Genie Discord forum

Author Avatarjeremiedb
2/9/2023, 10:46:36 PM

Is there any example showing how to have an image / png content within a Stipple reactive model?

The intent is to be able to display plots as static images, similar to what Shiny does. See for example: https://shiny.rstudio.com/gallery/conditionalpanel-demo.html

That would allows interactive application leveraging a broader set of plot librairies which have support for saving figures, including Plots.jl, but also CairoMakie and GGplot2 through RCall.

I want to avoid going through saving files on server, and rather directly pass the raw base64 content, again similar to how Shiny does it.

Following the above Shiny example, I'd expect to have something like the following in the frontend:

<img width="357.479553222656" height="300" alt="Plot object" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAb...">

On the backend, first step seems to save a plot into an IOBuffer:

io = IOBuffer()
p = Plots.scatter(rand(5), rand(5))
Plots.png(p, io)

Then, I thought of passing that io's data as a string to the model:

plt_static_1::R{Any} = String(take!(io))

And then passing this in the frontend as

<img alt="Plot object" src={{plt_static_1}}>

But this result in the follwing:

Any hint how to get this to work? I think it would be a very convenient feature to showcase.

Author Avatarjeremiedb
2/13/2023, 7:36:24 AM

For Plots.jl, the following provides a working solution.

Frontend:

<q-img :src="plt_static_1" spinner-color="white" style="height: auto; max-width: auto"/>

Backend:

using Base64
p = Plots.scatter(rand(5), rand(5))
m.plt_static_1[] = "data:image/png;base64," * stringmime("image/png", p)
Author Avatarjeremiedb
2/13/2023, 7:58:22 AM

Also works fine with Makie:

fig = CairoMakie.scatter(randn(20, 2), color = 1:20)
m.plt_static_2[] = "data:image/png;base64," * stringmime("image/png", fig)

Except for the string concatenation with "data:image/png;base64," that is not most elegant, integration happened to be surprinsingly more straighfoward than I thought!

Author Avatarjeremiedb
2/13/2023, 3:32:12 PM

Thanks for the reference! I'll look a little further in case there's an encoding mechanism that avoids the manual concatenation of the "data:image..." string, but the above snippets for Plots.jl and Makie.jl appers to do the job right in the context of Stipple, where reactivity to inputs is needed.

Author Avatarabhimanyuaryan
2/13/2023, 4:07:09 PM

have you uploaded those makie and plots examples of yours anywhere on Github(/OSS)?

I'll link them in resources here: https://github.com/GenieFramework/genie-awesome

Author Avatarjeremiedb
2/13/2023, 6:05:39 PM

Yes it's on https://github.com/jeremiedb/GenieExperiments.jl. Must go to /plotdemo as show in attached screenshot. I'll just make a quick cleanup to remove the PlotlyLight tests and move test to seperate ui/controller as I'm still trying to figure out how to get PlotlyLight to work!

Author Avatarjeremiedb
2/13/2023, 6:13:15 PM

Done, http://localhost:8002/plotdemo should now looks reasonably clean, with all plots being updated when clicking the RESET button.

Author Avatarjeremiedb
2/13/2023, 6:26:01 PM

@essenciary I haven't managed to get PlotlyLight working, but it looks like I'm only missing a small element. Here's where I got:

import Stipple: render
using PlotlyLight

function Stipple.render(p::PlotlyLight.Plot)
    return stringmime("text/html", p)
end

In the model, I tried both specifying vars as PlotlyLight.Plot and raw json.

    plt_2::PlotlyLight.Plot = PlotlyLight.Plot(
        x = 1:20,
        y = cumsum(randn(20)),
        type = "scatter",
        mode = "lines+markers",
    )
    plt_3 = stringmime("text/html", PlotlyLight.Plot(x = sort(rand(4)), y = randn(4)))

In the UI:

        <div class="col-sm-4">
            {{ plt_2 }}
        </div>
        <div class="col-sm-4">
            {{ plt_3 }}
        </div>

But the above looks to only print the html string, and not render them. I'm not suprised it does so with plt_3 which is effectively just a string in the Model, but I thought Stipple.render was meant to operate the magic for Stipple to actually injects the string as html to be rendered.

Author Avatarjeremiedb
2/22/2023, 5:15:36 AM

@abhimanyuaryan any idea what is missing to get the above json string to be actually rendered as plots (rather than just printing the literal string)?

Author Avatarabhimanyuaryan
2/23/2023, 11:57:41 PM

@jeremiedb sorry no. Is their an MWE that I can run and test? Is this last code with PlotlyLight the one?

Author Avatarhhaensel
3/8/2023, 12:07:29 AM

injecting html is via span(v__html="""myhtml text""")

Author Avatarhhaensel
3/8/2023, 12:10:42 AM

Stipple.render() converts the variables to be rendered in a julia variable that afterwards will be encoded by JSON3 and then transmitted. Very often it just doesn't do anything.

Author Avatarjeremiedb
3/9/2023, 5:35:03 AM

Thanks! I wasn't aware of that v__html feature. It works fine for simple html strings sent from the model, such as html_2::String = "<ul><li>item 1</li></ul>" which can then be included in a view with <span v-html="html_2"></span>.

However, including a more complex html such the one needed for plotly doesn't seem to work. <span v-html="plt_3"></span> results in no error but nothing is rendered, while the "static" insertion with $(model.plt_3[]) does render the plot (although with some strange display that doesn't respect the parent col class). I guess if the above would work smoothly, there wouldn't be the need to specify a plotly Vue component in https://github.com/GenieFramework/StipplePlotly.jl/blob/main/assets/js/vueplotly.js.

I think for now I'll stick to using the plotly Vue compoenent from StipplePlotly for situations where I want to reuse PlolyLight.jl recipes within Stipple.