[pixel,rasterize] Performance & Define UInt8Array with channel per bit

[dennemark]

Hi,
I am just experimenting with the @thi.ng/rasterize package. I want to see if it gives me some performance improvements over using a canvas or svg, since I want to change a WebGL texture in realtime so it needs to be fast (Edit: @thi.ng/rasterize is CPU based and not WebGL, therefore not comparably fast to svg or canvas).
But maybe I do not even need to look into this, since currently the drawCircle function needs around 900ms if I do it as shown in the thi.ng/rasterize example with defUVGradient (150ms if it is a floatBuffer).

In my case the texture is stored in a UInt8Array with [r,g,b,r,g,b,...]. I tried to use the defIntFormat function of thi.ng/pixel like this:

const rgbU8 = p.defIntFormat({
  channels: [
    { size: 8, lane: p.Lane.RED },
    { size: 8, lane: p.Lane.GREEN },
    { size: 8, lane: p.Lane.BLUE },
  ],
  size: 8,
  type: "u8",
})

But when using it on intBuffer(512,512,rgbU8) I still get a buffer of size 512x512. So I am not sure if it even is possible, to create a rgb buffer of 512x512x3 size.

It might only be worth looking into this, if there is really a performance benefit in using rasterize or for reasons of documentation. Would be happy to hear if some ideas come up concerning better performance. If not, is also ok.

Kind regards!

[dennemark]

Ok reading helps... looked closer at the code and how it is working, realising, that the shader function is js code, so cpu base.
The docs tells it:

Conceptually similar, but not to be equaled with actual WebGL fragement shaders,

But I would still be curious, if there is anything similar in webgl space. I thought I have seen something like writing to texture via real shaders in maplibre-gl. I am really not familiar with shaders. Would have been nice for me to write lines etc. to a texture easily, but seems like thats not how it works.

[postspectacular]

Hey @dennemark 👋 The rasterize package is really more meant as a generalized version of shape drawing and usable not just for pixel buffers (or to compete with native canvas/webgl). All of the supported shapes in that package are actually generator/iterator driven, meaning there're versions to just obtain sequences of coordinates of the "pixels" to draw, which is quite useful for a lot situations (e.g. the actual flood filling and mask transition shapes in De/Frag1 or the slow reveal of "macro brush strokes" lines in De/Frag3 are all created via this package). I'm also still planning to replace/consolidate the drawing functions in https://thi.ng/text-canvas with the ones from the rasterize pkg... that use case too isn't a "real" pixel buffer...

So tl;dr — this pkg is in no way meant to be super performant (though it's always an important concern for all my stuff). It's intended for use case where there're no other drawing solutions (e.g. no canvas and no webgl)...

As for using WebGL to draw directly into a texture, this is of course possible with the thi.ng/webgl package... I don't have anything simpler right now to show as example, but you can take a look at the webgl-float-fbo example to see the general idea. I will mock a more basic demo later tonight for you...

To answer your question about RGB pixel support in thi.ng/pixel - the 12 integer based pixel formats supported by that package are all either based on 8, 16 or 32 bit word sizes. It was simply a requirement to treat pixels as single values (so called packed integers) and their's no Uint24Array which would have supported this. Supporting 3 channel RGB or other non-power-of-2 channel formats would require the same handling as the float buffer and formats in that package (i.e. storing each channel as separate value). It's something I actually considered, but just never needed so far...

[dennemark]

Thanks for the explanations! The webgl example seems nice! With my basic knowledge of shaders i still seem to be able to read the code :) i think i would know how to approach the drawing on texture with it. But wont take this road for now, maybe another time. Though i am really curious about it :)

It's something I actually considered, but just never needed so far...

Lets leave it this way, i wont need it either :)

[postspectacular]

@dennemark - so here's the link to the new example (which also requires the latest version of thi.ng/webgl, just released this AM...). Please let me know if that's helpful to illustrate the overall approach (incl. drawing to and then reading from an offscreen render texture)

Demo (draw in the left/first square):
https://demo.thi.ng/umbrella/webgl-texture-paint/

Source:
https://github.com/thi-ng/umbrella/blob/develop/examples/webgl-texture-paint/src/index.ts