64 bit label support

[aschampion]

Taken from #1801.

Tested support

bits	ndarray	texture render	vis (shader hash)	shader ids	selection	issues
8	✅	✅ (as gl float)	✅			shader generality
16	✅	✅				shader generality
24/32	✅	✅ (as gl float rgba)	✅	✅	✅
32	✅	✅ (as gl float rgba)	✅			premultiplied alpha
48/64	✅	✅				16bpp, bigint, shader p
53/64	✅	✅				premultiplied alpha, 16bpp, bigint, shader p
64	✅	✅				premultiplied alpha, 16bpp, bigint, shader p, js numerics

Summary

Taken from lab wiki.

• ❌To avoid maintaining a Scijs fork to handle ImageBlocks in CATMAID, we will evaluate the TensorFlow.js base library. It is not clear if it fully supports 64 bit — tf.tensor takes any TypedArray as input (which includes BigInt64Array/BigUint64Array ), but the dtype specifier is tf.DataType which is 'float32'|'int32'|'bool'|'complex64'. If suitable, having tensors processing and eventually TensorFlow support in CATMAID may be useful for general image processing and in-browser inference.
• @clbarnes checked that 64 values do not work. Further, while the next release of tf.js will support importing int64 tensors from other TensorFlow backends, they are silently cast to int32.
• Instead, four options remain:
  1. Fork and patch SciJS
  2. Write minimal N-D slicing and padding in JS
  3. Write minimal N-D slicing and padding in TS
  4. Write a thin wasm-bindgen wrapper around necessary ops in rust ndarray
• 1 is lowest initial effort but perhaps a higher ongoing burden. 2 is the simplest solution in not introducing any new libraries, but has the least extensibility. 4 is most consistent with our strategic goal of relying as much as possible on a common backend between Rust, Python and JS. However, not introducing another large binary blob into the CATMAID repo would require merging webpack support (Infrastructure tracking: compilation, source mapping, and TypeScript #1338) first.

Roadmap

• To remove this as a blocker for the rest of label support, do 1 (fork SciJS for BigInt support) first, then consider more permanent options.

[aschampion]

Upstream PRs finally got merged last night, so I forked numjs with the remaining fixes for bigint support: https://github.com/aschampion/numjs/tree/features/bigint64-support

Since the maintainer is inactive I won't bother PRing until we're using the fork and have confirmed it's working for us.

[aschampion]

I have a branch fixing numjs' BigInt64 support, which fills out the first column of the matrix (ndarray). For the second column, actually getting the data into WebGL2 and rendered, while I have a branch getting textures uploaded in the correct GL internalformat/type/format triples, the next step is creating subclasses of Pixi.Sprite that are different object types with their own renders (extending Pixi.SpriteRenderer that have custom shaders with the appropriate glsl types. ImageBlockLayers will then re-init their tiles to the correct type once their tile source loads.

Note that this is a different problem from the rest of the shader columns, which refer to stack viewer filter shaders. Those issues and type-generality still need to be addressed, but we also need to factor the concepts of viewer filters (which apply to the entire render buffer) and layer filters (like object labeling, which apply to specific layers). This is because we likely want layers to be required to render down to normal GL floats for a sane pipeline, meaning the mapping from label IDs to colors should happen as part of the layer.