Which multiplication *is* Mul<LinSrgb>?

[ChasJC23]

I've been using this crate to avoid some boilerplate for a project involving a fairly common LED strip. Currently, as you might expect by me asking this question, the colours I'm getting aren't what I expect.

Although there are other issues I'm having which could be the cause (like finding out which is the most appropriate form of gamma correction to use, and the fact that sRGB probably isn't the triangle I want), I'd like some clarification on what the arithmetic operators are really doing, because I have suspicions that mul isn't doing what I expect, and the documentation is unhelpfully recursive:

impl<S, T> Add<S, T>> for Rgb<S, T>
...
Performs the + operation.

As I've done ray-marching and ray-tracing code before, my initial instinct was piecewise-adjacent: LinSrgb represents a decent approximation of luminous intensity, so addition and multiplication would be piecewise (like overlaying a light over another, or using a solid colour filter), and if they aren't, it would only be because of the alpha channel, which I'm not using so it shouldn't make a difference (much like palette::blend::Blend::multiply). However, a lot of the times I use * I just end up with black, or something only involving one of the LEDs, so clearly one of my assumptions has gone wrong somewhere.

I did initially check out the definition, but that didn't get me far as my macro knowledge hasn't gone any farther than how to avoid Rust syntax for generating things like mathematical vectors (curse you vec!). Even then, I still need something open to hold my hand in making them.

Are they piecewise (or close enough for the difference to not matter in my case)? If so, do the elements being multiplied actually represent something like luminous intensity for the vertices of the sRGB triangle?

I know I could just avoid this problem entirely by properly using palette::blend::Blend, but understanding why I'm wrong is better than knowing I'm wrong! Apologies if my digging through issues wasn't thorough enough.

[Ogeon]

Hi, I can give more details a bit later, but all the arithmetic operators are just simple operations on each component/channel separately. So multiplying RGB would be like Rgb::new(a.red * b.red, a.green * b.green, a.blue * b.blue). The same goes for alpha, though, so you could end up with entirely transparent colors, eventually.

[Ogeon]

Alright, I'm able to expand on my response a bit, but I hope it was the answer to the main question.

Are they piecewise (or close enough for the difference to not matter in my case)?

Yes. Your initial assumption was entirely correct, and if you stick to Srgb, LinSrgb and similar aliases without the a at the end, you shouldn't get any problems with alpha either, since those don't have the Alpha wrapper involved anywhere. But even with it, the multiplication operator doesn't mix the alpha values into the color values on its own. Here's what the macro expands to in the case of Rgb, for multiplying two colors:

impl<S, T> Mul<Self> for Rgb<S, T>
where
    T: Mul<Output = T>,
{
    type Output = Self;
    fn mul(self, other: Self) -> Self::Output {
        Rgb {
            red: self.red * other.red,
            green: self.green * other.green,
            blue: self.blue * other.blue,
            standard: PhantomData,
        }
    }
}

If so, do the elements being multiplied actually represent something like luminous intensity for the vertices of the sRGB triangle?

Linear RGB represent the light/power output within the triangle that the primaries form, which is why it's recommended in most use cases, especially in 3D rendering. It's different from the perceived lightness and color, both because our eyes and brain perceives it differently and because the medium (your LED strip) probably also changes the "signal". So the model is that a linear RGB value of 50% represent a light output of 50% from the maximum, but doesn't look like it because our biology applies a logarithmic curve.

The first gradient here is made in linear space and the second is made in sRGB space. The reason why the first one looks weird and the second doesn't is that sRGB's gamma encoding compensates quite well for how we perceive the lightness.

When using a regular ol' computer screen as the medium, it tends to expect the output to be in sRGB so it can set the sub-pixel brightness correctly (modern monitors with HDR and other fancy stuff support wider gamuts too). This isn't exactly because of how it matches our perception, but more of a legacy from CRT times because of their output curve. The rest is convenient coincidence, since it turns out to function as a good compression algorithm. A byte isn't exactly enough resolution to encode linear light without some bad banding in the darker ranges, where we are the most sensitive to differences, but sRGB does a much better job.

I can't say what your LED strip would expect, though. I did use a small LED strip before (some variant of this) and I remember that it didn't have good color reproduction at very low intensity. Could it be something similar in your case? It's hard for me to guess what the issue could be without knowing anything about the LEDs or what you are sending to them. 🤔

Something you could try is to make a gradient, like the one above, and set the LEDs show that gradient so it's the brightest in one end of the strip and darkest in the other. These should be similar to what they used in the images, but it's untested:

use palette::{Mix, Srgb, LinSrgb};

let x = /* Some value between 0.0 and 1.0 */;

// Gradient in linear space, output as sRGB u8
let linear_gradient_value: Srgb<u8> = LinSrgb::new(0.0f32, 0.0, 0.0).mix(LinSrgb::new(1.0, 1.0, 1.0), x).into_encoding();

// Gradient in sRGB space, output as sRGB u8
let srgb_gradient_value: Srgb<u8> = Srgb::new(0.0f32, 0.0, 0.0).mix(Srgb::new(1.0, 1.0, 1.0), x).into_format();

[Ogeon]

Adding to the hardware related explanation, I found this for the Neopixels: https://learn.adafruit.com/led-tricks-gamma-correction/the-issue. It seem like it has a gamma exponent of about 1/2.8 (sRGB is closer to 1/2.2) and also some severe quantization errors in the darker end, but your strip may of course be different.

[ChasJC23]

That's amazing, thank you! I did try doing a gradient type thing a while back so I could work out what there was built in for me in terms of gamma correction, but it looked like LinSrgb -> Srgb did the opposite of what I needed interestingly enough. Although I will still nab that code snippet for later, I originally coded the lerps myself... From the raytracing stuff I knew that if you're lazy with gamma correction you can sorta get by with just doing a square root at the end (I think the exponent you really want is ~0.55?), so I guess I assumed the conversion in the library did something like that.

I have since found a post that gives me all the information I need in terms of gamma correction so I should be fine in that regard, although it does seem strange that such a common standard gives me the opposite of what I want!

I think in terms of the LED strip it's a similar story to what you're saying about the NeoPixel where it's all a bit rubbish at low brightness (probably explains why it's backwards), although for the one I'm using (ws2812b) I have a global brightness thing I can send it, and my project is very much for a "bedroom rave" type deal so dark colours really aren't a concern.

I'm still not quite sure why my multiplying colours code seems so intent on returning me zero, if I try toy examples like "#f7f" * "#ff7" it just gives me black when in theory it should be something close to "#f77", although I am sending these instructions from a little website so I've probably got to do more translations than my code currently does. I do remember I managed to find other toy examples that did net me something, but I helpfully never made a note of them(!) Thinking about it, the function I've got for converting to the format for the LEDs is shockingly lazy, so I think I'll probably get to work getting all of the format changes done first!

fn reduce_colour(colour: LinSrgb) -> [u8; 4] {
    // I don't think I'm doing this right
    let colour: PackedBgra<[u8; 4]> = colour.into_format::<u8>().into();
    palette::cast::into_array(colour)
}

Thanks again, this has been a great help!

[Ogeon]

Nice, that page seem to have all the info for getting the gamma correction right. Looks like it's even the same as (or similar to) the LED the Neopixel used to have when I got mine.

it looked like LinSrgb -> Srgb did the opposite of what I needed interestingly enough

Was it a similar effect as in the first gradient in the picture I posted. In that case it could be because you made the gradient in linear space, like in the code snipped, and converted to sRGB after. Lightness gradients tend to look wrong in linear space, while chroma/color gradients (like red to blue) look better in linear space. This is where perceptually uniform spaces, like Lab, Oklab and also Luv, are useful. Their numerical properties are closer to what we experience.

From the raytracing stuff I knew that if you're lazy with gamma correction you can sorta get by with just doing a square root at the end (I think the exponent you really want is ~0.55?), so I guess I assumed the conversion in the library did something like that.

Yeah, the square root is a rough estimate that's relatively fast and can be good enough. The slower estimate is usually 1/2.2 (square root is 1/2), but sRgb is a bit more complicated than that if you want to be precise. That's why a lookup table is nice, which you get when using into_linear() and into_encoding() to also go from and to Srgb<u8> (skipping the into_format).

if I try toy examples like "#f7f" * "#ff7" it just gives me black when in theory it should be something close to "#f77"

Yeah, that doesn't seem right. I suppose something in the translation could be off, like you say. For example, how do you convert from hex to decimal? I would guess something like this would be what you are looking for:

// Assuming input is sRGB, since that's usually what you get in color pickers.
let color1 = Srgb::<u8>::from_str(hex_input1)?.into_linear::<f32>();
let color2 = Srgb::<u8>::from_str(hex_input2)?.into_linear::<f32>();

let result = color1 * color2;

// We can use this slightly obscure feature to gamma encode our colors in a typesafe way.
use palette::encoding::{self, gamma::{Number, Gamma}};
struct OutGamma;
impl Number for OutGamma {
    const VALUE: f64 =  1.0/0.45; // ???
}
type LedRgb<T> = palette::rgb::Rgb<Gamma<encoding::Srgb, OutGamma>, T>;

// You could do this like in your function, but set alpha to 0 if it's the white LED.
let (red, green, blue) = LedRgb::from_linear(result).into_format::<u8>().into();
let output = [blue, green, red, 0]; // The last one is a "clear" or white sub-LED, right?

[ChasJC23]

Was it a similar effect as in the first gradient in the picture I posted. In that case it could be because you made the gradient in linear space, like in the code snipped, and converted to sRGB after.

Yep, you're absolutely right, I did a pulsing type of thing from yellow to black or something like that. I guess I didn't think that making the gradient in linear space wouldn't be linear by the end of it all, although it does make a bit of sense now. I haven't heard of colour spaces like Lab, so I think I'll do a bit of reading! The fact that perception always seems to be logarithmic in some way can really make all this extra confusing, I know it's like that for sound as well, making dB a logarithmic scale in some sense.

For example, how do you convert from hex to decimal?

For that specifically, I'm doing the conversion over in the JavaScript world because I haven't felt the need to get WASM involved, and I thought keeping the events in a semi-consistent format throughout is good enough for me. Learning 2 or 3 new things for a project is great, but it can get pretty unwieldy if you're trying to learn 5 or 6. I'll leave WASM for another day. The function is a bit chunky so I won't put it here, but it seems to work fine for all the formats I wanted it to expect. If it doesn't understand the colour I currently have it return black so I might have a fiddle with that just in case.

Thanks for pointing out the feature for gamma encoding, that looks super useful! The fourth number in the colour array I think is a side effect of the library I'm using: it supports a whole bunch of LED strips in the same general category as mine, so even though mine doesn't have a white LED, it expects a value for it regardless. That lines up with what's in the datasheet (there's some shocking English in there) so I think I can get away with not caring about the specific value.

Thanks again very much, I've taken your recommendation for conversion on the way out for now as the conversion of the data coming in might be a bit harder to squeeze in at the moment: I've been using serde to parse the input data from the network into the data structures I'm using to represent updates to the LED strip, so inserting a Srgb -> LinSrgb conversion is a touch intrusive. I could do it on the JavaScript side but I think this is a good sign for doing some spring cleaning. Lot of work to do before I can make a decent test so it'll be a while before I can send a good update.

[Ogeon]

Nice that you found the cause! And you're welcome, I'm happy to help. 😁 I was going to say that I don't think you need to reach for WASM just for parsing hex values, so good that it wasn't that part that was problematic.

The fourth number in the colour array I think is a side effect of the library I'm using

Right, it was just a wild assumption from my part and can be ignored if it works anyway.

I've been using serde to parse the input data from the network into the data structures I'm using to represent updates to the LED strip, so inserting a Srgb -> LinSrgb conversion is a touch intrusive

Exactly what you send is an agreement between the client and server, it could be either format that works with the rest of the pipeline. You could potentially also push those conversions into the update functions, so they only do it when/if necessary. It all depends on what they need to do and how you structure them.