Future of Gradient / Color Interpolation

[NicolasKlenert]

Features

So I thought and learned a lot about interpolation methods recently and how the Gradient struct could be improved upon. To be on the same page, here are some features one could consider:

• Curve constructs like Bezier, BSpline, NURBS, etc.
• Curve fitting of these constructs
• Speed/Change adjustments of these constructs (more on this later)
• Surfaces and other multidimensional constructs (most curves do have a surface counterpart)
• Other interpolation/extrapolation methods, like noise
• Adapters which change the behaviour, common in animations: Slow-In Slow-Out, Repeating etc.

Some of these are definitly overkill for this crate. Such a possible extern crate may be already necessary for specific use cases.

Example

Let's assume we want to implement a Bezier Curve. The algorithm itself and it's representation are rather easy to implement. However our curve may seem nice at first, but Bezier Curves don't have a "constant speed" over the curve. That is, our resulting gradient may change it's color drastically at some point and almost not at all at other points. The same would be true of the linear implementation of the Gradient right now if it wouldn't have customised positions (from now on I'll call them knots). However it's not possible to use the same technic for bezier curves (BSplines do have knots, but they don't solve our problem). To achieve a constant change over the curve one can do an approximation and create a lookup table. In the end, one generates a linear interpolation over many equidistant sample points. (This is also the reason the constant gradients we (and matlab) use are given as a linear interpolation, even though they are basically curves.)

How would an extern crate work

For the usual interpolation methods and structures the elements which should be interpolated only have to have two traits:

• A way to calculate a weighted average (like our Mix trait)
• A way to calculate the "distance" between elements (this could be the percieved change of color or some other metric)

Such a good extern crate would only need these two traits to be implemented and then work out of the box. The only question left would be what happens to the constant gradients. I think the best option here would be to implement a linear interpolation for equidistant points/colors (if the extern crate is not used) and implement them that way. These would also have the benefit of a slight performance boost, as the implementation could create a color in O(1) instead of O(log n), if n is the number of colors, as no binary search of the right knots would be necessary.

Opinion

In my opinion, it seems like a good idea to outsource interpolation and point to an interpolation crate, as doing interpolation right can be rather hard and may blow the scope of this crate. However these are only my thoughts, so I would appreciate any discussion about it.

[Ogeon]

Thank you for this write-up! I agree completely regarding outsourcing anything more complex than linear gradients (or even just equidistant gradients). I would prefer it if this library can stay as focused on the color spaces and their direct operations as much possible, which is why I have had similar thoughts, and I'm open for scaling down the scope were possible. The Gradient is sort of a living fossil and has survived because people are still using it. But I have been thinking about this problem from time to time and it's actually somewhat related to the troubles around things like providing SIMD support. Here are some of those thoughts:

Integrating with external crates seem to work through three methods:

• Provide an API that the external crate implements. Examples in Palette are FromColor and Component. This requires a certain amount of ubiquity to be viable on a wide scale, otherwise almost nothing is compatible. A third party would have to explicitly support Palette to be compatible.
• Implement a third party API. Examples in Palette are conversion to and from slices and arrays, uses of Float and std/core traits and compatibility with serde. This allows "small" crates to gain wide compatibility without relying on a third party explicitly supporting it, but requires picking and choosing among the third party API.
• The users have to write wrappers and glue code. This would be required if SIMD support was added with today's ecosystem, since none of the crates shared a common API last time I checked.

What I want to say with that is that we need to decide if Palette should support the third party libraries or if the third party libraries have to support Palette integration. A model where Palette dictates the API could include a Gradient trait with a eval(t: Self::Scalar) -> Self::Value method. A model where Palette tries to be widely compatible could rely on the implementation of arithmetic traits, but that depends on how third party gradient and interpolation crates work. If they have their own trait for the values, Palette would have to either explicitly support them or require user written glue code. I would like to avoid that scenario as much as possible. I haven't surveyed the ecosystem, so I don't know what the current state is. And I don't know if Palette has accumulated enough "clout" yet to own the API.

I think it would be neat if it would be possible to be compatible in both directions. If Palette could implement some common traits that make the colors usable in completely agnostic third party crates, but Palette could also provide some quality of life methods if the gradient type implements a particular gradient trait. That is, if third party gradients aren't already more feature rich, in which case I'm more than happy to leave it to them.

[NicolasKlenert]

I do agree about trying to not include any implementation details of other crates into Palette and rather try to implement crate agnostic traits. However I don't think Palette owning the traits (which colors would implement) would be the right choice, as these traits have to enable the interpolation crates to, well, interpolate. So these crates should know best which traits are necessary.

If the color space of a color acts like a vector space (that is, it is linear, the color can be added with another color, it can be multiplied with a scalar, and maybe a metric of color change can be expressed as the euclidean metric) then only implementing Add and Mul from Core and maybe Distance from the norman crate could be enough for most interpolation purposes. Such the owner of the relevant traits would be rust core (and a crate specifically designed to "facilitate generic implementations of numeric algorithms"), which would be the best option in my opinion. Furthermore an interpolation crate would be also interested in the possibility of implementing defaults of necessary traits from the core traits, as this would greatly boost the amount of supported types to interpolate with.

So the question would be, if all (or most) color spaces we want to work with are like that? The linearity is more or less given, as this is one aim of Palette, but what about the other properties?

As for the quality of life methods, I assume it's an open question for now, as this greatly depends on the current and future existence and richness of interpolation crates. I did try to find some candidates, however either the scope was rather limiting or it was specialised for f64 and f64 vectors. However even if there is no good crate out there yet, I'd argue that it still would be better to create an extern crate for interpolation (and maybe having Palette in mind) than implementing "everything" in Palette itself.

[Ogeon]

It seems like we share the same opinions on this topic. The ideal situation, as I see it, is that some external crate sets the standard via a set of traits, similar to norman and num_traits.

The arithmetic traits should already be implemented wherever possible. It may be time to check again that all directions and combinations are there, but it's supposed to be possible to, for example, add both colors together and add colors and scalars. This should be sufficient for implementing the interpolation part.

Color distance in general is a bit of its own field, but naively treating them as vectors should work for this case. All of the color spaces that have been implemented so far can be treated as vector spaces. Hue based spaces are a bit special because of their cylindrical/polar properties (red to blue interpolation goes via purple in the typical case instead of red > orange > yellow > green > teal > blue). So interpolation within a certain color space is all about treating them as vector spaces and leveraging the visual effects that results in.

[Ogeon]

Hey, I saw your https://github.com/NicolasKlenert/enterpolation crate earlier and it looks really good! I would happily recommend it as sort of a "companion crate" that works well with Palette, and strip down or even remove the built-in gradient type. 😄 I may add implementations of Merge instead, via a cargo feature, for types like HSL that aren't automatically compatible.

[NicolasKlenert]

Thanks and that sounds awesome! As you might have guessed I did go down the rabbit hole and created a new crate with this goal in mind, but did not want to promote my own crate here. So I'm happy you happen to see it 😃.

Do you have in mind how you want to handle the default gradients? If the built-in gradient doesn't get removed entirely it wouldn't have to change much. If you decide to remove them entirely however, either a feature gate and an optional dependency to the crate would be necessary or another crate , depending on both Palette and Enterpolation, would have to store them. Personally I feel like the seperate crate seems to be the more elegant solution. I'm curious about your thoughts on this.

[Ogeon]

I'm sure I wouldn't have minded if you mentioned it, but I can relate. I don't generally advertise this library outside release posts and they are pretty sparse. 😄

I haven't decided on anything specific, but I'm considering only providing the color values (and knots if necessary) in the constants, without any actual gradient functionality. So it would just be a simple data source that can be dropped into whichever interpolation solution works best for the application. 🙂 The downside with this is that people have to figure out how to use them, but the upside is that they are free to choose the type of interpolation that fits their application, they can freely access and manipulate the values, and they can also pick a different interpolation library if they would want to and not have to pay for it in extra dependencies. That sounds to me like a net positive for the library users, which is what I try to prioritize.

But once again, I haven't made up my mind. It's just the best option I have come up with so far.

[NicolasKlenert]

I did'nt thought about only providing the data, but I do like it. As you said it would keep the dependencies at a minimum and there wouldn't be any data redundany spread across extern crates using different interpolation librarys.

[Ogeon]

I have finally opened #301 to axe the gradient module. Let me know if I made any mistakes with enterpolation in the examples. 😅 The named gradients will also go, after all, since it's a bit weird to have them sitting around as they are.

[NicolasKlenert]

I looked over it and it looks totally fine.

One small thing I noticed: One could change the feature flag of the example to #[cfg(not(any(feature = "std", feature = "libm")))], since the standard library is not neccessary, which would coincide with the requierements of palette in general.

[Ogeon]

Thanks for having a look! Good point about the cfg attribute, I should change it.

[apps4uco]

Hi, I just noticed that some code I had stopped compiling, and searched around for a while thinking that there might be a new feature flag, or that the crate had split into 2 parts. Would it be possible to add into the README a comment explaining this change, and possibly a reference to an example or 2 to recreate the original gradients...

[Ogeon]

Hi, sorry for any confusion. It's sometimes hard to keep everyone informed when removing features. I could admittedly have deprecated it for a while, but I don't remember why I decided against it.

Either way, there is an example in the crate's readme, that you can also find on https://crates.io/crates/palette, where it shows how to make a simple gradient with enterpolation. I would recommend that crate if you want a similar API, since it's partially inspired by the removed gradient from Palette.

There's also a section for it in this blog post, from when 0.7.0 was released, including examples. https://ogeon.github.io/2023/04/10/palette-0.7.0.html

I can try to be more clear next time, and I'm already using deprecation more as a warning, when possible.

[apps4uco]

No worries, thanks for all the hard work.

Maybe a link to the blog posts from the README would be good.