Path rendering on software renderer #6032

metehus · 2024-09-04T02:27:00Z

This PR aims to implement path rendering for the software renderer, which closes #4178.

This is still a draft PR starting as a proof-of-concept to validate the work, but I will try to contribute on this feature further along.

Important notes

This is my first time with rust (yes i know, not the best idea) and of course a lot is wrong and can be improved. For that I need some help and recommendations
I used tiny_skia to implement the rendering. I've seen the recommendation comment to use zeno, but I found that this crate does rasterization to a mask, which is not suitable for paths with stroke and fill (at least not easily)
As mentioned, I will need some help along, mainly on the crates configuration, performance, testing on different environments, best practices, etc.

TODO

CLAassistant · 2024-09-04T02:27:06Z

All committers have signed the CLA.

metehus · 2024-09-04T02:28:45Z

here's a poc with a basic path:

ogoffart · 2024-09-04T20:22:53Z

Thank you very much for the PR!
I'm busy this week so it will take some time until I can have a deeper look at it. But looks good at first glance.

One thing is that this is only for std, while the software renderer on desktop may be used also to preview how it'll look on no_std MCU. We'll have to think of a way to disable that. But regardless, I think it make sense to merge something like this.

metehus · 2024-09-05T01:03:51Z

Yes. I couldn't find a good crate for this without std. I thought of converting it to svg and using that to render, as this is already supported, but it seems like this si rasterized at compile time so wont work.

But I think it's better to have this supported only on std instead of no support on the software renderer, just configuring it to be enabled and imported only on std envs (I have no idea how to do it but i guess it's possible)

metehus · 2024-09-05T22:29:23Z

@ogoffart Hey! I was looking at the tiny-skia cargo file and noticed that it has support fot no_std environments by enabling the no-std-float feat. So this should be able to work on this environment, right? To be clear, I'm pretty new to rust and specially on embedded environments, so am I missing something too?

metehus · 2024-09-07T20:12:27Z

updates:
got it to render correctly and at a good performance:

but there are a few points:

I switched tiny_skia to zeno for the path rendering. Found out it had a better performance and it should use less memory (tiny_skia was saving one 32bit buffer, but with zeno it's saving a 16bit buffer (8 bit for both stroke and fill);
Because of this change, gradient fill and stroke won't work. It would need to render the gradient within the mask. maybe the draw_gradient_line could be used;
As this is prealocating a buffer of the element size, path areas that overflow this rectangle will always be clipped. zeno provides a function to measure the path boundings, but would need to have a reference location in order to counter offset the size;
The path joins are also not correct. even by setting it to mitter, they are different from the other renderers:

ogoffart

Thanks a lot for the PR.
This looks good.
I've added a few comments but they don't need all to be addressed before merging.
I think it is ok to merge it even if it is not fully complete.
All is required is that it passes CI. Ideally it can be disabled, which would be the case if the feature was not enabled by default.

ogoffart · 2024-09-09T07:31:46Z

internal/core/Cargo.toml

@@ -38,6 +38,8 @@ software-renderer = ["bytemuck"]
 image-decoders = ["dep:image", "dep:clru"]
 svg = ["dep:resvg", "shared-fontdb"]

+path = ["dep:zeno"]


perhaps should be renamed "software-renderer-path" since that doesn't impact other renderer.

ogoffart · 2024-09-09T07:32:17Z

internal/core/Cargo.toml

@@ -24,7 +24,7 @@ libm = ["num-traits/libm", "euclid/libm"]
 # Allow the viewer to query at runtime information about item types
 rtti = []
 # Use the standard library
-std = ["euclid/std", "once_cell/std", "scoped-tls-hkt", "lyon_path", "lyon_algorithms", "lyon_geom", "lyon_extra", "dep:web-time", "image-decoders", "svg", "raw-window-handle-06?/std", "chrono/std", "chrono/wasmbind", "chrono/clock"]
+std = ["euclid/std", "once_cell/std", "scoped-tls-hkt", "lyon_path", "lyon_algorithms", "lyon_geom", "lyon_extra", "dep:web-time", "image-decoders", "svg", "raw-window-handle-06?/std", "chrono/std", "chrono/wasmbind", "chrono/clock", "tiny-skia/std", "path"]


is tiny-skia still required?

ogoffart · 2024-09-09T07:33:41Z

internal/core/software_renderer.rs

@@ -2659,6 +2797,71 @@ impl<'a, T: ProcessScene> crate::item_rendering::ItemRenderer for SceneBuilder<'
    }
 }

+// impl From<Color> for tiny_skia::Color {


some leftover

ogoffart · 2024-09-09T07:40:40Z

internal/core/software_renderer.rs

+            stroke_brush: path_props.stroke(),
+            fill_mask,
+            fill_brush: path_props.fill(),


Note: the "global" alpha needs to be applied to that color. (That's why other functions do self.alpha_color

ogoffart · 2024-09-09T07:43:31Z

internal/core/software_renderer/draw_functions.rs

+    let stroke_mask = cmd.stroke_mask.as_ref();
+
+    let fill_color = cmd.fill_brush.color();
+    let fill_color_alpha = fill_color.alpha() as f32 / 255.0;


We should ideally try to avoid these kind of computation in floating point for every lines. That's why the other commands store the PremultipliedRgbaColor.

ogoffart · 2024-09-09T07:51:45Z

internal/core/software_renderer.rs

+        let mut zeno_pb: Vec<zeno::Command> = Vec::new();
+        let (logical_offset, path_events2) = path.fitted_path_events(item).unwrap();
+
+        for event in path_events2.iter() {


I wonder if we can implement zeno::PathData for our own path.

agreed! one thing that is blocking to render paths on no_std is the dependency of lyon_path on other parts.

ogoffart · 2024-09-09T08:16:02Z

internal/core/software_renderer.rs

+
+        let fill_mask = if !path_props.fill().is_transparent() {
+            let mut mask = Vec::new();
+            mask.resize((geometry.size.width * geometry.size.height) as usize, 0u8);


That seems big.
Internally, zeno renders row by row as well, I wonder if there is a way to do the actual path rendering in draw_zeno_path_line instead. (assuming that the storage needed in the rasterizer is less)
But anyway, there doesn't seem to be API in zeno to do that.

yes, it doesn't seem to be able to render it line by line. at least the command saves two u8 buffers, which is less if it was a rgba buffer. On some occasions, there may be big blank areas, like when full sizing on flex layouts. Zeno provides a way to make the buffer on the needed size, but it would need to limit this somehow (prevent huge buffers with overflow lines) and have a reference point to translate the buffer to the correct location

ogoffart · 2024-09-09T08:18:54Z

tests/cases/path/animation.slint

@@ -0,0 +1,40 @@
+export component MultiplePathTest inherits Window {


These files will need a copyright header like all other files

tronical

Great work!

internal/core/software_renderer.rs

ogoffart · 2024-09-09T19:43:38Z

internal/core/software_renderer.rs

@@ -1398,6 +1409,16 @@ struct GradientCommand {
    bottom_clip: PhysicalLength,
 }

+#[derive(Debug)]
+struct ZenoPathCommand {


(in fact, this is a simplified version of SharedBufferCommand, for an alphamap)

yes. saving as an alpha map reduces the memory size, but makes it heavier on the line drawing function, as gradients need to be calculated there. This shouldn't be a problem as gradient fill on rectangles already does that

I meant that the ZenoPathCommand is basically the same as two SharedBufferCommand { buffer: SharedBufferData::AlphaMap { ... }, ... }

In fact, the draw_texture_line can be re-used, i think (if you create a SceneTexture)

metehus · 2024-09-09T23:48:21Z

Thank you for the reviews! I will take some time to make some changes.
I've tried to run it with a MCU, as that was my goal, but it seems there are a bunch of other changes to be made (now i see the size of this problem), like changing lyon_path(std only) to another method on the parsing, which can impact on other renderers.
For now, I will fix the issues and make it ready as it is. This PR won't close #4178 anymore, but still will be usable as a standing point for that feature, and I'm hoping for this feature too!
This was something needed for my project. After seeing #2527, I will try the idea of making a callback function that receives some arguments and create the path image as a return to the slint runtime or something like that.

software-renderer: initial path rendering implementation

6f1f810

metehus added 4 commits September 5, 2024 22:39

software-renderer: add path rendering test case

4516619

software-renderer: change path rendering to zeno builder

8450dc4

software-renderer: create path test cases

65a80e3

software-renderer: create path screenshots tests

4d87d83

metehus added 3 commits September 7, 2024 17:12

software-renderer: change stroke join

25de8ce

chore: add .idea (rust rover) folder to gitignore

af28639

Merge branch 'slint-ui:master' into master

fa5e5b4

ogoffart reviewed Sep 9, 2024

View reviewed changes

tronical reviewed Sep 9, 2024

View reviewed changes

internal/core/software_renderer.rs Show resolved Hide resolved

ogoffart reviewed Sep 9, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Path rendering on software renderer #6032

Path rendering on software renderer #6032

metehus commented Sep 4, 2024 •

edited

Loading

CLAassistant commented Sep 4, 2024 •

edited

Loading

metehus commented Sep 4, 2024

ogoffart commented Sep 4, 2024

metehus commented Sep 5, 2024

metehus commented Sep 5, 2024

metehus commented Sep 7, 2024

ogoffart left a comment

ogoffart Sep 9, 2024

ogoffart Sep 9, 2024

ogoffart Sep 9, 2024

ogoffart Sep 9, 2024

ogoffart Sep 9, 2024

ogoffart Sep 9, 2024

metehus Sep 9, 2024

ogoffart Sep 9, 2024

metehus Sep 9, 2024

ogoffart Sep 9, 2024

tronical left a comment

ogoffart Sep 9, 2024

metehus Sep 9, 2024

ogoffart Sep 10, 2024

metehus commented Sep 9, 2024

		@@ -0,0 +1,40 @@
		export component MultiplePathTest inherits Window {

Path rendering on software renderer #6032

Are you sure you want to change the base?

Path rendering on software renderer #6032

Conversation

metehus commented Sep 4, 2024 • edited Loading

Important notes

TODO

CLAassistant commented Sep 4, 2024 • edited Loading

metehus commented Sep 4, 2024

ogoffart commented Sep 4, 2024

metehus commented Sep 5, 2024

metehus commented Sep 5, 2024

metehus commented Sep 7, 2024

ogoffart left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tronical left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

metehus commented Sep 9, 2024

metehus commented Sep 4, 2024 •

edited

Loading

CLAassistant commented Sep 4, 2024 •

edited

Loading