Add reference image tests for untested plots/blocks

[ffreyer]

Description

Plots/Blocks that do not have (direct) reference images:

• raincloud
• series
• stairs
• stem
• timeseries (unlikely to be consistent in refimage tests due to dependency on sleep/time)
• waterfall
• boxplot
• crossbar
• qqplot
• qqnorm
• ecdfplot
• Button
• IntervalSlider
• Slider
• Toggle
• LineSegmentBuffer, TextBuffer tested via axis3d!, considered internal, may disappear in the future

Plots that may need more:

• ablines
• hspan, vspan
• violin
• spy already has test testing multiple configurations
• volumeslices
• Textbox

Type of change

• Internal Improvements / testing

[MakieBot]

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(fig)

	using	create	display	create	display
GLMakie	4.98s (4.96, 5.04) 0.03+-	107.85ms (107.42, 108.68) 0.44+-	643.71ms (637.23, 648.39) 3.77+-	8.21ms (8.12, 8.28) 0.05+-	25.61ms (25.48, 25.69) 0.08+-
master	4.96s (4.94, 4.98) 0.01+-	107.99ms (107.14, 109.11) 0.65+-	411.46ms (406.73, 423.55) 5.83+-	9.34ms (9.25, 9.42) 0.06+-	25.69ms (25.54, 25.92) 0.15+-
evaluation	1.00x invariant, 0.02s (1.01d, 0.10p, 0.02std)	1.00x invariant, -0.14ms (-0.25d, 0.65p, 0.54std)	0.64x slower❌, 232.25ms (47.29d, 0.00p, 4.80std)	1.14x faster✅, -1.13ms (-20.64d, 0.00p, 0.05std)	1.00x invariant, -0.08ms (-0.64d, 0.26p, 0.11std)
CairoMakie	4.94s (4.88, 5.00) 0.04+-	111.09ms (109.63, 113.18) 1.24+-	170.67ms (167.34, 175.41) 3.18+-	9.40ms (9.17, 9.64) 0.15+-	1.21ms (1.16, 1.25) 0.05+-
master	4.94s (4.87, 5.02) 0.05+-	110.90ms (108.84, 115.36) 2.17+-	168.89ms (166.38, 177.56) 3.89+-	9.24ms (9.11, 9.43) 0.10+-	1.24ms (1.17, 1.26) 0.03+-
evaluation	1.00x invariant, -0.0s (-0.02d, 0.97p, 0.05std)	1.00x invariant, 0.19ms (0.11d, 0.84p, 1.71std)	0.99x invariant, 1.78ms (0.50d, 0.37p, 3.54std)	0.98x slower X, 0.15ms (1.19d, 0.05p, 0.13std)	1.02x invariant, -0.02ms (-0.61d, 0.28p, 0.04std)
WGLMakie	5.57s (5.44, 5.77) 0.11+-	113.28ms (109.32, 121.16) 3.94+-	5.10s (4.96, 5.22) 0.11+-	12.20ms (11.97, 12.97) 0.36+-	129.07ms (123.12, 136.47) 5.13+-
master	5.36s (5.20, 5.47) 0.09+-	109.83ms (108.99, 111.03) 0.92+-	5.02s (4.90, 5.32) 0.14+-	12.42ms (11.73, 14.24) 0.94+-	129.52ms (119.31, 138.16) 7.07+-
evaluation	0.96x slower X, 0.21s (2.07d, 0.00p, 0.10std)	0.97x invariant, 3.45ms (1.20d, 0.06p, 2.43std)	0.98x invariant, 0.09s (0.69d, 0.22p, 0.13std)	1.02x invariant, -0.22ms (-0.31d, 0.58p, 0.65std)	1.00x invariant, -0.45ms (-0.07d, 0.89p, 6.10std)

[EdsterG]

Is there any documentation about updating/creating ref image tests? Is the image pre-generated and uploaded or do you use some relative score between ref images dynamically generated during test time?

[ffreyer]

#4108 added some notes to prs about it.
If you want to add a refernce image test, you just need to write a

@reference_test "some name" begin
    # your code...
    fig
end

block, like in this pr. CI produces an image for each of these blocks and compares it to a reference image if it exists. When a new reference test is added (or if one gets changed) the maintainers need to upload the new image.

[jkrumbiegel]

I actually stopped doing reference tests where there's some kind of symmetry in the data. Like here, all groups have the same data basically, so if there was something wrong in the mapping from categories to groups, you wouldn't see it here. Instead I'd add the group values to the data maybe, so you'd see three shifted violins, or something to that effect.