Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarks: want summary boxplots generated during CI #471

Open
FelixMcFelix opened this issue Mar 8, 2024 · 0 comments
Open

Benchmarks: want summary boxplots generated during CI #471

FelixMcFelix opened this issue Mar 8, 2024 · 0 comments
Labels
cleanup Code cleanliness feature Planned and requested features repo Automated tooling for dependency version management, CI, etc.

Comments

@FelixMcFelix
Copy link
Collaborator

Currently, criterion generates very nice plots of the distribution of runtimes for all of our benchmarks. This is the right level of detail for honing in on a specific benchmark, e.g., the distribution of times received from FBT probes on rx.

A glance view of processing times in and between families of packets would be useful. Each set of results processed by criterion outputs a list of box-whisker values (tukey.json) and other summary statistics (estimates.json), which makes this data easily available. We're already pulling in plotters as a library via criterion, which may be a useful starting point (as handy as Gnuplot is under other circumstances).

@FelixMcFelix FelixMcFelix added cleanup Code cleanliness feature Planned and requested features repo Automated tooling for dependency version management, CI, etc. labels Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup Code cleanliness feature Planned and requested features repo Automated tooling for dependency version management, CI, etc.
Projects
None yet
Development

No branches or pull requests

1 participant