-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Visualization of daily perf loadgen data #63
Labels
Comments
mhofman
changed the title
Visualization of loadgen data
Visualization of daily perf loadgen data
Feb 9, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
This issue is about building a visualization dashboard which automatically displays the stats from the latest daily loadgen runs.
Context
The loadgen's primary use is to build a regression over time of the behavior of the SDK (see Agoric/agoric-sdk#3107). In this case time has 2 dimensions:
The first is mostly captured by running loadgen cycles split in stages (currently 4 stages of 6 hours for the daily perf run), and comparing stages to each other. The second is captured by comparing summarized metrics between revisions (different daily perf runs).
Current tooling
Currently the stats are saved in a
perf.jsonl
file which contains a stream of CPU and Memory usage stats, and a final summary of all other stats. #43 deals with unifying these so that individual stats data point are outputted in the stream, and only summaries are generated at the end, possibly including summaries of the CPU and memory usage.The visualization is done by extracting the stats summaries into a CSV file (see https://github.com/Agoric/testnet-load-generator/blob/main/scripts/perf_to_stats_csv.jq), and importing that in a Google Spreadsheet with some graphs.
Detailed requirements
We would like to have a dashboard that shows the data detailed in Agoric/agoric-sdk#3107, which is automatically updated to include the results from the latest daily run.
If a run fails, the dashboard should make it obvious or possibly send alerts. It should also alert if no data has been received recently (to highlight a stuck loadgen for example)
The dashboard does not need to show data for a in-progress loadgen, that is a separate issue (TBD)
It would be great if the dashboard allowed easily generating new graphs from the existing data, or perform queries.
The text was updated successfully, but these errors were encountered: