Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Stack Monitoring] Create simulator for stack monitoring data #117615

Closed
Tracked by #119491
matschaffer opened this issue Nov 5, 2021 · 6 comments · Fixed by #118302
Closed
Tracked by #119491

[Stack Monitoring] Create simulator for stack monitoring data #117615

matschaffer opened this issue Nov 5, 2021 · 6 comments · Fixed by #118302
Assignees
Labels
Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services

Comments

@matschaffer
Copy link
Contributor

matschaffer commented Nov 5, 2021

In order to test new kibana metrics formats (#113149) ahead of metricbeat being available (elastic/beats#27569) we'll need a way to generate fake data.

It'd be great to have this in general for things like testing pagination updates, mixed version scenarios, or even answering SDHs that involve tricky data collection models (mixed mode, via logstash or not, etc).

@matschaffer matschaffer added v8.0.0 Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services required-for-8.0 This work is required to be done before 8.0 lands, bc it relates to a breaking change or similar. Epic: Logs & Metrics 8.0 Plan labels Nov 5, 2021
@matschaffer matschaffer self-assigned this Nov 5, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

@matschaffer
Copy link
Contributor Author

@jasonrhodes let me know if this approach makes sense to to you. I could alternatively put this on ice until the beats work is done, then use a real stack to do dev UI work.

@matschaffer
Copy link
Contributor Author

Just wanted to note I also tried out some other data generator tooling:

  1. essiem - this has an esexport function I thought might be useful for exporting real data, but the randomization seemed very fixed on replacing only agent host name information
  2. https://github.com/elastic/kibana/tree/main/packages/elastic-apm-synthtrace - this might work, but seemed to have some baked in assumptions around apm data streams. Will chat with @dgieselaar to see if he thinks it'd fit to try to have synthtrace also produce non-data-stream feeds that SM uses today.

@matschaffer
Copy link
Contributor Author

I'm trying out #118302 for data simulation next

@matschaffer matschaffer removed their assignment Nov 18, 2021
@matschaffer matschaffer removed v8.0.0 required-for-8.0 This work is required to be done before 8.0 lands, bc it relates to a breaking change or similar. Epic: Logs & Metrics 8.0 Plan labels Nov 24, 2021
@matschaffer
Copy link
Contributor Author

This is no longer a blocker for 8.0 since I was able to get the data I needed by enabling kibana's internal collection.

@matschaffer
Copy link
Contributor Author

Moving to in progress since at least me an @miltonhultgren have been looking into it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services
Projects
None yet
3 participants