Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve otel-stream usability for local metrics troubleshooting #2364

Closed
jonathanj-square opened this issue Aug 14, 2024 · 1 comment · Fixed by #2378
Closed

Improve otel-stream usability for local metrics troubleshooting #2364

jonathanj-square opened this issue Aug 14, 2024 · 1 comment · Fixed by #2378
Assignees
@jonathanj-square

Comments

@jonathanj-square
Copy link
Contributor

jonathanj-square commented Aug 14, 2024 •
edited
Loading

Troubleshooting metrics instrumentation via otel-stream is slow and cumbersome. Console output is not ideal for this task and the rate at which metrics are collected is also too slow (logs are collected once per minute by default).

We should shift to a stack like Grafana that provides a UI for visualizing metrics.
@jonathanj-square jonathanj-square added the triage Issue needs triaging label Aug 14, 2024
@ftl-robot ftl-robot mentioned this issue Aug 14, 2024
Open
@gak
Copy link
Contributor

gak commented Aug 14, 2024

Look into the environment variable for collection period.

@github-actions github-actions bot removed the triage Issue needs triaging label Aug 14, 2024
@jonathanj-square jonathanj-square mentioned this issue Aug 15, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Aug 15, 2024
@jonathanj-square
feat: adopt Grafana for metrics visualization (#2364) (#2378)
7029173 
fixes: #2364 

Introduces a Grafana / OTEL container for capturing, ephemerally
storing, and visualizing OTEL metrics emitted by FTL. The collection
frequency is also increased to once per second (from a default once per
minute)

<img width="1246" alt="Screenshot 2024-08-14 at 7 58 39 PM"
src="https://github.com/user-attachments/assets/bd21cd33-dc93-4a79-9b2a-ad124f2db296">
safeer pushed a commit that referenced this issue Aug 19, 2024
@jonathanj-square @safeer
feat: adopt Grafana for metrics visualization (#2364) (#2378)
7bcf7d7 
fixes: #2364 

Introduces a Grafana / OTEL container for capturing, ephemerally
storing, and visualizing OTEL metrics emitted by FTL. The collection
frequency is also increased to once per second (from a default once per
minute)

<img width="1246" alt="Screenshot 2024-08-14 at 7 58 39 PM"
src="https://github.com/user-attachments/assets/bd21cd33-dc93-4a79-9b2a-ad124f2db296">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jonathanj-square jonathanj-square

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: adopt Grafana for metrics visualization (#2364) TBD54566975/ftl
2 participants
@gak @jonathanj-square