[Logs Overview] Overview component (iteration 1) (attempt 2) #195673

weltenwort · 2024-10-09T19:35:31Z

ℹ️ This is a re-submission of #191899 which was reverted due to a storybook build problem.

📝 Summary

This introduces a "Logs Overview" component for use in solution UIs behind a feature flag.

🏁 closes: [Logs Overview] Enhanced logs component for solution UIs #190848
🐞 affected by: change_point agg fails with IllegalArgumentException while indexing an array elasticsearch#112805

🏴‍☠️ Feature flag

The new logs overview is currently visible in the "Logs" tab of APM services when the feature flag observability:newLogsOverview is enabled in the advanced settings:

🎨 Previews

In APM UI

In the hosts UI

No log entries

⚠️ Known issues

Since this uses the change_point Elasticsearch aggregation extensively it is affected by change_point agg fails with IllegalArgumentException while indexing an array elasticsearch#112805.

❓ Open questions

See: #191899 (comment)

The change types returned by the change_point agg have some limitations:
- Some of them are hard to interpret (i.e. non-stationary)
- They are sensitive to slight changes in the distribution. A change can flap between a step, a spike and a distribution_change when just a few buckets are different. So it might be advisable to use the change point buckets detected and re-interpret their meaning using the histogram.
It's debatable whether a lexicographical sort order of the change types is a good idea. We might want to come up with more useful sorting weights.
Since the changes types shown in the design mock-ups don't line up with reality the assignment of the colors is unclear. Currently no types' badges are colored.

🕵️‍♀️ Review notes

This adds XState v5 as an additional dependency to start the transition. It can be imported as xstate5. Similarly, @xstate5/react is available.

Release Note

Add experimental logs overview to the observability hosts and service overviews

…mbeddable-iteration-1

elasticmachine · 2024-10-09T19:35:34Z

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

elasticmachine · 2024-10-09T19:35:38Z

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

sabarasaba

kbn management changes lgtm

crespocarlos

Nice additions to synthtrace. Thanks.

crespocarlos · 2024-10-10T07:34:25Z

...vability_solution/infra/public/pages/metrics/hosts/components/tabs/logs/logs_tab_content.tsx

@@ -84,6 +85,7 @@ export const LogsTabContent = () => {
            query={logsLinkToStreamQuery}
            logView={logView}
          />
+          ]


I guess we can remove this?

oops, indeed

crespocarlos · 2024-10-10T07:40:24Z

x-pack/plugins/observability_solution/apm/public/components/app/service_logs/index.tsx

+            {
+              term: {
+                [SERVICE_ENVIRONMENT]: environment,
+              },
+            },


We could use helper functions to build the filer clause. Wdyt?

...termQuery(SERVICE_ENVIRONMENT, environment),

Same for the exists below

existsQuery(SERVICE_ENVIRONMENT)

weltenwort · 2024-10-10T08:28:08Z

Hey @crespocarlos, thanks for the review and the suggestions. Since this is a re-submission of #191899, which was already merged but reverted, I hope you don't mind if I defer your suggested improvements to a follow-up PR?

crespocarlos · 2024-10-10T09:07:44Z

Hey @crespocarlos, thanks for the review and the suggestions. Since this is a re-submission of #191899, which was already merged but reverted, I hope you don't mind if I defer your suggested improvements to a follow-up PR?

Sure thing :)

elasticmachine · 2024-10-10T09:32:58Z

💚 Build Succeeded

Buildkite Build
Commit: 70000e3
Storybooks Preview
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-195673-70000e3daba3

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`datasetQuality`	224	225	+1
`infra`	1508	1509	+1
`logsExplorer`	562	563	+1
`logsShared`	228	298	+70
`observabilityLogsExplorer`	203	204	+1
`observabilityOnboarding`	255	256	+1
total			+75

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/apm-synthtrace-client`	208	223	+15
`@kbn/management-settings-ids`	142	143	+1
`@kbn/observability-logs-overview`	-	27	+27
`@kbn/xstate-utils`	13	14	+1
`logsShared`	281	284	+3
total			+47

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`apm`	3.4MB	3.4MB	+1.7KB
`infra`	1.6MB	1.6MB	+632.0B
`logsShared`	140.5KB	327.3KB	+186.8KB
total			+189.2KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`@kbn/apm-synthtrace-client`	33	36	+3
`@kbn/observability-logs-overview`	-	3	+3
`logsShared`	32	34	+2
total			+8

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`logsShared`	173.3KB	174.5KB	+1.1KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/apm-synthtrace-client`	208	223	+15
`@kbn/management-settings-ids`	144	145	+1
`@kbn/observability-logs-overview`	-	29	+29
`@kbn/xstate-utils`	13	14	+1
`logsShared`	310	313	+3
total			+49

async chunk count

id	before	after	diff
`logsShared`	14	15	+1

ESLint disabled line counts

id	before	after	diff
`@kbn/xstate-utils`	0	1	+1

Total ESLint disabled count

id	before	after	diff
`@kbn/xstate-utils`	0	1	+1

History

💚 Build #240885 succeeded 782e0b9

cc @weltenwort

afharo

Telemetry changes LGTM

weltenwort · 2024-10-10T10:44:36Z

Thanks everyone for re-approving so quickly ❤️

kibanamachine · 2024-10-10T10:46:51Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11272491981

kibanamachine · 2024-10-10T10:51:28Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 195673

Questions ?

Please refer to the Backport tool documentation

weltenwort · 2024-10-10T10:58:03Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…#195673) This is a re-submission of elastic#191899, which was reverted due to a storybook build problem. This introduces a "Logs Overview" component for use in solution UIs behind a feature flag. --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Kerry Gallagher <[email protected]> Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 0caea22) # Conflicts: # .github/CODEOWNERS # src/plugins/telemetry/schema/oss_plugins.json

…195673) (#195742) # Backport This will backport the following commits from `main` to `8.x`: - [[Logs Overview] Overview component (iteration 1) (attempt 2) (#195673)](#195673)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)

weltenwort added 30 commits August 20, 2024 18:11

Add rare logs scenario

b3a1af8

Add empty logs_overview package

0a43169

Add gaussian distribution to the unstructured logs scenario

d373e59

Add support for xstate5

278365e

Add xstate5 console inspector

1f4bd0f

Add cateogorization service

76ea2a5

Add categories grid

0528e8b

Enable translation linting for the package

0b7fc2c

Improve typing of the sorting config

b428bd4

Improve default column widths

5be73af

Add loading screen with cancellation

e0af1bb

Merge branch 'main' into logs-enhanced-embeddable-iteration-1

c70a37a

Bump XState 5 version

169df23

Start cleaning up names and introducing the overview component

901bee3

Provide a self-contained overview via logs_shared

1023d69

Fix unnecessary uiSettings dependency

0d35529

Fix some refactoring mistakes

d551da4

Memoize time overview props

47153c8

Update zod import

dc8ea70

Add service filters and error reporting

3793877

Fix fetching remaining categories

c69a83b

Add logs overview feature flag

870d347

Add logs overview in infra hosts

d08f104

Fix sampling probability of second pass

6ecd056

Signal categories limit being reached

b0b1fe7

Respect the abort signal when categorizing

8c09788

Add empty state screen

b028eab

Merge branch 'main' of github.com:elastic/kibana into logs-enhanced-e…

6a1ed55

…mbeddable-iteration-1

Merge branch 'main' of github.com:elastic/kibana into logs-enhanced-e…

463355c

…mbeddable-iteration-1

Fix race condition when rendering apm service details

6aad1a1

weltenwort requested review from a team as code owners October 9, 2024 19:35

botelastic bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Oct 9, 2024

sabarasaba approved these changes Oct 10, 2024

View reviewed changes

crespocarlos reviewed Oct 10, 2024

View reviewed changes

Remove accidental character in logs tab content

70000e3

Kerry350 approved these changes Oct 10, 2024

View reviewed changes

crespocarlos approved these changes Oct 10, 2024

View reviewed changes

delanni approved these changes Oct 10, 2024

View reviewed changes

afharo approved these changes Oct 10, 2024

View reviewed changes

weltenwort merged commit 0caea22 into elastic:main Oct 10, 2024
44 checks passed

kibanamachine added the v9.0.0 label Oct 10, 2024

weltenwort mentioned this pull request Oct 10, 2024

[8.x] [Logs Overview] Overview component (iteration 1) (attempt 2) (#195673) #195742

Merged

kibanamachine added the v8.16.0 label Oct 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Logs Overview] Overview component (iteration 1) (attempt 2) #195673

[Logs Overview] Overview component (iteration 1) (attempt 2) #195673

weltenwort commented Oct 9, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Oct 9, 2024

elasticmachine commented Oct 9, 2024

sabarasaba left a comment

crespocarlos left a comment

crespocarlos Oct 10, 2024

weltenwort Oct 10, 2024

crespocarlos Oct 10, 2024 •

edited

Loading

weltenwort commented Oct 10, 2024 •

edited

Loading

crespocarlos commented Oct 10, 2024

elasticmachine commented Oct 10, 2024 •

edited

Loading

API count

async chunk count

ESLint disabled line counts

Total ESLint disabled count

afharo left a comment

weltenwort commented Oct 10, 2024

kibanamachine commented Oct 10, 2024

kibanamachine commented Oct 10, 2024

weltenwort commented Oct 10, 2024

[Logs Overview] Overview component (iteration 1) (attempt 2) #195673

[Logs Overview] Overview component (iteration 1) (attempt 2) #195673

Conversation

weltenwort commented Oct 9, 2024 • edited by kibanamachine Loading

📝 Summary

🏴‍☠️ Feature flag

🎨 Previews

In APM UI

In the hosts UI

No log entries

⚠️ Known issues

❓ Open questions

🕵️‍♀️ Review notes

Release Note

elasticmachine commented Oct 9, 2024

elasticmachine commented Oct 9, 2024

sabarasaba left a comment

Choose a reason for hiding this comment

crespocarlos left a comment

Choose a reason for hiding this comment

crespocarlos Oct 10, 2024

Choose a reason for hiding this comment

weltenwort Oct 10, 2024

Choose a reason for hiding this comment

crespocarlos Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

weltenwort commented Oct 10, 2024 • edited Loading

crespocarlos commented Oct 10, 2024

elasticmachine commented Oct 10, 2024 • edited Loading

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

async chunk count

ESLint disabled line counts

Total ESLint disabled count

History

afharo left a comment

Choose a reason for hiding this comment

weltenwort commented Oct 10, 2024

kibanamachine commented Oct 10, 2024

kibanamachine commented Oct 10, 2024

💔 All backports failed

Manual backport

Questions ?

weltenwort commented Oct 10, 2024

💚 All backports created successfully

Questions ?

weltenwort commented Oct 9, 2024 •

edited by kibanamachine

Loading

crespocarlos Oct 10, 2024 •

edited

Loading

weltenwort commented Oct 10, 2024 •

edited

Loading

elasticmachine commented Oct 10, 2024 •

edited

Loading