[UII] Advanced agent monitoring options UI for HTTP endpoint and diagnostics #193361
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…migration to move them to the new properties
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
jen-huang
added
Team:Fleet
1 task
jen-huang
commented
Sep 18, 2024
|
Pinging @elastic/fleet (Team:Fleet) |
jen-huang
changed the title
elastic-vault-github-plugin-prod
bot
requested a review
from a team
as a code owner
…to feat/http-monitoring
… src/core/server/integration_tests/ci_checks'
…to feat/http-monitoring
|
The code changes look great! It would be nice to test that these options are actually applied on the agent side correctly. |
@juliaElastic I spent some time testing today:
|
juliaElastic
approved these changes
Sep 20, 2024
kilfoyle
approved these changes
Sep 20, 2024
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: cc @jen-huang |
gsoldevila
approved these changes
Sep 22, 2024
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Sep 22, 2024
…nostics (elastic#193361) ## Summary Resolves elastic#153950. This PR implements a UI to configure advanced Elastic Agent monitoring options under agent policy settings. These advanced options include enabling HTTP monitoring endpoints and various options for agent diagnostics. They are shown under an a toggle under the existing agent monitoring logs and metrics collection options: <img width="1326" alt="image" src="https://github.com/user-attachments/assets/ac8cbe00-d838-4c9a-8a35-3dbf31222dc9"> If the base HTTP monitoring endpoint is not enabled, the rest of the HTTP options are disabled: <img width="1328" alt="image" src="https://github.com/user-attachments/assets/2eac787c-3055-4862-b3eb-2566a39ee86c"> The following new fields are added to agent policy schema to support this: ``` monitoring_http monitoring_pprof_enabled monitoring_diagnostics ``` This work supersedes the previous `HTTP monitoring endpoint` options under `Advanced Settings` at the bottom of the page. Any previous configuration under an agent policy's `advanced_settings.agent_monitoring_http` saved object field are migrated over to the new `monitoring_http` field and the old field is deleted. See the migration fn `backfillAgentPolicyToV4`. These new options are compiled to agent yaml like this: ```yml agent: monitoring: enabled: true use_output: default logs: true metrics: true traces: true namespace: default pprof: enabled: true http: enabled: true host: localhost port: 6791 diagnostics: limit: interval: 1m burst: 1 uploader: max_retries: 10 init_dur: 1s max_dur: 10m ``` Summarize your PR. If it involves visual changes include a screenshot or gif. ### To-do - [x] API integration tests - [x] Full manual test of SO migration - [x] Full manual test with agent using these settings ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - elastic/ingest-docs#1333 - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 87cdc2d)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Sep 22, 2024
…d diagnostics (#193361) (#193658) # Backport This will backport the following commits from `main` to `8.x`: - [[UII] Advanced agent monitoring options UI for HTTP endpoint and diagnostics (#193361)](#193361) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Jen Huang","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-22T10:49:33Z","message":"[UII] Advanced agent monitoring options UI for HTTP endpoint and diagnostics (#193361)\n\n## Summary\r\n\r\nResolves https://github.com/elastic/kibana/issues/153950.\r\n\r\nThis PR implements a UI to configure advanced Elastic Agent monitoring\r\noptions under agent policy settings. These advanced options include\r\nenabling HTTP monitoring endpoints and various options for agent\r\ndiagnostics. They are shown under an a toggle under the existing agent\r\nmonitoring logs and metrics collection options:\r\n\r\n<img width=\"1326\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/ac8cbe00-d838-4c9a-8a35-3dbf31222dc9\">\r\n\r\nIf the base HTTP monitoring endpoint is not enabled, the rest of the\r\nHTTP options are disabled:\r\n\r\n<img width=\"1328\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/2eac787c-3055-4862-b3eb-2566a39ee86c\">\r\n\r\nThe following new fields are added to agent policy schema to support\r\nthis:\r\n```\r\nmonitoring_http\r\nmonitoring_pprof_enabled\r\nmonitoring_diagnostics\r\n```\r\n\r\nThis work supersedes the previous `HTTP monitoring endpoint` options\r\nunder `Advanced Settings` at the bottom of the page. Any previous\r\nconfiguration under an agent policy's\r\n`advanced_settings.agent_monitoring_http` saved object field are\r\nmigrated over to the new `monitoring_http` field and the old field is\r\ndeleted. See the migration fn `backfillAgentPolicyToV4`.\r\n\r\nThese new options are compiled to agent yaml like this:\r\n\r\n```yml\r\nagent:\r\n monitoring:\r\n enabled: true\r\n use_output: default\r\n logs: true\r\n metrics: true\r\n traces: true\r\n namespace: default\r\n pprof:\r\n enabled: true\r\n http:\r\n enabled: true\r\n host: localhost\r\n port: 6791\r\n diagnostics:\r\n limit:\r\n interval: 1m\r\n burst: 1\r\n uploader:\r\n max_retries: 10\r\n init_dur: 1s\r\n max_dur: 10m\r\n```\r\nSummarize your PR. If it involves visual changes include a screenshot or\r\ngif.\r\n\r\n### To-do\r\n- [x] API integration tests\r\n- [x] Full manual test of SO migration\r\n- [x] Full manual test with agent using these settings\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n - elastic/ingest-docs#1333 \r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"87cdc2db728b088a44ff6e1977679f326bfd38d2","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Fleet","v9.0.0","release_note:feature","backport:prev-minor"],"title":"[UII] Advanced agent monitoring options UI for HTTP endpoint and diagnostics","number":193361,"url":"https://github.com/elastic/kibana/pull/193361","mergeCommit":{"message":"[UII] Advanced agent monitoring options UI for HTTP endpoint and diagnostics (#193361)\n\n## Summary\r\n\r\nResolves https://github.com/elastic/kibana/issues/153950.\r\n\r\nThis PR implements a UI to configure advanced Elastic Agent monitoring\r\noptions under agent policy settings. These advanced options include\r\nenabling HTTP monitoring endpoints and various options for agent\r\ndiagnostics. They are shown under an a toggle under the existing agent\r\nmonitoring logs and metrics collection options:\r\n\r\n<img width=\"1326\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/ac8cbe00-d838-4c9a-8a35-3dbf31222dc9\">\r\n\r\nIf the base HTTP monitoring endpoint is not enabled, the rest of the\r\nHTTP options are disabled:\r\n\r\n<img width=\"1328\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/2eac787c-3055-4862-b3eb-2566a39ee86c\">\r\n\r\nThe following new fields are added to agent policy schema to support\r\nthis:\r\n```\r\nmonitoring_http\r\nmonitoring_pprof_enabled\r\nmonitoring_diagnostics\r\n```\r\n\r\nThis work supersedes the previous `HTTP monitoring endpoint` options\r\nunder `Advanced Settings` at the bottom of the page. Any previous\r\nconfiguration under an agent policy's\r\n`advanced_settings.agent_monitoring_http` saved object field are\r\nmigrated over to the new `monitoring_http` field and the old field is\r\ndeleted. See the migration fn `backfillAgentPolicyToV4`.\r\n\r\nThese new options are compiled to agent yaml like this:\r\n\r\n```yml\r\nagent:\r\n monitoring:\r\n enabled: true\r\n use_output: default\r\n logs: true\r\n metrics: true\r\n traces: true\r\n namespace: default\r\n pprof:\r\n enabled: true\r\n http:\r\n enabled: true\r\n host: localhost\r\n port: 6791\r\n diagnostics:\r\n limit:\r\n interval: 1m\r\n burst: 1\r\n uploader:\r\n max_retries: 10\r\n init_dur: 1s\r\n max_dur: 10m\r\n```\r\nSummarize your PR. If it involves visual changes include a screenshot or\r\ngif.\r\n\r\n### To-do\r\n- [x] API integration tests\r\n- [x] Full manual test of SO migration\r\n- [x] Full manual test with agent using these settings\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n - elastic/ingest-docs#1333 \r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"87cdc2db728b088a44ff6e1977679f326bfd38d2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193361","number":193361,"mergeCommit":{"message":"[UII] Advanced agent monitoring options UI for HTTP endpoint and diagnostics (#193361)\n\n## Summary\r\n\r\nResolves https://github.com/elastic/kibana/issues/153950.\r\n\r\nThis PR implements a UI to configure advanced Elastic Agent monitoring\r\noptions under agent policy settings. These advanced options include\r\nenabling HTTP monitoring endpoints and various options for agent\r\ndiagnostics. They are shown under an a toggle under the existing agent\r\nmonitoring logs and metrics collection options:\r\n\r\n<img width=\"1326\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/ac8cbe00-d838-4c9a-8a35-3dbf31222dc9\">\r\n\r\nIf the base HTTP monitoring endpoint is not enabled, the rest of the\r\nHTTP options are disabled:\r\n\r\n<img width=\"1328\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/2eac787c-3055-4862-b3eb-2566a39ee86c\">\r\n\r\nThe following new fields are added to agent policy schema to support\r\nthis:\r\n```\r\nmonitoring_http\r\nmonitoring_pprof_enabled\r\nmonitoring_diagnostics\r\n```\r\n\r\nThis work supersedes the previous `HTTP monitoring endpoint` options\r\nunder `Advanced Settings` at the bottom of the page. Any previous\r\nconfiguration under an agent policy's\r\n`advanced_settings.agent_monitoring_http` saved object field are\r\nmigrated over to the new `monitoring_http` field and the old field is\r\ndeleted. See the migration fn `backfillAgentPolicyToV4`.\r\n\r\nThese new options are compiled to agent yaml like this:\r\n\r\n```yml\r\nagent:\r\n monitoring:\r\n enabled: true\r\n use_output: default\r\n logs: true\r\n metrics: true\r\n traces: true\r\n namespace: default\r\n pprof:\r\n enabled: true\r\n http:\r\n enabled: true\r\n host: localhost\r\n port: 6791\r\n diagnostics:\r\n limit:\r\n interval: 1m\r\n burst: 1\r\n uploader:\r\n max_retries: 10\r\n init_dur: 1s\r\n max_dur: 10m\r\n```\r\nSummarize your PR. If it involves visual changes include a screenshot or\r\ngif.\r\n\r\n### To-do\r\n- [x] API integration tests\r\n- [x] Full manual test of SO migration\r\n- [x] Full manual test with agent using these settings\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n - elastic/ingest-docs#1333 \r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"87cdc2db728b088a44ff6e1977679f326bfd38d2"}}]}] BACKPORT--> --------- Co-authored-by: Jen Huang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Resolves #153950.
This PR implements a UI to configure advanced Elastic Agent monitoring options under agent policy settings. These advanced options include enabling HTTP monitoring endpoints and various options for agent diagnostics. They are shown under an a toggle under the existing agent monitoring logs and metrics collection options:
If the base HTTP monitoring endpoint is not enabled, the rest of the HTTP options are disabled:
The following new fields are added to agent policy schema to support this:
This work supersedes the previous
HTTP monitoring endpointoptions underAdvanced Settingsat the bottom of the page. Any previous configuration under an agent policy'sadvanced_settings.agent_monitoring_httpsaved object field are migrated over to the newmonitoring_httpfield and the old field is deleted. See the migration fnbackfillAgentPolicyToV4.These new options are compiled to agent yaml like this:
Summarize your PR. If it involves visual changes include a screenshot or gif.
To-do
Checklist
Delete any items that are not applicable to this PR.