[8.x] [Response Ops][Event Log] Updating event log mappings if data stream and index template already exist (#193205) #193589
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…and index template already exist (elastic#193205) Resolves elastic#192682 ## Summary As of 8.8, we started writing all event log documents to the `.kibana-event-log-ds` index. Prior to this, we created a new index template and data stream for every version (`.kibana-event-log-8.7` for example) so any mapping updates that were added for the version were created in the new index on upgrade. With the static index name and serverless, we need a way to update mappings in existing indices. This PR uses the same mechanism that we use for the alerts index to update the index template mappings and the mappings for the concrete backing indices of a datastream. ## To Verify Run ES and Kibana in `main` to test the upgrade path for serverless a. Check out `main`, run ES: `yarn es snapshot --license trial --ssl -E path.data=../test_el_upgrade` and Kibana `yarn start --ssl` b. Create a rule and let it run to populate the event log index c. Switch to this PR branch. Make a mapping update to the event log index: ``` --- a/x-pack/plugins/event_log/generated/mappings.json +++ b/x-pack/plugins/event_log/generated/mappings.json @@ -172,6 +172,9 @@ }, "rule": { "properties": { + "test": { + "type": "keyword" + }, "author": { "ignore_above": 1024, "type": "keyword", ``` d. Start ES and Kibana with the same commands as above e. Verify that the `.kibana-event-log-ds` index is created and has the updated mapping: - https://localhost:5601/app/management/data/index_management/templates/.kibana-event-log-template - https://localhost:5601/app/management/data/index_management/indices/index_details?indexName=.ds-.kibana-event-log-ds-2024.09.17-000001&filter=.kibana-&includeHiddenIndices=true&tab=mappings I also verified the following: 1. Run ES and Kibana in 8.7 to test the upgrade path from 8.7 (when event log indices were versioned) to now 2. Run ES and Kibana in 8.15 to test the upgrade path from the previous release to now However, I had to create an 8.x branch and cherry pick this commit because `main` is now on 9.0 and we can't upgrade directly from older 8.x version to 9.0! --------- Co-authored-by: Elastic Machine <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit e2798de)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Public APIs missing comments
To update your PR or re-run it, just comment with: cc @ymao1 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.x:Questions ?
Please refer to the Backport tool documentation