Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG][Segment Replication] The segment replication type doesn't seem to work with index or component templates. #11230

Closed
kksaha opened this issue Nov 16, 2023 · 4 comments · Fixed by #11417
Assignees
Labels
bug Something isn't working Indexing:Replication Issues and PRs related to core replication framework eg segrep v2.12.0 Issues and PRs related to version 2.12.0

Comments

@kksaha
Copy link

kksaha commented Nov 16, 2023

Describe the bug
The component/index template with replication type 'Segment' doesn't reflect on new indexes. It seems the replication type is still showing as 'Document'. I initially updated the index template, and the replication type didn't change to segment. Later on, I updated the component template, and only two index patterns were updated out of 20.

To Reproduce
Steps to reproduce the behavior:

Scenario 1:

  1. Index template with SEGMENT
    { "index_templates": [ { "name": "kk-execution-ppe", "index_template": { "index_patterns": [ "kk-execution-ppe-*" ], "template": { "settings": { "index": { "replication": { "type": "SEGMENT" }, "plugins": { "index_state_management": { "rollover_alias": "kk-execution-ppe" } } } } }, "composed_of": [ "kk" ], "priority": 100 } } ] }
  2. Index created with default index type Document
    { "kk-execution-ppe-kishore": { "settings": { "index": { "replication": { "type": "DOCUMENT" }, "mapping": { "total_fields": { "limit": "20000" } }, "indexing": { "slowlog": { "level": "warn", "reformat": "false", "threshold": { "index": { "warn": "10s", "trace": "500ms", "debug": "2s", "info": "5s" } } } }, ......... ..........

Scenario 2:

  1. Component Template with Segment replication type
    { "component_templates": [ { "name": "kk", "component_template": { "template": { "settings": { "index": { "replication": { "type": "SEGMENT" }, "mapping": { "total_fields": { "limit": "20000" } }, ......

  2. But indexes still creating with Document type except two index pattern.
    { "kk-legacy-ppe-000297": { "settings": { "index": { "replication": { "type": "DOCUMENT" }, "mapping": { "total_fields": { "limit": "20000" } }, "indexing": { "slowlog": { "level": "warn", "reformat": "false", "threshold": { "index": { "warn": "10s", "trace": "500ms", "debug": "2s", "info": "5s" } } } }, "plugins": { "index_state_management": { "rollover_alias": "kk-legacy-ppe" } }, "provided_name": "kk-legacy-ppe-000297", "creation_date": "1700120178163",

  3. But the one created with Segment replication
    { "kk-client-ppe-000187": { "settings": { "index": { "replication": { "type": "SEGMENT" }, "mapping": { "total_fields": { "limit": "20000" } }, "indexing": { "slowlog": { "level": "warn", "reformat": "false", "threshold": { "index": { "warn": "10s", "trace": "500ms", "debug": "2s", "info": "5s" } } } },

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [2.11]
@kksaha kksaha added bug Something isn't working untriaged labels Nov 16, 2023
@kksaha kksaha changed the title [BUG][Segment Replication] SEGMENT replication type doesn't seem to work with index/component templates [BUG][Segment Replication] The segment replication type doesn't seem to work with index or component templates. Nov 16, 2023
@msfroh msfroh added the Indexing:Replication Issues and PRs related to core replication framework eg segrep label Nov 16, 2023
@louzadod
Copy link

Same problem here and probably It started with 2.10.0. My previous version, 2.8.0, was OK.
In version 2.10.0 my index templates with replication type SEGMENT where ignored. All indexes where created with replication type DOCUMENT.

After migration to 2.11.0 the problem remains.

I need to customize replication type to accomodate different kind of use cases.

@anasalkouz anasalkouz added v2.12.0 Issues and PRs related to version 2.12.0 and removed untriaged labels Nov 27, 2023
@kotwanikunal kotwanikunal self-assigned this Nov 29, 2023
@kotwanikunal
Copy link
Member

Taking a look at this.

@kksaha
Copy link
Author

kksaha commented Dec 6, 2023

#11417 is still open, @kotwanikunal Do you know when it will be fixed please ?

@kotwanikunal
Copy link
Member

@kksaha The change has been merged into main and will be merged into 2.x soon.
The fix will be released with v2.12. In the meanwhile, please supply the replication type explicitly when creating the index as a workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Indexing:Replication Issues and PRs related to core replication framework eg segrep v2.12.0 Issues and PRs related to version 2.12.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants