[Feature]Add API Support for Re-adding Indices to Data Streams

[weiwang118]

Is your feature request related to a problem? Please describe.
Data Streams in OpenSearch simplify the initial setup process and work exceptionally well for time-based data like application logs, which are generally append-only. However, the current API support for data streams doesn't provide sufficient functionalities for modifying the data streams themselves.

• Use Case 1
  When indices of data streams are migrated to UltraWarm, automated snapshots will exclude those indices, but since those indices still are part of the data stream, the data stream metadata will still have these indices but automated snapshots don't. This will cause index not exists exception when customers try to restore data streams from snapshots.

• Use Case 2
  If a backing index is migrated to cold storage, it is removed from the data stream. Even if you move the index back to UltraWarm, this index remains not part of the original data stream. Since this index is already removed from the data stream, searching against the stream won't return any data from that index.

Describe the solution you'd like
I propose the development of a new API specifically crafted to add an index back into the data stream. Such an API would effectively address the above-mentioned use cases, ensuring seamless data stream management and retrieval.

[ashking94]

@weiwang118 - Some of these features are not present in OpenSearch. Can you elaborate more on the exact changes expected in OpenSource Opensearch?

[jed326]

Closing this in favor of

• [Feature Request] Ability to add a backing index to a data stream #8271