Add _flush API #7217

Naarcha-AWS · 2024-05-21T15:43:04Z

Closes #2096

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Archer <[email protected]>

_api-reference/index-apis/flush.md

Signed-off-by: Naarcha-AWS <[email protected]>

sohami · 2024-05-23T18:36:20Z

_api-reference/index-apis/flush.md

+| `expand_wildcards` | String | Specifies the types of indexes to which wildcard expressions can expand. Supports comma-separated values. Valid values are: <br> - `all`: Expand to all open and closed indexes, including hidden indexes. <br> - `open`: Expand to open indexes. <br> - `closed`: Expand to closed indexes. <br> - `hidden`: Include hidden indexes when expanding. Must be combined with `open`, `closed`, or both. <br> - `none`: Do not accept wildcard expressions. <br> Default is `open`. |
+| `force` | Boolean | When `true`, forces a flush to occur even when no changes to the index exist in-memory. Default is `true`. |
+| `ignore_unavailable` | Boolean | When `true`, OpenSearch ignores missing or closed indexes. If `false`, OpenSearch returns an error if the force merge operation encounters missing or closed indexes. Default is `false`. |
+| `wait_if_ongoing` | Boolean | When `true`, the Flush API does not run while another flush request is active. When `false`, OpenSearch returns an error if another flush request is active. |


Defaults to true

sohami · 2024-05-23T18:50:43Z

_api-reference/index-apis/flush.md

+**Introduced 1.0**
+{: .label .label-purple }
+
+The Flush API manually flushes in-memory data from an index onto disk. This ensures that any memory from the index stored in the buffer persists on disk, as opposed to writing all buffer data to the disk, which can affect the performance of your cluster.


The Flush API permanently stores all the in-memory operation to the lucene index on disk. The operations that are flushed to lucene index are not needed to be maintained in the transaction log anymore to recover in the event of the process restart. OpenSearch automatically performs the flush in background based on certain conditions such as the size of transaction log (controlled by index.translog.flush_threshold_size) vs performing for each operation for performance reasons.

@sohami: Does whether or not the indexing occured with Lucene matter in the cast of this API? In other words, is mentioning Lucene in the API definition required?

All the on-disk representation of the data is lucene index, hence the mention. Or we can say ... in-memory operation to the segments on disk. The operations that are flushed to the segments are not required..... Also tagging @mgodwan from indexing team to take a second look at the document

_api-reference/index-apis/flush.md

Signed-off-by: Naarcha-AWS <[email protected]>

_api-reference/index-apis/flush.md

Signed-off-by: Naarcha-AWS <[email protected]>

_api-reference/index-apis/flush.md

Signed-off-by: Naarcha-AWS <[email protected]>

vagimeli

@Naarcha-AWS Minimal changes to tighten or clarify wording. LGTM.

_api-reference/index-apis/flush.md

Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

natebower

@Naarcha-AWS LGTM with a few changes. Thanks!

_api-reference/index-apis/flush.md

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

* Add _flush API Signed-off-by: Archer <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update _api-reference/index-apis/flush.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update flush.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Archer <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Nathan Bower <[email protected]> (cherry picked from commit 9440e16) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Add _flush API Signed-off-by: Archer <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update _api-reference/index-apis/flush.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update flush.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Archer <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: RasonJ <[email protected]>

Add _flush API

66a324c

Signed-off-by: Archer <[email protected]>

Naarcha-AWS added 3 - Tech review PR: Tech review in progress backport 1.3 PR: Backport label for v1.3.x backport 2.14 PR: Backport label for 2.14 labels May 21, 2024

Naarcha-AWS self-assigned this May 21, 2024

Naarcha-AWS requested review from hdhalter, kolchfa-aws, vagimeli, AMoo-Miki, natebower, dlvenable, stephen-crawford and epugh as code owners May 21, 2024 15:43

Naarcha-AWS commented May 21, 2024

View reviewed changes

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

Apply suggestions from code review

2dfa4a5

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS added the API label May 22, 2024

sohami reviewed May 23, 2024

View reviewed changes

Naarcha-AWS commented May 23, 2024

View reviewed changes

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

Apply suggestions from code review

36a4666

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS commented May 29, 2024

View reviewed changes

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

Update _api-reference/index-apis/flush.md

de50ebb

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS added 4 - Doc review PR: Doc review in progress and removed 3 - Tech review PR: Tech review in progress labels Jun 4, 2024

Naarcha-AWS commented Jun 5, 2024

View reviewed changes

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

Naarcha-AWS added 2 commits June 5, 2024 12:08

Apply suggestions from code review

6e8c589

Signed-off-by: Naarcha-AWS <[email protected]>

Update flush.md

2c5a3cf

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Jun 5, 2024

vagimeli approved these changes Jun 5, 2024

View reviewed changes

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

Apply suggestions from code review

c5308c1

Co-authored-by: Melissa Vagi <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

natebower approved these changes Jun 7, 2024

View reviewed changes

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

_api-reference/index-apis/flush.md Outdated Show resolved Hide resolved

Apply suggestions from code review

9d54ef9

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS merged commit 9440e16 into main Jun 10, 2024
6 checks passed

opensearch-trigger-bot bot mentioned this pull request Jun 10, 2024

[Backport 1.3] Add _flush API #7344

Merged

opensearch-trigger-bot bot mentioned this pull request Jun 10, 2024

[Backport 2.14] Add _flush API #7345

Merged

github-actions bot pushed a commit that referenced this pull request Jun 10, 2024

Add _flush API (#7217) (#7344)

205ceae

github-actions bot pushed a commit that referenced this pull request Jun 10, 2024

Add _flush API (#7217) (#7345)

1971855

hdhalter added 3 - Done Issue is done/complete and removed 5 - Editorial review PR: Editorial review in progress labels Jul 19, 2024

Naarcha-AWS deleted the flush branch December 19, 2024 18:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add _flush API #7217

Add _flush API #7217

Naarcha-AWS commented May 21, 2024

sohami May 23, 2024

sohami May 23, 2024 •

edited

Loading

Naarcha-AWS May 23, 2024

sohami May 23, 2024

vagimeli left a comment

natebower left a comment

Add _flush API #7217

Add _flush API #7217

Conversation

Naarcha-AWS commented May 21, 2024

Checklist

sohami May 23, 2024

Choose a reason for hiding this comment

sohami May 23, 2024 • edited Loading

Choose a reason for hiding this comment

Naarcha-AWS May 23, 2024

Choose a reason for hiding this comment

sohami May 23, 2024

Choose a reason for hiding this comment

vagimeli left a comment

Choose a reason for hiding this comment

natebower left a comment

Choose a reason for hiding this comment

sohami May 23, 2024 •

edited

Loading