Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use refresh policy from config #530

Merged
merged 1 commit into from
Aug 7, 2024

Conversation

ykmr1224
Copy link
Collaborator

@ykmr1224 ykmr1224 commented Aug 6, 2024

Description

  • Use refresh policy from config when calling OpenSearch APIs
  • This will allow Flint to connect to OpenSearch Serverless collection. It originally use WAIT_FOR refresh policy for some requests and it failed against Serverless collection since it only accept NONE refresh policy.
  • To use Serverless collection as index store, spark.datasource.flint.write.refresh_policy=false config needs to be specified.
  • Added integration test for Serverless

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@ykmr1224 ykmr1224 force-pushed the refresh-policy branch 2 times, most recently from 4663fb4 to a08da93 Compare August 7, 2024 16:51
Signed-off-by: Tomoyuki Morita <[email protected]>
@ykmr1224 ykmr1224 marked this pull request as ready for review August 7, 2024 20:32
DEVELOPER_GUIDE.md Show resolved Hide resolved
@noCharger noCharger merged commit 773ad22 into opensearch-project:main Aug 7, 2024
4 checks passed
seankao-az pushed a commit that referenced this pull request Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants