Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add index settings option in create statement #44

Merged
merged 8 commits into from
Oct 4, 2023

Conversation

dai-chen
Copy link
Collaborator

@dai-chen dai-chen commented Sep 25, 2023

Description

  1. Add new index_settings option. Please see doc: https://github.com/dai-chen/opensearch-spark/blob/add-index-settings-option/docs/index.md#create-index-options
  2. Use a new FlintMetadata.indexSettings field to pass it to Flint client. Will refactor the Flint metadata and finalize spec in Refactoring Flint metadata and build index API #24

Example

spark-sql> CREATE INDEX orderkey_and_quantity
         > ON stream.lineitem_tiny (l_orderkey, l_quantity)
         > WITH (
         >   index_settings = '{"number_of_shards":9,"number_of_replicas":2}'
         > );

GET flint_stream_lineitem_tiny_orderkey_and_quantity_index/_settings
{
  "flint_stream_lineitem_tiny_orderkey_and_quantity_index": {
    "settings": {
      "index": {
        "creation_date": "1695680614547",
        "number_of_shards": "9",
        "number_of_replicas": "2",
        "uuid": "M-HlOt8yT1i7hKEm_GJmOw",
        "version": {
          "created": "136267827"
        },
        "provided_name": "flint_stream_lineitem_tiny_orderkey_and_quantity_index"
      }
    }
  }
}

Issues Resolved

#26

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dai-chen dai-chen added the enhancement New feature or request label Sep 25, 2023
@dai-chen dai-chen self-assigned this Sep 25, 2023
vmmusings
vmmusings previously approved these changes Sep 26, 2023
@dai-chen dai-chen force-pushed the add-index-settings-option branch from f904ed7 to deeb633 Compare September 29, 2023 00:15
@dai-chen dai-chen marked this pull request as ready for review September 29, 2023 00:33
@penghuo penghuo merged commit 654cb6d into opensearch-project:main Oct 4, 2023
4 checks passed
@dai-chen dai-chen deleted the add-index-settings-option branch October 4, 2023 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants