Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more index options in create statement #1

Draft
wants to merge 17 commits into
base: add-covering-index-sql-support
Choose a base branch
from

Conversation

dai-chen
Copy link
Owner

@dai-chen dai-chen commented Sep 8, 2023

Support more index options in WITH clause in CREATE statements:

CREATE SKIPPING/INDEX/MV ...
WITH (
  auto_refresh = [true | false],
  auto_start = [true | false],
  refresh_interval = '1s',
  checkpoint_location = 's3://...',
  index_settings = '{ "key": value ... }'
);

Example

CREATE SKIPPING INDEX ON ds_tables.lineitem_tiny
(
  l_shipdate VALUE_SET
)
WITH (
  auto_start = true,
  auto_refresh = false,
  checkpoint_location = 's3://test',
  index_settings = '{"number_of_shards":9,"number_of_replicas":2}'
);

GET flint_ds_tables_lineitem_tiny_skipping_index/_mapping
TODO

GET flint_ds_tables_lineitem_tiny_skipping_index/_settings
{
  "flint_ds_tables_lineitem_tiny_skipping_index": {
    "settings": {
      "index": {
        "creation_date": "1694202650976",
        "number_of_shards": "9",
        "number_of_replicas": "2",
        "uuid": "Xabjyw0XTbOMF6SIR2Voxw",
        "version": {
          "created": "136267827"
        },
        "provided_name": "flint_ds_tables_lineitem_tiny_skipping_index"
      }
    }
  }
}

TODO

After rebase, publish the final PR with the following items:

  1. Add UT for different options
  2. Add IT for some options, ex. auto_start, auto_refresh
  3. Should not pass options to indexBuilder.create?

Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
@dai-chen dai-chen force-pushed the add-covering-index-sql-support branch from 88ca8dd to ad7f353 Compare September 18, 2023 20:51
dai-chen pushed a commit that referenced this pull request Sep 3, 2024
* Fix distinct_count, Add test for distinct_count

Signed-off-by: Hendrik Saly <[email protected]>

* add tests for stats: distinct_count (#1)

add tests for stats: distinct_count

Signed-off-by: Kacper Trochimiak <[email protected]>

* Add functions to readme

Signed-off-by: Hendrik Saly <[email protected]>

* Fix comparePlans

Signed-off-by: Hendrik Saly <[email protected]>

---------

Signed-off-by: Hendrik Saly <[email protected]>
Signed-off-by: Kacper Trochimiak <[email protected]>
Co-authored-by: Kacper Trochimiak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant