Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Skipping index query rewriter cannot rewrite query on SQL view #199

Open
dai-chen opened this issue Dec 14, 2023 · 0 comments
Open
Labels
bug Something isn't working

Comments

@dai-chen
Copy link
Collaborator

What is the bug?

Flint successfully creates skipping indexes on SQL views. However, queries involving applicable filtering conditions aren't rewritten by the skipping index rewriter.

The root cause after investigation is: the skipping index query rewrite occurs post view resolution. While Flint's skipping index metadata stores the view as source, the query plan given to the rewriter is based on the source table with the view replaced already.

How can one reproduce the bug?

CREATE VIEW waf_logs_view_clientip 
AS SELECT httpRequest.clientIp as clientIp
FROM waf_logs;

CREATE SKIPPING INDEX ON waf_logs_view_clientip (
  clientIp VALUE_SET
) WITH (
  auto_refresh = true
);

# InMemoryFileIndex present without rewrite
EXPLAIN
SELECT * FROM glue.default.waf_logs_view_clientip
WHERE clientIp = "127.0.0.1"

What is the expected behavior?

Query should be rewritten with FlintSparkSkippingFileIndex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant