Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] The same log explorer used for Security Lake data source can’t query MV/CI indices #510

Open
A-Gray-Cat opened this issue Aug 1, 2024 · 2 comments
Labels
bug Something isn't working DataSource:OpenSearch

Comments

@A-Gray-Cat
Copy link

What is the bug?
After a MV/CI index is created using the securitylake log explorer, if I try to query the created index within the same explorer, it will return index not found error for both SQL and PPL.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Go to Data sources -> securitylake -> query data -> open log explorer
  2. Create a materialized view
CREATE MATERIALIZED VIEW last_1day_ct_2024_07_31_mv AS
    SELECT time_dt,
        actor.user.uid AS requestor_arn,
        accountid AS account_id,
        region AS region,
        src_endpoint.ip AS source_ip,
        api.service.name AS service,
        api.operation AS api_operation,
        api.request.data AS request_parameters,
        api.response.data AS response_elements,
        api.response.error AS error,
        api.response.message AS response_message,
        http_request.user_agent AS user_agent
     FROM securitylake.amazon_security_lake_glue_db_us_east_1.amazon_security_lake_table_us_east_1_cloud_trail_mgmt_2_0
     WHERE time_dt BETWEEN CURRENT_TIMESTAMP - INTERVAL '1' DAY AND CURRENT_TIMESTAMP
WITH ( auto_refresh = false)

REFRESH MATERIALIZED VIEW last_1day_ct_2024_07_31_mv

Run any query against the flint index you just created. E.g.

SELECT * FROM flint_securitylake_default_last_1day_ct_2024_07_31_mv
  1. It will error out and say index not found.

What is the expected behavior?
Quey results returned.

What is your host/environment?

  • OS: [e.g. iOS]
  • Version 2.13
  • Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

@A-Gray-Cat A-Gray-Cat added bug Something isn't working untriaged labels Aug 1, 2024
@dblock dblock removed the untriaged label Aug 19, 2024
@dblock
Copy link
Member

dblock commented Aug 19, 2024

Catch All Triage - 1, 2, 3

@dai-chen
Copy link
Collaborator

CI (Covering Index) will be utilized automatically when querying source table, as discussed in #298.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working DataSource:OpenSearch
Projects
None yet
Development

No branches or pull requests

3 participants