Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]FlintREPL error in parsing Flint query results #2408

Closed
YANG-DB opened this issue Nov 1, 2023 · 1 comment
Closed

[BUG]FlintREPL error in parsing Flint query results #2408

YANG-DB opened this issue Nov 1, 2023 · 1 comment
Assignees
Labels
bug Something isn't working Flint

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Nov 1, 2023

What is the bug?
Query from the cloudfront table (s3 based) results with the next error

{
    "data": {
        "ok": false,
        "resp": "Service Unavailable",
        "body": "{\n  \"status\": 503,\n  \"error\": {\n    \"type\": \"JSONException\",\n    \"reason\": \"There was internal problem at backend\",\n    \"details\": \"JSONObject[\\\"log_time\\\"] not found.\"\n  }\n}"
    }
}

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Create an S3 bucket based on the next raw data

Raw Data

2023-02-22	03:22:37	HKG62-C2	675	13.248.48.9	GET	d2wusnbjo8x1w7.cloudfront.net	/	304	-	Mozilla/5.0%20(Macintosh;%20Intel%20Mac%20OS%20X%2010_15_7)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/110.0.0.0%20Safari/537.36	-	-	Miss	Y5rIQOuGsI2vJN4hR3qLB55Cn4aoogvzPEnHhm5-0NiTtWDfTU5-vw==	d2wusnbjo8x1w7.cloudfront.net	https	536	0.623	-	TLSv1.3	TLS_AES_128_GCM_SHA256	Miss	HTTP/2.0	-	-	12812	0.623	Miss	-	-	-	-
2023-02-22	03:22:38	HKG62-C2	675	13.248.48.9	GET	d2wusnbjo8x1w7.cloudfront.net	/static/css/main.3c74189a.css	304	-	Mozilla/5.0%20(Macintosh;%20Intel%20Mac%20OS%20X%2010_15_7)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/110.0.0.0%20Safari/537.36	-	-	Miss	IPGkM0N8_4AU6ok71zDa4twWLigSM7Ib33IwRsBHm1hDSmIvWoNjBA==	d2wusnbjo8x1w7.cloudfront.net	https	118	0.656	-	TLSv1.3	TLS_AES_128_GCM_SHA256	Miss	HTTP/2.0	-	-	12812	0.656	Miss	-	-	-	-
2023-02-22	03:22:42	HKG62-C2	677	13.248.48.9	GET	d2wusnbjo8x1w7.cloudfront.net	/manifest.json	304	-	Mozilla/5.0%20(Macintosh;%20Intel%20Mac%20OS%20X%2010_15_7)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/110.0.0.0%20Safari/537.36	-	-	Miss	5sXuyCQs0mSgb2mN-KUcHW6z6LDQd12JBT0eE5E6RSJxwUZsxzT-kg==	d2wusnbjo8x1w7.cloudfront.net	https	410	0.582	-	TLSv1.3	TLS_AES_128_GCM_SHA256	Miss	HTTP/2.0	-	-	27007	0.582	Miss	-	-	-	-

Inside the SQL workbench:
2. Create a Flint based cloudfront Table

CREATE TABLE cloudfront (
    log_date STRING,
    log_time DATE,
    x_edge_location STRING,
    sc_bytes BIGINT,
    c_ip STRING,
    cs_method STRING,
    cs_host STRING,
    cs_uri_stem STRING,
    sc_status INT,
    cs_referer STRING,
    cs_user_agent STRING,
    cs_uri_query STRING,
    cs_cookie STRING,
    x_edge_result_type STRING,
    x_edge_request_id STRING,
    x_host_header STRING,
    cs_protocol STRING,
    cs_bytes BIGINT,
    time_taken float,
    x_forwarded_for STRING,
    ssl_protocol STRING,
    ssl_cipher STRING,
    x_edge_response_result_type STRING,
    cs_protocol_version STRING,
    fle_status STRING,
    fle_encrypted_fields STRING,
    c_port INT,
    time_to_first_byte float,
    x_edge_detailed_result_type STRING,
    sc_content_type STRING,
    sc_content_len STRING,
    sc_range_start STRING,
    sc_range_end STRING
)
USING csv
LOCATION 's3://bucket/cloudfront
OPTIONS (
  sep='\t'
);
  1. Query the data
select * from cloudfront limit 5;
  1. See error
{
    "data": {
        "ok": false,
        "resp": "Service Unavailable",
        "body": "{\n  \"status\": 503,\n  \"error\": {\n    \"type\": \"JSONException\",\n    \"reason\": \"There was internal problem at backend\",\n    \"details\": \"JSONObject[\\\"log_time\\\"] not found.\"\n  }\n}"
    }
}

What is the expected behavior?
A clear and concise description of what you expected to happen.

Do you have any screenshots?
Screenshot 2023-11-01 at 2 40 33 PM

Do you have any additional context?

OpenSearch query_execution_request_mys3 job metadata info:

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 6.137487,
    "hits": [
      {
        "_index": ".query_execution_request_mys3",
        "_id": "qidMEY1SDlzQk0ySG15czM=",
        "_score": 6.137487,
        "_source": {
          "queryId": "MEY1SDlzQk0ySG15czM=",
          "type": "jobmeta",
          "jobId": "00fedvohusqom00q",
          "applicationId": "00fd777k3k3ls20p",
          "isDropIndexQuery": false,
          "resultIndex": null,
          "sessionId": "cDlRM2F6OWVtMW15czM="
        }
      },
      {
        "_index": ".query_execution_request_mys3",
        "_id": "MEY1SDlzQk0ySG15czM=",
        "_score": 6.137487,
        "_source": {
          "version": "1.0",
          "type": "statement",
          "state": "success",
          "statementId": "MEY1SDlzQk0ySG15czM=",
          "sessionId": "cDlRM2F6OWVtMW15czM=",
          "applicationId": "00fd777k3k3ls20p",
          "jobId": "00fedvohusqom00q",
          "lang": "sql",
          "dataSourceName": "mys3",
          "query": "select * from mys3.default.cloudfront",
          "queryId": "MEY1SDlzQk0ySG15czM=",
          "submitTime": 1698873977721,
          "error": "unknown"
        }
      }
    ]
  }
}
@YANG-DB YANG-DB added bug Something isn't working untriaged and removed untriaged labels Nov 1, 2023
@YANG-DB YANG-DB self-assigned this Nov 2, 2023
@penghuo penghuo added the Flint label Nov 29, 2023
@penghuo
Copy link
Collaborator

penghuo commented Feb 2, 2024

Fix in 2.12. Verified.

@penghuo penghuo closed this as completed Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Flint
Projects
None yet
Development

No branches or pull requests

2 participants