Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] PPL describe couldn't handle fully qualified name #612

Closed
seankao-az opened this issue Aug 29, 2024 · 0 comments
Closed

[BUG] PPL describe couldn't handle fully qualified name #612

seankao-az opened this issue Aug 29, 2024 · 0 comments
Assignees
Labels
bug Something isn't working Lang:PPL Pipe Processing Language support

Comments

@seankao-az
Copy link
Collaborator

What is the bug?
describe myglue_test.default.http_logs

{
  "status": "FAILED",
  "error": "{\"Message\":\"Fail to run query. Cause: Invalid table name: myglue_test.default.http_logs Syntax: [ database_name. ] table_name\"}"
}

As a comparison, the following works:
describe default.http_logs

{
    "status": "SUCCESS",
    "schema": [
        {
            "name": "col_name",
            "type": "string"
        },
        {
            "name": "data_type",
            "type": "string"
        },
        {
            "name": "comment",
            "type": "string"
        }
    ],
    "datarows": [
        [
            "@timestamp",
            "timestamp",
            null
        ],
        [
            "clientip",
            "string",
            null
        ],
        [
            "request",
            "string",
            null
        ],
        [
            "status",
            "int",
            null
        ],
        [
            "size",
            "int",
            null
        ],
        [
            "year",
            "int",
            null
        ],
        [
            "month",
            "int",
            null
        ],
        [
            "day",
            "int",
            null
        ],
        [
            "# Partition Information",
            "",
            ""
        ],
        [
            "# col_name",
            "data_type",
            "comment"
        ],
        [
            "year",
            "int",
            null
        ],
        [
            "month",
            "int",
            null
        ],
        [
            "day",
            "int",
            null
        ],
        [
            "",
            "",
            ""
        ],
        [
            "# Detailed Table Information",
            "",
            ""
        ],
        [
            "Catalog",
            "spark_catalog",
            ""
        ],
        [
            "Database",
            "default",
            ""
        ],
        [
            "Table",
            "http_logs",
            ""
        ],
        [
            "Owner",
            "hadoop",
            ""
        ],
        [
            "Created Time",
            "Fri Aug 23 21:47:33 GMT 2024",
            ""
        ],
        [
            "Last Access",
            "UNKNOWN",
            ""
        ],
        [
            "Created By",
            "Spark 3.5.1-amzn-0",
            ""
        ],
        [
            "Type",
            "EXTERNAL",
            ""
        ],
        [
            "Provider",
            "json",
            ""
        ],
        [
            "Table Properties",
            "[isRegisteredWithLakeFormation=false]",
            ""
        ],
        [
            "Location",
            "s3://...",
            ""
        ],
        [
            "Serde Library",
            "org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe",
            ""
        ],
        [
            "InputFormat",
            "org.apache.hadoop.mapred.SequenceFileInputFormat",
            ""
        ],
        [
            "OutputFormat",
            "org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat",
            ""
        ],
        [
            "Storage Properties",
            "[compression=bzip2]",
            ""
        ],
        [
            "Partition Provider",
            "Catalog",
            ""
        ]
    ],
    "total": 31,
    "size": 31
}

What is your host/environment?

  • AOS: 2.15
  • EMR-S: emr-7.2.0
  • Flint: 0.5
@seankao-az seankao-az added bug Something isn't working untriaged labels Aug 29, 2024
@seankao-az seankao-az added the Lang:PPL Pipe Processing Language support label Aug 29, 2024
@YANG-DB YANG-DB self-assigned this Aug 30, 2024
@YANG-DB YANG-DB moved this to Todo in PPL Commands Aug 30, 2024
@YANG-DB YANG-DB removed the untriaged label Aug 30, 2024
@YANG-DB YANG-DB moved this from Todo to In Progress in PPL Commands Sep 12, 2024
@YANG-DB YANG-DB moved this from In Progress to InReview in PPL Commands Sep 12, 2024
YANG-DB added a commit to YANG-DB/opensearch-spark that referenced this issue Sep 13, 2024
YANG-DB added a commit that referenced this issue Sep 13, 2024
* fix describe ppl command to be aware of 3 parts FQN table that includes schema.catalog.table

Signed-off-by: YANGDB <[email protected]>

* fix describe ppl command to be aware of 3 parts FQN table that includes schema.catalog.table

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>
opensearch-trigger-bot bot pushed a commit that referenced this issue Sep 13, 2024
* fix describe ppl command to be aware of 3 parts FQN table that includes schema.catalog.table

Signed-off-by: YANGDB <[email protected]>

* fix describe ppl command to be aware of 3 parts FQN table that includes schema.catalog.table

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>
(cherry picked from commit 0be5697)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@YANG-DB YANG-DB moved this from InReview to Done in PPL Commands Sep 16, 2024
@YANG-DB YANG-DB closed this as completed by moving to Done in PPL Commands Sep 16, 2024
@YANG-DB YANG-DB removed the status in PPL Commands Sep 16, 2024
@YANG-DB YANG-DB moved this to Done in PPL Commands Sep 16, 2024
YANG-DB pushed a commit that referenced this issue Sep 16, 2024
* fix describe ppl command to be aware of 3 parts FQN table that includes schema.catalog.table



* fix describe ppl command to be aware of 3 parts FQN table that includes schema.catalog.table



---------


(cherry picked from commit 0be5697)

Signed-off-by: YANGDB <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Lang:PPL Pipe Processing Language support
Projects
Status: Done
Development

No branches or pull requests

2 participants