Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-45039][SQL] Include full identifier in Storage tab #42759

Closed

Conversation

planga82
Copy link
Contributor

What changes were proposed in this pull request?

If you have 2 databases with tables with the same name like: db1.table, db2.table

scala> spark.sql("use db1")
res8: org.apache.spark.sql.DataFrame = []

scala> spark.sql("cache table table")
res9: org.apache.spark.sql.DataFrame = []                                       

scala> spark.sql("use db2")
res10: org.apache.spark.sql.DataFrame = []

scala> spark.sql("cache table table")
res11: org.apache.spark.sql.DataFrame = []

You cannot differentiate it in the Storage tab

image

What is proposed is to add the full identifier

image

With this PR it’s also modified the node name in the sql tab

image

Why are the changes needed?

Facilitates the interpretation of the Storage tab

Does this PR introduce any user-facing change?

Yes, in the UI

How was this patch tested?

Unit testing and manual tests

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the SQL label Aug 31, 2023
@planga82
Copy link
Contributor Author

planga82 commented Sep 1, 2023

CC @cloud-fan

@planga82
Copy link
Contributor Author

planga82 commented Sep 2, 2023

The error in the test does not appear to be related to change

@planga82 planga82 changed the title [SPARK-45039][UI] Include full identifier in Storage tab [SPARK-45039][SQL] Include full identifier in Storage tab Sep 2, 2023
spark.sessionState.sqlParser.parseMultipartIdentifier(table) match {
case Seq(_) if spark.catalog.getTable(table).isTemporary => table

case CatalogAndIdentifier(cat, ident) => s"${cat.name()}.${ident.toString}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should ask the caller side to pass in a fully qualified table name, instead of relying on the current catalog/namespace here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, I will make the changes, thank you!

@planga82
Copy link
Contributor Author

planga82 commented Sep 9, 2023

@cloud-fan I made the changes, what do you think? thanks!

Copy link

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

@github-actions github-actions bot added the Stale label Dec 19, 2023
@github-actions github-actions bot closed this Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants