Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Issues triggered from investigating 1156 #1434

Open
1 of 5 tasks
amahussein opened this issue Nov 25, 2024 · 0 comments
Open
1 of 5 tasks

[BUG] Issues triggered from investigating 1156 #1434

amahussein opened this issue Nov 25, 2024 · 0 comments
Labels
? - Needs Triage bug Something isn't working core_tools Scope the core module (scala)

Comments

@amahussein
Copy link
Collaborator

amahussein commented Nov 25, 2024

Describe the bug

While investigating on issue #1156, there are some issues that need a separate handling. I am listing them here to keep track of them:

  • nds_q86 eventlog is generated with an old Spark (3.0.2) core/src/test/resources/spark-events-qualification/nds_q86_test:
    • Back then , there was a bug in duplicating planInfos that were supposed to be reused. SqlPlanParser added a special handling for that case, but we should not do that any longer. Those execs were executed twice and we should honor that even if we know that this is a bug.
    • Action: put an eventlog generated from more recent Spark version.
  • Refactor the code to use the ToolsPlanGraph as a result of processing the planInfo; instead of the spark graph class type.
  • Fix the same problem for Profiler tools.
  • It is hard to read the CSV file unsupported_ops.csv because the files are not sorted. It will be nice to sort the files.
  • [FEA] Make Execs report in Qualification more readable by sorting #1435
@amahussein amahussein added ? - Needs Triage bug Something isn't working core_tools Scope the core module (scala) labels Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
? - Needs Triage bug Something isn't working core_tools Scope the core module (scala)
Projects
None yet
Development

No branches or pull requests

1 participant