[BUG] REPL query failure results in unclear error message #445
Comments
|
After investigation, it appears that the root cause of the issue lies in the following line of code. The code is using the original exception 24/07/23 15:25:30 INFO FlintREPL: command complete: FlintStatement(
state=failed, query= CREATE TABLE my_glue1.default.flint_sql_test ( name STRING, age INT )
USING CSV OPTIONS ( header 'false', delimiter ' ' ) LOCATION 's3://test-123'; ,
statementId=8ba68f33-d83b-428a-82b9-60def7a29cd2, queryId=99, submitTime=1721773522390,
error=Some({"Message":"Fail to run query. Cause: No FileSystem for scheme \"s3\""}))
Code changes: diff --git a/spark-sql-application/src/main/scala/org/apache/spark/sql/FlintJobExecutor.scala
b/spark-sql-application/src/main/scala/org/apache/spark/sql/FlintJobExecutor.scala
index f38a27ef..c7dccf64 100644
--- a/spark-sql-application/src/main/scala/org/apache/spark/sql/FlintJobExecutor.scala
+++ b/spark-sql-application/src/main/scala/org/apache/spark/sql/FlintJobExecutor.scala
@@ -500,7 +500,7 @@ trait FlintJobExecutor {
val e = new SecurityException(ExceptionMessages.GlueAccessDeniedMessage)
handleQueryException(e, ExceptionMessages.QueryRunErrorPrefix)
} else {
- handleQueryException(ex, ExceptionMessages.QueryRunErrorPrefix)
+ handleQueryException(r, ExceptionMessages.QueryRunErrorPrefix)
}
}
}
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What is the bug?
When running queries in the REPL, the error message returned is generic and lacks specifics about the root cause. For instance, a failure in executing a query results in the following error message:
This message does not provide sufficient information to diagnose the actual issue, making troubleshooting difficult.
How can one reproduce the bug?
The bug can be reproduced using the following steps in the REPL IT environment:
class FlintREPLITSuite extends SparkFunSuite with OpenSearchSuite with JobTest { ... test("error") { try { createSession(jobRunId, "") threadLocalFuture.set(startREPL()) val createStatement = s""" | CREATE TABLE $testTable | ( | name STRING, | age INT | ) | USING CSV | OPTIONS ( | header 'false', | delimiter '\\t' | ) | LOCATION 's3://test-123'; |""".stripMargin submitQuery(s"${makeJsonCompliant(createStatement)}", "99") val selectQueryValidation: REPLResult => Boolean = result => { // noinspection ScalaStyle println(result) true } pollForResultAndAssert(selectQueryValidation, "99") } catch { case e: Exception => logError("Unexpected exception", e) assert(false, "Unexpected exception") } finally { waitREPLStop(threadLocalFuture.get()) threadLocalFuture.remove() } 24/07/23 15:01:16 INFO FlintREPL: command complete: FlintStatement( state=failed, query= CREATE TABLE my_glue1.default.flint_sql_test ( name STRING, age INT ) USING CSV OPTIONS ( header 'false', delimiter ' ' ) LOCATION 's3://test-123'; , statementId=e0cafbd7-6e28-4f6e-8076-78026e28d6c6, queryId=99, submitTime=1721772068015, error=Some({"Message":"Fail to run query. Cause: Exception thrown in awaitResult: "}))What is the expected behavior?
The system should provide a detailed error message that includes the root cause of the failure. Instead of a generic message, users should receive information that helps in diagnosing the underlying problem, such as specific exceptions or stack traces.
Do you have any additional context?
Spark log:
The text was updated successfully, but these errors were encountered: