You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, when the PPL parser fails to parse a query, the FlintSparkPPLParser class swallows the PPL parser error and falls back to the Flint SQL parser. This behavior can lead to confusing error messages for users, as the actual PPL parser error is replaced by a different error from the SQL parser.
Modify the FlintSparkPPLParser class to accept a SparkSession object in its constructor:
class FlintSparkPPLParser(sparkParser: ParserInterface, sparkSession: SparkSession) extends ParserInterface { // ...}
Update the FlintPPLSparkExtensions class to pass the SparkSession object to the FlintSparkPPLParser constructor:
class FlintPPLSparkExtensions extends (SparkSessionExtensions => Unit) {
override def apply(extensions: SparkSessionExtensions): Unit = {
extensions.injectParser { (spark, parser) =>
new FlintSparkPPLParser(parser, spark)
}
}
}
Instead of swallowing the PPL parser errors, throw the exception directly if the language type is specified as PPL (from spark conf) in the parsePlan method of FlintSparkPPLParser.
What alternatives have you considered?
An alternative approach would be to pass the PPL exception along with the CatalystPlanContext and determine which exception to throw from downstream components. However, this approach might introduce additional complexity and make the code harder to maintain and understand.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem?
Currently, when the PPL parser fails to parse a query, the
FlintSparkPPLParser
class swallows the PPL parser error and falls back to the Flint SQL parser. This behavior can lead to confusing error messages for users, as the actual PPL parser error is replaced by a different error from the SQL parser.opensearch-spark/ppl-spark-integration/src/main/scala/org/opensearch/flint/spark/ppl/FlintSparkPPLParser.scala
Lines 54 to 68 in beac01a
What solution would you like?
FlintSparkPPLParser
class to accept aSparkSession
object in its constructor:Instead of swallowing the PPL parser errors, throw the exception directly if the language type is specified as PPL (from spark conf) in the parsePlan method of FlintSparkPPLParser.
What alternatives have you considered?
An alternative approach would be to pass the PPL exception along with the CatalystPlanContext and determine which exception to throw from downstream components. However, this approach might introduce additional complexity and make the code harder to maintain and understand.
The text was updated successfully, but these errors were encountered: