Unescape query from EMR spark submit parameter

[seankao-az]

Description

In a previous PR in sql, query is escaped in spark submit parameter. This PR unescapes the query on the receiving end, retrieving the original query. This affects only the query in SparkConf.

End to end test is performed as follows:
Query:

CREATE INDEX cv
ON mys3.default.http_logs (`status`, `day`, `size`, `request`)
WITH (
  auto_refresh = true,
  refresh_interval="5 Minutes",
  checkpoint_location = 's3://test',
  index_settings = "{\"number_of_shards\":1,\"number_of_replicas\":1}"
)

Spark submit parameter for the EMR job (query escaped and wrapped with quotes):

--conf spark.flint.job.query="CREATE INDEX cv ON mys3.default.http_logs (`status`, `day`, `size`, `request`) WITH (auto_refresh = true, refresh_interval=\"5 Minutes\", checkpoint_location = 's3://test',index_settings = \"{\\\"number_of_shards\\\":1,\\\"number_of_replicas\\\":1}\")"

Query received on spark before this PR fix (query still escaped and result in syntax error):

CREATE INDEX cv ON mys3.default.http_logs (`status`, `day`, `size`, `request`) WITH (auto_refresh = true, refresh_interval=\"5 Minutes\", checkpoint_location = 's3://test',index_settings = \"{\\\"number_of_shards\\\":1,\\\"number_of_replicas\\\":1}\")

Query received on spark after this PR fix (query unescaped):

CREATE INDEX cv ON mys3.default.http_logs (`status`, `day`, `size`, `request`) WITH (auto_refresh = true, refresh_interval="5 Minutes", checkpoint_location = 's3://test',index_settings = "{\"number_of_shards\":1,\"number_of_replicas\":1}")

Issues Resolved

• [BUG] Async Job query can longer than 10280 chars sql#2376

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[noCharger]

let's include this in 0.3 after testing

[dai-chen]

let's include this in 0.3 after testing

Because we've started working on 0.4 task, is this the last PR for 0.3? If not, I think we'd better cut 0.3 branch now?

[dai-chen]

This may require sensitive info anonymized like: https://github.com/opensearch-project/sql/blob/2649200e065dff48282dce438ceb0ee5ac39054e/sql/src/main/java/org/opensearch/sql/sql/antlr/AnonymizerListener.java#L34. Probably we can do this later since this is also helpful for analyzing query pattern.

[dai-chen]

@seankao-az @noCharger could you address this in other 0.3 PR if any? Probably remove it for now?

[noCharger]

let's include this in 0.3 after testing

Because we've started working on 0.4 task, is this the last PR for 0.3? If not, I think we'd better cut 0.3 branch now?

Can we include this #303? I will add one more commit later today.

[dai-chen]

let's include this in 0.3 after testing

Because we've started working on 0.4 task, is this the last PR for 0.3? If not, I think we'd better cut 0.3 branch now?

Can we include this #303? I will add one more commit later today.

Sure. Will block other 0.4 PRs for now.

[noCharger]

let's include this in 0.3 after testing

Because we've started working on 0.4 task, is this the last PR for 0.3? If not, I think we'd better cut 0.3 branch now?

Can we include this #303? I will add one more commit later today.

Sure. Will block other 0.4 PRs for now.

We can cut the branch and backport too.

[noCharger]

@dai-chen cut 0.3 branch to unblock 0.4.