Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support parenthesized expression in filter #888

Merged
merged 4 commits into from
Nov 14, 2024

Conversation

LantaoJin
Copy link
Member

@LantaoJin LantaoJin commented Nov 11, 2024

Description

Parenthesized expression cannot work as expected in filter:
source = $testTable | where (country = 'Canada' or age > 60) and age < 25 | fields name, age, country failed in parser.

Related Issues

Resolves #887

Check List

  • Updated documentation (docs/ppl-lang/README.md)
  • Implemented unit tests
  • Implemented tests for combination with other commands
  • New added source code should include a copyright header
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@LantaoJin LantaoJin added Lang:PPL Pipe Processing Language support sanity-0.6 0.7 labels Nov 11, 2024
@LantaoJin LantaoJin marked this pull request as ready for review November 11, 2024 09:44
@@ -467,4 +467,33 @@ class FlintSparkPPLFiltersITSuite
val expectedPlan = Project(Seq(UnresolvedAttribute("state")), filter)
comparePlans(logicalPlan, expectedPlan, checkAnalysis = false)
}

test("test parenthesis in filter") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LantaoJin can u plz add a nested parenthesis test case ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in addition - can u plz add a correspondent logical unit test under the PPL*** relevant tests ?

Copy link
Member

@YANG-DB YANG-DB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LantaoJin Looks good - just maybe add an additional nested parentheses test case ?

@LantaoJin LantaoJin merged commit a80aa04 into opensearch-project:main Nov 14, 2024
4 checks passed
@opensearch-trigger-bot
Copy link

The backport to 0.6 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/opensearch-spark/backport-0.6 0.6
# Navigate to the new working tree
pushd ../.worktrees/opensearch-spark/backport-0.6
# Create a new branch
git switch --create backport/backport-888-to-0.6
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a80aa04b9bbadfe68a3ed0ff59c3edce5251aac6
# Push it to GitHub
git push --set-upstream origin backport/backport-888-to-0.6
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/opensearch-spark/backport-0.6

Then, create a pull request where the base branch is 0.6 and the compare/head branch is backport/backport-888-to-0.6.

LantaoJin added a commit to LantaoJin/opensearch-spark that referenced this pull request Nov 14, 2024
LantaoJin added a commit to LantaoJin/opensearch-spark that referenced this pull request Nov 14, 2024
LantaoJin added a commit that referenced this pull request Nov 14, 2024
(cherry picked from commit a80aa04)
Signed-off-by: Lantao Jin <[email protected]>
kenrickyap pushed a commit to Bit-Quill/opensearch-spark that referenced this pull request Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Parenthesized expression doesn't work in filter
2 participants