Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cidrmatch ppl command add logical tests and docs #865

Merged

Conversation

YANG-DB
Copy link
Member

@YANG-DB YANG-DB commented Nov 1, 2024

Description

In cidrmatch ppl command the logical plan tests are missing

Check List

  • Updated documentation (docs/ppl-lang/README.md)
  • Implemented unit tests
  • Implemented tests for combination with other commands
  • New added source code should include a copyright header
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[See additional command details](functions/ppl-ip.md)

- `source = table | where cidrmatch(ip, '192.169.1.0/24')`
- `source = table | where isV6 = false and isValid = true and cidrmatch(ipAddress, '192.168.1.0/24'`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

) is missing

Signed-off-by: YANGDB <[email protected]>
@YANG-DB YANG-DB merged commit b183b4a into opensearch-project:main Nov 4, 2024
4 checks passed
YANG-DB added a commit that referenced this pull request Nov 5, 2024
* update antlr grammar for (future) P1 command syntax

Signed-off-by: YANGDB <[email protected]>

* add trendline command

Signed-off-by: YANGDB <[email protected]>

* add expand command

Signed-off-by: YANGDB <[email protected]>

* add geoip command

Signed-off-by: YANGDB <[email protected]>

* PPl `flatten` command (#784)

* The flatten command implemented

Signed-off-by: Lukasz Soszynski <[email protected]>

* The flatten command integration tests were extended with additional checks for logical plans.

Signed-off-by: Lukasz Soszynski <[email protected]>

* flatten, added more tests related to plan translation and integration tests

Signed-off-by: Lukasz Soszynski <[email protected]>

* Flatten command added to command names list.

Signed-off-by: Lukasz Soszynski <[email protected]>

---------

Signed-off-by: Lukasz Soszynski <[email protected]>

* Extract source table names from mv query (#854)

* add sourceTables to MV index metadata properties

Signed-off-by: Sean Kao <[email protected]>

* parse source tables from mv query

Signed-off-by: Sean Kao <[email protected]>

* test cases for parse source tables from mv query

Signed-off-by: Sean Kao <[email protected]>

* use constant for metadata cache version

Signed-off-by: Sean Kao <[email protected]>

* write source tables to metadata cache

Signed-off-by: Sean Kao <[email protected]>

* address comment

Signed-off-by: Sean Kao <[email protected]>

* generate source tables for old mv without new prop

Signed-off-by: Sean Kao <[email protected]>

* syntax fix

Signed-off-by: Sean Kao <[email protected]>

---------

Signed-off-by: Sean Kao <[email protected]>

* Fallback to internal scheduler when index creation failed (#850)

* Fallback to internal scheduler when index creation failed

Signed-off-by: Louis Chu <[email protected]>

* Fix IT

Signed-off-by: Louis Chu <[email protected]>

* Fix IOException

Signed-off-by: Louis Chu <[email protected]>

---------

Signed-off-by: Louis Chu <[email protected]>

* New trendline ppl command (SMA only) (#833)

* WIP trendline command

Signed-off-by: Kacper Trochimiak <[email protected]>

* wip

Signed-off-by: Kacper Trochimiak <[email protected]>

* trendline supports sorting

Signed-off-by: Kacper Trochimiak <[email protected]>

* run scalafmtAll

Signed-off-by: Kacper Trochimiak <[email protected]>

* return null when there are too few data points

Signed-off-by: Kacper Trochimiak <[email protected]>

* sbt scalafmtAll

Signed-off-by: Kacper Trochimiak <[email protected]>

* Remove WMA references

Signed-off-by: Hendrik Saly <[email protected]>

* trendline - sortByField as Optional<Field>

Signed-off-by: Kacper Trochimiak <[email protected]>

* introduce TrendlineStrategy

Signed-off-by: Kacper Trochimiak <[email protected]>

* keywordsCanBeId -> replace SMA with trendlineType

Signed-off-by: Kacper Trochimiak <[email protected]>

* handle trendline alias as qualifiedName instead of fieldExpression

Signed-off-by: Kacper Trochimiak <[email protected]>

* Add docs

Signed-off-by: Hendrik Saly <[email protected]>

* Make alias optional

Signed-off-by: Hendrik Saly <[email protected]>

* Adapt tests for optional alias

Signed-off-by: Hendrik Saly <[email protected]>

* Adden logical plan unittests

Signed-off-by: Hendrik Saly <[email protected]>

* Add missing license headers

Signed-off-by: Hendrik Saly <[email protected]>

* Fix docs

Signed-off-by: Hendrik Saly <[email protected]>

* numberOfDataPoints must be 1 or greater

Signed-off-by: Hendrik Saly <[email protected]>

* Rename TrendlineStrategy to  TrendlineCatalystUtils

Signed-off-by: Hendrik Saly <[email protected]>

* Validate TrendlineType early and pass around enum type

Signed-off-by: Hendrik Saly <[email protected]>

* Add trendline chaining test

Signed-off-by: Hendrik Saly <[email protected]>

* Fix compile errors

Signed-off-by: Hendrik Saly <[email protected]>

* Fix imports

Signed-off-by: Hendrik Saly <[email protected]>

* Fix imports

Signed-off-by: Hendrik Saly <[email protected]>

---------

Signed-off-by: Kacper Trochimiak <[email protected]>
Signed-off-by: Hendrik Saly <[email protected]>
Co-authored-by: Kacper Trochimiak <[email protected]>

* update iplocation antlr

Signed-off-by: YANGDB <[email protected]>

* update scala fmt style

Signed-off-by: YANGDB <[email protected]>

* `cidrmatch` ppl command add logical tests and docs (#865)

* update logical tests and docs

Signed-off-by: YANGDB <[email protected]>

* update scala fmt style

Signed-off-by: YANGDB <[email protected]>

* fix type error

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* Support Lambda and add related array functions (#864)

* json function enhancement

Signed-off-by: Heng Qian <[email protected]>

* Add JavaToScalaTransformer

Signed-off-by: Heng Qian <[email protected]>

* Apply scalafmtAll

Signed-off-by: Heng Qian <[email protected]>

* Address comments

Signed-off-by: Heng Qian <[email protected]>

* Add IT and change to use the same function name as spark

Signed-off-by: Heng Qian <[email protected]>

* Address comments

Signed-off-by: Heng Qian <[email protected]>

* Add document and separate lambda functions from json functions

Signed-off-by: Heng Qian <[email protected]>

* Add lambda functions transform and reduce

Signed-off-by: Heng Qian <[email protected]>

* polish lambda function document

Signed-off-by: Heng Qian <[email protected]>

* polish lambda function document

Signed-off-by: Heng Qian <[email protected]>

* Minor fix

Signed-off-by: Heng Qian <[email protected]>

* Minor change to polish the documents

Signed-off-by: Heng Qian <[email protected]>

---------

Signed-off-by: Heng Qian <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>
Signed-off-by: Lukasz Soszynski <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Louis Chu <[email protected]>
Signed-off-by: Kacper Trochimiak <[email protected]>
Signed-off-by: Hendrik Saly <[email protected]>
Signed-off-by: Heng Qian <[email protected]>
Co-authored-by: lukasz-soszynski-eliatra <[email protected]>
Co-authored-by: Sean Kao <[email protected]>
Co-authored-by: Louis Chu <[email protected]>
Co-authored-by: Hendrik Saly <[email protected]>
Co-authored-by: Kacper Trochimiak <[email protected]>
Co-authored-by: qianheng <[email protected]>
kenrickyap pushed a commit to Bit-Quill/opensearch-spark that referenced this pull request Dec 11, 2024
…t#865)

* update logical tests and docs

Signed-off-by: YANGDB <[email protected]>

* update scala fmt style

Signed-off-by: YANGDB <[email protected]>

* fix type error

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.6 Lang:PPL Pipe Processing Language support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants