Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PPL command implementation for appendCol #990

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
6fe52e8
Update grammar def
andy-k-improving Dec 6, 2024
0b3b50c
Skeleton for Append fields
andy-k-improving Dec 6, 2024
50f4bd5
Visitor skeleton
andy-k-improving Dec 7, 2024
c74ac1a
Update import
andy-k-improving Dec 9, 2024
f23c2db
Update import
andy-k-improving Dec 9, 2024
893146a
Update osrt
andy-k-improving Dec 10, 2024
a9f10f0
Changes
andy-k-improving Dec 11, 2024
ed71d58
Consolidate String constant
andy-k-improving Dec 11, 2024
3d7b8b1
Update projection clause
andy-k-improving Dec 11, 2024
77fddf1
Remove dep on parent method
andy-k-improving Dec 11, 2024
0e7e65a
Consolidate relation inject logic
andy-k-improving Dec 11, 2024
d2aa146
Move constant
andy-k-improving Dec 11, 2024
477c4fc
Move out constant from lambda
andy-k-improving Dec 11, 2024
662a57c
Consolidate method
andy-k-improving Dec 11, 2024
365cc12
Update logic
andy-k-improving Dec 11, 2024
351ea88
Test 1 2
andy-k-improving Dec 12, 2024
16406a0
Test-cases 3 and 4
andy-k-improving Dec 13, 2024
13f4cb9
Update code format
andy-k-improving Dec 13, 2024
d34abf1
Update code style
andy-k-improving Dec 13, 2024
822ebd5
Update scala syntax
andy-k-improving Dec 13, 2024
25857f2
Override option
andy-k-improving Dec 13, 2024
cba664d
Update override option
andy-k-improving Dec 13, 2024
d790d20
Enable override option
andy-k-improving Dec 13, 2024
816f6d6
Override impl
andy-k-improving Dec 14, 2024
f6e03dd
Minimise cmd permission
andy-k-improving Dec 14, 2024
83d621e
Refactor util class
andy-k-improving Dec 14, 2024
b6d5ca0
Java doc
andy-k-improving Dec 14, 2024
c56e4e5
Integ test 1 2
andy-k-improving Dec 14, 2024
da242f5
Test cases 3 4
andy-k-improving Dec 14, 2024
9194ca6
Test code comments
andy-k-improving Dec 16, 2024
7e17ab9
Code tidy
andy-k-improving Dec 16, 2024
a3623e8
Code refactor
andy-k-improving Dec 16, 2024
04af9ad
ScalaFmt
andy-k-improving Dec 16, 2024
a588236
Remove sout
andy-k-improving Dec 16, 2024
8cf6f1a
Update doc
andy-k-improving Dec 16, 2024
6269b5b
Override option test case
andy-k-improving Dec 16, 2024
2847e5a
Code style
andy-k-improving Dec 16, 2024
d03f8bb
Code comment
andy-k-improving Dec 17, 2024
9db7927
Deprecate visit child (1)
andy-k-improving Dec 17, 2024
e73e15e
Minimise code diff
andy-k-improving Dec 17, 2024
4e6794a
Update override logic
andy-k-improving Dec 19, 2024
5970293
Update test-cases
andy-k-improving Dec 19, 2024
b219097
Integ
andy-k-improving Dec 20, 2024
6b3aeec
Make append alias distinct
andy-k-improving Dec 20, 2024
f2d54e5
Update integ for distinct tables
andy-k-improving Dec 20, 2024
03b2ba8
Update limitation
andy-k-improving Dec 20, 2024
7323cc6
Update code style
andy-k-improving Dec 20, 2024
33e6345
Update docs/ppl-lang/ppl-appendcol-command.md
andy-k-improving Dec 20, 2024
f378e9f
Update docs/ppl-lang/ppl-appendcol-command.md
andy-k-improving Dec 20, 2024
375415e
Update docs/ppl-lang/ppl-appendcol-command.md
andy-k-improving Dec 20, 2024
d13d53d
Update readme
andy-k-improving Dec 20, 2024
c44d1b6
Mark var as final
andy-k-improving Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/ppl-lang/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ For additional examples see the next [documentation](PPL-Example-Commands.md).

- [`expand commands`](ppl-expand-command.md)

- [`appendcol commands`](ppl-appendcol-command.md)
andy-k-improving marked this conversation as resolved.
Show resolved Hide resolved

* **Functions**

- [`Expressions`](functions/ppl-expressions.md)
Expand Down
83 changes: 83 additions & 0 deletions docs/ppl-lang/ppl-appendcol-command.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
## PPL `appendcol` command

### Description
Using `appendcol` command to append the result of a sub-search and attach it alongside with the input search results (The main search).

### Syntax - APPENDCOL
`APPENDCOL <override=?> [sub-search]...`

* <override=?>: optional boolean field to specify should result from main-result be overwritten in the case of column name conflict.
* sub-search: Executes PPL commands as a secondary search. The sub-search uses the same data specified in the source clause of the main search results as its input.


#### Example 1: To append the result of `stats ave(age) as AVG_AGE` into existing search result
andy-k-improving marked this conversation as resolved.
Show resolved Hide resolved

The example append the result of sub-search `stats ave(age) as AVG_AGE` alongside with the main-search.
andy-k-improving marked this conversation as resolved.
Show resolved Hide resolved

PPL query:

os> source=employees | FIELDS name, dept, age | APPENDCOL [ stats avg(age) as AVG_AGE ];
fetched rows / total rows = 9/9
+------+-------------+-----+------------------+
| name | dept | age | AVG_AGE |
+------+-------------+-----+------------------+
| Lisa | Sales------ | 35 | 31.2222222222222|
| Fred | Engineering | 28 | NULL |
| Paul | Engineering | 23 | NULL |
| Evan | Sales------ | 38 | NULL |
| Chloe| Engineering | 25 | NULL |
| Tom | Engineering | 33 | NULL |
| Alex | Sales | 33 | NULL |
| Jane | Marketing | 28 | NULL |
| Jeff | Marketing | 38 | NULL |
+------+-------------+-----+------------------+
andy-k-improving marked this conversation as resolved.
Show resolved Hide resolved


#### Example 2: Append multiple sub-search result

The example demonstrate multiple APPENCOL commands can be chained to provide one comprehensive view for user.

PPL query:

os> source=employees | FIELDS name, dept, age | APPENDCOL [ stats avg(age) as AVG_AGE ] | APPENDCOL [ stats max(age) as MAX_AGE ];
fetched rows / total rows = 9/9
+------+-------------+-----+------------------+---------+
| name | dept | age | AVG_AGE | MAX_AGE |
+------+-------------+-----+------------------+---------+
| Lisa | Sales------ | 35 | 31.22222222222222| 38 |
| Fred | Engineering | 28 | NULL | NULL |
| Paul | Engineering | 23 | NULL | NULL |
| Evan | Sales------ | 38 | NULL | NULL |
| Chloe| Engineering | 25 | NULL | NULL |
| Tom | Engineering | 33 | NULL | NULL |
| Alex | Sales | 33 | NULL | NULL |
| Jane | Marketing | 28 | NULL | NULL |
| Jeff | Marketing | 38 | NULL | NULL |
+------+-------------+-----+------------------+---------+

#### Example 3: Over main-search in the case of column name conflict

The example demonstrate the usage of `OVERRIDE` option to overwrite the `age` column from the main-search,
when the option is set to true and column with same name `age` present on sub-search.

PPL query:

os> source=employees | FIELDS name, dept, age | APPENDCOL OVERRIDE=true [ stats avg(age) as age ];
fetched rows / total rows = 9/9
+------+-------------+------------------+
| name | dept | age |
+------+-------------+------------------+
| Lisa | Sales------ | 31.22222222222222|
| Fred | Engineering | NULL |
| Paul | Engineering | NULL |
| Evan | Sales------ | NULL |
| Chloe| Engineering | NULL |
| Tom | Engineering | NULL |
| Alex | Sales | NULL |
| Jane | Marketing | NULL |
| Jeff | Marketing | NULL |
+------+-------------+------------------+

### Limitation:
When override is set to true, only `FIELDS` and `STATS` commands are allowed as the final clause in a sub-search.
Otherwise, an IllegalStateException with the message `Not Supported operation: APPENDCOL should specify the output fields` will be thrown.
Loading
Loading