PPL Lookup #407
PPL Lookup #407
Conversation
Signed-off-by: Kacper Trochimiak <[email protected]>
Signed-off-by: Kacper Trochimiak <[email protected]>
|
@salyh thanks for your contribution |
|
@YANG-DB any comments on the proposal so far? |
|
@rupal-bq @anasalkouz any comments on the proposal so far? |
| while(!root.getChild().isEmpty()) { | ||
| root = root.getChild().get(0); | ||
| } |
There was a problem hiding this comment.
This loop finds the root command as lookup's source table, but we may have multiple lookup commands, such as source=t1 | lookup t2 | ... | lookup t3
There was a problem hiding this comment.
The source=<table> is used to specify the "source" table name.
It's not required to specify (repeat) the name of the "source" table within the lookup command, so I guess finding the root Relation node is OK. Am I missing something?
|
|
||
| node.getChild().get(0).accept(this, context); | ||
|
|
||
| //TODO: not sure how to implement appendonly |
There was a problem hiding this comment.
To implement appendonly, the output Project (parent of Join) should be
Project [other fields in source, CASE WHEN isnull('source.copyField) THEN coalesce('source.copyField, 'lookup.copyField) AS copyField ELSE 'lookup.copyField END AS copyField]
+- Join LeftOuter
In CASE WHEN expression1 ELSE expression2, the expression1 is for appendonly (not performing when the source field already exist), the else is for overwriting existing field.
There was a problem hiding this comment.
Doesn't it mean that we need to know list of fields available in the "source" table, in order to build project consisting of two lists:
a list of fields which we want to overwrite with values from the lookup table and a list of fields which should remain untouched?
I'm not sure whether it is possible to retrieve such list of available fields.
|
Close this as issue resolved by #686 |
Description
Implement PPL Lookup
Issues Resolved
opensearch-project/sql#2651
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.