Translate PPL join Command

[LantaoJin]

Description

Syntax

SEARCH source=<left-table>
| <other piped command>
| [joinType] JOIN left = <leftAlias> [,] right = <rightAlias> ON joinCriteria <right-table>
| <other piped command>

joinType: [INNER] | LEFT [OUTER] | RIGHT [OUTER] | FULL [OUTER] | CROSS | [LEFT] SEMI | [LEFT] ANTI 

Example of migration from SQL query (TPC-H Q13):

SELECT c_count, COUNT(*) AS custdist
FROM
  ( SELECT c_custkey, COUNT(o_orderkey) c_count
    FROM customer LEFT OUTER JOIN orders ON c_custkey = o_custkey
        AND o_comment NOT LIKE '%unusual%packages%'
    GROUP BY c_custkey
  ) AS c_orders
GROUP BY c_count
ORDER BY custdist DESC, c_count DESC;

Rewritten by PPL Join query:

SEARCH source=customer
| FIELDS c_custkey
| LEFT OUTER JOIN left = c right = o
    ON c.c_custkey = o.o_custkey AND o_comment NOT LIKE '%unusual%packages%'
    orders
| STATS count(o_orderkey) AS c_count BY c.c_custkey
| STATS count(1) AS custdist BY c_count
| SORT - custdist, - c_count

- Limitation: sub-searches is unsupported in join right side because sub-searches is unsupported in PPL

If sub-searches is supported in future, above ppl query could be rewritten as:

SEARCH source=customer
| FIELDS c_custkey
| LEFT OUTER JOIN left = c right = o ON c.c_custkey = o.o_custkey
   [
      SEARCH source=orders
      | WHERE o_comment NOT LIKE '%unusual%packages%'
      | FIELDS o_orderkey, o_custkey
   ]
| STATS count(o_orderkey) AS c_count BY c.c_custkey
| STATS count(*) AS custdist BY c_count
| SORT - custdist, - c_count

Issues Resolved

Resolves #619

Check List

• Updated documentation (ppl-spark-integration/README.md)
• Implemented unit tests
• Implemented tests for combination with other commands
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[penghuo]

nit, if it is required field, why not using left and right, hint seems optional field?

[LantaoJin]

You are right. hint. seems optional item. Using left and right sounds better. But it bring me that actually the alias also could be optional if users prefer to use pattern $table.$field in join condition and no ambiguous naming related. For example:

SEARCH source=otel-v1-apm-span-000001
| <other PPL commands>
| LEFT JOIN
    ON name = serviceName AND traceGroupName = 'client_cancel_order'
    otel-v1-apm-service-map
| <other PPL commands>

Or ON otel-v1-apm-span-000001.name = otel-v1-apm-service-map.serviceName AND otel-v1-apm-service-map.traceGroupName = 'client_cancel_order'

Which solution do you prefer to? @penghuo

1. Keep the alias name hint.left and hint.right and set it to optional.
2. Remove the hint. prefix and keep it required.

[YANG-DB]

IMO I would keep the existing first option

[LantaoJin]

@YANG-DB I found that the side aliases must be added as required because we add a SubqueryAlias node for each side of the join.

[LantaoJin]

Doc updated. The hint. is removed in latest commit.

[LantaoJin]

@YANG-DB @penghuo can this PR be merged today? I am going to submit the lookup PR which is based on this.

[LantaoJin]

@YANG-DB This PR requires a 0.6 label since the Lookup has had.