Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimal expressions API for vortex #308

Closed
wants to merge 6 commits into from
Closed

Minimal expressions API for vortex #308

wants to merge 6 commits into from

Conversation

jdcasale
Copy link
Contributor

@jdcasale jdcasale commented May 10, 2024

Minimal set of expressions and operators for defining expressions over vortex arrays. Takes inspiration from datafusion but is restricted to filter predicates and rewritten to use the vortex type system.

I tried to restrict this to a manageable set of expressions that we will definitely need as to avoid bloat; the expectation here is that this may grow over time.

Subsequent changes will introduce:

  • Aggregation functions that we can push down to vortex metadata
  • A 'canonicalizer' mechanism to flatten expression trees
  • A pushdown API for vortex arrays
  • Array pushdown implementations for the various expressions

@jdcasale jdcasale force-pushed the jc/expr branch 3 times, most recently from 159cc45 to c374589 Compare May 14, 2024 14:53
@jdcasale jdcasale marked this pull request as ready for review May 14, 2024 16:31
vortex-dtype/src/ptype.rs Outdated Show resolved Hide resolved
vortex-expr/src/expression_fns.rs Outdated Show resolved Hide resolved
vortex-expr/README.md Outdated Show resolved Hide resolved
vortex-expr/src/expressions.rs Outdated Show resolved Hide resolved
vortex-expr/src/display.rs Outdated Show resolved Hide resolved
vortex-expr/src/operators.rs Outdated Show resolved Hide resolved
vortex-expr/README.md Outdated Show resolved Hide resolved
@jdcasale jdcasale changed the title [WIP] Minimal expressions API for vortex Minimal expressions API for vortex May 14, 2024
@jdcasale
Copy link
Contributor Author

closing in favor of #318

@jdcasale jdcasale closed this May 15, 2024
@jdcasale jdcasale mentioned this pull request May 15, 2024
2 tasks
gatesn pushed a commit that referenced this pull request May 15, 2024
Minimal set of expressions and operators for defining predicates over
vortex arrays.

This does **_not_** express recursive expression trees, but instead
restricts the algebra to disjunction-of-conjunction (OR-of-AND) of
field/value comparison operations.

I tried to restrict this to a manageable set of expressions that we will
definitely need as to avoid bloat; the expectation here is that this may
grow over time.

Subsequent changes will introduce:

- [ ] A pushdown API for vortex arrays
- [ ] Array pushdown implementations for the various expressions

Supercedes #308
@robert3005 robert3005 deleted the jc/expr branch August 13, 2024 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants