Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARROW-10908: [Rust][DataFusion] Update relevant tpch-queries with BETWEEN #8906

Closed
wants to merge 1 commit into from
Closed

ARROW-10908: [Rust][DataFusion] Update relevant tpch-queries with BETWEEN #8906

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions rust/benchmarks/src/bin/tpch.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,7 +366,7 @@ fn create_logical_plan(ctx: &mut ExecutionContext, query: usize) -> Result<Logic
where
l_shipdate >= date '1994-01-01'
and l_shipdate < date '1995-01-01'
and l_discount > 0.06 - 0.01 and l_discount < 0.06 + 0.01
and l_discount between 0.06 - 0.01 and 0.06 + 0.01
and l_quantity < 24;"
),

Expand Down Expand Up @@ -400,7 +400,7 @@ fn create_logical_plan(ctx: &mut ExecutionContext, query: usize) -> Result<Logic
(n1.n_name = 'FRANCE' and n2.n_name = 'GERMANY')
or (n1.n_name = 'GERMANY' and n2.n_name = 'FRANCE')
)
and l_shipdate > date '1995-01-01' and l_shipdate < date '1996-12-31'
and l_shipdate between date '1995-01-01' and date '1996-12-31'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use the TPC-H Query validation section to resolve the use of < or <= etc by running the query against SF1 data and comparing the result

According to the spec, the expected answer is 123141078.23 at SF1

http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-h_v2.17.2.pdf
page 38

REVENUE
123141078.23

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great idea, I agree this would be very valuable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am working with Andy to increase the testing provided by the TPC-H queries here: #8760 (comment)

The plan is to verify results against the correct results when run in test mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When implementing BETWEEN I went hunting for the ANSI SQL spec but the best I could do is find the Postgres implementation which confirms it is definitely INCLUSIVE.

https://www.postgresql.org/docs/13/functions-comparison.html

The BETWEEN predicate simplifies range tests:

a BETWEEN x AND y
is equivalent to

a >= x AND a <= y

) as shipping
group by
supp_nation,
Expand Down