Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP:Joezhong fix 320 #323

Closed
wants to merge 10 commits into from
Closed

WIP:Joezhong fix 320 #323

wants to merge 10 commits into from

Conversation

Beastjoe
Copy link
Contributor

Related Issue #320

Sorry for the delay. I tried to solve the case when scramble tables are involved in set operation. Currently, no AsyncAggExecutionNode is created if scramble tables are in set operation. The reason is that VerdictDB will select and group by verditdbtier column. However, in set operation, the number of selected columns should be the same. For instance,
select count(something) from (select something from scramble_table UNION select something from normal_table)
If we try to use AsyncAggExecutionNode here, the query created by the individual AggExecutionNode will become like
select something, verdictdbtier from scramble_table where verdictdbblock>=0 UNION select something from normal_table

@codecov-io
Copy link

codecov-io commented Dec 10, 2018

Codecov Report

Merging #323 into master will increase coverage by 0.33%.
The diff coverage is 81.04%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #323      +/-   ##
==========================================
+ Coverage   70.62%   70.94%   +0.33%     
==========================================
  Files         168      168              
  Lines       11492    11696     +204     
  Branches     1874     1935      +61     
==========================================
+ Hits         8115     8297     +182     
- Misses       2891     2904      +13     
- Partials      486      495       +9
Impacted Files Coverage Δ
.../main/java/org/verdictdb/sqlwriter/QueryToSql.java 64% <0%> (-4%) ⬇️
...verdictdb/core/querying/ola/InMemoryAggregate.java 83.46% <100%> (+0.12%) ⬆️
...ictdb/core/querying/QueryNodeWithPlaceHolders.java 88.28% <100%> (ø) ⬆️
...java/org/verdictdb/sqlwriter/SelectQueryToSql.java 83.16% <100%> (+0.45%) ⬆️
...org/verdictdb/sqlreader/ScrambleTableReplacer.java 78% <33.34%> (-4.22%) ⬇️
...verdictdb/core/sqlobject/SetOperationRelation.java 58.63% <52.64%> (-3.28%) ⬇️
...va/org/verdictdb/coordinator/ExecutionContext.java 85.1% <66.67%> (+0.09%) ⬆️
...main/java/org/verdictdb/sqlreader/RelationGen.java 89.31% <66.67%> (-2.5%) ⬇️
.../org/verdictdb/sqlreader/RelationStandardizer.java 79.88% <85.72%> (+0.4%) ⬆️
...tdb/core/querying/ola/AsyncQueryExecutionPlan.java 84.27% <88.6%> (-0.54%) ⬇️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2922390...f16d283. Read the comment docs.

…oezhong-fix-320

# Conflicts:
#	src/main/java/org/verdictdb/coordinator/ExecutionContext.java
…oezhong-fix-320

# Conflicts:
#	src/main/java/org/verdictdb/coordinator/ExecutionContext.java
@Beastjoe
Copy link
Contributor Author

Beastjoe commented Jan 9, 2019

@pyongjoo
I think now set operation is supported for async aggregation. When scramble tables have different probability distribution, VerdictDB will not create asyncAggExecutionNode.

…f set operation when one table is scrambled and another is not scrambled.
…oezhong-fix-320

# Conflicts:
#	src/main/java/org/verdictdb/sqlreader/ScrambleTableReplacer.java
…oezhong-fix-320

# Conflicts:
#	src/main/java/org/verdictdb/sqlreader/ScrambleTableReplacer.java
@pyongjoo pyongjoo closed this Oct 12, 2019
@pyongjoo pyongjoo deleted the joezhong-fix-320 branch October 12, 2019 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants