Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate the clean command to sql #6321

Merged
merged 4 commits into from
Jul 13, 2023
Merged

Migrate the clean command to sql #6321

merged 4 commits into from
Jul 13, 2023

Conversation

macneale4
Copy link
Contributor

@macneale4 macneale4 commented Jul 12, 2023

The dolt_clean() stored procedure pretty much does exactly what the CLI does, so this was an easy migration. What the dolt clean command doesn't have is tests, and neither does the dolt_clean() procedure for that matter. Unfortunately, the command experience itself isn't great, and differs significantly from git clean (#6313). So instead of spending time on testing and a full rewrite of dolt clean, I'm making this small tactical change to do nothing but the migration.

Related: #3922

@macneale4 macneale4 marked this pull request as ready for review July 12, 2023 01:02
@macneale4 macneale4 requested a review from stephkyou July 12, 2023 15:21
Copy link
Contributor

@stephkyou stephkyou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a few minor comments

integration-tests/bats/sql-local-remote.bats Outdated Show resolved Hide resolved
integration-tests/bats/sql-clean.bats Show resolved Hide resolved
go/cmd/dolt/commands/clean.go Outdated Show resolved Hide resolved
@macneale4 macneale4 merged commit 766916b into main Jul 13, 2023
19 checks passed
@github-actions
Copy link

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 0.87
adds_updates_deletes 60000 60000 60000 5.04
deletes_only 0 60000 0 2.35
updates_only 0 0 60000 3.2

@github-actions
Copy link

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.06 1
batching batch sql 10000 1 0.09 1.89
batching by line sql 10000 1 0.09 1.89
blob 1 blob 200000 1 1.12 3.93 4
blob 2 blobs 200000 1 1.18 4.78 4.84
blob no blob 200000 1 1.12 1.48 1.46
col type datetime 200000 1 1.01 1.97 2.03
col type varchar 200000 1 0.87 2.2 2.11
config width 2 cols 200000 1 0.94 1.44 1.39
config width 32 cols 200000 1 2.36 1.54 2.39
config width 8 cols 200000 1 1.19 1.55 1.84
pk type float 200000 1 1.09 1.3 1.29
pk type int 200000 1 0.93 1.46 1.4
pk type varchar 200000 1 1.84 1.12 1.02
row count 1.6mm 1600000 1 7.02 1.69 1.59
row count 400k 400000 1 1.78 1.58 1.54
row count 800k 800000 1 3.61 1.63 1.54
secondary index four index 200000 1 4.68 1.15 0.9
secondary index no secondary 200000 1 1.11 1.48 1.5
secondary index one index 200000 1 1.42 1.64 1.6
secondary index two index 200000 1 2.54 1.31 1.13
sorting shuffled 1mm 1000000 0 6.93 2.02 1.88
sorting sorted 1mm 1000000 1 6.86 2 1.84

@github-actions
Copy link

@coffeegoddd DOLT

name mean mean_mult
dolt_blame_basic 29.96 1.38
dolt_blame_commit_filter 209.39 2.83
dolt_commit_ancestors_commit_filter 0.38 0.97
dolt_commits_commit_filter 0.47 1.15
dolt_diff_log_join_from_commit 113.32 1.58
dolt_diff_log_join_to_commit 111.53 1.55
dolt_diff_table_from_commit_filter 0.74 1.21
dolt_diff_table_to_commit_filter 0.69 1.13
dolt_diffs_commit_filter 0.45 1.13
dolt_history_commit_filter 0.68 1.55
dolt_log_commit_filter 0.39 0.95

@macneale4 macneale4 deleted the macneale4/cli-sql-clean branch July 13, 2023 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants