Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"One file output" option added for csv/json. #21

Merged
merged 10 commits into from
Feb 19, 2024

Conversation

keltia
Copy link
Contributor

@keltia keltia commented Jan 29, 2024

The default output for csv and json files by datafusion is to output a series of files inside a directory named as the output file. I find it quite inconvenient so I have added -o/--one-file option. I have switched to clapa long time ago so I'm not sure how to rename the option (I agree -o is not ideal) so feel free to change it 😁. Thanks a lot for bdt BTW, it beats writing it myself.

@kassemz
Copy link

kassemz commented Feb 3, 2024

Thanks @andygrove for creating this nice tool! and thanks @keltia you added what I was looking for!

What about -ns/--no-split or -s/--single-file as other options?

@keltia
Copy link
Contributor Author

keltia commented Feb 19, 2024

I renamed -o into -s and --single-file. I also added support for Zstd. And updated datafusion and tokio.

Copy link
Collaborator

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @keltia!

@andygrove
Copy link
Collaborator

@keltia Could you fix the merge conflicts?

Cargo.toml Outdated Show resolved Hide resolved
@andygrove andygrove merged commit bbd4743 into datafusion-contrib:main Feb 19, 2024
1 check passed
@keltia keltia deleted the one-file-option branch February 20, 2024 19:13
@keltia
Copy link
Contributor Author

keltia commented Feb 20, 2024

Oh sorry, didn't see your request but I see you got it done, thanks for the merge!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants