Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sort.sh needs too much memory #34

Open
derhuerst opened this issue Mar 22, 2021 · 0 comments · May be fixed by #35
Open

sort.sh needs too much memory #34

derhuerst opened this issue Mar 22, 2021 · 0 comments · May be fixed by #35
Labels
help wanted Extra attention is needed

Comments

@derhuerst
Copy link
Member

derhuerst commented Mar 22, 2021 •
edited
Loading

Because of #27, I switched to Miller, but it just needs too much memory to sort the CSV files.

I aborted sorting stop_times.txt of the 2021-03-19 DELFI dataset after 1h and >12gb of memory usage.

Are there more efficient tools that can sort CSV files?
@derhuerst derhuerst added the help wanted Extra attention is needed label Mar 22, 2021
@derhuerst derhuerst linked a pull request Mar 23, 2021 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sort.sh: use GNU sort for stop_times.csv 💥 public-transport/gtfs-utils
1 participant
@derhuerst