Rust solution in 6 seconds

[rathijitpapon]

My implementation takes 6.7 seconds on average to run in my M3 Pro, 11 CPUs, and 18G memory.

Algorithm:

1. Divide the file into small buffer chunks and send them to threads.
2. Divide the threads into multiple stages and run them in parallel. In each stage, only a fixed number of threads will be run. And after completion, the next stage will start running.
3. Aggregate the results for all the threads and calculate the final result.

Initially, using standard library approach to split and parse the lines, it took around 13 seconds. Then I iterate over all the lines manually and do the calculation when iterating. It comes down to 6.7 seconds.

I believe there are some better ways to solve this. If anyone has any ideas to improve this, will appreciate them. 🤲

Source codes: https://github.com/rathijitpapon/onebrc

[raahulrahl]

Great Work <3