-
Notifications
You must be signed in to change notification settings - Fork 108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Benchmarking and mimalloc #67
Comments
Although the designs between mimalloc and snmalloc were very similar I think we had a different bias in why we had per slab free lists. In snmalloc it was primarily about fragmentation rather than performance. The mimalloc paper, demonstrated that this could lead to amazing performance. Current things we did taken from mimalloc paper/design
|
I am thinking about creating a benchmark website and update the results periodically (like some performance analysis website for the web frameworks). |
Hi @SchrodingerZhu , if you are planning to do that, I do think the ps: (Now, of course, I am already guilty myself by putting Anyway, just my 2c. |
@SchrodingerZhu I think it would be great to have something like this. But I also share Daan's (@daanx) worry. Personally, I use the micro-benchmarks for finding problems rather than justifying superiority. I have been using mimalloc-bench for the last two weeks in preparation for a release of snmalloc. I found four places where perf was significantly different to mimalloc. When I studied these I found issues in the code or benchmark. Leading to several of the recent PRs.
I would never want to do a parameter sweep to micro-optimise against this benchmark suite, but if you can justify the change independent of the benchmark, then I think it is great to let them find issues. You should also check why you are faster, not just the cases where you are slower. Perhaps, to help address Daan's concerns places where allocator are targeted at should have specific benchmarks built for them. So perhaps once we have a live site doing this. Working out how to demonstrate latency in a new benchmark would be really interesting. My change #158 that improves xmalloc-testN may harm latency, but perhaps not enough to notice. Also, at least allocators can be switched in real applications, so that can then address some of the larger scale benchmarking. I think something like Tech Empower benchmarks has been hugely influential in making web servers faster. |
work in progress at: https://github.com/SchrodingerZhu/bench_suite Though a preference of general performance can be useful, I agree that it is unfair for some allocators. (for instance, |
current output can be found at: https://github.com/SchrodingerZhu/bench_suite/blob/master/output/index.md. @mjp41 It seems that snmalloc is having some higher memory peak in some cases. |
Does the machine you run on have transparent hugh pages? You can check with
If it is set to always, I have found You can disable it with:
Not sure what the correct setting is to have. |
rust related simple benchmarks are added. |
Based on mimalloc's design we have managed to dramatically improve the performance on snmalloc.
We should
@daanx, @davidchisnall, @plietar, @sylvanc
The text was updated successfully, but these errors were encountered: