Publish benchmark results

[simnalamburt]

I’m curious about the details of this benchmark. Instead of running the benchmarks myself, it would be helpful if the benchmark results were published on a page (such as GitHub Pages or the KAIST-CP server) for easy access. If the results are already available and I’ve missed them, please let me know.

[powergee]

Thank you for the suggestion. At the moment, there isn't a dedicated webpage for benchmark results. However, you can refer to published papers that provide representative benchmark results for various reclamation schemes and data structures. I'd like to highlight two recent papers below: the first includes workloads involving reference counting schemes (e.g., CIRC and CDRC), and the second covers major manual schemes (e.g., EBR and HP). Please see the Experiment and Appendix sections.

• Jaehwang Jung, Jeonghyeon Kim, Matthew J. Parkinson, and Jeehoon Kang, Concurrent Immediate Reference Counting, PLDI 2024.
• Jeonghyeon Kim, Jaehwang Jung, and Jeehoon Kang, Expediting Hazard Pointers with Bounded RCU Critical Sections, SPAA 2024.

We agree that a public website with official benchmark results is necessary. We are working on it and aim to publish the website by the end of this year.

[simnalamburt]

@powergee Question: May I consider https://github.com/kaist-cp/circ as a SotA algorithm/implementation?

[powergee]

@simnalamburt

In the context of deferred reference counting schemes for unmanaged languages, I believe CIRC is the state-of-the-art, outperforming CDRC (the previous state-of-the-art) across overall workloads (see CIRC-EBR vs. CDRC-EBR in the second paper).

[simnalamburt]

Thank you for your response. From the user’s perspective, the most important and relevant point is that circ is the SotA, and the rest serves as reference material for study purposes.

Summarizing benchmark results and publishing a web version would take time, so if you’re pressed for time, simply adding one line to the README like “Our current recommendation: https://github.com/kaist-cp/circ” along with a brief summary of how much faster it is compared to std::sync::Arc should suffice.