This page summarise the mapping results.
| Study Title | Year | Benchmark Name | URL |
|---|---|---|---|
| Compiling a benchmark of documented multi-threaded bugs | 2004 | [Eytani et al.] | https://cs.haifa.ac.il (Inacessible) |
| Extraction of Bug Localization Benchmarks from History | 2007 | iBUGS | https://www.st.cs.uni-saarland.de/ibugs/ |
| Clash of the Titans: Tools and Techniques for Hunting Bugs in Concurrent Programs | 2009 | [Rungta et al.] | https://vv.cs.byu.edu/vv-lab/concurrency-tool-comparison#access-to-the-multi-threaded-programs |
| (Un-)Covering Equivalent Mutants | 2010 | JAVALANCHE Subject Programs | http://www.javalanche.org/ |
| Empirical Evaluation of Bug Linking | 2013 | [Bissyandé et al.] | http://momentum.labri.fr/bugLinking/ (Inacessible) |
| The Eclipse and Mozilla Defect Tracking Dataset: A Genuine Dataset for Mining Bug Information | 2013 | Eclipse and Mozilla Defect Tracking Dataset | https://github.com/ansymo/msr2013-bug_dataset |
| 42 Variability Bugs in the Linux Kernel: A Qualitative Analysis | 2014 | The Variability Bugs Database | http://vbdb.itu.dk/ |
| Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs | 2014 | Defects4J | http://defects4j.org |
| On the Effectiveness of Information Retrieval Based Bug Localization for C Programs | 2014 | [Saha et al.] | https://utexas.app.box.com/icsme2014-dataset (Inacessible) |
| Automated Bug Finding in Video Games: A Case Study for Runtime Monitoring | 2014 | [Varvaressos et al.] | https://github.com/kimlavoie/BeepBeepPingus |
| A Dataset of High Impact Bugs: Manually-Classified Issue Reports | 2015 | High Impact Bug Dataset | http://oss.sys.wakayama-u.ac.jp/?p=1009 |
| The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs | 2015 | ManyBugs and IntroClass | https://repairbenchmarks.cs.umass.edu/ |
| TaxDC: A Taxonomy of Non-Deterministic Concurrency Bugs in Datacenter Distributed Systems | 2016 | TaxDC | http://ucare.cs.uchicago.edu/projects/cbs/ |
| A Feasibility Study of Using Automated Program Repair for Introductory Programming Assignments | 2017 | [Yi et al.] | https://github.com/jyi/ITSP |
| Automatic detection and demonstrator generation for information flow leaks in object-oriented programs | 2017 | KEG Experiments | https://www.informatik.tu-darmstadt.de/se/se_research/se_projects/albia/albia_tools/keg.de.jsp |
| Codeflaws: A Programming Competition Benchmark for Evaluating Automated Program Repair Tools | 2017 | CodeFlaws | http://codeflaws.github.io/ |
| ELIXIR : Effective Object-Oriented Program Repair | 2017 | Bugs.jar | https://github.com/bugs-dot-jar/bugs-dot-jar |
| How Developers Debug Software The DBGBENCH Dataset | 2017 | DBGBENCH | http://www.st.cs.uni-saarland.de/debugging/dbgbench/ |
| QuixBugs: a multi-lingual program repair benchmark set based on the quixey challenge | 2017 | QuixBugs | https://github.com/jkoppel/QuixBugs |
| SECBENCH: A Database of Real Security Vulnerabilities | 2017 | Secbench | https://tqrg.github.io/secbench/ |
| Crashing Simulated Planes is Cheap: Can Simulation Detect Robotics Bugs Early? | 2018 | ArduBugs | https://github.com/squaresLab/ArduBugs |
| Large-Scale Analysis of Framework-Specific Exceptions in Android Apps | 2018 | Dataset Crash Analysis | https://crashanalysis.github.io/Dataset-CrashAnalysis.html |
| Mining repair model for exception-related bug | 2018 | Exception-related bugs | https://github.com/drzhonghao/exception.bugs |
| Pairika - A Failure Diagnosis Benchmark for C++ Programs | 2018 | Pairika | https://github.com/tum-i22/Pairika |
| Repairing Crashes in Android Apps | 2018 | Droixbench | https://droix2017.github.io/ |