Analyses for Learning from failure - guidelines for improving machine learning in medical imaging by Varoquaux & Cheplygina

In this repository you can find the data and code to reproduce Figures 1 to 3 from the paper.

Fig. 1 - Progress of Alzheimer's diagnosis challenges

• paper_tables_raw - tables as saved from HTML, or extracted by Tabula from the paper PDF.
• paper_tables_clean - same tables after cleaning by the raw tables
• plot_papers.py - code creating the figure

Fig. 2 - Lung vs breast cancer in medical vs medical machine learning papers

• publications - Excel file with publication numbers extracted from Dimensions.AI
• plot_publicationgap.py - code creating the figure

Fig. 3 - Analysis of gaps between public and private leaderboards on Kaggle

• kaggle - tables with the raw data about the challenges
• plot_kaggle.py - code creating the figure