TASK 1: convergence score significance

[dskhanirfan]

Hi What does a high or low convergence score indicate? and is it considered as a metric in performance evaluation for teams?

[sarthakpati]

Question is unclear. Please provide more details.

[dskhanirfan]

simulated time, convergence score, dice score and Housedorf distance is calculated. What is the significance of convergence score? and what does it represent?

[sarthakpati]

By convergence score, I assume you mean the time to convergence metric, and this will be computed as the area under the validation learning curve over 1 week of simulated time where the horizontal axis measures simulated runtime and the vertical axis measures the current best score, computed as the average of enhancing tumor, tumor core, and whole tumor DICE scores over the validation split of the training data.

The full description is given in https://github.com/FETS-AI/Challenge/tree/main/Task_1#time-to-convergence-metric-formerly-communication-cost

[dskhanirfan]

I mean Projected Convergence Score not Simulation Time

[Linardos]

I am also obtaining something called "convergence score", it doesn't say time to convergence and it's in decimals so it doesn't look like time:

[sarthakpati]

Tagging @alexey-gruzdev for more clarification.

[msheller]

The "convergence_score" metric is the same metric as detailed here. Apologies for the name mismatch.

The "projected convergence score" tells you what your "convergence_score" value would be if the model fails to improve during the remainder of the training (i.e. if the AUC were to remain flat from that point in time onward).

Does this clarify? Happy to add more detail if not.

[dskhanirfan]

Thanks. What does higher projected convergence score indicate (is 80 better than 60 and how) please add more detail