Leaderboard

2022

Task 1: Difficulty Level Classification

Team AUC F1 Rank AUC Rank F1 Avg. of Ranks Overall Team Rank
             
             
             
             
             
             

 

Task 2: Performance Error Prediction

Team RMSE Pearson Correlation
(PCorr)
Spearman Correlation
(SCorr)
Rank RMSE Rank PCorr Rank SCorr Avg. of Ranks Overall Team Rank
                 
                 
                 
                 
                 
                 

2021

Task 1: Difficulty Level Classification

*In ties, priority is given to greater AUC rank

Team AUC F1 Rank AUC Rank F1 Avg. of Ranks Overall Team Rank
USAFA-AFIT 0.75 0.47 1 1 1 1
RAFT LLC 0.69 0.35 2 3 2.5 2
BESPIN 0.68 0.39 3 2 2.5 3
ARL 0.63 0.47 4 1 2.5 4
GMU 0.49 0.26 5 4 4.5 5
HAF/A1XD 0.48 0.25 6 5 5.5 6
PTN 0.44 0.14 7 6 6.5 7

 

Task 2: Performance Error Prediction

*Statistically significant correlation, where indicated

Team RMSE

Pearson Correlation
(PCorr)

Spearman Correlation
(SCorr)

Rank RMSE Rank PCorr Rank SCorr Avg. of Ranks Overall Team Rank
RAFT LLC 2634 0.53* 0.42* 1 1 1 1 1
GMU 3499 0.33* 0.18 4 2 3 3 2
PTN 11674 0.33* 0.19 6 2 2 3.33 3
HAF/A1XD 2868 0.06 0.13 2 4 4 3.33 3
USAFA-AFIT 2973 0.10 0.04 3 3 5 3.67 4
ARL 7560 -0.05 -0.03 5 5 6 5.33 5