2022
Task 1: Difficulty Level Classification
Team | AUC | F1 | Rank AUC | Rank F1 | Avg. of Ranks | Overall Team Rank |
---|---|---|---|---|---|---|
RAFT | 0.914 | 0.74 | 1 | 1 | 1 | 1 |
Patrick Moore | 0.877 | 0.66 | 2 | 2 | 2 | 2 |
ARL-HCxS | 0.855 | 0.60 | 3 | 3 | 3 | 3 |
RSAF+ | 0.765 | 0.44 | 4 | 4 | 4 | 4 |
Noah Lee | 0.705 | 0.44 | 5 | 4 | 4.5 | 5 |
Team59TES | 0.699 | 0.41 | 6 | 6 | 6 | 6 |
Cog In The Machine | 0.653 | 0.41 | 7 | 6 | 6.5 | 7 |
John O'Donnell | 0.606 | 0.31 | 8 | 7 | 7.5 | 8 |
Dennis Brown | 0.528 | 0.29 | 9 | 8 | 8.5 | 9 |
Hans Gevers | 0.486 | 0.25 | 10 | 10 | 10 | 10 |
Task 2: Performance Error Prediction
Team | RMSE | Pearson Correlation (PCorr) |
Spearman Correlation (SCorr) |
Rank RMSE | Rank PCorr | Rank SCorr | Avg. of Ranks | Overall Team Rank |
---|---|---|---|---|---|---|---|---|
Team59TES | 3377.4 | 0.53* | 0.70* | 1 | 1 | 1 | 1 | 1 |
RAFT | 3459.7 | 0.48* | 0.46* | 2 | 2 | 2 | 2 | 2 |
ARL-HCxS | 3930.8 | 0.49* | 0.27* | 3 | 2 | 3 | 2.67 | 3 |
Dennis Brown | 4342.7 | -0.15 | -0.02 | 4 | 5 | 4 | 4.33 | 4 |
John O'Donnell | 4339.02 | -0.11 | -0.06 | 5 | 6 | 4 | 5 | 6 |
Noah Lee | 5513.1 | 0.18 | 0.03 | 6 | 4 | 4 | 4.67 | 5 |
2021
Task 1: Difficulty Level Classification
*In ties, priority is given to greater AUC rank
Team | AUC | F1 | Rank AUC | Rank F1 | Avg. of Ranks | Overall Team Rank |
---|---|---|---|---|---|---|
USAFA-AFIT | 0.75 | 0.47 | 1 | 1 | 1 | 1 |
RAFT LLC | 0.69 | 0.35 | 2 | 3 | 2.5 | 2 |
BESPIN | 0.68 | 0.39 | 3 | 2 | 2.5 | 3 |
ARL | 0.63 | 0.47 | 4 | 1 | 2.5 | 4 |
GMU | 0.49 | 0.26 | 5 | 4 | 4.5 | 5 |
HAF/A1XD | 0.48 | 0.25 | 6 | 5 | 5.5 | 6 |
PTN | 0.44 | 0.14 | 7 | 6 | 6.5 | 7 |
Task 2: Performance Error Prediction
*Statistically significant correlation, where indicated
Team | RMSE |
Pearson Correlation |
Spearman Correlation |
Rank RMSE | Rank PCorr | Rank SCorr | Avg. of Ranks | Overall Team Rank |
---|---|---|---|---|---|---|---|---|
RAFT LLC | 2634 | 0.53* | 0.42* | 1 | 1 | 1 | 1 | 1 |
GMU | 3499 | 0.33* | 0.18 | 4 | 2 | 3 | 3 | 2 |
PTN | 11674 | 0.33* | 0.19 | 6 | 2 | 2 | 3.33 | 3 |
HAF/A1XD | 2868 | 0.06 | 0.13 | 2 | 4 | 4 | 3.33 | 3 |
USAFA-AFIT | 2973 | 0.10 | 0.04 | 3 | 3 | 5 | 3.67 | 4 |
ARL | 7560 | -0.05 | -0.03 | 5 | 5 | 6 | 5.33 | 5 |