Submission

Steps to submit your solution for adjudication:

  1. Download the Evaluation Dataset (link coming soon)
  2. Apply your model to the Evaluation Dataset, and save the inference results in a CSV file, with the format specified below.
  3. Assemble your package of materials for submitting your solution
    • CSV file of challenge task predictions for the Evaluation Dataset
    • Completed questionnaire about the details of your solution (download link coming soon)
    • The code implementing your solution and instructions on how to run it. Comments and other description of the code will be much appreciated.
    • (Optional) Your Executive Pitch PowerPoint presentation document (template download link coming soon) 
    • (Optional) A video recording of your Executive Pitch, no longer than 10 minutes, summarizing your solution and what you’ve learned.
  4. Zip all the files above into one single zip file
  5. Complete the submission form below and upload your solution package as one zip file

CSV File Formats for CogPilot Data Challenge 2.0 Task Predictions

For all the runs in the evaluation set, please provide your predictions in a comma-delimited (CSV) file format. The required CSV format is different for each of the two challenge tasks.

Task 1: Difficulty Level Classification

For all the runs in the Evaluation Set, predict the difficulty level (DL) of each run. We will be assessing prediction accuracy of all four difficulty levels. The required columns and formatting are as follows:

Subject

Date

Run

Predicted DL

Probability Scores *

DL1

DL2

DL3

DL4

6

20210324

1

 

 

 

 

 

6

20210324

2

 

 

 

 

 

 

 

 

 

 

 

 

*These are the probability scores for the four difficulty levels, which your model uses to make its final prediction.

Task 2: Performance Error Estimation

For all the runs in the Evaluation Set, predict the Cumulative Performance Error of each run. The required columns and formatting are as follows:

Subject

Date

Run

Cumulative Error **

6

20210324

1

 

6

20210324

2

 

...

 

 

 

** Cumulative performance error is the sum of the three dimensions of flight performance error (airspeed, localizer, and glide slope) computed over the whole run.


Submission Form

 

Challenge Task 1: Predict Difficulty Level

Files must be less than 12 MB.
Allowed file types: .
Files must be less than 12 MB.
Allowed file types: .
Files must be less than 12 MB.
Allowed file types: .
Files must be less than 12 MB.
Allowed file types: html pdf ppt pptx.
Files must be less than 12 MB.
Allowed file types: avi mov mp3 ogg wav.

 

Challenge Task 2: Predict Difficulty Level

Files must be less than 12 MB.
Allowed file types: .
Files must be less than 12 MB.
Allowed file types: .
Files must be less than 12 MB.
Allowed file types: .
Files must be less than 12 MB.
Allowed file types: html pdf ppt pptx.
Files must be less than 12 MB.
Allowed file types: avi mov mp3 ogg wav.