ICArUS: Phase 2 Challenge Problem Design and Test Specification describes the challenge problem, and outlines the T&E approach for evaluating models.
ICArUS: Phase 2 Challenge Problem Design and Test Specification
Download Resources
PDF Accessibility
One or more of the PDF files on this page fall under E202.2 Legacy Exceptions and may not be completely accessible. You may request an accessible version of a PDF using the form on the Contact Us page.
Phase 2 of the IARPA program ICArUS (Integrated Cognitive-neuroscience Architectures for Understanding Sensemaking) requires a research problem that poses cognitive challenges of spatial-temporal sensemaking (BAA, 2010). The problem serves as a challenge for performers who are building integrated cognitive-neuroscience models, and as a tool for obtaining data from human experiments. This document describes the challenge problem, and outlines the T&E (Test & Evaluation) approach for evaluating models in Comparative Performance Assessment and Cognitive Fidelity Assessment (BAA, 2010). Normative (Bayesian) solutions to the challenge problem are derived, as needed to support the assessment of human and model performance. Opportunities are also identified for transition of the challenge problem design and results to the geospatial Intelligence Community.
Note: This document was originally prepared and delivered to IARPA in March, 2014, in order to support ICArUS Phase 2 T&E efforts that concluded in June, 2014.