Description: Procedures are commonly used by organizations to specify, document, and disseminate prescribed methods for performing tasks efficiently and effectively. However, even well-trained personnel can make errors when carrying out procedures. The risk of these errors increases when task loads are too low or too high, when multi-tasking or switching between tasks, when interrupted, when complex team coordination or handovers are required, and/or during stressful situations. In these situations, users can become fatigued or complacent, or they can lose situation awareness due to overloaded working memory, automaticity, loss of vigilance, cognitive tunneling, ineffective information scanning, or susceptibility to confirmation and other biases. When these cognitive states occur, users are prone to committing errors such as wrong steps, skipped steps, mode errors, completion errors, default errors, and perseveration.The goal of this project is to develop an intelligent assistant that monitors users, such as crewmembers performing procedural tasks, maintains estimates of the crewmembers' cognitive states (including situation awareness and affective state), identifies situations where the user is at risk of making errors, and selects appropriate interventions that reduce the likelihood of errors. Depending upon the situation and the cognitive state of the user, the assistant will select an intervention that increases user awareness of important situational elements that the user may be missing; and by changing the level of automation, the assistant will reduce user workload.The assistant's modular architecture facilitates plugging in of different data models and algorithms required to monitor user performance, assess situation awareness and cognitive states, identify states that might lead to errors, and intervene to prevent those errors.

Benefits: The technology resulting from this research will monitor user actions and physiological data and estimate the user cognitive state to detect increased risk of specific types of errors. In order to lower these risks, it will execute appropriate interventions such as changing the level of automation and/or the content and presentation of information. This capability will provide the greatest benefits when stress, tight deadlines, high task loads, multi-tasking, task-switching, complex team coordination, and handovers increase the risk of error. Candidate NASA applications include ISS flight crew operations, mission control operations, and the Orion Multi-Purpose Crew Vehicle (MPCV). In the case of near-Earth and deep space missions like MPCV, crews will need to be able to operate more autonomously with complex and sophisticated flight systems. Our technology will complement procedure tools and technologies planned for MPCV such as the presentation of procedures using tools like the MPCV eProc Viewer and augmented reality mobile glasses. This technology will also increase the ability for crew members to oversee the operation of teams of robots during lunar and Martian missions and, in the nearer term, during analog experiments on Earth.

This technology can be used to reduce the risk of errors in other domains that employ guided or memorized procedures, especially when errors are costly. In addition to NASA applications, other domains where loss of situation awareness or affective state might lead to error include aviation (air traffic control, pilot operations), medicine (nursing), nuclear plant operations, and, in general, monitoring, operating, maintaining, and repairing complex systems and processes. The technologies here proposed can also be used to enhance safety in the automotive industry and in the DoD-wide use of autonomous unmanned vehicles (UVs). Operating UVs will require intelligent adaptive interfaces to support new operator workload requirements as they change from many operators controlling one UV to one operator controlling many UVs.