Description: Current NASA missions to the International Space Station are heavily dependent upon ground controllers to assist crew members in performing routine operations and maintenance as well as responses to off-nominal situations. Standard operating procedures are at the heart of spacecraft operations, with almost 5000 procedures for ISS alone. Performing these procedures often requires close collaboration between ground controllers who have deep knowledge of the spacecraft's systems and crew members who have on-board situation awareness. This close collaboration will become more difficult in extended missions and crew members will need to have more autonomy. Augmented reality technology can help replace some of the guidance that ground controllers offer to crew members during procedure execution. Augmented reality can also provide continuous and just-in-time training opportunities during extended missions as well as entertainment and social connection opportunities. Context-sensitive augmented reality provides different support depending upon the on-board situation and ties directly to procedures, system data, daily plans, background information, and robotic assistants. TRACLabs has developed a procedure integrated development environment called PRIDE that is currently being used by NASA for ISS and Orion procedures. TRACLabs proposes to integrate augmented reality technologies into PRIDE in collaboration with the Georgia Tech Augmented Environmental Lab. In particular, Georgia Tech has developed an augmented realty capable web browser and Javascript framework that will complement the PRIDE web-based procedure execution system. These two industry-leading technologies will form the platform on which a suite of context-sensitive augmented reality applications can be quickly developed and deployed for a variety of NASA applications.

Benefits: This research can have immediate application to ISS operations because there are several iPads already on ISS and a Microsoft HoloLens on its way. We will use ISS procedures as test cases for this project. We will work closely with a variety of NASA personnel who are working on next generation procedure displays. We also see applications to robotic missions including control of NASA's R2 and R5 robots and robots on other planetary surfaces. PRIDE is being evaluated for use in ground operators for the Resource Prospector robotic mission to the moon being jointly developed by NASA JSC and ARC. Ground operations personnel are currently evaluating PRIDE and this technology would be able to assist them in their operations. We also have a close relationship with the Autonomous Mission Operations TOCA Autonomous Operations Project (AMO-TOCA)being tested on-board ISS. This could serve as a potential Phase~III testbed on ISS. Finally, we have connections to the Human Research Program (HRP) at NASA and will work with those personnel to identify applications, including analog test environments, for this work. We will meet with all of these individuals in Phase~I to determine their requirements and use cases for augmented reality.

TRACLabs is already selling PRIDE as a commercial product with oil field services company Baker Hughes as a launch customer. Baker Hughes is field-testing PRIDE at several sites world-wide before deployment in actual operations in mid-2016. PRIDE is proving automation assistance to drilling operations. Augmented reality would immediately increase the effectiveness of the PRIDE software in drilling operations by providing assistance in performing complex and dangerous procedures. TRACLabs expects additional customers in the oil and gas industry will deploy PRIDE once it has been proven effective by Baker Hughes. TRACLabs also sees application of this technology in the automotive manufacturing area. TRACLabs performed a small pilot project for automotive supplier Magna (second largest in the world with 285 manufacturing facilities and over 125,000 employees) on flexible robotic assembly. This was successful, and after a tour of several Magna manufacturing facilities in North America, TRACLabs personnel are negotiating a follow-on contract for research and development. Augmented reality would be used to assign personnel on the manufacturing floor in performing their tasks and validating their work. We expect other manufacturing companies to be interested as well. Sierra Nevada Corporation has also purchased PRIDE licenses for use in their Dream Chaser program, which was recently selected to deliver cargo to ISS.