Description: For future missions with human-robot teams, the NASA Space Technology Roadmap highlights challenges in human-robot interfaces for dexterous telerobotic control, gesture control of robotic assistants, and telesupervision. To help address these challenges, JPL is developing a sleeve-based gesture recognition interface that could be embedded and worn in IVA and EVA suits. The JPL BioSleeve incorporates electromyography sensors (to measure muscle activity), inertial sensors (IMUs), and magnetic field sensors, to provide intuitive force and position control signals from natural arm, hand, and finger movements. In the near-term, we plan to propose BioSleeve technology demonstrations aboard the ISS for gestural control of the Canadarm2 and SPHERES robot platforms. Problem Statement The current prototype has successfully achieved >95% accuracy in recognition of 20 static hand gestures and 10 dynamic arm gestures in ground testing. However, both capabilities are affected by the presence of gravity: the patterns of muscle activity change with respect to pose-dependent loading of the arm, and the gravity vector has been used with the magnetic field vector to predict arm pose. Because the near-term planned operational environment is on the International Space Station (ISS), we need to test the gesture recognition performance in the absence of gravity. The parabolic flight aircraft will allow testing in a microgravity environment, which will provide critical data on the performance of the interface, as well as give the team invaluable operational experience that may influence the design, development, and test program for the interface as it advances toward multiple flight opportunities.

Benefits: The BioSleeve can be used for human-robot interaction such as directing rovers by astronaut on EVA (extra-vehicular activities) or telerobotic control for remote manipulation such as Canadarm2. Future Customers This work is significant because NASA and commercial space companies could benefit from BioSleeve gesture control technology for human-robot interactions in multiple applications, such as: natural hand gestures to direct surface rovers by an astronaut on EVA; intuitive telerobotic control for remote manipulation and for high degree-of-freedom robots such as Robonaut and Canadarm2 on ISS; and force control for power-assisted EVA gloves or exoskeletons.