Description: Tangram Vision will develop a low-power sensor system for a Lunar Terrain Vehicle (LTV) that utilizes a wide-FOV stereo pair of event cameras (a.k.a. neuromorphic camera or dynamic vision sensor), along with an IMU, to enable quick localization and confident mapping in environments with extreme lighting conditions. This combination of novel sensor modalities will enable the creation of an event-informed dense depth map of the environment that is consistent over time and lighting conditions, due to an event camera's ability to identify occlusions and objects in motion relative to the world coordinate frame. Funding will be used to create a hardware prototype of the proposed sensor system. The remaining capital will be allocated to the technical experts for their investigations and contributions to the project. At the end of this phase, Tangram Vision will have demonstrated that a stable depth synthesis pipeline can be created from two stereo event cameras. If a stable pipeline has not been achieved by the end of this phase, Tangram Vision will report how such a pipeline could be achieved given different resources or approaches. Target markets for such a system extend well beyond lunar exploration. A sensor suite like the one proposed has applications in robotics, autonomy, drones, and any system moving at a high speed through challenging lighting conditions. We see this work as the first step towards fully commercialized event sensing, which currently does not have a presence in the sensor market.

Benefits: NASA's Artemis missions will return humans to the moon in order to establish long-term presence. Long-term autonomy is key to this mission, as NASA surface assets and vehicles will continue operations before, during, and after human presence on the lunar surface. In order to achieve this, new and novel sensing technologies are needed to enable long-term autonomy of LTVs and other robotic systems even through the extreme lighting conditions encountered at the lunar south pole. The proposed sensor suite uses commercial off-the-shelf (COTS) sensors in a new and innovative way, one that Tangram Vision predicts will achieve these objectives handily. Many of the challenges presented by the lunar surface are mitigated directly by event camera technology: - High dynamic range (120-150dB), allowing it to operate in scenes with intense bright and dark patches without losing fidelity - Extremely fast data readout speed, at <1 microsecond per event - Highly energy efficient in low-feature environments. If no event is triggered, no data is transmitted However, this technology has never been incorporated into a commercial offering that is easy-to-use, or that operates in any way like a conventional sensor suite. Tangram Vision's experience developing sensor suites for the consumer market gives us a deep understanding of these technologies. This understanding will allow quick iteration towards a sensor product that not only works in the lab, but in the long-term autonomy deployments necessary during Artemis missions. The promise of event cameras has captivated researchers, professionals, and customers in the world of autonomy and robotics for years. After all, many of the most vexing problems in perception — for instance, performance across a wide dynamic range of lighting — are easily handled by these sensors. Because of their superior performance in challenging environments, as well as their fundamental suitability for many autonomy applications (including robotics, autonomous vehicles, and drones), the market for event cameras is estimated to reach $5B in global sales by 2029, and $21.3B by 2034. Along similar lines, it is estimated that event-based vision will capture 9% of the total CMOS imaging market by 2030. By modifying an existing depth sensor design we already sell commercially to accommodate a stereo pair of event cameras as the primary sensing elements, Tangram Vision has the ability to rapidly develop and launch a first-to-market event-enabled 3D sensor. With additional contributions in partnership with Venturi Astrolab, Tangram Vision will be able to adapt this sensor to a variety of environments, from standard terrestrial environments to the harsh environment of the lunar surface. As a result, Tangram Vision will be among the first companies to commercialize event cameras in a real world ready package, thereby gaining access to an outsized share of the large, latent market for event cameras.