We propose to develop a method to effectively utilize the massive amount of image and range data that cameras and laser scanners can generate for an autonomous navigation system. Our innovative approach is to use deep learning to detect and segment only the most useful portions of the data and to use that to build better 3D models. We have proven methods for building accurate 3D models of the environment for robotic systems. [Wettergreen12] This new work will enable us to create better 3D models by identifying and incorporating the most salient information. These models will be more sparse but will have higher information content. This will improve ease of communication and quality of action planning. In Phase 1 we will prove the underlying concepts (use of salient features, segmentation by learning, efficient 3D modeling) in the context of one test environment. We will evaluate methods for determining the importance of object characteristics in their overall quality as a landmark. We will demonstrate improved landmark detection and selection in rocky and natural terrain. In Phase 2 we will generalize the work, considering several policies for salience and will implement system and test in multiple environments. We will demonstrate system learning over time and detecting new and reliable landmarks. Evaluate and demonstrate selective downlink methods that allow for offline training while also returning science-relevant data.

This work benefits NASA by advancing solutions to challenges identified in the 2015 NASA Technology Roadmap: TA4 Robotics and Autonomous Systems, specifically TA4.1 Sensing and Perception and TA4.2 Mobility, as they both relate to modeling the environment in three-dimensions. The proposed research and development will provide maps (3D models) for surface and above-surface mobility and manipulation. Our innovation will reduce requirements for onboard memory and computing power.

The challenges of modeling the environment confront almost every industrial and commercial application. In natural environments such as agriculture, forestry or undersea, where both vast scale and minute detail are important, accurate 3D models lead to efficient navigation and interaction. The same benefits will be realized in artificial environments indoors, such as in factories and warehouses, and outdoors as in mine and construction sites.