Description: Alphacore and Arizona State University (ASU) propose the development of hardware and software solutions for digital twins of regions of strategic interest on the moon, for robotic mission planning in preparation for and potentially during ARTEMIS human missions. We are developing a next generation imaging system for terrain mapping in dynamic lighting and thermal conditions. When deployed aboard rovers, such 3D imaging systems will produce multi-modal, high resolution imagery that can facilitate AR/VR enabled exploration of terrain at sub-cm ground sampling resolution. In this immersive digital twin, astronauts and stakeholders can carry out assessment and visioning activities as well as mission drills. We identify 1 mm/pixel as an ideal target resolution for terrain assessment, with sub-centimeter scale being the milestone for this proposal, with potential for millimeter scale in phase II if phase I is successful. We will leverage ML algorithms such as diffusion models and neural radiance fields (NeRF), for view synthesis from sparse imagery. Impact of generative features in astronaut mission planning will be assessed on earth analog sites in Arizona. Leveraging these sites, we will develop our methods focusing on synthesis of photorealistic and interactive geological features such as rocks, soil, lava tubes, and craters. Apollo astronaut imagery and ASU LROC imagery will be leveraged extensively. Our goal for phase I is to provide hardware solutions and a software pipeline for high-resolution terrain assessment (sub-cm resolution) with semantic and geologic trait metadata estimated by humans and AI. 3D rock models from photogrammetry, lidar scans, and spectroscopy will be assimilated to synthesize arbitrary views for multiple participants, enabling use in rendering engines at NASA (Unreal and Unity). Finally, this development will support also the creation of highly immersive and high-accuracy simulation environments for rover operations.

Benefits: The Exploration Systems Development Mission Directorate (ESDMD), Space Operations Mission Directorate (SOMD), Space Technology Mission Directorate (STMD), and Science Mission Directorate (SMD), as well as the Artemis and Gateway programs. Metaverse and digital twin technologies minimize costs and risks, enhance collaboration, training, and operations, and support education and outreach. Various NASA Programs and Directorates, including Aeronautics, Human Exploration, Science, Space Technology, Artemis, and Gateway, among others.

Companies specializing in gaming, entertainment, or social networking are driving the development of metaverse technologies, engineering and science. Climate and weather forecasting. Next-generation buildings and warehouses. Epic Games' Fortnite for interaction in a completely digital environment. The Roblox for content generated and linked into a metaverse. NVIDIA's Omniverse immersive platform.