Open Source Parallel Image Analysis and Machine Learning Pipeline
Status: Completed
Start Date: 2016-06-10
End Date: 2016-12-09
Description: Continuum Analytics proposes a Python-based open-source data analysis machine learning pipeline toolkit for satellite data processing, weather and climate data processing, and machine learning and prediction with optional proprietary cluster management tools for streamlined deployment for cloud providers and on-premises clusters. The innovative software will empower scientists and analysts to readily and seamlessly construct and test workflows that transparently and scalably perform calculations across cluster nodes for data-driven discovery. The simple API for homogenous processing of images, mosaics and tiles further improves ease of use for rapid testing and prototyping of analyses paradigms for multiple extremely large data sets. Today, NASA researchers must create, debug, and tune custom workflows for each analysis. Creation and modification of custom workflows is fragile, non-portable, and consumes time that could be better spent on advancing scientific discovery. The Phase I work plan will demonstrate that it is feasible to easily create and compose data manipulations and analytics from a variety of sources with a portable, reproducible, extensible process that can be deployed on a wide variety of systems and software. This is a major improvement over the current state-of-the-art because of reduced workflow creation time, portability of deployment and use, extensibility, and robustness.
Benefits: Continuum Analytics sees direct usage applications for the image analysis and machine learning pipeline in the areas of: - NASA land use and land cover analysis and change detection products - NASA volcanic thermal measurement and monitoring - NASA snow and water balance products - NASA classifications for hydrologic and geomorphic features, such as wetland delineation and mapping of nuisance algal blooms and other water column features, similar to the work described in the HyspIRI 2015 HyspIRI Aquatic Studies Group (HASG) Report
The team sees direct usage application of the image analysis and machine learning pipeline outside of NASA, such as: - NOAA mission-related research to predict changes in climate, weather, oceans and coast, and conserving and managing coasting and marine ecosystems and resources. - DOD/IC - foreign defense and homeland security applications - Commercial infrastructure and engineering, disaster management and mitigation analysis, natural resource monitoring, energy-related exploration and operational management. - Flood and floodplain mapping for insurance adjustments, bridge construction projects, FEMA floodplain definitions, river habitat and restoration projects, and emergency planning at local, state, and federal agencies - Forest disease and insect damage density identification for large commercial forest owners - Snow and ice cover and recession analysis useful in climate change and water management planning at federal, state, and local agencies - Developing spectral identifiers of agricultural crops in healthy versus water and nutrient stressed conditions - Classifying parking lots and roads based on the number of vehicles evidently in the image, an indicator of economic activity and also potentially useful in federal security applications - Mapping ecologically sensitive and geotechnically unstable areas, such as wetlands and mass wasting events, useful for reducing the cost of development review in local, state, and federal environmental agencies
The team sees direct usage application of the image analysis and machine learning pipeline outside of NASA, such as: - NOAA mission-related research to predict changes in climate, weather, oceans and coast, and conserving and managing coasting and marine ecosystems and resources. - DOD/IC - foreign defense and homeland security applications - Commercial infrastructure and engineering, disaster management and mitigation analysis, natural resource monitoring, energy-related exploration and operational management. - Flood and floodplain mapping for insurance adjustments, bridge construction projects, FEMA floodplain definitions, river habitat and restoration projects, and emergency planning at local, state, and federal agencies - Forest disease and insect damage density identification for large commercial forest owners - Snow and ice cover and recession analysis useful in climate change and water management planning at federal, state, and local agencies - Developing spectral identifiers of agricultural crops in healthy versus water and nutrient stressed conditions - Classifying parking lots and roads based on the number of vehicles evidently in the image, an indicator of economic activity and also potentially useful in federal security applications - Mapping ecologically sensitive and geotechnically unstable areas, such as wetlands and mass wasting events, useful for reducing the cost of development review in local, state, and federal environmental agencies
Lead Organization: Continuum Analytics, Inc.