CloudTurbine: Streaming Data via Cloud File Sharing

Status: Completed

Start Date: 2016-04-25

End Date: 2019-05-23

Description:

We propose a novel technology to leverage rapidly evolving cloud based infrastructure to improve time constrained situational awareness for real-time decision making. Our "CloudTurbine" innovation eliminates the distinction between files and streams. Streaming and static data have long been considered separately. Whereas streaming data protocols continue to fragment, a great unification of approach for static data has occurred. The paradigm for file-sharing services is simple: (1) put data in a local file folder, (2) it automatically shows up at other linked systems. Wouldn't it be nice if one could unify an approach for streaming data while leveraging the file-sharing cloud infrastructure? This is precisely what we propose. CloudTurbine is a streaming data interface to file-sharing and file-transport services such as Dropbox and FTP. It delegates much of the data transmittal, security, and server resources to the third-party service. It eliminates the distinction between files and streams, and enables a simple, cost effective new paradigm for streaming data middleware. Phase I demonstrated the features, utility, and performance of the prototype CloudTurbine to be very appropriate for a wide range of data-streaming applications. A significant performance issue is the inherent latency imposed by file bundling plus transfer time. Testing has proven this to be tractable, with low latency (2-5s) for several file sharing services, and very low latency (10-50ms) for protocols such as FTP and mapped drives. We propose an Open Source CloudTurbine web portal as a hub to provide this technology to NASA, scientific, and industrial communities. Following the legacy of the earlier-generation DataTurbine, we will vigorously move to establish a self-sustaining Open Source community which provides CloudTurbine access, development, services, spin-off products, and community support.

Benefits:

CloudTurbine would provide a significant advancement of the NASA "Virtual Presence" infrastructure for flight research data and wind tunnel test data remote collaboration. Other potential NASA applications include LVC-DE distributed flight simulation, SOFIA sensor data streaming to schools for educational outreach, and conversion of legacy flight data to a sustainable, transparent archive format. CloudTurbine is synergistic and compatible with the DataTurbine middleware and WebScan web viewer in use by NASA, and leverages the investment and utility of these legacy technologies. It brings new advantages and lowers the barriers to interfacing data at both source and consumer. Significantly, it brings the power and advantages of modern Cloud computing to streaming data for flight operations.

Potential non-NASA applications include scientific sensor applications, a DataTurbine compatible enhancement for scientific researchers at http://dataturbine.org, smartphone photo/video sharing, and a new platform for the rapidly growing "Internet of Things" market to monitor home appliance, utility and energy use.

Lead Organization: Cycronix