Systems and Services for Near-Real-Time Web Access to NPP Data
- Created: Saturday, 31 May 2014
Software for processing and interpreting S-NPP observations and related data has become publicly available and more readily usable.
Marshall Space Flight Center, Alabama
The recently launched Suomi National Polar-orbiting Partnership (SNPP) satellite, operated by NASA and the National Oceanic and Atmospheric Administration(NOAA), is providing multispectral global observations over the next several years to support a broad array of research and applications. SNPP data products consist of a complex set of data and metadata files in highly specialized formats, and the U.S. government’s operational ground segment delivers these to users with delays of several hours to a few days.
A growing set of antennas around the globe is capable of receiving S-NPP’s continuous, unencrypted data broadcast, and sharing the raw data via the Internet. Furthermore, software for processing and interpreting S-NPP observations and related data has become publicly available and more readily usable.
A suite of software was developed to couple near-real-time S-NPP data feeds with a streamlined, scalable processing chain and geospatial Web services, running in a scalable cloud computing environment. The system provides near-real-time access to data from the Visible Infrared Imaging Radiometer Suite (VIIRS), the Moderate Resolu tion Imaging Spectroradiometer (MODIS), and other observations. It functions 24/7 to retrieve these data from multiple sources over the Internet, processes them as quickly as possible using a scalable cloud computing environment, and delivers data products and visualizations on demand via standard Web services that can interoperate with a variety of end-user display and analysis tools.
The first component of the system (a Retrieval server) monitors data repositories at several remote S-NPP and Aqua MODIS receiving sites and fetches new satellite data files as soon as they become available. The second component (one or more Processing servers) runs science-validated algorithms on the newly fetched data, and prepares data products and visualizations for Web presentation to end users. The third component (a Map/Data server) draws on these data and visual products, using industry-standard protocols to serve a variety of software clients. Each component is distributed as an integrated machine image, ready to be instantiated on virtual machines in a cloud computing environment. As virtual machines, the three components can be reconfigured easily to monitor different data sources, and to perform different processing workflows on the retrieved data.
One advantage is the use of cloud computing for low-cost entry into full-scale scientific computing and radical scalability to meet wide variations in processing demand at moderate costs. Another is the use of virtual machines capable of running many different workflows, easily reconfigured to fetch, process, and deliver a variety of products from distributed sources.