The Science Processing and Data Management (SPDM) system’s primary functions are to control and monitor science processes and data products for Soil Moisture Active Passive (SMAP) Science Data System (SDS). SPDM adapts the Apache/OODT Framework 0.3 as the basic infrastructure. Although the OODT core framework is software-extensible, it provides limited, minimal, and rough functionalities. The adaptation effort has been extensive to fulfill SMAP SDS requirements including the integration of 26 processors with complex interactions among processors and files (≈70 static ancillary data types and ≈30 external dynamic ancillary types), pipeline automation to generate multiple products (LO-3) with large data volumes within a few hours to a few days of radar and radiometer instrument acquisition, monitoring and control of the data system, and interfacing with two NASA Data Active Archive Centers (DAACs).
The SPDM system has adapted the open source Apache/OODT 0.3. The key components selected for adaptation include File Manager, Workflow Manager, Resource Manager, Crawler, PGE Task Wrapper, and Balance Framework. These OODT components were re-architected, re-designed, customized, and extended to meet the SMAP SDS requirements.
SPDM Core contains a set of tools to provide the operation interface to the control, monitor, search, and report of SPDM services, processing, resources, and products. SPDM SIPS Interface reports the science data products’ readiness for publishing and distribution automatically.
SPDM Web Services is responsible for providing the search results to the SPDM Web Portal for tracking, monitoring, and reporting purposes. SPDM Web Portal Application provides the on-line Web monitoring of SPDM services, processing, resources, and products.