The NASA Earth Observing System Data and Information System (EOSDIS) is a data-centric system designed for processing and archiving from NASA’s Earth Observation missions and their distribution, as well as provision of specialized services to users. The major components of EOSDIS are 12 Distributed Active Archive Centers (DAACs), 14 Science Investigator-led Processing Systems (SIPSs), and the EOS Clearing House (ECHO). The Global Imagery Browse Services (GIBS) is the value-added image archive and access service for NASA Earth Science data products. It is the high-resolution global imagery solution for NASA. As the central imagery hub, GIBS is in need of an automated, scalable imagery capturing and generation solution.
The Imagery Exchange (TIE) delivers horizontal scaling solutions for image capturing to automate the generation of the Meta Raster Format (MRF) imagery products. TIE is built on the mature HORIZON framework described in the preceding article. It also leverages HORIZON’s event publication solution, the Significant Event Web service, and its Security service that integrates with Goddard Space Flight Center’s LDAP (Lunar Data Analysis Program) server.
Components of TIE include:
- Data Handlers: A growing collection of Data Handlers has been developed to support various delivery interfaces, which include FTP, HTTP/HTML, and HTTP/WMS, with various security schemes and data product formats. According to the Interface Control agreement with the provider, the handlers transform interface policies into application programs. They are responsible for capturing and staging the imagery data, and harvesting metadata for inventory.
- Inventory Service: The Inventory Service implements the HORIZON Inventory API and the GIBS-specific data schema. The schema captures imagery product information and traceability to the source observational data, which is managed at the data centers and discoverable at the Common Metadata Repository (CMR)/EOS Clearing Hours (ECHO).
- Data Subscribers: Data Subscribers automate the detection of newly ingested imagery products available in the TIE inventory. It implements the imagery-specific configuration for product generation. It is responsible for creating product generation work packages, and queues them onto the Apache ZooKeeper. Like the HORIZON’s Ingest Engine, the Product Generation service uses ZooKeeper as the facade for horizontal scaling, separating the job creators from the job performers.
- Product Generation: The Product Generation service is a client of the Apache ZooKeeper. It queries ZK for new work. The generator is responsible for updating job status on the ZooKeeper. The generator invokes the MRF generator to transform the global imagery product into a tiled, pyramid imagery product that is made accessible to the public via another GIBS subsystem called OnEarth.
- Product Type Management Tool: The PTMT is a product curation tool. It is a Web application for a provider to introduce new product types/imagery layers to the GIBS system. It provides a form-based interface with constraint checkers that mirrors the Inventory policies. The PTMT interfaces with the Inventory Service. PTMT is also the tool for making updates/corrections to product type metadata. It is also the tool to define the relationship between imagery products and the observational datasets, where the metadata is hosted at the CMR/ECHO.
This work was done by Thomas Huang, Christian Alarcon, Cristina M. De Cesare, and Nathan C. Tung of Caltech for NASA’s Jet Propulsion Laboratory. This software is available for license through the Jet Propulsion Laboratory, and you may request a license at: https://download.jpl.nasa.gov/ops/request/request_introduction.cfm . NPO-49883