With the goal of assessing the anthropogenic carbon-emission impact of urban centers on local and global climates, the Megacities Carbon Project has been building carbon-monitoring capabilities for the past two years around the Los Angeles metropolitan area as a pilot effort. Hundreds of megabytes of data are generated daily and distributed among data centers local to the sensor networks involved. These remotely generated data are then aggregated into a centralized data infrastructure located at the Jet Propulsion Laboratory (JPL) to provide collaboration opportunities on the data as well as generate refined data products through centralized data processing pipelines.
The goal of this informatics effort is to ensure near-real-time access to data products generated across the Los Angeles carbon monitoring sensor network, including data generated by at least a dozen ground-based sensors, satellite assets, and aircraft flyovers. These data are then made available to carbon researchers in formats and aggregation levels that seek to meet their data analysis needs.
In support of these goals, a reliable data aggregator was developed that can take near-real-time data collected from instrumentation, and process and present it in a consistent manner. The software syncs hourly and nightly data from heterogeneous sensing-network data sources, including Amazon Simple Storage Service (S3), Earth Networks servers, webcams, and various other data providers. It also provides data processing capabilities, integrating externally written scientific algorithms that transform collected data. This near-real-time data informatics infrastructure motivates a more up-to-date picture of atmospheric carbon above the Los Angeles metropolitan area and may one day aid disciplines as diverse as urban and regional policy making.
This work was done by Rishi Verma and Daniel J. Crichton of Caltech for NASA’s Jet Propulsion Laboratory. This software is available for license through the Jet Propulsion Laboratory, and you may request a license at: https://download.jpl.nasa.gov/ops/request/request_introduction.cfm . NPO-49614