IN31A-1119
A Distributed Event Detection System for the Solar Dynamics Observatory
The Solar Dynamics Observatory (SDO) instruments such as the
Atmospheric Imaging Assembly (AIA), in addition to other upcoming
missions, will provide a unique data processing challenge to the
community. Vast
amounts of data need to be processed in a largely automated, efficient
manner. We present the implementation of the AIA Event Detection System.
This software autonomously configures subscriptions between related
event detection algorithms submitted by collaborators, allowing
adjusting of algorithm parameters in response to events, and will run
them as it receives new data from AIA. Results will automatically be
published to the Helioinformatics Event Knowledgebase (HEK) with
provenance information. The system uses distributed processing and has
abilities to automatically reconfigure in case of node failures or
overloads. We demonstrate some of these abilities through feature
extraction algorithms for sunspot detection run on past SOHO MDI data.
http://helio-informatics.org
IN31A-1120
Improving the Forecast Accuracy of an Ocean Observation and Prediction System by Adaptive Control of the Sensor Network
The New York Harbor Observation and Prediction System (NYHOPS) is a real-time, estuarine and coastal ocean observing and modeling system for the New York Harbor and surrounding waters. Real-time measurements from in-situ mobile and stationary sensors in the NYHOPS networks are assimilated into marine forecasts in order to reduce the discrepancy with ground truth. The forecasts are obtained from the ECOMSED hydrodynamic model, a shallow water derivative of the Princeton Ocean Model. Currently, all sensors in the NYHOPS system are operated in a fixed mode with uniform sampling rates. This technology infusion effort demonstrates the use of Model Predictive Control (MPC) to autonomously adapt the operation of both mobile and stationary sensors in response to changing events that are -automatically detected from the ECOMSED forecasts. The controller focuses sensing resources on those regions that are expected to be impacted by the detected events. The MPC approach involves formulating the problem of calculating the optimal sensor parameters as a constrained multi-objective optimization problem. We have developed an objective function that takes into account the spatiotemporal relationship of the in-situ sensor locations and the locations of events detected by the model. Experiments in simulation were carried out using data collected during a freshwater flooding event. The location of the resulting freshwater plume was calculated from the corresponding model forecasts and was used by the MPC controller to derive control parameters for the sensing assets. The operational parameters that are controlled include the sampling rates of stationary sensors, paths of unmanned underwater vehicles (UUVs), and data transfer routes between sensors and the central modeling computer. The simulation experiments show that MPC-based sensor control reduces the RMS error in the forecast by a factor of 380% as compared to uniform sampling. The paths of multiple UUVs were simultaneously calculated such that measurements from on-board sensors would lead to maximal reduction in the forecast error after data assimilation. The MPC controller also reduces the consumption of system resources such as energy expended in sampling and wireless communication. The MPC-based control approach can be generalized to accept data from remote sensing satellites. This will enable in-situ sensors to be regulated using forecasts generated by assimilating local high resolution in-situ measurements with wide-area observations from remote sensing satellites.
IN31A-1121
A Framework for Accurate Geospatial Modeling of Recharge and Discharge Maps using Image Ranking and Machine Learning
This paper addresses the problem of accurate estimation of geospatial models from a set of groundwater
recharge & discharge (R&D) maps and from auxiliary remote sensing and terrestrial raster measurements.
The motivation for our work is driven by the cost of field measurements, and by the limitations of currently
available physics-based modeling techniques that do not include all relevant variables and allow accurate
predictions only at coarse spatial scales. The goal is to improve our understanding of the underlying physical
phenomena and increase the accuracy of geospatial models--with a combination of remote sensing, field
measurements and physics-based modeling. Our approach is to process a set of R&D maps generated from
interpolated sparse field measurements using existing physics-based models, and identify the R&D map that
would be the most suitable for extracting a set of rules between the auxiliary variables of interest and the
R&D map labels. We implemented this approach by ranking R&D maps using information entropy and
mutual information criteria, and then by deriving a set of rules using a machine learning technique, such as
the decision tree method. The novelty of our work is in developing a general framework for building
geospatial models with the ultimate goal of minimizing cost and maximizing model accuracy. The framework is
demonstrated for groundwater R&D rate models but could be applied to other similar studies, for instance, to
understanding hypoxia based on physics-based models and remotely sensed variables. Furthermore, our
key contribution is in designing a ranking method for R&D maps that allows us to analyze multiple plausible
R&D maps with a different number of zones which was not possible in our earlier prototype of the framework
called Spatial Pattern to Learn. We will present experimental results using examples R&D and other maps
from an area in Wisconsin.
http://isda.ncsa.uiuc.edu/groundwater
IN31A-1122
Autonomous Mission Design and Data Fusion: Laying the groundwork for Decadal Mission swath altimetry and ocean vector winds.
In the coming decade, the autonomous coordinated utilization of space, atmospheric, surface, and ocean assets, sensor webs, and data will assume more importance, as systems become more complex and tightly integrated, and as the need to know our environment with ever greater accuracy and precision becomes more acute. We have begun to address this issue with a prototype virtual ocean observatory that includes present and future NASA satellite missions (Jason-2 and QuikSCAT; and SWOT [swath altimetry] and XOVWM [ocean vector winds], respectively); atmosphere and ocean models (WRF/LAPS and ROMS, respectively); and in-situ sensors and platforms (underwater gliders). In our prototype system, the goal is to develop the architecture and implementation of the necessary software modules (e.g., automated data fusion/assimilation, and automated planning technology) to achieve adaptive in-situ sampling through feedback from space-based-assets (in this case via the SWOT simulator) thereby contributing to the orbit design during the first, experimental phase (~6-9 months) of the SWOT mission. This work is one step in the process of infusing technology into the development pipeline.
IN31A-1123
Regional Interagency Disaster Response Collaboration
In affiliation with the "Great Worden Quake II" (GWQII) disaster preparedness exercise, the NASA Ames Research Center, Moffett Field California, the Air Force National Guard (ANG) 129th Rescue Wing, Moffett Field, California, and the Bay Area Automated Mapping Association,led by the IT group for the City of Walnut Creek, California, will engage in a technology transfer demonstration utilizing the collaborative environment developed for NASA's very successful wildfire mapping campaigns during the years 2006-2008. The aircraft platform will be the ANG C-130, a viable candidate to substitute for the Ikana UAV, which cannot fly from Ames because of FAA restrictions on UAV flights over populated areas. In this technology transfer demonstration, we will: (1) Prove, document and train Regional Fire departments how to link and use NASA real-time data with existing software (ESRI, IRRIS, etc). (2) Demonstrate how to access and use this data as a bridge between the real-time (3) Refine the questions and capabilities that would be involved and developed with this type of real-time data available This paper describes this exercise.
IN31A-1124
A Prototype Land Information Sensor Web: Design, Implementation and Synthetic Experiments
To meet future earth system science challenges, NASA will develop constellations of smart satellites in sensor web configurations that provide timely on-demand data and analysis to users, and that be reconfigured based on the changing needs of science and available technology. A sensor web is more than a collection of satellite sensors. According to a most recent definition by NASA/AIST 2007 Sensor Web investigator meeting, "A Sensor Web is a coherent set of heterogeneous, loosely-coupled, distributed nodes, interconnected by a communications fabric that can collectively behave as a single dynamically adaptive and reconfigurable observing system". That means, a sensor web is a system composed of multiple platforms interconnected by a communication network for the purpose of performing specific observations and processing data required to support specific science goals. Sensor webs can eclipse the value of disparate sensor components by reducing response time and increasing scientific value, especially when integrated with science analysis, data assimilation, prediction modeling and decision support tools. The study of a prototype Land Information Sensor Web (LISW) is sponsored by NASA, trying to integrate the Land Information System (LIS) in a sensor web framework which allows for optimal 2-way information flow that enhances land surface modeling using sensor web observations, and in turn allows sensor web reconfiguration to minimize overall system uncertainty. This prototype is based on a simulated interactive sensor web, which is then used to exercise and optimize the sensor web - modeling interfaces. These synthetic experiments provide a controlled environment in which to examine the end-to-end performance of the prototype, the impact of various design sensor web design trade-offs and the eventual value of sensor webs for particular prediction or decision support. The Study of virtual Land Information Sensor Web (LISW) is expected to provide some necessary priori knowledge for designing and deploying the next generation Global Earth Observing System of systems (GEOSS). In this paper, the design, implementation and synthetic experiments, which were achieved from the LISW study, will be presented.
IN31A-1125
Cloud Computing Infusion for Generating ESDRs of Visible Spectra Radiances
The AIRS and AVHRR instruments have been collecting radiances of the Earth in the visible spectrum for over 25 years. These measurements have been used to develop such useful products as NDVI, Snow cover and depth, Outgoing long wave radiation and other products. Yet, no long-term data record of the level 1b visible spectra is available in a grid form to researchers for various climate studies. We present here an Earth System Data Record observed in the visible spectrum as gridded radiance fields of 8kmx10km grid resolution for the six years in the case of AIRS and from 1981 to the present for AVHRR. The AIRS data has four visible channels from 0.41μm to 0.94μm with an IFOV of 1 km and AVHRR has two visible channels in the 0.58μm to 1.00μm range also at 1 km. In order to process such large amounts of data on demand, two components need to be implemented,(i) a processing system capable of gridding TBs of data in a reasonable amount of time and (ii) a download mechanism to access and deliver the data to the processing system. We implemented a cloud computing approach to be able to process such large amounts of data. We use Hadoop, a distributed computation system developed by the Apache Software Foundation. With Hadoop, we are able to store the data in a distributed fashion, taking advantage of Hadoop's distributed file system (dfs). We also take advantage of Hadoop's MapReduce functionality to perform as much computations as is possible on available nodes of the UMBC bluegrit Cell cluster system that contain the data. We make use of the SOAR system developed under the ACCESS program to acquire and process the AIRS and AVHRR observations. Comparisons of the AIRS data witth selected periods of MODIS visible spectral channels on the same sattelite indicate the two instruments have maintained calibration consistency and continuity of their measurements over the six year period. Our download mechanism transfers the data from these instruments into hadoop's dfs. Our MapReduce implementation grids the data in a distributed fashion by returning each radiance as a key/value pair with the value being the radiance measurement and the key being a unique ID that corresponds to a particular grid box. The reducing function then performs averaging for each of the grid boxes, returning the final average radiance for each grid box. We implemented a generalized downloading mechanism in the SOAR system that is capable of intelligently streaming of data for these three instruments with a single interface. The downloader is capable of error checking and user defined operations when downloading data. The download mechanism takes advantage of multithreading to download the next dataset, while currently downloaded datasets are being processed. Additionally, the download mechanism allows for easy expansion to enable datasets for other instruments to be downloaded.
IN31A-1126
Making GOES Data Available to the GEOSS Community through Technology Infusion
The National Oceanic and Atmospheric Administration (NOAA) is operating two types of operational weather satellite systems: geostationary operational environmental satellites (GOES) and polar-orbiting satellites. GOES satellites provide continuous monitoring of Earth's atmosphere necessary for short-range warning and "now-casting" of weather conditions. The GOES imager acquires a multi-spectral image of a full-disc view of the Earth every thirty minutes with spatial resolution of 1 to 8 km at nadir. Because GOES satellites stay above a fixed spot on the surface, they provide a constant vigil for monitoring severe weather conditions such as tornadoes, thunderstorms, and hurricanes. Currently the GOES images are archived in and made available to publics through searching and ordering interfaces of NOAA's Comprehensive Large Array-data Stewardship System (CLASS). On a broader scale, the Group on Earth Observations (GEO) is leading the world-wide efforts on building the Earth Observation System of Systems (GEOSS) for making Earth observation data available to the world-wide user communities for applications in nine areas of societal benefits. GEOSS uses the service oriented architecture (SOA) and adopts specifications developed mainly by the Open Geospatial Consortium (OGC) for the interoperability arrangement. For the data discovery and access, the main specifications adopted by GEOSS are the OGC Catalog Service for Web (CS/W) for the data discovery and Web Coverage Service (WCS) for data access. Currently, CLASS has no WCS and CSW interfaces available. In the past several years with NASA funding support, we have developed OGC WCS and CSW service technology for making remote sensing data discoverable and accessible through such interfaces. During the GEOSS Architecture Implementation Pilot Phase II, we are developing a prototype that could infuse the technology into the CLASS system via the Simple NOAA Archive Access Protocol (SNAAP) to make GOES data accessible through WCS and CSW interfaces for supporting weather and related applications in the GEOSS communities. This infusion enables the persistence data services of current GOES data for serving the GEOSS user communities. These services will provide the proof of concept for contributing data from the future GOES-R satellite series to GEOSS in the future.
IN31A-1127
GeoBrainArc: Enabling Geospatial Interoperability in ArcGIS
In recent years, a growing number of geospatial Web services designed to deal with distributed geospatial information over network have emerged as the maturation of web service technologies. The Open Geospatial Consortium (OGC) has published a series of specifications that address geospatial interoperability requirement, standards and implementations to enhance the discovery, retrieval and handling of geospatial information and geospatial processing service. More and more government agencies, such as NASA, FGDC and EPA, publish their data using OGC protocols. ESRI is a leading global GIS software provider, and its flagship product ArcGIS Desktop has significant market share in commercial desktop solutions. To provide worldwide ArcGIS users an interoperable way of accessing OGC Web services for integrating and analyzing distributed heterogeneous geospatial data, we design and implement an extension of ArcGIS: GeoBrainArc. It can be easily installed as a component tool of ArcGIS. With the GeoBrainArc, ArcGIS users now is able to dynamically discover data and services over network using OGC Catalo Service for Web (CS/W), interactively access to and display remote sensing data from distributed OGC Web Coverage Service (WCS) and OGC Web Map Service (WMS), and visualize and analyze vector data from different OGC Web Feature Service (WFS). Thus, all those data from OGC Web services, just like other local data, is seamlessly integrated into the ArcGIS environment regardless of their locations, formats and projections.
IN31A-1128
DISCOVER: A Service Oriented Approach to Managing Earth Science Data Across Distributed Project-specific Repositories
The Global Hydrology Resource Center (GHRC), a NASA Earth Science data center managed by the University of Alabama in Huntsville, is one of twelve data centers that make up the Distributed Active Archive Centers (DAAC) Alliance. Over the years, GHRC staff have developed and evolved a production information management infrastructure to ingest, inventory, archive and distribute a variety of data products to our users. The GHRC has also collaborated with Remote Sensing Systems (RSS) over the course of three NASA Earth Science programs (ESIP, REaSON, and now MEaSURES) to develop valuable Earth science products and services, specifically for passive microwave sensors. This continued effort, known as the DISCOVER (Distributed Information Services for Climate and Ocean products and Visualizations for Earth Research) project, has been able to explore more experimental data services. A result of this collaboration is that services developed and hardened in the DISCOVER service oriented architecture may be integrated into the baseline GHRC infrastructure. For example, the GHRC Data Pool was originally developed for DISCOVER and is now supporting the inventory, search and distribution of science data products across multiple GHRC and DISCOVER data repositories. Distributed services for harvesting metadata and packaging data orders interoperate with two complementary search/access/order user interfaces through a central metadata and order tracking database. This presentation will discuss the science data tools and services developed by DISCOVER and the GHRC, with a focus on integration of new services into an established data management infrastructure.
IN31A-1129
Earth Science Mining Web Services
To allow scientists further capabilities in the area of data mining and web services, the Goddard Earth
Sciences Data and Information Services Center (GES DISC) and researchers at the University of Alabama in
Huntsville (UAH) have developed a system to mine data at the source without the need of network transfers.
The system has been constructed by linking together several pre-existing technologies: the Simple Scalable
Script-based Science Processor for Measurements (S4PM), a processing engine at the GES DISC; the
Algorithm Development and Mining (ADaM) system, a data mining toolkit from UAH that can be configured in
a variety of ways to create customized mining processes; ActiveBPEL, a workflow execution engine based on
BPEL (Business Process Execution Language); XBaya, a graphical workflow composer; and the EOS
Clearinghouse (ECHO). XBaya is used to construct an analysis workflow at UAH using ADaM components,
which are also installed remotely at the GES DISC, wrapped as Web Services. The S4PM processing engine
searches ECHO for data using space-time criteria, staging them to cache, allowing the ActiveBPEL engine to
remotely orchestrates the processing workflow within S4PM. As mining is completed, the output is placed in
an FTP holding area for the end user. The goals are to give users control over the data they want to
process, while mining data at the data source using the server's resources rather than transferring the full
volume over the internet.
These diverse technologies have been infused into a functioning, distributed system with only minor changes
to the underlying technologies. The key to this infusion is the loosely coupled, Web- Services based
architecture: All of the participating components are accessible (one way or another) through (Simple Object
Access Protocol) SOAP-based Web Services.
http://disc.gsfc.nasa.gov
IN31A-1130
Experiences with Transitioning Science Data Production from a Symmetric Multiprocessor Platform to a Linux Cluster Environment
NASA's Atmospheric Science Data Center at the NASA Langley Research Center performs all of the science data processing for the Multi-angle Imaging SpectroRadiometer (MISR) instrument. MISR is one of the five remote sensing instruments flying aboard NASA's Terra spacecraft. From the time of Terra launch in December 1999 until February 2008, all MISR science data processing was performed on a Silicon Graphics, Inc. (SGI) platform. However, dramatic improvements in commodity computing technology coupled with steadily declining project budgets during that period eventually made transitioning MISR processing to a commodity computing environment both feasible and necessary. The Atmospheric Science Data Center has successfully ported the MISR science data processing environment from the SGI platform to a Linux cluster environment. There were a multitude of technical challenges associated with this transition. Even though the core architecture of the production system did not change, the manner in which it interacted with underlying hardware was fundamentally different. In addition, there are more potential throughput bottlenecks in a cluster environment than there are in a symmetric multiprocessor environment like the SGI platform and each of these had to be addressed. Once all the technical issues associated with the transition were resolved, the Atmospheric Science Data Center had a MISR science data processing system with significantly higher throughput than the SGI platform at a fraction of the cost. In addition to the commodity hardware, free and open source software such as S4PM, Sun Grid Engine, PostgreSQL and Ganglia play a significant role in the new system. Details of the technical challenges and resolutions, software systems, performance improvements, and cost savings associated with the transition will be discussed. The Atmospheric Science Data Center in Langley's Science Directorate leads NASA's program for the processing, archival and distribution of Earth science data in the areas of radiation budget, clouds, aerosols, and tropospheric chemistry. The Data Center was established in 1991 to support NASA's Earth Observing System and the U.S. Global Change Research Program. It is unique among NASA data centers in the size of its archive, cutting edge computing technology, and full range of data services. For more information regarding ASDC data holdings, documentation, tools and services, visit http://eosweb.larc.nasa.gov
IN31A-1131
The Infusion of Dust Model Model Outputs into Public Health Decision Making - an Examination of Differential Adoption of SOAP and Open Geospatial Consortium Service Products into Public Health Decision Support Systems
Since 2004 the Earth Data Analysis Center, in collaboration with the researchers at the University of Arizona and George Mason University, with funding from NASA, has been developing a services oriented architecture (SOA) that acquires remote sensing, meteorological forecast, and observed ground level particulate data (EPA AirNow) from NASA, NOAA, and DataFed through a variety of standards-based service interfaces. These acquired data are used to initialize and set boundary conditions for the execution of the Dust Regional Atmospheric Model (DREAM) to generate daily 48-hour dust forecasts, which are then published via a combination of Open Geospatial Consortium (OGC) services (WMS and WCS), basic HTTP request-based services, and SOAP services. The goal of this work has been to develop services that can be integrated into existing public health decision support systems (DSS) to provide enhanced environmental data (i.e. ground surface particulate concentration estimates) for use in epidemiological analysis, public health warning systems, and syndromic surveillance systems. While the project has succeeded in deploying these products into the target systems, there has been differential adoption of the different service interface products, with the simple OGC and HTTP interfaces generating much greater interest by DSS developers and researchers than the more complex SOAP service interfaces. This paper reviews the SOA developed as part of this project and provides insights into how different service models may have a significant impact on the infusion of Earth science products into decision making processes and systems.
IN31A-1132
A Service Oriented Infrastructure for Earth Science exchange
NASA's Earth Science Distributed Information System (ESDIS) program has developed an infrastructure for the exchange of Earth Observation related resources. Fundamentally a platform for Service Oriented Architectures, ECHO provides standards-based interfaces based on the basic interactions for a SOA pattern: Publish, Find and Bind. This infrastructure enables the realization of the benefits of Service Oriented Architectures, namely the reduction of stove-piped systems, the opportunity for reuse and flexibility to meet dynamic business needs, on a global scale. ECHO is the result of the infusion of IT technologies, including those standards of Web Services and Service Oriented Architecture technologies. The infrastructure is based on standards and leverages registries for data, services, clients and applications. As an operational system, ECHO currently representing over 110 million Earth Observation resources from a wide number of provider organizations. These partner organizations each have a primary mission – serving a particular facet of the Earth Observation community. Through ECHO, those partners can serve the needs of not only their target portion of the community, but also enable a wider range of users to discover and leverage their data resources, thereby increasing the value of their offerings. The Earth Observation community benefits from this infrastructure because it provides a set of common mechanisms for the discovery and access to resources from a much wider range of data and service providers. ECHO enables innovative clients to be built for targeted user types and missions. There several examples of those clients already in process. Applications built on this infrastructure can include User-driven, GUI-clients (web-based or thick clients), analysis programs (as intermediate components of larger systems), models or decision support systems. This paper will provide insight into the development of ECHO, as technologies were evaluated for infusion, and a summary of how technologies where leveraged into a significant operational system for the Earth Observation community.
IN31A-1133
Infusing semantic web into operational data systems: real application experiences.
As part of our semantic data framework activities across disciplines from solid-earth, lower, middle and upper terrestrial atmosphere, and solaratmosphere, to integrative subjects such as climate response and space weather, we have collected a set of experiences: technical, collaboration and social that relate to how easy, or hard the infusion process has been. We cover both the semantic web and knowledge infusion as well as underlying service infusion such as catalogs and OPeNDAP data servers.