IN33B-1169
Fusing precipitation for NOAA's AWIPS DSS through a hydro-information system
Accurate precipitation estimation is essential to hydrologic modeling for flood and drought forecasts. With advancements in technology, precipitation can now be measured by a range of sensors, including the NOAA/National Weather Service NEXRAD radar network, satellites, and rain gauges. Each measurement platform and the data product(s) associated with it have their own strengths and weaknesses. There are different precipitation products derived from different data sources and from combinations of them. These data products vary in their spatial and temporal resolutions. In this study, we illustrate the integration of our MKF-based (Multiscale Kalman Filter) framework with our hydro-information system to fuse Stage III/Multi- sensor Precipitation Estimator (MPE) hourly NEXRAD precipitation data at approximately 4 by 4 square kilometer resolution with the precipitation data from LDAS (Land Data Assimilation Systems) at 1/8 degree resolution. Two data products from LDAS are investigated. One is the EDAS (NCEP's Eta-based 4-D Data Assimilation System) precipitation product, and the other is the combo precipitation product which is derived from the ?degree CPC (Climate Precipitation Center) daily precipitation data from rain gauges. The combo product is interpolated to 1/8 degree resolution based on the budget bilinear interpolation method. The daily time step of the combo product is disaggregated into hourly data based on either the weight of the hourly Stage II NEXRAD radar or EDAS hourly precipitation or uniformly, if there is no information from either Stage II NEXRAD radar or EDAS hourly precipitation. Our hydro-information system facilitates heterogeneous data retrieval from different data sources into the MKF-based data fusion framework, and then to the hydrological modeling system through an extension of the Hydrological Integrated Data Environment (HIDE) system. Initial results show significant differences in spatial coverage and magnitudes between the original LDAS precipitation data products and the fused precipitation at 1/8 degree resolution. Impacts of the fused precipitation on hydrologic simulations and on flood and drought forecasts will be investigated.
IN33B-1170
Issues in Data Fusion for Use in an Interactive Online Analysis System using MODIS Terra and Aqua Daily Aerosol Data
Data Fusion defined here as a consisting of merging and interpolation is a method of combining spatio- temporally near-coincident satellite observations to provide complete global or regional maps of geophysical variables for comparison with transport models and ground station observations. We investigate various methods, challenges and limitations of data fusion, with and without interpolation, as a first step towards merging datasets archived in the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) and made public through the Goddard Interactive Online Visualization and Analysis Infrastructure (Giovanni) data portals. As a prototype for the data fusion algorithm, this study uses daily global observations of Aerosol Optical Thickness (AOT), as measured by the MODerate resolution Imaging Spectroradiometer (MODIS) onboard the Terra and Aqua satellites. The goal is to develop a very fast online method for data fusion for implementation into Giovanni. We demonstrate three different methods for fusion (without interpolation): Simple Arithmetic Averaging (SIM), Maximum Likelihood Estimate (MLE) and Weighting by Pixel Counts (WPC). All three methods are roughly comparable, with the MLE (SIM) being slightly preferable when validating with respect to the AOT standard deviations (AOT means). To evaluate the fused product, we introduce two confidence functions, which characterize the percentage of the fused AOT pixels as a function of the relative deviation of the fused AOT from the initial Terra and Aqua AOTs. Gaps in the daily global maps of AOT's arise from regions in sun glint, clouds, gaps between orbit tracks at low latitudes, and other sources of missing data. Data fusion with spatial interpolation produces spatially contiguous fields (global and regional maps) for dust event tracking and comparison with and input to 3-D global and regional models. Eight combinations of merger-interpolation are applied to scenes with regular and irregular data gap patterns. The Cumulative SemiVariogram (CSV) was found to be sensitive to the spatial distribution and fraction of gap areas and, thus, useful for assessing the sensitivity and radius of influence of the merged data to gap patterns. Our results show that the merging-interpolation procedure can produce complete spatial fields with acceptable errors. In this work we also look at some of the challenges involved in data fusion as described above which include the treatment of biases in the individual measurements with respect to a validation standard and assumptions made about the spatial and temporal distribution of the parameter.
IN33B-1171
Using Geostatistical Inverse Modeling to Address Uncertainty in Sensor Data Fusion
One type of remote sensing data fusion problem is the estimation of ground surface properties based on observations from multiple sensors and/or platforms. It is important to note that because the problem involves representing a continuous variable (a ground surface property) from discrete data (sensor observations), there is inherent uncertainty in the estimation problem. In addition, remote sensing analyses are based on input data with errors due to sensor noise and involve physical sensor system models that have error characteristics. To effectively characterize and model these errors and uncertainties, a probabilistic framework approach is needed. Geostatistical Inverse Modeling (GIM) is one such probabilistic framework that can be used to explicitly model uncertainty for making estimates based on multiple sensor data sources. Geostatistical methods explicitly model error covariances as a function of spatial separation, and provide probabilistic estimates (both best estimation and estimation covariance). GIM can incorporate physical models of the sensor and sensing geometry as well as auxiliary data (such as known boundaries) to reduce the estimation uncertainty.
IN33B-1172
Merging the Data Models of NetCDF and DAP: Design Choices and Benefits
Beginning in 2008 OPeNDAP and Unidata have been working on an ambitious project to merge the functionality of two different implementations of the netCDF API into a single body of code. Unidata's implementation reads and writes to disk files while OPeNDAP's reads from data servers that support its Data Access Protocol (DAP). The reasons for combining the two are principally to reduce maintenance costs and delays for the introduction of new features, but a side affect has been to focus both groups on the issues of data model flexibility and simplicity. The netCDF format/API have been used in a wide range of contexts spanning the gamut of earth science disciplines including meteorology, oceanography, et c., as well as GIS applications. The Data Access Protocol has seen a similar breadth of use. Both of these software systems employ general data structuring technology based on well-understood information science principals such as data typing and grouping. However, the actual data models of DAP 2.0 and netCDF 3 are different in some significant ways. In merging the two both will require significant changes. We will discuss the on-going process of deciding which changes should be made, where they should be made and how to implement them without breaking software that uses the existing software and data models. In addition we will discuss the exciting prospects that combining these two libraries will provide, particularly how the combination of hierarchical, relational and array data types can facilitate data fusion.
IN33B-1173
The OPeNDAP and Remote NetCDF Invocation (RNI) middleware platform for Scientific Data Fusion.
Across geosciences, there are large data holdings being made available via the DAP protocol by means of OPeNDAP software. A lot of the underlying data are in the NetCDF format. Often, each individual dataset is a combination of hundreds of individual NetCDF files. Requesting such datasets for analysis is an expensive data fusion transaction, especially as the number and size of datasets increase. We present a set of solutions that instead request needed portions of the dataset fused just-in-time. The fusion includes both subsetting and agreggation operations as well as analysis and data manipulation steps. We have modified the NetCDF C library for Remote NetCDF Invocation (RNI), that is, to operate on remote dataset, over HTTP, HTTPS or gsiFTP (or any) protocols, individual NetCDF Application Programming Interface (API) calls as if they were local. This invocation model also can be applied to OPeNDAP data streams and local files. This mechanism resembles the well known Remote Procedure Call (RPC) yet it radically differs on the binding between local and remote operations. We describe our current approach, implementation and benefits obtained from this approach and indicate how it aids data fusion.
IN33B-1174
Automated Historical and Real-Time Cyclone Discovery With Multimodal Remote Satellite Measurements
Existing cyclone detection and tracking solutions involve extensive manual analysis of modeled-data and field campaign data by teams of experts. We have developed a novel automated global cyclone detection and tracking system by assimilating and sharing information from multiple remote satellites. This unprecedented solution of combining multiple remote satellite measurements in an autonomous manner allows leveraging off the strengths of each individual satellite. Use of multiple satellite data sources also results in significantly improved temporal tracking accuracy for cyclones. Our solution involves an automated feature extraction and machine learning technique based on an ensemble classifier and Kalman filter for cyclone detection and tracking from multiple heterogeneous satellite data sources. Our feature-based methodology that focuses on automated cyclone discovery is fundamentally different from, and actually complements, the well-known Dvorak technique for cyclone intensity estimation (that often relies on manual detection of cyclonic regions) from field and remote data. Our solution currently employs the QuikSCAT wind measurement and the merged level 3 TRMM precipitation data for automated cyclone discovery. Assimilation of other types of remote measurements is ongoing and planned in the near future. Experimental results of our automated solution on historical cyclone datasets demonstrate the superior performance of our automated approach compared to previous work. Performance of our detection solution compares favorably against the list of cyclones occurring in North Atlantic Ocean for the 2005 calendar year reported by the National Hurricane Center (NHC) in our initial analysis. We have also demonstrated the robustness of our cyclone tracking methodology in other regions over the world by using multiple heterogeneous satellite data for detection and tracking of three arbitrary historical cyclones in other regions. Our cyclone detection and tracking methodology can be applied to (i) historical data to support Earth scientists in climate modeling, cyclonic-climate interactions, and obtain a better understanding of the cause and effects of cyclone (e.g. cyclo-genesis), and (ii) automatic cyclone discovery in near real-time using streaming satellite to support and improve the planning of global cyclone field campaigns. Additional satellite data from GOES and other orbiting satellites can be easily assimilated and integrated into our automated cyclone detection and tracking module to improve the temporal tracking accuracy of cyclones down to ½ hr and reduce the incidence of false alarms.
IN33B-1175
Moving Beyond the Fourth Dimension of Environmental Data Fusion, Analysis and Visualization: Case Studies in Eonfusion
A significant challenge facing researchers is the bridging of domains between the natural and physical
sciences through the integration, analysis and visualization of multivariate environmental data from a variety
of sources. Typically the temporal, visual and analytical components are not well handled within a single
software application or by existing tools, and the final products are often difficult to share with collaborating
researchers and managers. To address these issues, Myriax has launched Eonfusion; the next generation of
geospatial analysis software which provides a state-of-the-art 4D-analysis environment that is intuitive and
readily extensible. The software significantly enhances the ease with which scientists can integrate diverse
data types (raster, vector and media) and share methods across disciplines. The powerful 4D-visualization
interface also incorporates a range of analysis and plotting tools as well as a time-stepping feature that allow
users to explore the temporal evolution of spatial structure and resolved relationships over time. Its
revolutionary fusion operator allows data at different temporal or spatial scales to be reconciled, attributes
merged and topological relationships identified. The software is customizable, possessing an integrated
coding environment (Expression Evaluator) for algorithm implementation and an API facilitating the
development of modules. While powerful, Eonfusion is easy to learn and has an intuitive, attractive user
interface. It is ideal for both undertaking complex analyses and communicating syntheses. In this paper we
present real-world examples of oceanographic and fisheries data which are integrated and visualized in
Eonfusion to reveal relationships about marine species and their environment so as to inform management
decisions.
http://eonfusion.myriax.com