SF23A-0024 1340h
A near real time system for producing and distributing EOS AQUA products to NWP centers
NOAA/NESDIS/ORA has designed, built, and currently operates a near real time data processing and distribution system for the EOS instruments (AIRS, AMSU, and HSB) aboard the AQUA satellite. AIRS, AMSU, and HSB level 0 orbital data packets are received and processed into level 1B (calibrated and navigated) six minute granules. These granules are both spectrally and spatially subset to produce files containing AIRS, AMSU, and HSB brightness temperatures, cloud-cleared radiances, principal components, and principal component reconstructions of radiances. AIRS level 2 temperature and water vapor profiles are generated from the level 1B cloud clear radiances. Data for AMSR-E and MODIS are received from NOAA/NESDIS/OSDPD where it has been processed up through level 1B. AMSR-E level 1B files are received as whole orbits in HDF-EOS format and are converted to BUFR. From the the level 1B, three separate AMSR-E level 2 products are generated for land, water, and atmosphere. MODIS level 1B 1 km observations are collocated to the AIRS footprints and averaged to produce clear and cloudy brightness temperatures, radiances, as well as level 2 products. All the AQUA data products discussed are output in BUFR and over eighty percent are distributed within three hours of observation to meet the operational requirements of NWP centers. Development of a near real time processing/distribution system for the IASI instrument (scheduled to launch on the MetOp 1 satellite in October 2005) is currently ongoing. The design of the EOS AQUA near real time system is providing a design template and useful experience for this and future projects.
SF23A-0025 1340h
AIRS Data Product Size Reduction
AIRS standard data products have been designed for ease of use. However, data volume has emerged as a limiting factor in usability and data access. AIRS standard products are designed with full HDF-EOS 4 compliance for its transportability across architectures and for its self-describing nature. In addition, all scientific quantities are expressed simply as floating-point numbers in physical units in order to avoid cumbersome re-scaling of the data prior to use. However, the trade-off has been large data files that can be difficult to use and access because of their size. We have looked at various compression methods to decrease the size of AIRS standard products while maintaining ease of use. Internal HDF compression provides some reduction in file size while remaining transparent to the user, but does not give substantial reduction when used as the only method. Rounding the floating-point numbers by zeroing the least-significant bits of their mantissas allows AIRS products to be compressed significantly without loss of scientific quality. This paper details the trade-offs in choosing how many bits to keep as well as the mechanics of the approach.
SF23A-0026 1340h
The ASTER emergency scheduling system: A new project linking near-real-time satellite monitoring of disasters to the acquisition of high-resolution remote sensing data
Numerous government agencies and university partnerships are currently utilizing orbital instruments with high-temporal/low-spatial resolution (e.g. MODIS, AVHRR) to monitor hazards. These hazards are varied and include both natural (volcanic eruptions, severe weather, wildfires, earthquake damage) and anthropogenic (environmental damage, urban terrorism). Although monitoring a hazardous situation is critical, a key strategy of NASA's Earth science program is to develop a scientific understanding of the Earth system and its responses to changes, as well as to improve prediction of hazard onset. In order to develop a quantitative scientific basis from which to model transient geological and climatological hazards, much higher spatial/spectral resolution datasets are required. Such datasets are sparse, currently available from certain government (e.g. ASTER, Hyperion) and commercial (e.g. IKONOS, QuickBird) instruments. However, only ASTER has the capability to acquire high spatial resolution data from the visible to thermal infrared (TIR) wavelength region in conjunction with digital elevation models (DEM) generation. These capabilities are particularly useful for numerous aspects of volcanic remote sensing. For example, multispectral TIR data are critical for monitoring low temperature anomalies and mapping both chemical and textural variations on volcanic surfaces. Because ASTER data are scheduled in advance and the raw data are sent to Japan for calibration processing, rapid acquisition of hazards becomes problematic. However, a "rapid response" mode does exist for ASTER data scheduling and processing, but its availability is limited and requires significant human interaction. A newly-funded NASA ASTER science team project seeks to link this ASTER rapid response pathway to larger-scale monitoring alerts, which are already in-place and in-use by other organizations. By refining the initial event detection criteria and improving interfaces between these organizations and the ASTER project, we expect to minimize lag time and use existing monitoring tools as triggers for the emergency response of ASTER. The first phase of this project will be integrated into the Alaska Volcano Observatory's current near-real-time volcanic monitoring system, which relies on high temporal/low spatial resolution orbital data. This synergy will allow small-scale activity to be targeted for science and response, and a calibration baseline between each sensor to be established. If successful, this will be the first time that high spatial resolution, multispectral satellite data will be routinely scheduled, acquired, and analyzed in a "rapid response" mode within an existing hazard monitoring framework. Initial testing of this system is now underway using data from previous eruptions in the north Pacific region, and modifications to the rapid data flow procedure within the ASTER science and support structure has begun.
SF23A-0027 1340h
Visualization, Analysis and Subsetting Tools for EOS Aura Data Products in HDF-EOS5
Aura data products are among the first to use the new version 5 of the Hierarchical Data Format for the Earth Observing System, or HDF-EOS5. This presentation discusses the common HDF-EOS5 file layout that is adopted for most of the EOS Aura standard data products. Details of the various tools that can be used to access, visualize and subset these data will also be provided. Aura, the NASA Earth Observing System's atmospheric chemistry mission, was successfully launched July 15, 2004. The Aura spacecraft includes four instruments: the High Resolution Dynamics Limb Sounder (HIRDLS), the Microwave Limb Sounder (MLS), the Ozone Monitoring Instrument (OMI), and the Tropospheric Emission Spectrometer (TES). Data from the HIRDLS, MLS and OMI will be archived at the NASA Goddard Earth Sciences (GES) Distributed Active Archive Center (DAAC), while TES data will be archived at the NASA Langley Research Center DAAC.
http://daac.gsfc.nasa.gov
SF23A-0028 1340h
Hierarchical Data Format (HDF) for Geospatial Data - the State of the Art.
HDF is the format for over 3.5 petabytes of NASA's Earth Observation System standard products as well as other NASA satellite remote sensing products. HDF will also be the native format for the wide variety of operational Environmental Data Records from the future National Polar-orbiting Operational Environmental Satellite System (NPOESS). The next generation of NetCDF,(network Common Data Form), the popular application programming interface for scientific modeling and observation data, will also use HDF as its underlying framework. A variety of commercial scientific visualization and analysis packages support HDF and HDF has been shown to be compatible with a range of international interoperability standards. This poster will survey the state of the art for representation of geospatial data using HDF and explore remaining challenges to even wider use of the format in geoscience.
SF23A-0029 1340h
Improving NGDC Track-line Data Quality Control
Ship-board gravity, magnetic and bathymetry data archived at the National Geophysical Data Center (NGDC) represent decades of seagoing research, containing over 4,500 cruises. Cruise data remain relevent despite the prominence of satellite altimetry-derived global grids because many geologic processes remain resolvable by oceanographic research alone. Due to the tremendous investment put forth by scientists and taxpayers to compile this vast archive and the significant errors found within it, additional quality assessment and corrections are warranted. These can best be accomplished by adding to existing quality control measures at NGDC. We are currently developing open source software to provide additional quality control. Along with NGDC's current sanity checking, new data at NGDC will also be subjected to an along-track ``sniffer'' which will detect and flag suspicious data for later graphical inspection using a visual editor. If new data pass these tests, they will undergo further scrutinization using a crossover error (COE) calculator which will compare new data values to existing values at points of intersection within the archive. Data passing these tests will be deemed ``quality data`` and suitable for permanent addition to the archive, while data that fail will be returned to the source institution for correction. Crossover errors will be stored and an online COE database will be available. The COE database will allow users to apply corrections to the NGDC track-line database to produce corrected data files. At no time will the archived data itself be modified. An attempt will also be made to reduce navigational errors for pre-GPS navigated cruises. Upon completion these programs will be used to explore and model systematic errors within the archive, generate correction tables for all cruises, and to quantify the error budget in marine geophysical observations. Software will be released and these procedures will be implemented in cooperation with NGDC staff.
SF23A-0030 1340h
The development of an OpeNDAP satellite data server for CEOP
This paper describes a project that develops an OPeNDAP server to serve satellite data to the Coordinated Enhanced Observing Period (CEOP) community. CEOP, which is built as the foundation of the World Climate Research Program (WCRP) in Cooperation with World Meteorological Organization (WMO) and the Committee on Earth Observation Satellites (CEOS) under the Framework of Integrated Global Observing Strategy Partnership (IGOS-P), seeks to establish an integrated global observing system for the water cycle to respond to both scientific and social needs. CEOP uses data from field observation, data assimilation, model outputs, and satellite remote sensing in research. The multi-source data integration is one of keys for the success of the CEOP program. Much of the satellite data identified in CEOP are Level-1B and Level-2 products. Data in these products are in Swath coordinates. While CEOP users commonly use the OPeNDAP protocols to access CEOP data for research, most of swath data are not available via this protocol. Instead, many space agencies have developed satellite data servers that implement the Open GIS Consortium (OGC)'s Web Coverage Service (WCS) Specification for serving satellite data to geospatial community. In order to provide satellite data to CEOP community, we developed a middleware, which act as a wrapper around an OpenGIS WCS implementation providing a gateway from the OPeNDAP protocols. The combination of the wrapper and any OGC-compliant WCS server acts as an OPeNDAP server. To provide the capabilities required to convert from Swath coordinates to an equirectangular latitude-longitude coordinate reference system, as well as perform grid cell interpolation and geo-spatial selection the server leverages the capabilities provided by an OGC WCS implementation. Basically, the middleware module does three things: 1). Translate the client requests in OpeNDAP protocols to WCS protocols and pass the requests to a WCS server; 2). Translate the server response in WCS protocols to DAP protocols and pass the response back to client; and 3). Handle the difference between the DAP protocols and WCS protocols. This paper discusses the details of the implementation.
http://laits.gmu.edu
SF23A-0031 1340h
The Amma-Sat Database
The African Monsoon Multidisciplinary Analysis project is a French initiative, which aims at identifying and analysing in details the multidisciplinary and multi-scales processes that lead to a better understanding of the physical mechanisms linked to the African Monsoon. The main components of the African Monsoon are: Atmospheric Dynamics, the Continental Water Cycle, Atmospheric Chemistry, Oceanic and Continental Surface Conditions. Satellites contribute to various objectives of the project both for process analysis and for large scale-long term studies: some series of satellites (METEOSAT, NOAA,.) have been flown for more than 20 years, ensuring a good quality monitoring of some of the West African atmosphere and surface characteristics. Moreover, several recent missions, and several projects will strongly improve and complement this survey. The AMMA project offers an opportunity to develop the exploitation of satellite data and to make collaboration between specialist and non-specialist users. In this purpose databases are being developed to collect all past and future satellite data related to the African Monsoon. It will then be possible to compare different types of data from different resolution, to validate satellite data with in situ measurements or numerical simulations. AMMA-SAT database main goal is to offer an easy access to satellite data to the AMMA scientific community. The database contains geophysical products estimated from operational or research algorithms and covering the different components of the AMMA project. Nevertheless, the choice has been made to group data within pertinent scales rather than within their thematic. In this purpose, five regions of interest where defined to extract the data: An area covering Tropical Atlantic and Africa for large scale studies, an area covering West Africa for mesoscale studies and three local areas surrounding sites of in situ observations. Within each of these regions satellite data are projected on a regular grid with a spatial resolution compatible with the spatial variability of the geophysical parameter. Data are stored in NetCDF files to facilitate their use. Satellite products can be selected using several spatial and temporal criteria and ordered through a web interface developed in PHP-MySQL. More common means of access are also available such as direct FTP or NFS access for identified users. A Live Access Server allows quick visualization of the data. A meta-data catalogue based on the Directory Interchange Format manages the documentation of each satellite product. The database is currently under development, but some products are already available. The database will be complete by the end of 2005.
SF23A-0032 1340h
GAPS and RADS: International Collaboration in Provision of Altimetry Datasets
Two existing Altimeter Data services, both with significant numbers of users, have been interlinked to produce a coherent data service with alternative web interfaces and configurable users access. We will discuss issues of migrating users from existing services to new services such as this. The GAPS (Global Altimeter Processing Scheme) database provides collocated ocean altimeter data. The system has several stages: read the incoming Geophysical Data Records and convert to a consistent format, applying all known data updates and corrections; determine and apply a consistent set of geophysical corrections; collocate to an along track grid. The output includes collocated values of applied corrections so they can be removed or replaced. The system has remained relatively static for several years, giving a stable, but dated, product. The RADS (Radar Altimeter Database System) system, at the Technical University of Delft, is a sophisticated database system that is more flexible in its approach. The system carries out the first 2 stages of the GAPS scheme, allowing users to retrieve non-collocated altimetric parameters. The geophysical corrections are applied at the time of extraction via user-defined corrections sets, allowing more frequent updates to the database. A new system uses RADS as the input data source to create a reference GAPS data set, allowing users to easily extract a collocated, internally consistent dataset with recommended geophysical parameters. The extraction process allows any field in the RADS database to be collocated to the reference grid with little additional computational overhead and no storage overhead. Users can also extract their preferred parameters, using their preferred correction set, either with or without collocation, as required. This system is efficient, but requires the database operator to decide which fields to maintain in the database. Additional data fields must be calculated for all available data, even if never used. A further development of this system is now planned, to make use of recent GRID services advances. The source database will be replaced by a system that retrieves source data on request from remote certified sources.
SF23A-0033 1340h
Using Application Servers to Build Distributed Data Systems
Space and Earth scientists increasingly require data products from multiple sensors. Frequently these data are widely distributed and each source may have very different types of data products. For instance a single space science research project can require data from more than one instrument on more than one spacecraft, data from Earth based sensors and results from theoretical models. These data and model results are housed at many locations around the world. The location of the data may change with time as spacecraft are complete their missions. Unless care is taken in providing access to these data, using them will require a great deal of effort on the part of individual scientists. Today's data system designers are challenged to link these distributed sources and make them work together as one. One approach to providing universal support is to base the core functionality of each data provider on common technology. An emerging technology platform is Sun's Java Application Server. With an application server approach all services offered by the data center are provided through Java servlets that can be invoked through the application server while responding to a request for a specific URL. The benefits of using an application server include a well established framework for development, broad corporate support for the technology and increased sharing of implementations between data centers. We will illustrate the use of an application server by describing the system currently being deployed at the Planetary Plasma Interactions Node of NASA's Planetary Data System.
SF23A-0034 1340h
Importance of SECAA Web Services for the Integrated SEC Data Environment
The Space Physics Data Facility (SPDF) at NASA GSFC has developed new and state-of-the-art, distributed programming (web services) interfaces to SPDF's space science mission services and data. This work is an essential step in integrating these powerful and popular SPDF software systems (and the NASA mission data they now serve) with external science data and model software systems under a NASA Virtual Observatory paradigm for enhancing the scientific return of space physics research. The distributed components of a space physics virtual observatory work together via standard interfaces and metadata agreements to form a system that appears as a single super-instrument providing geophysical measurements and models across time and space, enabling researchers to easily and seamlessly analyze data from many more sources than possible before. We are providing a critical set of these lower level components, leveraging our data format expertise and our existing and very popular science and orbit data web-based services, Coordinated Data Analysis Web [CDAWeb] and Satellite Situation Center Web [SSCweb], by adding web services for orbit location, data finding across FTP sites and in CDAWeb, data file format translation, and display. These services tie together existing data holdings, standardize and simplify their use, and enable much enhanced interoperability and data analysis. A Java3D application, TIPSOD (Tool for Interactive Plotting, Sonification and 3-D Orbit Display), developed in-house, makes use of these web services. CDAWeb, SSCWeb, CDF tools, and web services are a joint effort of the Sun-Earth Connections (SEC) Active Archive (SECAA) in the NASA GSFC Space Physics Data Facility (SPDF) and the National Space Science Data Center (NSSDC); see $<$http://spdf.gsfc.nasa.gov/$>$.
http://spdf.gsfc.nasa.gov/
SF23A-0035 1340h
Peer-to-Peer Science Data Environment
The goal of P2PSDE is to provide a convenient and extensible Peer-to-Peer (P2P) network architecture that allows: distributed science-data services-seamlessly incorporating collaborative value-added services with search-oriented access to remote science data. P2PSDE features the real-time discovery of data-serving peers (plus peer-groups and peer-group services), in addition to the searching for and transferring of science data. These features are implemented using "Project JXTA", the first and only standardized set of open, generalized P2P protocols that allow arbitrary network devices to communicate and collaborate as peers. The JXTA protocols standardize the manner in which peers discover each other, self-organize into peer groups, advertise and discover network services, and securely communicate with and monitor each other-even across network firewalls. The key benefits include: Potential for dramatic improvements in science-data dissemination; Real-time-discoverable, potentially redundant (reliable), science-data services; Openness/Extensibility; Decentralized use of small, inexpensive, readily-available desktop machines; and Inherently secure-with ability to create variable levels of security by group.
SF23A-0036 1340h
Earth Science Data Grid System
The Earth Science Data Grid System (ESDGS) is a software in support of earth science data storage and access. It is built upon the Storage Resource Broker (SRB) data grid technology. We have developed a complete data grid system consistent of SRB server providing users uniform access to diverse storage resources in a heterogeneous computing environment and metadata catalog server (MCAT) managing the metadata associated with data set, users, and resources. We are also developing additional services of 1) metadata management, 2) geospatial, temporal, and content-based indexing, and 3) near/on site data processing, in response to the unique needs of Earth science applications. In this paper, we will describe the software architecture and components of the system, and use a practical example in support of storage and access of rainfall data from the Tropical Rainfall Measuring Mission (TRMM) to illustrate its functionality and features.
SF23A-0037 1340h
The ISLSCP Initiative II Data Collection
The International Satellite Land Surface Climatology Project (ISLSCP) sponsored the production of the first, co-registered, peer-reviewed, documented interdisciplinary data collection that included global, monthly surface meteorology, vegetation, soils, surface routing and runoff, atmospheric radiation data and clouds for 1987 and 1988 at a 1-degree spatial resolution. The ISLSCP Initiative II collection is greatly expanded in both time and spatial resolution. Initiative II is a 10-year core global data collection spanning the years 1987 to 1995 with improved spatial and temporal resolution (one-quarter to 1 degree) using improved data generation algorithms. In addition, Initiative II includes some data sets spanning the 18-year period, 1982-1999. The data collection includes additional carbon and socioeconomic data sets uniquely designed to support global carbon cycling studies. Initiative II provides a comprehensive collection of high priority global data sets in a consistent data format and Earth projection. The full Initiative II collection consists of some 47 different data types with 230 different parameters shown below (number of parameters in parentheses). * Radiation and Clouds (45) * Hydrology, Topography and Soils (39) * Near-Surface Meteorology (95) * Vegetation (29) * Fixed (11) * Carbon (15) * Monthly (12) * Snow and Sea Ice (4) * Monthly 3-hourly (44) * Oceans (1) * 3-hourly (28) * Socioeconomic (2) Each of the 47 data sets and accompanying documents have undergone two peer reviews, one by reviewers familiar with the data set (but not the producer of it) and a second reviewer, a potential user who has no previous experience with it. The data set draft is complete and is to be followed by an evaluation of the entire collection prior to final publication by the Initiative II staff and external users. This collection-overview evaluation will include an assessment of the individual data series against each other and independent data sources and an evaluation of the value of the ISLSCP II collection as a whole.The Initiative II collection is available at http://islscp2.sesda.com/.
http://islscp2.sesda.com/
SF23A-0038 1340h
Merging Real-Time and Retrospective Data Services, NOAA's Solar X-Ray Imager
The ground systems team for NOAA's first Solar X-ray Imager (SXI) proposed a merger of real-time and retrospective data services with two goals in mind. First, it was anticipated that this would be a more economical approach than legacy systems that divided these services between two separate organizations within NOAA. Also, unifying these services would naturally provide a simpler, and more consistent public interface for all SXI data users. The implementation of this innovative approach has been successful on both accounts. NOAA's Space Environment Center (SEC) receives the telemetry stream from SXI and generates the raw and processed imagery that they use in their Space Weather alert and forecast services. These data are instantaneously transferred to NOAA's National Geophysical Data Center through a combination of data push and pull protocols. The result is an interface that provides access to all SXI data, including images that are less than two minutes old. The success of this system has prompted its use in the ground systems design for the SXI and Space Environment Monitor (SEM) data collected from GOES-N, schedule for launch in December 2004.
http://sxi.ngdc.noaa.gov
SF23A-0039 1340h
NPOESS Interface Data Processing Segment Architecture and Software
The National Oceanic and Atmospheric Administration (NOAA), Department of Defense (DoD), and National Aeronautics and Space Administration (NASA) are jointly acquiring the next-generation weather and environmental satellite system; the National Polar-orbiting Operational Environmental Satellite System (NPOESS). NPOESS is an estimated \$6.5 billion program replacing the current Polar-orbiting Operational Environmental Satellites (POES) managed by NOAA and the Defense Meteorological Satellite Program (DMSP) managed by the DoD. The NPOESS satellites carry a suite of sensors that collect meteorological, oceanographic, climatological, and solar-geophysical observations of the earth, atmosphere, and space. The ground data processing segment for NPOESS is the Interface Data Processing Segment (IDPS). The IDPS processes NPOESS satellite data to provide weather, oceanographic, and environmental data products to NOAA and DoD processing centers and field terminals operated by the United States government. This paper describes Raytheon's high performance computer and software architecture for the NPOESS IDPS. NOAA, the DoD, and NASA selected this architecture after a 2.5-year Program Definition and Risk Reduction (PDRR) competition. The PDRR phase concluded in August of 2002, and has been followed by the NPOESS Preparatory Project (NPP) phase. The NPP satellite, scheduled to launch in late 2006, will provide risk reduction for the future NPOESS satellites, and will enable data continuity between the current EOS missions and NPOESS. Efforts within the PDRR and NPP phases consist of: requirements definition and flowdown from system to segment to subsystem, Object-Oriented (OO) software design, software code development, science to operational code conversion, integration and qualification testing. The NPOESS phase, which supports a constellation of three satellites, will also consist of this same lifecycle during the 2005 through 2009 timeframe, with operations and support efforts extending out to approximately the year 2020. The IDPS must process a data volume an order of magnitude greater than the current POES and DMSP systems and within significantly reduced processing times. This will also result in 3.8 TB of data during NPP and 7.8 TB of data during NPOESS delivered to the users on a daily basis. The IDPS architecture meets the following key NPOESS design requirements: * Daily processing of 650 GB of satellite data resulting in 3.8 (NPP)/ 7.8 (NPOESS) TB of data products * Generate 95% of the NPOESS data products within 5 minutes * Process and deliver data products for at least 99.99% of the NPOESS satellite data * A scalable architecture producing all NPOESS data products for the NOAA and DOD processing centers while also deploying the architecture on shipboard, battlefield, and mobile "laptop" field terminals * DII-COE compliant and Joint Technical Architecture Interoperability * Robust architecture exceeding an availability of 99.99% * Maximize use of COTS hardware and software This paper will describe the architecture approach to hardware and software that is necessary to meet these challenging NPOESS IDPS design requirements.
SF23A-0040 1340h
Utilizing the Extensible Stylesheet Language Transformations (XSLT) standard to Enhance Interoperability between NASA and NOAA XML Metadata-Databases
The NASA Global Change Master Directory (GCMD) and the NOAA National Climatic Data Center (NCDC) have been exchanging metadata content for more than a decade. Previous transfers of metadata were performed by a tedious process involving converting NCDC's Content Standard for Digital Geospatial Metadata (CSDGM) formatted files to Standard Generalized Markup Language (SGML) and then translating the files to Directory Interchange Format (DIF) using custom GCMD software. Finally, after extensive manual editing to fix mismatched and missing fields due to the two levels of translation, the files were committed to the GCMD database. The complete exchange of metadata was time-consuming and as a result, the NCDC metadata held in the GCMD was frequently out of date. In the Spring of 2004 both organizations began discussions to determine how to streamline the process of exchanging metadata. Since both systems currently hold metadata in the Extensible Markup Language (XML) format, we were in a position to utilize the latest tools available for translating XML. Extensible Stylesheet Language Transformations (XSLT), Version 1.0, is the W3C standard for translating one XML document to another XML (or HTML) document. Demonstrated will be the application of XSLT to transform NCDC's CSDGM metadata to DIF format for referencing in the GCMD XML database. Also shown are the methods and open source tools used for translating, parsing, and formating the XML. We have found that XSLT produces a more accurately translated metadata file with less loss of information and mismatched fields. Therefore, minimal manual manipulation is required and the time it takes to synchronize the database has been significantly reduced.
http://gcmd.nasa.gov/
SF23A-0041 1340h
Demonstrating Interoperability and Heterogeneous Resource Access: The Scientific Resource Access System (SRAS) and the Space Physics Data Markup Language (SPDML)
Answering complex science questions often means that a scientist must find and use a variety of scientific resources. These resources can include everything from static data in various formats to complex data assimilation models to format converters and analysis tools. We have developed a testbed that demonstrates an innovative approach to interoperability--integrating a prototype heterogeneous resource access system and a new, extensible space science data description language to provides easy, quick, and conceptual access to interdisciplinary science data. The Scientific Resource Access System (SRAS) prototypes novel concepts to create a loosely coupled architecture, making it possible to integrate heterogeneous resources, including other systems, into a single conceptual interface. The Space Physics Data Markup Language (SPDML) is a prototype system that provides a standard method of data querying that enables the development of an extensible standard for next generation multi-mission catalog searching capabilities (http://sd-www.jhuapl.edu/SPDML). Each system exists independently, but the two can interoperate--with even more capability to support interdisciplinary scientific enquiry.
SF23A-0042 1340h
Processing Spacecraft Data Without Confusion
Producing multiple versions of the same data product for the same time frame with the same remotely sensed inputs can be a recipe for disaster. Yet, amidst the commotion of satellite launch and early operations (LEO), such data processing is needed. After LEO, the situation gets worse. Processing newly arriving data ("forward processing") is augmented with reprocessing and algorithm development, comparison, evaluation, and testing -- often happening all at the same time. The problem can be analyzed in three main parts: maintaining multiple versions of algorithms and data so that end-product users are not overwhelmed, allocating computer resources efficiently, and simplifying production operations so that vast amounts of data can be processed with minimal staff and fewer errors. OMIDAPS provides a framework for execution of algorithms that transform lower level data acquired by OMI on NASA's Aura satellite into higher level science data products. In contrast to traditional science data processing systems, we address all parts of the problem with an innovative approach allowing multiple data processing to run within a single physical system. The data products, imports, exports, and execution planning are all segregated into distinct "ArchiveSets." This paper describes reasons for multiple concurrent productions on a typical satellite data processing project using OMI as an example. It describes the virtual data processing system concept and its advantages over separate physical processing strings. It explores the specific implementation of the virtual systems within OMIDAPS and discusses some of the implications of our approach and describes how virtual processing is used to accomplish the overall mission of OMI data processing.
SF23A-0043 1340h
Overview of an Algorithm Plugin Package (APP)
Science software that runs operationally is fundamentally different than software that runs on a scientist's desktop. There are complexities in hosting software for automated production that are necessary and significant. Identifying common aspects of these complexities can simplify algorithm integration. We use NASA's MODIS and OMI data production systems as examples. An Algorithm Plugin Package (APP) is science software that is combined with algorithm-unique elements that permit the algorithm to interface with, and function within, the framework of a data processing system. The framework runs algorithms operationally against large quantities of data. The extra algorithm-unique items are constrained by the design of the data processing system. APPs often include infrastructure that is vastly similar. When the common elements in APPs are identified and abstracted, the cost of APP development, testing, and maintenance will be reduced. This paper is an overview of the extra algorithm-unique pieces that are shared between MODAPS and OMIDAPS APPs. Our exploration of APP structure will help builders of other production systems identify their common elements and reduce algorithm integration costs. Our goal is to complete the development of a library of functions and a menu of implementation choices that reflect common needs of APPs. The library and menu will reduce the time and energy required for science developers to integrate algorithms into production systems.
SF23A-0044 1340h
Do Phases of the Moon Affect Phases of Science Software Development?
This paper is based on observation of a number of satellite remote sensing missions and their associated data production efforts. Projects typically transition from initial concept through pre-launch development, launch and early operations, initial release of data, release of "validated" data and into final reprocessing and end of life. Different drivers affect development of the science code during each phase. Science teams tend to create new algorithm versions in distinct patterns during each project phase. One result of this is that a multitude of software and data set versions are created for each data product. This paper will delineate typical project phases, examine the main drivers behind algorithm change, and illustrate typical patterns of science software versions for each phase. Understanding these phases and the nature of the data prodceed in each can improve estimates of both the time it will take and the magnitude and schedule of resources needed to develop, produce, archive and distribute data. It can also help users understand when they will be able to obtain and use data from new projects and how the data sets will change with time. By the way we have not found any connection between moon phases and data set development phases.
SF23A-0045 1340h
The NASA EOS User Services Offices: Supporting Earth Science Data
The primary goal for NASA's Sun-Earth System Division is to use satellite remote sensing to examine the Sun and Earth as a single connected system. Within the Sun-Earth System Division, the Earth Observing System (EOS) is composed of a series of satellites, scientific research, and a data collection and management system known as EOS Data and Information System (EOSDIS). EOSDIS has nine discipline-specific data centers that manage, document, archive, and distribute a variety of Earth system science data. The data centers provide an assortment of services to their data users via their User Services Offices (USO). The nine USOs communicate regularly by email, phone, and teleconference, and have meetings twice a year during which they analyze, discuss, and determine how to better serve the Earth science community. The sharing of information among USO representatives within the User Services Working Group (USWG) results in an understanding of user needs and problems with data sets within EOS. By identifying these needs, we can improve our services and data distribution methods for users, and advocate solutions on behalf of the user community to the EOS project. Each User Services Office provides timely assistance answering a variety of user questions about its data and services, assists users with their data orders, provides referrals to other data centers, and establishes data subscriptions when applicable. USO troubleshoots problems with data sets and data distribution, recommends and supports tools for data subsetting, searching and ordering, handling, and manipulation, and communicates user needs to data and software developers. The USO is each data center's interface to the public, and has many resources available to assist the user, including data set guide documents, science team members, and programmers. Additionally, the USWG represents the nine data centers in the OneNASA outreach effort. Users will always find ready support for NASA Earth science data through the data centers' User Services Offices.
http://nasadaacs.eos.nasa.gov
SF23A-0046 1340h
Access to Land Data Products Through the Land Processes DAAC
The Land Processes Distributed Active Archive Center (LP DAAC) was established as part of NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) initiative to process, archive, and distribute land-related data collected by EOS sensors, thereby promoting the inter-disciplinary study and understanding of the integrated Earth system. The LP DAAC is responsible for archiving, product development, distribution, and user support of Moderate Resolution Imaging Spectroradiometer (MODIS) land products derived from data acquired by the Terra and Aqua satellites and processing and distribution of Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) data products. These data are applied in scientific research, management of natural resources, emergency response to natural disaster, and Earth Science Education. There are several web interfaces by which the inventory may be searched and the products ordered. The LP DAAC web site (http://lpdaac.usgs.gov/) provides product-specific information and links to data access tools. The primary search and order tool is the EOS Data Gateway (EDG) (http://edcimswww.cr.usgs.gov/pub/imswelcome/) that allows users to search data holdings, retrieve descriptions of data sets, view browse images, and place orders. The EDG is the only tool to search the entire inventory of ASTER and MODIS products available from the LP DAAC. The Data Pool (http://lpdaac.usgs.gov/datapool/datapool.asp) is an online archive that provides immediate FTP access to selected LP DAAC data products. The data can be downloaded by going directly to the FTP site, where you can navigate to the desired granule, metadata file or browse image. It includes the ability to convert files from the standard HDF-EOS data format into GeoTIFF, to change the data projections, or perform spatial subsetting by using the HDF-EOS to GeoTIFF Converter (HEG) for selected data types. The Browse Tool also known as the USGS Global Visualization Viewer (http://lpdaac.usgs.gov/aster/glovis.asp) provides a easy online method to search, browse, and order the LP DAAC ASTER and MODIS land data by viewing browse images to define spatial and temporal queries. The LP DAAC User Services Office is the interface for support for the ASTER and MODIS data products and services. The user services representatives are available to answer questions, assist with ordering data, technical support and referrals, and provide information on a variety of tools available to assist in data preparation. The LP DAAC User Services contact information is: LP DAAC User Services U.S. Geological Survey EROS Data Center 47914 252nd Street Sioux Falls, SD 57198-0001 Voice: (605) 594-6116 Toll Free: 866-573-3222 Fax: 605-594-6963 E-mail: edc@eos.nasa.gov "This abstract was prepared under Contract number 03CRCN0001 between SAIC and U.S. Geological Survey. Abstract has not been reviewed for conformity with USGS editorial standards and has been submitted for approval by the USGS Director."
http://lpdaac.usgs.gov
SF23A-0047 1340h
Customer Support Operations In Support Of The NASA/JPL Physical Oceanography Distributed Active Archive Center (PO.DAAC)
PO.DAAC is responsible for the ingest, archive and distribution of data relevant to the physical state of the ocean. The PO.DAAC provides a level of service for customer support for core Earth Observing System Data Information System (EOSDIS) missions such as TOPEX/POSEIDON, Jason, SeaWinds on QuikSCAT, SeaWinds on ADEOS-II, NOAA AVHRR and MODIS. PO.DAAC's level of support has broadened recently to include missions outside of the EOSDIS including, WindSat, GHRSST, Naval Oceanographic, Monterey Bay Aquarium, AirSAR and GOES. Customer support operations is managed and conducted in partnership between Raytheon ITSS and JPL and includes a full complement of services to accommodate the various PO.DAAC and Earth Observing System user communities. Customer support activities are ubiquitous to service industries from banking to shopping and this presentation will detail how two operational applications have been adopted for use in Earth Science data communities. Customer Response Management Operations System Integrated system that combines PO.DAAC's communications channels including web, email, phone and personal interactions into one intelligent knowledge base. The knowledge is shared with customers and the entire team to deliver consistent and timely information. The system allows 24 x 7 customer support service via extensive on-line searchable knowledge base. PO.DAAC has customized the system to fully integrate with PO.DAAC's existing legacy database and order tracking system. Website Communication Channel The newly redesigned PO.DAAC website (http://podaac.jpl.nasa.gov) is the central gateway to all of the Data, Tools and Services offered at the data center. The consistent look and feel was developed to enhance the ease of searching and ordering earth science data in particular. Data is accessible within a few clicks using the improved dynamic data catalog interface. New data products and information are delivered quickly to customers via the dynamic system and templates.
SF23A-0048 1340h
Data Tools and Services at the NASA/JPL Physical Oceanography Distributed Active Archive Center (PO.DAAC)
The PO.DAAC is responsible for archiving and distributing data relevant to the physical state of the ocean. PO.DAAC products, largely derived from satellite missions such as TOPEX/POSEIDON, Jason, GRACE, SeaWinds on QuikSCAT, SeaWinds on ADEOS-II, NOAA AVHRR, MODIS and GOES include ocean surface topography, ocean wind and sea surface temperature. To support users PO.DAAC provides various data tools and services that allow for subsetting by region and parameter, multiple output formats, global and regional quick-look of near-real-time data products, browse image and animation interfaces for selected data products, automated distribution of mission critical data, as well as use of a fast transport interface for downloading data. The following Data Tools and Services at the PO.DAAC will be highlighted: OCEANIDS (near-real-time data distribution), NEREIDS (near-real-time image distribution), POET (subsetting and vizualization interface), Selected Dataset Browse Image Tools, OPeNDAP (DODS) (remote data set access), VAP (scatterometry image and animation web server), EOS Data Gateway, Aspera Web Client Data Transfer Tool, and PO.DAAC's future release of a subscription interface for Seawinds on QuikSCAT data products.
http://podaac.jpl.nasa.gov/