American Geophysical Union Become an AGU Member
Subscribe to AGU Journals		 AGU Home AGU Publications

Read Full Article (file size: 5752485 bytes)    Cited by

JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 113, D06104, doi:10.1029/2007JD008972, 2008

Performance metrics for climate models

P. J. Gleckler

Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, Livermore, California, USA


K. E. Taylor

Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, Livermore, California, USA


C. Doutriaux

Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, Livermore, California, USA


Abstract

Objective measures of climate model performance are proposed and used to assess simulations of the 20th century, which are available from the Coupled Model Intercomparison Project (CMIP3) archive. The primary focus of this analysis is on the climatology of atmospheric fields. For each variable considered, the models are ranked according to a measure of relative error. Based on an average of the relative errors over all fields considered, some models appear to perform substantially better than others. Forming a single index of model performance, however, can be misleading in that it hides a more complex picture of the relative merits of different models. This is demonstrated by examining individual variables and showing that the relative ranking of models varies considerably from one variable to the next. A remarkable exception to this finding is that the so-called “mean model” consistently outperforms all other models in nearly every respect. The usefulness, limitations and robustness of the metrics defined here are evaluated 1) by examining whether the information provided by each metric is correlated in any way with the others, and 2) by determining how sensitive the metrics are to such factors as observational uncertainty, spatial scale, and the domain considered (e.g., tropics versus extra-tropics). An index that gauges the fidelity of model variability on interannual time-scales is found to be only weakly correlated with an index of the mean climate performance. This illustrates the importance of evaluating a broad spectrum of climate processes and phenomena since accurate simulation of one aspect of climate does not guarantee accurate representation of other aspects. Once a broad suite of metrics has been developed to characterize model performance it may become possible to identify optimal subsets for various applications.

Received 15 May 2007; accepted 21 November 2007; published 20 March 2008.

Keywords: Performance metrics; climate models; model evaluation.

Index Terms: 1626 Global Change: Global climate models (3337, 4928); 1610 Global Change: Atmosphere (0315, 0325); 1620 Global Change: Climate dynamics (0429, 3309); 1616 Global Change: Climate variability (1635, 3305, 3309, 4215, 4513); 1630 Global Change: Impacts of global change (1225).

Read Full Article (file size: 5752485 bytes)    Cited by

Citation: Gleckler, P. J., K. E. Taylor, and C. Doutriaux (2008), Performance metrics for climate models, J. Geophys. Res., 113, D06104, doi:10.1029/2007JD008972.