Evaluation of climate model simulations using observed data contributes to the assessment of confidence in model predictions of future climate change. The mid-Holocene represents an opportunity to evaluate model simulations of El Niño–Southern Oscillation (ENSO) in comparison with coral proxy evidence of reduced ENSO amplitude. Quantitative comparisons between coral records and model output have been limited by (1) the use of different measures of ENSO amplitude, (2) possible sampling of natural variability in short records, and (3) uncertainty about the stationarity of the relationship between central Pacific sea surface temperature (SST) variability and ENSO signals at the coral site. We examine these issues using modern and fossil coral records from the western Pacific and model simulations of preindustrial and mid-Holocene climate. As a measure of ENSO amplitude, the standard deviation is found to be preferable to event frequency or size as event-based measures are highly dependent on the choice of threshold and may be unreliable for a small number of events. Model ENSO amplitude is found to be strongly dependent on the choice of averaging period, with calendar year averages smoothing the seasonal ENSO signal. A relatively robust relationship between SST variability in the NINO3.4 region and the ENSO SST and precipitation anomalies archived in corals is demonstrated for the instrumental period and for a set of model simulations. Remaining uncertainty about changes in ENSO teleconnections under paleoclimate conditions implies the need for additional proxy records from ENSO-sensitive regions before quantitative reconstructions of ENSO amplitude can be used to evaluate model sensitivity.