03 · Location
All threeThe intervention: a mean-bias correction. What moved: the location.
tailspec · the bias correction
A Mean Shift Scored as Tail Skill
A skill score computed inside bands of the observed value rewards a shift in the forecast mean, so it cannot separate a change of location from skill in the tail. The worked example is arXiv:2608.09972, Do AI weather models miss extremes? Its reported gale-force ranking exists only after a rolling mean-bias correction, and reverses with the correction switched off.
the worked example
Gale-Force Skill With and Without the Correction
The paper verifies forecast systems against European station observations and scores the windiest five per cent of hours. Before scoring, each forecast has a causal rolling four-week, per-hour-of-day mean bias subtracted, fitted on the same stations it is verified against (the authors' section 3.5). The correction is the same for every system and is fair to all-conditions skill, but only the corrected numbers are reported.
Every system underforecasts the windiest band. Adding a positive constant therefore lowers a forecast's error there and raises it in the calm band, whatever its tail does.
The published aggregates include both settings. With the correction off, the two systems the abstract names as leading at gale force, DWD ICON Global and EPT-2.1 Europa, score 0.0 and −4.9 points relative to the reference instead of +9.5 and +9.0. The uncorrected leader is NOAA GFS, at +13.6, from a 0.155 m/s high wind bias. That is the same mechanism in the other direction, so neither ordering ranks tail skill.
the mechanism
Raw Mean Bias and the Skill the Correction Adds
- 1 EPT-2.1 Europa (the authors' own) −0.726 m/s · 13.9 pp
- 2 DWD ICON Global −0.609 m/s · 9.5 pp
- 3 ECMWF AIFS −0.364 m/s · 1.0 pp
- 4 EPT-2 HRRR (the authors' own) −0.359 m/s · 2.2 pp
- 5 EPT-2 Reasoning (the authors' own) −0.342 m/s · −0.6 pp
- 6 EPT-2e (the authors' own) −0.331 m/s · 0.0 pp
- 7 Microsoft Aurora −0.289 m/s · −1.6 pp
- 8 ECMWF ENS (mean) −0.188 m/s · −5.9 pp
- 9 NOAA GFS 0.155 m/s · −13.1 pp
The systems with the largest raw negative bias gain the most skill from the correction: r = −0.988 across 9 systems, so r² = 0.977 of the cross-system variance in that gain is accounted for by one all-conditions number that says nothing about tails.
This is close to true by construction. The shift applied is roughly the negated mean bias, and in this band the error is larger than the shift and has the same sign for every system, so the change in error is close to affine in the mean bias. The gap between the reported and uncorrected orderings is almost entirely mean bias.
other variables
Temperature, Solar and Precipitation
In the heat tail EPT-2 HRRR leads both with the correction, at +19.6 points, and without it, at +15.6.
Solar and precipitation behave like wind. The paper's largest reported number, EPT-2.1 Helios in the clear-sky solar tail at +24.8 points, is +5.6 uncorrected. A four-week lagged hourly mean undercorrects a strongly seasonal quantity, and undercorrects the reference (the denominator of the skill score) most. In the wettest band EPT-2 Reasoning goes from +1.7 points to −1.9. Across the wind systems the correction moves skill by up to ±14 points, compared with the +9.0 to +9.5 point leads it supports.
the family's line
The Location Case
The headline is contingent on a mean-bias correction estimated on the verification network; remove the flag and it reverses. It is the location case, and the premise of the event set stands: missing extremes is not a class property of AI emulators.
Source: the benchmark's own published aggregates (data/derived/final_aggregates_climatology.parquet), recomputed with the correction switched off, generated 2026-09-03.
get in touch
The code behind this site is in a private repository for now. If any of it is useful to you, get in touch.