tailspec · tail-risk evaluation of AI weather models
A forecast distribution has a location, a scale and a shape. Every intervention that claims to fix a tail moves one of them. Almost none move the shape, and nobody is required to say which one moved.
Three case files, one per moment: a bias correction, a guided sampler, and the AI weather emulators themselves. Each names the intervention, what it moved, and the number.
the case files
case file 01 · location
A benchmark of eleven forecast systems reports AI models leading at gale-force wind. Its published aggregates carry a mean-bias correction estimated on the verification stations themselves. Remove the flag and the headline reverses.
Case file in preparation
case file 02 · scale
Classifier guidance steers a generative weather emulator toward rare events and enriches them by a factor of 4 to 90. The enrichment is variance collapse rather than a longer tail, and the method’s estimator fails its own reliability gate.
case file 03 · shape
Six AI weather emulators match the physics-driven models on the usual scores. Every one of them has a wind tail thinner than ERA5’s, and the shortfall grows with rarity. Priced on a real delivery book, none separates from the operational ensemble.
A fourth case file is reserved: building an event set honestly, scored in EP-curve space.
The code behind this site is in a private repository for now. If any of it is useful to you, get in touch.