tailspec · tail-risk evaluation of AI weather models
Three ways to fix a tail
A forecast distribution has a location, a scale and a shape. A bias correction moves the location and a guided sampler moves the scale. The shape sets how rare the worst events are, and all six AI emulators tested draw it too thin.
Read the emulator evaluation: AIFS, GenCast, GraphCast, Pangu-Weather, FourCastNet 3 and NeuralGCM
×0.17
forecast mass beyond the observed 1-in-100 level, schematic
−0.60
move of the 1-in-100 level, observed scale units
NeuralGCM has the thinnest wind tail of the six emulators, shape −0.196 where ERA5 fits −0.107. Read more →
Shape, Scale, Location
01 · Shape
AI weather emulators: AIFS, GenCast, GraphCast, Pangu-Weather, FourCastNet 3, NeuralGCM
6 of 6
emulators draw a thinner wind tail than ERA5
Read the emulator evaluation
02 · Scale
Classifier guidance on a diffusion weather emulator
×0.59
width of the guided upper tail, exploratory (n = 81)
Read what it moved
03 · Location
A mean-bias correction, estimated on the verification stations
±14 pp
swing in >P95 wind skill under the correction
Read what it moved
in practice
Using the Results
metrics
The Metric Catalogue
get in touch
The code behind this site is in a private repository for now. If any of it is useful to you, get in touch.