T-21 Province VI · The Survey Office The Survey Office 5 min 21 of 21 New
Evaluating World Models: Utility, Robustness & Open Problems
How to tell whether a world model is actually good: not by how real it looks, but by whether decisions made with it turn out well.
In one sentence A world model is only as good as the decisions it improves – judge it by those, not by its pictures.
Picture two weather maps on the fridge. One is beautiful, with soft colours and tiny drawn clouds. The other is plain, just a few lines and numbers.
Which one is better? You cannot tell by looking. A weather map can be beautifully drawn and still wrong about tomorrow's rain.
The only fair test is the week that follows. Did you carry an umbrella on the days it rained? Did you leave it home on the sunny days? The better map is the one that kept you dry.
World models are the same. Some make pictures that look real. Looking real is easy to measure and easy to fake.
Scientists tested video models that look very real. Many still got simple physics wrong.
So people who test world models ask harder questions. Does the model react correctly when you change something? If you open a window in its pretend room, does the curtain move?
Does the model know when it is unsure? A good weather map says "maybe rain" on the days it truly cannot tell.
Does it hold up on a day it has never seen, like the first snow? And when you make plans with it, do the plans work in real life?
Those are the honest tests. Pretty pictures are a bonus.
Try this at home for one week. Each morning, guess whether it will rain after school. Also write how sure you are, a little or a lot. Each evening, check.
At the end of the week, count your right guesses. Then check the days you were very sure. Were you right more often on those days?
Evaluation has three layers. Most published numbers sit in the first.
Fidelity metrics measure how close predictions look to reality. Pixel and perceptual error belong here. So does FVD, a video score for quality and temporal coherence (Unterthiner et al., 2018). Fidelity is necessary and rarely sufficient.
Physics-IQ made that gap measurable (Motamed et al., 2025). Its authors found physical understanding in video models severely limited. They also found it unrelated to visual realism.
Controllability and consistency come second. Do predictions respond correctly to actions and interventions? Do they stay coherent over the horizon? Valid next steps can still imply an incoherent world (Vafa et al., 2024).
Utility is the third layer. It measures downstream outcomes. Do policies trained or planned with the model succeed in the real environment?
Model-based reinforcement learning learned this the hard way. One-step prediction likelihood is not always correlated with control performance (Lambert et al., 2020). They named it objective mismatch.
Four properties cut across all three layers:
- calibration, whether stated uncertainty matches actual error
- out-of-distribution behaviour on states the model never saw
- drift over long rollouts
- latency for interactive or control use
Rare failures need their own method. Black-box safety validation runs the system inside a simulator (Corso et al., 2021). It searches for disturbances that cause failure and estimates how likely failure is. That turns evaluation from averages into a hunt for the worst plausible case.
Take a load model for a distribution feeder that feeds a switching planner. Fidelity is the forecast error on held-out days. It will look good, because most days are ordinary.
Utility is a replay. Make switching plans with the model on last summer's peak days. Compare them with the plans operators actually ran. Count overloads avoided and customer minutes saved.
Calibration asks whether the 90% band holds the real load nine days in ten. Drift shows up after a heat wave or a wave of new chargers. Stress testing looks for the sequence that breaks the plan. Think of a late meter feed during a peak with one switch out of service.
Latency belongs in the same report. A plan that arrives after the operator has switched is worth nothing. Time the model inside the planner's real decision window.
Open problems remain. We need benchmarks that reward causal correctness over appearance. We need protocols that survive when the model under test generated its own training data.
I would recommend one report for every world model, with four sections. Show fidelity, controllability and utility in the real setting. Then list the failures the stress test found.
A world model is only worth funding if decisions made with it turn out better. Everything else is a proxy.
Vendors and research teams tend to show fidelity. That means how real the predictions look. It is the easiest number to produce and the easiest to fake. Recent research found video models that look realistic yet fail basic physics.
The evidence that matters is utility. Plans made with the model should beat plans made without it. That comparison should run in your real setting. The margin should be large enough to matter to the business.
For a utility, that might mean switching plans on peak days. It might mean crew dispatch during storms or billing exception handling. Each one has a current way of working to beat. Each one has a cost per wrong decision you can estimate.
Ask for failure analysis as readily as headline results. Where does the model break? How does it behave on a day unlike any in its training data? Does it say when it is unsure?
Trust the vendor who can describe exactly where their model breaks. Silence on failure modes usually means nobody looked.
I would set three gates before money moves:
- a utility test against current practice, in our data
- a calibration check on the model's own uncertainty
- a stress test aimed at rare, costly events
Budget for evaluation as part of the programme. A model you cannot evaluate is a model you cannot defend to a regulator.
So ask one question. In our own setting, by how much did plans made with this model win?
Quick check
Key terms
Further reading
- Objective Mismatch in Model-based Reinforcement Learning Lambert, Amos, Yadan, Calandra, 2020 Better one-step likelihood does not always mean better control.
- Do generative video models understand physical principles? Motamed, Culp, Swersky, Jaini, Geirhos, 2025 Physics-IQ. Physical understanding is severely limited and unrelated to visual realism.
- Evaluating the World Model Implicit in a Generative Model Vafa, Chen, Rambachan, Kleinberg, Mullainathan, 2024 Coherence tests that expose an impossible map behind accurate next steps.
- Towards Accurate Generative Models of Video: A New Metric & Challenges Unterthiner, van Steenkiste, Kurach, Marinier, Michalski, Gelly, 2018 Introduces FVD, the standard fidelity score for video.
- A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems Corso, Moss, Koren, Lee, Kochenderfer, 2021 JAIR 2021. Falsification and failure-probability methods for rare failures.
- The Quest for a Common Model of the Intelligent Decision Maker Richard Sutton, 2022 Evaluating models by the decisions they serve.
- Awesome World Models (Dong et al.) Dong et al., 2026 A maintained index of papers and benchmarks, updated through 2026.