Atlasof World Models
Start

T-01 Province I · The Founding Coast The Mainland 5 min 1 of 21

World Models: An Overview

What a "world model" actually is, and why prediction, planning, and control all live on the same map.

World Models: An Overview: the world-model loop Plain words: what you see, the picture in your head, your step, your guess and fixing the picture when the guess is wrong. Architect: observation o_t, encoder, state s_t, action a_t, transition f(s_t, a_t) giving s_t+1 and reward r_t, prediction error. CIO: data in, model of the grid, decision, forecast and the audit record of forecasts against what happened. T-01 · THE LOOP what you see the picture in your head your step your guess wrong? what happened fix the picture guess, then check Guess before you step. When the guess is wrong, fix the picture. ot observation encoder state st action at st+1, rt transition + reward error ot+1 update on the error st+1 = f(st, at) One learned transition. Planning and control sit on top of it. data in model of the grid decision forecast miss what happened audit record rehearse before acting Trust comes from the record of forecasts against what happened.
A white scale model of a city on a long table in an old survey office, one cobalt thread running across it.

In one sentence A world model is a small imagined copy of reality that lets a machine guess what happens next before it acts.

Illustration of Sonu, a ten-year-old girl with a dark ponytail in a yellow shirt, holding a small notepad. Sonu, 10

Think about the walk from your bedroom to the kitchen at night. The lights are off. You still do not crash into the couch. How?

You carry a picture of the house in your head. You know the couch sits on the left. You know the hallway turns after about eight steps.

You even know the kitchen floor feels cold under your feet. Nobody has to tell you each time.

That picture in your head is a world model. It is a small copy of the house that lives inside you. You use it to guess what happens next, before you take the step.

Now say Mom moved a chair into the hallway today. Your picture does not know that. Your toe finds the chair. Ouch.

Right then your head fixes the picture. Tomorrow night you walk around the chair without thinking.

So a world model guesses before you move. When the guess is wrong, it gets fixed. It also remembers things you cannot see right now, like the couch in the dark.

Computers can learn this too. A robot vacuum needs a picture of the rooms. A game player on a computer needs a picture of what is off the screen.

Scientists teach machines to build these pictures by watching the world again and again. The machine guesses the next moment. Then it checks and fixes the picture a little.

I work on this for electricity. A power company cannot see every wire at once. Its computers keep a picture of the grid. It is the same trick you use for the house.

Try this at home with a grown-up next to you. Stand at your bedroom door with your eyes closed. Guess how many steps it takes to reach your bed.

Then walk it slowly and count. Was your picture right? If not, try again and see if it gets better.

Illustration of the Senior AI Architect, a person in glasses and a blue shirt holding a pen. Senior AI Architect

A world model is a learned transition function. It takes a state and an action and predicts the next state. Most versions also predict a reward or the next observation. Planning, control and training in imagination all sit on top of this one predictive object.

Four terms carry the field. The observation is what the sensors report. The state is a compressed summary of everything relevant so far.

The action is what the agent does. The transition is how the state changes in response.

Ha and Schmidhuber made this concrete in "World Models" (2018). A variational autoencoder compressed each CarRacing frame into 32 numbers. A recurrent network predicted the next code. A controller with 867 parameters chose the action.

The agent averaged 906 over 100 random tracks. The bar for solving is 900. They also trained a VizDoom agent entirely inside the model's own dream. The policy then worked in the real game.

The idea is older than deep learning. Sutton's Dyna (1991) mixed real experience with experience imagined from a learned model. In 2022 Sutton proposed a common model of the decision maker. A world model is one of its internal parts.

LeCun's position paper from 2022 goes further. It puts a predictive world model at the center of an autonomous agent.

The engineering tension is what to keep. A pixel-perfect simulator keeps everything and costs a great deal to run. A hand-written physics equation keeps almost nothing and cannot flex.

Learned world models sit between these poles. Most of this atlas is a record of where that trade has moved.

Here is the utility version. A customer's monthly bill says 900 kWh. That number is the observation. The hourly usage behind it is the state.

Three customers can share that bill and live very different days. Will a time-of-use rate save one of them money? Will a demand response event at 4 pm shift anything? Is the night load a new EV or a failing meter?

None of these is answerable from the bill. All of them are answerable from interval history. A world model for customer decisions has to carry that history as state.

Two failures show up early. The first is a state that drops what the decision needs. The model looks accurate on its own loss and still plans badly.

The second is model exploitation. A planner pushes into actions the model never saw and trusts the fiction there. In one lab an unbounded actor drove a battery setpoint to about 88 MW. Its critic had only seen actions within 3 MW either way.

I would recommend to measure three things before trusting a world model. First, prediction error at the horizon the planner uses. One-step error flatters almost every model.

Second, decision error. Does the chosen action change when the model is wrong? Third, calibration, so stated confidence tracks real accuracy.

Illustration of My CIO, a grey-haired person in a purple jacket with arms folded. My CIO

A world model is a bet that simulation is cheaper than reality. A utility cannot try a new switching plan on a live feeder just to see. A world model gives a stand-in for the grid or the customer base.

A plan can then be rehearsed thousands of times before it touches anything real. Robotics, logistics and customer operations face the same wall. Every one of them has things it cannot test in production. The world model is the sandbox where that test happens.

The value sits in data most utilities already hold. Meter interval reads, outage logs, crew dispatch records and control loops are all sequential. They record decisions and what followed.

That is the fuel a world model needs. Organizations sitting on this kind of data will see value first.

The risk is a confident model of the wrong things. A model can look accurate on a dashboard and still mislead a plan. It may have dropped the variable the decision turns on.

It may also be asked about conditions it never saw. A storm season unlike any training year is the classic case.

The money question is where the model is allowed to act. Advice to a planner is cheap to get wrong. Automated switching or billing changes are expensive to get wrong.

I would recommend to start where a wrong answer costs a review. Grow the scope as the evidence builds.

Watch one capability. Can the model answer what-if questions cheaply, with answers the team has reason to trust? That trust comes from a record of past predictions against what actually happened. Polished demos and generated video do not create that record.

The question I would ask the team is this one. At our planning horizon, how often was this model right and how do we know?

Quick check

3 questions. Pick one answer each. Get them all right to mark this territory as checked.

1What does a world model predict?

2Three customers have the same 900 kWh monthly bill. Why can a model not tell whether a time-of-use rate helps them?

3In a planning study, which number best shows whether you can trust a world model?

Key terms

Further reading

  1. World Models David Ha, Jürgen Schmidhuber, 2018 The essay that gave the field its name. A 32-number code, a recurrent predictor and an 867-parameter controller solved CarRacing.
  2. Dyna, an Integrated Architecture for Learning, Planning, and Reacting Richard S. Sutton, 1991 The first clean recipe for mixing real experience with experience imagined from a learned model.
  3. The Quest for a Common Model of the Intelligent Decision Maker Richard S. Sutton, 2022 One vocabulary for state, action and prediction across psychology, control, economics and AI.
  4. A Path Towards Autonomous Machine Intelligence Yann LeCun, 2022 The position paper that puts a predictive world model at the center of an agent and introduces JEPA.

Search the atlas