-
Mission · Sonu, 10
Why predicting pixels fails
A model that must draw the future wastes its effort on texture, so joint embedding models predict a summary of the future instead.
6 cards, 2 of them quick checks. About 3 minutes. One idea per card, so read one, then look away and say it back.
Arrow keys or swipe to move.
-
Idea · 1 of 6
Two places to predict
A world model can predict the raw future, like every pixel or every meter read. It can also predict a short summary of the future. This course is about the second choice.
-
Try this · 2 of 6
Name the gist
Look out of a window for ten seconds. Write one line on what will happen in the next minute. Then try to sketch the exact scene. Notice which one you trust.
-
Idea · 3 of 6
Averages make blur
When many futures are possible, squared error rewards the average of them. Average ten possible cloud shapes and you get grey fog. That fog is what pixel models often draw.
-
Quick check · 4 of 6
Quick check on blur
A model must guess a future that can go five ways. It is trained with squared error. What does it learn to output?
Say your answer, then show it
The average of the five, which looks like a blur.
-
Quick check · 5 of 6
Quick check on fit
When is pixel reconstruction still right?
Say your answer, then show it
When the output people need is the picture or the signal itself. Generation needs detail. Decisions need state.
-
Recap · 6 of 6
What to carry forward
Pixel targets average uncertain futures into blur and spend capacity on noise. Joint embedding models predict summaries. That freedom invites a trap called collapse, which is next.
-
Mission · Senior AI Architect
Why predicting pixels fails
A model that must draw the future wastes its effort on texture, so joint embedding models predict a summary of the future instead.
10 cards, 3 of them quick checks. About 5 minutes. One idea per card, so read one, then look away and say it back.
Arrow keys or swipe to move.
-
Idea · 1 of 10
Two places to predict
A world model can predict the raw future, like every pixel or every meter read. It can also predict a short summary of the future. This course is about the second choice.
-
Idea · 2 of 10
Averages make blur
When many futures are possible, squared error rewards the average of them. Average ten possible cloud shapes and you get grey fog. That fog is what pixel models often draw.
-
Idea · 3 of 10
Most bits are noise
Texture, glare and sensor noise fill most of a frame. A pixel loss pays for all of it. In meter data the same role goes to kettles and dryers.
-
Try this · 4 of 10
Find the noise in a meter
Open any 15-minute load profile you can reach. Circle the spikes no one could forecast. Then circle the evening peak. Which one drives a feeder decision?
-
Quick check · 5 of 10
Quick check on blur
A model must guess a future that can go five ways. It is trained with squared error. What does it learn to output?
Say your answer, then show it
The average of the five, which looks like a blur.
-
Idea · 6 of 10
The joint embedding move
Encode the context. Encode the target. Train a predictor to map one summary to the other. The encoder may now drop detail that nothing could predict.
-
Try this · 7 of 10
Write the two losses
On paper, write the pixel loss. It is the gap between a drawing and the real frame. Write the joint loss as the gap between two summaries. Mark which one can ignore texture.
-
Quick check · 8 of 10
Quick check on the evidence
I-JEPA swapped its targets on 1 percent of ImageNet labels. Which won?
Say your answer, then show it
Representation targets reached 66.9 percent. Pixel targets reached 40.7.
-
Quick check · 9 of 10
Quick check on fit
When is pixel reconstruction still right?
Say your answer, then show it
When the output people need is the picture or the signal itself. Generation needs detail. Decisions need state.
-
Recap · 10 of 10
What to carry forward
Pixel targets average uncertain futures into blur and spend capacity on noise. Joint embedding models predict summaries. That freedom invites a trap called collapse, which is next.
-
Mission · My CIO
Why predicting pixels fails
A model that must draw the future wastes its effort on texture, so joint embedding models predict a summary of the future instead.
7 cards, 2 of them quick checks. About 4 minutes. One idea per card, so read one, then look away and say it back.
Arrow keys or swipe to move.
-
Idea · 1 of 7
Two places to predict
A world model can predict the raw future, like every pixel or every meter read. It can also predict a short summary of the future. This course is about the second choice.
-
Idea · 2 of 7
Averages make blur
When many futures are possible, squared error rewards the average of them. Average ten possible cloud shapes and you get grey fog. That fog is what pixel models often draw.
-
Quick check · 3 of 7
Quick check on blur
A model must guess a future that can go five ways. It is trained with squared error. What does it learn to output?
Say your answer, then show it
The average of the five, which looks like a blur.
-
Idea · 4 of 7
Pay for what a decision uses
Compute that models noise is compute no decision uses. For grid and billing work, the output is usually a decision. That points to representation-space prediction.
-
Try this · 5 of 7
Audit one project
Pick one forecasting project. List what it predicts. Strike every item no one acts on. Whatever is left is the summary it should have predicted.
-
Quick check · 6 of 7
Quick check on fit
When is pixel reconstruction still right?
Say your answer, then show it
When the output people need is the picture or the signal itself. Generation needs detail. Decisions need state.
-
Recap · 7 of 7
What to carry forward
Pixel targets average uncertain futures into blur and spend capacity on noise. Joint embedding models predict summaries. That freedom invites a trap called collapse, which is next.