Say your friend is coming to pick you up at the school gate. She has never met you. You call her and describe yourself.

You do not list every freckle. You do not count your hairs. You say something like this. Red jacket, curly hair, a blue backpack with a fox on it.

Those few words are enough. Out of hundreds of kids, she finds you. That short description is a representation. It keeps the few facts that matter and drops the rest.

A photo of you holds millions of tiny dots. Each dot has its own color number. Almost none of them help your friend.

Representation learning teaches a computer to squeeze those dots into a short description. The description still has to find you.

There is a funny way to get this wrong. Say the computer gets a gold star whenever two descriptions of the same kid match. A lazy computer learns a trick. It describes every kid as "a kid."

Now every description matches, so it earns every gold star. But your friend cannot find you anymore. Scientists call this collapse. Much of this field is about stopping it.

One fix is to also say how kids differ. Red jacket is not green jacket.

Another fix is a guessing game. Cover half of a photo of the playground. The computer describes what is hidden, like a swing. It does not have to draw every leaf.

That guessing game is the idea behind a model called JEPA. I have been studying it this autumn. It learns to describe hidden parts of the world without drawing them.

Try this at school. Pick a classmate without saying who. Describe them in five words to a friend.

Can your friend find them? Then try with three words. Which words did you keep?

Representation learning decides what a model keeps from raw input. Every world model downstream inherits that choice. Three self-supervised recipes dominate. Each keeps different things.

Masked prediction hides part of the input and reconstructs it. MAE (He et al., 2021) masked 75 percent of image patches. It trained at least 3 times faster and reached 87.8 percent on ImageNet-1K with ViT-Huge. Reconstruction keeps fine detail, which helps generation.

Contrastive learning pulls two augmented views of one input together. It pushes different inputs apart.

SimCLR (Chen et al., 2020) reached 76.5 percent top-1 with a linear classifier on ImageNet. It needs many negatives and large batches. The augmentations decide what gets ignored.

Asymmetric methods drop the negatives. BYOL (Grill et al., 2020) has an online network predict a slow-moving target network. It reached 74.3 percent top-1 with a ResNet-50.

Joint-embedding predictive architectures predict the representation of a hidden region. They never predict its pixels.

I-JEPA (Assran et al., 2023) predicts target blocks from one context block. It needs no hand-crafted augmentations. It trained a ViT-Huge/14 on 16 A100 GPUs in under 72 hours. Skipping pixels saves capacity that would go to unpredictable noise.

V-JEPA 2 (Assran et al., 2025) took this to video. It pretrained on over 1 million hours of video and images. Under 62 hours of robot video then let it plan zero-shot pick-and-place on Franka arms.

LeJEPA (Balestriero and LeCun, 2025) replaces stop-gradients and teachers with one regularizer, SIGReg. It pushes embeddings toward an isotropic Gaussian with a single trade-off hyperparameter.

The failure all of these fight is collapse. Train with positives only and the encoder outputs one vector for every input.

In the course notebook I studied, the loss reached 0.000000. Embedding spread fell to 0.001 of its start. A nearest-neighbor test scored 15 percent against 10 percent for guessing.

A second detail matters in practice. Most methods train a projection head on the encoder, then throw it away. SimCLR tested a linear classifier on each output. The encoder output beat the head output by more than 10 points.

Here is the meter version. A month of smart-meter data is 30 days of 96 reads, or 2,880 numbers. An encoder can turn that into a 384-number usage fingerprint.

Trained once on a narrow task, it can serve theft detection or EV-charger detection. It can also feed rate-plan advice.

Collapse also has a meter twin. A broken meter that reports the same reading every month passes a month-to-month consistency rule.

I would recommend to measure spread or effective rank of the embeddings first. Then run a linear probe on a signal your decision depends on. A falling loss proves nothing by itself.

This territory is where "garbage in, garbage out" gets decided for every downstream world model. The encoder chooses which details survive. Every planner built on top can only see what survived.

The danger is quiet. A representation can discard a safety-relevant cue because it looked like noise in training. Nothing breaks.

The model simply never sees that cue again. No later stage can recover it.

Pretrained encoders are attractive for cost reasons. One encoder, trained once, can feed many tasks. In utility work one meter fingerprint might feed theft detection and rate advice. The savings are real when the fingerprint holds the right signal.

So the diligence question changes. Benchmark numbers on public image sets say little about meter data or field photos. Ask instead whether the encoder was trained to keep the signal our domain needs.

There is a second check that costs almost nothing. A training loss near zero can mean the model learned nothing at all. This failure has a name, collapse. Teams should report a probe on a real business signal, beside the loss.

Recent work makes this cheaper. JEPA-style models learn from unlabeled data, which utilities hold in volume. Meta's V-JEPA 2 learned from over a million hours of video before any robot data.

The lesson for us is scale on unlabeled history first. A small domain step comes after.

The decision is where to invest. A shared encoder for one data type, like interval reads, can pay back across teams. It needs one owner and one test suite.

The question I would ask is plain. Which signal does our decision depend on? Have we tested that this encoder keeps it?