Lesson 4 of 7 · I-JEPA, predicting the hidden parts of an image
-
Mission · Sonu, 10
I-JEPA, predicting the hidden parts of an image
I-JEPA sees one large block of an image and predicts the summaries of several hidden blocks, with no hand-made augmentations.
5 cards, 1 of them quick checks. About 3 minutes. One idea per card, so read one, then look away and say it back.
Arrow keys or swipe to move.
-
Idea · 1 of 5
See a block, predict the gaps
I-JEPA shows its encoder one large block of an image. A predictor then guesses the summaries of four hidden blocks.
-
Try this · 2 of 5
Cover and describe
Cover two parts of a photo with sticky notes. Say in four words what each hidden part shows. Lift the notes and compare.
-
Idea · 3 of 5
What the summaries keep
A separate decoder rendered the predictions. Object parts kept their pose, like a bird's back. Fine detail and background were dropped.
-
Quick check · 4 of 5
Quick check on limits
Where did the pixel method beat I-JEPA?
Say your answer, then show it
Clevr counting, 90.5 against 86.7. Fine local detail still favours pixels a little.
-
Recap · 5 of 5
What to carry forward
One context block, four large target blocks, a slow target encoder and a narrow predictor. Strong frozen probes at low compute. Weaker on fine detail.
-
Mission · Senior AI Architect
I-JEPA, predicting the hidden parts of an image
I-JEPA sees one large block of an image and predicts the summaries of several hidden blocks, with no hand-made augmentations.
9 cards, 3 of them quick checks. About 5 minutes. One idea per card, so read one, then look away and say it back.
Arrow keys or swipe to move.
-
Idea · 1 of 9
See a block, predict the gaps
I-JEPA shows its encoder one large block of an image. A predictor then guesses the summaries of four hidden blocks.
-
Idea · 2 of 9
Three networks
A context encoder sees the visible block. A target encoder sees the whole image and moves as a slow average. A narrow predictor maps one to the other.
-
Idea · 3 of 9
Big targets on purpose
Targets cover 15 to 20 percent of the image each. Small targets would teach texture. Large ones force the model to predict whole parts of objects.
-
Try this · 4 of 9
Draw the masks
Draw a 14 by 14 grid. Shade four target blocks of about 35 cells each. Shade a large context block. Erase any context cell inside a target.
-
Quick check · 5 of 9
Quick check on targets
Where do I-JEPA's target features come from?
Say your answer, then show it
The target encoder's output on the full image. They are never raw pixels.
-
Idea · 6 of 9
What the summaries keep
A separate decoder rendered the predictions. Object parts kept their pose, like a bird's back. Fine detail and background were dropped.
-
Quick check · 7 of 9
Quick check on results
What did a frozen ViT-H/14 reach on the ImageNet linear probe?
Say your answer, then show it
79.3 percent after 300 epochs. MAE ViT-H/14 reached 77.2 after 1600.
-
Quick check · 8 of 9
Quick check on limits
Where did the pixel method beat I-JEPA?
Say your answer, then show it
Clevr counting, 90.5 against 86.7. Fine local detail still favours pixels a little.
-
Recap · 9 of 9
What to carry forward
One context block, four large target blocks, a slow target encoder and a narrow predictor. Strong frozen probes at low compute. Weaker on fine detail.
-
Mission · My CIO
I-JEPA, predicting the hidden parts of an image
I-JEPA sees one large block of an image and predicts the summaries of several hidden blocks, with no hand-made augmentations.
7 cards, 1 of them quick checks. About 4 minutes. One idea per card, so read one, then look away and say it back.
Arrow keys or swipe to move.
-
Idea · 1 of 7
See a block, predict the gaps
I-JEPA shows its encoder one large block of an image. A predictor then guesses the summaries of four hidden blocks.
-
Idea · 2 of 7
What the summaries keep
A separate decoder rendered the predictions. Object parts kept their pose, like a bird's back. Fine detail and background were dropped.
-
Try this · 3 of 7
Name a field use
Think of one kind of utility photo with many images and few labels. Write it down. Then name the label you would test first.
-
Idea · 4 of 7
Cheap enough to own
The largest model trained on 16 GPUs in under three days. That puts an in-house image encoder within a normal project budget.
-
Try this · 5 of 7
Plan two probes
Write one semantic label and one fine-detail label for a pilot. Example pair is asset type and hairline crack. Who owns each label set?
-
Quick check · 6 of 7
Quick check on limits
Where did the pixel method beat I-JEPA?
Say your answer, then show it
Clevr counting, 90.5 against 86.7. Fine local detail still favours pixels a little.
-
Recap · 7 of 7
What to carry forward
One context block, four large target blocks, a slow target encoder and a narrow predictor. Strong frozen probes at low compute. Weaker on fine detail.