Atlasof World Models
Start

T-13 · Spatial World Models I: Geometry & 3D Representations

  1. Mission · Sonu, 10

    Spatial World Models I: Geometry & 3D Representations

    Store the world as shapes in space, not flat pictures, and every viewpoint agrees with every other.

    5 idea cards, 3 questions and a recap. About 6 minutes. One idea per card, so read one, then look away and say it back.

    Arrow keys or swipe to move.

  2. Idea 1 of 5

    Close one eye and hold out a finger. Ask someone to hold up a pencil. Try to touch its tip with your finger. It is harder than you think.

    Now open both eyes and try again. Easy.

    Your two eyes see two flat pictures that are a tiny bit different. Your brain mixes them into depth. Depth is your sense of how far away things are.

  3. Idea 2 of 5

    Keep in your head Close one eye and hold out a finger.

    Moving your head helps too. That is how you know the cup is in front of the jug.

    Spatial world models do something similar. A camera takes many flat photos of a place from different spots. The computer rebuilds the place as shapes in space.

    Try this Close the screen. Tell someone at home what this card says, in your own words.

  4. Idea 3 of 5

    Keep in your head Moving your head helps too.

    It works out where each surface is. It works out how far away each one sits. It even works out the colour from each side.

    Here is why that matters. Suppose the computer kept only a stack of photos. Its guess from the left might put the chair in one spot. Its guess from the right might put it somewhere else.

  5. Idea 4 of 5

    Keep in your head It works out where each surface is.

    A 3D model keeps one chair in one spot. So every view agrees with every other view.

    Some of these models are made of millions of tiny soft blobs. Think of coloured cotton balls floating in the air. From far away, the blobs blend into walls and trees and floors. Computers can now draw them fast enough to walk around inside.

    Try this Draw this card as one small picture. Use three lines or fewer.

  6. Idea 5 of 5

    Keep in your head A 3D model keeps one chair in one spot.

    Try this at home. Put a toy on the table. Walk around it and take eight photos from different sides.

    Lay the photos out in a circle on the floor. Can you point to where you stood for each one? Which photo was hardest to place?

  7. Words to keep

    Try this Cover the list. Say each word and what it means, then check.

  8. Quick check 1 of 3

    1Why does a 3D representation keep every view consistent?

  9. Quick check 2 of 3

    2What did 3D Gaussian Splatting make possible?

  10. Quick check 3 of 3

    3What does NeRF need as input besides the images?

  11. Recap

    What you just learned

    1. Close one eye and hold out a finger.
    2. Moving your head helps too.
    3. It works out where each surface is.
    4. A 3D model keeps one chair in one spot.
    5. Try this at home.

  1. Mission · Senior AI Architect

    Spatial World Models I: Geometry & 3D Representations

    Store the world as shapes in space, not flat pictures, and every viewpoint agrees with every other.

    6 idea cards, 3 questions and a recap. About 7 minutes. One idea per card, so read one, then look away and say it back.

    Arrow keys or swipe to move.

  2. Idea 1 of 6

    Representations trade fidelity against memory and editability. Depth maps and point clouds are explicit and cheap, but sparse. Occupancy grids answer one question per voxel, filled or empty. They suit robot planning.

    NeRF (Mildenhall et al., 2020) stores a scene inside a network. The network maps a 5D coordinate to density and colour. That coordinate is position plus view direction.

  3. Idea 2 of 6

    Keep in your head Representations trade fidelity against memory and editability.

    It renders by volume integration along camera rays. The results are photorealistic and slow to render.

    3D Gaussian Splatting (Kerbl et al., 2023) represents the scene as anisotropic 3D Gaussians. A real scene uses millions of them. The method rasterizes them instead of marching rays.

    Try this Look away and say this card back in one sentence.

  4. Idea 3 of 6

    Keep in your head It renders by volume integration along camera rays.

    It reached real-time rendering at 30 frames per second or more at 1080p. That made photoreal 3D interactive for the first time.

    Every representation needs a consistent coordinate frame and camera poses. Structure-from-motion usually supplies them. NeRF takes known poses as input.

    DUSt3R (Wang et al., 2023) regresses pointmaps from image pairs with no calibration or poses. It recovers the cameras as a by-product. That removes a fragile step from field capture.

  5. Idea 4 of 6

    Keep in your head It reached real-time rendering at 30 frames per second or more at 1080p.

    Pose errors are a quiet failure. A few wrong cameras bend the whole reconstruction. Check reprojection error before anyone looks at the pretty render. A clean render can hide a bent yard.

    For world models, explicit 3D gives what video models lack by default. Multi-view consistency holds by construction. Memory has a place to live, in coordinates. A revisit reads the same geometry it wrote.

    Try this Name one system you have shipped where this would break first.

  6. Idea 5 of 6

    Keep in your head Pose errors are a quiet failure.

    That is why several interactive video systems now add a 3D scaffold underneath. The previous territory covers that pattern.

    Take a distribution substation. A technician walks the yard with a phone and captures a few minutes of video. A splat reconstruction gives a walkable model for planning a transformer swap.

  7. Idea 6 of 6

    Keep in your head That is why several interactive video systems now add a 3D scaffold underneath.

    Before trusting it for clearances, check it against surveyed points. Splats optimise for how the scene looks. Geometry can drift where texture is plain, such as a bare concrete pad.

    Three failure modes matter in a utility yard:

    • Floaters, which are blobs of density hanging in empty air. - Thin structures such as conductors and guy wires, which few views capture well. - Reflective and transparent surfaces, which confuse both depth and colour.

    Measure geometry and image quality separately. Use geometric error against surveyed control points for the first. Use quality on held-out views for the second.

    Then track the capture date of each region. A correct model of last year's yard is still wrong today.

    Try this Find "depth" on this card. Define it in one line without looking.

  8. Words to keep

    Try this Cover the list. Say each word and what it means, then check.

  9. Quick check 1 of 3

    1Why does a 3D representation keep every view consistent?

  10. Quick check 2 of 3

    2What did 3D Gaussian Splatting make possible?

  11. Quick check 3 of 3

    3What does NeRF need as input besides the images?

  12. Recap

    What you just learned

    1. Representations trade fidelity against memory and editability.
    2. It renders by volume integration along camera rays.
    3. It reached real-time rendering at 30 frames per second or more at 1080p.
    4. Pose errors are a quiet failure.
    5. That is why several interactive video systems now add a 3D scaffold underneath.
    6. Before trusting it for clearances, check it against surveyed points.

  1. Mission · My CIO

    Spatial World Models I: Geometry & 3D Representations

    Store the world as shapes in space, not flat pictures, and every viewpoint agrees with every other.

    4 idea cards, 3 questions and a recap. About 5 minutes. One idea per card, so read one, then look away and say it back.

    Arrow keys or swipe to move.

  2. Idea 1 of 4

    Spatial models matter wherever physical layout drives decisions. Sites, facilities, warehouses and field assets all qualify. For a utility that covers substations, yards and the equipment inside them.

    Capturing a space once and querying it from any angle is already practical. Phone video and Gaussian splatting do it today. A planner can walk a yard on a laptop before a crew rolls a truck.

  3. Idea 2 of 4

    Keep in your head Spatial models matter wherever physical layout drives decisions.

    The money case is field time. Every site visit avoided or shortened saves crew hours and drive time. A shared model also cuts the back-and-forth between planners and crews. Contractors can see the site before they bid, which tightens estimates.

    The open question for most organisations is staying current. Capture quality is rarely the problem.

    Try this Name the one decision this card changes for you.

  4. Idea 3 of 4

    Keep in your head The money case is field time.

    The real place changes. Equipment gets replaced. Fences move. Trees grow toward the lines every season.

    A model that was right last spring can mislead today. So plan for refresh cycles from the start. A one-off scan is a photo album with depth.

    Each model needs a capture date and an owner. It also needs a trigger for recapture, such as a completed work order.

  5. Idea 4 of 4

    Keep in your head The real place changes.

    The second risk is precision. A model that looks right is not always measured right. Clearances and safety distances need a check against survey before anyone relies on them.

    The decision is the refresh cadence and who owns it. I would recommend to tie recapture to work orders. Then the model updates as part of normal work, with no separate program to fund.

    Ask the team one question. Who owns keeping each site model current after the place changes?

    Try this Say what this card would cost you if a vendor got it wrong.

  6. Words to keep

    Try this Cover the list. Say each word and what it means, then check.

  7. Quick check 1 of 3

    1Why does a 3D representation keep every view consistent?

  8. Quick check 2 of 3

    2What did 3D Gaussian Splatting make possible?

  9. Quick check 3 of 3

    3What does NeRF need as input besides the images?

  10. Recap

    What you just learned

    1. Spatial models matter wherever physical layout drives decisions.
    2. The money case is field time.
    3. The real place changes.
    4. The second risk is precision.

Search the atlas