The four major families of model that learn to generate: VAEs, GANs, autoregressive models, and diffusion/flow methods.
Ask four different artists to paint "a chair," and each will have a different process: one sketches a rough shape and refines it, one paints boldly and asks a critic for feedback, one draws it stroke by stroke in order, one starts from scribbles and gradually cleans them into a picture. Generative models work the same way — there are a handful of fundamentally different processes for turning nothing into something that looks real, and each has different strengths.
Variational autoencoders (VAEs) learn a latent variable z and a decoder, optimizing a tractable lower bound on likelihood (the ELBO); their well-known failure mode, posterior collapse, occurs when the decoder ignores z entirely. GANs replace explicit likelihood with an adversarial game between a generator and a discriminator, trading likelihood guarantees for often sharper samples at the cost of training instability. Autoregressive models factorize a joint distribution into a product of conditionals and generate token-by-token, trained with teacher forcing. Diffusion and flow-matching models learn to reverse a gradual noising process (or, in flow matching, to follow a continuous path from noise to data), currently the dominant recipe for high-fidelity image and video synthesis due to their stable training and strong sample quality.
Within a world model, the choice of generative backbone determines the character of its imagination: autoregressive backbones compose naturally with sequential decision-making; diffusion and flow backbones currently dominate wherever visual fidelity is the priority.
None of these four families is strictly "best" — they are different points on a trade-off between sample quality, training stability, likelihood guarantees, and inference speed. A team choosing a generative backbone for a world model should be explicit about which of those four properties the application actually needs, rather than defaulting to whichever architecture is currently most publicized. Diffusion's dominance in visual quality, for instance, comes with a real inference-speed cost relevant to any real-time or interactive use case.