Diffusion models, the technology behind Stable Diffusion, Midjourney, and DALL-E, generate images through a genuinely counterintuitive process: learning to reverse a gradual noising process.
The Core Idea
Training: take a real image, gradually add random noise over many steps
until it's pure noise. Train the model to predict and REMOVE
a bit of that noise at each step (reverse the process).
Generation: start with PURE random noise, and run the learned
denoising process in reverse - each step removes a bit
of noise, gradually revealing a coherent image
Why Start from Noise at All
Starting generation from random noise means every generation is genuinely different, even with the same prompt (unless you fix the random seed) — the model learned to denoise toward plausible images conditioned on a text prompt, not to memorize and reproduce specific training images directly.
Why Text Conditioning Works
The denoising process is conditioned on a text embedding (from a separate text encoder like CLIP) at every step — the model learns to remove noise in a way that’s consistent with the described prompt, which is exactly how “a cat wearing a spacesuit” steers each denoising step toward an image matching that description rather than a generic, unconditioned image.
Why This Approach Won
Diffusion models proved more stable to train than GANs and scale more predictably with more data and compute — the same underlying recipe (a denoising network plus a text conditioning signal) scales from small research models to today’s large, high-fidelity commercial image generators.