What is Diffusion model?

A diffusion model creates something new by learning to clean up noise. Most of today's image and video generators are built on the idea.

How it learns

Training starts with real examples, such as photos. Engineers add a little random noise to a photo, then a little more, over many steps, until nothing but static is left. The model is then trained to reverse that: given a noisy image and how noisy it is, predict the noise that was added so it can be removed. The 2020 paper "Denoising Diffusion Probabilistic Models" by Ho, Jain and Abbeel made this approach competitive with older image generators, and later systems such as Stable Diffusion and Imagen build on the same noise-prediction setup.

How it generates

To make a new image, the model starts from pure random noise and runs the cleanup many times. Each pass removes a bit of noise and moves the picture closer to something realistic. Your text prompt steers the process, so the result drifts toward "a lighthouse at dusk" instead of any random image.

Think of a sculptor who starts with a rough block and keeps chipping away, checking the brief each time.

Why it matters to you

• Speed versus quality. Because generation takes many sequential steps, diffusion models are slower than one-shot generators. Much recent work goes into cutting the number of steps, for example by training a faster "step-distilled" version using distillation.
• Memory. Image models are usually far smaller than chat models, so many run on a single consumer graphics card. How much VRAM you have decides which versions fit.
• Customization. You can adapt a diffusion model to a style or a product with fine-tuning, and LoRA makes that cheap.
• Beyond images. The same recipe is used for video and audio, and for some experimental text models.

How it differs from a chat model

A chat model is built on predicting the next token in a sequence. A diffusion model refines a whole image at once, over repeated passes. They can be combined: many image tools use a language model to read your prompt and a diffusion model to draw it. Models that handle several kinds of data together are covered under multimodal model.