Denoising: The loop that produces every image

Introduction

A diffusion model is never asked to draw. It is asked one question, over and over: given this noisy thing, what is the noise in it? Everything else a generation request does is arrange that question and decide what to do with the answer.

What the model learned

Training runs the process in the other direction. Take a real image, compress it into a latent, add a measured amount of noise, and ask the model to say what was added. Enough examples at enough noise levels and the model has a usable answer for any noisy latent it is handed, including one that is nothing but noise.

That is the entire trained skill. At no point during training does the model produce an image.

One step of the loop

Generation is that answer applied repeatedly:

1. Hand the model the current latent and the noise level it is at.
2. The model predicts the noise component.
3. The scheduler decides how much of that prediction to remove.
4. The result is a latent at a lower noise level, and the question is asked again.

The first latent is pure noise, and the seed is what decides which noise it is. The last one goes to the VAE decoder and becomes the picture you receive. Nothing in between is an image, and there is nothing to look at until the decoder runs.

Why the number of steps matters

The model's answer is a direction, exact only for an infinitesimally small move. A step of any real size leaves along that direction and lands a little off the trajectory the model was describing, so the next question is asked about a latent that is already slightly wrong.

This is what steps is really setting. Few steps do not render a rougher version of the same picture. They produce a different picture, because the path ended somewhere else. And once the hops are short enough that the remaining drift is smaller than anything the decoder can express, more of them buy nothing at all, which is why every model has a count past which the output stops changing.

What guidance adds

Each step can be run twice: once with your conditioning and once without it. The difference between the two answers is the part of the prediction your prompt is responsible for, and CFGScale multiplies that difference before it is removed.

Two consequences follow. Guidance costs roughly twice the compute per step, because the model runs twice. And pushing the multiplier high amplifies a difference that was only an estimate, which is how high values burn contrast and flatten detail rather than making the result more faithful.

What the scheduler decides

Step sizes are not uniform. Most schedules take large hops early and small ones late, because the trajectory bends hardest near pure noise and is nearly straight by the end.

Some schedulers add fresh noise back at each step rather than only removing it. Those reach a result by a different route, and they are the reason a fixed seed does not always reproduce an identical image.

Where this stops being true

Flow matching architectures ask a slightly different question: not what noise is in this latent, but which direction moves it toward the data. The loop has the same shape, and the trajectory it follows is closer to a straight line, which is part of why those models reach a result in fewer steps.

Autoregressive models do not run this loop at all. They extend a sequence one piece at a time, and a step count is not a dial you are given.