AI Image Generators Start With Random Noise. Here’s What Happens Next

AI image generators do not sketch pictures.

A diffusion model starts with random noise and repeatedly predicts and removes a small amount of noise until an image emerges.

In this video, I break down the mechanism behind modern text-to-image systems usinIt never sketches anything. A text-to-image model starts from a screen of pure random noise and runs a thousand small correction steps -- guess what noise is there, subtract a little, repeat -- until a picture falls out the other end.

Comment MECHANISM and I'll reply with the exact papers this is sourced from.

Here's the whole argument, in the order the video covers it:

Forward vs. reverse. Destroying a photo into noise is a fixed, unlearned recipe -- the original DDPM paper (Ho, Jain, Abbeel) runs it over 1,000 steps. The only thing the network ever learns is the reverse: given a noisy image, predict the noise.

What "generation" really is. Training undoes noise added to a real photo, where the answer is known. Generation runs the identical loop on noise that was never attached to anything real -- there's no photo anywhere in the process.

Why it isn't instant. The original paper's own numbers: sampling just 256 tiny 32x32 images took 17 seconds. Scaled to real photos, this is why full pixel-space diffusion "often consumes hundreds of GPU days" to train (Rombach et al., the Latent Diffusion paper).

Latent diffusion -- the actual fix. Run the entire thousand-step loop on a small compressed representation instead of raw pixels, then decompress once at the end. This is the specific idea behind the name "Stable Diffusion."

How your prompt steers the picture. Cross-attention layers let every single denoising step consult an encoding of your prompt -- not just the first step.

"CFG scale" -- the most misleading slider in the toolbox. It isn't classifier guidance at all. Classifier-free guidance asks the SAME model twice (once with your prompt, once with nothing) and extrapolates past the difference. No separate classifier model exists in the version almost every tool actually ships.

Three separate knobs. Step count, CFG scale, and the model itself are three independent levers on three different parts of the same loop -- turning one up doesn't fix what another got wrong.

Sources, fetched live for this video:
- "Denoising Diffusion Probabilistic Models" (Ho, Jain, Abbeel) -- arxiv.org/abs/2006.11239
- "High-Resolution Image Synthesis with Latent Diffusion Models" (Rombach et al.) -- arxiv.org/abs/2112.10752
- "Classifier-Free Diffusion Guidance" (Ho, Salimans) -- arxiv.org/abs/2207.12598

Chapters:
0:00 It's Not Drawing
0:11 Why It Feels Like Drawing (And Isn't)
1:03 What's Covered
1:55 Forward vs. Reverse Process
3:11 What the Network Actually Predicts
4:32 How Generation Runs the Loop Backward
5:36 The Pixel-Space Problem
6:44 Latent Diffusion: The Fix
7:58 How Your Prompt Steers the Image
9:12 Classifier-Free Guidance Explained
10:58 Three Separate Knobs
12:02 The Big Picture
13:15 Recap

More from skill.md:
YouTube: [at]skilldotmd
TikTok: [at]skill.md_

#diffusionmodels #stablediffusion #aiexplained #machinelearning #howaiworks #texttoimage #deeplearning #generativeai