how diffusion model train and use to generate image from prompt

6 Просмотры

Спасибо! Поделитесь с друзьями!

Вам не понравилось видео. Спасибо за то что поделились своим мнением!

Издатель Jul 8, 2026

During Training:
The diffusion model is trained using a noisy latent image, the prompt embeddings, and the timestep embeddings. The prompt embeddings guide the U-Net so it knows what content should appear in the image, while the timestep tells it how much noise is present. Using these inputs, the U-Net learns to predict the Gaussian noise that was added to the latent image. After training on millions of image–text pairs, the U-Net becomes very good at predicting the added noise while being guided by the prompt embeddings. Once the U-Net has learned this mapping, it is ready to generate new images.

Generating an image from a prompt:
During generation, we start with pure random latent noise instead of a real image. We then use the trained U-Net, which receives the current noisy latent, the prompt embeddings, and the current timestep. The U-Net predicts the noise present in the latent, and the scheduler uses that prediction to remove part of the noise, producing a slightly cleaner latent. The timestep gradually decreases, and this denoising process is repeated many times until it reaches timestep 0. At that point, the latent contains a clean representation of the image described by the prompt, and the VAE decoder converts the latent into the final image.