What it is
A diffusion model generates images (and now video and audio) by starting from pure random noise and removing it step by step until a picture appears. During training the opposite is done: real images are gradually covered in noise and the network learns to undo one small step of that. Text prompts steer the denoising towards what you describe.
What it is for
Creating illustrations and photos from a description, editing parts of an image, extending a picture, making variations, and producing video. It is also useful as data augmentation and for design exploration. The same technology raises questions about copyright and fake images.
How to do it
- Pick a tool that fits your need: a hosted image service for quick results, or an open model you run yourself for control.
- Write a prompt that names the subject, the style, the composition and the light, and say what to avoid.
- Generate several images with different random seeds: each seed gives a different picture from the same prompt.
- Refine: change one thing in the prompt at a time, or use image-to-image or inpainting to change only part of a picture.
- Upscale and fix details, such as hands and text, which these models often get wrong.
- Check the licence and the rules of the platform before publishing, and label generated images honestly.