Mini WorldLaplace Plate 44 — AI

Plate 44 · AI

Pictures Out of Noise

Many of today’s image generators begin every picture with meaningless random static. They don’t look up a picture and paste it together; they guess, over and over, which part of what they have is noise, and wipe a little of it away. Watch a model small enough to follow at every step, and you find that it makes things it has never seen precisely because it hasn’t learned perfectly.

Try it
Chapters · 6
  1. 01Every Picture Starts as Static
  2. 02Ruining a Picture Is Easy
  3. 03Learning to Walk It Back
  4. 04A Perfect Student Only Copies
  5. 05Two Networks, One New Answer
  6. 06Your Turn

Scroll

01

Every Picture Starts as Static

A photograph is really a long list of numbers: one each for the red, green and blue of every pixel. Even a small 256-by-256 photo has nearly 200,000. Treat those numbers as coordinates, and every photo is a single point in a space with 200,000 directions.

No one can draw such a space, so here it shrinks to three directions. Each bright dot is a “picture” made of just three numbers, and the pictures we want all lie on a knotted loop. For now, the dots are only random noise.

Each dot is a “picture” of three numbers · the pictures we want lie on the knot

02

Ruining a Picture Is Easy

Start with 1,000 good pictures as examples to practise on: 1,000 points on the knot. Add a little random noise to each, then a little more, and more again.

Every step is only a nudge, but enough of them and the knot falls apart, leaving a round cloud with no trace of what it was. This half is plain arithmetic; nothing needs to be learned.

03

Learning to Walk It Back

The hard part is going the other way. In 2020, researchers at Berkeley trained a neural network to do one small job: given a noisy point, and told how far the noising has gone, guess the noise that was added.

Then start from fresh noise and, step after step, subtract a little of the guessed noise. The network here is small, two layers of 96 neurons, taking 100 steps. Watch the scattered dots draw together and settle into the knot.

04

A Perfect Student Only Copies

What if the guessing were perfect? For a fixed set of examples, the best possible guess can simply be calculated. Here it is, given only 64 examples, the copper beads.

Almost every dot heads straight for one of the beads; nothing new gets made. Learning the examples perfectly means memorising them. As a 2024 paper puts it, theory says the best possible model could only give back its examples, yet real models make new pictures.

05

Two Networks, One New Answer

A 2024 study ran a clever experiment: train two networks on two sets of examples with nothing in common, then start both from the same noise. Here we do the same. Amber dots come from one network, teal from the other, and a thread joins each pair that set out from the same place.

First each network gets only 8 examples: copper beads for one, teal-grey for the other. Each memorises its own, and the pairs land far apart. Then each gets 1,000. Now the two land almost on the same spots, though neither network ever saw the other’s examples. What they have learned is no longer the examples but the knot. In the study, two networks each trained on 100,000 different photos of faces drew nearly the same face from the same noise.

06

Your Turn

Choose how many examples, which denoiser and how many steps, then press Generate. See what happens with only 5 steps, and whether the two networks still agree with 8 examples each.

Try thisWith 1,000 examples, compare “Perfect” and “Network”: which one copies its examples?

What the model leaves out

The “pictures” here have three numbers; real images have hundreds of thousands to millions. The networks in the 2020 DDPM paper had 35.7 million to about 256 million parameters; each one here has about 11,000, was trained in advance, and runs for real in your browser. This plate walks back with the deterministic method (DDIM), so the same noise always gives the same result; the original DDPM took 1,000 steps and added a little fresh randomness at each one. The small networks here make new points mainly because they are too small to memorise 1,000 examples one by one (8 they can). For image networks built from convolutions, a 2024 theory points to two other limits: each part of the network sees only a small patch of the image and treats every position alike, so it stitches patches of its training images into new wholes.

A common misreading

An image generator neither searches a database and pastes pictures together nor “imagines” the way a person does. It does one thing, again and again: guess the noise and take a little away. It can make new pictures because it hasn’t learned its examples perfectly, and real models do sometimes reproduce training images almost exactly: researchers have pulled more than a thousand training images back out of such models.

Sources

  1. Sohl-Dickstein et al. (2015), the first diffusion models, arXiv:1503.03585
  2. Ho, Jain and Abbeel (2020), DDPM, arXiv:2006.11239
  3. Song, Meng and Ermon (2021), DDIM, arXiv:2010.02502
  4. Nichol and Dhariwal (2021), the cosine noise schedule, arXiv:2102.09672
  5. Kadkhodaie et al. (2024, ICLR), generalisation in diffusion models, arXiv:2310.02557
  6. Kamb and Ganguli (2024), an analytic theory of creativity in convolutional diffusion models, arXiv:2412.20292
  7. Carlini et al. (2023), extracting training data from diffusion models, arXiv:2301.13188

Follow the question

Plate 43More Features Than NeuronsA single neuron in a neural network often stands for several unrelated things. A model with just two neurons can arrange five features into a pentagon and keep them all. Plate 16Where Spots and Stripes Come FromA leopard’s spots and a fish’s stripes may come from the same simple recipe: two ingredients, one fast and one slow.

All plates