ContentsThe library

How Noise Becomes a Picture

If the Path Is Yours to Pick, Pick a Line

Last timeWhy a Thousand Steps Became Fifty

The path from noise to picture was inherited from a process built for destroying things, and choosing a straight one instead makes every stride along it cheaper.

Every lesson so far has treated the path from picture to noise as given. It

arrived from a process designed to destroy a picture gradually, and the shape it

traces was never chosen for anything. This lesson asks what the method actually

requires of that path, finds the answer is very little, and picks a better one.

What the path has to provide

Two things. A way to construct an input at any point between picture and noise,

which training needs in order to manufacture examples. And a target at that

point saying which way to move to continue towards the picture, which is what the

network regresses on. The noise schedule provides both, and so does the simplest

interpolation there is.

FIG 1The straight path and the velocity along it
a fraction between zero and one, no longer an index into a thousand step schedule
the picture, at the t equals zero end
the noise draw, at the t equals one end
the velocity, which is the same at every point along this path
Compare this with the closed form from lesson two, which has square roots and a schedule behind it. Here there is one line, no schedule to design, and a velocity that does not depend on where along the path you are. The training objective is the squared error between the network output and that velocity, which is the same manufactured regression as before.
FIG 2Two paths between the same noise draw and the same picture
-101012halfway in time, and a long way off the lineone direction through the space of picturesanother direction
the straight paththe path a noise schedule traces
Both paths start at the same noise draw and end at the same picture, so both are valid things to train against. The marked point is where the curved path has wandered furthest. A stride taken there in the direction the curve is currently heading leaves the path immediately, which is the error every extra sampling step is paying to suppress.

Why straightness is the thing worth buying

A stride assumes the direction holds for its whole length. The error it incurs is

therefore set by how much the direction turns along the stride, and nothing else.

A path that does not turn at all can be crossed in one stride with no error

whatsoever. All the sampling steps in this course have been paying for curvature.

The lesson stops here

3 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the planThe contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The rest of this course

  1. 01A Thousand Small Steps From a Photograph to Static
  2. 02A Thousand Steps Collapse Into One Lineopening only
  3. 03Five Lines of Training, and One Squared Erroropening only
  4. 04One Step Back, a Thousand Timesopening only
  5. 05The Network Is Pointing Uphillopening only
  6. 06Overshooting the Condition on Purposeopening only
  7. 07The Same Model, Twenty Times Fasteropening only
  8. 08If the Path Is Yours to Pick, Pick a Lineyou are here

Read alongside