Basis Lab

How models work

Fit a straight line to data by hand, understand every piece of the machinery you used, and then recognise that same machinery, a prediction, a loss, a gradient and a step, inside every larger model you will ever meet

The Simplest Model That Learns

One model, small enough to do entirely by hand, containing every part a neural network has. The honest first model rather than a toy.

8 lessons, written and corrected before you arrived. Reading them here needs no account. The first reads the whole way through; the others open and then stop, because a page nobody owns cannot tell who is reading it. Starting the course gives you your own copy, where every idea has problems standing under it and you can ask about any sentence.

Start reading

This is course 1 of 8 in a reading track: build a model by hand, then find the same parts inside every large one. The track is an order, not a gate, and the rest of it is on one page.

  1. 01A Guess With Knobs On ItA model is a formula with adjustable numbers in it, and separating those numbers from the data is the distinction the whole subject rests on.
  2. 02Measuring How Wrong It Isopening onlyFitting needs a single number saying how bad a setting of the knobs is, and the choice of that number is a decision about which mistakes matter.
  3. 03Which Way to Turn the Knobsopening onlyThe slope of the score with respect to each knob says which way is downhill, and stepping that way repeatedly is the method every model in use is fitted with.
  4. 04The Answer Without Any Stepsopening onlyFor a straight line and the squared score the best knobs can be written down directly, and understanding why that shortcut exists explains why almost nothing else has one.
  5. 05More Than One Inputopening onlySeveral inputs need one knob each and nothing else changes, but each knob now means the effect of its input with the others held still, which is not what most people read it as.
  6. 06Predicting Yes or Noopening onlyPredicting a probability needs a squashed output and a matching score, and swapping those two pieces leaves the slopes and the stepping method exactly as they were.
  7. 07When It Learns the Noiseopening onlyA model with enough knobs will fit the accidents in its training data, and the cure is to hold the knobs back with a penalty and to measure on data the fitting never saw.
  8. 08The Same Four Parts, Much Biggeropening onlyA neural network is a prediction, a score, a gradient and a step, which are the four parts of this course, and the three things that are genuinely new are worth naming exactly.