Basis Lab

How models work

Build a model out of yes-or-no questions, understand why one such tree is unreliable and why hundreds of them together are among the strongest things available on ordinary table-shaped data, and say which of the two standard committees to reach for

Trees, and Committees of Them

The family that actually wins on spreadsheet data. A tree of questions, why a single one cannot be trusted, and the two ways of combining many that fix it.

8 lessons, written and corrected before you arrived. Reading them here needs no account. The first reads the whole way through; the others open and then stop, because a page nobody owns cannot tell who is reading it. Starting the course gives you your own copy, where every idea has problems standing under it and you can ask about any sentence.

Start reading

This is course 7 of 8 in a reading track: build a model by hand, then find the same parts inside every large one. The track is an order, not a gate, and the rest of it is on one page.

  1. 01Splitting on One QuestionA single yes or no question divides the data into two groups, and a measure of how mixed a group is turns the choice of question into arithmetic.
  2. 02Growing a Whole Treeopening onlyApplying the same question search to each group that comes out of the last one builds a tree, and a prediction is made by following a case down to a leaf.
  3. 03Knowing When to Stop Growingopening onlyA tree grown until every leaf is pure memorises its training data exactly, so growth has to be limited, and cutting back after growing works better than stopping early.
  4. 04One Tree Is Unstableopening onlyChange a handful of training rows and a tree can come out looking entirely different, which makes a single tree untrustworthy and is also the opening the rest of the course walks through.
  5. 05Averaging Many Treesopening onlyGrowing hundreds of trees on resampled data, with the questions at each split drawn from a random handful of inputs, and averaging them removes the instability without making the model any more wrong.
  6. 06Each Tree Fixing the Lastopening onlyInstead of averaging independent trees, each new tree is fitted to what the committee so far got wrong, which is more accurate and far less forgiving.
  7. 07Which Input Actually Matteredopening onlyA committee can be asked which inputs it leaned on, by three different methods that disagree, and the ranking that comes back is read wrongly in two standard ways.
  8. 08When to Reach for a Treeopening onlyOn data that lives in a spreadsheet a committee of trees usually wins, on data made of pixels, sound or words it usually loses, and the shape of the data says which case you are in.