More Than One Input

Last timeThe Answer Without Any Steps

Several inputs need one knob each and nothing else changes, but each knob now means the effect of its input with the others held still, which is not what most people read it as.

One knob each

A house has a size, a number of bedrooms, an age and a distance from the

station. The model that uses all four is the obvious one.

FIG 1Four inputs, five knobs

: the four inputs of one house: size, bedrooms, age, distance
: one knob per input
: the offset, still the prediction when every input is zero

Five knobs rather than two and nothing else different. The offset has stopped being readable: a house of zero size, zero bedrooms, zero age and zero distance from the station is not a house, so the offset is now purely positional.

Everything from the earlier lessons carries over without amendment. The score

is the same average of squared errors. Each knob has its own slope, computed

exactly as before except that the error is weighted by that knob's own input.

The step is the same line. The stopping rule is the same.

FIG 2The slope for any one knob

: the knob belonging to input j
: input j of example i
: the error on example i, which is the same for every knob

One formula for all of them, with j picking out which. The errors are computed once and reused for every knob, which is why fitting five knobs costs barely more than fitting two, and why fitting five million is possible at all.

FIG 3The same fitting loop with four inputs

python

rows = [  # size, bedrooms, age, km from station, price in thousands
    [52, 1, 40, 0.4, 196], [61, 2, 12, 1.1, 230], [75, 2, 35, 0.6, 265],
    [80, 3, 8, 2.2, 290], [94, 3, 25, 0.9, 330], [110, 4, 60, 1.5, 385]]
w, b, step = [0.0, 0.0, 0.0, 0.0], 0.0, 0.0001
for t in range(20000):
    errs = [sum(wj * x for wj, x in zip(w, r[:4])) + b - r[4] for r in rows]
    n = len(errs)
    for j in range(4):
        w[j] -= step * 2 * sum(e * r[j] for e, r in zip(errs, rows)) / n
    b -= step * 2 * sum(errs) / n
print("knobs", [round(v, 3) for v in w], "offset", round(b, 2))
print("the loop is the lesson three loop with one extra line of its own")

The only structural change is a loop over the knobs. Compare this against the code in the third lesson: the error line, the step line and the stopping arrangements are identical, which is the claim this lesson exists to demonstrate.

What a knob now means

This is where the trouble starts, and it is the kind of trouble that produces

confident wrong statements rather than error messages.

In the single input model the slope was the effect of size on price, and that

was both what it computed and what people read it as. With several inputs the

knob on size is the effect of size on price with the bedrooms, the age and the

distance held still. Those are different quantities, and in most data they

differ substantially, because bigger houses tend to have more bedrooms and the

data contains almost no pairs of houses that differ in size alone.

The lesson stops here

5 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

More Than One Input

One knob each

What a knob now means

5 more paragraphs to go

This is the reading half

Read alongside