Several Steps and a Budget
Last timeWhen a Tool Fails
A loop with no limit is not a capability, it is an invoice. Bounding the steps, the time and the money is what makes a multi step system safe to run.
Three budgets, not one
The step limit in the second lesson is the obvious bound and on its own it is
the weakest of the three.
Twenty steps might be two seconds of fast lookups or two minutes of web
fetches. It might be a tenth of a penny or several pounds, depending on how
long the history has grown and which model is being used. A limit expressed in
steps says almost nothing about the two things people actually care about,
which are how long they wait and what it costs.
- what one task may cost
- the model cost of one step
- the tool cost of one step
- steps the budget affords
So three budgets, all checked in the loop. Steps, because it is the cheapest to
enforce and catches runaway behaviour immediately. Time, because somebody is
waiting and a correct answer after four minutes is often worth less than an
admission after thirty seconds. And money, because a loop with no cost limit
has no upper bound on a bad day and bad days are when it will run.
| catches a stuck loop | protects the person wait | protects the bill | cheap to enforce | |
|---|---|---|---|---|
| a step limit | 1 | 0 | 0 | 1 |
| a time budget | 1 | 1 | 0 | 1 |
| a cost budget | 0 | 0 | 1 | 0 |
| all three together | 1 | 1 | 1 | 1 |
Why the cost grows faster than the length
The second lesson showed the curve and this is the reasoning behind it.
At step one the model reads the question. At step two it reads the question,
the first request and the first result. At step three it reads all of that
again plus one more exchange. Nothing is ever read once.
If each step adds about the same amount to the history, then the total read
across a task of many steps is the sum of one plus two plus three and so on,
which is about half the square of the number of steps. Twice as many steps is
roughly four times the reading.
The lesson stops here
3 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents