Basis Lab

Many Models, One Job

The Loop That Will Not Settle, and the Loop That Settles on Something Wrong

Last timeChecking Before Passing It On

A retry loop that fixes a draft six times in ten takes 1.56 passes on average, and the cap you put on it moves the worst case rather than that average.

The service answers a customer's written question about an order in four steps.

It sorts the request, finds the matching order and the policy that applies,

drafts the reply, and checks the draft against that policy. The first three

steps run once each. The fourth is different, because it can say no, and the

only sensible thing to do when it says no is to draft again.

That turns a chain into a loop, and a loop has a property the chain did not.

Where it stops is no longer a fact about the system. It is a decision somebody

has to make, and if nobody makes it the loop stops when the money runs out.

The loop and the three numbers that control it

Write pp for the chance that one pass through draft and check produces a draft

the checker accepts. Measured on this service, pp is 0.6. Write q=1−pq = 1 - p,

which is 0.4, for the chance a pass does not settle it. Write NN for the cap,

the number of passes allowed before the loop gives up, and let N=3N = 3.

The prices are the ones every step in this service is charged at. Each step

sends 1,500 words of instruction and context and gets 300 back, input is 3 units

per million words and output is 15, so a step costs $1500 \times 3 / 10^6 =

0.0045$ for what it sends and 300×15/106=0.0045300 \times 15 / 10^6 = 0.0045 for what it

returns, which is 0.009 a step, and takes 0.8 seconds. Drafting and checking are

two steps, so one pass of the loop is 0.018 and 1.6 seconds. Sorting and finding

are the other two, also 0.018 and 1.6 seconds, and they happen once.

FIG 1The loop, with both of its exits
The two exits at the bottom are both failures and they have nothing in common. One costs three passes and produces a queue somebody can see and work through. The other costs one pass, produces a satisfied log line, and reaches the customer. A loop is usually tuned against the first and judged by the second.

Three passes, written out

The cleanest way to get the pass count is to run a thousand requests through on

paper, assuming for now that the checker is perfect, so that it rejects a draft

exactly when the draft is wrong.

The lesson stops here

7 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the planThe contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The rest of this course

  1. 01What Splitting a Job Across Model Calls Buys, and What It Charges
  2. 02A Step Good 95 Times in 100 Is Not Good Enough to Be the Fourth of Fouropening only
  3. 03Chain, Fork, Loop and Crowd, Priced on One Jobopening only
  4. 04Who Decides the Next Step, and What Deciding Costsopening only
  5. 05What One Step Has to Tell the Next, and What It Must Notopening only
  6. 06What a Gate Catches, What It Destroys and What It Costsopening only
  7. 07The Loop That Will Not Settle, and the Loop That Settles on Something Wrongyou are here
  8. 08Three Points of Accuracy for Twice the Costopening only

Read alongside