The Loop

Last timeSaying Something and Doing Something

The whole of a tool using system is a short cycle: ask, call, return the result, ask again. What stops it is the interesting part, because nothing stops it by itself.

The cycle itself

Here is the whole thing.

FIG 1The loop

python

messages = [{"role": "user", "content": question}]

for step in range(MAX_STEPS):
    reply = model(messages, tools=TOOLS)
    messages.append(reply)
    if not reply.calls:
        return reply.text                   # the model answered, so we are done
    for call in reply.calls:
        result = run(call)                  # every check lives inside run
        messages.append({"role": "tool", "id": call.id, "content": result})

return "stopped after the step limit"

Twelve lines, and this is not a simplification for teaching. Production systems of this shape differ by having more inside run, more careful handling of what goes into the result, and more logging, and the loop itself stays about this long. The common experience of reading a framework and being unable to find where the work happens is because this is where it happens, buried under layers that do not add to it.

Three features are worth naming. The conversation is accumulated rather than

replaced, so the model sees everything that has happened so far. The tool

result goes back as a message rather than as a return value, because the model

has no memory between turns other than this list. And the loop runs for a

bounded number of steps, which is the subject of the next section and of the

sixth lesson.

What ends it

The loop above has two exits and only one of them is reliable.

The model can stop asking for tools and simply answer, which is the normal

case and is entirely at the model discretion. Or the step limit is reached,

which is the program decision and always happens eventually.

The lesson stops here

4 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The Loop

The cycle itself

What ends it

4 more paragraphs to go

This is the reading half

Read alongside