Answers · Putting a model into service
How do I know if a failed task was the model's fault or the tool's?
In shortYou cannot tell from the reply alone: grade the final state of the data, log what each tool actually returned, and assign every failure to the model, a tool, or your own loop code.
You cannot tell from the reply. A failed run and a successful one produce transcripts that read almost identically, because the model sounds equally confident in both and the tool output that caused the trouble usually looks like an ordinary result. What separates the three possible culprits is a checkable final state to grade against, plus a record written while the task runs of what each tool actually sent back. With both in hand every failure lands in one of three places: the model made a wrong decision given correct information, a tool returned something wrong or nothing when it should have worked, or the loop you wrote dropped, truncated, retried or cut short something recoverable.
Three buckets, and the one nobody counts
The three causes have nothing in common except the outcome. A model failure is a wrong tool chosen, a sensible result ignored, a conclusion that does not follow from what came back. A tool failure is a tool returning something wrong or nothing at all when it should have worked; the repair lives in the tool and sometimes in somebody else's service. A loop failure is everything in between, and it is the residual: whatever is left once the first two are counted.
Twenty two per cent of tasks failing, with nine of those points traced to wrong decisions and five to tools returning bad results, leaves eight per cent belonging to the loop: the code you wrote. Take that eight seriously. In systems measured honestly the loop share is usually the largest of the three, and it is the only one that is an ordinary bug with an ordinary fix. A model fault needs a changed description or a different model, a tool fault may need another team, and a loop fault needs an afternoon.
The reply cannot tell you
The natural guess is to read each reply, mark the bad ones, and charge every one of them to the model. That fails four ways at once. It rewards tone rather than outcome, it cannot see an amount that was wrong by a penny, it cannot see an action performed twice, and it cannot see a confident report that nothing happened when something did. Each of those four is perfectly compatible with fluent, well punctuated, reasonable sounding text, which is exactly what the system produces when it is failing.
| can be checked mechanica | catches a wrong amount | catches a double action | catches a confident fals | |
|---|---|---|---|---|
| grade the final reply | 0 | 0 | 0 | 0 |
| grade the final state of | 1 | 1 | 1 | 1 |
| grade both | 1 | 1 | 1 | 1 |
The truncation case shows why. A log line reading that a tool returned twenty rows does not say whether twenty rows was everything in the table or everything the loop chose to pass on. If it was the second, the model is being blamed for reasoning it was never given the inputs for. The same goes the other way: a tool that returns an empty list for a customer id it does not recognise is reporting absence, and the model will faithfully report that the customer has no orders. Neither of those is a decision error, and no amount of prompt tightening will fix either.
Per step grading is also flattering in a way that hides all of this. At 0.97 per step over a twelve step task, 0.97 squared is 0.9409, the fourth power is 0.9409 squared, 0.8853, the eighth is 0.8853 squared, 0.7837, and the twelfth is the eighth times the fourth, 0.7837 times 0.8853, which is 0.694. So thirty one per cent of tasks fail while nearly every step looks fine. At 0.99 per step the same twelve steps give 0.886, so two points per step becomes nineteen points per task. Quote the step rate and you will never see the eight per cent that belongs to your own code.
| what you see | who owns it | the repair that works |
|---|---|---|
| refunded the wrong order | model: used the customer id as the order id | state the id format in the description and reject a malformed id before the call |
| reported that a customer has no orders | tool: returned an empty list for an unknown id | return a reason rather than a bare absence |
| gave up at step twenty, half done | loop: the step limit | find out why the task needed twenty steps |
| refunded twice | loop: a timeout retried by the model | retry in the program with an identifier |
| summarised twenty rows as if they were all | loop: the result was truncated silently | say in the result that it was truncated |
Deciding between them therefore depends on records made as the task runs, not reconstructed afterwards. The minimum is one record per step: which tool was called, the arguments after validation, whether validation rejected it and why, how long the tool took, whether it succeeded, the full result that went back to the model, and the running totals of steps and cost. Plus one record per task: the request, the final state, whether it matched, and which of the three causes the miss was assigned to.
That last field needs a person. The assignment cannot be derived from the transcript, because nothing in the text distinguishes twenty rows from all the rows, and somebody has to look and write it down. If the full result was never logged, the honest label is unknown, which is a fourth bucket that sends you to fix the logging before anything else. And if the task has no checkable final state, there is nothing to attribute against at all: you are back to grading prose, and a tool fault will keep presenting itself as a careless model.
Worth keeping
- A tool using system is judged on finished tasks against a checkable final state, because the transcript of a failed run reads the same as the transcript of a successful one.
- A system right ninety seven per cent of the time per step finishes about sixty nine per cent of twelve step tasks, so a per step rate never describes the system.
- Failures belonging to the loop around the model are usually the largest share and the cheapest to repair, which is the opposite of where prompt work goes.
Where this goes next
This question is one step of Models That Use Tools, a course that works through its subject from the beginning. The page that teaches this part of it is Judging Whether the Whole Thing Worked, and it reads the whole way through whether or not you have an account.
What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.
Read that pageThe whole courseOther questions people ask about this
- Why does the model pick the wrong tool when two of them sound almost the same?The model only sees the written description, so when two descriptions overlap it cannot tell the tools apart and confidently picks the more familiar reading.
- Can a web page my agent reads make it do something I never asked for?Yes. A fetched web page can contain a sentence aimed at the model, and nothing in the model's view marks it as different from something you wrote yourself.
- Why does the model have to be asked again after the tool hands back its result?Because a tool call is a request, not an answer. The result must be added to the conversation and the model called again to read it and reply.
- Why does the model return JSON wrapped in extra text when I asked for JSON?A request for JSON does not force the model to return only JSON, so you need to validate the reply and repair it once rather than assume it will parse.
- Why do the numbers look good while users keep complaining?Because the score measures a fixed set of cases with a fixed rule, and the users are living through behaviour that the set and the rule never covered.
- How do I make the model answer from the documents I give it instead of its own memory?Fence the documents as labelled reference material, put them before the question, tell the model to prefer them over memory, and require a passage number for each claim.