Asking for a Shape

Last timeShowing Instead of Explaining

When a program has to read the answer, the shape has to be specified, validated and repaired, because it will come back broken often enough to matter.

Asking for a shape

Once a program rather than a person reads the answer, the answer needs a shape,

and the shape should be the simplest one that carries what is needed.

Every piece of structure is something that can come back wrong. A single word

from a fixed list almost never fails. A flat record of four fields fails

occasionally. A nested structure with a list of records inside it, each with

optional fields, fails regularly. So the first question is whether the task

genuinely needs the structure it is being given.

The specification itself works best as a named shape with an example filled in.

A description of the fields leaves a dozen things open, and one filled example

settles them: whether a missing value is an empty string or absent, whether

numbers are quoted, whether the list may be empty, whether the order matters.

FIG 1A shape, specified and then validated

python

SHAPE = """Reply with exactly this shape and nothing else:
{"category": one of billing, delivery, technical, other,
 "urgency": a whole number 1 to 5,
 "order_id": the id if the message names one, otherwise null,
 "summary": one sentence}

For example:
{"category": "delivery", "urgency": 3, "order_id": "A-4192",
 "summary": "Asks where a parcel posted on Tuesday has got to."}"""

ok, problem = validate(reply, SHAPE)       # shape only, not meaning
if not ok:
    reply = model(body + broken_note(reply, problem))   # one repair attempt

The example in the middle is doing most of the work, since it settles that the identifier is a string, that a missing one is the word null rather than an empty string, and that the summary is a sentence rather than a phrase. The two lines at the end are the whole repair mechanism: validate, and if it fails, hand the broken output back with the specific complaint. Note what is not here, which is any check that the category is right.

When it comes back broken

However clearly the shape is specified, some share of answers will not match

it. A field will be missing, a number will arrive as a word, the output will be

wrapped in an explanatory sentence, or a list will have one item too many.

FIG 2What a repair pass has to fix

: the share of answers that arrive broken
: the share you are willing to leave broken
: the share of broken answers the repair must fix

Four per cent arriving broken and a tolerance of eight in a thousand means the repair has to fix eighty per cent of them, which a single retry with the specific complaint attached comfortably achieves. The arithmetic also shows where this stops working: at a twenty per cent break rate the repair would have to fix ninety six per cent, which no retry does, and the answer is then to simplify the shape rather than to improve the repair.

FIG 3How the break rate moves with the complexity of the shape

flat recordnested, with optional fields

Both lines are illustrative and the gap between them is the finding. A flat record degrades gently with the number of fields. The same information arranged in nested structures with optional parts degrades much faster, because there are more ways to be wrong. Where a nested shape is genuinely needed the usual repair is to ask for it in two calls, each producing a flat part, rather than to ask more firmly for the nested one.

Validating and repairing

The mechanism is small and the details matter.

Validate the shape as soon as the answer arrives, against the same definition

that was sent in the request, so the two cannot drift apart.

On a failure, make exactly one more attempt, with the broken output and the

specific complaint attached: the field that was missing, or the value that was

not one of the allowed ones. A general note saying the output was invalid works

much less often than a specific one, for the same reason a good rejection

message works in a tool using system.

Stop after that one attempt. A second repair rarely succeeds where the first

failed, and the right behaviour is to record the case and return a failure the

rest of the system can handle.

The lesson stops here

2 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

Asking for a Shape