Counting Things as They Arrive
Somebody Has Already Seen the Number You Are About to Change
Last timeDeciding a Window Is Finished
An event arrives after its window closed. There are three things you can do with it, and the difference between them is not technical: it is what happens to a figure already published.
Drop, hold, or correct
The last lesson established that a window is declared finished by a guess, and
that the guess is wrong for a measurable fraction of events. Here is one of
them. A sale with an event time of 09:40 arrives at 11:15. The nine o'clock
window closed at 10:05 and its total was published at 10:06.
There are three things you can do, and that is the complete list.
Drop it. The nine o'clock total stays as published. The sale is not counted
anywhere, the daily total is one short, and nothing in the system records that
this happened unless you arrange for it to.
Hold the window open longer. Had the allowance been two hours rather than five
minutes, this event would have been included. The cost is that every nine
o'clock total is now published at 11:05 rather than 10:05, and every open window
holds its state for two hours instead of five minutes.
Correct it. Publish 10:06 as before, then at 11:15 publish a revised nine
o'clock total. This is the only option that is both complete and prompt, and it
is the one with a requirement attached: everything downstream has to accept that
a number it already has can change.
| step | option | what the reader saw at 10:06 | what the reader sees at 11:16 | what it cost | what happened |
|---|---|---|---|---|---|
| 1 | drop | £41200 | £41200 | the figure is short by £680 forever | Stable, prompt, and quietly wrong. No error appears anywhere and no reader can detect it from the number. |
| 2 | hold open two hours | nothing yet | £41880 at 11:05 | every figure an hour late, state held 24 | Complete and correct, and the nine oclock figure is not available during the morning when somebody wanted it. |
| 3 | correct | £41200 marked provisional | £41880 marked revised | downstream must handle a changed number | Complete and prompt. The whole cost sits in the word provisional and in whether anything reading this figure can cope with it moving. |
| 4 | correct, delta only | £41200 | an extra +£680 record | downstream must add rather than replace | A variant worth knowing. Instead of restating the total, emit the difference. Cheaper for the pipeline, and it requires the consumer to be an accumulator rather than a display, which is a real constraint rather than a detail. |
What a published number promised
The reason correcting is hard has almost nothing to do with the pipeline.
An unpublished figure can be revised for free. Recompute, overwrite, done. The
moment a figure has been seen by a person, it has started being copied: into a
slide for a meeting on Thursday, into a spreadsheet somebody keeps, into a
commission calculation, into an invoice, into a decision that has already been
communicated. Each copy is a thing your revision does not reach.
| consumers must tolerate | the figure is eventually | the figure is late | the figure is quietly wr | |
|---|---|---|---|---|
| drop | 0 | 0 | 0 | 1 |
| hold open | 0 | 1 | 1 | 0 |
| correct, full restatemen | 1 | 1 | 0 | 0 |
| correct, delta only | 1 | 1 | 0 | 0 |
So the practical question is not can we correct this but what reads this
number. A figure consumed only by a dashboard that nobody screenshots can be
corrected freely. A figure that feeds a monthly invoice cannot be corrected
after the invoice is sent, which means the restatement window has to close
before invoicing does, which is a scheduling constraint derived from a
streaming decision.
The lesson stops here
3 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents