Answers · The systems around the model

Why does concurrent code fail only sometimes even when there is no syntax error?

In shortConcurrent code can fail only sometimes because a multi-step update can be interrupted between its read and its write, so two workers act on the same stale value and one update is lost without any error.

It fails only sometimes because an update such as counter = counter + 1 is not one action. It is a read of the stored value, a calculation on the value held by that worker, and a write back to storage; a second worker can enter the gap between the read and the write, so both workers act on the same starting value and one result overwrites the other. The code has no syntax error because every instruction is legal and every worker does exactly what it was told. The defect is in the interleaving, not in any single line.

The missing increment

Take the stored counter at 41. Worker one reads 41 and computes 42. Before worker one writes, worker two reads the stored value, which is still 41, and computes 42. Worker one writes 42. Worker two writes 42. Two increments were requested, two workers did their arithmetic correctly, and the stored counter ends at 42 rather than 43. At no point is 42 wrong for the worker that computed it: 41 + 1 = 42. One event has vanished, and no error was raised because no value written was inconsistent with what that worker had read.

FIG 1Two workers incrementing the same counter

step	worker one's held value	worker two's held value	the stored counter	increments requested so far	what happened
1	0	0	41	0	Starting state. Nobody has read anything yet.
2	41	0	41	1	Worker one reads. It holds 41 and the stored value is still 41.
3	41	41	41	2	Worker two reads in the gap. It also holds 41.
4	42	41	42	2	Worker one writes its computed value. The stored counter is correct right now.
5	42	42	42	2	Worker two writes its computed value from a value now stale. Two requests, one increment.

5 steps

Follow the stored counter. It holds a value somebody computed honestly at every step, including the last. What is lost is not a value but an event.

Why it hides

The window from read to write is short. For an in-memory counter it can be tens of nanoseconds; for a counter behind a network call it can be milliseconds. A collision needs two workers to arrive inside the same window. The expected number of colliding pairs in a period comes from pairing the operations and multiplying by the chance that a pair lands close enough together.

E = \frac{n\,(n-1)}{2}\cdot\frac{w}{T}

With a window $w = 100$ microseconds and a period $T = 1$ hour $= 3{,}600{,}000{,}000$ microseconds, $w/T = 100/3{,}600{,}000{,}000 = 1/36{,}000{,}000$ . At $n = 1{,}000$ operations, the number of possible pairs is $1{,}000 \times 999 / 2 = 499{,}500$ , so $E = 499{,}500/36{,}000{,}000 = 0.013875$ colliding pairs per hour. At $n = 10{,}000$ operations, the pair count is $10{,}000 \times 9{,}999 / 2 = 49{,}995{,}000$ , so $E = 49{,}995{,}000/36{,}000{,}000 = 1.38875$ colliding pairs per hour. Ten times the traffic gives a hundred times the expected collisions because the pair count grows with the square. Nothing in the code changed between those two rates.

The natural guess is that a real concurrency bug should show up every run, or at least often enough for a test to catch it. That would be true if the fault were in a line of code. Here the fault is a required coincidence: the second read must land after the first read and before the first write. A test that runs the operation a few hundred times in sequence never creates that overlap, so it passes. The same code under production traffic runs the operation millions of times with real overlap and drifts by a small fraction. The rarity is not evidence of safety; it is a property of the window.

Anything that checks a condition and then acts on what it saw has the same shape. Two workers both check that no record exists for an address, both see none, and both create one; every later operation has to choose between duplicates. Two workers both check a balance is sufficient, both see enough, and both spend; the balance can go negative. The check was true when made and false by the time the action happened, and nothing announced the change.

Where the interleaving explanation stops

Interleaving is not the only reason two workers can disagree. Even with no interleaving of your own steps, one processor can read a value that another has already changed, because each processor keeps its own copy of recently used memory and the compiler or processor can reorder independent operations. A language's memory model states when a write by one worker is guaranteed to be visible to another. Without something the language promises will synchronise, the read can see an old value even though no two of your steps overlapped.

Worth keeping

An operation that reads and then writes has a gap, and a second worker inside that gap makes the result wrong without breaking anything.
The failure is rare in proportion to the window width, so it survives testing and arrives with traffic.
Unless something synchronises two workers, one is not guaranteed to see the other's writes at all.

Where this goes next

This question is one step of Doing Two Things at Once, a course that works through its subject from the beginning. The page that teaches this part of it is Two Things, One Piece of State, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course