Answers · The systems around the model

What does it actually mean to say a copy will be 'eventually' consistent?

In shortIt means the copies will agree in the end, and that for some unspecified time a read may return an out of date value.

It means the system guarantees only that the copies converge once writes stop, and guarantees nothing whatever about how long that takes. At any single instant a read may return an old value, so "eventually" is a statement about the limit, not about an effort being made.

What the window actually looks like

Take three copies of one record, with a single copy accepting writes. A write lands on that copy at t = 0 and is acknowledged straight away. Follower one receives it at 4 ms and follower two at 9 ms, which are ordinary numbers on a local network. At t = 0 exactly one copy of the three holds the new value, so two are stale and a read sent to a copy chosen at random returns the old value two times in three, about 67 per cent. At 4 ms one copy of three is stale, about 33 per cent. At 9 ms none is. The window is 9 ms wide, and the system is out of step for the whole of it.

Suppose those copies are serving 4200 reads per second. Over 9 ms they serve 4200 x 0.009 = 37.8 reads. During the first 4 ms the stale fraction is 2/3, so 4200 x 0.004 x 2/3 is about 11.2 reads coming back old. During the remaining 5 ms the fraction is 1/3, so 4200 x 0.005 x 1/3 is about 7.0 more. Roughly 18 reads in 38 return a value that has already been overwritten, and the system has broken no promise to any of those readers.

FIG 1

interval	copies with the write	stale chance, per cent	reads served	reads served stale
0 to 4 ms	1 of 3	67	16.8	11.2
4 to 9 ms	2 of 3	33	21.0	7.0
after 9 ms	3 of 3	0	n/a	0

The natural reading is that "eventually" is a promise about speed: the system will do its best and get there in a moment. It is close to the opposite. The guarantee puts no ceiling on the window at all. A follower four minutes behind satisfies eventual consistency exactly as well as a follower nine milliseconds behind, because both converge once the writes stop and neither word says when. When one follower did fall four minutes behind, the read rate on the remaining copies collapsed from 4200 to 1100 per second, and that collapse had to be handled by someone, not by the guarantee.

Which makes the awkward case a continuous stream of writes rather than a slow copy. Convergence is defined only on the assumption that writes cease. If they never cease, a copy four minutes behind is not converging to anything: it is following a moving target at a fixed distance, and the moment at which it becomes correct never arrives.

Turning it into a number

If a bound on the window is genuinely needed it has to be built, and the usual construction makes the write set and the read set overlap. With $n$ copies in total, acknowledge a write once $w$ of them have it, and answer a read by consulting $r$ of them. Then:

w + r > n

With $n = 3$ , $w = 2$ and $r = 2$ : 2 + 2 = 4, which is greater than 3, so a write set of two and a read set of two cannot be disjoint and every read touches at least one copy carrying the newest value. Drop to $w = 1$ and $r = 1$ and the inequality fails: 1 + 1 = 2, well under 3. The sets can now miss each other entirely, and a read hits the single written copy one time in three, so it is stale two times in three, which is exactly the 67 per cent above. Raising $w$ does not abolish the window, it narrows it by making the acknowledgement wait, and that wait is paid on every write.

What people actually complain about is narrower than any of this. A user who posts a comment and reloads does not need three copies to agree; they need their own read to land on a copy that has their own write. Remember which copy took the write, and send that user's reads there for the next minute, or record the position in the write sequence and hold their reads until a copy has reached it. Either costs a fraction of what waiting for every copy costs, and together they remove the complaint that actually gets filed.

Where the word stops being useful

With one copy accepting writes, a partition does not produce staleness, it produces unavailability. Followers on the far side of the split are frozen at a point in the stream, and if the writing copy is on the minority side there is no stream at all until the split heals. The window that was 9 ms is now however long the network takes, which is not a number anybody can write down. A system called eventually consistent is usually one whose window is short in the ordinary case and unbounded in the worst case, and the second half of that sentence is the half to design against.

Worth keeping

Eventual consistency bounds nothing: a copy nine milliseconds behind and a copy four minutes behind both satisfy it.
With three copies, writing to two and reading from two forces the two sets to overlap in at least one copy, and that copy holds the newest value.
The guarantee users notice they want is that their own writes do not vanish, which is far cheaper to provide than making every copy agree.

Where this goes next

This question is one step of When One Machine Is Not Enough, a course that works through its subject from the beginning. The page that teaches this part of it is Keeping Copies In Step, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course