A System Design in Forty Minutes
Where 360,000 Reads a Second Actually Go
Last timeThe Simplest Thing That Works
Derive the cache hit rate that brings a peak read load inside what the store can serve, the memory that hit rate costs, and the number of copies needed the moment the cache is empty.
The estimates said 100 reads for every write. At the peak that is 360,000
redirects a second against 3,600 link creations, and the broken number from the
single machine version was the first of those two, 36 times over the 10,000 a
second one machine serves out of memory.
There are exactly two things to do with a read load. Put more machines on it, so
that the same question is answered by more than one of them, and stop the
question from arriving, by keeping the answer somewhere nearer. Both are on the
table here, and which one does the work is a question the ratio answers.
The ratio, not the volume
Ninety nine out of every hundred requests this service handles is a read. That
single fact, and not the size of the traffic, is what makes the read path the
design. Take the same 3,600 writes a second and set the reads to 3,600 as well,
and the write arithmetic is unchanged while the drawing becomes unrecognisable,
because there is nothing left for the copies and the cache to repair.
It is worth being precise about what a cache does not do, because this is where
most designs quietly go wrong. The baseline figure of 10,000 a second from
memory belongs to a machine that is also running the service, and a cache
sitting beside the service on those machines is subject to the same number, so
serving 360,000 a second that way takes at least
$$
machines answering reads, and no hit rate changes that count. What the cache
changes is what kind of machine those 36 are, and how many reads continue past
them into the store. A tier of machines doing nothing but key lookups is the
other arrangement, and one of those serves several times more than 10,000 a
second, which is how the final drawing of this design gets away with four of
them. The store is the thing holding 18 TB a year, the thing that has to be
durable and the thing that is expensive to copy, so keeping load off it is worth
a great deal even though it does not reduce the machine count on the front.
The hit rate the store needs
Write for the peak reads a second arriving, which is 360,000. Write for
the reads a second one copy of the store can serve, which is the baseline 10,000.
Write for how many copies of the store answer reads, and for the fraction
of arriving reads the cache answers without touching the store. Misses arrive at
the store at a second, and the store can take , so the design
needs .
The lesson stops here
6 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents