What You Actually Buy With a Hit
Last timeWhy the Same Question Keeps Arriving
A hit rate is not a score. It converts into two very different things, a faster average and a much lighter load on whatever sits behind the cache, and the second usually matters more.
Somebody reports that the cache is at ninety percent and everyone nods. It is
worth slowing down on what that sentence actually promises, because it converts
into two quite different currencies and the conversion rate is not what most
people assume.
The blend
Every request takes one of two paths, and both have a cost you can measure. The
average is therefore a straightforward blend.
- the average cost over all requests, which is what a dashboard usually shows you
- the hit rate, as a fraction between zero and one
- what a hit costs, usually a fraction of a millisecond
- what a miss costs, which includes producing the answer the expensive way
- the miss rate, which is the term that actually governs the result
Put numbers on it. A hit costs one millisecond, a miss costs forty. At a hit
rate of fifty percent the average is twenty and a half. At ninety percent it is
four point nine. At ninety-nine percent it is one point four. The first forty
percentage points of hit rate bought you a saving of fifteen and a half
milliseconds; the last nine bought three and a half. Measured in the average,
early hit rate improvements look like the valuable ones.
The lesson stops here
6 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents