If a Cache Hit Took One Second, Main Memory Would Take Four Minutes
Last timeThe Page That Was Not There
The gaps in the memory hierarchy are two orders of magnitude and nobody can feel a nanosecond. Rescale the whole ladder to seconds and the design rules become obvious.
The ladder and its rungs
Here are the numbers. They vary across machines by a factor of two and the
shape never changes.
| nanoseconds | seconds, if one cycle is | typical size in kilobyte | |
|---|---|---|---|
| nearest cache | 1 | 1 | 64 |
| second cache | 4 | 4 | 1024 |
| third cache | 12 | 12 | 32768 |
| main memory | 80 | 240 | 33554432 |
| solid state drive | 100000 | 300000 | 1000000000 |
| spinning disk | 8000000 | 24000000 | 2000000000 |
The left column is correct and useless. Nobody has any intuition for the
difference between one nanosecond and eighty nanoseconds, because nobody has
experienced either.
The same ladder in seconds
So rescale. Take one processor cycle, about a third of a nanosecond, and call
it one second. Everything else scales by the same factor, and the ladder turns
into durations from ordinary life.
Reaching the nearest cache: one second. Reaching the second cache: four
seconds, about as long as it takes to find something on your desk. The third
cache: twelve seconds, walking to a filing cabinet. Main memory: four minutes,
which is walking to another floor of the building. A solid state drive: eleven
days. A spinning disk: nine months.
That rescaling is the single most useful thing in this course, because it
converts an abstract ratio into a decision you already know how to make. Nobody
would walk to another floor of the building eight hundred times to collect
eight hundred items one at a time. They would carry a box. That instinct,
transplanted back down to nanoseconds, is the whole content of the next two
lessons.
The lesson stops here
4 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents