Layers, and Why One Line Rebuilds Sixty Megabytes
Last timeLimits on What It Uses
The build cache has one rule and it chains, so the cost of an edit is the step you touched plus every step above it, and a later deletion never reaches the bytes below.
Two complaints usually arrive together and they have one cause. The build takes
nine minutes to absorb a change of one line. The image is 180 MB although the
program inside it is 12 MB. Neither needs measuring to explain. Both are
arithmetic over a list of five numbers, and the list is short enough to put on
the page.
The image is the one this course has been using throughout: a small web service
in five layers. 120 MB of base system, 48 MB of installed dependencies, 12 MB of
application code, and two steps that add no bytes at all. 120 plus 48 plus 12 is
180, and the two empty steps are not an oversight. One sets the working
directory and one records the command to run, and a step that changes no file
still produces a layer, which happens to be an empty one.
A layer, and its two names
A layer is not a copy of a filesystem. It is the set of changes one build step
made to the filesystem it was handed: the files it added, the files it replaced,
and the files it removed. Stack them bottom to top, apply each one to the result
of the one below it, and the top of the stack is the root directory the process
starts with.
Two names hang off a layer and keeping them apart is most of this lesson.
The first is the layer's own name, which is a hash of the layer's bytes. That is
what lets an image be shared: two images built on the same base refer to the
same 120 MB by the same name, so it is stored once on disk and fetched once over
the network.
The second is the key the build cache files a step under, and it is a different
hash of different things. It is computed from three ingredients. The text of the
instruction, which is obvious. The contents of the files being brought in, for a
step that copies files in, which is slightly less obvious and is the reason an
edit is noticed at all. And the key the step below it got, which is neither
obvious nor usually mentioned, and is the whole of the rule everybody states.
The one rule, and why it chains
The rule is that a changed step invalidates every step after it. The derivation
is one sentence long. A step is reused when its key is found in the cache; its
key is computed partly from its parent's key; so a step whose parent's key
changed computes a different key itself; and a key nobody ever stored cannot be
found. The later instruction being identical character for character makes no
difference, because it is not the instruction being looked up. It is the key.
So the cost of an edit follows. Number the steps 1 to from the bottom. Let
be the seconds step takes when it has to be built, and let be the
position of the earliest step whose key changed. Every step below is found
and costs nothing. Every step from upwards is built. The time an edit costs
is therefore
The lesson stops here
9 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents