Keeping a Secret in Production
Follow the Chain Until It Stops
Last timeWhere It Can Live Instead
A process has no credentials and must end up with all of them. Every scheme for that is a chain, and the only question worth asking is where the chain stops.
What starting up looks like
A process begins with nothing. It has a user identity, a working directory, a
set of file handles, and no knowledge of any password anywhere. A few hundred
milliseconds later it is talking to a database, a payment provider and an
object store. Something happened in between, and that something is the subject
of this lesson.
The shape is always the same. The process presents one thing it already has.
It receives a short-lived token in exchange. It uses that token to fetch the
credentials it needs. It holds those in memory and uses them until it exits.
The one it already has
There is a step that cannot be fetched. Before anything can be requested,
something has to be presented, and that something was placed there by someone
or something else. This is the bootstrap credential, and it is the quietest
part of every secrets architecture.
It is quiet for a reason that is almost funny. The diagram on the whiteboard
shows the secret service in the middle with arrows coming out of it, and
everybody looks at the arrows. The thing that opens the service is drawn as a
small unlabelled line at the edge, because it was handled during setup, by one
person, probably in an afternoon, and never revisited.
The consequence is a system where the credentials are held carefully, rotated
on a schedule, and audited on read, and all of them are reachable by anybody
who can present one unrotated value that three people know and that is also in
a configuration management repository from four years ago.
| nothing is stored on dis | works unattended | survives moving to anoth | resists somebody who own | |
|---|---|---|---|---|
| platform identity grante | 1 | 1 | 0 | 0 |
| certificate installed wh | 0 | 1 | 1 | 0 |
| a value a person types a | 1 | 1 | 1 | 1 |
| hardware root on the mac | 0 | 0 | 1 | 1 |
Where the chain ends
Four endings, and each one is an assumption worth stating out loud.
A platform identity is the common modern answer. The process runs somewhere,
and the place it runs vouches for it: anything running in this position is this
service. Nothing is stored anywhere, which is a genuine improvement, and the
assumption is that nothing else can get into that position. That assumption is
exactly as strong as the controls on who can start something there, which is a
different team and a different review.
A certificate placed into the image at build time pushes the problem to the
build system, which now holds a credential that can obtain every credential.
Build systems are frequently the least protected part of an estate, with broad
access, many contributors and extensive logging.
A person typing a value at start is the strongest against replay and the
weakest operationally. Nothing on the machine can reproduce it, and nothing
restarts without a human. A few systems genuinely warrant this, and most do
not, and the ones that adopt it informally usually end up with the value saved
somewhere so that the restart works at night.
A hardware root ties the ability to bootstrap to a specific physical machine.
It is the strongest ending and it ends there: somebody who can run code on that
machine can still ask it to do the bootstrap.
The lesson stops here
2 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents