Start With What You Would Hate to Lose
Before any mention of attacks, write down what the system holds and what would be bad about losing it. A threat list with no sense of loss becomes a list of everything.
What counts
The instinct, when somebody says threat modelling, is to start listing
attacks. Resist it for twenty minutes. A list of attacks with no sense of
what each one costs cannot be ordered, and a list that cannot be ordered
becomes a list of everything, which is the same as a list of nothing.
Start instead with what the system holds that you would hate to lose. Four
categories cover nearly everything, and the last two are the ones left out.
Data people entrusted to you. Addresses, messages, health details,
locations, anything they gave because they had to. Money and the things
that move it: balances, payment details, the ability to issue a refund. The
ability to act as somebody, which is not data at all but a capability, and
which covers every credential in the previous course. And finally
reputation and availability: the fact that the service is up, and the fact
that people believe it is safe to use.
asset | disclosed | altered | unavailable | falls on
customer addresses | severe | medium | low | them
payment card details | severe | severe | low | them
order history | medium | medium | low | both
the refund capability | low | severe | medium | us
internal pricing rules | medium | severe | medium | us
the service being up | n/a | n/a | severe | both
application logs | medium | low | low | us
belief that we are safe | n/a | n/a | severe | both
reading it
severe we would be explaining this in public
medium a bad week, recoverable, some people harmed
low annoying, contained, nobody outside noticesTwo entries in that register are worth defending as entries. The refund
capability is not data; it is the ability to do something, and the harm is
entirely in the alteration column. And belief that the service is safe
cannot be disclosed or altered, only destroyed, and its loss is often the
largest number on the page.
Three ways to lose it
Every asset can go wrong in three ways, and the three are genuinely
independent. Disclosure, somebody sees it who should not. Alteration,
somebody changes it. Unavailability, nobody can get at it.
The reason to insist on all three for every asset is that people
instinctively think about only one, usually disclosure, and the instinct is
frequently wrong about which one matters.
| cost if disclosed | cost if altered | cost if unavailable | highest of the three | |
|---|---|---|---|---|
| customer addresses | 3 | 2 | 1 | 3 |
| payment card details | 3 | 3 | 1 | 3 |
| the refund capability | 1 | 3 | 2 | 3 |
| application logs | 2 | 1 | 1 | 2 |
| the service being up | 0 | 0 | 3 | 3 |
| internal pricing rules | 2 | 3 | 2 | 2 |
| the scheduled export job | 1 | 2 | 3 | 2 |
Notice the refund capability row. Disclosing it means nothing, since the
existence of refunds is public. Altering it, meaning being able to issue
one you should not, is as bad as anything on the page. If your model only
asks who can read what, that row never appears.
Whose loss is it
Now a distinction that changes how the ranking comes out: harm to you and
harm to the people who trusted you are not the same thing, and they come
apart more often than is comfortable.
The asset whose loss falls entirely on somebody else is the one most likely
to be underprotected. Not from malice; from the ordinary fact that the
people deciding how much effort to spend do not feel the consequence. A
customer address list that leaks costs you a difficult week and costs some
of those customers a great deal more than that, and the difficult week is
the only part of it that anybody in the room experiences.
The practical remedy is the last column of the register. Writing falls on
them next to an asset is a small act that makes the asymmetry visible at
the moment of the decision, which is the only moment it can be acted on.
The third-party slice is small and worth a sentence. Your system may hold
data about people who never agreed to anything: the recipient of a message,
the person in the photograph, the referee on an application. They have no
relationship with you and no way to complain, and they belong in the
register.
| step | loss | what happens first | who notices | what it costs | what happened |
|---|---|---|---|---|---|
| 1 | disclosed | a copy appears somewhere | often nobody, for months | severe and permanent, it cannot be undon | The defining feature of disclosure is that it is irreversible. There is no action available afterwards that restores the previous state. |
| 2 | altered | a balance or a permission is wrong | the person affected, quickly | medium, if you can tell what it was befo | Recoverable in proportion to how good your records are, which is why an audit trail is itself an asset worth listing. |
| 3 | unavailable | requests fail | everybody, immediately | medium and bounded, it ends when the ser | The loudest and usually the least serious, which is exactly why it absorbs attention disproportionate to its place in the register. |
Rank it first
Now put the list in order, and do it before anybody has described a single
attack.
- expected loss from one asset
- plausibility that it goes wrong at all
- cost if it does
The ordering matters more than the numbers, and the reason to do it before
thinking about attacks is a bias worth naming. Once a clever attack is in
your head, every asset it touches feels valuable, because the vividness of
the attack transfers to the thing it reaches. Teams reliably overrate the
asset at the end of the most interesting chain and underrate the dull asset
that is simply sitting there in a bucket with the wrong permissions.
Fixing the cost half of the product first, while the attack half is still
blank, keeps it honest. When the threats arrive in the next three lessons,
they are multiplied against an ordering that was made for its own reasons,
and the two disagree in useful ways.
What you should have at the end of this is one page, fifteen or twenty
lines, with three scores and an owner of the harm against each. It takes
about twenty minutes and it is the input to everything that follows. The
next lesson draws the system those assets live in, which is where the
findings actually come from.
Recap
- A list of threats cannot be prioritised without a list of assets, so the exercise starts with what you hold rather than with what could go wrong.
- Every asset can be lost in three distinct ways, and the same asset usually matters very differently under each.
- Rank the assets before you have thought about any attack, because once an attack is in your head the ranking quietly becomes a ranking of how clever the attack was.
This is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents