Answers · Keeping it safe
Isn't threat modelling just brainstorming what could go wrong?
In shortNo. It is a fixed checklist of six questions asked at every boundary, which gives a list with a known shape and a record of what was examined.
No. Threat modelling is a fixed set of six questions asked at every boundary on a drawing, and the output is an unranked list with a known shape plus a record of what was examined. A room full of people saying what worries them is a different activity, and it produces a different kind of result: findings without a coverage claim.
At each boundary the questions are about the line, not the components: could somebody pretend to be a party they are not; change what crosses in flight; do something and then deny it; read what crosses or what is stored; stop it working; or end up with more permission than was granted. Each question gets one of three answers. Nothing, with a short reason. A finding, written in five fields. Unknown, with the name of somebody who would know. The three answers matter because a pass that only records findings runs out of patience at a later boundary. A pass that records nothings as well produces a statement of coverage, and unknown is a better answer than an invented one in either direction.
Take a drawing with eleven boundaries. Eleven times six is 66 questions. At an average of one and a half minutes per question, 66 times 1.5 is 99 minutes, which is 1 hour 39 minutes. At three quarters of a minute, realistic when the answer is a nothing with a short reason, 66 times 0.75 is 49.5 minutes. The arithmetic is worth doing before anybody agrees to the exercise, because the objection is always time and the number is usually under two hours. Most answers will be nothing. The value sits in the small number that are not.
| step | question | answer | what an attacker gets | where it goes | what happened |
|---|---|---|---|---|---|
| 1 | pretend to be another party | nothing: the callback is signed with a per-merchant secret | nothing | nothing, with the reason recorded | A nothing with a reason. If the secret stops being per merchant, this line is where the assumption breaks. |
| 2 | change what crosses in flight | finding: the amount is outside the signed body | an order for goods at a lower price than paid | finding, five fields | Something is signed, so everybody assumed everything was. The finding names the part that is not covered. |
| 3 | deny having done it | finding: nothing records which operator approved a refund | a refund with no attributable approver | finding, five fields | The category people skip, and the finding is invisible until somebody has to investigate. |
| 4 | read what crosses | nothing: the channel is encrypted and the body carries no card data | nothing | nothing, with the reason recorded | A short reason that a reader who knows better can challenge. |
| 5 | stop it working | unknown: nobody present knows whether the callback endpoint is rate limited | possibly a cheap outage | unknown, named owner | Unknown rather than a guess. It becomes a question with an owner. |
| 6 | end up with more permission than granted | nothing: the callback can only mark one order paid, and the order id is checked against the merchant | nothing | nothing, with the reason recorded | The check is specific, so the nothing is specific. |
The mistake
The natural guess fails at the step that feels most productive. A brainstorm returns what the room already knows. The person who spent last quarter on authentication raises authentication; the engineer who was on call during the outage raises availability. Those are real concerns, and they were already concerns before anybody sat down, so the meeting has produced no new information at considerable expense. What a brainstorm cannot give is a statement of what was looked for and found absent. After a brainstorm the team knows what it found. After a fixed pass it knows what it examined, and the second claim is the one that survives a redesign, an audit, or a change of personnel.
Writing one down
A finding written badly cannot be ranked, which wastes the work at the last step. Five fields, and no adjectives. Name the boundary as it appears on the drawing. Name the category, one of the six. State what an attacker obtains as an outcome rather than as a mechanism: an order for goods that were not paid for, not a missing signature check. State what would have to be true for it to work, which is where the attacker list matters, because the answer might be that somebody holds a valid client key. State how anybody would notice, which is often nothing, and that is itself worth recording. Serious, critical and high risk are banned from the list. Ranking happens later with numbers, and a finding that arrives pre-ranked by its author distorts the comparison.
The procedure is dull on purpose. Take the next unexamined boundary, read aloud what crosses it, and ask the six questions in order. Reading aloud matters because somebody will say that is not all it carries, and that sentence is where findings come from. Do not fix anything during the pass. The moment an editor opens, the pass stops, the remaining boundaries are never examined, and the fix is chosen before its severity has been compared with the others. Do not rank during the pass either. A finding that sounds trivial on its own is sometimes the second step of a chain, and the chain is invisible until the list is complete.
Where it stops
The checklist is not a theory and not a proof. Its coverage is only as good as the boundary list. If a component is missing from the drawing, six questions at the boundaries that remain will not find it. The six categories are a list that has survived use, not a guarantee that every attack has been imagined. And the nothings have to be written down with their reasons. Without those lines there are findings but no statement of coverage, which is the thing the exercise was supposed to buy. Unknown is legitimate and better than a guess, but it becomes a question with an owner, not a finding.
Worth keeping
- A brainstorm finds what the room already worries about; a checklist finds what nobody thought to worry about.
- Six questions at every boundary produce coverage that can be stated, not just findings that can be remembered.
- A finding needs the boundary, the category, what an attacker gets and what must be true, or it cannot be ranked.
Where this goes next
This question is one step of Thinking Like an Attacker, a course that works through its subject from the beginning. The page that teaches this part of it is Going Through It Systematically, and it reads the whole way through whether or not you have an account.
What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.
Read that pageThe whole courseOther questions people ask about this
- Is it actually OK to look at a risk and decide not to fix it?Yes, if the decision not to fix it is written down with a reason, a named owner and a date or trigger for looking again; otherwise it is just an oversight.
- How does text I saved in a comment end up running as JavaScript in someone else's browser?The comment stayed text in storage; a page that later pasted it into its own markup made the browser read it as code.
- Why does escaping the quotes in user input not stop SQL injection?Escaping quotes only cleans one context; SQL injection stops only when query text and values travel separately, so the value is never parsed as code.
- Why isn't encrypting the password enough to keep it safe if the database leaks?Because encryption can be undone by your own login code, so the key that lets you check a password usually sits near the database, and anyone who copies both can read every password.
- How do I stop one service from reading another service's password?Give every service its own identity, grant only the credentials it uses, and test that it cannot read another service's password.
- Why does storing a random salt with each hash stop rainbow tables?Because the salt is different for every account, a table built once from common passwords matches at most one row, so the work has to be redone account by account.