Letting a Model Call Your Code Safely
Five Ceilings, and the Twenty Cores They Imply
Last timeTaking the Privileges Away
Processor time, wall clock, memory, writing and processes each get a number, each one stops a different failure, and together they decide how many cores the service has to own.
Taking the privileges away closes the attempts that needed a permission. Five
of the ten remain, and they are the five that need no permission at all:
filling the disk, making processes without stopping, spinning the processor,
taking memory until the host suffers, and leaving something running after the
answer has gone back. Every one of them is built out of the same sixty calls
the job itself cannot work without.
So they cannot be refused. They can only be metered.
| meter | the number | the failure it prevents |
|---|---|---|
| processor time | 2 seconds | a submission that spins, holding a core until somebody notices |
| wall clock | 10 seconds | a submission that sleeps or blocks, holding its slot while spending nothing |
| memory | 256 MB | a submission that allocates until the host starts refusing other people's allocations |
| writing | 64 MB | a submission that fills the disk, which takes the whole host down rather than one slot |
| processes | 32 | a submission that makes processes faster than anything can reap them |
Five meters, not one
The first two look like one meter written twice, and they are the pair worth
being careful about, because they catch opposite failures. A submission that
spins reaches 2 seconds of processor time in about 2 seconds of wall clock, so
the processor ceiling fires first and the clock never matters. A submission that
blocks on reading from something that will never produce anything spends perhaps
a hundredth of a second of processor time and would sit there until the service
was restarted. Nothing but the clock on the wall can end it.
Which of the two fired is also the most useful single fact in the log. One says
the submission was computing and the other says it was waiting, and those are
different conversations with whoever submitted it.
The last three are not equally serious, and it is worth being clear about why.
Memory and processes are spent inside the slot, so a submission that abuses
either one mostly ruins its own run. Writing is different. A full disk is not a
property of one slot, it is a property of the host, and it stops every other
reader on that host as well as the service's own logging, which is the thing
that would have told you about it. Of the five numbers, the 64 MB is the one to
set tightest and to express as the size of a mount rather than as a counter
anybody has to check.
One more decision hides inside the wall clock, which is when it starts. If it
starts when the request arrived, then a submission that sat in a queue for 9
seconds because the service was busy is refused for the service's own
congestion, which is a refusal the reader cannot act on and should not be sent.
The clock starts when the submitted program starts. Time spent queuing is
measured too, and it is a capacity problem rather than a submission problem.
Where the numbers come from
None of the five numbers above is a round number chosen because it sounded
safe. Each one is a ratio to the honest workload, and the ratio is the thing to
write down, because it is what you defend when somebody asks for more.
Measure the honest submissions first. Suppose the slowest one in a hundred
finishes in 0.2 of a second of processor time and holds about 40 MB. Then 2
seconds is ten times the processor headroom an honest submission needs, since 2
divided by 0.2 is 10, and 256 MB is a little over six times its memory, since
256 divided by 40 is 6.4. A ceiling set at ten times the honest extreme is
generous enough that no honest reader ever meets it and tight enough that a
hostile one is stopped within a couple of seconds.
The ceiling has to sit in the gap between two numbers: above the worst honest
submission by enough that nobody legitimate is refused, and below the point at
which a hostile submission hurts the host. If no such gap exists, the thing that
is wrong is the job, not the ceiling.
The lesson stops here
6 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents