Not Running Is Two Different Problems With One Symptom

Last timeTaking Turns

A program that is not running is either waiting for a turn or waiting for a device, and the fixes are opposites. Here is what each one does to the numbers you can see from outside.

Two reasons, opposite fixes

A program takes four seconds to do something that should take one. From inside

the program, the picture is the same in both of the cases below: a line of code

took longer than expected.

The two cases are these. The program was ready to run and the processors were

busy with other work, so it waited its turn. Or the program was not ready to

run at all, because it had asked for something and the answer had not arrived.

These have opposite remedies. The first is a capacity problem: there is more

demand than machine, and the answer is less work, better work, or more

processors. The second is a latency problem: nothing is short of capacity, the

program is waiting on a disk or a network or another process, and adding

processors changes nothing whatsoever. Doubling the machine for a program in

the second state is the single most common wasted response to a performance

problem.

So the diagnosis cannot come from inside. A timer around a slow line reports

elapsed time, which includes both, and tells you nothing about which it was.

FIG 1The three things wall-clock time is made of

: elapsed time, which is what your stopwatch measures
: time actually running on a processor
: time ready to run but not chosen, because the processors were busy
: time blocked, waiting for something that had not happened

Every performance investigation is the question of which of the three terms on the right grew. The system can tell you the first directly, since it charges processes for it. The third is usually inferred from what the process was waiting on. The middle term is the one nobody measures by default and the one most often misattributed, because a process that is merely waiting its turn looks exactly like a process that is slow.

What blocking is

Blocking is not a state of nature; it is a specific sequence of things the

kernel does. Worth having exactly, because the next lesson is about avoiding

it.

Your program makes a request. The kernel discovers it cannot be satisfied

immediately: the data is not in memory, the socket has nothing in it, the lock

is held. So it puts the process on a list of things waiting for that particular

event, marks the process not runnable so the scheduler stops considering it,

and switches to somebody else. Your process is now not a candidate for the

processor at all, and the scheduler from the previous lesson never sees it.

Later, something happens. A disk controller raises an interrupt, a packet

arrives, another process releases the lock. The kernel finds the processes

waiting on that event, marks them runnable, and they rejoin the comparison from

the previous lesson, where, having accumulated almost no charged time, they are

chosen almost immediately. Your process resumes execution in the middle of the

request it made, and returns from it with an answer. From inside, one line of

code took eight milliseconds.

FIG 2The states, and what moves a process between them

Trace the two paths out of running into waiting and notice they differ only in whether signals are delivered. That single difference accounts for the whole experience of a process that cannot be killed: it is not refusing, nobody is delivering the signal, and it will leave the state when the device replies or never. Also notice there is no arrow from a waiting state directly to running, because a woken process has to be chosen like everybody else.

Telling them apart from outside

The system records enough to attribute a slowdown, and the trick is knowing

which numbers move for which cause.

The lesson stops here

2 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents