ContentsThe library

Memory, All the Way Down

One Allocator Adds a Number, the Other Goes Looking

Last timeLaying the Data Out

The two places a program puts things differ by a factor of a hundred in cost, by everything in failure mode, and the choice between them is about lifetime rather than speed.

The stack is one pointer

A register holds the address of the top of the current stack. Entering a

function subtracts the size of its local variables from that register, and

everything the function declares lives in the region that just appeared.

Returning adds the same number back.

That is the entire mechanism. There is no list of allocations, no record of

sizes, no search for a suitable gap, and no way for two threads to interfere,

because each thread has its own. Allocating a kilobyte and allocating a byte

cost exactly the same, which is one instruction.

The addresses it produces are the best possible ones for everything in the

last three lessons. Successive frames are adjacent, the memory was almost

certainly used moments ago by a previous call, and so it is already in cache

and already translated. Stack memory is fast in two different ways and the

second one is larger.

What the mechanism cannot do is free out of order. The only operation is

moving the top, so the thing allocated last is freed first, always. If

something must outlive the function that created it, the stack cannot hold

it, and this is not an implementation limit that a better design could remove.

It is what makes the addition and subtraction sufficient.

What the heap costs you

A general allocation call has to do four things the stack does not.

FIG 1What a general allocation does
Five things that can happen, against one addition on the stack. The fast path is perhaps twenty to fifty nanoseconds and the slow path can be hundreds, and nothing in your code says which one you are about to take.
FIG 2The two places, compared
nanosecondshidden bytes per allocatfreed in reverse order ocan fail while memory is
stack frame1010
heap, fast path301601
heap, slow path3001601
A factor of three hundred between the marked figures, but the two right-hand columns are the ones that decide the design. The stack is restricted and cannot fail in an interesting way; the heap is unrestricted and can fail in a way that depends on the program history.

There is a third cost that does not fit in the table. Heap addresses are

unrelated to each other. Two objects allocated one after another may be

adjacent or may be in different regions entirely, depending on what was freed

earlier, so a data structure built from many small heap allocations is

scattered by construction. Everything in the last three lessons then works

against you: poor block use, no prefetching, and a traversal that is a chain

of dependent misses.

The lesson stops here

3 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the planThe contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The rest of this course

  1. 01Two Programs Can Hold the Same Address and Never Collide
  2. 02A Few Hundred Entries Decide Whether Your Program Falls Off a Cliffopening only
  3. 03Most Faults Cost a Microsecond and One Costs Ten Thousandopening only
  4. 04If a Cache Hit Took One Second, Main Memory Would Take Four Minutesopening only
  5. 05You Asked for Four Bytes and Sixty-Four Arrivedopening only
  6. 06Predict the Speedup on Paper Before You Change a Lineopening only
  7. 07One Allocator Adds a Number, the Other Goes Lookingyou are here
  8. 08Two Threads, No Shared Variables, and One of Them Is Ten Times Sloweropening only

Read alongside