Turning Text Into Instructions
Your Instructions Arrive in Order and Are Executed in Whatever Order Suits
Last timeToo Few Places to Put Things
The compiler rearranged your program and then handed it to a processor that rearranges it again at run time. The second reordering is invisible until a second thread is watching.
Not the order you wrote
Seven lessons have followed your program from text to machine instructions. The
last transformation happens after all of that, at run time, inside the
processor, and it is the largest rearrangement of the lot.
The instructions arrive in order. They are not executed in order. A modern
processor fetches a long way ahead, keeps a hundred or more instructions in
flight at once, and starts each one at the moment its inputs exist and a
suitable unit is free. An instruction whose input is still coming from memory
sits and waits while a dozen instructions written after it run to completion.
What stops this from being chaos is the last step. Results are held aside as
they are produced and made official in the original program order. Nothing
becomes visible out of sequence, so from the point of view of the thread doing
the work, the program ran exactly as written.
Why it is worth doing
The cost of this machinery is large. A substantial share of the transistors on
a processor, and of its power budget, goes to keeping instructions in flight
and sorting out their order. It is worth it because of one number.
- cycles lost to waiting
- number of times the program stops to wait
- cycles each wait costs
Lesson eight of the previous course on this shelf established the number: an
access that reaches main memory costs a few hundred cycles. In an in-order
machine everything behind that access stops for the duration, including
instructions that have nothing to do with it. There is almost always work
available that does not depend on the value being fetched, and out-of-order
execution exists to find it and do it during the wait.
The lesson stops here
5 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents