Basis Lab

Markov Chains

Where You Are After Many Hops

Last timeWhere You Are, and One Hop

Two hops means adding up every route through a middle situation. That sum is exactly matrix multiplication, so the table for any number of hops is the one hop table raised to that power.

The table says what happens in one hop. Almost no question anybody actually has

is about one hop. What is the chance the machine is running a fortnight from

now, what fraction of readers reach the checkout page eventually, how long

before this process has forgotten where it started: all of these are questions

about many hops.

It turns out that the many hop question needs no new ideas at all. It needs the

one hop table and one piece of arithmetic, applied repeatedly.

Two hops, by listing the routes

Start with two. To get from a situation to another in exactly two hops, the

process has to be somewhere in between, and the list of candidates for that

somewhere is the list of situations, which is finite and known.

Fix one candidate. The chance of taking that particular route is the chance of

the first hop multiplied by the chance of the second, because both have to

happen and, by the memoryless assumption, the second hop's chance depends only

on where the process is at the middle moment and not on where it came from.

Now notice that the different candidates are mutually exclusive: at the middle

moment the process is in exactly one situation. So the chances of the routes

add. The two hop chance is a sum of products, one term for each situation.

FIG 1Every route from waiting to running in two hops
steproutefirst hopsecond hopchance of the route
1waiting, then running, then runningwaiting to running is impossible, chance noughtrunning to running, nine in tennought
2waiting, then under repair, then runningwaiting to repair, three in tenrepair to running, one in twothree in ten times one in two, which is fifteen in a hundred
3waiting, then waiting, then runningwaiting to waiting, seven in tenwaiting to running is impossible, chance noughtnought
4all routes togetherthree candidates for the middle situation, no othersthe list is exhaustive because the process is somewherenought plus fifteen in a hundred plus nought, which is fifteen in a hundred
4 steps
The machine from the previous lesson. Two of the three routes contribute nothing, and they still have to appear in the list: knowing that a term is zero is a result of the calculation, not a reason to skip it. The answer, fifteen in a hundred, is the entry in the bottom left of the two hop table, and it says something the one hop table could not: a machine waiting for a part can be running the morning after next, even though it cannot be running tomorrow.

The route sum is matrix multiplication

Write the sum down in symbols and look at what it is doing. It runs along the

row of the starting situation and down the column of the destination,

multiplying the pairs it meets and adding the products. That is the definition

of multiplying two matrices together, with the table playing both parts.

The lesson stops here

4 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the planThe contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The rest of this course

  1. 01Where You Are, and One Hop
  2. 02Where You Are After Many Hopsyou are here
  3. 03What Is Reachable, and What Traps Youopening only
  4. 04The Distribution the Rule Leaves Aloneopening only
  5. 05Why the Chain Forgets Where It Startedopening only
  6. 06How Long Until It Gets Thereopening only
  7. 07Chains That Look the Same Run Backwardsopening only
  8. 08Building a Chain to Reach a Chosen Distributionopening only

Read alongside