Basis Lab

Answers · Making it fast enough

How does merge sort get n log n by splitting and merging?

In shortSplitting halves the number of runs each round, so there are about log base 2 of n rounds; merging every item once per round gives n log n.

The nlog⁡nn\log n comes from two separate counts multiplied together: the number of halving rounds is log⁡2n\log_2 n, and each round merges every item once, which costs nn. Splitting is only how you arrange the rounds; the work is in the merges.

The merge is a single pass

Take two runs that are already in order, say 2,4,82,4,8 and 1,5,91,5,9. Look only at the two front items. The smaller front item is the smallest item not yet output, because everything behind it in its own run is at least as large. Take it, advance that run's pointer, and repeat. Each comparison places exactly one item, so merging runs with total length mm takes at most mm placements and at most m−1m-1 comparisons. That is linear in mm, not logarithmic. The final tail of a run is appended without any comparison, which is the only saving and the reason the cost is stated as at most the combined length.

FIG 1Merging two ordered runs
stepfront of the left runfront of the right runtakenoutput so farwhat happened
12111compare 2 with 1, take 1
22521 2compare 2 with 5, take 2
34541 2 4compare 4 with 5, take 4
48551 2 4 5compare 8 with 5, take 5
58981 2 4 5 8compare 8 with 9, take 8
6none left99, then the rest1 2 4 5 8 9left run empty, append the rest
6 steps
Six items are placed in five comparisons. The final tail is appended without comparison, which is why the linear cost is stated as at most the combined length.

Counting the rounds

Start with nn runs of length 1. After one round there are n/2n/2 runs of length 2; after two rounds n/4n/4 runs of length 4; after kk rounds n/2kn/2^k runs of length 2k2^k. The merging stops when one run remains, which needs 2k≥n2^k \ge n, so k=⌈log⁡2n⌉k=\lceil \log_2 n\rceil. For eight items, 8→4→2→18\to4\to2\to1 is three rounds. For a million, 219=524,2882^{19}=524{,}288 and 220=1,048,5762^{20}=1{,}048{,}576, so the number of rounds is 20.

C=n⏟one pass×log⁡2n⏟rounds=nlog⁡2nC = \underbrace{n}_{\text{one pass}} \times \underbrace{\log_2 n}_{\text{rounds}} = n\log_2 n
FIG 2
roundruns at the startlength of each runitem placements in that round
start810
1818
2428
3248
total1824

At a million items the same arithmetic gives 20 rounds and 20×1,000,000=20,000,00020 \times 1{,}000{,}000 = 20{,}000{,}000 item placements. The worst case for comparisons is the same order. Comparing every pair would take 1,000,000×999,999/2=499,999,500,0001{,}000{,}000 \times 999{,}999 / 2 = 499{,}999{,}500{,}000 comparisons, which is about 25,000 times larger. The gap is not a constant improvement; it is the difference between a line and a curve that bends upward.

Why the log is not in the merge

The usual wrong picture is that splitting costs log⁡n\log n and merging costs something similar, or that merging two halves is logarithmic because it sounds like binary search. Merging is not searching. A binary search halves one run, but a merge cannot discard half of a run: the smallest remaining item might be at the front of either run, so both fronts must be watched. Watching two fronts and advancing one pointer per output is a single walk over the combined length. The logarithm is purely the number of times you must repeat that linear walk. If you split and then merge by scanning one half for each item of the other, the merge itself becomes quadratic, and the total is no longer nlog⁡nn\log n.

Where it stops being true

This is a comparison sort bound. It assumes the only way to learn about an item is to compare it with another. If you can inspect the key, as with digits or small integer ranges, you can sort in linear time per digit or per range and beat nlog⁡nn\log n asymptotically. The bound is also asymptotic: at ten items the constants and the cost of allocating half-arrays can dominate, and an insertion sort may be faster. The balanced halving matters too. Splitting at the midpoint gives the log factor; a merge that repeatedly scans from the beginning, or an unbalanced split that leaves one huge piece every time, loses the guarantee. Array merge sort also needs O(n)O(n) extra space for the merged output, which is a real cost even when the comparison count is optimal.

Worth keeping

  • A merge of two ordered runs with total length m takes at most m placements and m minus one comparisons, because each comparison places one item.
  • The number of halving rounds needed to reduce n items to single items is the base-2 logarithm of n, and each round touches every item once.
  • No comparison sort can beat n log n asymptotically: c yes-or-no comparisons distinguish at most two to the c orders, while there are n factorial possible orders.

Where this goes next

This question is one step of Keeping Things in Order, a course that works through its subject from the beginning. The page that teaches this part of it is Putting It in Order in the First Place, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. Why do databases use B-trees rather than binary search trees for indexes?Because a database index pays for disk blocks read, not comparisons made, and a wide shallow tree needs only a few block reads per lookup.
  2. How does a heap know the smallest item without keeping the whole collection sorted?A heap only keeps each parent no larger than its children, so the top is always the smallest even though the rest is not sorted.
  3. Why does binary search only work if the list is sorted?Because only in a sorted list does one comparison against the middle tell you that half the items cannot be the one you want.
  4. Why does inserting already sorted numbers turn a binary search tree into a linked list?Because each new number is bigger than every number already stored, so it is always attached down the right side, and the tree becomes a chain as deep as the number of items.
  5. Why doesn't breadth-first search give the cheapest path once the links have different costs?Breadth-first search counts links rather than costs, so it can return a route with fewer links that is more expensive than a longer route.
  6. Why is O(2n) treated the same as O(n)?Because it describes how cost grows rather than exact cost, doubling every cost does not change the growth class.

Read alongside