Answers · Making it fast enough
Which grows faster, n log n or n squared?
In shortThe squared option grows faster: doubling the input multiplies the log-times-linear option by a little over two, but multiplies the squared option by four.
grows faster. Doubling the input multiplies by a little over two, while it multiplies by four, and the gap widens at every larger size.
The comparison at real sizes
Take logarithms base 2, which is the usual convention for counting halving steps. At , , so . Meanwhile . The quadratic count is already about times larger. At , is about , so is about , while is . Dividing gives a ratio near .
| n | n log2 n | n squared | n squared divided by n log2 n |
|---|---|---|---|
| 8 | 24 | 64 | 2.67 |
| 64 | 384 | 4,096 | 10.67 |
| 1,024 | 10,240 | 1,048,576 | 102.4 |
| 1,000,000 | 19,931,569 | 1,000,000,000,000 | about 50,172 |
The last column is the answer at that size. At the quadratic count is only times the count, which is why small tests can look harmless. At it is times. At it is times. At a million it is about times. The ratio is , and that quotient has no ceiling: as grows, the numerator grows while the denominator creeps up by one each time doubles. This is why an method and an method can look comparable in a toy test and be in different worlds in production.
The natural guess
Because contains an extra logarithm, it is tempting to read it as the more decorated and therefore heavier expression. The decoration is not an extra factor of . Write the two as times something: is times , while is times . For every integer , , so . The logarithm grows very slowly: at a million, is still under , which is nothing beside the million in the other factor. A constant factor can hide this at small sizes, but it cannot change which expression eventually dominates.
The doubling ratio is the quickest mental test. For , replacing by gives , so the cost quadruples exactly. For , the same replacement gives , which is plus . The ratio is , so it is slightly more than and falls towards as grows. At it is . At doubling to : becomes , a ratio of , while becomes , a ratio of . A procedure that quadruples when the input doubles is pulling away from one that a little more than doubles.
The asymptotic statement is one directional. is because the ratio tends to . It is not true that is , since the reverse ratio tends to infinity. That is the formal version of the same arithmetic.
Where the ordering can reverse
The statement is about growth, not about a stopwatch at every size. At small , lower order terms and constant factors can make a quadratic procedure faster than an procedure. Insertion sort, which is quadratic in the worst case, often beats merge sort below a few dozen items because its inner loop is tiny and merge sort allocates and copies. The crossover is a measured number, and below it the worse growth shape can be the right choice. The same caution applies to the logarithm's base: changing from base 2 to natural log multiplies by a constant, which shifts timings but does not change the growth class. The formal order is still below ; only the measured winner at that size changes.
At a million rows, the procedure is still standing: its count is about million, while the procedure is at a million million. If a design choice is between the two, the crossover question is only relevant below the size where constants matter; above it, the side wins by the ratio .
Worth keeping
- Doubling the input multiplies an n log n cost by a little over two and an n squared cost by four.
- The ratio of n squared to n log n is n divided by log base 2 n, so it grows without bound.
- At small sizes constants can make the quadratic method faster, but that is a crossover, not a change in which curve eventually wins.
Where this goes next
This question is one step of How Long It Takes, a course that works through its subject from the beginning. The page that teaches this part of it is The Shape of the Curve, and it reads the whole way through whether or not you have an account.
What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.
Read that pageThe whole courseOther questions people ask about this
- Why is O(2n) treated the same as O(n)?Because it describes how cost grows rather than exact cost, doubling every cost does not change the growth class.
- Why do nested loops multiply their running times instead of adding them?Because the inner loop runs in full once for every pass of the outer loop, so the cost is one count times the other, not one plus the other.
- Why doesn't breadth-first search give the cheapest path once the links have different costs?Breadth-first search counts links rather than costs, so it can return a route with fewer links that is more expensive than a longer route.
- How does a hash function turn a word into an array index?A hash function reads the word as bytes, mixes them into one large number, then takes the remainder after dividing by the table size to get an index from zero to size minus one.
- How can you check whether something is in a set without storing every item?Record a few hashed marks per item in a bitmap; all marks set means probably present, any mark clear means certainly absent.
- What's the difference between memoized recursion and filling a DP table bottom-up?Memoised recursion keeps the calls and only solves states it reaches; a bottom-up table removes the calls, fills every entry in a dependency-safe order, and is usually faster when most entries are needed.