Basis Lab

Answers · Making it fast enough

Why is O(2n) treated the same as O(n)?

In shortBecause it describes how cost grows rather than exact cost, doubling every cost does not change the growth class.

Because big O describes the growth of cost, not the exact cost, and multiplying every cost by two does not change growth. Doubling every term moves the line up but leaves it a line. The rest is the definition doing exactly what it says.

T(n)=O(g(n))means∃ c>0,  n0  such that  T(n)≤c g(n)  for all  n≥n0.T(n)=O(g(n))\quad\text{means}\quad \exists\, c>0,\; n_0 \;\text{such that}\; T(n)\le c\,g(n) \;\text{for all}\; n\ge n_0.

The definition does it

Write the claim in full: T(n)=O(g(n))T(n)=O(g(n)) means there are a constant c>0c>0 and a size n0n_0 such that T(n)≤c g(n)T(n)\le c\,g(n) for every n≥n0n\ge n_0. The 22 in 2n2n is not sitting outside that inequality. It is one more number that can be folded into cc. To show 2n2n is O(n)O(n), take g(n)=ng(n)=n. Choose c=2c=2 and n0=1n_0=1. Then for every n≥1n\ge1, 2n≤2n2n\le2n, which is true by equality. The claim is satisfied, and nothing about the factor 22 has been denied.

Put numbers in and the scale is plain. At n=1,000,000n=1{,}000{,}000, a cost of 2n2n is 2,000,0002{,}000{,}000 operations and a cost of nn is 1,000,0001{,}000{,}000. The ratio is 22. A quadratic cost at the same size is 1,000,000,000,0001{,}000{,}000{,}000{,}000 operations, which is 500,000500{,}000 times the linear one: 1012/2,000,000=500,00010^{12}/2{,}000{,}000=500{,}000. The factor 22 is a difference of two; the shape is a difference of five hundred thousand. At the scale where growth classes are meant to be read, the second difference is the one that decides.

The natural guess is that O(2n)O(2n) records the 22 and O(n)O(n) does not, so the first must be a larger class. That treats the expression inside OO as if it were the running time itself. It is not. O(2n)O(2n) is not twice O(n)O(n); the notation does not multiply that way. Both expressions name the same set of functions, the ones bounded by a straight line for large enough nn. If the 22 were meant to survive, the question would be about the exact cost, or about which of two linear procedures is faster, and OO is deliberately blind to both.

The equality is not just one absorption; it is mutual. If T(n)=O(2n)T(n)=O(2n), then some cc and n0n_0 give T(n)≤c(2n)T(n)\le c(2n) for n≥n0n\ge n_0. Set c′=2cc'=2c. Then T(n)≤c′nT(n)\le c'n, so T(n)=O(n)T(n)=O(n). Conversely, if T(n)=O(n)T(n)=O(n) with constant cc, then T(n)≤c n=(c/2)(2n)T(n)\le c\,n=(c/2)(2n), so T(n)=O(2n)T(n)=O(2n). A factor of two can be moved from the function into the constant or from the constant into the function. That is why the two claims are the same claim.

T(n)=O(2n)⟺T(n)=O(n),T(n)≤c(2n)⇒T(n)≤(2c)n,T(n)≤cn⇒T(n)≤c2(2n).T(n)=O(2n)\quad\Longleftrightarrow\quad T(n)=O(n),\qquad T(n)\le c(2n)\Rightarrow T(n)\le (2c)n,\qquad T(n)\le c n\Rightarrow T(n)\le \frac{c}{2}(2n).

The factor is not unreal

None of this says the factor 22 is imaginary. It says the factor belongs to the implementation and the machine, and to the small inputs, rather than to the growth class. A procedure that does 2n2n work is twice as slow as one that does nn work at the same nn, if the constant is the same. On a list of ten items that is the difference between twenty operations and ten. If the input never gets large, that difference is the whole story, and asking for the growth class is asking the wrong question. The convention is a statement about the limit, not a promise about today.

A larger constant can even reverse the early winner. Compare 100n100n with n2n^2. At n=10n=10, 100n=1,000100n=1{,}000 and n2=100n^2=100, so the linear procedure is ten times more work. At n=100n=100, both are 10,00010{,}000. At n=10,000n=10{,}000, 100n=1,000,000100n=1{,}000{,}000 and n2=100,000,000n^2=100{,}000{,}000, so the quadratic procedure is one hundred times more work. The constant set the crossing point at 100100 and the shape controlled everything after it. Dropping constants is safe for the eventual comparison and unsafe for the early one.

Same shape, no answer

The other place the convention goes quiet is when two procedures have the same shape. If both are O(n)O(n), the growth statement rates them identically and is correct to: neither collapses as the data grows. It cannot tell you which to ship today. The only instrument left is measurement, because the difference is now entirely in the constant, in cache behaviour, in the language, and in the inner loop. A factor of two hundred between two linear procedures is possible and the OO notation will not show it. That is not a failure of the notation; it is the boundary of what the notation was built to answer.

Worth keeping

  • A constant factor moves the running time up or down but does not change how it grows with input size.
  • The definition of big O hides a constant, so a factor of two can be absorbed by changing that constant.
  • Constants decide small inputs and equal shapes; growth shapes decide large inputs.

Where this goes next

This question is one step of How Long It Takes, a course that works through its subject from the beginning. The page that teaches this part of it is Why the Constants Get Dropped, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. Which grows faster, n log n or n squared?The squared option grows faster: doubling the input multiplies the log-times-linear option by a little over two, but multiplies the squared option by four.
  2. Why do nested loops multiply their running times instead of adding them?Because the inner loop runs in full once for every pass of the outer loop, so the cost is one count times the other, not one plus the other.
  3. Why doesn't breadth-first search give the cheapest path once the links have different costs?Breadth-first search counts links rather than costs, so it can return a route with fewer links that is more expensive than a longer route.
  4. How does a hash function turn a word into an array index?A hash function reads the word as bytes, mixes them into one large number, then takes the remainder after dividing by the table size to get an index from zero to size minus one.
  5. How can you check whether something is in a set without storing every item?Record a few hashed marks per item in a bitmap; all marks set means probably present, any mark clear means certainly absent.
  6. What's the difference between memoized recursion and filling a DP table bottom-up?Memoised recursion keeps the calls and only solves states it reaches; a bottom-up table removes the calls, fills every entry in a dependency-safe order, and is usually faster when most entries are needed.

Read alongside