Answers · Where the data lives
If SQL is declarative, who decides how the join actually gets run?
In shortThe database's query planner chooses the join method and order; your SQL only says which rows the answer must contain.
The database's query planner decides. It takes your JOIN as a statement about which rows must appear, then chooses an access path for each table, a join method for each pair, an order for the joins, and where sorting and grouping happen. None of those choices changes the answer. All of them change the time.
The text is a specification
An ordinary query asks for orders placed last month by customers in one country, with the customer name, sorted by date. The text fixes the tables, the conditions, the relationship, the columns returned and the output order. It does not say to use an index on the country column, or to start from customers, or to combine with orders by nested loops, or to sort at the end rather than walking an index that is already in date order. Those are method questions. The query leaves every one of them open, so the database has to settle them before it reads a row.
Read a query by asking what it did not say. For each table there is a choice between a scan and a seek into one of its indexes, and if several indexes apply there is a choice between them. For each pair of inputs there is a choice among join methods with different cost shapes. For the whole set of tables there is an order. For a sort or a group there is a place in the plan to do it, and an input already in the right order can make it free. The SQL names the result; the planner names the route.
How many routes there are
Here is the number of tables, is the number of join methods, three in most systems, and is the number of access paths per table, a scan plus the usable indexes. Three tables with two access paths each gives plans. Six tables gives . Work that out by hand: , then plans. The factorial dominates. The planner cannot enumerate that in a few milliseconds, so it searches and prunes, which means the best plan is sometimes never considered.
The natural guess is that writing JOIN fixes the join. It does not. JOIN says the two inputs are related and how; it does not say nested loop, hash or merge. The planner compares plans by estimated cost, not by running them, so it chooses the cheapest estimate. That distinction matters: the planner can choose correctly over numbers that are badly wrong. The numbers come from statistics about the tables, and statistics are a summary, not the data.
Where the estimate fails
Suppose the statistics say last month's orders are a small slice when they are 4.1 million rows. The planner may start from the large side and scan. One plan starts from the small filtered side, reaches the orders by index seek, then filters to the month. Another starts from all 4.1 million orders, hashes them against the customer table, and only then filters to the country. The first reads 11,400 rows in total. The second reads 8,300,000. Same query, same answer, same database.
Where this stops being true is not in the language but in the assumptions. If the statistics are fresh and the columns independent, the estimate can be close. If the data is skewed, if two conditions are correlated, if the value is supplied as a parameter, or if the statistics are stale, the estimate can be off by orders of magnitude. The planner also prunes: it does not consider every plan, so a good plan may be missed. And the fix depends on the cause. A missing index changes the access paths; stale statistics change the estimates; a query rewritten to expose a filter can change the join order. Adding an index to a query whose estimate is wrong can make the plan worse.
The declaration ends at the answer. The method is the planner's, and the plan is its guess at the cheapest way to produce that answer from the statistics it has. When the guess is wrong, the query is not a little slower: the same text can read hundreds of times more rows.
Worth keeping
- SQL specifies which rows the answer contains, not the access path, join method, join order or sort placement.
- The planner picks the cheapest estimated plan, so a wrong row estimate can make a sensible join method look correct.
- The same query and the same data can read 11,400 rows under one plan and 8,300,000 under another, a factor of just over 728.
Where this goes next
This question is one step of What the Planner Decides, a course that works through its subject from the beginning. The page that teaches this part of it is What You Did Not Say, and it reads the whole way through whether or not you have an account.
What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.
Read that pageThe whole courseOther questions people ask about this
- How does the database estimate how many rows a WHERE clause will return?It does not scan the table: it uses stored summary statistics to turn each filter into a fraction of rows and multiplies the fractions.
- Why does the planner underestimate rows when I filter on two columns that are related?It multiplies the two filter shares as if the columns were unrelated, so when one column already implies the other the estimate comes out far too low.
- Why does the order I join tables in matter if joins are commutative?The final result is the same, but the rows carried between joins are not, and that is what the work is paid on.
- Will refreshing statistics fix a bad query plan?Only if the summary was taken before the data changed; otherwise refreshing re-measures the same assumptions and the plan stays wrong.
- Why is the date off by one day after I save it and read it back?You saved a calendar date as if it were a precise moment, so showing it in another time zone can move it to the day before or after.
- Is UTF-8 the same thing as Unicode, or what's the difference?No: Unicode assigns a number to each character, while UTF-8 is a way of writing those numbers as bytes, using one to four bytes per character.