How Many Groups Are There

Last timeGrouping Around Centres

The number of groups has to be argued for, and the usual argument is a bend in a falling curve, which is vague, so the better one compares the data against the same data with its structure destroyed.

The question the previous lesson assumed away

The method needs a number of groups before it starts. Where does the number

come from?

Sometimes from outside the data, and when it does, this lesson is

unnecessary. Marketing can run four campaigns, so there are four groups. A

tray holds six compartments. That is a legitimate answer and usually the best

one available, because it ties the grouping to what it is for.

When there is no such constraint, the number has to be argued for from the

data, and the standard argument is weaker than it looks.

The curve that always falls

Run the method for every number of groups from one to twelve and plot the

total within group spread against the number.

The curve falls. It always falls, because allowing one more group can only

reduce the total, and at the extreme where every point is its own group the

total is zero. So the direction of the curve says nothing whatsoever.

FIG 1The same curve on data with four groups and on data with none

four real groupsno structure at all

Both fall, both flatten, and neither reaches zero on this range. The one with real groups falls faster early and flattens sooner, which is the only difference, and it is a difference in curvature rather than in anything a single curve shows. This is why the second curve has to be put on the chart, and why the usual practice of showing only the first proves nothing.

Looking for the bend

The standard argument is to find the bend: the number of groups after which

the curve stops falling steeply. Before the bend, each extra group is buying

a real division. After it, each extra group is cutting an existing group in

half for a modest gain.

The reasoning is sound and the instrument is not. The bend is a visual

judgement about a smooth curve. It moves if the vertical scale changes. Two

people reading the same chart choose different numbers, and on real data the

curve very often has no visible bend at all, just a steady flattening.

The lesson stops here

5 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents