Answers · How models work

Why does a convolution make the image smaller if you don't pad it?

In shortA convolution shrinks the image because the window must fit completely inside it, so the first and last few positions have no full window. Padding adds extra values at the border so those positions can be used.

Because a window has to sit entirely inside the picture, the positions where it would hang over an edge are not valid placements. A row of 100 values with a three-wide window gives 98 outputs, not 100, because only starts 1 through 98 have three values available; starts 99 and 100 would need values beyond the row. The same count happens independently down the other axis, so a 32 by 32 image with a 3 by 3 window becomes 30 by 30.

n_{\text{out}} = \left\lfloor \frac{n_{\text{in}} + 2p - k}{s} \right\rfloor + 1

The formula is a count of starting positions. The numerator $n_{\text{in}}+2p$ is the width after padding, subtracting $k$ leaves the span over which a start can occur without the window overhanging, dividing by $s$ counts only the starts actually used, and the final $+1$ is the first start. The floor is not decoration: if the span is not an exact multiple of the stride, the last partial step has no complete window and is discarded. For $n_{\text{in}}=10$ , $k=3$ , $p=0$ , $s=2$ , the span is $10-3=7$ , so there are $\lfloor7/2\rfloor+1=3+1=4$ outputs, at starts 1, 3, 5, 7; a start at 9 would need values 9, 10, 11 and value 11 does not exist.

Counting complete windows

With stride one and no padding the count along one axis is $n_{\text{in}} - k + 1$ : the first start is at the first value, and every start after that moves the window one place. For $k=3$ and $n_{\text{in}}=100$ that is $100-3+1=98$ . In two dimensions the two axes are counted separately, so $32-3+1=30$ in each direction. Stack twenty such layers and each axis loses $20 \times 2 = 40$ positions: $224-40=184$ . Nothing has been thrown away by the arithmetic; there are simply fewer legal window positions.

FIG 1

input size	window	padding	stride	output size
32	3	0	1	30
32	3	1	1	32
32	3	1	2	16
32	5	2	2	16
32	3	0	3	10

Read the first row and the second row together. The first has no padding, so a 32 input with a 3 window gives 30. The second puts one value on each side, so the padded width is $32+2=34$ , and $34-3+1=32$ : the two lost positions are returned. The third row keeps that padding but moves two at a time, so the span is still 31 but only every second start is used, giving $\lfloor31/2\rfloor+1=16$ . The fourth row uses a five-wide window and padding of two, so the padded width is $32+4=36$ , and with stride two we get $\lfloor(36-5)/2\rfloor+1=\lfloor31/2\rfloor+1=16$ . The fifth row removes padding and moves three at a time: $\lfloor(32-3)/3\rfloor+1=\lfloor29/3\rfloor+1=9+1=10$ .

Padding is not only about size

The common mistake is to treat padding as a way of keeping the output the same size and nothing more. It does that, but it also changes how often each input value is read. With a $3\times3$ window and no padding, a corner value belongs to exactly one window, because only the window whose top-left corner sits on that corner can contain it. An edge value away from the corner belongs to three windows: the start row is fixed to the edge, and the start column can be one of three. An interior value belongs to nine: three possible start rows times three possible start columns. Those counts are 1, 3 and 9. The border is therefore contributing less evidence than the middle, and padding is what raises its count. Padding of one takes a corner from one window to four and an edge from three to six, which narrows the gap; full padding by $k-1$ would give every position the same count.

p = \frac{k-1}{2} \quad \Longrightarrow \quad n_{\text{out}} = n_{\text{in}} \;\text{when}\; s=1

Where the rule stops

The size-preserving value $p=(k-1)/2$ assumes an odd window and a stride of one. An even window has no centre position, so it cannot be padded symmetrically and the output is offset by half a pixel. A stride above one shrinks the map even when the padding is size-preserving: with $n_{\text{in}}=32$ , $k=3$ , $p=1$ and $s=2$ , the formula gives $\lfloor(32+2-3)/2\rfloor+1 = \lfloor31/2\rfloor+1 = 15+1=16$ . The floor is where a final partial step is dropped. If the stride does not divide the span, the last strip of the input is read by no window, so two implementations can differ by one pixel without either being obviously wrong. Zero padding is also a fabrication: a filter half over real pixels and half over zeros sees a strong edge that is not in the picture, which puts a bright frame around feature maps unless the padding is reflective or replicated.

Worth keeping

Without padding, a k-wide window has n_in - k + 1 full placements along an axis, so a 100-wide row with a 3-wide window gives 98 outputs.
Padding of (k-1)/2 restores the input size only for odd k and stride one; with stride two the same padding halves a 32-wide input to 16.
Zero padding does not just add empty space: it gives edge filters a fabricated strong edge, which trained networks can learn to use as a position signal.

Where this goes next

This question is one step of How a Filter Reads a Picture, a course that works through its subject from the beginning. The page that teaches this part of it is Edges, and Skipping Positions, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course