Why does a planet speed up when it gets closer to the Sun?

A planet speeds up as it approaches the Sun because the product of its distance from the Sun and the sideways component of its speed is fixed, so a smaller distance forces a larger sideways speed. Gravity also does work on the planet as it falls inward, so the total speed rises even when the sideways component alone does not account for all of it.

Write the planet's position from the Sun as $(x,y)$ and its velocity as $(v_x,v_y)$. The combination $x\,v_y - y\,v_x$ is large when the planet is moving across the line to the Sun and zero when it is moving straight at or away from the Sun. For a planet with sideways speed $v_\perp$ at distance $r$, this reduces to $L=m\,r\,v_\perp$, because the sideways velocity is $r$ times the rate at which the angle changes. A force aimed at the Sun has no sideways component at the planet's position, so it cannot change $L$. If $L$ is fixed and $m$ is fixed, then $r\,v_\perp$ is fixed. Halve $r$ and $v_\perp$ must double. The rate of turning, $d\theta/dt = v_\perp/r$, goes as $1/r^2$, so it rises by four when $r$ halves.

The natural guess is that a pull toward the Sun should only produce motion toward the Sun. That guess treats the planet as starting from rest. It does not. The planet already has sideways motion, and the pull is at right angles to that sideways direction, so it bends the path without directly cancelling the circulation. What changes the sideways speed is not a sideways force; it is the shrinking of $r$ in the fixed product $r\,v_\perp$. If the Sun were switched off, the planet would fly in a straight line at constant speed. With the Sun on, the inward fall converts stored gravitational energy into kinetic energy, so the total speed rises, but the sideways part is forced up by geometry as well.

Angular momentum alone does not give the whole speed at every point on an ellipse. It gives the sideways part. The radial part is set by energy. At the two ends of the orbit the radial speed is zero, so the total speed is the sideways speed, and the comparison is clean. Take an orbit with aphelion at $r=1$ and perihelion at $r=0.5$, with $m=1$ and $L=0.8$. At aphelion $v_\perp=0.8$; at perihelion $v_\perp=1.6$. The kinetic energies are $0.5\times0.8^2=0.32$ and $0.5\times1.6^2=1.28$. The difference is $0.96$. Matching the two ends with a potential $-\mu/r$ gives $0.32-\mu=1.28-2\mu$, so $\mu=0.96$. The potential energies are $-0.96$ and $-1.92$, and the totals are both $-0.64$. Gravity has done $0.96$ of work, which is exactly the rise in kinetic energy.