Why can a higher hit rate still mean the cache is making things slower?

A higher hit rate can make things slower because the hit rate counts events, not work saved. It gives a hit on a two-second answer the same weight as a hit on a tenth-of-a-millisecond answer, and it says nothing about the cost the cache adds to every read.

The average read costs $p h + (1-p)(h+c)$ with the cache and $c$ without it. Subtracting gives $p c - h$ per read. So the cache helps only when $p c$ exceeds $h$, and the size of $p$ alone cannot settle it. Suppose $c = 10$ ms and $h = 12$ ms. A hit costs 12 ms, a miss costs 22 ms, and at $p = 0.90$ the average read costs $0.90 \times 12 + 0.10 \times 22 = 10.8 + 2.2 = 13$ ms. Going straight to the store costs 10 ms. The cache is 3 ms slower per read at a 90 per cent hit rate. At 500 reads per second that is $500 \times 3 = 1500$ ms, or 1.5 seconds of extra work every second.

There is a second way a high hit rate misleads. A hit is fast because it is a hit, so it cannot be in the slowest requests. Every request in the tail went to the store. Moving from 80 to 95 per cent can leave the 99th percentile unchanged, because the remaining 5 per cent are the only requests that were ever slow. Raising the hit rate changes how many reads are slow. Improving the miss path changes how slow a miss is. Those are different projects.

If every miss costs the same $c$ and the lookup overhead $h$ is negligible, then $p c - h$ is ordered by $p$, and a higher hit rate does mean more work saved. The ordering also holds if you compare caches on the same mix of answers and the same $h$, because $c$ is then a common factor. It fails when the mix changes, when the cache is consulted on every read, or when the hit rate rises by keeping cheap answers and evicting expensive ones. Then the high hit rate can be a record of many small savings and one large loss.