How can I tell if a prompt change actually helped or if I just got a lucky answer?

A prompt change helped only if it raises the pass rate on a fixed set of cases that you ran before and after, by more than the wobble of that set. One impressive answer after the change is not evidence, because that answer is the case the change was aimed at. The apparatus is one written file: thirty to a hundred inputs, each with a note saying what a good answer has to contain. The set must be fixed before either run, so the two runs are comparable, and it must include the awkward inputs, an empty one and an angry one among them, because ordinary traffic cannot detect a change. The same grading method is applied to both versions, whether a person reads the answers, a program checks a property, or a model checks each requirement.

The number of cases is set by the difference you want to see. For a rate near eighty per cent and a wobble of four points, put $a=0.8$ and $w=0.04$: $n=0.8\times0.2/0.04^2=0.16/0.0016=100$. For a wobble of two points, the same numerator is divided by $0.02^2=0.0004$, giving $0.16/0.0004=400$. The denominator is squared, so halving the wobble quadruples the cases. At twenty cases and a rate near eighty per cent the wobble is $w=\sqrt{0.8\times0.2/20}=\sqrt{0.008}=0.089$, which is 8.9 points. A five point improvement is inside that, so a twenty case set cannot see it. That is why a small set is a check for large breakages, not a measurement of small gains.

With a set of eighty cases, suppose fifty eight pass before and sixty four after. The rates are $58/80=0.725$ and $64/80=0.8$, a difference of $0.075$, or 7.5 points. The wobble on a rate near three quarters with eighty cases is $\sqrt{0.725\times0.275/80}=\sqrt{0.00249}=0.0499$, about 5.0 points. A difference of two measurements carries the noise of both, so compare 7.5 points with $\sqrt{5.0^2+5.0^2}=7.1$ points. Seven and a half against seven point one is suggestive and not settled. On a hundred cases at a rate near eighty per cent the wobble is 4.0 points, and a difference of two such runs needs about $1.4\times4.0=5.7$ points; an eight point gain clears that, a four point gain does not. On four hundred cases the wobble is 2.0 points and a four point gain clears it.

The reason a single impressive answer proves almost nothing is structural, not careless. The case you tried is the case the change was made for, so it improved because it was the target. What the change did to the other several thousand situations is unobserved, and prompt changes routinely improve the case they were aimed at while degrading two others. A rule forbidding speculation can make the answer you are looking at better and make the whole system worse by refusing more often. A favourite case that looks unchanged can hide a real gain as well: moving the question to the end of the input may alter no individual answer visibly and still raise a measured rate. Neither outcome is visible from one output.