One similarity cutoff is always wrong
'What's 2+2?' needs a near-exact match to reuse safely. 'Give me creative ideas for X' can tolerate a loose one. A single global threshold guarantees you're too strict somewhere and too loose somewhere else.
Every semantic cache has to draw a line: how similar is similar enough to reuse an answer? The tempting move is to pick one number and apply it everywhere. It's also a guaranteed mistake. A factual lookup needs an almost-exact match before you dare reuse it. A brainstorming prompt can share an answer with a much looser cousin. One line can't serve both.
The bar for reuse moves with the question, strict where correctness is binary, relaxed where it isn't.
And it isn't static. When users correct an answer or regenerate one, that's a signal the threshold for that kind of question was too loose, and it tightens. The cache tunes its own caution per category instead of forcing you to pick one compromise that's wrong everywhere.
The bottom line
Similarity isn't one bar; it's many. Match the strictness to the stakes of the question, and you stop trading correctness for hit rate, or hit rate for correctness.