My benchmark graded '7! = 5040' as wrong — and two other ways it lied to me
Re-running my own LLM benchmark, I found a bug that had inflated the quality scores in posts I'd already published. Then a second bug. Then a third. Here's how a wrong number looks exactly like a right one — and why you spot-check the failures, not the passes.
Jun 7, 2026