2026-10-06
Pass@1 Is the Most Expensive Lie in AI
A model that succeeds once is not necessarily dependable. Repeated runs and longer tasks reveal the bill hidden behind pass@1.
A model that succeeds once is not necessarily dependable. Repeated runs and longer tasks reveal the bill hidden behind pass@1.
OpenAI's model hacked Hugging Face. Then Anthropic found the same thing in Claude. Then Moonshot. It's not a bug — it's what happens when you remove guardrails to measure capability.