Over-Refusal Is a Bug Too

One of the adapters in the gym came back from a training run last month and, mid-conversation, refused to summarize a public news article. Nothing sensitive in it, no strange phrasing in the request, just a normal ask. My first reaction was to let it slide. It's erring toward caution, I thought. That's the direction you want a model to err in.

Except that's exactly the reasoning I've learned to distrust. The gym runs a promotion gate before any new adapter version replaces the one currently serving traffic, and for a long time that gate only checked one kind of failure: did the model get something wrong, give a bad answer, or follow an instruction it shouldn't have. A model that refused too much wasn't scored against anything. It just sat there, technically passing, quietly useless for a growing share of ordinary requests.

The temptation is to treat that as a rounding error. You spend a training cycle making a model more resistant to prompt injection or more skeptical of instructions buried in tool output, and its refusal reflex creeps up everywhere else too. That feels like a side effect of doing the safety work correctly, a bit of overspray from a coat of paint you actually wanted. Free, basically. Nobody complains, and the eval dashboard for the thing you were optimizing looks great.

But a model that refuses to translate a sentence or explain how a heat pump works isn't doing its job, and it doesn't matter that the failure looks polite. Functionally it's the same outcome as a model that gives a wrong translation or a wrong explanation: the user doesn't get what they asked for. The only difference is that a wrong answer looks broken and gets flagged immediately, while an unnecessary refusal looks responsible enough that nobody bothers flagging it at all.

Grading refusals like any other wrong answer

So the gate is two-sided now. A candidate adapter has to hold its own on whatever it's specifically being trained for that cycle: accuracy, tool-use correctness, resistance to bad instructions, whatever it is. Separately, it has to run against a fixed set of ordinary, unambiguous, slightly boring prompts, summarize this, translate that, explain how a thing works, draft a short email. Requests where there's no trick and nothing to be cautious about. If the new version refuses more of those than its predecessor did, it doesn't get promoted, no matter how good the rest of its numbers look.

Writing that prompt set was less interesting than it sounds. Mostly it meant sitting down one evening and typing out forty or fifty requests that any reasonable person would expect a model to just answer, then resisting the urge to sneak in anything remotely edgy to make it feel more rigorous. The boringness was on purpose. If a model can't clear a bar that low, tightening its instincts elsewhere hasn't made it safer. It's made it worse at the one thing it was built to do.

I won't pretend this catches everything. Refusal rate against a fixed prompt set is a blunt instrument, and some future adapter will probably find a new way to be unhelpful that the set doesn't cover. But at least the failure mode that used to hide behind good intentions now has to show up as a number, sitting right next to all the other numbers that decide whether a model ships.