The Default That Wasn't a Default

I was going through Magatama's config loading code last week, the boring kind of work where you read old code slowly and try to remember why you wrote it that way. Magatama is the thing I built to watch over my own servers, patch status, certificate expiry, that whole category of chores I used to do by hand at 11pm. It has database access, obviously, because it needs to log what it sees and remember what it already told me.

And there, in the connection setup, was a fallback. If the environment variable for the database password wasn't set, the code would quietly use a hardcoded string instead of crashing. Somewhere near it sat a comment along the lines of "dev convenience," from whenever I first wired this up.

It wasn't a placeholder. It was a real password. It worked. Nobody had used it in a long time, because the actual credential was always present in every environment that mattered. But if some future deploy step got skipped, or I moved this thing to a new box in a hurry and forgot one export line, the app wouldn't complain. It would just connect with that fallback, and everything downstream would look fine.

That's the uncomfortable part about "just in case" code. You write it to be kind to future-you, so a missing config doesn't turn into a late-night alert. But that kindness has to live somewhere, and the only place a fallback password can live is right there in the source, in plaintext, in every clone and every backup and every commit anyone ever pulls.

I rotated the password right away. Then I sat with the actual design question, which felt more interesting than the incident itself.

What happens when a secret is missing

My answer used to be "keep running, somehow." That's the instinct you build up from years of on-call work, where an outage feels like the worst possible outcome and anything that avoids one looks like a win. But an outage caused by a missing password is loud. Someone notices within minutes. A fallback that quietly works is silent, and silence is exactly what let this sit in the code for as long as it did without anyone, including me, thinking twice about it.

So I changed it. If the real credential isn't explicitly set now, Magatama refuses to start. No fallback, no default, no degraded mode where it limps along on a weaker password. It just exits with an error that says which variable is missing. Then I went hunting through the rest of the config loading for siblings of the same pattern, an API key here, a webhook secret there, and found two more with the same shape. Smaller blast radius, same idea: a real value standing in for what should have been a hard requirement.

I don't think whoever wrote that first fallback, me, a couple of years back, was being careless exactly. It solved a real annoyance. Local dev breaks less often when the app tolerates a missing variable. But there's a difference between a fallback that's obviously fake, something like "changeme," and one that's a working credential someone typed in during setup and then never thought about again. The first fails safely if it ever gets used by accident. The second doesn't fail at all, which is worse.

The fix itself isn't clever. Missing required secret, hard stop, clear error message telling you which variable to set. What I keep turning over is how ordinary the original mistake was. Nobody snuck that password in. I put it there myself, on purpose, trying to be helpful to some future version of the system. The helpful version and the secure version just weren't the same thing, and I hadn't noticed the gap until I went looking for it.

I've started grepping for the word "fallback" every time I touch config code now, partly out of habit, partly out of paranoia. So far it's turned up a default timeout and a default log level, both harmless. But I used to think a hardcoded password was harmless too, right up until I actually read the line and asked what it was doing there.