Load-Bearing Duct Tape

Every resilience budget I've ever seen follows the same pattern. Somebody buys a second data center, or a fancier dashboard, or a shiny new observability platform with a name that sounds like a Norse god. Leadership gets a demo. Everyone nods. Meanwhile the thing actually keeping the lights on is a switch in a closet that three people know exists, one of whom left the company in 2019.

Call it the weakest link problem, and skip the metaphor caveat: it's literal. In any system built from interconnected parts, overall resilience is bounded by the worst-maintained component, not the best one. You can build a redundant, load-balanced, geographically distributed everything, and none of it matters if the whole stack still routes through one unmonitored box nobody has touched since the Obama administration.

Let's talk about the usual suspects, because they deserve a lineup.

The Ancient Switch

Every org of a certain age has one: a piece of network hardware physically installed by someone whose LinkedIn now says “Retired.” Its firmware predates several major CVEs. Nobody has the admin password written down anywhere sane, so when it finally dies, the fix involves someone standing in a closet with a paperclip and a prayer. It has never appeared on an architecture diagram, because architecture diagrams get drawn by people who know what they're documenting.

The Temporary Workaround

Somewhere in your stack is a cron job or a manual step someone bolted on three years ago to hit a launch deadline. It was supposed to last two weeks. Three years later, it's load-bearing. Half the org doesn't know it exists; the other half assumes someone else owns it. The person who wrote it remembers roughly what it does, the way you remember a dream: confidently, and wrong.

The Vendor Integration Frozen in Time

Somebody configured an integration once, for a project that shipped years ago and got forgotten right along with it. It authenticates with credentials rotated exactly zero times since setup. Nobody has tested the failure path, because nobody has thought about it hard enough to worry. It works, right up until the vendor changes an API response format on a Tuesday and something three systems downstream quietly starts eating garbage.

None of these show up in a slide deck. Resilience investment chases visibility, not risk. A new data center is fundable because you can point at it and put a photo of it in the annual report. Auditing the closet switch is not fundable, because the best-case outcome of that audit is "nothing happened," and nothing happened is a hard sell to a budget committee. Boring work that prevents a disaster looks, from the outside, exactly like work that was never necessary.

There's a selection bias baked into this too. The components people show off got built recently, on purpose. The ones nobody shows off accreted over years of "we'll fix it later." Later never comes, because later always loses to next quarter's shiny thing, and shiny things have better lobbyists.

I've come around to thinking the fix isn't more dashboards. What works is uglier: an inventory of shame, a document reviewed every quarter that names every component nobody wants to touch and who's responsible for it. Add one column for what happens if it dies at 3am on a Sunday, and you've got a real risk register instead of a wiki page nobody opens.

Because the alternative is what most organizations already do. The ancient switch fails. It gets treated like a shocking one-off, complete with a very serious postmortem. Then everyone goes straight back to funding the next dashboard. The weakest link doesn't announce itself. It just sits there, quietly outperforming your entire disaster recovery plan at the one job that matters: being the thing that breaks first.

So here's a cheap diagnostic for your next infrastructure review. Ask the room one question: what's the oldest thing in this system that nobody wants to touch? Watch the silence. That silence is your real risk register, and it's been sitting there unfunded the whole time.