Never Silently Drop an Alert

I built Magatama to watch my own server fleet so I don't have to stare at logs at midnight. It reads alerts and decides what's worth waking me up for. Somewhere in that pipeline, untrusted text reaches a language model: alert descriptions, threat-feed entries, whatever some remote system decided to hand it. Any time untrusted text reaches a model, you have to assume someone will eventually try to make the model do something other than what you asked.

So there's a scanner in front of everything. Before incoming text touches the model, it gets checked for prompt-injection patterns: instructions embedded in what's supposed to be a plain log line, formatting that mimics a system message. Fairly standard if you've built anything that touches an LLM and the open internet.

What took longer to work out wasn't the scanner itself. It was what happens after the scanner fires.

My first instinct was the obvious one: if something looks like an injection attempt, block it, don't let it near the model. That's how most security tooling works. When in doubt, deny. A firewall rule doesn't ask the packet to explain itself.

Then I thought about what that means for a system whose entire job is triaging alerts. Say a real alert, something genuinely wrong on a box, happens to contain text that trips the injection scanner. If my response is to drop it, I haven't stopped an attack. I've made a real problem invisible. Nobody gets paged, nothing shows up in a queue. The alert just doesn't fail loudly, and a quiet failure is worse, because a loud one at least tells you something broke.

That's the part I had to actually sit with rather than just nod along to: a false block on a real alert is a security incident on its own. If an attacker's goal is to suppress detection rather than hijack a model's output, an overzealous scanner does that job for free, no working exploit required. Blocking on suspicion turns my own caution into the attack surface.

So I flipped the default. Magatama is not allowed to silently drop or ignore an incoming alert because it looks like it might contain an injection attempt. The suspicious content gets flagged and wrapped before it goes near the model, treated as inert data rather than as instructions, whatever it claims to be. The underlying alert still gets triaged, every single time. The pipeline doesn't get a vote on whether something is too suspicious to look at.

Writing that rule down felt wrong for a while. Nearly everything I know about building security-adjacent systems says fail closed. Here I was deliberately building something that fails open on the exact axis where failing closed seemed obviously right. I went back and forth on it more than I'd like to admit, mostly arguing with myself over whether I was rationalizing something convenient rather than something correct.

What settled it, eventually, was pinning down what actually worried me: a real incident sitting untriaged in a pile because a scanner got twitchy, more than a model getting talked into saying something dumb inside a log entry nobody reads. One of those failure modes gets noticed and fixed. The other sits quietly until something breaks badly enough on its own to surface it, and by then the alert that would have caught it earlier is already gone.

I still don't love the phrase "fails open" next to anything with the word security in it. But the alternative was a system that could make itself blind to the exact thing it exists to catch, and call that blindness a safety feature. I'd rather deal with the occasional weird log line than build a very expensive way of not noticing.