One GPU, Many Models: Building a Fair Queue
A home fine-tuning setup kept corrupting runs until every training script agreed to ask permission first. The lease was the easy part; finding every script that skipped it was not.
A batch of data my own scorer stamped "gold" turned out to be well-formatted noise, which meant admitting my heuristics were measuring the wrong thing.
A home fine-tuning setup kept corrupting runs until every training script agreed to ask permission first. The lease was the easy part; finding every script that skipped it was not.
Magatama can act on its own when a signal comes from scanners I built and vetted myself. Everything else, no matter how alarming it looks, only gets to flag something, never touch it.
A hardcoded database password sat quietly in Magatama's config code for years, meant as a harmless fallback. Rotating it was the easy part, deciding what should happen when a secret goes missing was the actual work.
Magatama monitors my server fleet so I don't have to. Recently I had to admit it needed monitoring of its own, and figuring out how much was the actual problem.
My prompt-injection scanner only checked for English keywords, which works fine until someone writes the same instruction in another language. Extending it to two dozen languages turned up a dumber problem: a word-boundary check that had quietly never worked against non-Latin scripts.
Magatama, the agent that watches my server fleet, can write a flawless rollback plan on its own. It still isn't allowed to execute one without me. Here's why undo turned out to be the hardest permission to hand over, not the easiest.
A rule I made for myself while hardening Magatama, my own ops agent: every fix has to make the system stricter, never more convenient. It turns out the hardest part is being the only person that rule ever inconveniences.
The instinct in security tooling is to block anything that looks suspicious. For an agent that triages real alerts, I had to convince myself that blocking is sometimes the more dangerous choice.
A defensive try/catch in Magatama's advisory-feed fetcher quietly turned a failed request into a "fresh" empty cache, erasing the last good data with nothing anywhere to flag it. The fix wasn't clever, just three changes that stopped the code from confusing "returned" with "worked."