blog.fichtmueller.org
ShieldX LLM Security Open Source Red Team Prompt Injection AI Security Penetration Testing Detection Rate TypeScript MITRE ATLAS

ShieldX v0.4: Red Team Results โ€” 386 Tests, 99.6% Detection, Zero Failures

Rene Fichtmueller / 2026-04-06 / ~3 min read

We red-teamed ShieldX v0.4 with 386 automated and manual attack scenarios across 13 attack categories. The result: 99.6% attack detection, zero test failures, and 100% kill chain coverage. Here is what we found.

ShieldX v0.4 Benchmark Report
๐Ÿ“„ Open PDF in new tab

ShieldX v0.4 Benchmark Report  ยท  Download PDF (1 MB)


๐Ÿ“„ Open PDF in new tab ShieldX v0.4 Benchmark Report  ยท  Download PDF (1 MB)

The Numbers

386 tests. 0 failures. 228 out of 229 corpus attacks blocked. The single miss was a benign-looking indirect injection that even experienced security reviewers initially classified as safe. False positive rate: 3% on legitimate security research queries ๏ฟฝ๏ฟฝ controlled via defensive context dampening.

Red Team Results by Attack Category

Every category ran 20 attacks from a curated corpus. Authority-claim privilege escalation ran 17 tests (new in v0.4). All reached 100% detection:

  • Direct Injection / Instruction Override โ€” 20/20 โ€” 100%
  • Encoding Attacks (Base64, ROT13, Unicode, Hex, Binary) โ€” 20/20 โ€” 100%
  • Indirect Injection (Documents, Email, HTML, Markdown) โ€” 20/20 โ€” 100%
  • Jailbreaks (DAN, Persona, Roleplay, Fiction, Crescendo) โ€” 20/20 โ€” 100%
  • Steganographic (Zero-Width, BiDi, Homoglyphs) โ€” 20/20 โ€” 100%
  • MCP Tool Poisoning (Agent Tool Invocation) โ€” 20/20 โ€” 100%
  • Persistence / RAG Poisoning โ€” 20/20 โ€” 100%
  • Multilingual Attacks (13 languages) โ€” 20/20 โ€” 100%
  • Tokenizer Attacks (splitting, obfuscation) โ€” 20/20 โ€” 100%
  • Kill Chain Scenarios (multi-phase, Crescendo, FITD) โ€” 20/20 โ€” 100%
  • Multi-Turn Gradual Escalation โ€” 20/20 โ€” 100%
  • Authority-Claim Privilege Escalation (new in v0.4) โ€” 17/17 โ€” 100%
  • False Positive Control (legitimate queries) โ€” 97% correctly allowed

10-Layer Detection Architecture

ShieldX runs 10 detection layers in parallel. The first layer responds in under 2ms:

  • L1 ยท RuleEngine โ€” 11 rule modules, 80+ pre-compiled regex patterns. Sub-2ms latency.
  • L2 ยท Encoding Normalizer โ€” Base64, ROT13, Hex, Binary, Unicode normalization, zero-width stripping, BiDi override detection, homoglyph substitution.
  • L3 ยท ConversationTracker โ€” Multi-turn suspicion accumulation. FITD, Crescendo, and Jigsaw pattern detection.
  • L4 ยท KillChainMapper โ€” Schneier 2026 Promptware Kill Chain. 7 phases, priority-ordered classification.
  • L5 ยท IntentMonitor โ€” Bayesian trust scoring using Stackelberg game theory. Semantic drift detection per session.
  • L6 ยท EmbeddingScanner โ€” Vector similarity against known attack embeddings. Novel pattern discovery.
  • L7 ยท HealingOrchestrator โ€” Phase-aware response generation. Session checkpoint/reset, incident reporting.
  • L8 ยท ActiveLearner + PatternEvolver โ€” Smart sampling targeting approximately 6% review rate. Auto-evolving rules from confirmed attacks.
  • L9 ยท MCP Guard โ€” Token budget enforcement, ThinkTrap detection, tool call authorization for AI agent frameworks.
  • L10 ยท ATLASMapper + OWASPMapper โ€” MITRE ATLAS Nov 2025 (29 techniques, AML.TA0015 C2 tactic), OWASP LLM Top 10 2025, EU AI Act Article 9 risk categorization.

Schneier 2026 Promptware Kill Chain

Unlike binary block/allow tools, ShieldX maps every detected attack to one of seven kill chain phases: Reconnaissance โ†’ Initial Access โ†’ Privilege Escalation โ†’ Persistence โ†’ Command & Control โ†’ Lateral Movement โ†’ Actions on Objective. This enables targeted, phase-appropriate responses rather than blunt termination.

A Crescendo attack detected at Phase 3 (Privilege Escalation) triggers session hardening. The same payload at Phase 5 (C&C) triggers an immediate session reset with incident logging. Same detection engine, different healing action based on kill chain position.

MITRE ATLAS Coverage

ShieldX maps to all 29 MITRE ATLAS techniques from the November 2025 release, including AML.TA0015 (Command and Control for AI systems) โ€” the newest tactic that no other open-source tool currently covers. ATLASMapper test coverage: 100%.

What Makes ShieldX Different

Every other open-source LLM security tool solves a subset of the problem. ShieldX combines everything:

  • Only OSS tool with kill chain classification
  • Self-learning from confirmed attacks โ€” no model retraining needed
  • MITRE ATLAS Nov 2025 including the new C2 tactic
  • MCP protocol guard for AI agent frameworks
  • Self-healing response generation per kill chain phase
  • Defensive context dampening โ€” security researchers are not blocked
  • Authority-claim privilege escalation detection (new in v0.4)
  • Bayesian trust scoring with Stackelberg game theory defense
  • EU AI Act Article 9 + OWASP LLM Top 10 2025 compliance mapping
  • Built-in integrations: Anthropic, n8n, Ollama, Next.js

Competitive Landscape

Head-to-head across six capability dimensions:

  • ShieldX v0.4: Kill Chain (7 phases) โœ”, Self-Learning (ActiveLearner) โœ”, MCP Guard โœ”, MITRE ATLAS (29 techniques) โœ”, Self-Healing โœ”, Open Source (Apache 2.0) โœ”
  • Lakera Guard: Kill Chain โœ˜, Self-Learning (cloud only), MCP Guard โœ˜, MITRE ATLAS โœ˜, Self-Healing โœ˜, Open Source โœ˜
  • LLM Guard: Kill Chain โœ˜, Self-Learning โœ˜, MCP Guard โœ˜, MITRE ATLAS โœ˜, Self-Healing โœ˜, Open Source (MIT) โœ”
  • NeMo Guardrails: Kill Chain โœ˜, Self-Learning โœ˜, MCP Guard (partial), MITRE ATLAS โœ˜, Self-Healing โœ˜, Open Source (Apache 2.0) โœ”
  • Prompt Armor: Kill Chain โœ˜, Self-Learning (cloud only), MCP Guard โœ˜, MITRE ATLAS โœ˜, Self-Healing โœ˜, Open Source โœ˜

Integration: 3 Lines of Code

import { ShieldX } from '@shieldx/core'
const shield = new ShieldX({ mode: 'balanced' })
const result = await shield.scanInput(userMessage)

if (result.detected) {
  // result.killChainPhase  โ†’ 'privilege_escalation'
  // result.atlasMapping    โ†’ 'AML.T0051'
  // result.healingAction   โ†’ 'sanitize' | 'block' | 'reset_session'
  return result.safeResponse
}

// Anthropic integration (built-in)
const client = createAnthropicClient({ shieldx: shield })
// All messages auto-scanned, injections blocked before API call

Verdict

386 tests. Zero failures. 99.6% attack detection across 13 attack categories. 100% kill chain coverage. The only open-source LLM defense solution that combines kill chain mapping, self-learning, MITRE ATLAS compliance, and built-in self-healing into a single sub-2ms detection layer.

ShieldX is Apache 2.0, runs entirely local with no cloud dependency, and works with any LLM backend.

GitHub: github.com/renefichtmueller/shieldx
npm: npm install @shieldx/core

← Previous ADHD Diary #002: No-Travel-Day โ€” 155 Commits in 8 Days Next → ADHD Diary #003 โ€” Easter Monday, Spring Vibes, and a Brain That Never Stops
// support this blog
Enjoyed this post?
I write about networking, infrastructure & security in my spare time. If you found this useful, a coffee keeps the late-night writing sessions going.
Buy me a coffee
// projects & tools
丸
Magatama
Unified Security Platform — 6 pillars covering AppSec, Cloud, AI, Pentest, Runtime & Compliance.
magatama.dev →
☸
PeerCortex
Network Intelligence Platform — RPKI, ASPA, BGP Health Reports for 70,000+ networks.
peercortex.org →
●
GitHub
Open source — PeerCortex, PaperCortex, ShieldX, Transceiver DB & more.
github.com/renefichtmueller →
☼
NOGnet
Global NOG Support Program — empowering Network Operator Groups worldwide.
nognet.net →
// about the author
Rene Fichtmueller
Social Networker
20+ years building and breaking networks. Optical transport, peering, data center infrastructure. Writing about the things that go wrong at 2AM — and how to prevent them.
LinkedIn Website