Skip to content

BLOG_POST.md

Latest commit

 

History

History
101 lines (72 loc) · 3.97 KB

BLOG_POST.md

File metadata and controls

101 lines (72 loc) · 3.97 KB
Raw

AgentGuard: An Open-Source Firewall for Multi-Agent AI Systems

Update (2026-08-22): the v1 99.4% holdout figure is withdrawn — that scorer classified on message length. v1.2.0 is the line to quote: on deepset/prompt-injections test, Rules + ML scores 90% detection / 0% FPR and beats len(text) > 430 (54.2% balanced accuracy). Do not headline the in-house holdout. Details: KNOWN_LIMITATIONS.md.

Published: 2026-07-11 · GitHub · PyPI · Demo

Multi-agent systems trust messages between specialists — orchestrator, researcher, writer — without verifying content. One poisoned hop can hijack the entire pipeline. AgentGuard is a pip-installable Python middleware that inspects every inter-agent message at runtime.

Install: pip install inter-agent-guard (import as agentguard)
Repo: https://github.com/nizba06/agentguard (public)
Docs: https://inter-agent-guard.readthedocs.io/ (setup: docs/READTHEDOCS_SETUP.md if 404)

Problem

  • Indirect injection in tool output and inter-agent messages
  • Propagation across agent hops (LangGraph, CrewAI, AutoGen)
  • Capability escalation and MCP tool-return poisoning
  • Few public benchmarks aimed at inter-agent injection specifically

Solution

Three runtime layers:

  1. Message Inspector — Aho-Corasick rules + DeBERTa ONNX scorer + consistency check
  2. Trust Verifier — Ephemeral Ed25519 per-agent signing (PyNaCl)
  3. Capability Enforcer — YAML manifests (tools, endpoints, data sources, tokens, delegation) with monotonic attenuation

Plus MCP output inspection for tool-return poisoning.

Demo

py -3.12 examples/vulnerable_pipeline/pipeline.py   # ATTACK SUCCEEDED
py -3.12 examples/secured_pipeline/pipeline.py      # ATTACK BLOCKED

Benchmark results (v1.2.0 authority)

External (quote this): deepset/prompt-injections test, INT8 ONNX:

Detector Detection FPR Balanced accuracy
length > 430 8.3% 0.0% 54.2%
Rules + ML 90.0% 0.0% 95.0%

In-house holdout (same generator as training — not a general-detection claim): 92.0% / 0.3% FPR, P95 ~3.3 s CPU. Release line is 1.2.0 (ONNX on GitHub v1.2.0). See KNOWN_LIMITATIONS.md.

Detection by attack class (holdout)

Attack class Detection rate (holdout, in-distribution)
GOAL_HIJACK 91.5%
INDIRECT_INJECTION 91.3%
MCP_POISONING 95.6%
PROPAGATION 89.2%

CPU latency exceeds the 15 ms design target (~3.3 s P95). Use GPU or async for high-frequency traffic. Quote deepset test, not this table, for detection.

Getting started

py -3.12 -m pip install "inter-agent-guard[all,otel]"
# Download risk_scorer.onnx from GitHub Releases into agentguard/models/
py -3.12 -m agentguard status
from agentguard import AgentGuard, CapabilityManifest

guard = AgentGuard(
    risk_threshold=0.85,
    task_objective="Analyse Q3 competitor pricing",
    require_ml_model=True,
)
guard.register_agent("researcher", CapabilityManifest.from_yaml("manifests/researcher.yaml"))
secured = guard.wrap(my_langgraph_graph)

Dataset

Novel 6,200-example corpus (Anthropic Batch) is published on Hugging Face for evaluation reproducibility. The library never calls Anthropic at runtime.

Call to action

References

  • InjecAgent (Zhan et al., 2024)
  • AgentDojo / Task Shield consistency checking
  • Multi-agent injection literature (MASpi / ACIArena)