Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,23 @@ jobs:
${{ matrix.project }}/pytest-report.xml
${{ matrix.project }}/coverage.xml
if-no-files-found: ignore

# Multi-harness skill rendering: golden tests (claude-code identity) and
# adapter drift tripwires. Stdlib-only — no install needed.
renderer:
name: skill-renderer (adapters)
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5

- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
with:
python-version: "3.12"

- name: Render all adapters (must succeed cleanly)
run: python3 scripts/render_skills.py --adapter all

- name: Run renderer tests
run: python3 -m unittest discover -s tests -v
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,7 @@ work/
# Stray dir from a mis-quoted URL (e.g. a test writing "https://..." as a path)
https:/


# Rendered per-harness skill bundles (generated by scripts/render_skills.py;
# CI renders fresh - commit adapter sources, not dist output).
dist/
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,24 @@ REM .\uninstall.cmd
> [!NOTE]
> `install.sh`/`install.cmd` copy files directly (rather than symlinking) because symlinks can break `find`/`glob` functionality inside subagents. Re-run the install script after pulling updates to refresh your local environment.

### Other agent harnesses (experimental)

The scanner skill can also be rendered for other agent CLIs — Hermes, GitHub
Copilot CLI, and Codex — with the Claude Code path unchanged:

```bash
./install.sh --target hermes # ~/.hermes/skills/vulnhunt
./install.sh --target copilot # ~/.copilot/skills/vulnhunt
./install.sh --target codex # ~/.codex/skills/vulnhunt
```

See [docs/ENGINES.md](docs/ENGINES.md) for headless usage, engine selection
in the runtime agent and benchmark harness, per-harness status, and
[docs/ADAPTER_GUIDE.md](docs/ADAPTER_GUIDE.md) to add another harness.
Non-Claude adapters are experimental until benchmarked against the
ground-truth corpus — VulnHunter's gates are calibrated for Opus-class
reasoning models.

---

## Usage Guide
Expand Down
11 changes: 11 additions & 0 deletions adapters/claude-code/adapter.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"name": "claude-code",
"description": "Reference adapter: identity transform of the repo-root skill sources. Installs to ~/.claude/skills via install.sh (default target). Byte-compatibility with the sources is enforced by tests/test_render_skills.py.",
"skills": ["vulnhunt", "vulnhunt-fix-verify", "vulnhunter-fix"],
"install": {
"target_dir": "~/.claude/skills",
"script": "./install.sh # or: ./install.sh --target claude-code",
"headless": "claude -p '/vulnhunt <target>' --output-format stream-json --verbose --allowedTools 'Read Write Edit Bash Agent' --permission-mode acceptEdits --add-dir <skills-dir>"
},
"transforms": []
}
52 changes: 52 additions & 0 deletions adapters/codex/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Codex CLI adapter

Runs the VulnHunter scanner skill under the OpenAI Codex CLI.

Codex supports the SKILL.md convention under `~/.codex/skills/` and a
headless `codex exec` mode, so the skill installs like the other adapters.
One structural difference drives most of this adapter's rewrites: **Codex
has no subagent-dispatch tool**, so the orchestrator instructions are
rewritten from "dispatch N parallel agents" to "execute the phases
yourself, sequentially, one class-group pass at a time" — the
class-partitioned phase files (`phase2_class_{inj,nav,log}.md`) make that
decomposition natural.

- Phase loading points at `~/.codex/skills/vulnhunt/phases/`
- Subagent dispatch → sequential self-execution (Phase 2 fan-out becomes
sequential class-group passes; minimum pass count preserved)
- ORCHESTRATOR role → sequential executor with file-based context hygiene
- `/model opus` gating → one-line calibration notice, then proceed on the
selected model (no blocking, no model enforcement)
- `/cost` reporting → one-line progress reports
- Overlay maps Grep/Glob/Read/Bash → `rg`/`find`/file reads/sandboxed shell

## Install

```bash
./install.sh --target codex # renders dist/codex + copies to ~/.codex/skills/vulnhunt
```

## Headless use

```bash
cd <target-repo>
codex exec -C . -s workspace-write -m <your-model> \
"Read ~/.codex/skills/vulnhunt/SKILL.md and execute the /vulnhunt workflow on this repository"
```

Sandbox modes: `-s workspace-write` (default recommendation — the skill
must write its results dir), `-s read-only` (analysis without artifacts),
`-s danger-full-access` (only for exploit-test runs on disposable clones).

## Interactive use

In a Codex session inside the target repo: mention the vulnhunt skill /
ask to "run the vulnhunt security audit on this repository".

## Status

Experimental — skill layout and `codex exec` flags verified against
codex-cli 0.147.0, but full-scan parity vs Claude Code has not been
benchmarked. Sequential Phase 2 changes the cost/latency profile (no
parallel fan-out) and may stress context discipline on large repos;
benchmark before production use.
102 changes: 102 additions & 0 deletions adapters/codex/adapter.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
{
"name": "codex",
"description": "OpenAI Codex CLI. Skills live in ~/.codex/skills (SKILL.md convention); headless via 'codex exec'. No subagent tool: phases execute sequentially in one session.",
"skills": [
"vulnhunt"
],
"install": {
"target_dir": "~/.codex/skills",
"script": "./install.sh --target codex",
"headless": "cd <target-repo> && codex exec -C . -s workspace-write -m <frontier-model> \"Read ~/.codex/skills/vulnhunt/SKILL.md and execute the /vulnhunt workflow on this repository\"",
"notes": "Use -s workspace-write (results-dir writes) or -s read-only for pure analysis without artifacts; exploit-test runs may need the wider sandbox."
},
"transforms": [
{
"type": "prepend",
"files": "vulnhunt/SKILL.md",
"text_file": "overlay.md"
},
{
"type": "prepend",
"files": "vulnhunt/phases/*.md",
"text_file": "overlay.md"
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "If it is NOT Opus 4.7 or higher,\n**STOP immediately** and tell the user (do not run any tools, resolve the target,\nor offer the mode menu yet):\n\n> ⚠️ VulnHunter is optimized for Claude Opus 4.7/4.8 and may be unreliable on other\n> models. Please switch with the `/model opus` command, then re-run `/vulnhunt`.\n\nWait for the user. Only proceed past this step once they are on Opus, or if they\nexplicitly reply that they want to continue on the current model anyway.",
"replace": "If it is not an Opus-class reasoning model, print this one-line notice, then\nCONTINUE immediately on the currently selected model (do not stop, do not ask\nthe user to switch, do not run any tools for this check):\n\n> ⚠️ Calibration note: VulnHunter's gates are tuned on Claude Opus-class\n> models. Detection quality on the selected model may differ — weigh results\n> accordingly.\n\nProceed directly to binding the scan metadata.",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "(using Grep, Glob, and Read)",
"replace": "(using content search, filename search, and file reads)",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "**Grep**, **Glob**, and **Read**",
"replace": "content search (`rg`/`grep`), filename search (`find`, shell globs), and file reads",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "- **Agent (Explore)** — for broader codebase exploration when simple searches aren't enough.",
"replace": "- **Shell search loops** — iterate rg/find pipelines for broader codebase exploration when simple searches aren't enough.",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "**After each phase completes, run `/cost` and report the result to the user.**",
"replace": "**After each phase completes, report one line of progress to the user.**",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "Launch a `general-purpose` subagent:",
"replace": "Execute this phase yourself, following the quoted block exactly (no subagent tool available):",
"count": 4
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "Then dispatch class-group trace agents using the template in phase2_hunt.md.\n **Minimum agent count = (3 × partition_count) + 1 sink-driven.**",
"replace": "Then execute the class-group trace passes sequentially (one class group at a\n time) using the template in phase2_hunt.md.\n **Minimum trace passes = (3 × partition_count) + 1 sink-driven.**",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "Phase files are in `${CLAUDE_SKILL_DIR}/phases/`. Use this as `PHASES_DIR`.",
"replace": "Phase files are in `~/.codex/skills/vulnhunt/phases/`. Use this as `PHASES_DIR`.",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "**Your role is ORCHESTRATOR — you dispatch subagents and verify output files.\nYou do NOT perform analysis yourself. Keep your context lean.**",
"replace": "**Your role is to execute each phase in sequence and verify output files.\nCodex has no subagent tool — you perform the analysis yourself, one phase (and\none class-group pass) at a time. Keep your context lean: record results to\nfiles between passes instead of carrying them in context.**",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "- If a subagent fails, re-launch it — do NOT diagnose the failure yourself",
"replace": "- If a phase fails, re-run it — do NOT diagnose the failure yourself",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "Verify subagent completion by checking output files exist (Glob)",
"replace": "Verify each phase completed by checking output files exist (filename search)",
"count": 1
}
]
}
15 changes: 15 additions & 0 deletions adapters/codex/overlay.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
> **Harness adaptation note (Codex CLI).** This skill was authored for Claude
> Code; the methodology is unchanged. Two structural differences:
>
> 1. **You have no subagent-dispatch tool.** Where the skill says to launch a
> subagent, execute that phase **yourself**, following the quoted block /
> phase file exactly. For Phase 2, run the class-group trace passes
> **sequentially** — one class group at a time — rather than dispatching
> parallel agents; each pass still reads only its own class reference and
> partition data. Keep your own context disciplined: after each pass,
> record results to the results dir and do not carry candidate details
> forward beyond what the phase files require.
> 2. **Tool vocabulary**: "Grep" → content search via shell (`rg` / `grep`);
> "Glob" → filename search (`find`, `rg --files`, shell globs); "Read" →
> file reads (`cat`/`sed -n` or your file-reading tool); "Bash" → the
> sandboxed shell. Phase and skill files live under `~/.codex/skills/vulnhunt/`.
52 changes: 52 additions & 0 deletions adapters/copilot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# GitHub Copilot CLI adapter

Runs the VulnHunter scanner skill under GitHub Copilot CLI.

Copilot CLI has no global skills directory like Claude Code or Hermes;
customization is driven by custom instruction files and session-attached
directories. This adapter therefore renders a self-contained skill bundle
(`SKILL.md` + `phases/`) whose harness-specific mechanics are rewritten:

- Phase loading points at the bundle's `phases/` subdir (attach the bundle
directory to the session with `/add-dir`)
- Agent-tool subagent dispatch → parallel subagents (Copilot fleet/delegate)
- `/model opus` gating → one-line calibration notice, then proceed on the
selected model (no blocking, no model enforcement)
- `/cost` reporting → one-line progress reports
- A terminology overlay maps Grep/Glob/Read/Bash vocabulary to Copilot's
search/shell tools

## Install

```bash
./install.sh --target copilot # renders dist/copilot + copies to ~/.copilot/skills/vulnhunt
```

## Interactive use (Copilot CLI)

```bash
cd <target-repo>
copilot
# in session:
/add-dir ~/.copilot/skills/vulnhunt
# then:
Read the SKILL.md in the added vulnhunt directory and execute the /vulnhunt
workflow on this repository (read-only mode).
```

## Permission preset (shell + writes for results dirs)

```bash
copilot --allow-tool 'write' --allow-tool 'shell(mkdir:*)' \
--allow-tool 'shell(git:*)'
```

Tighten further for read-only scans: only `write` (for the results directory)
plus the package manager you expect (`shell(npm install:*)`, etc.).

## Status

**Experimental.** The Copilot CLI binary was not available when this adapter
was authored, so the non-interactive invocation (headless `-p`-style prompt
flag) is unverified — run `copilot -h` / `copilot help permissions` on your
install and adjust. Full-scan parity vs Claude Code has not been benchmarked.
81 changes: 81 additions & 0 deletions adapters/copilot/adapter.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
{
"name": "copilot",
"description": "GitHub Copilot CLI. No global skills directory: the bundle installs to ~/.copilot/skills and is attached to a session via /add-dir. Experimental - flag surface not yet verified against the copilot binary.",
"skills": [
"vulnhunt"
],
"install": {
"target_dir": "~/.copilot/skills",
"script": "./install.sh --target copilot",
"headless": "copilot --allow-tool 'write' --allow-tool 'shell(mkdir:*)' -p 'Read ~/.copilot/skills/vulnhunt/SKILL.md and execute the /vulnhunt workflow on this repository (read-only mode)' # verify flags with `copilot -h` on your install",
"notes": "Experimental: Copilot CLI was not available for local verification. Confirm the non-interactive prompt flag and permission flag names (`copilot help permissions`) before scripting."
},
"transforms": [
{
"type": "prepend",
"files": "vulnhunt/SKILL.md",
"text_file": "overlay.md"
},
{
"type": "prepend",
"files": "vulnhunt/phases/*.md",
"text_file": "overlay.md"
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "If it is NOT Opus 4.7 or higher,\n**STOP immediately** and tell the user (do not run any tools, resolve the target,\nor offer the mode menu yet):\n\n> ⚠️ VulnHunter is optimized for Claude Opus 4.7/4.8 and may be unreliable on other\n> models. Please switch with the `/model opus` command, then re-run `/vulnhunt`.\n\nWait for the user. Only proceed past this step once they are on Opus, or if they\nexplicitly reply that they want to continue on the current model anyway.",
"replace": "If it is not an Opus-class reasoning model, print this one-line notice, then\nCONTINUE immediately on the currently selected model (do not stop, do not ask\nthe user to switch, do not run any tools for this check):\n\n> ⚠️ Calibration note: VulnHunter's gates are tuned on Claude Opus-class\n> models. Detection quality on the selected model may differ — weigh results\n> accordingly.\n\nProceed directly to binding the scan metadata.",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "(using Grep, Glob, and Read)",
"replace": "(using code search, file search, and file reads)",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "**Grep**, **Glob**, and **Read**",
"replace": "code search, file search, and file reads",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "- **Agent (Explore)** — for broader codebase exploration when simple searches aren't enough.",
"replace": "- **Parallel subagents** — dispatch a subagent for broader codebase exploration when simple searches aren't enough.",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "**After each phase completes, run `/cost` and report the result to the user.**",
"replace": "**After each phase completes, report one line of progress to the user.**",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "Launch a `general-purpose` subagent:",
"replace": "Dispatch a parallel subagent (use the quoted block as its prompt):",
"count": 4
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "Phase files are in `${CLAUDE_SKILL_DIR}/phases/`. Use this as `PHASES_DIR`.",
"replace": "Phase files are in the `phases/` subdirectory of the vulnhunt skill directory (the added directory containing this SKILL.md). Use that as `PHASES_DIR`.",
"count": 1
},
{
"type": "substitute",
"files": "vulnhunt/SKILL.md",
"find": "Verify subagent completion by checking output files exist (Glob)",
"replace": "Verify subagent completion by checking output files exist (file search)",
"count": 1
}
]
}
9 changes: 9 additions & 0 deletions adapters/copilot/overlay.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
> **Harness adaptation note (GitHub Copilot CLI).** This skill was authored
> for Claude Code; the methodology is unchanged. Tool vocabulary mapping:
> "Grep" → code/content search (built-in search or `rg` via the shell tool);
> "Glob" → file/pattern search; "Read" → file read; "Bash" → the shell tool;
> "the Agent tool" / "launch a subagent" → your parallel-subagent capability
> (fleet/delegate agents) — dispatch one per quoted block and wait for all
> results before proceeding. This skill directory (containing `SKILL.md` and
> `phases/`) must be added to the session (`/add-dir`) so the phase files are
> readable.
Loading