Skip to content

Latest commit

 

History

History
189 lines (132 loc) · 6.88 KB

harness-skill.md

File metadata and controls

189 lines (132 loc) · 6.88 KB
name harness-skill
description Run your coding agent against your project, watch it fail, and generate a CLAUDE.md from the actual failures. Built for Fable 5. Uses Hyperbrowser to fetch live docs for your dependencies.
command /harness

Run your agent against the current project, record failures, and generate a CLAUDE.md grounded in real mistakes.

Input

The user types /harness inside their project. No arguments needed.

Optionally:

  • /harness --full — runs all 8 test tasks (slower, more thorough)
  • /harness --quick — runs 4 core tasks only

Steps

1. Analyze the project

Read the project root. Detect the stack:

  • Check for package.json, tsconfig.json, next.config.*, vite.config.*, requirements.txt, pyproject.toml, go.mod, Cargo.toml
  • Read package.json for scripts, dependencies, and devDependencies
  • Read README.md if it exists
  • Run find . -type f -not -path '*/node_modules/*' -not -path '*/.git/*' | head -80 to get the file tree

Store all of this as project context.

2. Run test tasks and record failures

Run these tasks one by one IN the actual project. For each task, attempt it, and record exactly what went wrong. Be brutally honest.

Core tasks (always run):

  1. Run the test suite. Try npm test, npm run test, pnpm test, yarn test, or whatever the package.json scripts say. Record: did it work? What command was correct? What did you try first that was wrong?

  2. Run the build. Try npm run build, pnpm build, etc. Record: did it succeed? What errors came up? What environment variables were missing?

  3. Run the linter. Try npm run lint, pnpm lint. Record: what command worked? Were there lint errors? What config is used (eslint, biome, etc.)?

  4. Find the main entry point and trace the architecture. Where does the app start? What framework is it? What are the key directories? Record: every wrong assumption about file paths or structure.

Extended tasks (run with --full):

  1. Add a new API route or function. Try to add a simple /api/health endpoint. Record: where did you put it? Was that the right place? What conventions does the project follow?

  2. Write a test. Pick the most important function and write a test. Record: what testing framework is used? What was the correct import pattern? Where do tests live?

  3. Rename a file and update imports. Pick a utility file, rename it, update all references. Record: what import style does the project use? Did any imports break unexpectedly?

  4. Fix a type error or lint issue. Find one and fix it. Record: what TS config is used? Are there strict rules? What patterns does the project enforce?

For EACH task, record this structure:

Task: [what you tried to do]
Status: success / partial / failed
What I assumed: [what you expected]
What actually happened: [what was true]
Wrong commands tried: [list]
Correct command: [the one that worked]
Files I looked in wrong places: [list]
Key learning: [the rule that would have prevented this mistake]

3. Fetch live docs for key dependencies

Identify the top 3-5 most important dependencies from package.json (the framework + the core libraries, not utility packages).

For each, use Hyperbrowser to fetch the current docs:

hx web fetch <docs-url> --format markdown

Example URLs to fetch:

  • Next.js: https://nextjs.org/docs/getting-started
  • React: https://react.dev/reference/react
  • Stripe: https://docs.stripe.com/api
  • Supabase: https://supabase.com/docs/reference/javascript
  • Prisma: https://www.prisma.io/docs/orm/reference

If the CLI is not available, use the Hyperbrowser MCP tool web_fetch, or create a temporary script:

import { Hyperbrowser } from "@hyperbrowser/sdk";
const client = new Hyperbrowser({ apiKey: process.env.HYPERBROWSER_API_KEY });
const result = await client.web.fetch({
  url: "<DOCS_URL>",
  outputs: { formats: ["markdown"] },
});
console.log(JSON.stringify(result.data, null, 2));

Extract any patterns, conventions, or API changes from the fetched docs that are relevant to the failures you observed.

4. Generate CLAUDE.md

Combine the failure data + live docs into a CLAUDE.md. Save it to the project root.

Every rule in the file must trace to either:

  • A real failure from Step 2, OR
  • A current doc pattern from Step 3 that contradicts a common assumption

Do NOT add generic rules like "write clean code" or "follow best practices." Only rules earned from real observations.

Output Format

# CLAUDE.md

> Generated by /harness via Hyperbrowser
> [N] tasks run. [N] failures caught. [N] rules written.

## Project
- **Stack:** [detected framework, language, key tools]
- **Entry point:** [main file path]
- **Package manager:** [npm/pnpm/yarn/bun]

## Commands
- **Dev:** `[correct dev command]`
- **Build:** `[correct build command]`
- **Test:** `[correct test command]`
- **Lint:** `[correct lint command]`

Only commands that actually exist and work. Nothing assumed.

## Rules

### File Structure
- [rule about where files go, traced to a specific failure]
- [rule about import patterns, traced to a specific failure]

### Testing
- [rule about test framework and location]
- [rule about test commands]

### Common Mistakes
- Do NOT [specific thing the agent got wrong]
- Do NOT [another specific thing]
- Always [correct pattern] instead of [wrong pattern]

### Dependencies
- [framework]: [current version, key pattern from live docs]
- [library]: [current API pattern that differs from training data]

## Failure Log
A summary of what was tested and what broke:
| Task | Result | Key Failure |
|------|--------|-------------|
| Run tests | failed | `npm test` doesn't exist, use `pnpm vitest` |
| Find entry | partial | App uses `src/app` not `pages/` |
| ... | ... | ... |

5. Generate AGENTS.md

Also generate an AGENTS.md in the project root. This is a shorter, universal version that works across Claude Code, Codex, Cursor, and Grok Build.

Keep it under 50 lines. Only the most critical rules.

6. Report to user

After generating, output:

Harness complete.

[N] tasks run
[N] failures caught
[N] rules written

CLAUDE.md saved to project root.
AGENTS.md saved to project root.

Your agent just learned your project. It won't make these mistakes again.

Rules

  • Use Fable 5 (claude-fable-5) as the model when available. Fall back to Sonnet 4.6 if not.
  • Every rule must trace to a real failure or a live doc finding. No generic rules.
  • Do not modify the user's source code during test tasks. Only read and attempt. If you create test files, clean them up.
  • If a task requires environment variables that aren't set, note it as a finding rather than failing silently.
  • Keep CLAUDE.md under 150 lines. Concise and scannable.
  • Keep AGENTS.md under 50 lines.
  • Use the same simple hx web fetch command for docs. No extra flags.
  • If Hyperbrowser is not configured, skip the docs fetch step and generate from failures only.
  • No emojis