Skip to content
probelabsPublic

About

Deterministic checklist gate for AI-driven development (Cosmopolitan APE)

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

12 Commits

Folders and files

Repository files navigation

qc

Deterministic checklist gate for AI-driven development. Agents are non-deterministic; a content-addressed attestation is not. qc records evidenced answers bound to file bytes and refuses to clear until every applicable item has one.

Quick start

Install (macOS / Linux / Git Bash; one static binary, no runtime needed):

curl -fsSL https://raw.githubusercontent.com/probelabs/qc/main/scripts/install.sh | sh

Turn it on in your repo:

cd your-repo
qc init
qc --version   # 0.2.0 or newer for everything below

Windows PowerShell and other install options: Install.

Now hand your agent a checklist. There are two ways: paste it into the chat, or point the agent at a markdown file.

Main use: give your agent a checklist

A checklist is plain markdown. qc turns it into a gate: the agent has to answer every item with evidence tied to the exact bytes it checked, and qc verify stays red until it does.

1. Paste a checklist into the chat

Paste this into Claude Code, Cursor, Codex, or any agent that can run shell commands. The first line is the instruction; the checklist goes under it:

Using the QC CLI, follow this checklist until it is done. If this checklist is only in the message, save it to a file first, then use that path. When qc verify is clear, show me the qc report --md output.

---
id: release-check
scope: [src/**]
---

- [ ] behaviour :: Does the change still do what the ticket says? @evidence(the function and line that implement it)
  - [ ] happy :: Run the example. Does the normal case work? @evidence(the output line, copied verbatim)
  - [ ] edge :: Is the boundary case covered by a test? @evidence(the test name and the test summary line with its count)
- [ ] tests :: Does the suite pass? @run(make test)
- [ ] files :: Which files changed? @scope(src/**, tests/**) @evidence(the `git status --short` output)

qc cannot read the chat, so the agent first saves the checklist as release-check.md (qc help tells it to). Then it runs qc plan, does the work, marks each item, runs qc verify, and shows you the report.

2. Point the agent at a checklist file

Keep checklists in your repo (a release checklist, a review checklist, a runbook) and name the file:

Using the QC CLI, follow the checklist in docs/release-check.md until it is done. When qc verify is clear, show me the qc report --md output.

The agent passes the path to every command. You can run the same loop yourself:

export QC_BY=agent
qc plan --checklist docs/release-check.md                          # questions + what evidence each needs
qc mark release-check/happy --pass -m "make run printed \`total=42\`" --checklist docs/release-check.md
qc verify --checklist docs/release-check.md                        # exit 0 only when every item is attested
qc report --md --checklist docs/release-check.md                   # the evidence table

QC_CHECKLIST=docs/release-check.md does the same as the flag. To make a checklist part of every qc verify, copy it to qc/checklists/<id>.md and drop the flag.

What you get back

From a real Claude Code run of examples/paste-a-checklist/ (trimmed):

| item | status | by | digest | evidence | parent | asked for |
|---|---|---|---|---|---|---|
| reservation-release/promise | CLEAR | agent | 2bf544b4… | src/gate.py:9 'return is_retry and age_hours <= 24' | | the line number and text of the return statement in src/gate.py |
| ↳ reservation-release/row-10 | CLEAR | agent | b11d0e9c… | python3 src/gate.py row: 'retry=True age=10 same=True expected=True' | promise | the age=10 output row, copied verbatim |
| ↳ reservation-release/boundary | CLEAR | agent | 7df1f0c3… | Added test_retry_at_exactly_24_hours_is_same to tests/test_gate.py; 'Ran 4 tests' / 'OK' | promise | the new test name and the unittest summary line with its count |
| reservation-release/suite | CLEAR | qc | | qc ran `python3 -m unittest discover -s tests -q`: exit 0 | | |
| reservation-release/bytes | CLEAR | agent | 102dd37b… | git status --short: ' M tests/test_gate.py' | | the `git status --short` output, one entry per file |

7 of 7 items attested to the current bytes.

Each row says who answered, what they cited, and what the checklist asked for. The digest covers the scoped files. If one of them changes later, the item opens again and qc verify exits 1.

Try it in two minutes: cd examples/paste-a-checklist && COPY=1 ./setup.sh. That makes a throwaway repo and puts the message on your clipboard. Paste it into Claude Code.

Checklist format

  • Frontmatter id must equal the filename stem: save id: release-check as release-check.md. scope is the files every item depends on.
  • - [ ] <name> :: <question> is one item. Indent two spaces to nest. Children are marked before their parent; a parent pass is refused while a child is open.
  • @evidence(...) tells the agent what its mark must name. qc plan shows it under the question, and qc report --md shows it next to the answer.
  • @run(cmd) items are run by qc itself. The agent cannot mark them.
  • @scope(...) overrides the files an item depends on. Edit one of them and the item opens again.
  • @human items are for a person. An agent is refused.
  • A mark like tests pass, done or lgtm exits 2. Evidence has to name a path, a number, a URL or a backticked command.
  • Prose between items is free text for the agent (for example: "go one item at a time and sleep 2 between items so people can follow"). qc parses only the - [ ] lines.

Install

qc is a Cosmopolitan APE universal binary — one release asset runs on Linux, macOS, and Windows from a shell (do not open it from a GUI file manager). Release assets: qc and qc.exe (same bytes; Windows-friendly name).

One-liners

macOS / Linux

curl -fsSL https://raw.githubusercontent.com/probelabs/qc/main/scripts/install.sh | sh

Installs to ~/.local/bin/qc. No root required.

Windows (PowerShell)

irm https://raw.githubusercontent.com/probelabs/qc/main/scripts/install.ps1 | iex

Installs to %LOCALAPPDATA%\qc\bin\qc.exe. The installer prints a PATH export if needed.

Windows (Git Bash / MSYS / Cygwin)

Same curl|sh as Unix — detects the Windows-ish host and installs ~/.local/bin/qc.exe:

curl -fsSL https://raw.githubusercontent.com/probelabs/qc/main/scripts/install.sh | sh

Overrides (pass env to the installer process, not only to curl/irm):

  • Unix / Git Bash: curl … | QC_INSTALL_DIR=~/bin sh, curl … | QC_VERSION=v0.2.0 sh
  • PowerShell: $env:QC_INSTALL_DIR = "$env:USERPROFILE\bin"; $env:QC_VERSION = "v0.2.0" then irm … | iex

Then:

qc help

On Apple Silicon, the first run may self-extract an APE loader under TMPDIR/HOME (no sudo required for a smoke test).

Build from source

Needs cosmocc:

git clone https://github.com/probelabs/qc.git
cd qc

# Makefile defaults COSMOCC to a local path; override for your machine:
export COSMOCC=/path/to/cosmocc/bin/cosmocc
make                 # writes ./qc (APE). Invoke it from a shell as ./qc
./qc help

In a consumer repo, qc init vendors that APE as qc/qc. Call it as ./qc/qc, or put qc/ on your PATH so plain qc works (the quickstart below assumes that).

60-second quickstart

From a git repo (throwaway is fine). Assumes qc resolves to the APE (export PATH="/path/to/qc-build:$PATH" before init, or export PATH="$PWD/qc:$PATH" after).

qc init
qc template list
qc add docs
qc plan

plan prints the ordered worklist. A checklist with applies_when starts with a decision:

export QC_BY=agent
qc decide docs --yes -m "README.md documents the public install and usage surface for this change"
qc plan

Answer with falsifiable evidence (a real path, digit, URL, or backtick span). Vague ticks are rejected; the tool names the next command, never a passing phrase:

qc mark docs/accuracy --pass -m "checked README.md section headings against the files present in the worktree"
qc mark docs/stale --pass -m "README.md is new; no prior install steps to retire"
qc verify                 # loop mode: scratch + committed state

When the worktree is what you intend to ship:

git add qc/
qc seal --staged         # promote matching scratch into this branch segment
qc verify --staged       # predicts CI: committed/index state only, no scratch
qc report --md           # claims table for the PR summary

Exit codes: 0 all clear · 1 open items · 2 config/parse error.

Core concepts

Digest and attestation

An attestation is a claim about content. Its logical key is (item_ref, digest). The digest is a 16-hex SHA-256 prefix over the item identity text plus the sorted, normalized hashes of files in the item's scope. Rewording the question or changing scoped bytes forces the item again. Forcing means staleness, never, or expiry — there is no second mechanism.

qc/ vs .qc/

Path Git Role
qc/ committed Vendored APE (qc/qc), checklists, config, state segments/base — the review surface
.qc/ ignored Scratch attestations, manifests, cache — disposable local workspace

rm -rf .qc/ is always safe (same class as qc reset). Hand-editing qc/state/ or .qc/scratch.qcs is not; only qc mark / qc decide / qc baseline / qc seal / qc compact write state.

Exit codes

Code Meaning
0 All clear
1 Open items (forced, failed, or awaiting decision)
2 Config or parse error

This triple is the machine API. Help and errors name the next command; they never name a string that would satisfy the gate (I5).

Verify modes

Mode State sources Question
qc verify scratch + committed Am I ready? (agent loop)
qc verify --staged index/committed only Will this commit pass CI?
qc verify --ci committed only, read-only Gate of record

Unsealed scratch is invisible to --staged and --ci. Forgetting seal does not corrupt state; it leaves items forced where it matters.

Also: --only <ref>, --quiet, --json.

Branch state without merge fights

mark / decide / baseline write scratch only. seal appends to a branch-owned qc/state/seg-*.qcs. compact (trunk only) folds segments into base.qcs. Parallel branches write different segment files, so merges are both-add — correctness does not depend on merge drivers.

I5 — every line is a prompt

Help, errors, and worklists name the next qc … command. They never print a phrase that would satisfy the evidence checker. Agents copy NEXT lines; applause tokens (lgtm, done, ok) are rejected at mark time.

QC_BY=agent

Set export QC_BY=agent (or pass --by agent) so attestations record provenance. Agents must not mark @human items — the CLI refuses and tells you to surface the item to a person.

Command reference

Topic help: qc help then qc help <topic>. Every topic ends with NEXT.

qc help [topic]

qc help
qc help mark
qc help verify

Topics: init add template plan decide mark baseline seal reset verify compact report.

qc init

Creates committed qc/ (vendors this APE as qc/qc), ignored .qc/, and one-line .gitignore / .gitattributes updates.

qc init
# Created qc/ (tool + checklists, committed) and .qc/ (local workspace, ignored)
# NEXT: qc template list

qc template list / qc template show <t>

Gallery of bundled checklists. show prints the minimal file you would copy.

qc template list
#   docs                 claims verified against code
#   code-quality         quality questions for application code
#   …

qc template show docs
# ---
# id: docs
# applies_when: The change is documentation
# scope: [docs/**, README.md]
# …

qc add <t> [--full]

Copies a template into qc/checklists/. Minimal by default; --full keeps every item.

qc add docs
# Copied builtin/docs@1 (minimal) → qc/checklists/docs.md

qc add security-review --full
# Copied builtin/security-review@1 (full) → qc/checklists/security-review.md

Adapt after copy; there is no runtime extends (legibility beats DRY).

qc plan [--json] [--checklist <path>]

Ordered worklist before work: forced items, questions, previous answers, decision requests. Nested items are indented, and each @evidence(...) hint is printed under its question as Evidence -m must name: …. --checklist <path> (or QC_CHECKLIST) adds a checklist file that lives outside qc/checklists/; every command accepts it.

qc plan
# 1  docs/@applies   FORCED (never)
#    Q: The change is documentation
#    NEXT: qc decide docs --yes|--no -m "…"

qc plan --json | head -c 200
# {"items":[{"ref":"docs/@applies","state":"FORCED (never)",…},…]}

qc decide <id> --yes|--no -m "…"

Applicability for checklists with applies_when. Writes scratch only. Exclusions are diffable Statements of Applicability.

export QC_BY=agent
qc decide docs --yes -m "README.md documents the public install and usage surface for this change"
qc decide code-quality --no -m "this change is docs-only README.md; src.rs is a stub not shipped"

qc mark <ref> --pass|--fail|--na -m "…"

Evidenced answer bound to the current worktree digest → scratch. A --pass on a parent item is refused (exit 2) while any of its children is open.

qc mark docs/accuracy --pass -m "checked README.md section headings against the files present in the worktree"
qc mark docs/stale --fail -m "README.md still mentions cosmocc.zip path that Makefile no longer uses"
qc mark docs/stale --na -m "docs/ is empty; only README.md exists and it is new in this PR"

qc baseline [-m "…"] [--item <ref>]

Grandfather current content at adoption. Writes scratch; seal afterward so trunk sees it.

qc baseline -m "adopting docs checklist on an existing README.md that already matches the code"
qc baseline --item code-quality/naming -m "greenfield src.rs with a single fn main; naming debt accepted at adoption"
qc seal --staged

qc seal [--staged]

Promote scratch entries whose digest still matches into this branch's qc/state/seg-*.qcs.

git add qc/ README.md
qc seal --staged
# seal: promoted 4 of 4 → qc/state/seg-main-28cd64.qcs
# NEXT: qc verify --staged

Without --staged, seal uses the worktree view; --staged matches what the index will commit.

qc reset [--item <ref>]

Wipe scratch, or one scratch entry. Committed qc/state/ is untouched. Same safety class as rm -rf .qc/.

qc reset --item docs/stale
# removed docs/stale from scratch. Committed state is untouched.
qc reset                 # wipe all scratch
qc plan

qc verify modes

qc verify                         # loop: scratch + committed
qc verify --staged                # index/committed only — predicts CI
qc verify --ci                    # committed only, read-only gate of record
qc verify --only docs/accuracy    # single ref
qc verify --quiet                 # exit code only (still prints forced/failed lines)
qc verify --json                  # machine-readable items[]

Example --only clear:

qc verify --only docs/accuracy
# CLEAR              docs/accuracy
# all clear

qc compact (trunk only)

Fold segments into base.qcs, one line per item, drop orphans, commit. Refuses off-trunk so parallel branches cannot rewrite shared base.

git checkout main
qc compact

# On a feature branch:
qc compact
# REFUSED — compact rewrites shared base.qcs; it runs on trunk only.
# NEXT: git checkout main && qc compact

qc report [--md]

Claims table for reviewers, CI summaries, and the end of an agent's task. age is when the answer was recorded. parent and asked for (the @evidence hint) come from the checklist. @run rows show qc ran \`: exit N, by qc`.

qc report --md
# | item | status | by | age | digest | evidence | parent | asked for |
# |---|---|---|---|---|---|---|---|
# | docs/accuracy | CLEAR | agent | 2026-10-28T15:04:14Z | b1a1bbdb28626343 | checked README.md section headings… | | |
#
# 1 of 1 items attested to the current bytes.

Evidence, @human, and @run

Evidence rejection (I5)

Every pass / fail / n_a needs -m evidence. Shape check at mark time:

  • Rejects empty applause (done, ok, lgtm, … and deny_evidence).
  • Requires enough substance and at least one falsifiable token: an existing repo path, a digit, a URL, or a backticked span.
  • Refuses common secret shapes (AWS keys, ghp_, sk-, PEM blocks, high-entropy blobs).
qc mark docs/accuracy --pass -m "lgtm"
# REJECTED — evidence must point at something falsifiable (a path, a number, a command, a link),
# not restate that checking happened.
# NEXT: qc mark <ref> --pass -m "…"

Rejection messages state the principle and name the next qc mark command. They do not reveal a phrase that would pass.

@human refuse

Items tagged @human require a human --by <name>. With QC_BY=agent the CLI refuses:

export QC_BY=agent
qc mark code-quality/diff-read --pass -m "read src.rs top to bottom; only fn main present"
# REFUSED — this item is @human. An agent must not attest it.
# Principle: human approval is provenance the PR author cannot forge locally.
# NEXT: surface code-quality/diff-read to the user; they run
#       qc mark code-quality/diff-read --pass -m "…" --by <their-name>

@run fail-closed

@run(cmd) items are executed by the gate (exit 0 clears). A missing binary is a loud FAILED with an install hint — never a skip.

qc verify --only code-quality/lint
# ruff: No such file or directory
# FAILED             code-quality/lint
#   @run exited 127. binary ruff not found. Install the command named in @run, then: qc verify
#   → The gate does not skip a missing checker. NEXT: install it, then qc verify

For agents

Start here: docs/agentic-guide.md — Claude Code / Cursor / AGENTS.md wiring, exact loop, CI sketch, failure recovery, and copy-paste snippets.

Paste-a-checklist walkthrough you can run in two minutes: examples/paste-a-checklist/.
Short skill card to load into the agent: SKILL.md.
Worked demo with a real fail-then-pass transcript: examples/agent-demo/.

Agent workflow

Point agents at SKILL.md (full guide: docs/agentic-guide.md). Contract:

  1. Set QC_BY=agent.
  2. Run qc plan at task start; treat questions as requirements, not an end exam.
  3. Mark at natural boundaries with falsifiable evidence.
  4. Never mark @human items — surface them to the user. The CLI refuses by:agent on @human.
  5. Never hand-edit qc/state/ or .qc/scratch.qcs.
  6. Loop: qc plan → work → qc mark … → qc seal --staged && qc verify --staged.
  7. On failure, read the worklist top-to-bottom and fix; do not re-attest blindly.

Bundled templates

Template Use when
code-quality Application behavior changes
api-change Public surface / compatibility
security-review Auth, secrets, untrusted input
data-migration Stored data or schema (includes @strict killers)
dependency-update Add or bump a dependency
release Release or cut
docs Documentation changes
ai-generated-code Agent-authored edits (memory APIs, weakened tests, suppressions)

qc add copies the minimal variant unless --full. Adapt after copy; there is no runtime extends (legibility beats DRY).

Build and test

make              # cosmocc → ./qc (APE)
make test         # kernel + shell acceptance + acceptance binary
make cover-host   # host gcc-15 MC/DC path (not the APE; see scripts/host-c-mcdc.sh)
make clean

Host coverage needs Homebrew gcc-15 / gcov-15 as wired in scripts/host-c-mcdc.sh. The APE product binary remains the cosmocc build.

Spec and skill

  • SPEC.md — specification v0.4 (purpose, invariants, digest, state, evidence, CLI, phasing).
  • SKILL.md — short agent loop contract.
  • docs/agentic-guide.md — agent wiring (Claude Code, Cursor, CI) + demo pointer.

Phase 1 is what this binary implements. Features marked Phase 2 in the spec (template diff, qc update, qc stats, qc diff / --carry, merge-queue mode, …) are not shipped here.

License

No LICENSE file is present in this repository. Treat the code as unlicensed until the maintainers add one — do not assume an SPDX identifier.

About

Deterministic checklist gate for AI-driven development (Cosmopolitan APE)

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages