All notable changes to this project are documented here. The format follows Keep a Changelog and this project adheres to Semantic Versioning.
The judge now has to quote the passage it judges, and an opinion whose quote is not there is dropped and said to be. The check is exact at word and number boundaries, so "2 million" is not found in "1.2 million". The AVeriTeC eval can now show the judge the NEIs too. It fixed 5 claims and broke none, but at p = 0.062 that is short of the bar set beforehand, so the product does not do it.
- The judge's quote is checked, and shown. A
SUPPORTEDorREFUTEDopinion now has to return the exact words from the passage that decide it, in a newquotefield. An opinion is dropped, with a note saying why, when its quote has no words, when a part of it is shorter than two words, or when it is not in the passage it was shown at word boundaries ("5 million" is not in "15 million"). The match folds case, curly quotes, dashes and the minus sign, spaces, the fraction slash and invisible characters such as the soft hyphen, and nothing else: a superscript stays a superscript, and a sign or a percent is never trimmed....may elide the middle of a quote as long as every part is in the passage, in order. The verified quote is printed in the judge line,judge (<model>): SUPPORTED — "<quote>" — <rationale>, and JSON gains aquotefield. AnNEIopinion asserts nothing and may quote nothing. Cached opinions are now keyed by the passage they were checked against, and those cached before this change are not served again: the next run asks.
- AVeriTeC eval:
--judge-neiand--no-fallback(scripts/eval_averitec.py).--judge-nei(requires--judge) also sends every read source whose verdict is NEI to the judge, with the passage the models came closest to deciding on, and the hypothetical column gains a second number in which those opinions vote too, with its per-label counts, its wrong-way count beside the product's, and an exact McNemar test against the product claim by claim. The report also counts opinions by model and kind of escalation, and the ones the quote check dropped by reason.--no-fallback(requires--judge) turns the judge's local fallback off for the run, so every opinion is from the configured model. Both flags get their own results file and report name. Measurement only: the product still never sends an NEI to the judge.
proofpath update keeps proofpath current, and it keeps the extras you installed. A
judge that hits Groq's per-minute limit now returns to Groq once the limit resets,
instead of staying on the local model for the rest of the run. The AVeriTeC numbers
were measured again, with the browser and the judge: search with judge-written queries
scores 0.483, the one gain that cleared the noise. That is still below the 0.708
baseline, and the report now says how often search calls a false claim supported.
- AVeriTeC eval:
--browser,--judgeand--fresh(scripts/eval_averitec.py). Every source now records the ladder step that read it and, under--judge, the judge's opinion when its verdict was in the escalation band. The report adds two numbers beside the product's: the accuracy with the browser's pages dropped, and a hypothetical one in which the judge decides the low band. The product never does the second, and the report says so. Each run setup gets its own results file and report. Measured on 2026-10-05: gold URLs 0.371; search with sentence queries 0.371; search with the judge writing the queries 0.483 (14 claims fixed, 4 broken, p = 0.031). The majority baseline is 0.708. proofpath updateinstalls the latest release from PyPI, andproofpath update --checkonly says whether there is one. In the TUI they are/updateand/update --check. Running the command is the consent, so nothing asks. uv tool, pipx and pip installs are each updated with their own tool; pip falls back touv pipwhen it fails and uv is on PATH. An editable install is never touched: the command names its source directory and says to update it withgit pullanduv sync.- The installed extras are kept. They are written into the install spec
(
proofpath[browser]), never with a version pin. A plainuv tool upgradewas measured to drop packages added to the tool's environment after it was installed, which would have removed the browser wheels you had consented to. - The new version is read back from a fresh process. An update the resolver held
back below the latest version is a failure that names the version it installed.
On Windows a failed command adds "close proofpath and run the command above",
since a running
proofpath.execan be locked. - Temporary environments are left alone. In a
uvxorpipx runenvironment nothing runs. The command says thatuvx proofpath@latestorpipx run --no-cache proofpathalready fetches the newest. - Commands paste back on every platform. Each command is logged as
$ <cmd>; the one printed for running by hand has no$. On Windows both usecmd.exequoting, and so does the browser install's log. The version check after the update runs the interpreter isolated (-I), so a checkout in the working directory cannot answer for the install. - In the TUI, an update and a run never overlap. A bare
/updateis refused while a run or a mirrored verb is still going, because the install replaces the code they are executing;/update --checkis not refused. While an update is installing, every/checkand mirrored verb is refused until it finishes, including a second/update. - Exit codes:
0for up to date, updated, ahead of PyPI, editable, temporary, or (with--check) an update available.1means the check or the update failed.permissions.networkis read by the fetch ladder's own rule:deny, andaskwithout a terminal, send nothing to PyPI and fail rather than report "up to date".askin a terminal goes ahead with a note.
- At 80x24 the
/list now caps./updatemakes eleven verbs and 80x24 has room for ten rows. The last two,/quitand/update, sit behind... 2 more.updateis last on purpose: the banner's hint line already names/quit, and/helpstays visible. One more row, at 80x25, shows them all. packagingis a declared dependency. It was already installed through other packages;updatenow imports it directly.- The judge goes back to Groq once its minute limit resets. A 429 whose
Retry-Afteris at most 120 seconds (60 when it names none) now switches to the local model only until that wait has passed:Groq limit reached — judging with local ollama qwen3.5:9b until it resets (~30s). The first request after it asks Groq again, from one thread only; if Groq answers, the judge stays on it and says so once (Groq answering again — local ollama qwen3.5:9b stood in for 4 requests), on stderr, in the TUI and in the report's footer. Another short 429 starts a new wait, quietly. Nobody waits: the local model answers while Groq cools down. A daily limit (a longerRetry-After), a 413, a rejected key, a 5xx or no answer still switches for the rest of the run, with the same notice as before.
An opt-in, larger NLI model for scientific sources, chosen by a measured bake-off and downloaded only with your consent. Reports now admit when a model has no real low confidence tier, and CI type-checks the whole package.
- An opt-in accurate NLI model. The bake-off's winner,
MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli, runs at k=2 with cuts calibrated on SciFact train. Its SciFact dev macro-F1 is 0.697, against 0.580 for the default model. It is not better on news: on AVeriTeC claims with a readable source it scores 0.475, against 0.537 for the default model at k=2. It is recommended for scientific sources, not for news. It is 643 MB, against 244 MB, and the default install does not change. proofpath check --accurateuses the accurate model for one run.[models] nli = "accurate"uses it for every run; set it withproofpath config set models.nli accurateor in the/configpanel.permissions.install_modelguards the 643 MB download. It isaskby default: the run asks once, naming the model and its size, with the browser question's answers (once, always, no, never), and "always" and "never" are saved. With no terminal,askis treated asdenyand the denial is reported. A denied or failed download falls back to the default model and says so in three places: a note with the reason and what would change it (dropped by-q, and on stderr under--format jsonorsarif); in text mode, annli default model chosen; requested accurate (<reason>)line that-qkeeps; andnli_requested: accurate (<reason>)in the report'smodelsblock. That block now also names thekthat ran.
- A note when a run has no real low tier. A run whose
mediumcut sits within 0.01 ofdecidenow says "this model has no real low tier on SciFact, so an asserted verdict is almost always medium or better". Default runs say it too: the default'smedium(0.457948) sits 0.008 abovedecide(0.45), which the 2026-09-12 tier sweep already measured. When the no-high-tier note also applies, the two are joined with "; ". - The no-high-tier note now reads "this model earns no high tier on SciFact dev, so medium is the strongest confidence shown".
- CI runs strict mypy over the whole of
src/.
- An NLI bake-off (
scripts/eval_nli_bakeoff.py, report indocs/eval/2026-10-01-nli-bakeoff.md). It compares three NLI models, three values of k and three aggregation rules, 27 combinations in all. Each combination is calibrated on SciFact train and scored on SciFact dev and a frozen AVeriTeC snapshot. The winner is picked by a rule written down before any number was seen.- Winner:
MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanliwith k=2. SciFact dev macro-F1 is 0.697, against 0.580 for today's default and 0.608 for the best configuration of the default model. - AVeriTeC stays below its majority baseline: 0.475 on claims with a readable source, against 0.708.
- The report lists its own limits. The main one is that dev both chose the winner and reports its score.
- Winner:
Wikipedia can be read again, and a verdict now says more about how much it rests
on: an abstract, or a claim too short to pin down. The Tavily key is pasted straight
into /config.
Tavily API:in/config. The search section of the settings panel has a key row: press Enter, paste the key, Enter again. It is saved to the.envin your config directory (mode 0600 on macOS/Linux), never toconfig.tomlor the project's.env, and it is never shown: the row says only whether a key is saved and where. Backspace removes it. Saving a key does not turn search on: whilesearch.provideris nottavilythe row says the key is unused, and withtavilyselected and no key it says one is needed.- A note on claims too short to pin down. A claim like "openai won" is supported
by almost any page about any win. A claim with two content words or fewer, and no
linking verb (
is,are,was, …) between them, now carries a "short claim" note on each of its verdict rows, SARIF included. The note tells you to check that the quoted passage is about the event you meant. It is a rule, not a model, and it changes no verdict, tier or exit code.
- Wikipedia pages no longer read as
UNVERIFIED (blocked). The bot-wall check matchedcaptchainside MediaWiki's inline configuration script, so every Wikipedia page was reported blocked at all four ladder steps despite answering 200. Challenge phrases are now matched outside<script>/<style>contents and only on small pages.cf-chland Cloudflare's_cf_chl_optstill mark a wall wherever they appear. Othercf_chltokens (such as__cf_chl_f_tk) count only on small pages, because they linger in the URLs of real articles. - A verdict judged against an abstract now says so on its own row. A short page
or an abstract-only source already had its
LOW CONFIDENCE (abstract only)row, but the claim's verdict row (SUPPORTED (found by proofpath),NOT SUPPORTED, …) did not, so read alone it looked like a full-text verdict. It now carries "judged against the abstract only, not the full text", in the terminal, Markdown, JSON and SARIF.
A text that cites nothing can now be checked: proofpath searches the web for it. And the judge no longer goes quiet when Groq's free tier runs out.
- Evidence search for claims with no source (experimental). A pasted post or plain
text with no link, no
[12]marker and no bibliography used to end in a parse error. Withsearch.providerset totavily(a free key inTAVILY_API_KEY) orsearxng(plussearch.base_url), up to 5 check-worthy sentences are searched, up to 3 pages each. Every page is fetched and read like any other source; the search result's own snippet is never evidence. Findings are markedFOUND BY PROOFPATH (not cited by the author):SUPPORTED/REFUTED (found by proofpath)carry their passage, and a claim nothing confirmed isNO EVIDENCE FOUND (searched), neverREFUTED. Setting the provider is the consent;--no-searchturns it off for one run andpermissions.web_search = denyfor good. A run that searched only some of its check-worthy claims says so and exits 1. AVeriTeC search mode scores 0.404 against a 0.708 majority baseline, which is why it is marked experimental. - Queries and translation by the judge. With
--judge, the model writes the search queries and translates a non-English claim; the report shows your sentence and achecked as: …line. Without a judge, a claim that is not in English isUNVERIFIED (language not supported)rather than checked by English-only models. - Local fallback judge. When Groq hits its rate limit, rejects the key or stops
answering, the run switches at once to
qwen3.5:9bin your local Ollama, if it is installed. There is no retry and no wait, and the request that failed is sent again, so no batch is lost. The switch and its cause appear on stderr, in the TUI just above the prompt, and in the report. The next run tries the configured API first again. proofpath never pulls the model;judge.fallback offturns this off. Without the model, the run says how to add it (install Ollama,ollama pull qwen3.5:9b). /configrows for[search]andpermissions.web_search, and footer lines for what was searched.scripts/eval_averitec.py --searchmeasures the product's own search path, leaving out the claim's fact-checking site.
- The
ollamajudge's default model is nowqwen3.5:9b(wasllama3.1), always asked with reasoning off. - The X hint says a pasted text with no link is searched when a provider is set.
- Pages proofpath found on its own are fetched without the contact address.
- A bibliography entry that is only a DOI or arXiv URL is never reported as a
GHOST REFERENCE. It resolves, or it isAMBIGUOUS. A cache migration (v5) drops ghost results cached under the old rule, so they are checked again.
Two more platforms a post can be read from without an account: Lobste.rs and Lemmy.
- Lobste.rs.
check --url https://lobste.rs/s/<id>reads a story — its title, its submitted address first and the links in its text — and/c/<id>or a#c_<id>fragment reads one comment, never the thread above it. Read from the.jsontwin of each page; no key, no account. A moderated comment isUNVERIFIED (blocked), a deleted oneUNVERIFIED (unreachable), as spec §15 keeps them apart. An anchor pointing back at the story's own discussion page is not a source and is left out. - Lemmy.
check --url https://<instance>/post/<n>and/comment/<n>read from the/api/v3every instance serves without a login. A post's submitted address comes first, then the markdown links in its body.removed(a moderator's doing) isUNVERIFIED (blocked);deletedisUNVERIFIED (unreachable). A private instance answers an anonymous reader with the same400as a missing post andnot_logged_inin the body, so the body's token decides:not_logged_inisUNVERIFIED (blocked),couldnt_find_postisUNVERIFIED (unreachable), and a400naming neither isUNVERIFIED (unavailable)— never a guess about the post. Lemmy is known by its host — thelemmy.prefix or one oflemm.ee,sh.itjust.works,beehaw.org,programming.dev,feddit.org— never by/post/<n>alone, which is how any blog addresses an article; a post on an instance outside that rule is read as a page, as before, and no/api/v3request is ever sent to a host that did not say it is Lemmy.
- The "platform not read in this version" hint, the
check --urlhelp and the README table name all seven platforms. polite.ProviderErrorcarriescode: the one snake_case token a JSON error body'serrorfield named, when it named one, and nothing else from the body. It is what lets the Lemmy reader tell the two facts above apart; no other caller reads it yet.
The terminal looks like the product: the wordmark in both themes, a settings panel, and commands that show themselves as you type.
- TUI: slash suggestions. Typing
/lists every command above the bar with what it wants and what it does; the list narrows per keystroke (/h→/help),Tabcompletes the selected row,↑/↓move the selection while the list shows,Enterruns what is in the bar. On a short terminal the list windows around the selection and says… n more(docs/superpowers/specs/2026-09-18-wordmark-banner-design.md§10). - TUI:
/configis a settings panel. Arrow keys move between the settings,←/→change a choice (permissions.install_browser,permissions.network,fetch.respect_robots,judge.provider),Enteredits a text setting in place,Backspacerestores its default,Esccloses; every change is written at once and reported under the panel; thekeyline says where the judge's API key was found and never shows it./config show|path|set|checkare unchanged (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md§12).
- TUI: the wordmark everywhere. The
plaintheme draws the sameProofPathmark in ASCII and the raven is gone from the terminal;richruns it through five crimson bands left to right with the shadow one tone darker; a full-width rule closes the banner in both themes; at 129 columns and up the version and hint rows sit beside the mark, narrower terminals keep them under it, and below 69 columns only the two rows and the rule are drawn (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md§8). - TUI: the bar's frame runs the same gradient. The rounded frame around the input
is drawn in the wordmark's five crimson bands, light at the left, dark at the
right; the caret keeps the next run's accent
(
docs/superpowers/specs/2026-09-18-wordmark-banner-design.md§9).
- TUI: typing always reaches the bar. A click on the log's background, the banner
or a panel no longer takes the keyboard away; a printable key typed while a finding
line or a permission button is focused goes to the bar
(
docs/superpowers/specs/2026-09-18-wordmark-banner-design.md§11). - Tests: the cache CLI test no longer expires with the calendar. Its "fresh" entry was dated 2026-09-11 and crossed the 7-day TTL on its own.
The bar takes a whole post, and the rich theme opens with the wordmark.
- TUI: the
richbanner is a wordmark.ProofPathin block letters, crimson on deep crimson, with the version and hint lines under it; the Braille raven is gone. The ASCII raven of theplaintheme and the website's raven stay (docs/superpowers/specs/2026-09-18-wordmark-banner-design.md).
- TUI: a multi-line paste is no longer cut to its first line. Textual's input
keeps the first line of a bracketed paste; the bar now holds the whole text behind
a one-line summary (
pasted · 3 lines · 118 chars · enter to check, esc to drop),Enterchecks it whole andEscdrops it. The run is labelled/check pasted textand the paste stays out of the command history (docs/superpowers/specs/2026-09-17-multiline-paste-design.md).
Paste any link: a page is read as the document itself.
- A page as the target. A bare address on any host that is not a social platform
— pasted into the TUI bar or given to
check --url— is read as the document itself instead of being refused as "a page, not a post". The page is fetched up the same ladder its sources are, cut into one paragraph per block, and cites the way a post does: each paragraph's sentences are checked against the pages that paragraph links to. A page that prints a reference list is paired by citation number like a paper. A page the ladder could not read ends the run in the ladder's own words (UNVERIFIED (blocked),… (unreachable), …), with the setting to type when the browser step was the missing one; a page with no links is reported rather than passed as clean; the page's own address is never one of its sources (docs/superpowers/specs/2026-09-17-page-target-design.md).
The TUI says what to type when it stops you, and every tag gets a GitHub Release.
- GitHub Releases. The release workflow now creates the GitHub Release for each
tag, with the version's CHANGELOG section as its notes (
scripts/release_notes.py) and the wheel and sdist attached; the eight earlier tags were released the same way by hand on 2026-09-17.
- Permission hints. The TUI's consent question names the
/allowanswers instead of the terminal's keys; a run that skipped sources because the browser was not permitted says which setting turns it on (/config set …in the TUI,proofpath config set …elsewhere), and a run the network setting stopped says so in its once-per-runnetwork: not permittednote; a page given where a post was expected says to paste the claim as text;/allowwith nothing waiting says where the question appears. - README screenshots are real runs with the raven (
docs/eval/tui-raven-{rich,plain}.svg, method indocs/eval/2026-09-17-tui-raven.md).
loading models …no longer outlives the loading in the TUI. The note is transient and is taken back when theVerifyingstage ends; a finished run no longer looks like it is still loading.
A raven in the terminal, and a website to go with it.
- A website, under
site/: a static Astro page with a replay of a real/checkrun, the three questions, the six rules, and the CLI surface. The engravings are public-domain plates (Met Open Access, Wikimedia Commons) halftoned at build time bysite/scripts/dither.mjsfromsite/images.manifest.json; the outputs are committed, so the build never fetches. Design notes indocs/superpowers/specs/2026-09-16-landing-page-design.md.
- The pet is a raven. The ferret is gone from both themes:
richdraws a two-tone pixel raven in Braille cells (fifteen columns, eight rows, the text beside it, its ground running to the right edge),plaina small ASCII one. The[PROOF]stamp and the blink/busy/tail animation are gone with it; the widget'sset_busy/flashremain as no-ops. Design notes indocs/superpowers/specs/2026-09-16-raven-pet-design.md.
The TUI gets a shell's reflexes, and sits on the terminal's own background.
- The TUI bar behaves like a shell prompt.
↑/↓walk a command history kept across sessions (state_dir()/history, 500 lines);Tabcompletes/verbs,/allowanswers and/cancel #n;PageUp/PageDown,Shift+↑/↓andCtrl+Home/Endscroll the log without leaving the bar;↑/↓on a focused log line step between lines and typing returns to the bar;Ctrl+Lclears finished runs and keeps the coverage footer; a mouse selection is copied byCtrl+C/⌘Cinstead of quitting./helplists the keys.
- The TUI sits on the terminal's own background instead of Textual's grey: every
surface is
ansi_defaultand palette names reach the terminal untranslated, so the app no longer shows up as a layer over the terminal it runs in.
Sources that are not papers: a post's links, and the coverage block that finally says why
a source could not be read. Also the first end-to-end measurement on real web claims, which
says plainly that this is not yet the thing to point at a news story
(docs/eval/2026-09-16-averitec.md).
check --urlreads a post and verifies the links inside it, never the post's own words (spec §6.2). Bluesky (public.api.bsky.app) and Hacker News (the Firebase API) are first-class and need no account; Reddit reads with a free app you register yourself (REDDIT_CLIENT_ID/REDDIT_CLIENT_SECRETin.env); Mastodon is best effort per instance; X cannot be read and says so, asking you to paste the text. Pasted text with links in it is verified the same way.UNVERIFIED (credentials missing)(spec §15): a platform that reads only with a credential this machine does not have is its own state, never "unreachable". No request is made, both variable names are printed, and the secret itself never reaches a log, an error or arepr.- The coverage block now prints a line per reason on the terminal and in the markdown
report, not only in SARIF.
blocked: 3,credentials missing: 1,unreachable: 2— a source with no recorded reason is listed rather than dropped, so the lines account for every unverified source. Product rule 6 on its main surface. The TUI footer does not carry them yet. - A
providers/package (academic,web,social) behind theEvidenceProviderprotocol of spec §5.2, so a new source family no longer means touching the core. The academic and web paths came through byte-identical, pinned by a golden report. docs/eval/2026-09-16-averitec.md: 100 AVeriTeC dev claims through the whole product.
- A bare DOI or arXiv URL is resolved as a record again instead of being fetched as a web page, so it keeps its retraction check and its open-access full text.
- A
doi:source is read through the open-access chain and retraction-checked by the academic provider whoever resolved the entry — a warm cache could previously route both to the web ladder, where a missing retraction notice was reported as "no notice". - The
Resolvingstage no longer names Crossref and Semantic Scholar on a run that asked neither. - A document that cites by linking is never paired as if it printed a numbered bibliography, and vice versa: a PDF with a bibliography and no detected marker could previously pair a body sentence to a reference it never cited.
check --urlconsults the network permission before reading anything; a denied run makes no request at all.
- AVeriTeC dev, 100 claims: 0.270 3-way accuracy against a 0.708 majority baseline
(4-way 0.240). A third of the claims had no readable source; on the rest the score is
0.361. Every
Supportedclaim was missed. Nothing was tuned after the measurement.
The judge layer: an opt-in LLM second opinion and an opt-in model-written summary.
The default run still makes zero LLM calls, and nothing the model says can change a
verdict. Live run on Groq in docs/eval/2026-09-15-judge-live.md.
check --judge(judge.py,verify.py). After the local verdicts are final, thelow-tier ones — never a numeric mismatch, never a claim without a quoted passage — go to the model in batches of up to 20 items (about 7k tokens), each with its claim and passage. The opinion (SUPPORTED | REFUTED | NEIplus a one-sentence rationale) is attached beside the verdict: a= judge (groq openai/gpt-oss-120b): …line under a finding it disagrees with, ajudgecolumn in the markdown## Checkedtable, andjudgefields in the JSON. The localVerdict, the finding kind and every state are untouched (spec §11.1). Opinions are cached in the newjudgementstable (schema v4, additive; wiped with the verdicts when a source's text changes), so a re-run asks nothing, andmodel_idis untouched, so toggling--judgenever invalidates a verdict.check --summarizeand the TUI's/summarize: one extra call over the finished markdown report, run after the report is final, off by default in both front-ends, printed assummary (model-written, groq openai/gpt-oss-120b) …and as## Summary (model-written, …)in the file.--summarizealone is exactly one call.JudgeClient: one adapter for Groq (defaultopenai/gpt-oss-120b), Gemini and Ollama over the OpenAIchat/completionsshape; strict JSON-schema output with ajson_objectfallback,reasoning_effort=lowwith a fallback for providers that reject it,Retry-Afteron 429 (capped, accounted), exponential backoff on 5xx, thenJudgeUnavailable. The key comes from the environment or.env, never from config, never appears inrepr, errors or logs; provider bodies are never echoed. Prompts are packaged template files (proofpath/prompts/review.md,summarize.md).- Cost on every surface: the
JudgingandSummarisingstage lines carry calls and prompt/completion tokens; the footer counts the calls;Report.judge_costandmodels["judge"]land in the JSON. - An unanswered judge is reported, not hidden:
judge unavailable after N calls (HTTP 401 from …); local verdicts standin the stage line, the report header (judge status:/summary status:), the JSON and an unsuppressed terminal line — a-qor piped run cannot look like a judged-clean one. The SARIF log does not carry it. The run never fails because of the judge. - Gemini prints its data-use warning once per run (spec §11).
check --judge/--summarizeno longer exit witharrives in v0.3.- Cache schema v4 (
judgements); a v1 file still migrates through the whole chain. - The markdown
## Checkedtable always carries ajudgecolumn; without--judgeevery cell is—, so a v0.2 report and a v0.3 one differ by that column alone. --format jsongainedsummary,summary_model,judge_costand per-resultjudgefields; every one of them isnullon a default run.
The TUI's second look. No behaviour change: every state word, every honesty sentence,
the exit codes, the scheduler and the one-shot CLI are exactly v0.2.0's. Design in
docs/superpowers/specs/2026-09-15-tui-v2-design.md; the by-hand session in both
themes, with SVG screenshots, in docs/eval/2026-09-15-tui-v2-live.md.
- Two themes, one truth (
tui/theme.py).richdraws box borders, Unicode glyphs and truecolor tones when the terminal gives evidence of them (COLORTERM, Windows Terminal, iTerm2, kitty, WezTerm, Ghostty, VS Code, Terminal.app);plainis a pure-ASCII, ANSI-16 look (v0.2.0's⏺ ✗ ⚠ › ⧉become* x ! > [copy], and findings now print their= note:lines as the CLI does), chosen underNO_COLOR,--no-color,-q,TERM=dumb, legacy conhost, a CJK locale, or any session without truecolor evidence (an SSH or tmux session that stripsCOLORTERMgetsplain).PROOFPATH_THEME=rich|plainoverrides detection. The meaning colours stayui.py's tables; a theme changes how a meaning looks, never what a word means. Spec §13.1's "pure ASCII" rule for the pet now bindsplainonly. - The pet (
tui/pet.py):richdraws a seven-line ferret with a real tail running to the[PROOF]stamp;plainkeeps the three-line one unchanged. Same eyes, same blink, same reactions; the tail wags inrichwhile a run works. - Run panels (
tui/widgets/run_block.py): inricheach run is a rounded panel in its accent, the command and the state word on the top border, the coverage on the bottom one; the stages are a fixed-column table (symbol, name, summary with·separators, attribution, elapsed) and the active stage carries a real▰▱progress bar withdone/total. A finished stage that left something unverified keeps the⏺mark instead of a tick. Below 60 columns the borders go and the flat rows return; a run that crossed the floor either way says its coverage exactly once. - Findings (
tui/widgets/finding.py): the state word is a badge, the location a fixed cell, the tier right-aligned; the finding's notes and the claim (you) and the passage (source) it was checked against are printed under it. A tier-less badge keeps its row at 60–79 columns.plainprints the notes as the CLI's own= note:rows and keeps[copy]. - Footer (
tui/widgets/footer.py):richdraws the coverage as a proportional█▓░bar in the three meaning colours over the counts line;plainkeeps thekvlines. The prompt wears a one-line rounded border in the run's accent. - Structure (OPEN-ITEMS 12.4):
tui/app.pyis split —theme.py,pet.py,verbs.pyandtui/widgets/(run_block,finding,footer,prompt,banner,_shared) — and is now 681 lines of composition, scheduler wiring and slash commands, no rendering.
The interactive front-end, author-year citations, SARIF output and a cache for the
two lookups that made a warm re-run slow. Bare proofpath now opens the TUI; the
one-shot verbs are unchanged for CI. Live checks in docs/eval/2026-09-15-v0.2-live.md.
- TUI (spec §13.1): bare
proofpathopens atextualsession — a four-line banner with the pet, one prompt bar, and a scrolling log of run blocks. Every CLI verb is a slash command (/check,/resolve,/fetch,/config,/cache; plus/allow,/cancel #n,/help,/quit), a bare verb waits for its argument (Escleaves it), and a line that is not a command is something to check. Runs are scheduled concurrently through oneScheduler(network stages in parallel under a shared politeness limiter, the NLI model one run at a time), each run is cancellable mid-flight and keeps what it had decided, the coverage footer never scrolls away, and the §7.1 permission prompt is drawn inline under the stage that hit the wall with[allow once] [always] [no] [never]buttons (or/allow …). Every log line works by mouse and by keyboard: run headers fold, stage rows hide their summary, findings open the full quoted passage,⧉/ccopies it, a finding's reference is an OSC 8 link to its source. The pet's eyes blink, watch a run and react to its result — only with colour, never under--no-coloror-q. - Author-year citations (spec §9, §17):
(Smith et al., 2020),Smith (2020),(Smith, 2020; Jones, 2019),2020a/2020bcollisions,ibid.andop. cit.back-references, and mixed(Smith, 2020; [12])all pair with their bibliography entry. Hand-built set of 55 passages, 83 expectations, rate 0.940 (docs/eval/2026-09-12-pairing-author-year.md); the numeric set is unchanged at 108/109. A marker no entry matches is reported asUNRESOLVED MARKER, never guessed. check --format sarif(spec §13.2): the run as a SARIF 2.1.0 log for VS Code or any SARIF 2.1.0 viewer — one result per finding on its line, the quoted passage in the message of every asserting result, the exact honesty state inproperties, and the run's coverage inruns[0].properties.--out FILEwrites the same document. Validated against the schemastore schema, vendored intests/data/sarif-schema-2.1.0.json; not yet exercised against GitHub code scanning.- Resolution and retraction cache (cache schema v3):
resolutions(30-day TTL, keyed by the marker-free folded entry) andretractions(30 days for a notice, 7 for its absence). Neither anUNVERIFIED (provider unavailable)nor a retraction check every provider failed is ever stored. A warm re-run of the seven-reference draft: 1.35 s wall clock, 0 network calls (v0.1: 15.8 s).proofpath cachecounts both tables. - The §7.1 aggregate line
skipped N source(s) because the browser was not permittedis printed bychecktoo (terminal and markdown), never dropped by-q. browser_binary_present(): the consent gate checks for a chromium build underPLAYWRIGHT_BROWSERS_PATHor the platform default, so a half-installed environment runs the idempotent installer instead of failing inside the fetch; the consent log gainsbrowser binary: found|missing.commands.py: the shared wiring behind every mirrored verb (resolve_reference,fetch_target,config_*,cache_*) returns result objects and never prints, socli.pyand the TUI can only differ in how a result is drawn.- Bibliography fallback for PDFs without a
Referencesheading (paged formats): the last contiguous run of numbered paragraphs is read as the list. AlphaFold: 0 → 17 references found, 97 markers now reported as unresolved instead of invisible. - Resolver rescues:
First Last and First Lastauthor lists resolve instead of landing inAMBIGUOUS; an arXiv id whose record agrees on author and year is accepted asRESOLVED (low confidence)the way a DOI already was. Ghost set 274 rows: false-ghost 0.0 %, ghost recall 99.1 % (docs/eval/2026-09-12-ghosts.md). tests/data/draft-author-year.md, a committed author-year draft with an offline smoke test; live: 6 citations paired, 0 unresolved.
- Cache schema v3 (migrated in place from v1/v2; versions compared as integers).
Claims.unsupportednow means "a style this version cannot pair" and is empty by construction;UNSUPPORTED CITATION STYLEis left for footnote-only and superscript-letter styles. Markers that pair with nothing areunresolved.resolve.looks_unindexedreads only the initials-path author patterns: a full-name list (First Last and First Last) no longer counts as "this entry printed an author list", so a title-first book keeps its title (see Known issues for the price).--format sarifonresolveorfetchnow saysapplies to check onlyinstead of naming a future version.- Stage row
ClaimsreadsN citations, M unresolved(wasM unsupported).
- A provider answering with a non-JSON body (a bot wall or maintenance page under a
200) escaped
resolve.pyas a rawJSONDecodeErrortraceback. Every provider body is decoded in one place and that failure is aProviderError, soresolve,fetchandcheckreportUNVERIFIED (provider unavailable)and never read it as evidence the work does not exist. - The §7.1 prompt said
blocked this request (HTTP 200)for the empty-body bot wall; it now saysanswered without readable text (HTTP 200). resolveon the CLI leaked one HTTP client per invocation;commands.resolve_referenceowns and closes it.- A retraction check every provider failed was recorded as "not retracted". It now
raises, is reported as
retraction check unavailableand counted in the stage summary (1 unavailable), and nothing is cached, so the next run asks again (product rule 2). Found alongside it: the fetching stage replaced a source's notes instead of appending, which would have dropped that very note. - Versions in the cache file compared as strings (
"10" < "9"); compared as integers now.
api_callsin the footer counts LLM calls only (none yet), so a cold run that spent 30 s on Crossref and Semantic Scholar still prints0 API calls; the provider lookups are shown on the stage lines instead.- A cached resolution reprints the notes it was stored with (for example an
openalex unavailable (HTTP 429)from the day it was resolved) as if they were current. proofpath check -names its SARIF artifact-; give the draft a file name when the log is meant for a viewer.- Author-year pairing matches the first author and the exact year;
(Lindqvist, 2019)against "Okafor, C. and Lindqvist, S." and a year off by one are reported unresolved, and two surnames sharing a last word (Berg/van der Berg) are reported ambiguous rather than guessed.Smith 2020with no comma is not a marker;(WHO, 2020)does not pair with "World Health Organization". - An unnumbered two-column bibliography is cut at line breaks (RoBERTa: 103 entries for ~50), so author-year pairing over such a list resolves few items. The largest open item for author-year pairing.
- The headless bibliography fallback takes only the last run of numbered paragraphs (AlphaFold: 17 of 84 entries) and drops the prose printed before a block's first entry.
- A fabricated
First Last and First Last. Title. Venue.with no year between the names and the title reachesUNVERIFIED (not in bibliographic indexes), notGHOST(the price of the rescue above; rule 3 outranks recall). A proceedings-volume record can accept a fabricated paper cited into that volume asRESOLVED (low confidence)— the ghost set's one fabricated acceptance (0.9 %). - A cold run of a long bibliography is still serial through the providers (129 references: minutes); only the TUI's concurrent scheduler runs them in parallel.
- In the TUI the
loading models …note is drawn after theVerifyingrow it precedes (it is emitted inside that stage). - A
#nrun reference in the TUI is a click target that folds its block, not a hyperlink (spec §13.1); a run number has no address to open. - A provider body that is valid JSON but not an object (a bare list or string) still
escapes
resolveas an error, exit 2, rather than being reported asUNVERIFIED (provider unavailable). /quitin the TUI waits for an in-flight mirrored/fetchto finish before the app exits; a/checkrun is cancelled, a/fetchis not.
First working release: proofpath check verifies a document's citations end to end
and writes a report that states its own coverage.
-
proofpath check TARGETas the v0.1 surface: compiler-style diagnostics, a markdownreport.md,--format json,-q, and exit codes0/1/2. -
Confidence tiers calibrated on SciFact dev rather than chosen by hand (
decide=0.45,medium=0.457948,high=0.99933;docs/eval/2026-09-12-tiers.md), shipped aspipeline.DEFAULT_THRESHOLDS. -
Nine live user-like runs recorded in
docs/eval/2026-09-12-v0.1-live.md, andscripts/zero_network_check.py, which guards both HTTP clients and re-verifies a document: with the network gone the fetch ladder makes no attempt at all, because resolution produces no identifiers to fetch with. -
Config and permissions module:
config.tomlunder the platform config dir,proofpath config/proofpath config set permissions.<key>, and the rule that anaskpermission without a TTY resolves todenyand is reported (spec §7.1). -
Core types (
Verdictcannot beSUPPORTED/REFUTEDwithout a passage), device selection (CUDA → CoreML → CPU), sentence retrieval overfastembedembeddings scanned with numpy, ONNX NLI entailment oncross-encoder/nli-deberta-v3-base, and the aggregation pipeline. -
Reference resolution (spec §8): Crossref + Semantic Scholar first, then arXiv, Open Library and OpenAlex before any ghost call; identity decided only by field agreement against the raw string; DOI / arXiv id resolved directly; retraction check via Crossref's Retraction Watch data and OpenAlex; new state
UNVERIFIED (not in bibliographic indexes); per-host throttling,Retry-After, OpenAlex daily-budget handling.proofpath resolve "<reference>"on the CLI. Hand-built ghost set (106 real, 100 fabricated, 20 mutated) andscripts/eval_ghosts.py: false-ghost rate 0 %. -
Persistent cache: one plain SQLite file (
sources,raw_textwith 7-day TTL,chunkswith float32 embeddings,verdicts), a schemaCHECKthat refuses an asserted verdict without a passage, andproofpath cache/cache path/ls/show/clear [--expired]. -
Numeric claim layer (spec §10): percentages, factors and unit counts with direction are compared before NLI; an unambiguous contradiction is refuted by rule with both figures named (
Verdict.reason). Conservative by design: one comparable figure on each side, change never against level. -
Judge settings (
[judge]in config, Groq default) withproofpath config checkandproofpath config set judge.<key>; API key resolved from the environment or.env, never stored or printed..env.exampleadded. -
SciFact loader pinned to the AI2 tarball by sha256, evaluation metrics, and
scripts/eval_scifact.py. First measured result: dev accuracy 0.606 vs 0.406 trivial baseline (docs/eval/2026-09-11-scifact-dev.md). -
Fetch ladder (spec §7):
httpx→curl_cffiTLS impersonation → browser engine behind the §7.1 consent prompt → Wayback Machine;robots.txtviaprotego; content type from headers; per-host throttling and backoff shared inpolite.py; fetched text cached with the 7-day TTL. Distinct honesty states for blocked, blocked-by-robots, browser-not-permitted, unreachable, provider-unavailable and network-denied — never collapsed. -
Consent gate for the ~280 MB browser engine: asks at most once per run, never without a TTY,
always/neverpersist to config, installs withpip(oruv) andscrapling install, and reports how many sources were skipped. -
Open-access chain: Semantic Scholar → Crossref TDM links → Unpaywall (only with a contact address) → Europe PMC → arXiv → landing page → abstract (OpenAlex last); abstract-only results labelled
LOW CONFIDENCE (abstract only). DataCite arXiv DOIs resolve straight to the arXiv PDF.proofpath fetch <url|doi|arXiv id>. -
scripts/eval_coverage.py: measured 72 % full text / 18 % abstract / 10 % none on 50 DOIs (docs/eval/2026-09-11-coverage.md). -
Document ingest and claim extraction (spec §9):
document.pyvalue types with page/line locators;ingest.pyfor PDF (pymupdf blocks, per-page line numbers, running header/footer removal, superscript citations), docx (paragraphs and tables), markdown and plain text, with the bibliography kept as raw strings, every unparseable page reported as aPageErrorand a page holding an image and no text reported as a scan rather than passed on as an empty page;claims.pypairs numeric markers ([12],[12,15],[12-15]) with their sentence, applies thePARAGRAPH-SCOPEDrule, reports author-year markers asUNSUPPORTED CITATION STYLEand any number the bibliography does not print as unresolved. Hand-built pairing set (61 passages, rate 0.98) andscripts/eval_pairing.py; four real documents (three PDFs and one extracted text) measured indocs/eval/2026-09-11-pairing.md. -
proofpath check(spec §9, §13.2, §15): oneverify()entry point built asprepare()(parse, claims, resolve, retractions, fetch) anddecide_all()(retrieval, numeric rule, NLI, cached per source and claim); compiler-style diagnostics with the quoted passage, the confidence tier and the exact honesty state; a coverage block in every run and a "coverage is weak" line when a quarter or more of the sources could not be read;report.mdwritten by default,--format json,-q,--out,check -for stdin, a real Ctrl-C that keeps what was decided (exit 2). A second run of the same document re-decides nothing: chunks and verdicts come from the cache and neither model scores again, although the models are still loaded and reference resolution and the retraction check still go to the network. Cache schema v2: chunks carry the text digest they were cut from and a source's verdicts are dropped when its text changes.permissions.network = denynow also skips reference resolution and the retraction check, each reported as not attempted.
- CLI surface (spec §13.3):
permissionsandjudgegroups replaced byconfig(config/show/path/set SECTION.KEY VALUE/check); global--no-colorand-q; oneui.pylayer overrichowns every colour and the 10-column key/value layout;resolve --format json; "provider unavailable" exits1(a finding), no longer2. A resolved but retracted reference and a URL that was reached but yielded no text (reached but no text extracted) are findings too (1).skipped N source(s)counts sources, not the URLs tried for them;--format jsonaddsbrowser.skipped_urls.permissions.networkbinds the open-access providers as well as the ladder, andaskwithout a TTY isdeny, reported. - Retrieval no longer depends on
sqlite-vec: a numpy cosine scan is faster at every measured scale and the plain SQLite file opens in any GUI. - Spec:
PARAGRAPH-SCOPEDandUNSUPPORTED CITATION STYLEstates, three-tier confidence display, 7-day raw-text cache TTL, v0.1 limited to numeric citation markers.
- A real reference whose author list carries a surname particle (
van der Walt) was called aGHOST REFERENCE— product rule 3. Reference resolution now understands particles (including gluedal-/el-forms), scores the title against every title-like segment, and accepts a DOI whose record agrees on first author and year asRESOLVED (low confidence)instead of a ghost. The ghost set grew to 258 rows (docs/eval/2026-09-12-ghosts.md): false-ghost rate 0.0 %, fabricated recall 100 %. - A fabricated reference in a numbered bibliography was reported as
UNVERIFIED (not in bibliographic indexes)instead of a ghost, because the printed marker ([7]) blinded the "is this even a paper" check. The marker is stripped once at the resolver's entry, guarded so it can never remove a year, an identifier or a title that begins with a number. - A markdown draft piped through
proofpath -q check -lost its## References. - The consent-gated browser step (fetch-ladder step 3) could not install itself
(
scraplinghas no__main__); it now runs the package's own CLI afterpip/uv, and a paywalled Cell landing page was read through it live (3,632 words). - A piped run occasionally aborted with exit
134from ONNX runtime teardown after printing a complete report; the engine now releases both models explicitly and the CLI flushes its streams before exiting (20 of 20 consecutive runs exit1). - A document with citation markers but no detected bibliography printed a clean-looking
0 %coverage block; it now saysno bibliography was found; N citation markers could not be checked.
- Reference resolution and the retraction check are not cached, so a warm re-run is still a network run: a 68-reference PDF takes about 13 minutes cold and 3.7 minutes cached, a 129-reference one 24 minutes cold. Concurrent resolution and a resolution cache are v0.2 work.
- Two-author lists written
First Last and First Lastresolve toAMBIGUOUS, notRESOLVED; an arXiv id whose record agrees on author and year is not yet rescued the way a DOI is. browser.is_installed()checks that the packages import, not that a browser binary exists; a half-installed environment skips the installer and fails inside the browser fetch (reported as an unverified source, never a crash).
First release. The verification pipeline is not implemented; this reserves the name and establishes the interface, packaging and CI that later phases build on.
- Design specification with measured source-access data (spec §6), the fetch ladder and its permission model (§7), and corrected reference resolution (§8).
- Phased implementation plan, ordered by risk retired rather than user-visible progress.
proofpathcommand. A bare invocation is a first-class entry point rather than a help screen, which is where the TUI will attach. Exit codes are fixed:0clean,1findings,2the run itself failed.- Cross-platform CI on Linux, macOS and Windows, and PyPI publishing through trusted publishing rather than a stored API token.