[TheGreatAxios]

Supersedes #692.

Builds on #690 (merged), which moved the judge's limits into quality.ts and caps the patch the facts step reads at 2,000,000 chars per pull request.

The judge already splits a large diff by file across several calls, up to 64 per pull request. It combines focused and docs in code. Focused fails when any slice is a separate change, passes when every slice serves the purpose, and is Unsure otherwise. Docs fails when any file part needs docs. What left #688 Unsure was the judge's own patch budget: it read only 270,000 chars, three calls' worth, so later files reached it without lines and focused listed them as unread.

The judge no longer has a patch budget of its own. It reads every patch the facts step read, each cut at 65,536 chars as before, split across as many first-round calls as it takes up to 64. Files past the facts step's 2,000,000-char ceiling are listed as unread and the rule is Unsure, naming them. A diff that needs more than 64 first-round calls is still judged: the first 64 are sent (63 when duplicates has a request, which keeps its slot), and the files only later calls would hold are unread, so the rules that ask about them are Unsure, naming them. Before, such a diff sent nothing and left every judged rule Unsure. The worst case is 95 calls: the 64 first-round calls, the verdict call, and at most 30 focused re-sends, since re-sends stop once 30 files are named. That is about $0.26 at $0.0027 per full call. Nothing is trimmed silently, and small pull requests are unchanged. Bitbucket Data Center gives no patches, so it is unaffected.

• quality.ts: judgedFiles drops the 270,000-char total. Past 64 first-round calls, the plan drops the later calls and marks their files unread for focused, docs and scoped custom rules, instead of refusing the whole pull request.
• pr-triage.test.ts: the multi-call test, which runs the facts step against a fake GitHub, now uses more than 270,000 chars of patch. It checks that every file reaches the judge across at least 4 calls.
• facts.test.ts: drops fix(triage-workflows): read every patch for the rules that run in code #690's test that the judge reads at most three calls' worth. Its pattern-rule half is already covered by the ceiling test.
• quality.test.ts: a new test builds a diff under the 2,000,000-char ceiling (71 files of 28,000 chars, 24 KB of docs, an 8 KB body, long paths) that needs 71 calls. With a duplicates shortlist on, it checks that 63 file calls and the duplicates call are sent, the 8 files past them are unread for focused and docs, and every other file is asked about.
• docs/DECISION_MODEL.md and docs/ARCHITECTURE.md: they now say the judge reads as much as the facts step read, up to 64 first-round calls, state the 95-call worst case and its cost, and say that files past the 64th call are unread.
• The verdict question is still one call holding as much of the diff as fits. Splitting it is out of scope.

Verification

• I replayed feat: make the policy file the one source of what Triage may do #688 offline through the real listFiles and prepareQualityEvaluation, using its files from the GitHub API. Before this change: 4 calls and 77 files unread by focused. With this change: 9 calls and 0 unread.
• The pr-triage test fails on fix(triage-workflows): read every patch for the rules that run in code #690's quality.ts with 10 files missing and passes here.
• The new cap test fails on main (10 calls, 61 files unread) and on this branch's previous head (no call, "pull request too large for the decision model (71 requests)"), and passes here.
• bun --conditions=intx-src test packages/triage-workflows: 297 pass, 0 fail. bun run lint: exit 0, no new warnings. bun run check: exit 0.
• bun run test:chain hung locally with no output, so CI runs it. Its fixture is a one-line patch.
• Judge eval: not run, because it calls a paid model. Its result is the same before and after by construction. The change only matters above 270,000 chars of patch or 64 calls, and the largest eval pull request has 83,508 chars (refactor(web): rebuild the repository panel in the mockup's sentence layout #157) and takes 2 calls. The largest mix is about 105,000 chars (feat(triage): let a policy request changes on a pull request #190 with refactor(hub): sign the GitHub App setup state instead of storing it #186). Every case was measured through the GitHub compare API.

Closes CL-11613

[linear-code]

CL-11613

CL-11710

[TheGreatAxios]

Independent review of 54e65fb against origin/main.

Checked: bun run lint exits 0 with no warnings in the touched files. bun --conditions=intx-src test packages/triage-workflows passes, 297 tests, 0 failures.

The code change is right. judgedFiles now only drops lockfiles and cuts each patch at 65,536 chars, so the judge gets every patch the facts step read. The 64 cap is unchanged: 64 requests are sent, 65 or more send nothing and set judgeError to "pull request too large for the decision model (N requests)". When the cap is full, duplicates gives up its own request, so 63 plus duplicates still fits. The rules step sets judgeLimitExceeded, and renderVerdict then returns Unsure with that reason, unless a deterministic rule already hard-fails. Nothing named MAX_PATCH_CHARS, the 270,000 figure or "the judge's budget" is left in code or docs. Apart from changes in the rebased base text, the diff has the same content as #692's final diff. #692 also removed the NO_FACTS import. This PR keeps it, which is correct, because #690 added a use of it at facts.test.ts:158. The pr-triage test is load-bearing: the old budget would have left the later files out of every request, so its path-set assertion would fail on main.

On the DECISION_MODEL row "Patch text in one facts step's output ... one pull request per step": it is correct. The hub's manager intake splits backlog and items mails into one record per pull request, and the listener deploys one manager per pull request. The bullet just below it, which says "each pull request in a backlog batch counting its own", is also true, because facts.ts still accepts batch input. Read together, the two lines look like they contradict each other. They came from #690, not this PR.

Findings

1. docs/DECISION_MODEL.md:131: "up to 64 requests, at most about $0.17" is not an upper bound. The 64 cap covers only the first round. The verdict request and the re-sent request for each failing focused slice (focusedPartRequests) come on top, so the worst case is about 129 requests, roughly $0.35. Either say "64 first-round requests" or give the real bound. docs/ARCHITECTURE.md:557 says "across up to 64 requests" and needs the same fix.
2. packages/triage-workflows/src/logic/facts.test.ts:107: the judge half of this test was removed. What is left no longer touches the judge, but the title still says "several judge requests", and the threshold at :116 (3 * MAX_SYSTEM_ONE_STATE_BYTES) is the value of the deleted MAX_PATCH_CHARS. The ceiling test at :138 already checks that about 1.9M chars are read whole across pages for the pattern rules. Drop this test, or retitle it and remove the threshold if the late match at line 201 is the point.

Notes, not blocking

• No test covers exactly 64 against 65 requests. The comparison did not change, but it is now the judge's only bound.
• Before and after this PR, a default pull-request-scope custom rule is asked only in the first request, so it can Pass without seeing files in later requests. This PR does not change what the first request holds, but large diffs now spread across more requests.
• The rules step output carries both the facts (up to 2M chars of patch) and every chunk's state (up to 64 × 90 KB). That bound is unchanged, but ordinary large pull requests now come close to it, where the old 270,000-char budget kept them far below.

[TheGreatAxios]

Fixed in 37ada0c: docs state the true worst case (64 first-round + verdict + at most 30 focused re-sends, since re-sends stop at 30 named files = 95 requests, about $0.26, not ~129), the stale facts.test.ts test is dropped, and quality.test.ts pins 64 requests sent / 65 refused with the request-count reason.

[TheGreatAxios]

Re-review at 37ada0c.

No findings.

Both earlier findings are fixed. The cost line in docs/DECISION_MODEL.md, its limits row and docs/ARCHITECTURE.md now name 64 first-round requests, plus the verdict request and at most 30 focused re-sends. The leftover facts.test.ts test is removed, and its pattern-rule coverage is still there in the code-rules ceiling tests.

I checked the bound against quality.ts and the evaluate and parts actions. The 64 cap applies only to the first round. focusedPartRequests only adds a request that names at least one new file and stops at MAX_PART_FILES (30), so there are at most 30 re-sends. There is one verdict request. 95 is a correct upper bound. It is not tight, though: choiceOf skips the verdict question whenever a rule fails, and re-sends happen only when focused fails, so the two never happen in the same run and the real maximum is 94. That does not need a change.

The new cap test is load-bearing. I changed the comparison to >=, and the test failed with "pull request too large for the decision model (64 requests)". After reverting, it passes.

bun run lint exits 0 with no warnings in the touched files. bun --conditions=intx-src test packages/triage-workflows passes: 297 tests, 0 failures.

[corbits-triage]

Triage: In progress

Triage is waiting for CI to finish before it decides. This comment shows the verdict once it has.

[corbits-triage]

Removes the judge's own 270,000-char patch budget so the judge reads every patch the facts step reads, capped by 64 first-round calls instead of by character count. Two concerns stand out: large but legitimate diffs can exceed the request cap and get no judgment at all, and the cap can bind before the 2,000,000-char ceiling the PR describes as the only limit.

P2 · The 64-request cap refuses diffs the facts step read in full, so the judge does not read every patch it was given · packages/triage-workflows/src/logic/quality.ts:562

The PR says the facts step's 2,000,000-char ceiling is the one limit and that the judge reads every patch the facts step read. In practice the 64-call cap binds first. Each request has about 90,000 bytes of file room, minus metadata, docs and the path list. The PR describes up to about 24 KB of docs and up to 8 KB each for paths and directories. A file larger than about half that room gets its own request. A diff of many files in the 25–35 KB range, all under the ceiling, therefore needs more than 64 first-round requests. The judge then sends nothing and every judged rule goes Unsure, even though the facts step read those patches in full.

Trigger: a PR of about 1.5–2.0 MB of patch made of many files around 25–35 KB each, with docs near 24 KB and long paths. Its first-round split needs more than 64 requests, so the code returns the 'pull request too large' refusal even though the facts step read every patch.

Fix: either bound the request count by the same measure the facts step uses, or pack files into requests more tightly, or document the mismatch and correct the PR's claim that the 2,000,000-char ceiling is the only limit.

if (requests.length > MAX_JUDGE_REQUESTS) return { plan, ...(focused && { focused }), requests: [], judgeError: ${QUALITY_EVALUATION_LIMIT_ERROR} (${requests.length} requests) };

[corbits-triage]

P2 · Large diffs that were partly judged now get no judgment at all

Without the 270,000-char budget, every patch up to the facts step's 2,000,000-char ceiling enters the request plan. Per-request room is about 90 KB, and docs (up to 24 KB), metadata and the path list take a share of that. For a wide PR the path list alone can be tens of KB, so a legitimate diff can need more than 64 first-round requests. Before this change, later files were sent without patches and focused/docs still got answered. Now no request is sent and every judged rule, including docs and custom rules, is Unsure, with no partial degradation.

Trigger: a PR with many files whose path list leaves about 25–30 KB of file room per request, and roughly 1.5–2 MB of patch, for example about 1,000 files with sizeable diffs.

Fix: when the request count exceeds the cap, degrade gracefully by sending the first 64 requests' worth of patches and marking the remaining files unread, so only the affected rules become Unsure and the others still get answered. Alternatively, reduce per-request overhead (path list, docs) so the room per request is larger.

if (patch === undefined) return file;

[TheGreatAxios]

Reproduced (71 files under the 2,000,000-char ceiling needed 71 calls and got none); fixed in 1592b62: the first 64 calls are sent and only files past them are unread, pinned by a test in quality.test.ts.

[corbits-triage]

Triage requests changes:

• code-review: Code review found 2 problems on this head.

[corbits-triage]

Triage requests changes:

• CI green: required checks are failing

[TheGreatAxios]

Review at 1592b62.

Findings

1. packages/triage-workflows/src/logic/quality.ts:582: on any diff whose first round needs 64 or more requests, duplicates is dropped and its reason always says "(65 requests)". The first round is now cut to 64 before duplicates is appended, so requests.length + 1 is always 65, whatever the split needed. I confirmed this with a 100-file diff that needed about 100 requests: the reason was "pull request too large for the decision model (65 requests)". Duplicates' request carries no diff, so now that later file requests can be left unread, it could keep its slot: cut the first round at 63 when duplicates has a request, and the one extra file request becomes unread like the rest. If dropping it stays, the reason should not give a request count that the code itself produced.

Notes, not blocking

• leaveUnread is correct for every rule kind. I checked it with a probe: 100 files, plus focused, docs, duplicates and custom rules scoped to the pull request, a file and a hunk. 64 requests went out. Focused, docs and both scoped rules each listed the same 36 later files as unread. Hunk citations were reduced to their paths and listed once. No file was both asked and unread for the file-scoped rule. The whole-PR custom question stayed in request 1. render.ts never passes a rule that has an unread file: focused and fileCheck go unconfirmed unless an answered item fails, and docs passes only on a changed doc file, by rule. A file split across the cut keeps its docs part question and is also unread, so it can fail docs but cannot pass it.
• Determinism: the plan is a pure function of facts.files, which keeps GitHub's file order. Two runs on the same input gave the same JSON, so the same files are unread on every run of a head. The files left unread are the tail of that order, which is about path order and not risk. Putting files that docs or scoped rules ask about first, or the smallest files first, would leave fewer files unread. That is optional.
• A whole-PR custom rule is still answered only from request 1. Before, a diff past the cap got no answer at all. Now it gets one that did not see the later files. The semantics are the same as for any multi-request diff, so this needs no change.
• The new cap test is load-bearing. With the rest of the branch in place, swapping in main's quality.ts fails it at toHaveLength (10 requests), and swapping in 82e6866's fails it at judgeError ("71 requests"). Neither deleted test is the only cover for a behaviour that still exists: refusal past the cap is gone, and the KB refusal is still tested at :138. The new test does not cover scoped custom rules or the hunk-citation branch of leaveUnread, which only the probe above exercised.
• The docs match the code. The DECISION_MODEL.md line "Hunk-scoped custom rules add one question per hunk, uncapped, so wide diffs can need more requests" could add that past 64 requests the later files are unread.
• bun run lint exits 0 with no warnings in the touched files. bun run check exits 0. bun --conditions=intx-src test packages/triage-workflows: 297 pass, 0 fail. Both commits are Conventional, with no attribution lines.

[TheGreatAxios]

Fixed in aae86b7: with a duplicates request the first round is cut at 63, so duplicates keeps its slot and no reason quotes a request count; the 71-file test now asserts it is sent, and the hunk-scoped docs line notes files past the 64th request are unread.

[corbits-triage]

Triage requests changes:

• Merge conflicts: merge conflicts with base

[TheGreatAxios]

Re-review at aae86b7.

The duplicates finding is fixed. When duplicates has a request, the first round is cut at 63 and the duplicates request is always sent, and the "(65 requests)" reason is gone. Compared with 1592b62, only quality.ts, quality.test.ts and docs/DECISION_MODEL.md changed. The updated cap test is load-bearing: with 1592b62's quality.ts swapped in, it fails at the openPullRequests assertion. The cost bound is still right, since duplicates now counts within the 64 first-round requests.

Nits (wording only)

1. In docs/DECISION_MODEL.md, two places still say 64 where it is 63 when duplicates has a request: the "Unread files" bullet ("only requests past the 64th hold it"), and the sentence about files past the 64th request at the end of the hunk-scoped line in the cost section. The "Request splitting" bullet already says "(63 when duplicates has a request)", and these two could say the same.
2. The title of the cap test in quality.test.ts says "reading the first 64", but the test sends 63 file requests and the duplicates request.

bun --conditions=intx-src test packages/triage-workflows in a fresh worktree: 297 pass, 0 fail. bun run lint exits 0. bun run check exits 0.

[TheGreatAxios]

Both nits applied: DECISION_MODEL.md says 63 where duplicates has a request; the test title matches what it sends.

[TheGreatAxios]

Re-review at 950e733.

Compared with aae86b7, only the two nit lines and the test title changed, and all three are right.

One nit of the same kind, which I missed last round: the Focused bullet under "Unread files" in docs/DECISION_MODEL.md still says "part of it was only in requests past the 64th". When duplicates has a request, that is the 63rd. This is wording only and is not blocking.

[TheGreatAxios]

No findings.

[corbits-triage]

Triage requests changes:

• Merge conflicts: merge conflicts with base

[TheGreatAxios]

Rebased onto main after #711; the only conflict was the facts paragraph in docs/ARCHITECTURE.md, resolved by keeping this PR's paragraph and main's route and judge lines.

[corbits-triage]

Removes the judge's separate 270,000-char patch budget so it reads every patch the facts step read, split across up to 64 first-round calls. Files past the cap are listed as unread and the affected rules return Unsure, naming them. No review findings were supplied for this change.