Skip to content

Commit c8be353

Browse files
committed
chore(api): refresh custom-code enforcement tooling
Castiron-Internal-PR: openai/openai-java-internal#349 Castiron-Source-SHA: a3de1ef7e29982e9808a5c0ba5577f7466a606b9 Castiron-Public-Base-SHA: 65bd9e4
1 parent 65bd9e4 commit c8be353

15 files changed

Lines changed: 2533 additions & 159 deletions

‎.castiron.stats.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
schema_version: 1
2-
generation_id: ed22133c-9fa7-43fe-8968-e883cf5c1247
2+
generation_id: 93c08dac-e499-4867-b9bc-750ce37a6066
33
openapi_spec_hash: 28504532afa0c3f477f005e641ec7d6d
44
openapi_transformed_spec_hash: 13422b63a47a5d25533cf55573de7632
55
config_hash: 4446d9cc8e7a00d29ce72282aa5616f9
6-
codegen_sha: 60fe0b2314d2370d16ef526cebfd5aa058962d68
7-
codegen_hash: e20a23b060bd9cb27f5b9e51b6bff14be4f845d2585d0ef54dfb6693747439b5
8-
public_codegen_sha: 63178f1c3de0fdf686ebb74b4233a6dc7b8b6eaf
6+
codegen_sha: d80319ea71677c74e7568f024d8825d726e44cd1
7+
codegen_hash: f60b0457f387b97e5aa58dc87abc845c5a6ad3389cba01af02d0416abf233ec7
8+
public_codegen_sha: a4f3f72d94bdb04745fff76735dbf5dfa6b5751b

‎.github/workflows/castiron-custom-code-comment.yml‎

Lines changed: 294 additions & 22 deletions
Large diffs are not rendered by default.

‎.github/workflows/castiron-custom-code.yml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ name: Castiron custom code
44
on:
55
pull_request:
66
types: [opened, synchronize, reopened, ready_for_review, edited]
7+
pull_request_review:
8+
types: [submitted, edited, dismissed]
79
# Notify the existing trusted handler for queue candidates.
810
merge_group:
911
types: [checks_requested]
@@ -16,11 +18,11 @@ concurrency:
1618
cancel-in-progress: false
1719

1820
env:
19-
REPORTER_SHA256: 6b7cfda1f85426daaa2e4f6186e2d1b20ac638dadaa54be8142ecec89fefd880
21+
REPORTER_SHA256: a52a1aabf4047f4c20e96d1d788c25a4aa78c1987b4fa73a13d22c92ef213a3f
2022

2123
jobs:
2224
queue-signal:
23-
if: github.event_name == 'merge_group'
25+
if: github.event_name == 'merge_group' || github.event_name == 'pull_request_review'
2426
runs-on: ubuntu-latest
2527
timeout-minutes: 1
2628
permissions: {}

‎scripts/castiron/CUSTOM_CODE.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,24 @@ and human review, and [AGENTS.md](../../AGENTS.md#custom-code-budget) for agent
99

1010
## What is counted
1111

12+
Repositories may separately activate `"ratchet": "go"` in their policy after
13+
this checker reaches trusted main. Activation must be a human-reviewed,
14+
policy-only PR; SDK promotion does not carry policy edits. Keep the existing
15+
`max_custom_patch_lines` ceiling.
16+
17+
The Go ratchet rejects net growth in generated-owned custom patch additions
18+
plus deletions **and physical lines of handwritten Go** compared with the
19+
captured live target. It includes helpers, tests, comments, blank lines, nested
20+
modules, and SDK-owned CI Go retained in private checkpoints. Non-text and
21+
nonregular handwritten Go fail validation. Both revisions use their own verified
22+
generated snapshots, read as Git objects without executing candidate code.
23+
24+
Stale PRs compare with the live target, not their historical merge base. Merge
25+
queues check the complete combined candidate. Every landed reduction becomes
26+
the next baseline automatically. An activated ratchet cannot be disabled;
27+
raising the independent patch-only ceiling does not restore retired headroom.
28+
Existing policy isolation, human review, and snapshot ownership rules still apply.
29+
1230
The budget is **added lines plus deleted lines in the entire current custom patch**
1331
against the candidate's verified pure-generated snapshot. It is not the PR's diff
1432
against main, and additions and deletions never cancel. Replacing 500 generated
@@ -193,6 +211,20 @@ from a repository owner, member, or collaborator, in addition to ordinary
193211
CODEOWNERS and required reviews. Bot reviews, review-body markers, stale reviews,
194212
and dismissed approvals do not authorize activation.
195213

214+
For these policy-only reviews, the approving reviewer must explicitly attest that
215+
they personally performed the human review. The entire review body must be the
216+
following sentence, substituting the exact full current head SHA:
217+
218+
```text
219+
I personally reviewed and approve this policy change as a human: <40-character head SHA>
220+
```
221+
222+
Blank reviews, ordinary code-review text, quoted attestations, and attestations for
223+
an earlier head do not qualify. Agents must never submit this attestation, even
224+
through a human account. This is an explicit reviewer attestation, not a claim that
225+
GitHub account type or team membership proves who operated the account. Ordinary
226+
SDK code reviews continue to accept automation under their normal repository gates.
227+
196228
For initial adoption only, a base with no pin may accept an exact policy-only
197229
pin-initialization PR under authenticated current-main checker/policy, with the
198230
same human review requirement. The proposed pin is validated but never executed
@@ -215,3 +247,28 @@ immediately before publication. Dismissed, replaced, stale, automated or missing
215247
approvals fail closed even if ordinary automated approvals keep the queue live.
216248
The publisher consumes the selector's versioned review rules, with cross-runtime
217249
predicate fixtures; candidate artifacts never supply authorization or outcomes.
250+
251+
### Review changes after publication
252+
253+
The unprivileged reporter workflow signals submitted, edited and dismissed reviews.
254+
For a current next PR changing the policy file, the trusted main handler marks its
255+
required budget statuses and affected live queue statuses pending, then requests
256+
normal reporter reruns. Only the ordinary authenticated evaluator may restore
257+
success; the review signal never selects executable code, approval rules or an
258+
arbitrary status target. Fork runs with empty PR lists use authenticated commit
259+
associations and open-PR lookup, retaining live head/merge binding. The handler
260+
marks each authenticated snapshot pending before slow run pagination, then retries
261+
discovery if the PR or queue changes. Pending statuses on superseded affected
262+
SHAs cannot authorize a merge; observed successes stay revoked even if a later
263+
lookup fails. Reporter reruns must resolve to this same PR under the canonical
264+
main-preference rule, not merely share its head SHA. Missing or ambiguous reporter
265+
runs remain pending. Status writers share a serialization group.
266+
267+
Retain native required approving reviews and CODEOWNERS at merge time. Review-event
268+
delivery and reevaluation are asynchronous; this mechanism is not an atomic human
269+
approval check at merge. In particular, automated reviews using human accounts do
270+
not become human policy authorization. Do not weaken native gates or treat this
271+
handler as their replacement. A failed invalidation or rerun must be investigated;
272+
missing reporter runs remain pending until the normal trusted run is requested.
273+
Install and validate the review signal on the protected target before activating
274+
next policy checks, including actual PR and queue revocation after dismissal.

‎scripts/castiron/custom_code_budget.py‎

100644100755
Lines changed: 102 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,10 @@ def strict_policy(repo: Path, revision: str) -> bool:
5050
return cast(int, read_policy(repo, revision)["schema_version"]) == 2
5151

5252

53+
def go_ratchet(repo: Path, revision: str) -> bool:
54+
return cast(bool, read_policy(repo, revision).get("ratchet") == "go")
55+
56+
5357
MANIFEST = b"scripts/castiron/generated-files.json"
5458
# Accept the relocated tooling only from the verified checkpoint. Deploy this
5559
# reader before changing the producer so trusted-main checks can validate moves.
@@ -266,6 +270,8 @@ def check_isolation(repo: Path, base: str, head: str) -> int | None:
266270
return None
267271
if strict_policy(repo, base) and not strict_policy(repo, head):
268272
raise ValueError("strict generated-file ownership cannot be downgraded")
273+
if go_ratchet(repo, base) and not go_ratchet(repo, head):
274+
raise ValueError("Go customization ratchet cannot be disabled")
269275
if paths != {POLICY.encode()}:
270276
raise ValueError(ISOLATION_MESSAGE)
271277
return proposed
@@ -290,6 +296,40 @@ def outcome(state: str, description: str) -> dict[str, str]:
290296
return {"state": state, "description": description}
291297

292298

299+
def go_customization(repo: Path, revision: str, measured: dict[str, Any]) -> dict[str, Any]:
300+
"""Count patches plus physical handwritten Go, using only verified Git objects."""
301+
added, removed = count_custom_lines(measured)
302+
# Use the reporter's ownership view. Private snapshots also retain SDK-owned
303+
# CI files excluded from that view; those Go files are still handwritten.
304+
generated = report.tree_entries(repo, measured["after"]["commit"])
305+
handwritten = {
306+
path: entry
307+
for path, entry in report.tree_entries(repo, revision, include_workflows=True).items()
308+
if path.endswith(b".go") and path not in generated
309+
}
310+
for path, entry in handwritten.items():
311+
if entry.mode not in {b"100644", b"100755"}:
312+
raise ValueError(f"nonregular handwritten Go: {path!r}")
313+
blobs = report.read_blobs(repo, handwritten)
314+
files: list[dict[str, Any]] = []
315+
for path, entry in sorted(handwritten.items()):
316+
content = blobs[entry.oid]
317+
content.decode("utf-8")
318+
if b"\0" in content:
319+
raise ValueError(f"non-text handwritten Go: {path!r}")
320+
lines = content.count(b"\n") + int(bool(content) and not content.endswith(b"\n"))
321+
files.append({"path": path.decode("utf-8"), "lines": lines})
322+
handwritten_lines = sum(file["lines"] for file in files)
323+
return {
324+
"revision": revision,
325+
"generated": measured["after"],
326+
"patch_lines": added + removed,
327+
"handwritten_lines": handwritten_lines,
328+
"handwritten_files": files,
329+
"total": added + removed + handwritten_lines,
330+
}
331+
332+
293333
def evaluate(
294334
repo: Path,
295335
base: str,
@@ -362,11 +402,38 @@ def evaluate(
362402
result["checked_limit"] = checked_limit
363403
state = "success" if total <= checked_limit else "failure"
364404
description = f"+{added} / -{removed} = {total} custom lines; limit {checked_limit}."
405+
if any(go_ratchet(repo, revision) for revision in (policy, base, head)):
406+
# The report's historical merge base is for presentation only. Measure
407+
# the captured live target against its own hash-verified snapshot.
408+
baseline, _ = report.build_report(
409+
repo,
410+
base,
411+
base,
412+
public=public,
413+
fetch=fetch,
414+
require_head_hash=True,
415+
target_branch=target_branch,
416+
)
417+
before = go_customization(repo, base, baseline)
418+
after = go_customization(repo, head, measured)
419+
delta = after["total"] - before["total"]
420+
result["ratchet"] = {"kind": "go", "before": before, "after": after, "delta": delta}
421+
description += (
422+
f" Go customization {before['total']} -> {after['total']} (delta {delta:+d})."
423+
)
424+
if delta > 0:
425+
state = "failure"
426+
if go_ratchet(repo, base) and not go_ratchet(repo, head):
427+
state = "failure"
428+
description += " Go customization ratchet cannot be disabled."
365429
# Main's strict policy applies even if the candidate tries to remove it.
366430
# A proposed activation must itself satisfy the stricter contract.
367431
if strict_policy(repo, policy) or strict_policy(repo, base) or strict_policy(repo, head):
368432
violations = verify_generated(
369-
repo, measured["after"]["commit"], head, previous=measured["before"]["commit"]
433+
repo,
434+
measured["after"]["commit"],
435+
head,
436+
previous=(measured["before"] or measured["adoption_baseline"])["commit"],
370437
)
371438
result["strict_violations"] = violations
372439
if violations:
@@ -512,24 +579,23 @@ def github_evaluate(
512579
f"{branch} must require a merge queue before activating the budget gate"
513580
)
514581
page += 1
515-
if branch == "next":
516-
# This receipt is created locally by the trusted-main selector before the
517-
# pinned checkout. It must never come from candidate workflow artifacts.
518-
if (
519-
selection is None
520-
or type(selection.get("schema_version")) is not int
521-
or any(
522-
selection.get(key) != value
523-
for key, value in {
524-
"schema_version": 1,
525-
"target_branch": branch,
526-
"base_sha": base,
527-
"head_sha": head,
528-
"policy_sha": main,
529-
}.items()
530-
)
531-
):
532-
raise ValueError("trusted next policy selection does not match evaluation")
582+
# This receipt is created locally by the trusted-main selector before the
583+
# pinned checkout. It must never come from candidate workflow artifacts.
584+
if (
585+
selection is None
586+
or type(selection.get("schema_version")) is not int
587+
or any(
588+
selection.get(key) != value
589+
for key, value in {
590+
"schema_version": 1,
591+
"target_branch": branch,
592+
"base_sha": base,
593+
"head_sha": head,
594+
"policy_sha": main,
595+
}.items()
596+
)
597+
):
598+
raise ValueError("trusted policy selection does not match evaluation")
533599
measurement = None
534600
if trusted_report_dir is not None:
535601
# Only the preceding main-branch step may supply this directory. Never
@@ -556,19 +622,6 @@ def github_evaluate(
556622
report.git(
557623
repo, "fetch", "--quiet", "--no-tags", "origin", *dict.fromkeys((main, base, head))
558624
)
559-
if branch == "next":
560-
assert selection is not None
561-
base_pin = read_policy(repo, base).get("next_policy")
562-
if base_pin is None:
563-
if selection.get("bootstrap") is not True or changed_paths(repo, base, head) != {
564-
POLICY.encode()
565-
}:
566-
raise ValueError("unreviewed next policy initialization")
567-
elif selection.get("bootstrap") is not False or base_pin != {
568-
"repository": repository,
569-
"main_sha": main,
570-
}:
571-
raise ValueError("trusted next base pin changed; rerun policy selection")
572625
pull_heads: list[str] | None = None
573626
if run["event"] == "merge_group":
574627
if report.git(repo, "merge-base", base, head).decode().strip() != base:
@@ -585,6 +638,22 @@ def github_evaluate(
585638
pull_heads.append(candidate)
586639
if not pull_heads:
587640
raise ValueError("cannot verify merge-group PR membership; retry the current group")
641+
if branch == "next":
642+
assert selection is not None
643+
base_pin = read_policy(repo, base).get("next_policy")
644+
if base_pin is None:
645+
# evaluate() checks each verified queue constituent for policy isolation.
646+
if (
647+
selection.get("bootstrap") is not True
648+
or "next_policy" not in read_policy(repo, head)
649+
or (pull_heads is None and changed_paths(repo, base, head) != {POLICY.encode()})
650+
):
651+
raise ValueError("unreviewed next policy initialization")
652+
elif selection.get("bootstrap") is not False or base_pin != {
653+
"repository": repository,
654+
"main_sha": main,
655+
}:
656+
raise ValueError("trusted next base pin changed; rerun policy selection")
588657
result, patch = evaluate(
589658
repo,
590659
base,
@@ -602,7 +671,7 @@ def github_evaluate(
602671

603672
def main() -> int:
604673
if sys.argv[1:2] == ["select-policy"]:
605-
return policy.main(sys.argv[1:])
674+
return cast(int, policy.main(sys.argv[1:]))
606675
parser = argparse.ArgumentParser(description=__doc__)
607676
commands = parser.add_subparsers(dest="command", required=True)
608677
generated = commands.add_parser("verify-generated")

0 commit comments

Comments
 (0)