Skip to content

eval: separate fixture history from declared Git identity - #50

Merged
myobie merged 6 commits into
mainfrom
schickling-assistant/2026-07-30-signal-rename-frozen-identity
Jul 31, 2026
Merged

eval: separate fixture history from declared Git identity#50
myobie merged 6 commits into
mainfrom
schickling-assistant/2026-07-30-signal-rename-frozen-identity

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Why

Free gates and runtime materializers were using porcelain commits to construct synthetic history. That conflated deterministic fixture data with an agent-authored commit lifecycle: repository commit policy and hooks could reject the setup or inject private provenance into scanner and oracle inputs.

What

Separate the two boundaries:

  • Gate-owned mutation history and immutable materializer seed history use deterministic Git plumbing with explicit fixture metadata.
  • Agent-authored product commits continue to use ordinary porcelain Git commands and a declared worktree identity policy.
  • Weird-Git owns the neutral actor identity Eval Worktree Actor <worktree-actor@eval.invalid> in both worktree-local configs. Its judge dynamically compares the feature commit to that pinned identity.
  • Frozen historical authors and real agent-workspace attribution remain unchanged.

Synthetic roots and mutation inputs now use write-tree, commit-tree, and update-ref, so constructing test data does not invoke authored commit hooks. The signal twins grade only BASE..HEAD. Weird-Git excludes its root structurally, retains extensions.worktreeConfig, and compares the feature tip against its fixture-declared worktree identity.

Validation

Restacked on current main commit 0d5ba20fc5ab28e7e8085cd00070baa9897acf5d.

Focused gates are green:

  • shell syntax and git diff --check
  • fixture reset is reproducible and terminal-independent
  • Weird-Git mutation proof preserves the seeded regression contract
  • VRS mutation families and no-PII history use deterministic plumbing
  • the Weird-Git actor identity is fixture-owned; no ambient, principal, source-repo, or SPEC_DIR identity is read

The complete pinned model-free preflight exits 0:

  • pinned st2 source 9887b28, binary SHA-256 d49d44fd4f3f6f655455c212353a469fefa956082bedf22163deb767d8a36a0d
  • 168 shell files parse
  • 77 KDL files validate
  • 47 maintained cells and 10 nested catalogs load
  • 62 model seats preserve the canonical harness
  • 16 model-free exclusions remain explicit
  • all fixture, VRS, Weird-Git, preflight-mutation, overnight-policy, no-PII history, generated-catalog, and diff gates pass

No model/provider call or deployment was made.

@schickling-assistant schickling-assistant changed the title eval: preserve host identity in signal materializers eval: preserve Git identity policy in free gates Jul 30, 2026
agent-session-id: dev3.dotfiles-cos-misc-st2-resource-design
agent-tool: Codex
agent-tool-version: 0.145.0
agent-model: gpt-5.6-sol
agent-runtime-profile: /home/schickling/.local/state/agent-session-recovery/2026-07-29-pty-st2-cutover/runtime-profile/profile-without-null-opencode.json
agent-skills-manifest: /nix/store/kx5j47nghj1yps2v693ryb6wnf1c2xhb-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@16a22c6
agent-session-id: dev3.dotfiles-cos-misc-st2-resource-design
agent-tool: Codex
agent-tool-version: 0.145.0
agent-model: gpt-5.6-sol
agent-runtime-profile: /home/schickling/.local/state/agent-session-recovery/2026-07-29-pty-st2-cutover/runtime-profile/profile-without-null-opencode.json
agent-skills-manifest: /nix/store/kx5j47nghj1yps2v693ryb6wnf1c2xhb-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@16a22c6
agent-session-id: dev3.dotfiles-cos-misc-st2-resource-design
agent-tool: Codex
agent-tool-version: 0.145.0
agent-model: gpt-5.6-sol
agent-runtime-profile: /home/schickling/.local/state/agent-session-recovery/2026-07-29-pty-st2-cutover/runtime-profile/profile-without-null-opencode.json
agent-skills-manifest: /nix/store/kx5j47nghj1yps2v693ryb6wnf1c2xhb-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@16a22c6
agent-session-id: dev3.dotfiles-cos-misc-st2-resource-design
agent-tool: Codex
agent-tool-version: 0.145.0
agent-model: gpt-5.6-sol
agent-runtime-profile: /home/schickling/.local/state/agent-session-recovery/2026-07-29-pty-st2-cutover/runtime-profile/profile-without-null-opencode.json
agent-skills-manifest: /nix/store/kx5j47nghj1yps2v693ryb6wnf1c2xhb-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@16a22c6
agent-session-id: dev3.dotfiles-cos-misc-st2-resource-design
agent-tool: Codex
agent-tool-version: 0.145.0
agent-model: gpt-5.6-sol
agent-runtime-profile: /home/schickling/.local/state/agent-session-recovery/2026-07-29-pty-st2-cutover/runtime-profile/profile-without-null-opencode.json
agent-skills-manifest: /nix/store/kx5j47nghj1yps2v693ryb6wnf1c2xhb-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@16a22c6

myobie commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Current-main reconciliation found an unresolved runtime identity-input blocker; this draft is not ready to retarget.

A clean local five-commit restack onto main 0d5ba20fc5ab28e7e8085cd00070baa9897acf5d produces only the intended 11-file diff, and syntax plus git diff --check pass. However, the complete pinned free preflight exits 1 at bin/check-fixture-reset-terminal.shbin/check-fixture-reset.sh while materializing weird-git-setup.

Exact cause: setup-megarepo.sh reads git config --get user.name / user.email after entering $CATALOG. st2 run steps also execute with cwd $CATALOG; the authorized identity is repository-local, and no source/spec identity context is exported to the run step. In an isolated fixture copy, AUTHOR_NAME is empty and the materializer exits 1.

Injecting identity only into the free gate would make the proof diverge from the real eval, while embedding a principal identity is not portable. No branch update, model/provider call, or deployment was made. The provisional local restack head is 81196929ad21335f99090abff66e80fc928d0cfe; it is evidence only and has not been pushed.

@myobie
myobie force-pushed the schickling-assistant/2026-07-30-signal-rename-frozen-identity branch from e5f2f98 to 684e9fd Compare July 31, 2026 11:38
@myobie myobie changed the title eval: preserve Git identity policy in free gates eval: separate fixture history from declared Git identity Jul 31, 2026
@myobie
myobie changed the base branch from schickling-assistant/2026-07-30-2026-07-30-inbox-hygiene-reset to main July 31, 2026 11:38

myobie commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Resolved by the accepted fixture-owned identity contract and current-main restack at exact head 684e9fd92b480b0b80fc97e8ffe37d877f19edfe.

Weird-Git now pins Eval Worktree Actor <worktree-actor@eval.invalid> in both worktree-local configs. Its dynamic judge compares the ordinary porcelain feature commit to that declared identity; immutable seed and mutation histories remain separate deterministic plumbing data. No ambient, principal, source-repo, or SPEC_DIR identity is read.

Focused gates and the complete pinned model-free preflight are green. PR state is draft, cleanly mergeable, and thread-clean. No model/provider call or deployment was made.

@myobie
myobie marked this pull request as ready for review July 31, 2026 11:40
@myobie
myobie merged commit 2eacf39 into main Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants