Skip to content

Correct local model memory evidence#4

Draft
alan-d-smith wants to merge 1 commit into
mainfrom
codex/correct-phi-container-evidence
Draft

Correct local model memory evidence#4
alan-d-smith wants to merge 1 commit into
mainfrom
codex/correct-phi-container-evidence

Conversation

@alan-d-smith

Copy link
Copy Markdown
Member

What changed

  • Records cgroup memory pressure counters (max, oom, oom_kill, and related v2/v1 evidence) in llama.cpp smoke and local-route benchmark outputs.
  • Preserves at-load baselines, current counters, and per-run deltas.
  • Corrects the optimization log: all four tested Phi GGUFs run individually inside the constrained environment; reaching memory.max is pressure/reclaim evidence, not an automatic OOM failure.
  • Documents the strict 4,000,000,000-byte cold-cache Phi-4 Mini Q4_K_S run and keeps it as an exact-image/narrow-route candidate rather than promoting it prematurely.
  • Rebuilds the two measurement manifests without changing their holdout states.

Why

The previous decision treated a cgroup peak equal to the configured ceiling as proof that a model could not run. That conclusion was not in the participant guide and did not distinguish reclaim/limit-hit events from actual OOM or OOM-kill events. The corrected run produced all 40 answers with zero OOMs and zero OOM kills.

Impact

The scored runtime and published image are unchanged. No GGUF weight or llama.cpp runtime dependency is shipped by this PR. Future local-model decisions now carry explicit memory-pressure evidence, and Phi-4 Mini Q4_K_S remains eligible for a separate exact baked-image experiment.

Validation

  • 580 tests passed; 1 Windows symlink test skipped
  • Ruff check and format check passed
  • Strict mypy passed across 72 source files
  • Both measurement manifests verify
  • Strict cold-cache benchmark: 35/40 strict accuracy, 0 runtime failures, 0 invalid outputs, max task latency 15.1s, 214.6s total, 0 OOM, 0 OOM-kill

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant