Repository navigation

Report held-out evaluation during training - #137

Merged

jeffhandley merged 15 commits into

mainfrom

matouskozak-held-out-evaluation-foundation

Oct 24, 2026

Merged

Report held-out evaluation during training#137
jeffhandley merged 15 commits into
mainfrom
matouskozak-held-out-evaluation-foundation

Commits on Oct 19, 2026
- Add saved held-out model evaluation
  
  matouskozak
  and
  Copilot
  committed
  e605f0b
- Keep held-out foundation scope minimal
  
  matouskozak
  and
  Copilot
  committed
  65969f3
Commits on Oct 20, 2026
- Use the simpler frozen holdout evaluation
  
  matouskozak
  and
  Copilot
  committed
  b6a5ecf
- Keep held-out evaluation details out of README
  
  matouskozak
  and
  Copilot
  committed
  8f82525
- Reuse dataset caches with a fixed train-test split
  
  matouskozak
  and
  Copilot
  committed
  b354513
- Consolidate held-out scoring into Tester
  
  matouskozak
  and
  Copilot
  committed
  14b3ddb
- Evaluate the holdout during training and keep Test historical
  
  matouskozak
  and
  Copilot
  committed
  8b7b731
Commits on Oct 21, 2026
- Report coverage at held-out precision targets
  
  matouskozak
  and
  Copilot
  committed
  7e8fe41
- Shorten the historical test warning
  
  matouskozak
  and
  Copilot
  committed
  e6602cb
- Preserve the original Trainer entry point
  
  matouskozak
  and
  Copilot
  committed
  694aec5
- Limit trainer tests to critical paths
  
  matouskozak
  and
  Copilot
  committed
  82acf5f
- Keep action descriptions concise and remove README expansion
  
  matouskozak
  and
  Copilot
  committed
  d4a2e16
- Keep trainer tests focused on behavior
  
  matouskozak
  and
  Copilot
  committed
  ce8eeb7
- Run trainer tests in parallel
  
  matouskozak
  and
  Copilot
  committed
  1baf249
Commits on Oct 22, 2026
- Move held-out evaluation types to their own file
  
  matouskozak
  and
  Copilot
  committed
  ac3443d

Commits

Commits on Oct 19, 2026

Commits on Oct 20, 2026

Commits on Oct 21, 2026

Commits on Oct 22, 2026