Skip to content

Fix SCHEMATIC silently shipping uninstrumented binaries - #71

Merged
byeongjee merged 10 commits into
mainfrom
byeongjee/piddock
Aug 17, 2026
Merged

Fix SCHEMATIC silently shipping uninstrumented binaries#71
byeongjee merged 10 commits into
mainfrom
byeongjee/piddock

Conversation

@byeongjee

Copy link
Copy Markdown
Owner

Summary

Fixes a latent SCHEMATIC bug that silently shipped uninstrumented binaries, and makes the benchmark result readable from NVM without a device-debug build.

The bug

ckpt bench/intermittent schematic --device-debug collected the execution trace with device_debug=False hardcoded (bench/schematic.py), then applied it to a -DDEVICE_DEBUG build. That define expands BENCH_INIT/BENCH_EXIT differently, renaming blocks in main, so TraceLoader dropped every function trace:

TraceLoader: BB 'do.end3' not found in main, skipping trace
TraceLoader: loaded 0 function traces, 39 loop traces for main
SCHEMATIC: no function traces for main — traces are required

solveFunction returned false, the driver ignored it, opt exited 0, and the pipeline linked and flashed an uninstrumented binary that died at the first outage (region_violation). Latent since the bash→Python rewrite; it only bites with --device-debug.

Minimal repro (aes, cap board):

trace collected build outcome
no-debug device-debug 0 function traces → 0 region boundaries
device-debug device-debug 1 function trace → 10 region boundaries

Changes

  • Collect SCHEMATIC traces with the caller's device-debug setting — thread device_debug into collect_trace().
  • Make SCHEMATIC trace-loading failures fatal — a missing or mismatched trace means the pipeline is wrong, so report_fatal_error instead of skipping the function. Infeasible solves still return false; the toolchain reports those as an infeasible result.
  • Drop the mutatedIR out-parameter — it existed only to report PreservedAnalyses accurately after a failed solve; alloca hoisting runs before any failure, so returning PreservedAnalyses::none() unconditionally says the same thing without the flag.
  • Commit the benchmark result to NVM in non-debug builds too__nvm_result is now defined unconditionally and written by bench_commit_result(int), one extra FRAM word at BENCH_EXIT. Intermittent runs use --device-debug off (its counters and UART distort the energy behavior), and could not report correctness before. bench_commit_done and the identical debug_exit_commit collapse into bench_commit_result.

Verification

ckpt intermittent, trace 1, cap board, --device-debug off — 8/8 ok, every result matching the uninstrumented baseline (aes=107, crc=39423):

algo bench status result recovery exec (us) boundaries
milp aes ok 107 0 793,045 5
milp crc ok 39423 0 253,957 2
rockclimb aes ok 107 5 60,516,443 17
rockclimb crc ok 39423 4 51,363,351 8
schematic aes ok 107 10 126,451,035 10
schematic crc ok 39423 2 26,454,973 3
schematicO3 aes ok 107 0 1,100,873 3
schematicO3 crc ok 39423 0 364,818 3

uv run pytest tests/ -m "milp or rockclimb or schematic or unit" → 249 passed, 3 failed. All three fail identically on main: test_schematic_o3_dijkstra_loop_budget_uses_rare_inner_branch, and two rockclimb tests missing the adjcallstackdown/adjcallstackup energy keys.

Not addressed

  • schematicO3/aes hits region_violation with --device-debug at cap board. Not an algorithm defect: the debug runtime's UART and counters cost energy the model does not account for, and schematicO3 has the thinnest margin (3 boundaries). Lowering the configured capacity to 75% makes it pass with debug on, and it passes at 100% with debug off.
  • SCHEMATIC boundary placement is non-monotonic in configured capacity (38122→3 boundaries, 19061→2, 9530→3, 4765→6). No failure observed, but unexplained.

-DDEVICE_DEBUG expands BENCH_INIT/BENCH_EXIT differently, which renames blocks in main; a trace collected without it made TraceLoader drop every function trace of a --device-debug build.
Skipping the function instead linked an uninstrumented binary that dies at the first outage, with every stage up to the flashed target reporting success. Infeasible solves keep returning false: the toolchain reports those as an infeasible result.
Alloca hoisting runs before a solve can fail, so the pass may mutate IR whatever the outcome; reporting PreservedAnalyses::none() unconditionally says the same thing without threading a flag through the call.
One extra FRAM word at BENCH_EXIT, so intermittent runs — which run with device debug off because its counters and UART distort the energy behavior — can still be checked for correctness by return code. bench_commit_done and the identical debug_exit_commit collapse into bench_commit_result.
Anchoring trace 3's peak on 3.6 V let one outlier sample set the factor, leaving the rest of the trace below the voltage the capacitor needs to reach the wait threshold; the trace powered no region and every algorithm failed on it.
Renaming bench_commit_done left the ignore list naming two functions that no longer exist, so BENCH_EXIT's commit call was costed as region energy.
At the previous counts these three outlast every trace under RockClimb and SCHEMATIC, so no intermittent run completes. The clang frontend emits the same blocks and loops at both sizes.
A supply glitch cost the data point outright; the run is now reflashed and repeated up to three times, and run_attempts records how many it took. A region that really exceeds its budget fails every attempt, so a genuine violation still surfaces.
The previous results predate the assembly_params.json update, the workload sizes, and the SCHEMATIC trace fix. Adds cuckoo_filter, sha256_fixed, and stringsearch_fixed; drops fft.
@byeongjee
byeongjee merged commit 4fc0822 into main Aug 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant