Fix SCHEMATIC silently shipping uninstrumented binaries - #71
Merged
Conversation
-DDEVICE_DEBUG expands BENCH_INIT/BENCH_EXIT differently, which renames blocks in main; a trace collected without it made TraceLoader drop every function trace of a --device-debug build.
Skipping the function instead linked an uninstrumented binary that dies at the first outage, with every stage up to the flashed target reporting success. Infeasible solves keep returning false: the toolchain reports those as an infeasible result.
Alloca hoisting runs before a solve can fail, so the pass may mutate IR whatever the outcome; reporting PreservedAnalyses::none() unconditionally says the same thing without threading a flag through the call.
One extra FRAM word at BENCH_EXIT, so intermittent runs — which run with device debug off because its counters and UART distort the energy behavior — can still be checked for correctness by return code. bench_commit_done and the identical debug_exit_commit collapse into bench_commit_result.
Anchoring trace 3's peak on 3.6 V let one outlier sample set the factor, leaving the rest of the trace below the voltage the capacitor needs to reach the wait threshold; the trace powered no region and every algorithm failed on it.
byeongjee
force-pushed
the
byeongjee/piddock
branch
from
August 16, 2026 21:50
b4d71b4 to
42e42da
Compare
Renaming bench_commit_done left the ignore list naming two functions that no longer exist, so BENCH_EXIT's commit call was costed as region energy.
At the previous counts these three outlast every trace under RockClimb and SCHEMATIC, so no intermittent run completes. The clang frontend emits the same blocks and loops at both sizes.
A supply glitch cost the data point outright; the run is now reflashed and repeated up to three times, and run_attempts records how many it took. A region that really exceeds its budget fails every attempt, so a genuine violation still surfaces.
The previous results predate the assembly_params.json update, the workload sizes, and the SCHEMATIC trace fix. Adds cuckoo_filter, sha256_fixed, and stringsearch_fixed; drops fft.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a latent SCHEMATIC bug that silently shipped uninstrumented binaries, and makes the benchmark result readable from NVM without a device-debug build.
The bug
ckpt bench/intermittent schematic --device-debugcollected the execution trace withdevice_debug=Falsehardcoded (bench/schematic.py), then applied it to a-DDEVICE_DEBUGbuild. That define expandsBENCH_INIT/BENCH_EXITdifferently, renaming blocks inmain, soTraceLoaderdropped every function trace:solveFunctionreturned false, the driver ignored it,optexited 0, and the pipeline linked and flashed an uninstrumented binary that died at the first outage (region_violation). Latent since the bash→Python rewrite; it only bites with--device-debug.Minimal repro (aes, cap board):
Changes
device_debugintocollect_trace().report_fatal_errorinstead of skipping the function. Infeasible solves still return false; the toolchain reports those as an infeasible result.mutatedIRout-parameter — it existed only to reportPreservedAnalysesaccurately after a failed solve; alloca hoisting runs before any failure, so returningPreservedAnalyses::none()unconditionally says the same thing without the flag.__nvm_resultis now defined unconditionally and written bybench_commit_result(int), one extra FRAM word atBENCH_EXIT. Intermittent runs use--device-debugoff (its counters and UART distort the energy behavior), and could not report correctness before.bench_commit_doneand the identicaldebug_exit_commitcollapse intobench_commit_result.Verification
ckpt intermittent, trace 1, cap board,--device-debugoff — 8/8 ok, every result matching the uninstrumented baseline (aes=107, crc=39423):uv run pytest tests/ -m "milp or rockclimb or schematic or unit"→ 249 passed, 3 failed. All three fail identically onmain:test_schematic_o3_dijkstra_loop_budget_uses_rare_inner_branch, and two rockclimb tests missing theadjcallstackdown/adjcallstackupenergy keys.Not addressed
region_violationwith--device-debugat cap board. Not an algorithm defect: the debug runtime's UART and counters cost energy the model does not account for, and schematicO3 has the thinnest margin (3 boundaries). Lowering the configured capacity to 75% makes it pass with debug on, and it passes at 100% with debug off.