Skip to content

Benchmark stack: main + #294 - #317

Draft
ValarDragon wants to merge 4 commits into
mainfrom
bench/main-plus-pr294-20260902
Draft

Benchmark stack: main + #294#317
ValarDragon wants to merge 4 commits into
mainfrom
bench/main-plus-pr294-20260902

Conversation

@ValarDragon

Copy link
Copy Markdown
Contributor

Benchmark-only branch for current main at c0d8189 plus PR #294. Main already contains #267.\n\nValidation:\n- cargo fmt --all --check\n- cargo test -p zakura-halo2-proofs precomputed_evaluation_and_powers_preserve_proof_bytes --release\n- cargo test -p zakura-halo2-proofs sparse_commitment_matches_dense --release\n- cargo check -p orchard-ios-benchmark --release --features prepared-prover\n\nThe iOS harness remains isolated from this PR. Firebase matrix link will be added after completion.

@ValarDragon

ValarDragon commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Firebase physical-iPhone results for benchmark commit b9f61b5 (current main c0d8189 + #294), prepared prover, Orbits off, release/fat LTO:\n\n| Device | current main median | main + #294 median | observed latency delta |\n|---|---:|---:|---:|\n| iPhone 8 / A11 | 464.68 ms | 465.19 ms | +0.11% |\n| iPhone 11 Pro / A13 | 386.41 ms | 388.85 ms | +0.63% |\n| iPhone SE 3 / A15 | 251.95 ms | 285.30 ms | +13.24% |\n| iPhone 14 Pro / A16 | 253.45 ms | 243.71 ms | -3.84% |\n| iPhone 16 Pro / A18 Pro | 213.16 ms | 199.12 ms | -6.59% |\n\nImportant A15 qualification: this is an observed delta for the latest combined stack, not a demonstrated #294 regression. An earlier stack containing #294 measured 252.09 ms on the same Firebase model, essentially matching the latest-main control. The source delta between that earlier stack and this one includes newly merged #314/#315 and the finalized #267 integration. Isolating causality requires an exact-base A/B or ablation.\n\nA15 pools two 10-sample runs per arm; main range 248.50–256.08 ms and latest-stack range 279.97–318.83 ms. All retained runs ended nominal.\n\nMatrices:\n- stack: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/6598991501901938802\n- main control: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/7030891744298490504\n- SE main rerun: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/4977806520349634840\n- SE stack rerun: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/5211575053577454689\n\nLocal physical iPhone 17 Pro / A19 Pro (iOS 26.6): 174.81 ms proof median, 173.86 ms keygen median, 74.96 ms preparation median; nominal throughout.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant