Benchmark stack: main + #294 - #317
Conversation
|
Firebase physical-iPhone results for benchmark commit b9f61b5 (current main c0d8189 + #294), prepared prover, Orbits off, release/fat LTO:\n\n| Device | current main median | main + #294 median | observed latency delta |\n|---|---:|---:|---:|\n| iPhone 8 / A11 | 464.68 ms | 465.19 ms | +0.11% |\n| iPhone 11 Pro / A13 | 386.41 ms | 388.85 ms | +0.63% |\n| iPhone SE 3 / A15 | 251.95 ms | 285.30 ms | +13.24% |\n| iPhone 14 Pro / A16 | 253.45 ms | 243.71 ms | -3.84% |\n| iPhone 16 Pro / A18 Pro | 213.16 ms | 199.12 ms | -6.59% |\n\nImportant A15 qualification: this is an observed delta for the latest combined stack, not a demonstrated #294 regression. An earlier stack containing #294 measured 252.09 ms on the same Firebase model, essentially matching the latest-main control. The source delta between that earlier stack and this one includes newly merged #314/#315 and the finalized #267 integration. Isolating causality requires an exact-base A/B or ablation.\n\nA15 pools two 10-sample runs per arm; main range 248.50–256.08 ms and latest-stack range 279.97–318.83 ms. All retained runs ended nominal.\n\nMatrices:\n- stack: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/6598991501901938802\n- main control: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/7030891744298490504\n- SE main rerun: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/4977806520349634840\n- SE stack rerun: https://console.firebase.google.com/project/library-testing-def50/testlab/histories/bh.a0f11618fe2721b3/matrices/5211575053577454689\n\nLocal physical iPhone 17 Pro / A19 Pro (iOS 26.6): 174.81 ms proof median, 173.86 ms keygen median, 74.96 ms preparation median; nominal throughout. |
Benchmark-only branch for current main at c0d8189 plus PR #294. Main already contains #267.\n\nValidation:\n- cargo fmt --all --check\n- cargo test -p zakura-halo2-proofs precomputed_evaluation_and_powers_preserve_proof_bytes --release\n- cargo test -p zakura-halo2-proofs sparse_commitment_matches_dense --release\n- cargo check -p orchard-ios-benchmark --release --features prepared-prover\n\nThe iOS harness remains isolated from this PR. Firebase matrix link will be added after completion.