Adapt Groth16 batch parallelism to pool size - #483
Open
ValarDragon wants to merge 3 commits into
Open
ValarDragon wants to merge 3 commits into
ValarDragon wants to merge 3 commits into
Conversation
|
Warning An unexpected error occurred while creating the run. Please try again or use the dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adapt multicore Groth16 Miller-loop chunk size to the batch size and Rayon
thread count. The verifier aims for one work item per thread, with chunks
capped at eight proofs. A one-thread pool keeps eight-proof chunks.
Each chunk still receives a nonzero random weight; the same batch pairing
equation and single final exponentiation are used. Smaller chunks expose more
parallelism, especially for batches below the thread count.
The batch benchmark now consumes verification results and includes the multicore
path. This avoids measuring work the optimizer could discard.
Measurement
Local Criterion on a 16-logical-core Mac using the default Rayon pool, valid
one-public-input MiMC proofs:
A seven-public-input diagnostic that matches Sapling Spend's input count
measured 7.65 to 3.83 ms (8 proofs), 7.78 to 5.20 ms (32), and 8.37 to
6.79 ms (64) with one-proof chunks. It uses an expanded key and therefore
invalid proofs, but executes the full batch pairing equation. The diagnostic
code is not included. The adaptive choice for a 16-thread pool is one, two,
and four proofs per chunk respectively, so these figures apply directly only
to the eight-proof case.
On Linux 1 (AMD EPYC, eight logical cores), the original eight-proof chunks
measured 12.03, 13.66, and 13.06 ms for batches of 8, 32, and 64. The
adaptive sizes for that pool are one, four, and eight; the pushed revision
measured 6.40, 8.33, and 13.04 ms respectively. The host was idle before the
run.
With Rayon limited to one thread, one-proof chunks regressed from 40.05 to
45.98 ms for 64 proofs. The adaptive implementation retains eight-proof
chunks in that case. With two threads and 64 proofs, eight-proof chunks
measured 22.24 ms versus 23.19 ms for one-proof chunks. With four threads
and 64 proofs, four-proof chunks measured 12.17 ms versus 12.60 ms for
one-proof chunks.
Checks
cargo test -p zakura-bellman --test mimc batch_verify --locked(valid anddeliberately invalid multicore batches)
cargo test -p zakura-sapling-crypto --lib --locked(69 tests)cargo check -p zakura-bellman --no-default-features --features groth16 --tests --benches --lockedcargo fmt --all -- --checkgit diff --checkNo public or crate-visible Rust API changes.