feat(ethexe): observer alloy chunked event#5392
Conversation
Replace the hand-rolled 256-block chunking around `eth_getLogs` in `EthereumBlockLoader::load_many` with `alloy::contract::ChunkedEvent`, which adds an optimistic full-range query first and a per-block fallback when an individual chunk fails. Block-header batching is kept and now runs in parallel with the chunked log query. Also adds an ignored `bench::bench_load_many` test that times `load_many` against an in-process Anvil. Local numbers (1500 blocks): old custom chunked ≈131 ms avg, new alloy ChunkedEvent ≈120 ms avg. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Gregory Sobol <grishasobol@users.noreply.github.com>
Changed Files
|
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request updates the observer's block loading logic to utilize alloy's Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
I would like to see how it works with reth node where chunk size can be |
| /// Max number of blocks per `eth_getBlockByNumber` JSON-RPC batch. | ||
| const MAX_BLOCK_BATCH_SIZE: usize = 256; | ||
| /// Block-window size passed to alloy's [`alloy::contract::ChunkedEvent`] when fetching logs. | ||
| const LOGS_CHUNK_SIZE: u64 = 256; |
There was a problem hiding this comment.
it can be up to 100k, mb make this configurable
There was a problem hiding this comment.
Code Review
This pull request refactors the Ethereum block and log loading logic in ethexe-observer. It introduces the use of alloy's ChunkedEvent helper for more robust and efficient log fetching, allowing for configurable chunk sizes and concurrency. The load_many function was updated to fetch block headers and logs in parallel, improving performance. Additionally, constants were renamed for clarity, and an empty range check was added. I have no feedback to provide.
|
@claude fix review |
|
Claude finished @grishasobol's task in 1m 53s —— View job Fixed Review Feedback
Changes made ( Added two builder methods to EthereumBlockLoader::new(provider, router_address)
.with_logs_chunk_size(100_000)
.with_logs_max_concurrency(5)The constants |
…onfigurable Add builder methods `with_logs_chunk_size` and `with_logs_max_concurrency` to `EthereumBlockLoader` so callers (e.g. reth nodes) can override the defaults for chunk size (up to 100k) and concurrency. Co-authored-by: Gregory Sobol <grishasobol@users.noreply.github.com>
Numbers (local Anvil, 5 runs each)