perf(batch-builder): read sender balances through one state provider (#1303) - #1306
Open
MavenRain wants to merge 9 commits into
Open
perf(batch-builder): read sender balances through one state provider (#1303)#1306MavenRain wants to merge 9 commits into
MavenRain wants to merge 9 commits into
Conversation
…1303) build_batch read each sender's canonical balance one address at a time. Every BlockchainProvider::basic_account call builds a fresh ConsistentProvider: a new MDBX read transaction plus a MemoryOverlayStateProvider over the in-memory canonical blocks. The sender count is bounded only by the 30M-gas/1MB batch limits, so one build could open ~1,400 read transactions on the hot path that gates batch sealing. Replace TxPool::get_account_balance with a batched get_account_balances that acquires ONE state provider (BlockchainProvider::latest) for the whole set, the way reth's own load_accounts does. The conservative U256::ZERO fallback for a missing account or read error is unchanged, as is the per-sender mined-cost debit at the call site. TestPool keeps its U256::MAX default for tests that do not exercise balance. Closes #1303. Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…nce-reads Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…nce-reads Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…nce-reads Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…nce-reads Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
grantkee
requested changes
Sep 8, 2026
grantkee
left a comment
Contributor
There was a problem hiding this comment.
One small nit that I think is within scope for this PR. Everything else is solid
Co-authored-by: Grant Kee <49913008+grantkee@users.noreply.github.com>
Integrate main at 4ded535. Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1303.
Problem
build_batchconstructs the optimisticchanged_accountsupdate by reading each sender's canonical balance one address at a time throughTxPool::get_account_balance. In reth v1.11.3BlockchainProvider::basic_accountis not a cheap read against a held provider: every call builds a freshConsistentProvider, which opens a new MDBX read transaction and a newMemoryOverlayStateProviderover the in-memory canonical blocks, plus the incidental hash-to-number and prune-checkpoint lookups each one needs.This is the hot path, not a recovery path. It runs on every batch build, synchronously inside the spawned
next-batchtask, once per round per worker. The distinct-sender count is bounded only by the 30M gas / 1MB batch limits, so a batch of minimal transfers can hold ~1,400 distinct senders, i.e. ~1,400 read transactions and memory overlays per build. No attacker is required for the steady-state cost; an adversarial many-sender transaction mix pins it at the worst case every round. The pool resync path caps the same pattern at 100 addresses per iteration; this call site had no cap at all.There is no correctness issue: the pool has no cross-address invariant, so reading each sender at a slightly different canonical tip was fine. This is purely cost (runtime-thread occupancy and allocation churn on the path that gates batch sealing).
Fix
Acquire one state provider for the whole sender set, the way reth's own
load_accountsdoes.crates/tn-reth/src/txn_pool.rs: replaceTxPool::get_account_balancewith a batchedTxPool::get_account_balances(&self, addresses: &[Address]) -> HashMap<Address, U256>. The doc contract requires an entry for every requested address and keeps the documented conservative fallback: a missing account or a read error yieldsU256::ZERO, which can only keep a sender's remaining transactions parked, never promote an unfunded one, and the engine's authoritative canonical update corrects it within the same consensus round.WorkerTxPoolimpl acquires ONE state provider (self.1.latest()) and loops the addresses against it. A provider-acquisition failure now degrades the WHOLE set to the conservative zero instead of one address, so it is logged withwarn!(error, address count) rather than silently, and the trait doc states the all-or-nothing degradation explicitly.crates/batch-builder/src/batch.rs: the single production caller collects the sender set and issues oneget_account_balancescall. The per-sender mined-cost debit (saturating_sub, issue Optimistic U256::MAX balance in batch builder amplifies invalid transactions into follow-up batches (griefing/DoS) #1158) is unchanged.crates/batch-builder/src/test_utils.rs:TestPoolmaps the requested set over its balance table, preserving theU256::MAXdefault for tests that do not exercise balance.get_account_balancehad exactly one production caller, so it is replaced outright rather than kept as a single-address convenience.Testing
cargo +nightly fmt -- --checkgreen (nightly, perrustfmt.tomlcomment wrapping).cargo +1.94 check --workspace --all-targetsgreen under the pinned toolchain (trait signature change, so the full-workspace caller sweep is the gate).cargo +1.94 test -p tn-batch-buildergreen: the existing Optimistic U256::MAX balance in batch builder amplifies invalid transactions into follow-up batches (griefing/DoS) #1158 regression tests drivebuild_batchthrough the batched accessor and assert the exact optimistic balances (real balance minus mined cost, nonce advanced past mined transactions), so a broken batched read fails them.-D warnings, nextest) fires on push; the pinned+1.94attest runs post-push on the second machine.