perf(state): avoid full header decode in deprecated state backend by ongyimeng · Pull Request #3789 · NethermindEth/juno · GitHub
Skip to content

perf(state): avoid full header decode in deprecated state backend#3789

Merged
rodrodros merged 3 commits into
mainfrom
perf/deprecated-state-partial-header
Jul 3, 2026
Merged

perf(state): avoid full header decode in deprecated state backend#3789
rodrodros merged 3 commits into
mainfrom
perf/deprecated-state-partial-header

Conversation

@ongyimeng

@ongyimeng ongyimeng commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Extends the partial-header-decode optimization from #3782 to the
production-default deprecatedStateBackend, avoiding full header decodes
(including EventsBloom) when only one field or an existence check is needed.

  • HeadState: dropped the discarded GetBlockHeaderByNumber fetch;
    GetChainHeight is now only used as the head-exists guard.
  • StateAtBlockNumber: gated on RequireStateRetainedByBlockNumber,
    which reads only the block hash + hash→number index.
  • StateAtBlockHash: resolves the block number via the hash→number
    index without decoding the header.

Why

These paths back latest/head and historical-block state RPCs (getNonce,
getStorageAt, getClass*, call, estimateFee, tracing, etc.). The full
header decode was pure overhead on the hot read path.

Behavior note

HeadState and StateAtBlockHash no longer perform a full header decode
as an incidental DB-corruption/header-existence guard. The GetChainHeight
check and the hash→number index lookup remain as guards, which is correct
for a consistent DB.

Benchmark

Note: I could not use the recent mainnet data sampled earlier because these state-read benchmarks require contiguous chain data with matching parent hashes. The available fixture-sync path only provides a contiguous mainnet chain through block 2, so this uses shallow fixture data instead.

Path / op sec/op base sec/op new sec/op Δ B/op base B/op new B/op Δ allocs/op base allocs/op new allocs/op Δ
Latest/Nonce 68.14µ 11.41µ −83.3% (p=0.002) 4.558Ki 1.784Ki −60.9% 39 15 −61.5%
Latest/ClassHashAt 70.30µ 10.56µ −85.0% (p=0.002) 4.558Ki 1.784Ki −60.9% 39 15 −61.5%
Latest/StorageAt 95.23µ 17.42µ −81.7% (p=0.015) 5.066Ki 2.285Ki −54.9% 51 27 −47.1%
ByNumber/Nonce 67.57µ 50.79µ −24.8% (p=0.009) 4.766Ki 2.684Ki −43.7% 48 60 +25.0%
ByNumber/ClassHashAt 68.38µ 52.83µ ~ (p=0.065) 4.766Ki 2.680Ki −43.8% 48 60 +25.0%
ByNumber/StorageAt 76.44µ 68.49µ ~ (p=0.132) 5.471Ki 3.393Ki −38.0% 61 73 +19.7%
ByHash/Nonce 73.20µ 34.53µ −52.8% (p=0.002) 4.770Ki 2.172Ki −54.5% 48 26 −45.8%
ByHash/ClassHashAt 65.38µ 34.79µ −46.8% (p=0.002) 4.766Ki 2.172Ki −54.4% 48 26 −45.8%
ByHash/StorageAt 77.86µ 48.41µ −37.8% (p=0.002) 5.473Ki 2.877Ki −47.4% 61 39 −36.1%

All B/op deltas have p=0.002.

Tests

  • Added focused tests for the new partial accessor and retention guard.

@ongyimeng ongyimeng force-pushed the perf/deprecated-state-partial-header branch from fa5458a to c31cef8 Compare July 3, 2026 08:56
@ongyimeng ongyimeng marked this pull request as ready for review July 3, 2026 09:08
@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.30435% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.24%. Comparing base (7dbcfd9) to head (7bd87fa).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
blockchain/statebackend/deprecated.go 75.00% 0 Missing and 1 partial ⚠️
pruner/retention.go 87.50% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3789      +/-   ##
==========================================
- Coverage   75.34%   75.24%   -0.10%     
==========================================
  Files         438      438              
  Lines       39528    39569      +41     
==========================================
- Hits        29781    29773       -8     
- Misses       7676     7707      +31     
- Partials     2071     2089      +18     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished @ongyimeng's task in 2m 32s —— View job


PR Review: perf(state): avoid full header decode in deprecated state backend

  • Gather context (diff, changed files)
  • Review core/accessors.go partial accessor
  • Review pruner/retention.go retention guard
  • Review blockchain/statebackend/deprecated.go changes
  • Verify retention/pruning semantics
  • Review tests
  • Post findings

I traced the retention semantics end-to-end to confirm the behavior change is safe. The optimization is correct — no blockers.

Correctness verification

The key question is whether dropping the full header decode weakens the state-retention guard. It doesn't:

  • The pruner signals state pruned by deleting the hash→number index (DeleteBlockHeaderNumberByHash in pruner/accessors.go:183,203), while keeping the header-by-number (needed for block-level data). So the index lookup is the retention signal.
  • StateAtBlockNumberRequireStateRetainedByBlockNumber: does GetBlockHeaderHashByNumber (header exists) + GetBlockHeaderNumberByHash(hash) (the retention signal). Same two lookups as the old HeaderByNumberIfStateRetained, just a partial decode. ✅
  • StateAtBlockHashBlockNumberByHashIfStateRetained: the old path resolved the index then decoded the header only to read header.Number; on a consistent DB index[hash] == header.Number, so returning the index value directly is equivalent. The dropped header decode was an incidental corruption guard, correctly called out in the PR description. ✅
  • HeadState: dropping the discarded GetBlockHeaderByNumber(height) is pure win — GetChainHeight remains the head-exists guard. ✅
  • Partial CBOR decode into struct{ Hash *felt.Felt } is sound: Header uses canonical named-field (map) CBOR, so unknown fields are ignored — same established pattern as GetGlobalStateRootByBlockNumber. The added nil-check makes a corrupt/missing Hash stricter than before, not weaker. ✅
  • Old HeaderByNumberIfStateRetained / HeaderByHashIfStateRetained are still used by the non-deprecated stateBackend (which genuinely needs GlobalStateRoot/Number), so they're not dead code and the scoping is right.

Findings

Blocker: none.

Important: none.

Nit (posted inline):

  • core/accessors_test.go:299 — the table-driven refactor dropped the old "missing global state root" subtest, so the header.GlobalStateRoot == nil / header.Hash == nil guards are now uncovered (matches Codecov's missing-lines report). Worth re-adding a malformed-header case.
  • pruner/retention.go:75 — doc comments on RequireStateRetainedByBlockNumber and BlockNumberByHashIfStateRetained don't start with the function name (unlike the neighbouring helpers); minor godoc/revive consistency.

Optional follow-up (out of scope): the non-deprecated stateBackend.StateAtBlockNumber (statebackend.go:37) could adopt the same GetGlobalStateRootByBlockNumber partial decode for the same win.
· branch perf/deprecated-state-partial-header

Comment thread core/accessors_test.go
Comment thread pruner/retention.go Outdated
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

@rodrodros rodrodros merged commit c68a109 into main Jul 3, 2026
28 checks passed
@rodrodros rodrodros deleted the perf/deprecated-state-partial-header branch July 3, 2026 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants