core: reset tx lookup cache if necessary #28865

rjl493456442 · 2024-01-24T08:01:44Z

This pull request resets the txlookup cache if certain transactions are reorg'd out, preventing them from remaining reachable. It addresses failures in the hive tests.

lightclient · 2024-01-25T17:03:52Z

So it looks like before #28703, the eth api backend would call rawdb.ReadTransaction(..) to retrieve txs to serve eth_getTransactionReceipt. That function would then check for a txLookupEntry (which was deleted in the code that this PR is touching) and if not found, return nil.

In #28703, the eth api backend was modified to immediately call GetTransactionLookup(..) which first checks the lookup cache before it falls back to the db.

That is why purging the cache upon reorgs resolves the hive test issues.

--

It seems like there are three was to proceed:

merge this PR and purge the cache on reorgs
revert the logic change to the eth api backend and continue using rawdb.ReadTransaction(..)
modify the hive test to get the tx receipt, then query the number to see if the canonical hash matches the block has the tx was mined in

I think my preference is 1) or 2). This is a breaking change in terms of API behavior so unless there is reason to retain the new behavior I think we should return to the old behavior.

lightclient · 2024-01-25T17:04:19Z

Also the test this PR is failing seems to be flaky, it passes locally unless I push -count past 5ish.

holiman

LGTM

MariusVanDerWijden

LGTM, the txlookupcache is only used by the api, so the cost of purging the cache only really impacts nodes that do a lot of transaction lookups (due to tracing or rpc). Normal nodes should not be impacted by this change at all

vr-devil · 2024-01-29T12:36:51Z

waiting a release online ~ come on ~

lightclient · 2024-01-29T18:54:26Z

core/blockchain.go

+		indexesBatch = bc.db.NewBatch()
+		diffs        = types.HashDifference(deletedTxs, addedTxs)
+	)
+	for _, tx := range diffs {


If we're going to purge before writing canonical chain, can we revert this chunk?

They are equal, i am just too lazy to wait another round of CI. I think it should be fine.

* params: begin v.1.13.12 release cycle * internal/flags: fix typo (ethereum#28876) * core/types: fix and test handling of faulty nil-returning signer (ethereum#28879) This adds an error if the signer returns a nil value for one of the signature value fields. * README.md: fix travis badge (ethereum#28889) The hyperlink in the README file that directs to the Travis CI build was broken. This commit updates the link to point to the corrent build page. * eth/catalyst: allow payload attributes v1 in fcu v2 (ethereum#28882) At some point, `ForkchoiceUpdatedV2` stopped working for `PayloadAttributesV1` while `paris` was active. This was causing a few failures in hive. This PR fixes that, and also adds a gate in `ForkchoiceUpdatedV1` to disallow `PayloadAttributesV3`. * docs/postmortems: fix outdated link (ethereum#28893) * core: reset tx lookup cache if necessary (ethereum#28865) This pull request resets the txlookup cache if chain reorg happens, preventing them from remaining reachable. It addresses failures in the hive tests. * build: fix problem with windows line-endings in CI download (ethereum#28900) fixes ethereum#28890 * eth/downloader: fix skeleton cleanup (ethereum#28581) * eth/downloader: fix skeleton cleanup * eth/downloader: short circuit if nothing to delete * eth/downloader: polish the logic in cleanup * eth/downloader: address comments * deps: update memsize (ethereum#28916) * core/txpool/blobpool: post-crash cleanup and addition/removal metrics (ethereum#28914) * core/txpool/blobpool: clean up resurrected junk after a crash * core/txpool/blobpool: track transaction insertions and rejections * core/txpool/blobpool: linnnnnnnt * core/txpool: don't inject lazy resolved transactions into the container (ethereum#28917) * core/txpool: don't inject lazy resolved transactions into the container * core/txpool: minor typo fixes * core/types: fix typo (ethereum#28922) * p2p: fix accidental termination of portMappingLoop (ethereum#28911) * internal/flags: fix --miner.gasprice default listing (ethereum#28932) * all: fix typos in comments (ethereum#28881) * Makefile: add help target to display available targets (ethereum#28845) Co-authored-by: Martin HS <martin@swende.se> Co-authored-by: Felix Lange <fjl@twurst.com> * core: cache transaction indexing tail in memory (ethereum#28908) * eth, miner: fix enforcing the minimum miner tip (ethereum#28933) * eth, miner: fix enforcing the minimum miner tip * ethclient/simulated: fix failing test due the min tip change * accounts/abi/bind: fix simulater gas tip issue * core/state, core/vm: minor uint256 related perf improvements (ethereum#28944) * cmd,internal/era: implement `export-history` subcommand (ethereum#26621) * all: implement era format, add history importer/export * internal/era/e2store: refactor e2store to provide ReadAt interface * internal/era/e2store: export HeaderSize * internal/era: refactor era to use ReadAt interface * internal/era: elevate anonymous func to named * cmd/utils: don't store entire era file in-memory during import / export * internal/era: better abstraction between era and e2store * cmd/era: properly close era files * cmd/era: don't let defers stack * cmd/geth: add description for import-history * cmd/utils: better bytes buffer * internal/era: error if accumulator has more records than max allowed * internal/era: better doc comment * internal/era/e2store: rm superfluous reader, rm superfluous testcases, add fuzzer * internal/era: avoid some repetition * internal/era: simplify clauses * internal/era: unexport things * internal/era,cmd/utils,cmd/era: change to iterator interface for reading era entries * cmd/utils: better defer handling in history test * internal/era,cmd: add number method to era iterator to get the current block number * internal/era/e2store: avoid double allocation during write * internal/era,cmd/utils: fix lint issues * internal/era: add ReaderAt func so entry value can be read lazily Co-authored-by: lightclient <lightclient@protonmail.com> Co-authored-by: Martin Holst Swende <martin@swende.se> * internal/era: improve iterator interface * internal/era: fix rlp decode of header and correctly read total difficulty * cmd/era: fix rebase errors * cmd/era: clearer comments * cmd,internal: fix comment typos --------- Co-authored-by: Martin Holst Swende <martin@swende.se> * core,params: add holesky to default genesis function (ethereum#28903) * node, rpc: add configurable HTTP request limit (ethereum#28948) Adds a configurable HTTP request limit, and bumps the engine default * all: fix docstring names (ethereum#28923) * fix wrong comment * reviewers input * Update log/handler_glog.go --------- Co-authored-by: Martin HS <martin@swende.se> * ethclient/simulated: fix typo (ethereum#28952) (ethclient/simulated):fix typo * eth/gasprice: fix percentile validation in eth_feeHistory (ethereum#28954) * cmd/devp2p, eth: drop support for eth/67 (ethereum#28956) * params, core/forkid: add mainnet timestamp for Cancun (ethereum#28958) * params: add cancun timestamp for mainnet * core/forkid: add test for mainnet cancun forkid * core/forkid: update todo tests for cancun * internal/ethapi: add support for blobs in eth_fillTransaction (ethereum#28839) This change adds support for blob-transaction in certain API-endpoints, e.g. eth_fillTransaction. A follow-up PR will add support for signing such transactions. * internal/era: update block index format to be based on record offset (ethereum#28959) As mentioned in ethereum#26621, the block index format for era1 is not in line with the regular era block index. This change modifies the index so all relative offsets are based against the beginning of the block index record. * params: go-ethereum v1.13.12 stable --------- Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: alex <152680487+bodhi-crypo@users.noreply.github.com> Co-authored-by: protolambda <proto@protolambda.com> Co-authored-by: KeienWang <42377006+keienWang@users.noreply.github.com> Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com> Co-authored-by: rjl493456442 <garyrong0905@gmail.com> Co-authored-by: Péter Szilágyi <peterke@gmail.com> Co-authored-by: zoereco <158379334+zoereco@users.noreply.github.com> Co-authored-by: Chris Ziogas <ziogaschr@gmail.com> Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com> Co-authored-by: Halimao <1065621723@qq.com> Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: lmittmann <3458786+lmittmann@users.noreply.github.com> Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com> Co-authored-by: Austin Roberts <austin.roberts@rivet.cloud>

) This pull request resets the txlookup cache if chain reorg happens, preventing them from remaining reachable. It addresses failures in the hive tests. Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

core: reset tx lookup cache if necessary

7b97c59

rjl493456442 requested review from karalabe and holiman as code owners January 24, 2024 08:01

tobidae-cb mentioned this pull request Jan 26, 2024

Transaction lookup using eth_getTransactionReceipt returns the wrong information after reorg #28885

Closed

rjl493456442 added this to the 1.13.12 milestone Jan 29, 2024

core: reset txlookup cache if reorg happens

064315d

holiman approved these changes Jan 29, 2024

View reviewed changes

MariusVanDerWijden approved these changes Jan 29, 2024

View reviewed changes

core: clear cache before writing canonical chain

2173b6f

lightclient reviewed Jan 29, 2024

View reviewed changes

rjl493456442 merged commit eaac53e into ethereum:master Jan 30, 2024
3 checks passed

lightclient mentioned this pull request Feb 2, 2024

Incorrect transaction receipts after reorg on Holesky #28918

Closed

pratikspatil024 mentioned this pull request Feb 7, 2024

core: reset tx lookup cache if necessary maticnetwork/bor#1157

Merged

18 tasks

rjl493456442 mentioned this pull request Feb 7, 2024

Calling the eth_getTransactionReceipt method yielded different results #28938

Closed

NickKelly1 mentioned this pull request Feb 15, 2024

rpc eth_getTransactionByHash and eth_getTransactionReceipt inconsistency after reorgs #28992

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

core: reset tx lookup cache if necessary #28865

core: reset tx lookup cache if necessary #28865

rjl493456442 commented Jan 24, 2024 •

edited

lightclient commented Jan 25, 2024

lightclient commented Jan 25, 2024 •

edited

holiman left a comment

MariusVanDerWijden left a comment

vr-devil commented Jan 29, 2024

lightclient Jan 29, 2024

rjl493456442 Jan 30, 2024

core: reset tx lookup cache if necessary #28865

core: reset tx lookup cache if necessary #28865

Conversation

rjl493456442 commented Jan 24, 2024 • edited

lightclient commented Jan 25, 2024

lightclient commented Jan 25, 2024 • edited

holiman left a comment

Choose a reason for hiding this comment

MariusVanDerWijden left a comment

Choose a reason for hiding this comment

vr-devil commented Jan 29, 2024

lightclient Jan 29, 2024

Choose a reason for hiding this comment

rjl493456442 Jan 30, 2024

Choose a reason for hiding this comment

rjl493456442 commented Jan 24, 2024 •

edited

lightclient commented Jan 25, 2024 •

edited