time to push first transaction in the network has increased since Monday

[h4ck3rk3y]

Hey!

I am Gyani and I work on Kurtosis. We maintain an example https://github.com/kurtosis-tech/awesome-kurtosis/tree/main/smart-contract-example where we show how a smart contract can be deployed against an Etheruem test net spun up using Kurotsis https://github.com/kurtosis-tech/ethereum-package. By default we spin up geth + lighthouse, latest images of both.

From Tuesday we were noticing that the test was failing https://app.circleci.com/pipelines/github/kurtosis-tech/awesome-kurtosis/1476/workflows/7c9ac168-3b25-43fc-9232-b2da96660edc/jobs/3985

The failure talks about ProviderError: transaction indexing is in progress; after much debugging and not finding anything on the internet I decided to read your source code

which is rather new and only got added on Monday - https://github.com/ethereum/go-ethereum/pull/28703/files#diff-af55d4d4253e0b8b994cedaa88607f84009a0b2971f9b3514afa71efc906a9c5

When I switched to using 1.13.10 https://github.com/kurtosis-tech/awesome-kurtosis/pull/282/files things started working again as expected.

I am wondering if this is a bug or is there some change you expect us to make to the smart contract example.

Tagging @leoporoli from Kurtosis who I have been debugging this with

[h4ck3rk3y]

Switching v1.13.10 to v1.13.11 you will get the bug that I am talking about

Note the run script inside the smart contract example is Linux only; but can be run with some tweaks on Mac using gsed or a different sed sed -i '' "s/<PORT>/$PORT/" hardhat.config.ts

[h4ck3rk3y]

It seems like transactions that are being posted later are being processed just fine. If I try deploying the same thing with hardhat a little later it goes through.

[h4ck3rk3y]

Summarizing, while running against the latest published since Monday

1. The smart contracts we are deploying are here; these are example/toy smart contracts https://github.com/kurtosis-tech/awesome-kurtosis/tree/main/smart-contract-example/contracts
2. The error i get is transaction indexing is in progress; if I try submitting after a while then the error goes away and the network resumes as normal
3. Here is the log of my geth client running v1.13.11; notice how the error happens in the beginning but fades away and the network resumes as normal https://gist.github.com/h4ck3rk3y/169fe3d2374afaffed75e976bc7ef579

[lightclient]

Can you please describe the order of events of your test? E.g. are you starting a chain from scratch? When is the block produced? Do you query for receipt immediately after the block is produced?

With #28703, we no longer index txs during sync so after a node is synced you'll have to wait a few minutes to query. In your case, it appears you are starting a fresh chain though, so I wouldn't expect this error to pop up, but maybe if you outline the events it will be clearer.

[h4ck3rk3y]

I am starting a chain from scratch. Going by the logs the first transaction is being posted even before block number=1.

[antazoey]

I am also hitting this issue in my local development node that uses geth.
I believe my flow is similar:

1. Init chain from genesis
2. Make first transaction - a deploy - works great
3. Second transaction - a contract function invoke - it fails if I try to wait for the tx too faster after sending it with this exact error message

I am not a Go dev but looking at the code, it looks like this error message is returned at the interface level: https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/errors.go#L78

[rjl493456442]

it fails if I try to wait for the tx too faster after sending it with this exact error message

What's the waiting time configured after sending the transaction? Originally the returned receipt will be none if it's not indexed yet internally, but now an error(transaction indexing is still in progress) is bubbled up.

Does this change affect your workflow?

[antazoey]

What's the waiting time configured after sending the transaction?

After sending, we call wait right away. Here is the wait function:

tx = web3.eth.wait_for_transaction_receipt(hex_hash)

I opened a similar issue in web3.py to maybe adjust the waiting function to handle this error.

Does this change affect your workflow?

I believe it affects web3.py wait for transaction receipt method. right now, it fails instead of waiting.

So I am 2 dependencies down from this change

[rjl493456442]

@h4ck3rk3y are you using the hardhat in your project? I think it has the same issue as web3.py has ethereum/web3.py#3212 (comment)

Would be nice to open a ticket there too and to align with the new behavior in Geth.

[fselmo]

Related to the web3.py ticket, it would be quite nice to get unique error codes in some situations like these in order to distinguish errors a bit better without relying on the message alone.

[rjl493456442]

Yeah, I did plan to apply a unique error code for it. But it's more like a geth internal thing and error-code spec is not maintained properly? So i just return -320000, a common error code for everything.

We do need to improve it

[rjl493456442]

@h4ck3rk3y @antazoey could you please try the latest web3.py?

[antazoey]

@h4ck3rk3y @antazoey could you please try the latest web3.py?

Yes, the web3.py upgrade did resolve my issue.

[holiman]

This issue seems to be resolved

[rjl493456442]

@h4ck3rk3y Which SDK you are using to interact with blockchain?

[h4ck3rk3y]

Hey, sorry for the silence so far @rjl493456442. I am using Hardhat for this project yes, v2.13. I'll try upgrading tomorrow to check if the behavior is fixed in hardhat or not.