Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make EstimateGas bail early when a tx reverts for any reason other than Out of Gas #27502

Closed

Conversation

roberto-bayardo
Copy link
Contributor

@roberto-bayardo roberto-bayardo commented Jun 17, 2023

EstimateGas currently conducts a binary search to determine a suitable gas limit for a tx, and continues searching with a higher gas limit whenever the transaction's execution results in revert. While this makes sense if the reason for revert is out of gas, increasing the gas limit is highly unlikely to allow a non-OOG reverting transaction to succeed (unless it is making unusual use of the GAS opcode).

This PR makes EstimateGas return immediately when a transaction reverts for any reason other than out of gas. This has two benefits:

  1. Saves compute overhead.

  2. Eliminates a race condition where a reverting transaction causes the gas limit lowerbound to increase repeatedly during gas estimation, only for a state change to allow the transaction to later succeed later in the loop, resulting in an unreasonably high estimate being returned. (We believe this is happening in practice in one of our applications)

@roberto-bayardo roberto-bayardo changed the title make EstimateGas bail early when a transaction reverts for any reason than Out of Gas make EstimateGas bail early when a tx reverts for any reason other than Out of Gas Jun 17, 2023
@joohhnnn
Copy link
Contributor

In the case where the error is not due to insufficient gas, it will return an error. Here is the code snippet:

if err != nil {
	if errors.Is(err, core.ErrIntrinsicGas) {
		return true, nil, nil // Special case, raise gas limit
	}
	return true, nil, err // Bail out
}

Additionally, when an error is encountered, it will immediately return. Here is the code snippet:

if err != nil {
	return 0, err
}

Therefore, I believe the functionality you mentioned has already been implemented.

@roberto-bayardo
Copy link
Contributor Author

roberto-bayardo commented Jun 18, 2023

In the case where the error is not due to insufficient gas, it will return an error. Here is the code snippet:

My concern is not errors (which as you note are handled correctly) but transactions reverting. This case does not appear to result in an error being returned from DoCall(), but rather a result being returned with a non.nil Err field. At least that is what is implied by the implementation of Call() that wraps DoCall() to convert tx revert outcomes into errors (but is not called here):

	result,_ err := DoCall(...)
	if err != nil {
		return nil, err
	}
	// If the result contains a revert reason, try to unpack and return it.
	if len(result.Revert()) > 0 {
		return nil, newRevertError(result)
	}
	return result.Return(), result.Err

@roberto-bayardo
Copy link
Contributor Author

@mdehoog has noted that because of the GAS opcode, transactions could revert for non-out of gas reasons based on gas remaining, therefore one might need to continue the binary search in such cases to get an accurate gaslimit.

Another fix would be to lock the VM during the entire binary search, but that would result in one hell of a blocking call.

@jwasinger
Copy link
Contributor

If I understand correctly, the racey behavior you're describing is caused by Geth retrieving the state at each iteration of the loop inside estimate gas. If the pending/latest state is used, this could change mid-call causing strange results.

I opened a PR to address that issue.

@roberto-bayardo
Copy link
Contributor Author

If I understand correctly, the racey behavior you're describing is caused by Geth retrieving the state at each iteration of the loop inside estimate gas. If the pending/latest state is used, this could change mid-call causing strange results.

I opened a PR to address that issue.

yes, brilliant!

@roberto-bayardo
Copy link
Contributor Author

Closing in favor of @jwasinger's fix above.

@roberto-bayardo roberto-bayardo deleted the roberto/estimate-gas branch June 19, 2023 03:19
holiman pushed a commit that referenced this pull request Jun 20, 2023
#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: #27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
spencer-tb pushed a commit to spencer-tb/go-ethereum that referenced this pull request Jul 7, 2023
ethereum#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
MoonShiesty pushed a commit to MoonShiesty/go-ethereum that referenced this pull request Aug 30, 2023
ethereum#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
darioush pushed a commit to ava-labs/coreth that referenced this pull request Sep 1, 2023
…s (#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum/go-ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.

Signed-off-by: Darioush Jalali <darioush.jalali@avalabs.org>
devopsbo3 pushed a commit to HorizenOfficial/go-ethereum that referenced this pull request Nov 10, 2023
ethereum#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
cyyber pushed a commit to cyyber/go-zond that referenced this pull request Nov 27, 2023
…s (#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum/go-ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
minh-bq pushed a commit to minh-bq/ronin that referenced this pull request Jan 9, 2024
…s (#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum/go-ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
minh-bq pushed a commit to minh-bq/ronin that referenced this pull request Jan 10, 2024
…s (#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum/go-ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
minh-bq pushed a commit to minh-bq/ronin that referenced this pull request Feb 15, 2024
…s (#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum/go-ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.
minh-bq added a commit to axieinfinity/ronin that referenced this pull request Feb 15, 2024
…#392)

* internal/ethapi: make EstimateGas use `latest` block by default (#24363)

* EstimateGas should use LatestBlockNumber by default

* graphql: default to use latest for gas estimation

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>

* ethclient/gethclient: fix bugs in override object encoding (#25616)

This fixes a bug where contract code would be overridden to empty code ("0x")
when the Code field of OverrideAccount was left nil. The change also cleans up
the encoding of overrides to only send necessary fields, and improves documentation.

Fixes #25615

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>

* internal/ethapi: add block overrides to eth_call (#26414)

Adds an optional config parameter to eth_call which allows users to override block context fields (same functionality that was added to traceCall in #24871)

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>

* internal/ethapi: use same state for each invocation within EstimateGas (#27505)

EstimateGas repeatedly executes a transaction, performing a binary search with multiple gas prices to determine proper pricing. Each call retrieves a new copy of the state (https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/api.go#L1017) . Because the pending/latest state can change during the execution of EstimateGas, this can potentially cause strange behavior (as noted here: ethereum/go-ethereum#27502 (comment)).

This PR modifies EstimateGas to retrieve the state once and use a copy of it for every call invocation it does.

* internal/ethapi: add state override to estimateGas (#27845)

* internal/ethapi: optimize & clean up EstimateGas (#27710)

Optimizations:

- Previously, if a transaction was reverting, EstimateGas would exhibit worst-case behavior and binary search up to the max gas limit (~40 state-clone + tx executions). This change allows EstimateGas to return after only a single unconstrained execution in this scenario.
- Uses the gas used from the unconstrained execution to bias the remaining binary search towards the likely solution in a simple way that doesn't impact the worst case. For a typical contract-invoking transaction, this reduces the median number of state-clone+executions from 25 to 18 (28% reduction).

Cleanup:

- added & improved function + code comments
- correct the EstimateGas documentation to clarify the gas limit determination is at latest block, not pending, if the blockNr is unspecified.

* eth/gasestimator, internal/ethapi: move gas estimator out of rpc (#28600)

* eth/gasestimator: allow slight estimation error in favor of less iterations (#28618)

* eth/gasestimator: early exit for plain transfer and error allowance

* core, eth/gasestimator: hard guess at a possible required gas

* internal/ethapi: update estimation tests with the error ratio

* eth/gasestimator: I hate you linter

* graphql: fix gas estimation test

---------

Co-authored-by: Oren <orenyomtov@users.noreply.github.com>

* internal/ethapi: partially copy the api_test from go-ethereum

This commit only copies the TestEstimateGas and TestCall from the original file
in go-ethereum.

---------

Co-authored-by: zhiqiangxu <652732310@qq.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Janko Simonovic <simonovic86@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: jwasinger <j-wasinger@hotmail.com>
Co-authored-by: Amin Talebi <talebi242@gmail.com>
Co-authored-by: Roberto Bayardo <roberto.bayardo@coinbase.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Oren <orenyomtov@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants