Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/geth: implement dev mode for post-merge #27327

Merged
merged 49 commits into from Jul 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
667a585
devmode (rebased again)
jwasinger Jun 12, 2023
b4d5583
break withdrawal logic into separate structure
jwasinger Jun 12, 2023
e5ca14a
fix/add comments
jwasinger Jun 12, 2023
59850ce
rename clmock->simulated_beacon
jwasinger Jun 12, 2023
797494a
remove unused file. remove unused code from node.go. move mutex ope…
jwasinger Jun 13, 2023
a33ad4a
move feeRecipient access into private function. remove some empty lines
jwasinger Jun 13, 2023
3d7dc00
use shutdown channel instead of context and cancel function
jwasinger Jun 13, 2023
fe5db72
check if we have any transactions, without relying on the pending block
jwasinger Jun 13, 2023
c32fc8d
use Sign instead of Bitlen to check if big.Int == 0
jwasinger Jun 13, 2023
fa9267d
simulated_beacon: fixes
holiman Jun 13, 2023
8d91eaa
don't validate withdrawal indices
jwasinger Jun 13, 2023
5f04806
rename simulated_beacon package to simulatedbeacon. move under cmd/geth
jwasinger Jun 13, 2023
650374a
lint
jwasinger Jun 13, 2023
92441a1
cmd/geth/simulatedbeacon: refactorings
holiman Jun 14, 2023
942a841
rename withdrawals->withdrawalQueue. re-include withdrawals in blocks
jwasinger Jun 14, 2023
a0938fb
remove usage of log.Crit
jwasinger Jun 14, 2023
434cc20
fix withdrawals not being included when dev period is 0
jwasinger Jun 15, 2023
b7408af
don't register catalyst service twice in dev mode.
jwasinger Jun 19, 2023
c8c920d
rename simulatedbeacon files to reflect package name
jwasinger Jun 19, 2023
59df562
Update cmd/geth/config.go
jwasinger Jun 19, 2023
9c91d84
Update cmd/geth/simulatedbeacon/simulatedbeacon.go
jwasinger Jun 19, 2023
cca59ec
Update cmd/geth/simulatedbeacon/simulatedbeacon.go
jwasinger Jun 19, 2023
06213ac
goimports
jwasinger Jun 19, 2023
12b410a
add license
jwasinger Jun 20, 2023
7d4d5cf
use common.Hash instead of *common.Hash for feeRecipient apis
jwasinger Jun 20, 2023
30ca6f5
instantiate withdrawalQueue with new(...)
jwasinger Jun 20, 2023
f59982e
move withdrawals into single array in withdrawalQueue. rename getQue…
jwasinger Jun 20, 2023
7c98200
remove uneccesary calls to engineAPI.ExchangeTransitionConfigurationV1
jwasinger Jun 20, 2023
f687ee4
move SimulatedBeacon.shutdownCh initialization into constructor
jwasinger Jun 20, 2023
1bb92db
move simulatedbeacon package contents under eth/catalyst
jwasinger Jun 20, 2023
be5e29e
rename getFeeRecipient() -> feeRecipient
jwasinger Jun 20, 2023
c8d9ef3
Update eth/catalyst/simulated_beacon.go
jwasinger Jun 20, 2023
bb7b0cd
Update eth/catalyst/simulated_beacon.go
jwasinger Jun 20, 2023
402ac2e
Update eth/catalyst/simulated_beacon.go
jwasinger Jun 20, 2023
27f0bb0
move beginSealing and finalizeSealing out of loop
jwasinger Jun 20, 2023
8d3330a
cmd/geth: prefer utils.Fatalf over returning an error object in makeF…
jwasinger Jun 21, 2023
e21dc29
only clear as many pending withdrawals from the queue as were include…
jwasinger Jun 25, 2023
e3ab2db
change dev.period flag to uint64 flag type
jwasinger Jun 26, 2023
714bf98
don't register catalyst service if in light sync mode
jwasinger Jun 27, 2023
4fd9197
better naming for feeRecipientMutex. move initialization of Simulate…
jwasinger Jun 27, 2023
5eac860
Update eth/catalyst/simulated_beacon.go
jwasinger Jun 27, 2023
1340352
wip: prepare for using new miner GetFullPayload api
jwasinger Jun 27, 2023
c6b50e9
eth/catalyst, miner: add GetFullPayload API
rjl493456442 Jun 26, 2023
e83d22d
somewhat works... but is mistakenly tracking multiple payloads simult…
jwasinger Jun 27, 2023
68608ca
refactor simulated_beacon based on miner API changes
jwasinger Jun 28, 2023
3a16d22
remove logging. add comment
jwasinger Jun 28, 2023
c592736
remove period from developer genesis configuration
jwasinger Jun 28, 2023
a504a80
comments
jwasinger Jun 28, 2023
ba286e5
cmd, eth/catalyst, miner: terminate block building by ResolveFull
rjl493456442 Jun 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
18 changes: 18 additions & 0 deletions cmd/geth/config.go
Expand Up @@ -32,6 +32,8 @@ import (
"github.com/ethereum/go-ethereum/accounts/scwallet"
"github.com/ethereum/go-ethereum/accounts/usbwallet"
"github.com/ethereum/go-ethereum/cmd/utils"
"github.com/ethereum/go-ethereum/eth/catalyst"
ethcatalyst "github.com/ethereum/go-ethereum/eth/catalyst"
"github.com/ethereum/go-ethereum/eth/downloader"
"github.com/ethereum/go-ethereum/eth/ethconfig"
"github.com/ethereum/go-ethereum/internal/ethapi"
Expand Down Expand Up @@ -193,6 +195,22 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
if ctx.IsSet(utils.SyncTargetFlag.Name) && cfg.Eth.SyncMode == downloader.FullSync {
utils.RegisterFullSyncTester(stack, eth, ctx.Path(utils.SyncTargetFlag.Name))
}

// Start the dev mode if requested, or launch the engine API for
// interacting with external consensus client.
if ctx.IsSet(utils.DeveloperFlag.Name) {
simBeacon, err := catalyst.NewSimulatedBeacon(ctx.Uint64(utils.DeveloperPeriodFlag.Name), eth)
if err != nil {
utils.Fatalf("failed to register dev mode catalyst service: %v", err)
}
catalyst.RegisterSimulatedBeaconAPIs(stack, simBeacon)
stack.RegisterLifecycle(simBeacon)
} else if cfg.Eth.SyncMode != downloader.LightSync {
err := ethcatalyst.Register(stack, eth)
if err != nil {
utils.Fatalf("failed to register catalyst service: %v", err)
}
}
return stack, backend
}

Expand Down
4 changes: 2 additions & 2 deletions cmd/geth/main.go
Expand Up @@ -128,8 +128,8 @@ var (
utils.NodeKeyHexFlag,
utils.DNSDiscoveryFlag,
utils.DeveloperFlag,
utils.DeveloperPeriodFlag,
utils.DeveloperGasLimitFlag,
utils.DeveloperPeriodFlag,
utils.VMEnableDebugFlag,
utils.NetworkIdFlag,
utils.EthStatsURLFlag,
Expand Down Expand Up @@ -408,7 +408,7 @@ func startNode(ctx *cli.Context, stack *node.Node, backend ethapi.Backend, isCon
}

// Start auxiliary services if enabled
if ctx.Bool(utils.MiningEnabledFlag.Name) || ctx.Bool(utils.DeveloperFlag.Name) {
if ctx.Bool(utils.MiningEnabledFlag.Name) {
// Mining only makes sense if a full Ethereum node is running
if ctx.String(utils.SyncModeFlag.Name) == "light" {
utils.Fatalf("Light clients do not support mining")
Expand Down
7 changes: 2 additions & 5 deletions cmd/utils/flags.go
Expand Up @@ -160,7 +160,7 @@ var (
Usage: "Ephemeral proof-of-authority network with a pre-funded developer account, mining enabled",
Category: flags.DevCategory,
}
DeveloperPeriodFlag = &cli.IntFlag{
DeveloperPeriodFlag = &cli.Uint64Flag{
Name: "dev.period",
Usage: "Block period to use in developer mode (0 = mine only if transaction pending)",
Category: flags.DevCategory,
Expand Down Expand Up @@ -1835,7 +1835,7 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) {
log.Info("Using developer account", "address", developer.Address)

// Create a new developer genesis block or reuse existing one
cfg.Genesis = core.DeveloperGenesisBlock(uint64(ctx.Int(DeveloperPeriodFlag.Name)), ctx.Uint64(DeveloperGasLimitFlag.Name), developer.Address)
cfg.Genesis = core.DeveloperGenesisBlock(ctx.Uint64(DeveloperGasLimitFlag.Name), developer.Address)
if ctx.IsSet(DataDirFlag.Name) {
// If datadir doesn't exist we need to open db in write-mode
// so leveldb can create files.
Expand Down Expand Up @@ -1910,9 +1910,6 @@ func RegisterEthService(stack *node.Node, cfg *ethconfig.Config) (ethapi.Backend
Fatalf("Failed to create the LES server: %v", err)
}
}
if err := ethcatalyst.Register(stack, backend); err != nil {
rjl493456442 marked this conversation as resolved.
Show resolved Hide resolved
Fatalf("Failed to register the Engine API service: %v", err)
}
stack.RegisterAPIs(tracers.APIs(backend.APIBackend))
return backend.APIBackend, backend
}
Expand Down
2 changes: 1 addition & 1 deletion console/console_test.go
Expand Up @@ -94,7 +94,7 @@ func newTester(t *testing.T, confOverride func(*ethconfig.Config)) *tester {
t.Fatalf("failed to create node: %v", err)
}
ethConf := &ethconfig.Config{
Genesis: core.DeveloperGenesisBlock(15, 11_500_000, common.Address{}),
Genesis: core.DeveloperGenesisBlock(11_500_000, common.Address{}),
Miner: miner.Config{
Etherbase: common.HexToAddress(testAddress),
},
Expand Down
11 changes: 3 additions & 8 deletions core/genesis.go
Expand Up @@ -554,21 +554,16 @@ func DefaultSepoliaGenesisBlock() *Genesis {
}

// DeveloperGenesisBlock returns the 'geth --dev' genesis block.
func DeveloperGenesisBlock(period uint64, gasLimit uint64, faucet common.Address) *Genesis {
func DeveloperGenesisBlock(gasLimit uint64, faucet common.Address) *Genesis {
// Override the default period to the user requested one
config := *params.AllCliqueProtocolChanges
config.Clique = &params.CliqueConfig{
Period: period,
Epoch: config.Clique.Epoch,
}
config := *params.AllDevChainProtocolChanges
jwasinger marked this conversation as resolved.
Show resolved Hide resolved

// Assemble and return the genesis with the precompiles and faucet pre-funded
return &Genesis{
Config: &config,
ExtraData: append(append(make([]byte, 32), faucet[:]...), make([]byte, crypto.SignatureLength)...),
GasLimit: gasLimit,
BaseFee: big.NewInt(params.InitialBaseFee),
Difficulty: big.NewInt(1),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it a bit weird that the genesis block has a non-zero difficulty:

> eth.getBlock(0).difficulty
131072

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was very difficult genecizing this genesis block

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difficulty formula is undefined for difficulties <131072. There is a minimum difficulty, and below that, the spec can be interpreted in two different ways, depending on at what exact step one decides to apply the minimum threshold.

Difficulty: big.NewInt(0),
Alloc: map[common.Address]GenesisAccount{
common.BytesToAddress([]byte{1}): {Balance: big.NewInt(1)}, // ECRecover
common.BytesToAddress([]byte{2}): {Balance: big.NewInt(1)}, // SHA256
Expand Down
29 changes: 20 additions & 9 deletions eth/catalyst/api.go
Expand Up @@ -407,7 +407,22 @@ func (api *ConsensusAPI) GetPayloadV2(payloadID engine.PayloadID) (*engine.Execu

func (api *ConsensusAPI) getPayload(payloadID engine.PayloadID) (*engine.ExecutionPayloadEnvelope, error) {
log.Trace("Engine API request received", "method", "GetPayload", "id", payloadID)
data := api.localBlocks.get(payloadID)
data := api.localBlocks.get(payloadID, false)
if data == nil {
return nil, engine.UnknownPayload
}
return data, nil
}

// getFullPayload returns a cached payload by it. The difference is that this
// function always expects a non-empty payload, but can also return empty one
// if no transaction is executable.
//
// Note, this function is not a part of standard engine API, meant to be used
// by consensus client mock in dev mode.
func (api *ConsensusAPI) getFullPayload(payloadID engine.PayloadID) (*engine.ExecutionPayloadEnvelope, error) {
log.Trace("Engine API request received", "method", "GetFullPayload", "id", payloadID)
data := api.localBlocks.get(payloadID, true)
if data == nil {
return nil, engine.UnknownPayload
}
Expand Down Expand Up @@ -715,8 +730,8 @@ func (api *ConsensusAPI) ExchangeCapabilities([]string) []string {
return caps
}

// GetPayloadBodiesV1 implements engine_getPayloadBodiesByHashV1 which allows for retrieval of a list
// of block bodies by the engine api.
// GetPayloadBodiesByHashV1 implements engine_getPayloadBodiesByHashV1 which
// allows for retrieval of a list of block bodies by the engine api.
func (api *ConsensusAPI) GetPayloadBodiesByHashV1(hashes []common.Hash) []*engine.ExecutionPayloadBodyV1 {
var bodies = make([]*engine.ExecutionPayloadBodyV1, len(hashes))
for i, hash := range hashes {
Expand All @@ -726,8 +741,8 @@ func (api *ConsensusAPI) GetPayloadBodiesByHashV1(hashes []common.Hash) []*engin
return bodies
}

// GetPayloadBodiesByRangeV1 implements engine_getPayloadBodiesByRangeV1 which allows for retrieval of a range
// of block bodies by the engine api.
// GetPayloadBodiesByRangeV1 implements engine_getPayloadBodiesByRangeV1 which
// allows for retrieval of a range of block bodies by the engine api.
func (api *ConsensusAPI) GetPayloadBodiesByRangeV1(start, count hexutil.Uint64) ([]*engine.ExecutionPayloadBodyV1, error) {
if start == 0 || count == 0 {
return nil, engine.InvalidParams.With(fmt.Errorf("invalid start or count, start: %v count: %v", start, count))
Expand All @@ -753,23 +768,19 @@ func getBody(block *types.Block) *engine.ExecutionPayloadBodyV1 {
if block == nil {
return nil
}

var (
body = block.Body()
txs = make([]hexutil.Bytes, len(body.Transactions))
withdrawals = body.Withdrawals
)

for j, tx := range body.Transactions {
data, _ := tx.MarshalBinary()
txs[j] = hexutil.Bytes(data)
}

// Post-shanghai withdrawals MUST be set to empty slice instead of nil
if withdrawals == nil && block.Header().WithdrawalsHash != nil {
withdrawals = make([]*types.Withdrawal, 0)
}

return &engine.ExecutionPayloadBodyV1{
TransactionData: txs,
Withdrawals: withdrawals,
Expand Down
7 changes: 5 additions & 2 deletions eth/catalyst/queue.go
Expand Up @@ -73,7 +73,7 @@ func (q *payloadQueue) put(id engine.PayloadID, payload *miner.Payload) {
}

// get retrieves a previously stored payload item or nil if it does not exist.
func (q *payloadQueue) get(id engine.PayloadID) *engine.ExecutionPayloadEnvelope {
func (q *payloadQueue) get(id engine.PayloadID, full bool) *engine.ExecutionPayloadEnvelope {
q.lock.RLock()
defer q.lock.RUnlock()

Expand All @@ -82,7 +82,10 @@ func (q *payloadQueue) get(id engine.PayloadID) *engine.ExecutionPayloadEnvelope
return nil // no more items
}
if item.id == id {
return item.payload.Resolve()
if !full {
return item.payload.Resolve()
}
return item.payload.ResolveFull()
}
}
return nil
Expand Down