Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLI] Add support for running standard indexer processors to the local testnet #10254

Merged

Conversation

banool
Copy link
Contributor

@banool banool commented Sep 26, 2023

Stack

Description

This PR adds support for running indexer processors to the local testnet. The processors you run is configurable with --processors, which uses the new ProcessorName enum.

Note: I will update the dependency from that branch to a specific rev later.

Test Plan

cargo run -p aptos -- node run-local-testnet --assume-yes --force-restart --with-indexer-api
Readiness endpoint: http://0.0.0.0:8070/

Postgres is starting, please wait...
Transaction stream is starting, please wait...
Node API is starting, please wait...
Faucet is starting, please wait...

Completed generating configuration:
        Log file: "/Users/dport/.aptos/testnet/validator.log"
        Test dir: "/Users/dport/.aptos/testnet"
        Aptos root key path: "/Users/dport/.aptos/testnet/mint.key"
        Waypoint: 0:2cc2b83a3435b6b38f5f8ef84afe96aef2458c524445d3d72cd55b0459e627c5
        ChainId: 4
        REST API endpoint: http://0.0.0.0:8080
        Metrics endpoint: http://0.0.0.0:9101/metrics
        Aptosnet fullnode network endpoint: /ip4/0.0.0.0/tcp/6181
        Indexer gRPC node stream endpoint: 0.0.0.0:50051

Aptos is running, press ctrl-c to exit

Node API is ready. Endpoint: http://0.0.0.0:8080/
Postgres is ready. Endpoint: postgres://postgres@127.0.0.1:5433/local_testnet
Transaction stream is ready. Endpoint: http://0.0.0.0:50051/
Faucet is ready. Endpoint: http://127.0.0.1:8081/

Applying post startup steps...

Setup is complete, you can now use the local testnet!
local_testnet-# \x
Expanded display is on.
local_testnet-# select * from events limit 1;
ERROR:  syntax error at or near "select"
LINE 2: select * from events limit 1
        ^
local_testnet=# select * from events limit 1;
-[ RECORD 1 ]------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sequence_number          | 3
creation_number          | 3
account_address          | 0x0000000000000000000000000000000000000000000000000000000000000001
transaction_version      | 4
transaction_block_height | 3
type                     | 0x1::block::NewBlockEvent
data                     | {"hash": "0x1cce75abaf027f116439ef26040f64f6d5337f6a04f0b093bed08a51b3da5981", "epoch": "2", "round": "2", "height": "3", "proposer": "0xa03f668876e2ec4d37d3b7f33ad29b5d01c777a189c2dfc0b03d67046c19c7d9", "time_microseconds": "1695901972557097", "failed_proposer_indices": [], "previous_block_votes_bitvec": "0x80"}
inserted_at              | 2023-09-28 11:52:53.356014
event_index              | 0

local_testnet=#

I also tested that works when using --with-host-postgres:

cargo run -p aptos -- node run-local-testnet --assume-yes --force-restart --with-indexer-api --postgres-user $USER --use-host-postgres

@banool banool changed the base branch from main to banool/local-testnet-structure September 26, 2023 18:34
@banool banool force-pushed the banool/local-testnet-structure branch from 5076245 to cb317a0 Compare September 26, 2023 18:38
@banool banool force-pushed the banool/local-testnet-processors branch from 5f28de8 to 9c50340 Compare September 26, 2023 18:38
@banool banool force-pushed the banool/local-testnet-structure branch 4 times, most recently from 3b28941 to f59af3e Compare September 27, 2023 12:57
@banool banool force-pushed the banool/local-testnet-processors branch from 9c50340 to 3e46168 Compare September 27, 2023 12:58
@banool banool force-pushed the banool/local-testnet-processors branch from 3e46168 to cea8653 Compare September 27, 2023 15:39
@banool banool changed the base branch from banool/local-testnet-structure to banool/local-testnet-postgres September 27, 2023 15:40
@banool banool force-pushed the banool/local-testnet-postgres branch from fcb0465 to fe141c9 Compare September 28, 2023 11:12
@banool banool force-pushed the banool/local-testnet-processors branch 3 times, most recently from 9ec8ad5 to eaa59f2 Compare September 28, 2023 13:07
@banool banool force-pushed the banool/local-testnet-postgres branch from fe141c9 to 25fe203 Compare September 28, 2023 13:48
@banool banool force-pushed the banool/local-testnet-processors branch from eaa59f2 to c984694 Compare September 28, 2023 13:49
@banool banool force-pushed the banool/local-testnet-postgres branch from 25fe203 to ec1bb8f Compare September 28, 2023 17:20
@banool banool force-pushed the banool/local-testnet-processors branch 3 times, most recently from dfc812a to 48f2219 Compare September 29, 2023 08:09
@banool banool force-pushed the banool/local-testnet-postgres branch from ec1bb8f to a20df49 Compare September 29, 2023 08:17
@banool banool force-pushed the banool/local-testnet-processors branch from 48f2219 to 90c7967 Compare September 29, 2023 08:17
@banool banool marked this pull request as ready for review September 29, 2023 08:59
@banool banool force-pushed the banool/local-testnet-postgres branch from a20df49 to 0166c9f Compare September 29, 2023 10:03
@banool banool force-pushed the banool/local-testnet-processors branch from 90c7967 to 4d7e72f Compare September 29, 2023 10:03
@banool banool force-pushed the banool/local-testnet-postgres branch from 0166c9f to 543cdac Compare September 29, 2023 18:48
@banool banool force-pushed the banool/local-testnet-processors branch from 4d7e72f to 8e61ff4 Compare September 29, 2023 18:48
@banool banool force-pushed the banool/local-testnet-postgres branch from 543cdac to 446a5fb Compare October 1, 2023 15:35
@banool banool force-pushed the banool/local-testnet-processors branch from 8e61ff4 to abcebd8 Compare October 1, 2023 15:35
@banool banool force-pushed the banool/local-testnet-postgres branch from 446a5fb to 57b3c28 Compare October 1, 2023 17:56
@banool banool force-pushed the banool/local-testnet-processors branch 2 times, most recently from 169b0fb to 0d385ad Compare October 1, 2023 17:58
@banool banool force-pushed the banool/local-testnet-postgres branch from 57b3c28 to 126f8d8 Compare October 1, 2023 22:36
@banool banool force-pushed the banool/local-testnet-processors branch from 0d385ad to 4e0b6d2 Compare October 1, 2023 22:37
@banool banool force-pushed the banool/local-testnet-postgres branch from 126f8d8 to 7f3c143 Compare October 2, 2023 11:19
@banool banool force-pushed the banool/local-testnet-processors branch from 4e0b6d2 to 20d4773 Compare October 2, 2023 11:19
@banool banool force-pushed the banool/local-testnet-postgres branch from 7f3c143 to b8a554c Compare October 4, 2023 08:12
@banool banool force-pushed the banool/local-testnet-processors branch from 20d4773 to 0455b2f Compare October 4, 2023 08:12
@banool banool force-pushed the banool/local-testnet-postgres branch from b8a554c to c1a3508 Compare October 4, 2023 15:15
@banool banool force-pushed the banool/local-testnet-processors branch from 0455b2f to 981ae09 Compare October 4, 2023 15:15
@banool banool force-pushed the banool/local-testnet-postgres branch from c1a3508 to 0c3cb4b Compare October 4, 2023 17:42
@banool banool force-pushed the banool/local-testnet-processors branch from 981ae09 to ee9d4d2 Compare October 4, 2023 17:42
Copy link
Contributor

@0xmigo 0xmigo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I will let Greg or Bowen to double check on indexer processor stuff.

Comment on lines +30 to +43
#[clap(
long,
value_enum,
default_values_t = vec![
ProcessorName::AccountTransactionsProcessor,
ProcessorName::CoinProcessor,
ProcessorName::DefaultProcessor,
ProcessorName::EventsProcessor,
ProcessorName::FungibleAssetProcessor,
ProcessorName::StakeProcessor,
ProcessorName::TokenProcessor,
ProcessorName::TokenV2Processor,
ProcessorName::UserTransactionProcessor,
],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clap with default values - This is convenience! TIL

Comment on lines +63 to +86
let processor_config = match processor_name {
ProcessorName::AccountTransactionsProcessor => {
ProcessorConfig::AccountTransactionsProcessor
},
ProcessorName::AnsProcessor => {
bail!("ANS processor is not supported in the local testnet")
},
ProcessorName::CoinProcessor => ProcessorConfig::CoinProcessor,
ProcessorName::DefaultProcessor => ProcessorConfig::DefaultProcessor,
ProcessorName::EventsProcessor => ProcessorConfig::EventsProcessor,
ProcessorName::FungibleAssetProcessor => ProcessorConfig::FungibleAssetProcessor,
ProcessorName::NftMetadataProcessor => {
bail!("NFT Metadata processor is not supported in the local testnet")
},
ProcessorName::StakeProcessor => ProcessorConfig::StakeProcessor,
ProcessorName::TokenProcessor => {
ProcessorConfig::TokenProcessor(TokenProcessorConfig {
// This NFT points thing doesn't exist on local testnets.
nft_points_contract: None,
})
},
ProcessorName::TokenV2Processor => ProcessorConfig::TokenV2Processor,
ProcessorName::UserTransactionProcessor => ProcessorConfig::UserTransactionProcessor,
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit - Personally I would add the unsupported processor into a set, then check from the list/set.

@banool banool merged commit 9aaf63f into banool/local-testnet-postgres Oct 5, 2023
3 checks passed
@banool banool deleted the banool/local-testnet-processors branch October 5, 2023 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants