Skip to content

Commit

Permalink
p2p: enable dht based disovery (#4736)
Browse files Browse the repository at this point in the history
this makes use of "native" libp2p discovery by enabling kad dht, peers exchange (adjusting score) and relay servers. the primary reason is to benefit from hole punching and multiple transports support (quic, webrtc).

in general peers will be obtained by one of the following methods:
- getting more peers from dht. in order for peer to be added to public dht it needs to have dialable address and advertise it to the neighbors. this is pretty much analogous to our current setup, but it also supports multiple addresses so we can enable quic in next change. as it is udp based it may work on nat/firewall devices where tcp was restricted.
- peer exchange (as a part of gossipsub) will advertise peers to other nodes as part of PRUNE message, once other node sees relayed address it will initiate holepunch.  if peer is not publicly dialable it will request relay reservation from the bootnodes. once received this reservation will be communicated to bootnode as one of the peer addresses.
- additionally we will persist peers in p2p/connected.txt every 30minutes, and try to reconnect with them on startup to decrease load on bootnodes/relays. 

configuration changes:
- `p2p-bootnode` option needs to be set explicitly on bootnodes in order to start dht in server mode, and adjust gossipsub configuration
- `p2p-reachability` can be either `public` or `private`. we have to set `public` on bootnodes, otherwise autonat might be confused by out setup, beside that it is only useful for testing.
- `p2p-holepunching` will enable relay client and holepunching. enabled by default
- `p2p-disable-legacy-discovery` disables our own discovery. it can be run in parallel, but we will have no reason to keep it if all works as expected
- `p2p-private-network` disables dht filtering by public address, it is mostly useful for testing
- `relay-server` section:
     - `enable` enable relay server, has to be enabled on bootnodes
     - `reservations` number of relay reservations, 512 by default
     -  `ttl` reservation ttl, 20 minutes  

example of the p2p config section:
```json
"p2p": {
        "listen": "/ip4/0.0.0.0/tcp/5000",
        "p2p-disable-legacy-discovery": true,
        "relay-server": {
            "enable": "true"
        },
        "p2p-reachability": "public",
        "p2p-bootnode": true,
}
```
  • Loading branch information
dshulyak committed Jul 31, 2023
1 parent a975106 commit 516214f
Show file tree
Hide file tree
Showing 13 changed files with 791 additions and 149 deletions.
5 changes: 4 additions & 1 deletion cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,10 @@ func AddCommands(cmd *cobra.Command) {
cfg.P2P.Bootnodes, "entrypoints into the network")
cmd.PersistentFlags().StringVar(&cfg.P2P.AdvertiseAddress, "advertise-address",
cfg.P2P.AdvertiseAddress, "libp2p address with identity (example: /dns4/bootnode.spacemesh.io/tcp/5003)")

cmd.PersistentFlags().BoolVar(&cfg.P2P.Bootnode, "p2p-bootnode", cfg.P2P.Bootnode,
"gossipsub and discovery will be running in a mode suitable for bootnode")
cmd.PersistentFlags().BoolVar(&cfg.P2P.DisableLegacyDiscovery, "p2p-disable-legacy-discovery", cfg.P2P.DisableLegacyDiscovery, "custom legacy discovery is disabled")
cmd.PersistentFlags().BoolVar(&cfg.P2P.PrivateNetwork, "p2p-private-network", cfg.P2P.PrivateNetwork, "discovery will work in private mode. mostly useful for testing, don't set in public networks")
/** ======================== TIME Flags ========================== **/

cmd.PersistentFlags().BoolVar(&cfg.TIME.Peersync.Disable, "peersync-disable", cfg.TIME.Peersync.Disable,
Expand Down
56 changes: 36 additions & 20 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,12 @@ require (
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/go-retryablehttp v0.7.4
github.com/hashicorp/golang-lru/v2 v2.0.4
github.com/ipfs/go-ds-leveldb v0.5.0
github.com/ipfs/go-log/v2 v2.5.1
github.com/libp2p/go-libp2p v0.27.5
github.com/libp2p/go-libp2p v0.27.7
github.com/libp2p/go-libp2p-kad-dht v0.24.2
github.com/libp2p/go-libp2p-pubsub v0.9.3
github.com/libp2p/go-libp2p-record v0.2.0
github.com/mitchellh/mapstructure v1.5.0
github.com/multiformats/go-multiaddr v0.10.1
github.com/multiformats/go-varint v0.0.7
Expand All @@ -45,6 +48,7 @@ require (
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.16.0
github.com/stretchr/testify v1.8.4
github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7
github.com/zeebo/blake3 v0.2.3
go.uber.org/atomic v1.11.0
go.uber.org/zap v1.24.0
Expand Down Expand Up @@ -74,7 +78,7 @@ require (
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.1.0 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/elastic/gosigar v0.14.2 // indirect
github.com/emicklei/go-restful/v3 v3.9.0 // indirect
Expand All @@ -83,7 +87,8 @@ require (
github.com/flynn/noise v1.0.0 // indirect
github.com/francoispqt/gojay v1.2.13 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/go-logr/logr v1.2.3 // indirect
github.com/go-logr/logr v1.2.4 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.1 // indirect
Expand All @@ -95,43 +100,48 @@ require (
github.com/golang/snappy v0.0.4 // indirect
github.com/google/gnostic v0.5.7-v3refs // indirect
github.com/google/gopacket v1.1.19 // indirect
github.com/google/pprof v0.0.0-20230405160723-4a4c7d95572b // indirect
github.com/google/pprof v0.0.0-20230602150820-91b7bce49751 // indirect
github.com/google/s2a-go v0.1.4 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
github.com/googleapis/gax-go/v2 v2.11.0 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/huin/goupnp v1.1.0 // indirect
github.com/huin/goupnp v1.2.0 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/ipfs/boxo v0.10.0 // indirect
github.com/ipfs/go-cid v0.4.1 // indirect
github.com/ipfs/go-datastore v0.6.0 // indirect
github.com/ipfs/go-log v1.0.5 // indirect
github.com/ipld/go-ipld-prime v0.20.0 // indirect
github.com/jackpal/go-nat-pmp v1.0.2 // indirect
github.com/jbenet/go-temp-err-catcher v0.1.0 // indirect
github.com/jbenet/goprocess v0.1.4 // indirect
github.com/jessevdk/go-flags v1.5.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.16.4 // indirect
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
github.com/klauspost/compress v1.16.5 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect
github.com/koron/go-ssdp v0.0.4 // indirect
github.com/libp2p/go-buffer-pool v0.1.0 // indirect
github.com/libp2p/go-cidranger v1.1.0 // indirect
github.com/libp2p/go-flow-metrics v0.1.0 // indirect
github.com/libp2p/go-libp2p-asn-util v0.3.0 // indirect
github.com/libp2p/go-libp2p-kbucket v0.6.3 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect
github.com/libp2p/go-nat v0.1.0 // indirect
github.com/libp2p/go-netroute v0.2.1 // indirect
github.com/libp2p/go-reuseport v0.2.0 // indirect
github.com/libp2p/go-reuseport v0.3.0 // indirect
github.com/libp2p/go-yamux/v4 v4.0.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd // indirect
github.com/mattn/go-isatty v0.0.18 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/miekg/dns v1.1.53 // indirect
github.com/miekg/dns v1.1.54 // indirect
github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b // indirect
github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
Expand All @@ -144,26 +154,28 @@ require (
github.com/multiformats/go-multiaddr-dns v0.3.1 // indirect
github.com/multiformats/go-multiaddr-fmt v0.1.0 // indirect
github.com/multiformats/go-multibase v0.2.0 // indirect
github.com/multiformats/go-multicodec v0.8.1 // indirect
github.com/multiformats/go-multihash v0.2.1 // indirect
github.com/multiformats/go-multicodec v0.9.0 // indirect
github.com/multiformats/go-multihash v0.2.3 // indirect
github.com/multiformats/go-multistream v0.4.1 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/nullstyle/go-xdr v0.0.0-20180726165426-f4c839f75077 // indirect
github.com/nxadm/tail v1.4.8 // indirect
github.com/onsi/ginkgo/v2 v2.9.2 // indirect
github.com/onsi/ginkgo/v2 v2.9.7 // indirect
github.com/opencontainers/runtime-spec v1.0.2 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect
github.com/pelletier/go-toml/v2 v2.0.8 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/polydawn/refmt v0.89.0 // indirect
github.com/prometheus/client_model v0.4.0 // indirect
github.com/prometheus/procfs v0.10.1 // indirect
github.com/pyroscope-io/dotnetdiag v1.2.1 // indirect
github.com/quic-go/qpack v0.4.0 // indirect
github.com/quic-go/qtls-go1-19 v0.3.2 // indirect
github.com/quic-go/qtls-go1-20 v0.2.2 // indirect
github.com/quic-go/quic-go v0.33.0 // indirect
github.com/quic-go/webtransport-go v0.5.2 // indirect
github.com/quic-go/webtransport-go v0.5.3 // indirect
github.com/raulk/go-watchdog v1.3.0 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/shirou/gopsutil v3.21.11+incompatible // indirect
Expand All @@ -172,13 +184,16 @@ require (
github.com/spf13/cast v1.5.1 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/subosito/gotenv v1.4.2 // indirect
github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 // indirect
github.com/tklauser/go-sysconf v0.3.10 // indirect
github.com/tklauser/numcpus v0.5.0 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 // indirect
github.com/yusufpapurcu/wmi v1.2.2 // indirect
go.opencensus.io v0.24.0 // indirect
go.uber.org/dig v1.16.1 // indirect
go.opentelemetry.io/otel v1.16.0 // indirect
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/otel/trace v1.16.0 // indirect
go.uber.org/dig v1.17.0 // indirect
go.uber.org/fx v1.19.2 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/crypto v0.11.0 // indirect
Expand All @@ -189,9 +204,10 @@ require (
golang.org/x/term v0.10.0 // indirect
golang.org/x/text v0.11.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/tools v0.7.0 // indirect
golang.org/x/tools v0.9.1 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
gonum.org/v1/gonum v0.13.0 // indirect
google.golang.org/api v0.126.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230706204954-ccb25ca9f130 // indirect
Expand All @@ -206,7 +222,7 @@ require (
k8s.io/klog/v2 v2.90.1 // indirect
k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f // indirect
k8s.io/utils v0.0.0-20230209194617-a36077c30491 // indirect
lukechampine.com/blake3 v1.1.7 // indirect
lukechampine.com/blake3 v1.2.1 // indirect
nhooyr.io/websocket v1.8.7 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
Expand Down

0 comments on commit 516214f

Please sign in to comment.