[Merged by Bors] - prune old proposals after block is generated #4993
Conversation
countvonzero
requested review from
dshulyak,
fasmat and
poszu
as code owners
Codecov Report
@@ Coverage Diff @@
## develop #4993 +/- ##
=========================================
- Coverage 77.1% 77.0% -0.1%
=========================================
Files 254 254
Lines 30281 30295 +14
=========================================
- Hits 23367 23357 -10
- Misses 5399 5419 +20
- Partials 1515 1519 +4
|
countvonzero
force-pushed
the
prune-prune-prune
branch
from
a21e500
to
df99f0f
Compare
| @@ -167,6 +167,12 @@ func (g *Generator) run() error { | |||
| if len(g.optimisticOutput) > 0 { | |||
| g.processOptimisticLayers(maxLayer) | |||
| } | |||
| if err := proposals.Delete(g.cdb, out.Layer); err != nil { | |||
There was a problem hiding this comment.
what about old proposals and transactions link? will it conflict with sync if someone will request proposals a bit later?
i thought that it may be better to prune old proposals & certs every 30m or so in separate goroutine, and run autovacuum there as well but every day instead
There was a problem hiding this comment.
i was thinking this can be shipped ASAP, as proposals is the major pain point and the db will stop growing at least.
later i will do something like you suggested and with vaccum because the timing to prune certificates and proposal_transactions are trickier.
one depends on hdist, and the other depends on mempool txs.
proposal syncing is only done during hare rounds. there it only prune up to layer N-1 at the end of hare for N.
so it's not likely that proposal sync will happen for old layers.
There was a problem hiding this comment.
btw, mempool never cross-referenced with the proposal tables so it's ok that proposals are deleted.
|
merging in case we want to make a release this week. bors merge |
## Motivation prune mesh data ## Changes after block is generated for layer N, prune proposals < N
|
Pull request successfully merged into develop. Build succeeded! The publicly hosted instance of bors-ng is deprecated and will go away soon. If you want to self-host your own instance, instructions are here. If you want to switch to GitHub's built-in merge queue, visit their help page.
|
bors
bot
changed the title
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
Motivation
prune mesh data
Changes
after block is generated for layer N, prune proposals < N