New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Merged by Bors] - prune old proposals after block is generated #4993
Conversation
Codecov Report
@@ Coverage Diff @@
## develop #4993 +/- ##
=========================================
- Coverage 77.1% 77.0% -0.1%
=========================================
Files 254 254
Lines 30281 30295 +14
=========================================
- Hits 23367 23357 -10
- Misses 5399 5419 +20
- Partials 1515 1519 +4
|
a21e500
to
df99f0f
Compare
@@ -167,6 +167,12 @@ func (g *Generator) run() error { | |||
if len(g.optimisticOutput) > 0 { | |||
g.processOptimisticLayers(maxLayer) | |||
} | |||
if err := proposals.Delete(g.cdb, out.Layer); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what about old proposals and transactions link? will it conflict with sync if someone will request proposals a bit later?
i thought that it may be better to prune old proposals & certs every 30m or so in separate goroutine, and run autovacuum there as well but every day instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i was thinking this can be shipped ASAP, as proposals is the major pain point and the db will stop growing at least.
later i will do something like you suggested and with vaccum because the timing to prune certificates and proposal_transactions are trickier.
one depends on hdist, and the other depends on mempool txs.
proposal syncing is only done during hare rounds. there it only prune up to layer N-1 at the end of hare for N.
so it's not likely that proposal sync will happen for old layers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw, mempool never cross-referenced with the proposal tables so it's ok that proposals are deleted.
merging in case we want to make a release this week. bors merge |
## Motivation prune mesh data ## Changes after block is generated for layer N, prune proposals < N
Pull request successfully merged into develop. Build succeeded! The publicly hosted instance of bors-ng is deprecated and will go away soon. If you want to self-host your own instance, instructions are here. If you want to switch to GitHub's built-in merge queue, visit their help page.
|
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
## Motivation there are a lot of old proposals floating in the gossip network. deleting old proposals in #4993 forced the node to process/save these old proposals repeatedly ## Changes - refuse proposals older than current layer - add metrics/log for time it takes to delete proposals
Motivation
prune mesh data
Changes
after block is generated for layer N, prune proposals < N