ethdb/pebble: sync pebble writes #27522
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Some assorted charts. This PR is yellow, master green
Somehow this PR managed to do faster ingress No loss in disk write speed
All in all, this PR was about 1 hour faster syncing
So, I don't believe this PR makes things faster, but the important point is that at least there's no evidence of this change incurring any massive slowdown. |
|
More info in this ticket cockroachdb/pebble#2624 |
spencer-tb
pushed a commit
to spencer-tb/go-ethereum
that referenced
this pull request
Jul 7, 2023
This is likely the culprit behind several data corruption issues, e.g. where data has been written to the freezer, but the deletion from pebble does not go through due to process crash.
MoonShiesty
pushed a commit
to MoonShiesty/go-ethereum
that referenced
this pull request
Aug 30, 2023
This is likely the culprit behind several data corruption issues, e.g. where data has been written to the freezer, but the deletion from pebble does not go through due to process crash.
devopsbo3
pushed a commit
to HorizenOfficial/go-ethereum
that referenced
this pull request
Nov 10, 2023
This is likely the culprit behind several data corruption issues, e.g. where data has been written to the freezer, but the deletion from pebble does not go through due to process crash.
devopsbo3
added a commit
to HorizenOfficial/go-ethereum
that referenced
this pull request
Nov 10, 2023
This reverts commit c42afc4.
devopsbo3
added a commit
to HorizenOfficial/go-ethereum
that referenced
this pull request
Nov 10, 2023
This reverts commit c42afc4.
devopsbo3
added a commit
to HorizenOfficial/go-ethereum
that referenced
this pull request
Nov 10, 2023
This reverts commit c42afc4.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is likely the culprit behind several data corruption issues, e.g. where data has been written to the freezer, but the deletion from leveldb does not go through due to process crash.
Pebble has pretty strange sync options:
sync=true: writes to the WAL are flushed to OS, and fsync:ed to disk.sync=false: writes to the WAL are in-mem, and flushed to OS at some future time.Out of durability guarantees,
With leveldb, we could choose to land on
2: survive process crash, but potentially corrupt data on os crash. With pebble, it's either option1or3. Current master uses3.This PR uses option
1.Benchmarking, this PR is on
07, master on08: