Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a FS flag to detect and correct corruption #12408

Closed
wants to merge 3 commits into from

Conversation

anand1976
Copy link
Contributor

Add a flag in IOOptions to request the file system to make best efforts to detect data corruption and reconstruct the data if possible. This will be used by RocksDB to retry a read if the previous read returns corrupt data (checksum mismatch). Add a new op to FSSupportedOps that, if supported, will trigger this behavior in RocksDB.

@anand1976 anand1976 added the WIP Work in progress label Mar 6, 2024
@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@anand1976 anand1976 changed the title Add a FS flag to detect and correct corruption [RFC] Add a FS flag to detect and correct corruption Mar 6, 2024
@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@anand1976 has updated the pull request. You must reimport the pull request before landing.

@anand1976
Copy link
Contributor Author

Changed the name of the new flag to make its purpose clearer.

@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@anand1976 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@anand1976 anand1976 changed the title [RFC] Add a FS flag to detect and correct corruption Add a FS flag to detect and correct corruption Mar 8, 2024
@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@anand1976 merged this pull request in 179afd5.

kwadhwa18 pushed a commit to kwadhwa18/rocksdb that referenced this pull request Mar 18, 2024
Summary:
Add a flag in `IOOptions` to request the file system to make best efforts to detect data corruption and reconstruct the data if possible. This will be used by RocksDB to retry a read if the previous read returns corrupt data (checksum mismatch). Add a new op to `FSSupportedOps` that, if supported, will trigger this behavior in RocksDB.

Pull Request resolved: facebook#12408

Reviewed By: akankshamahajan15

Differential Revision: D54564218

Pulled By: anand1976

fbshipit-source-id: bc401dcd22a7d320bf63b5183c41714acdce39f5
anand1976 added a commit that referenced this pull request Mar 18, 2024
Summary:
Add a flag in `IOOptions` to request the file system to make best efforts to detect data corruption and reconstruct the data if possible. This will be used by RocksDB to retry a read if the previous read returns corrupt data (checksum mismatch). Add a new op to `FSSupportedOps` that, if supported, will trigger this behavior in RocksDB.

Pull Request resolved: #12408

Reviewed By: akankshamahajan15

Differential Revision: D54564218

Pulled By: anand1976

fbshipit-source-id: bc401dcd22a7d320bf63b5183c41714acdce39f5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants