Added randomness to lookup_htbl. #4839
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This makes it harder to extract a user's location via cache-timing, but not impossible. The hash function is extended to 2 rounds and is turned into a hash rather than a pseudo-random permutation. This modified 2-round hash passes the dieharder tests when feeding it with a counter, and outputting the lwoer 32 bits of every hash. This is more than enough proof that the hash function output is suitable for a hash table.
This no longer passes dieharder tests, but all we really want is a nice random-ish distribution that does not have more collisions in the hash table than expected. We now measure this in a collision test, and one round is good enough. This is a both faster and a better distribution than the ahash::RandomState algorithm.
andrisaar
approved these changes
Feb 26, 2024
jblebrun
pushed a commit
to jblebrun/oak
that referenced
this pull request
Feb 28, 2024
* Added randomness to lookup_htbl. This makes it harder to extract a user's location via cache-timing, but not impossible. The hash function is extended to 2 rounds and is turned into a hash rather than a pseudo-random permutation. This modified 2-round hash passes the dieharder tests when feeding it with a counter, and outputting the lwoer 32 bits of every hash. This is more than enough proof that the hash function output is suitable for a hash table. * Updated Cargo.lock file. * Updated Cargo.lock file. * Just reformatted 1 file. * Reduced hashing back down to 1 round. This no longer passes dieharder tests, but all we really want is a nice random-ish distribution that does not have more collisions in the hash table than expected. We now measure this in a collision test, and one round is good enough. This is a both faster and a better distribution than the ahash::RandomState algorithm. * Blocked clippy from trying to make an iterator to traverse the hash table.
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This makes it harder to extract a user's location via cache-timing, but not impossible.
The new hash function distributes keys very evenly in tests. In contrast, the ahash crate produces poor hashes, resulting in more hash collisions. I've submitted Issue #210 upstream to ahash about this.
The average number of collisions in a 2B entry swiss table with 50% load factor was 3.50 for ahash. For the custom hasher, it is 2.54, which is identical to what we see using true random numbers as hash values. The custom hash function is also about 20% faster than ahash on my laptop.