Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend Checkpoints with new sizes and lookup mechanisms #3589

Merged
merged 53 commits into from
Aug 30, 2022
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
4176852
move mock
Amxx Jul 28, 2022
63edb74
Expand Checkpoint library
Amxx Jul 28, 2022
da0ea31
rename upperLookupExpEnd → upperLookupRecent
Amxx Jul 28, 2022
ed81388
sanity check when pushing checkpoints
Amxx Jul 28, 2022
b91832d
Skip redundant optimisation in VotesQuorumFraction
Amxx Jul 29, 2022
d2a829c
Merge branch 'master' into feature/checkpoint
Amxx Jul 29, 2022
2a6adb3
reset previous history lookup algorithm and reenable quorum checks
Amxx Jul 29, 2022
28f6e6f
Use unsafeAccess to save gas
Amxx Jul 29, 2022
b7e2f64
Merge branch 'master' into feature/checkpoint
Amxx Jul 29, 2022
616c51b
Regenerate contracts
Amxx Jul 29, 2022
3fb185b
Add changelog entry
Amxx Jul 29, 2022
ce0c7a6
use StorageSlot for unsafeAccess returns
Amxx Jul 29, 2022
33741e7
minor rewrite for readability
Amxx Jul 29, 2022
8425e4a
fix typo
Amxx Jul 29, 2022
74fef95
testing unsafeAccess
Amxx Jul 29, 2022
df3a9b1
fix lint
Amxx Jul 29, 2022
e182730
Update CHANGELOG.md
Amxx Jul 29, 2022
9b1f559
Merge branch 'master' into feature/checkpoint
Amxx Aug 14, 2022
768a681
Apply suggestions from code review
Amxx Aug 17, 2022
8d9b177
Update Arrays.sol
Amxx Aug 17, 2022
8dc9d4f
Run prettier as part of the procedural generation
Amxx Aug 17, 2022
4e6aa05
Reactor checkpoints to keep legacy types
Amxx Aug 17, 2022
fffde15
add a comment about files being procedurally generated
Amxx Aug 17, 2022
ac5466a
add extensions for the comments
Amxx Aug 17, 2022
a9a69cf
lint
Amxx Aug 17, 2022
4654237
shorter message
Amxx Aug 17, 2022
b486b6b
Merge branch 'master' into feature/checkpoint
Amxx Aug 25, 2022
2848c3a
regenerate & codespell
Amxx Aug 25, 2022
4c6c65e
update generation script path resolution
Amxx Aug 25, 2022
068513c
address comment from PR
Amxx Aug 26, 2022
bf12a2b
add more neatspec
Amxx Aug 26, 2022
b1d5a09
add more neatspec
Amxx Aug 26, 2022
543d516
Apply suggestions from code review
Amxx Aug 26, 2022
634e279
regenerate
Amxx Aug 26, 2022
17e1ec7
don't insert newline when no previous version is recorded
Amxx Aug 26, 2022
d1af38d
improve checkpoint opts generation
Amxx Aug 26, 2022
253e12a
codespell
Amxx Aug 26, 2022
1ccff75
test coverage
Amxx Aug 26, 2022
4d1d3da
grammar
frangio Aug 29, 2022
2d03e54
improve tests
Amxx Aug 30, 2022
69ac69e
Merge remote-tracking branch 'amxx/feature/checkpoint' into feature/c…
Amxx Aug 30, 2022
5661e71
Update test/utils/Checkpoints.test.js
Amxx Aug 30, 2022
6cd42ce
Apply suggestions from code review
Amxx Aug 30, 2022
9c84034
fix generation
Amxx Aug 30, 2022
b878b23
Update scripts/generate/templates/Checkpoints.js
Amxx Aug 30, 2022
c832e53
Update scripts/generate/templates/Checkpoints.js
Amxx Aug 30, 2022
a450545
Update scripts/generate/templates/Checkpoints.js
Amxx Aug 30, 2022
96122a3
Update scripts/generate/templates/Checkpoints.js
Amxx Aug 30, 2022
c891f58
fix generation
Amxx Aug 30, 2022
47ed78e
add History.getAtRecentBlock & tests
Amxx Aug 30, 2022
533ccef
wording
Amxx Aug 30, 2022
6caea6a
Merge branch 'master' into feature/checkpoint
Amxx Aug 30, 2022
22491c0
wrap docs
frangio Aug 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion contracts/mocks/CheckpointsMock.sol
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/CheckpointsMock.js`.
// This file was procedurally generated from scripts/generate/templates/CheckpointsMock.js.

pragma solidity ^0.8.0;

Expand Down
2 changes: 1 addition & 1 deletion contracts/mocks/EnumerableMapMock.sol
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/EnumerableMapMock.js`.
// This file was procedurally generated from scripts/generate/templates/EnumerableMapMock.js.

pragma solidity ^0.8.0;

Expand Down
2 changes: 1 addition & 1 deletion contracts/mocks/EnumerableSetMock.sol
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/EnumerableSetMock.js`.
// This file was procedurally generated from scripts/generate/templates/EnumerableSetMock.js.

pragma solidity ^0.8.0;

Expand Down
2 changes: 1 addition & 1 deletion contracts/mocks/SafeCastMock.sol
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/SafeCastMock.js`.
// This file was procedurally generated from scripts/generate/templates/SafeCastMock.js.

pragma solidity ^0.8.0;

Expand Down
6 changes: 3 additions & 3 deletions contracts/utils/Arrays.sol
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ library Arrays {
/**
* @dev Access an array in an "unsafe" way. Skips solidity "index-out-of-range" check.
*
* WARNING: only use if you are certain pos is lower than the array length.
* WARNING: Only use if you are certain `pos` is lower than the array length.
*/
function unsafeAccess(address[] storage arr, uint256 pos) internal pure returns (StorageSlot.AddressSlot storage) {
bytes32 slot;
Expand All @@ -67,7 +67,7 @@ library Arrays {
/**
* @dev Access an array in an "unsafe" way. Skips solidity "index-out-of-range" check.
*
* WARNING: only use if you are certain pos is lower than the array length.
* WARNING: Only use if you are certain `pos` is lower than the array length.
*/
function unsafeAccess(bytes32[] storage arr, uint256 pos) internal pure returns (StorageSlot.Bytes32Slot storage) {
bytes32 slot;
Expand All @@ -82,7 +82,7 @@ library Arrays {
/**
* @dev Access an array in an "unsafe" way. Skips solidity "index-out-of-range" check.
*
* WARNING: only use if you are certain pos is lower than the array length.
* WARNING: Only use if you are certain `pos` is lower than the array length.
*/
function unsafeAccess(uint256[] storage arr, uint256 pos) internal pure returns (StorageSlot.Uint256Slot storage) {
bytes32 slot;
Expand Down
93 changes: 86 additions & 7 deletions contracts/utils/Checkpoints.sol
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/Checkpoints.js`.
// OpenZeppelin Contracts (last updated v4.5.0) (utils/Checkpoints.sol)
// This file was procedurally generated from scripts/generate/templates/Checkpoints.js.

pragma solidity ^0.8.0;

Expand Down Expand Up @@ -69,6 +69,9 @@ library Checkpoints {
return push(self, op(latest(self), delta));
}

/**
* @dev Pushes a (`key`, `value`) pair into a ordered list of checkpoints, either by inserting a new checkpoint, or by updating the last one.
*/
function _insert(
Checkpoint[] storage self,
uint32 key,
Expand All @@ -77,10 +80,10 @@ library Checkpoints {
uint256 pos = self.length;

if (pos > 0) {
// Use of memory is important here.
// Copying to memory is important here.
Checkpoint memory last = _unsafeAccess(self, pos - 1);

// Checkpoints keys must be increassing.
// Checkpoints keys must be increasing.
require(last._blockNumber <= key, "Checkpoint: invalid key");

// Update or push new checkpoint
Expand All @@ -96,6 +99,12 @@ library Checkpoints {
}
}

/**
* @dev Return the index of the last checkpoint for which `key` >= checkpoint.key, i.e. the most recent checkpoint which key is lower or equal than the search key.
* `low` and `high` define a section where to do the search, with an `low` inclusive and `high` exclusive.
*
* WARNING: `high` should not be greater than the array's length.
*/
function _upperBinaryLookup(
Amxx marked this conversation as resolved.
Show resolved Hide resolved
Checkpoint[] storage self,
uint32 key,
Expand All @@ -113,6 +122,12 @@ library Checkpoints {
return high;
}

/**
* @dev Return the index of the first checkpoint for which `key` <= checkpoint.key, i.e. the oldest checkpoint which key is higher or equal than the search key.
* `low` and `high` define a section where to do the search, with an `low` inclusive and `high` exclusive.
*
* WARNING: `high` should not be greater than the array's length.
*/
function _lowerBinaryLookup(
Checkpoint[] storage self,
uint32 key,
Expand Down Expand Up @@ -146,11 +161,19 @@ library Checkpoints {
uint224 _value;
}

/**
* @dev Returns the value in the latest checkpoint, or zero if there are no checkpoints.
*/
function latest(Trace224 storage self) internal view returns (uint224) {
uint256 pos = self._checkpoints.length;
return pos == 0 ? 0 : _unsafeAccess(self._checkpoints, pos - 1)._value;
}

/**
* @dev Pushes a (`key`, `value`) pair into a History so that it is stored as the checkpoint.
*
* Returns previous value and new value.
*/
function push(
Trace224 storage self,
uint32 key,
Expand All @@ -159,18 +182,27 @@ library Checkpoints {
return _insert(self._checkpoints, key, value);
}

/**
* @dev Return the value most recent checkpoint which key is lower or equal than the search key.
*/
function lowerLookup(Trace224 storage self, uint32 key) internal view returns (uint224) {
uint256 length = self._checkpoints.length;
uint256 pos = _lowerBinaryLookup(self._checkpoints, key, 0, length);
return pos == length ? 0 : _unsafeAccess(self._checkpoints, pos)._value;
}

/**
* @dev Return the value oldest checkpoint which key is higher or equal than the search key.
*/
function upperLookup(Trace224 storage self, uint32 key) internal view returns (uint224) {
uint256 length = self._checkpoints.length;
uint256 pos = _upperBinaryLookup(self._checkpoints, key, 0, length);
return pos == 0 ? 0 : _unsafeAccess(self._checkpoints, pos - 1)._value;
}

/**
* @dev Return the value oldest checkpoint which key is higher or equal than the search key (similarly to {upperLookupRecent}) doing an exponential lookup first to optimize for recent checkpoints.
*/
function upperLookupRecent(Trace224 storage self, uint32 key) internal view returns (uint224) {
uint256 length = self._checkpoints.length;
uint256 offset = 1;
Expand All @@ -186,6 +218,9 @@ library Checkpoints {
return pos == 0 ? 0 : _unsafeAccess(self._checkpoints, pos - 1)._value;
}

/**
* @dev Pushes a (`key`, `value`) pair into a ordered list of checkpoints, either by inserting a new checkpoint, or by updating the last one.
*/
function _insert(
Checkpoint224[] storage self,
uint32 key,
Expand All @@ -194,10 +229,10 @@ library Checkpoints {
uint256 pos = self.length;

if (pos > 0) {
// Use of memory is important here.
// Copying to memory is important here.
Checkpoint224 memory last = _unsafeAccess(self, pos - 1);

// Checkpoints keys must be increassing.
// Checkpoints keys must be increasing.
require(last._key <= key, "Checkpoint: invalid key");

// Update or push new checkpoint
Expand All @@ -213,6 +248,12 @@ library Checkpoints {
}
}

/**
* @dev Return the index of the last checkpoint for which `key` >= checkpoint.key, i.e. the most recent checkpoint which key is lower or equal than the search key.
* `low` and `high` define a section where to do the search, with an `low` inclusive and `high` exclusive.
*
* WARNING: `high` should not be greater than the array's length.
*/
function _upperBinaryLookup(
Checkpoint224[] storage self,
uint32 key,
Expand All @@ -230,6 +271,12 @@ library Checkpoints {
return high;
}

/**
* @dev Return the index of the first checkpoint for which `key` <= checkpoint.key, i.e. the oldest checkpoint which key is higher or equal than the search key.
* `low` and `high` define a section where to do the search, with an `low` inclusive and `high` exclusive.
*
* WARNING: `high` should not be greater than the array's length.
*/
function _lowerBinaryLookup(
Checkpoint224[] storage self,
uint32 key,
Expand Down Expand Up @@ -267,11 +314,19 @@ library Checkpoints {
uint160 _value;
}

/**
* @dev Returns the value in the latest checkpoint, or zero if there are no checkpoints.
*/
function latest(Trace160 storage self) internal view returns (uint160) {
Amxx marked this conversation as resolved.
Show resolved Hide resolved
uint256 pos = self._checkpoints.length;
return pos == 0 ? 0 : _unsafeAccess(self._checkpoints, pos - 1)._value;
}

/**
* @dev Pushes a (`key`, `value`) pair into a History so that it is stored as the checkpoint.
Amxx marked this conversation as resolved.
Show resolved Hide resolved
*
* Returns previous value and new value.
*/
function push(
Trace160 storage self,
uint96 key,
Expand All @@ -280,18 +335,27 @@ library Checkpoints {
return _insert(self._checkpoints, key, value);
}

/**
* @dev Return the value most recent checkpoint which key is lower or equal than the search key.
*/
function lowerLookup(Trace160 storage self, uint96 key) internal view returns (uint160) {
uint256 length = self._checkpoints.length;
uint256 pos = _lowerBinaryLookup(self._checkpoints, key, 0, length);
return pos == length ? 0 : _unsafeAccess(self._checkpoints, pos)._value;
}

/**
* @dev Return the value oldest checkpoint which key is higher or equal than the search key.
*/
function upperLookup(Trace160 storage self, uint96 key) internal view returns (uint160) {
uint256 length = self._checkpoints.length;
uint256 pos = _upperBinaryLookup(self._checkpoints, key, 0, length);
return pos == 0 ? 0 : _unsafeAccess(self._checkpoints, pos - 1)._value;
}

/**
* @dev Return the value oldest checkpoint which key is higher or equal than the search key (similarly to {upperLookupRecent}) doing an exponential lookup first to optimize for recent checkpoints.
*/
function upperLookupRecent(Trace160 storage self, uint96 key) internal view returns (uint160) {
uint256 length = self._checkpoints.length;
uint256 offset = 1;
Expand All @@ -307,6 +371,9 @@ library Checkpoints {
return pos == 0 ? 0 : _unsafeAccess(self._checkpoints, pos - 1)._value;
}

/**
* @dev Pushes a (`key`, `value`) pair into a ordered list of checkpoints, either by inserting a new checkpoint, or by updating the last one.
*/
function _insert(
Checkpoint160[] storage self,
uint96 key,
Expand All @@ -315,10 +382,10 @@ library Checkpoints {
uint256 pos = self.length;

if (pos > 0) {
// Use of memory is important here.
// Copying to memory is important here.
Checkpoint160 memory last = _unsafeAccess(self, pos - 1);

// Checkpoints keys must be increassing.
// Checkpoints keys must be increasing.
require(last._key <= key, "Checkpoint: invalid key");

// Update or push new checkpoint
Expand All @@ -334,6 +401,12 @@ library Checkpoints {
}
}

/**
* @dev Return the index of the last checkpoint for which `key` >= checkpoint.key, i.e. the most recent checkpoint which key is lower or equal than the search key.
* `low` and `high` define a section where to do the search, with an `low` inclusive and `high` exclusive.
*
* WARNING: `high` should not be greater than the array's length.
*/
function _upperBinaryLookup(
Checkpoint160[] storage self,
uint96 key,
Expand All @@ -351,6 +424,12 @@ library Checkpoints {
return high;
}

/**
* @dev Return the index of the first checkpoint for which `key` <= checkpoint.key, i.e. the oldest checkpoint which key is higher or equal than the search key.
* `low` and `high` define a section where to do the search, with an `low` inclusive and `high` exclusive.
*
* WARNING: `high` should not be greater than the array's length.
*/
function _lowerBinaryLookup(
Checkpoint160[] storage self,
uint96 key,
Expand Down
2 changes: 1 addition & 1 deletion contracts/utils/math/SafeCast.sol
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/SafeCast.js`.
// OpenZeppelin Contracts (last updated v4.7.0) (utils/math/SafeCast.sol)
// This file was procedurally generated from scripts/generate/templates/SafeCast.js.

pragma solidity ^0.8.0;

Expand Down
2 changes: 1 addition & 1 deletion contracts/utils/structs/EnumerableMap.sol
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/EnumerableMap.js`.
// OpenZeppelin Contracts (last updated v4.7.0) (utils/structs/EnumerableMap.sol)
// This file was procedurally generated from scripts/generate/templates/EnumerableMap.js.

pragma solidity ^0.8.0;

Expand Down
2 changes: 1 addition & 1 deletion contracts/utils/structs/EnumerableSet.sol
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated by `scripts/generate/run.js` from the template file `scripts/generate/templates/EnumerableSet.js`.
// OpenZeppelin Contracts (last updated v4.7.0) (utils/structs/EnumerableSet.sol)
// This file was procedurally generated from scripts/generate/templates/EnumerableSet.js.

pragma solidity ^0.8.0;

Expand Down
7 changes: 4 additions & 3 deletions scripts/generate/run.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,15 @@ for (const [ file, template ] of Object.entries({
'utils/Checkpoints.sol': './templates/Checkpoints.js',
'mocks/CheckpointsMock.sol': './templates/CheckpointsMock.js',
})) {
const script = __filename.replace(path.join(__dirname, '../../'), '');
const script = path.relative(path.join(__dirname, '../..'), __filename);
const input = path.join(path.dirname(script), template);
const output = `./contracts/${file}`;
const version = getVersion(output);
const content = format(
'// SPDX-License-Identifier: MIT',
`// This file was procedurally generated by \`${script}\` from the template file \`${input}\`.`,
(version ? version + ` (${file})\n` : ''),
...(version ? [ version + ` (${file})` ] : []),
`// This file was procedurally generated from ${input}.`,
'',
require(template),
);

Expand Down