tftypes: Reduce AttributePath and Value type compute and memory usage #308
+1,006
−182
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #307
These optimizations were performed by adding benchmark tests against a set of 1000 "simple" objects and viewing the cpu/memory profiles. The original implementations were spending an immense amount of time dealing with memory allocations and garbage collection, so reducing memory allocations was the main target of these optimizations, which in turn, also reduced compute time.
The largest changes were accomplished by removing
(Value).Diff()
from the logic paths which were only testing equality. The new(Value).deepEqual()
implementation started as a duplicate of that logic, removing allValueDiff
allocations. Then, the walking ofValue
needed a methodology to stop the walk immediately to prevent further allocations. The two changes in that regard were introducing astopWalkError
sentinel error type for which callback functions can signal that no remainingValue
are necessary to traverse and updating the internalwalk()
implementation to include a new bool return to fully accomplish the intended behavior.The other changes were smaller optimizations noticed from the profiling, such as avoiding the Go runtime immediately reallocating a larger slice with the
AttributePath
methods, avoiding memory allocations caused by(Type).Is()
usage instead of using type switches, and directly referencingList
,Map
,Object
,Set
, andTuple
value storage instead of allocating an unneeded variable. This logic is heavily used both by this Go module and others, so even these minor optimizations have impact at scale.benchstat
differences between prior implementation and proposed changes: