-
-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: sparse: Speed up _add_sparse
for DIA format
#20722
Conversation
I ran the
In these benchmarks the A matrix is in DIA format, and B is in CSR format. So the AB, BA, and BB tests aren't exercising the updated code path, which is why they don't show up in the timing diffs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code changes here look good!
[I especially like _invert_index
which I've done for lists and dicts, but not seen before for numpy.]
The bench.py
file checks all-overlap and no-overlap, so I tried some-overlap. Most of these cases were also faster than the main branch. But in one case , the new code is slightly slower than the main branch. I don't believe it is significant enough to keep the old methods. But it might be worth trying the suggestion to check both self
and other
for whether they include all new offsets.
The above makes B
a 3-diagonal poisson1d_in2d
constructor with offsets [-1, 0, 1]. Thus they are a subset of the offsets for the poisson2d
A
.
Most of the variants I tried end up looking like the following, with the new code faster than the main branch. These have B being a poisson2d with offsets [-N, -2, 0, 2, N]. And the same A and B as the some-overlap case above, only B+A instead of A+B. That is what made me think of symmetrizing the new code -- but the main branch code is not symmetric anyway -- B+A is more than 50% different from A+B sometimes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me!
_add_sparse
for DIA format
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a nice improvement! Thanks @perimosocordiae, and thanks for the review @dschult
drafted the release note |
This PR rewrites the
_add_sparse
logic for DIA-format sparse arrays/matrices to avoid expensive allocations and Python looping.My local benchmarks show that this is faster, in the easy case where the sparsity pattern doesn't change, but also in the more complex case where the sets of diagonal offsets are non-overlapping.
Generated via:
bench.py