Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve entropy recalculations #1879

Merged
merged 5 commits into from
Nov 5, 2024

Conversation

jameshadfield
Copy link
Member

Improves the general speed of Auspice when the Entropy panel is not displayed (either toggled off or off-screen). The downside is a small amount of time where there's a blank entropy panel when it scrolls back into view after a state change which requires the entropy data to be recalculated. (The win is that we skip these recalculations so everything else is faster.)

Eyes on the behaviour changes in this PR would be very welcome!

There's more we can do here, but I'll leave the following for subsequent PRs:

  • I'd like to investigate shifting the entire recalculation off to a worker thread. The mutation data can be transferred once (it never changes) so I think the IO should be fast.
  • I suspect the slowness is the myriad string slice operations - how much faster would it be to pre-process these and store as arrays. (And how much more memory would this take?)
  • The debouncing could be smarter, perhaps utilising the requestIdleCallback API

Related to #1878

@nextstrain-bot nextstrain-bot temporarily deployed to auspice-james-improve-e-3gx9pf October 28, 2024 04:06 Inactive
@jameshadfield jameshadfield force-pushed the james/improve-entropy-recalculations branch from 2c81abc to 4588d6c Compare October 28, 2024 19:08
@jameshadfield jameshadfield temporarily deployed to auspice-james-improve-e-3gx9pf October 28, 2024 19:09 Inactive
@jameshadfield
Copy link
Member Author

Updated to fix a bug when viewing narratives and also fix linting errors

Copy link
Contributor

@joverlee521 joverlee521 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I played around with the groups/blab/ncov-orf8ko/WA/20k dataset and definitely see improvements in responsiveness!

I only left non-blocking comments. In general, I find the split of management of entropy data between the Redux actions and Entropy component a bit weird. It feels like the Redux actions should trigger that entropy data updates appropriately and the Entropy component should just remain a consumer of that data.

@jameshadfield jameshadfield force-pushed the james/improve-entropy-recalculations branch 2 times, most recently from 65a982c to 446d5c9 Compare November 5, 2024 00:24
jameshadfield added a commit that referenced this pull request Nov 5, 2024

Unverified

This user has not yet uploaded their public signing key.
Prompted by <#1879 (comment)>

Verified

This commit was signed with the committer’s verified signature. The key has expired.
davidkarlsen David J. M. Karlsen
by not recalculating data on the leading edge of the debounced
recalculation action. For big trees (which take >500ms of time to
redraw) the main thread is still blocked for roughly the same amount of
time, but the tree is redrawn faster. For small trees which redraw
quicker than that the entropy doesn't update until the debounce 500ms
timeout is reached, resulting in slightly odd behaviour.

(Reducing this timout also results in less-than-ideal behaviour as
(e.g.) dragging the date range of a tree results in interruptions while
the entropy calcs run which is worse IMO.)

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Improves the performance of most interactions in Auspice when the
`<Entropy>` panel's not displayed by not constantly updating the entropy
calculations. The downside is when the panel's toggled on things are a
little slower as we recalculate at that point.

Note that the Entropy panel should never be rendered with invalid /
stale / uncomputed data. Any action which results in the panel being
shown is responsible for ensuring the underlying data is updated as
appropriate.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Use an intersection observer to detect when the entropy panel is visible
on the screen (viewport). When it's offscreen we don't update entropy
data (an expensive calculation). When it comes back onscreen we
recalculate the entropy data if it has become stale.

This results in slightly strange behaviour when the entropy panel will
be shown with no bars and they'll be drawn after a slight delay (while
the data's recalculated). The wins are much improved performance when
the entropy panel is not on screen (which is common).

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Prompted by <#1879 (comment)>
@jameshadfield jameshadfield force-pushed the james/improve-entropy-recalculations branch from 3799a98 to 5543a86 Compare November 5, 2024 20:54
@jameshadfield
Copy link
Member Author

Rebased onto master which now includes e3cfb27 as part of #1883. Merging.

@jameshadfield jameshadfield merged commit def2ffb into master Nov 5, 2024
26 checks passed
@jameshadfield jameshadfield deleted the james/improve-entropy-recalculations branch November 5, 2024 21:34
jameshadfield added a commit that referenced this pull request Nov 12, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
The bug arose because window-resizes trigger a complete re-render of
the entropy panel's SVG and this didn't correctly handle an edge case
where the entropy data was stale (a concept introduced in #1879).

For specific steps to reproduce see <#1895 (comment)>

Closes #1895
jameshadfield added a commit that referenced this pull request Nov 17, 2024
Changing between "Entropy - Events" results in two dispatches
(`ENTROPY_COUNTS_TOGGLE` and `ENTROPY_DATA`) however the entropy code
would ignore the first update if it didn't contain new entropy data.
This bug has probably existed for the life of the entropy panel however
was masked by those two actions happening back-to-back and being
(always?) bundled together by redux.

Recent work in #1879 (v2.60.0) meant these actions were no longer
back-to-back and thus the bug was revealed.

Closes #1905
jameshadfield added a commit that referenced this pull request Nov 17, 2024
This reverts commit 0c91724.

That commit was the first performance optimisation within #1879. The
idea was to skip the expensive recalculation when initially dragging the
date slider. In other contexts this change was a negative, specifically
when there was only one action (e.g. changing entropy to events) as we
essentially introduced a 500ms delay.

Note that when the panel is off-screen subsequent work in #1879 means
that the recalculation is skipped anyway, so reverting this commit only
affects performance when the panel is on screen and we're dragging the
date slider. To remedy this we should add a code path which specifically
skips the leading edge calculation when dragging the date slider.

Essentially closes #1899 - that issue was partly the recalculation
on-demand which remains, but greatly exacerbated by the unnecessary
500ms delay.
@jameshadfield jameshadfield mentioned this pull request Nov 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants