Frequent errors when clicking on crate Dependents link

[atouchet]

Current Behavior

I am experiencing frequent errors when trying to load the Dependents link from a crate page. This issue has been happening for a while now.

Expected Behavior

I expected the Dependents page to load as it is supposed to.

Steps To Reproduce

I have not found a way to consistently reproduce the issue 100% of the time but this is an example of the issue happening:

1. Go to a crate page (ex. https://crates.io/crates/serde)
2. Click "Dependents" (ex. https://crates.io/crates/serde/reverse_dependencies)
3. The page does not finish loading and returns an error (ex. "Could not load reverse dependencies for the "serde" crate: Internal Server Error").

Environment

• Browser: Firefox 120.0
• OS: Windows 10

Anything else?

Error image:

[Turbo87]

unfortunately this is caused by a performance issue with the corresponding database query. on less depended-upon crates it works fine but for cases like werde it takes a long time to respond. some attempts have been made to improve the situation, but it's still very slow unfortunately.

[qarmin]

Serde have 33840 reverse dependencies, so I think that this is expected.

But I have sometimes problem with printpdf, which have only 20 reverse dependencies, which is more strange - https://crates.io/crates/printpdf/reverse_dependencies

[osa1]

I've encountered the same issue in https://crates.io/crates/salsa/reverse_dependencies.

This package only has 34 reverse dependencies.

Refreshing the page a few times worked.

[CAD97]

Mainly just recording/calling out that it's reporting 500 internal error, and a change to report a timeout instead (if that's indeed what's ocurring) would be nice and less work than actually addressing the issue. (Turbo admitted overlooking that on Zulip.)

The fix would probably be a caching/invalidation layer around frequent/expensive rdep queries. Which certainly isn't trivial to be fair, but also probably any query that doesn't need to reflect registry updates immediately to be useful will likely end up wanting such a layer as the registry grows.

Errors even with a low count of rdeps don't seem too strange, since if it's a performance issue, load can probably impact adjacent queries' performance/success as well.