Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grpc-js: Fix connectivity state change event sequencing #2421

Merged

Conversation

murgatroid99
Copy link
Member

The test in #2419 revealed a couple of bugs related to the order of connectivity state change events:

  • PickFirstLoadBalancer#pickSubchannel informs its parent of the state change before resolving its own internal state, and as a result arbitrary code can run at that point.
  • The calls to the list of listeners in Subchannel#transitionToState can eventually result in API calls that cause additional state changes, causing potential reentrancy in transitionToState which can result in out-of-order state change notifications.

The changes to fix this are:

  • Move the state reporting in PickFirstLoadBalancer#pickSubchannel to the end.
  • Refactor the listener list in Subchannel to a set to simplify interacting with it and to allow removing listeners during the iteration.
  • Ensure all calls to transitionToState are asynchronous with respect to surface API calls, to avoid reentrant transitionToState calls.

I adapted the test from #2419 to test this combination of bugs.

@murgatroid99 murgatroid99 merged commit e94b8c5 into grpc:@grpc/grpc-js@1.8.x Apr 12, 2023
5 checks passed
@@ -227,6 +227,8 @@ export class Subchannel {
}
const previousState = this.connectivityState;
this.connectivityState = newState;
process.nextTick(() => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@murgatroid99
What's the purpose of this call?
It looks like some debugging leftover. It doesn't make anything asynchronous, it just adds an empty function to be called by Event Loop after synchronous part of the code is executed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. That was part of a change that I was trying out before I went in a different direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants