grpc: support channel idleness #6263

easwars · 2023-05-09T00:58:14Z

This PR adds support for channel idleness. Summary of changes:

Added a new component idlenessManager that
- keeps track of RPC activity on the channel, and
- instructs the channel to enter or exit idle mode
ClientConn
- methods to enter and exit idle mode
  - these are invoked by the newly added idlenessManager
  - these take care of shutting down and recreating the name resolver, load balancer, and blocking picker
  - these also set connectivity state appropriately
- call into the idlenessManager from Invoke() and NewStream()
  - these are the RPC entry points
- refactor DialContext a little bit for better code flow
- addTraceEvent helper to emit channelz trace events
Add a WithIdleTimeout dial option to set idle_timeout
- Defaults to 30m if unset (current default used by Java)
- Disables channel idleness if explicitly set to 0
balancer wrapper
- methods to enter and exit idle mode by shutting down and recreating the balancer respectively
- not forwarding calls from the balancer to grpc when the channel is in idle mode
picker wrapper (or blocking picker)
- methods to enter and exit idle mode
- when in idle mode, drops picker updates
resolver wrapper
- methods to enter and exit idle mode by shutting down and recreating the name resolver respectively

RELEASE NOTES:

grpc: support channel idleness using WithIdleTimeout dial option

easwars · 2023-05-09T00:59:12Z

I will be adding e2e tests soon and will push a commit for the same. But the PR is ready to looked at.

easwars · 2023-05-09T22:29:01Z

Looks like there is a race around closing and entering/exiting idle. I hit it with my first e2e test. Will ping the PR when this is ready to be looked at again. Sorry.

dfawley

I don't think I'll be able to finish a pass today, but here are my thoughts on the idle manager so far...

dfawley · 2023-05-12T22:10:10Z

idle.go

+)
+
+// For overriding in unit tests.
+var newTimer = func(d time.Duration, f func()) *time.Timer {


Nit: afterFunc or timeAfterFunc please

dfawley · 2023-05-12T22:14:45Z

idle.go

+
+	enforcer   idlenessEnforcer // Functionality provided by grpc.ClientConn.
+	timeout    int64            // Idle timeout duration nanos stored as an int64.
+	isDisabled bool             // Disabled if idle_timeout is set to 0.


Similar, but how about returning a nil *idlenessManager instead and make the receivers short-circuit via nil? Or don't call from cc if nil. Or make this implement an interface and have a disabledIdlenessManager type that is returned instead. It seems unusual to have a field that indicates the whole struct shouldn't even be used, given that it doesn't change dynamically.

Done. Used the short-circuit via nil option.

idle.go

dfawley · 2023-05-12T22:16:36Z

idle.go

+		i.timer = newTimer(time.Duration(i.timeout), i.handleIdleTimeout)
+		i.isIdle = false


These things seem like they'd belong in exitIdleMode, no?

After the switch to the atomic idleness manager, this is not applicable anymore.

idle.go

…ver wrapper

dfawley · 2023-05-18T22:46:08Z

balancer_conn_wrappers.go

+		// Reset the current balancer name so that we act on the next call to
+		// switchTo by creating a new balancer specified by the new resolver.
+		ccb.curBalancerName = ""


Would it be better to do this when entering idle instead of exiting?

…idle

dfawley · 2023-05-19T22:58:17Z

clientconn.go

+	if cc.dopts.idleTimeout == 0 {
+		cc.idlenessMgr = newDisabledIdlenessManager()
+	} else {
+		cc.idlenessMgr = newAtomicIdlenessManager(cc, cc.dopts.idleTimeout)
+	}


Can we simplify to cc.idlenessMgr = newIdlenessManager(cc, cc.dopts.idleTimeout) and leave the complexity of the implementation up to the implementation?

dfawley · 2023-05-19T22:58:51Z

call.go

 	}
-	return invoke(ctx, method, args, reply, cc, opts...)
+	cc.idlenessMgr.onCallEnd()


Optional: defer instead and leave the same?

dfawley · 2023-05-19T23:04:43Z

test/idleness_test.go

+	sCtx, sCancel := context.WithTimeout(ctx, 3*defaultTestShortIdleTimeout)
+	defer sCancel()
+	go func() {
+		for ; sCtx.Err() == nil; <-time.After(defaultTestShortTimeout) {


Maybe defaultTestShortIdleTimeout/2 for this instead? Otherwise these things that are otherwise not dependent on each other are actually dependent on each other for this test's correctness.

Done. Thanks.

dfawley · 2023-05-19T23:13:58Z

test/idleness_test.go

+	// restarted when exiting idle, it will push the same address to grpc again.
+	r := manual.NewBuilderWithScheme("whatever")
+	backend := stubserver.StartTestService(t, nil)
+	t.Cleanup(func() { backend.Stop() })


Optional nit: t.Cleanup(backend.Stop)

It seems most of the other cleanups can't be simplified similarly (cc.Close errors and things that take parameters have to b e wrapped). 😢

Nice. Thanks. I've gotten so used to using t.Cleanup(func() { ... }) that I didn't even stop to think about the signature of this method.

dfawley · 2023-05-19T23:15:26Z

test/idleness_test.go

+	sCtx, sCancel := context.WithTimeout(ctx, 3*defaultTestShortIdleTimeout)
+	defer sCancel()
+	if cc.WaitForStateChange(sCtx, connectivity.Ready) {
+		t.Fatalf("Connectivity state changed to %q when expected to stay in READY", cc.GetState())
+	}


Use these instead!

func awaitState(ctx context.Context, t *testing.T, cc *grpc.ClientConn, stateWant connectivity.State) func awaitNotState(ctx context.Context, t *testing.T, cc *grpc.ClientConn, stateDoNotWant connectivity.State)

Or factor out similarly, next to these (in clientconn_state_transition_test.go), since this particular test doesn't seem to be able to reuse those.

Done. And also, added a awaitNoStateChange alongside the existing ones. At some point, we can move them to the testutils package.

easwars

Thanks for the review!!

easwars · 2023-05-19T23:46:17Z

call.go

 	}
-	return invoke(ctx, method, args, reply, cc, opts...)
+	cc.idlenessMgr.onCallEnd()


easwars · 2023-05-19T23:48:01Z

clientconn.go

+	if cc.dopts.idleTimeout == 0 {
+		cc.idlenessMgr = newDisabledIdlenessManager()
+	} else {
+		cc.idlenessMgr = newAtomicIdlenessManager(cc, cc.dopts.idleTimeout)
+	}


easwars · 2023-05-19T23:51:03Z

test/idleness_test.go

+	sCtx, sCancel := context.WithTimeout(ctx, 3*defaultTestShortIdleTimeout)
+	defer sCancel()
+	go func() {
+		for ; sCtx.Err() == nil; <-time.After(defaultTestShortTimeout) {


Done. Thanks.

easwars · 2023-05-19T23:53:11Z

test/idleness_test.go

+	// restarted when exiting idle, it will push the same address to grpc again.
+	r := manual.NewBuilderWithScheme("whatever")
+	backend := stubserver.StartTestService(t, nil)
+	t.Cleanup(func() { backend.Stop() })


Nice. Thanks. I've gotten so used to using t.Cleanup(func() { ... }) that I didn't even stop to think about the signature of this method.

easwars · 2023-05-20T00:13:31Z

test/idleness_test.go

+	sCtx, sCancel := context.WithTimeout(ctx, 3*defaultTestShortIdleTimeout)
+	defer sCancel()
+	if cc.WaitForStateChange(sCtx, connectivity.Ready) {
+		t.Fatalf("Connectivity state changed to %q when expected to stay in READY", cc.GetState())
+	}


Done. And also, added a awaitNoStateChange alongside the existing ones. At some point, we can move them to the testutils package.

easwars · 2023-05-20T00:51:45Z

Pulled the final set of changes into g3 and the tests seem healthy. Will merge on Monday.

easwars requested a review from dfawley May 9, 2023 00:58

easwars assigned dfawley May 9, 2023

easwars added the Type: Feature New features or improvements in behavior label May 9, 2023

easwars added this to the 1.56 Release milestone May 9, 2023

easwars removed the request for review from dfawley May 10, 2023 16:39

easwars unassigned dfawley May 10, 2023

easwars force-pushed the channel_idleness_support_2 branch from 41500ec to 3649ed2 Compare May 12, 2023 00:42

easwars marked this pull request as draft May 12, 2023 00:43

easwars force-pushed the channel_idleness_support_2 branch 10 times, most recently from 8ee9c19 to 7d21f4f Compare May 12, 2023 19:20

easwars marked this pull request as ready for review May 12, 2023 19:30

easwars requested a review from dfawley May 12, 2023 19:33

easwars assigned dfawley May 12, 2023

easwars force-pushed the channel_idleness_support_2 branch from 7d21f4f to a3771e2 Compare May 12, 2023 20:02

dfawley reviewed May 12, 2023

View reviewed changes

easwars force-pushed the channel_idleness_support_2 branch from 0e5b271 to 6c1a3ea Compare May 15, 2023 16:32

dfawley assigned easwars and unassigned dfawley May 15, 2023

easwars added 2 commits May 16, 2023 14:22

grpc: support channel idleness

bb51629

call cc.Connect only when IDLE at blocking dial time

968a05f

easwars added 6 commits May 18, 2023 11:02

create and destroy resolver wrapper instead of supporting idleness in it

2b04a5b

start channel in idle, and kick it out of idle at the end of Dial

0748ab4

review comments + remove mutexIdlenessManager

93c990e

reset ccb.curBalancerName when exiting idle

a0eaa5c

remove the mutex idleness manager from tests

37c286a

todo to switch balancer/picker wrappers to the same approach as resol…

0682572

…ver wrapper

dfawley reviewed May 18, 2023

View reviewed changes

easwars assigned dfawley and unassigned easwars May 18, 2023

easwars added 2 commits May 18, 2023 16:09

remove unused consts

460400c

reset ccb.curBalancerName when entering idle instead of when exiting …

f13ba8a

…idle

dfawley reviewed May 19, 2023

View reviewed changes

dfawley approved these changes May 19, 2023

View reviewed changes

dfawley assigned easwars and unassigned dfawley May 19, 2023

easwars added 3 commits May 19, 2023 16:41

refactor new idleness manager to hide implementation details

2068ba0

defer the call to onCallEnd()

d69513f

test cleanups

f1f091a

easwars commented May 20, 2023

View reviewed changes

handle small flake by trying the RPC more frequently

f94f923

easwars merged commit 9b7a947 into grpc:master May 22, 2023
11 checks passed

easwars deleted the channel_idleness_support_2 branch May 22, 2023 19:42

dfawley mentioned this pull request Jul 5, 2023

Support IDLE_TIMEOUT (client-side) #1719

Closed

crazy-max mentioned this pull request Jul 28, 2023

update grpc/protobuf dependencies and protoc moby/buildkit#4074

Closed

github-actions bot locked as resolved and limited conversation to collaborators Nov 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grpc: support channel idleness #6263

grpc: support channel idleness #6263

easwars commented May 9, 2023

easwars commented May 9, 2023

easwars commented May 9, 2023

dfawley left a comment

dfawley May 12, 2023

easwars May 12, 2023

dfawley May 12, 2023

easwars May 12, 2023

dfawley May 12, 2023

easwars May 18, 2023

dfawley May 18, 2023

easwars May 18, 2023

dfawley May 19, 2023

easwars May 19, 2023

dfawley May 19, 2023

easwars May 19, 2023

dfawley May 19, 2023

easwars May 19, 2023

dfawley May 19, 2023

easwars May 19, 2023

dfawley May 19, 2023

dfawley May 19, 2023

easwars May 20, 2023

easwars left a comment

easwars May 19, 2023

easwars May 19, 2023

easwars May 19, 2023

easwars May 19, 2023

easwars May 20, 2023

easwars commented May 20, 2023

		i.timer = newTimer(time.Duration(i.timeout), i.handleIdleTimeout)
		i.isIdle = false

grpc: support channel idleness #6263

grpc: support channel idleness #6263

Conversation

easwars commented May 9, 2023

easwars commented May 9, 2023

easwars commented May 9, 2023

dfawley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

easwars left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

easwars commented May 20, 2023