xds: support pick_first custom load balancing policy (A62) #6314

dfawley · 2023-05-24T18:49:01Z

RELEASE NOTES:

xds: support pick_first custom load balancing policy (A62)

zasweq · 2023-05-24T19:02:12Z

Will take a look, but fyi this is failing vet and tests :).

dfawley · 2023-05-24T20:08:55Z

Will take a look, but fyi this is failing vet and tests :).

Forgot to remove a line from my testing...should be fixed now.

zasweq · 2023-05-24T21:34:23Z

test/pickfirst_test.go

@@ -382,6 +383,8 @@ func (s) TestPickFirst_StickyTransientFailure(t *testing.T) {
 }

 func (s) TestPickFirst_ShuffleAddressList(t *testing.T) {


Top level comment about setup and expectation.

I'd rather not be detailed and explain the whole test setup and everything in the comment. The name is very self-explanatory, but I added a short comment.

That's not what I mean. I just feel like it should be briefly explain stuff like configures pick_first as top level lb of the channel, sends it address list before knob and connection isn't random, after knob it is random (with our determinisitic randomness algorithm plumbed). See

grpc-go/test/pickfirst_test.go

Line 239 in 8d2f69a

// TestPickFirst_NewAddressWhileBlocking tests the case where pick_first is

for an example of what I'm going for.

This comment is unnecessarily detailed and a recipe for the test & comment diverging when someone decides to change the test to add an extra check or do things in a different order because of a race, etc. I'd rather not over-document our tests, and would definitely not require detailed documentation anywhere. It's actually pretty rare to have any comments at the top of a test.

E.g. https://cs.opensource.google/go/go/+/refs/tags/go1.20.4:src/net/http/client_test.go;l=550

zasweq · 2023-05-24T21:34:27Z

test/pickfirst_test.go

@@ -431,3 +434,58 @@ func (s) TestPickFirst_ShuffleAddressList(t *testing.T) {
 		t.Fatal(err)
 	}
 }
+
+func (s) TestPickFirst_ShuffleAddressListDisabled(t *testing.T) {


Same here. Please add top level comment with setup and expectation.

Same comment as above.

pickfirst.go

test/pickfirst_test.go

xds/internal/xdsclient/xdslbregistry/converter/converter.go

zasweq · 2023-05-25T21:48:11Z

xds/internal/xdsclient/xdslbregistry/xdslbregistry_test.go

 				},
 			},
+			wantConfig: `[{"pick_first": { "shuffleAddressList": true }}]`,
 		},


Optional: can you add a test case for it being unset and it creating false? I guess that is specific and might be more appropriate (unset and also set to false -> false as per my comment on the main code) in ParseConfig test in pick_first rather than scope it to this test here which tests the registry. Actually since the registry does incorporate this proto -> json conversion, maybe we should add tests here. Or maybe that would scale up the codebase too much but I'm now mindful of those 3 buckets (not set, set and zero, set and non zero) in both the xDS flow through proto and the user writing json itself in the file system e.g. and how those 3 buckets map to our balancer's eventual internal configuration type returned from ParseConfig().

There is no "unset" in the PF proto input. This is a boolean field. If it's missing from the struct, it defaults to its zero value of false. That said, testing that it outputs false when the input is false/absent-so-zero is worthwhile, so done.

Oh. I've seen the proto bools be wrapped in a proto pointer type. Right, now if it's not explicitly set it gets it's zero (false, not nil because it's a bool value type), and it should make a false config in both cases.

I've seen the proto bools be wrapped in a proto pointer type.

There are wrapper types for use cases where presence is important. That's not in use here, because presence is not important for this field (missing=default=zero=false).

xds/internal/xdsclient/xdslbregistry/xdslbregistry_test.go

test/pickfirst_test.go

zasweq · 2023-05-25T22:04:19Z

Btw there's a ton of merge conflicts with go.mod/go.sum stuff.

dfawley

Thanks; comments addressed

test/pickfirst_test.go

dfawley · 2023-05-25T22:09:11Z

test/pickfirst_test.go

@@ -382,6 +383,8 @@ func (s) TestPickFirst_StickyTransientFailure(t *testing.T) {
 }

 func (s) TestPickFirst_ShuffleAddressList(t *testing.T) {


I'd rather not be detailed and explain the whole test setup and everything in the comment. The name is very self-explanatory, but I added a short comment.

dfawley · 2023-05-25T22:09:17Z

test/pickfirst_test.go

@@ -431,3 +434,58 @@ func (s) TestPickFirst_ShuffleAddressList(t *testing.T) {
 		t.Fatal(err)
 	}
 }
+
+func (s) TestPickFirst_ShuffleAddressListDisabled(t *testing.T) {


xds/internal/xdsclient/xdslbregistry/xdslbregistry_test.go

pickfirst.go

dfawley · 2023-05-25T22:48:24Z

xds/internal/xdsclient/xdslbregistry/xdslbregistry_test.go

@@ -198,10 +223,7 @@ func (s) TestConvertToServiceConfigSuccess(t *testing.T) {
 					},
 				},
 			},
-			wantConfig: &internalserviceconfig.BalancerConfig{
-				Name:   "myorg.MyCustomLeastRequestPolicy",
-				Config: customLBConfig{},


I don't understand what you're trying to say.

We need to register the LB policy in order for the code to work, right? The custom LB policy would not be emitted if it's not registered.

dfawley · 2023-05-25T22:48:40Z

xds/internal/xdsclient/xdslbregistry/xdslbregistry_test.go

 				t.Fatalf("ConvertToServiceConfig() got unexpected output, diff (-got +want): %v", diff)
 			}
 		})
 	}
 }

+func jsonMarshal(t *testing.T, x interface{}) string {


x LGTM for a 5 line function.

dfawley · 2023-05-25T22:50:40Z

I'll fix the go.mod/sum stuff before merging

zasweq · 2023-05-26T01:19:39Z

xds/internal/xdsclient/xdslbregistry/xdslbregistry_test.go

@@ -301,6 +314,8 @@ func (s) TestConvertToServiceConfigSuccess(t *testing.T) {
 			if err != nil {
 				t.Fatalf("ConvertToServiceConfig(%s) failed: %v", pretty.ToJSON(test.policy), err)
 			}
+			// got and want must be unmarshalled since JSON strings shouldn't
+			// generally be directly compared.


Nit: briefly mention why. E.g. removes whitespaces/nondeterministic creation/functionality equiavlent JSON even with whitespaces.

I think we can assume a programmer understands why JSON strings can't be directly compared.

zasweq · 2023-05-26T01:21:22Z

pickfirst.go

@@ -112,7 +113,7 @@ func (b *pickfirstBalancer) UpdateClientConnState(state balancer.ClientConnState
 		b.cfg = cfg
 	}

-	if b.cfg != nil && b.cfg.ShuffleAddressList {
+	if envconfig.PickFirstLBConfig && b.cfg != nil && b.cfg.ShuffleAddressList {


Does every language guarantee this optimization that we as a team (I use it a lot) to protect panics if x == nil || x.y (will panic but first check not exiting guarantees this won't panic). Hopefully Go never changes this early evaluation of the conditional.

Yes, short-circuiting is fundamental to every major programming language.

zasweq · 2023-05-26T01:22:49Z

xds/internal/xdsclient/xdslbregistry/xdslbregistry_test.go

+				t.Fatalf("Error unmarshalling rawJSON (%q): %v", rawJSON, err)
+			}
+			var want []map[string]interface{}
+			if err := json.Unmarshal(json.RawMessage(test.wantConfig), &want); err != nil {


Can you make sure this is WAI by deleting a recursion layer of wrrLocalityBalancerConfig and making sure this fails?

Not sure what you're concerned about specifically, but I did this and it failed.

zasweq

LGTM outside of a few trailing comments/nits.

xds: support pick_first custom load balancing policy

e13e8c5

dfawley added the Type: Feature New features or improvements in behavior label May 24, 2023

dfawley added this to the 1.56 Release milestone May 24, 2023

dfawley requested a review from zasweq May 24, 2023 18:49

dfawley assigned zasweq May 24, 2023

envconfig knob

1f3f8d6

dfawley force-pushed the pf_xds branch from 9688d93 to 1f3f8d6 Compare May 24, 2023 20:08

go mod tidy everywhere

6c64630

zasweq requested changes May 25, 2023

View reviewed changes

zasweq assigned dfawley and unassigned zasweq May 25, 2023

review comments

8d2f69a

dfawley commented May 25, 2023

View reviewed changes

dfawley assigned zasweq and unassigned dfawley May 25, 2023

zasweq reviewed May 26, 2023

View reviewed changes

zasweq approved these changes May 26, 2023

View reviewed changes

zasweq assigned dfawley and unassigned zasweq May 26, 2023

dfawley added 3 commits May 26, 2023 11:11

small extra cleanup

57a3f29

Merge master branch

54330ac

re-update go-control-plane to repair after merge

90ca8aa

dfawley merged commit 6c2529b into grpc:master May 30, 2023
11 checks passed

dfawley deleted the pf_xds branch May 30, 2023 16:52

github-actions bot locked as resolved and limited conversation to collaborators Nov 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xds: support pick_first custom load balancing policy (A62) #6314

xds: support pick_first custom load balancing policy (A62) #6314

dfawley commented May 24, 2023 •

edited

zasweq commented May 24, 2023

dfawley commented May 24, 2023

zasweq May 24, 2023

dfawley May 25, 2023

zasweq May 26, 2023

dfawley May 26, 2023

zasweq May 24, 2023

dfawley May 25, 2023

zasweq May 26, 2023

zasweq May 25, 2023

dfawley May 25, 2023

zasweq May 26, 2023 •

edited

dfawley May 26, 2023

zasweq commented May 25, 2023

dfawley left a comment

dfawley May 25, 2023

dfawley May 25, 2023

dfawley May 25, 2023

dfawley May 25, 2023

dfawley commented May 25, 2023

zasweq May 26, 2023

dfawley May 26, 2023

zasweq May 26, 2023

dfawley May 26, 2023

zasweq May 26, 2023

dfawley May 26, 2023

zasweq left a comment

		@@ -382,6 +383,8 @@ func (s) TestPickFirst_StickyTransientFailure(t *testing.T) {
		}

		func (s) TestPickFirst_ShuffleAddressList(t *testing.T) {

xds: support pick_first custom load balancing policy (A62) #6314

xds: support pick_first custom load balancing policy (A62) #6314

Conversation

dfawley commented May 24, 2023 • edited

zasweq commented May 24, 2023

dfawley commented May 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq May 26, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq commented May 25, 2023

dfawley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dfawley commented May 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq left a comment

Choose a reason for hiding this comment

dfawley commented May 24, 2023 •

edited

zasweq May 26, 2023 •

edited