Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cdsbalancer: test cleanup part 2/N #6554

Merged
merged 3 commits into from Aug 18, 2023

Conversation

easwars
Copy link
Contributor

@easwars easwars commented Aug 15, 2023

This PR cleanup us tests in the cds balancer relating to security configuration. This is a pre-requisite for switching the cds balancer to use the generic xdsClient watch API.

RELEASE NOTES: none

@easwars easwars requested a review from zasweq August 15, 2023 22:02
@easwars easwars added this to the 1.58 Release milestone Aug 15, 2023
@easwars easwars force-pushed the cdsbalancer_test_cleanup_part_2 branch from 5addb43 to 663f6d0 Compare August 17, 2023 18:30
Copy link
Contributor

@zasweq zasweq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great overall. Some minor nits, but I'll go ahead and approve it since that's all I had.

Comment on lines 46 to 48
// Equal reports whether the handshake info structs are identical (have the
// same pointer). This is sufficient as all subconns from one CDS balancer use
// the same one.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update this comment to what you changed this Equal() function to do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks.


func (tcc *testCCWrapper) NewSubConn(addrs []resolver.Address, opts balancer.NewSubConnOptions) (balancer.SubConn, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you mention that this requires exactly one address specified with handshake info present?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}
t.Cleanup(xdsClose)

// Register a wrapped cds LB policy (child policy of the cds LB policy) for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cds LB policy is the child policy of cds LB policy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦

Done.

}
t.Cleanup(xdsClose)

// Register a wrapped cds LB policy (child policy of the cds LB policy) for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here about child policy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 213 to 231
stub.Register(cdsName, stub.BalancerFuncs{
Init: func(bd *stub.BalancerData) {
ccWrapper = &testCCWrapper{
ClientConn: bd.ClientConn,
handshakeInfoCh: make(chan *xdscredsinternal.HandshakeInfo, 1),
}
bd.Data = cdsBuilder.Build(ccWrapper, bd.BuildOptions)
},
ParseConfig: func(lbCfg json.RawMessage) (serviceconfig.LoadBalancingConfig, error) {
return cdsBuilder.(balancer.ConfigParser).ParseConfig(lbCfg)
},
UpdateClientConnState: func(bd *stub.BalancerData, ccs balancer.ClientConnState) error {
bal := bd.Data.(balancer.Balancer)
return bal.UpdateClientConnState(ccs)
},
Close: func(bd *stub.BalancerData) {
bal := bd.Data.(balancer.Balancer)
bal.Close()
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as previous PR. This seems to be common functionality (overriding cds balancer with testCCWrapper to get the NewSubConn handshake info passed in. Can you please pull into helper?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

t.Fatal(err)
}

// Verify that a successful RPC can be made.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mention something about secure connection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// Make a NewSubConn and verify that attributes are added.
if _, err := makeNewSubConn(ctx, edsB.parentCC, tcc, false); err != nil {
t.Fatal(err)
// Verify that a successful RPC can be made.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here on a secure connection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines +735 to +736
// Wait for the connection to move to the new backend that expects
// connections without security.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my understanding, how does this work? How does the switching logic know? Does it fail the first one since the client has no security and fallback to this backend which works because it expects no security?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new cluster resource pushed from here does not contain any security information. Once cds receives this, it is expected to not use the fallback credentials, which in our case is insecure creds.

It was hard to get the same old server (which was expecting a secure connection) to now expect an insecure connection. Hence I spun up a new one, and changed the EDS resource also to point to the new one.

The reason I have a while loop here is because it can take some time for the security configuration to be received and processed by the cds LB policy. And the corresponding EDS resource to be processed by the child policies down the tree.

// bootstrap file contents. Verifies that the connection between the client and
// the server is secure. Subsequently, the cds LB policy receives a cluster
// resource that is NACKed by the xDS client. Test verifies that the cds LB
// policy continue to use the previous good configuration, but the error from
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

continues

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines +759 to +760
// Register a wrapped clusterresolver LB policy (child policy of the cds LB
// policy) for the duration of this test that makes the resolver error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This child policy makes sense :).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol, switched this to the helper I added as part of the previous PR.

@zasweq zasweq assigned easwars and unassigned zasweq Aug 18, 2023
@easwars
Copy link
Contributor Author

easwars commented Aug 18, 2023

Thank you for the review. Addressed all the comments.

@easwars easwars merged commit 8a2c220 into grpc:master Aug 18, 2023
10 of 11 checks passed
@easwars easwars deleted the cdsbalancer_test_cleanup_part_2 branch August 18, 2023 02:50
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants