Change subchannel BalancerAddress when attributes change #2228

JamesNK · 2023-08-04T00:42:02Z

Fixes #2225

This PR:

Updates how BalancerAttributes are stored to always use Dictionary<string, object>. This is to improve comparison performance.
Adds an equals comparison on subchannels that includes BalancerAttributes.
Changes subchannel transport to just store the DnsEndPoint. The endpoint is then used to look up the balancer attributes to use in the subchannel.
Changes subchannel transport to use DnsEndPoint instead of BalancerAttributes in all logs.
A BalancerAddress with different attributes replaces the address on the subchannel. If the subchannel has an open connection, it stays open.

cc @kalduzov

kalduzov · 2023-08-22T06:35:16Z

@JamesNK hi.

And why is the work on this PR not moving?

src/Grpc.Net.Client/Balancer/BalancerAddress.cs

mgravell · 2023-08-22T10:34:28Z

src/Grpc.Net.Client/Balancer/BalancerAttributes.cs

@@ -38,20 +38,22 @@ public sealed class BalancerAttributes : IDictionary<string, object?>, IReadOnly
    /// <summary>
    /// Gets a read-only collection of metadata attributes.
    /// </summary>
-    public static readonly BalancerAttributes Empty = new BalancerAttributes(new ReadOnlyDictionary<string, object?>(new Dictionary<string, object?>()));


you cite performance as the reason for this change; I would be genuinely surprized if Dictionary<string,object? out-performed ReadOnlyDictionary<string,object?> except on the custom struct iterator - but if perf is key, I also wonder whether we should be looking to FrozenDictionary<string, object?> on TFMs where it is available, via #if; is there data behind this change? If the enumerator is the big difference, FrozenDictrionary<string,object?> has a custom iterator too, and foreach isn't a common scenario on dictionary, usually

The performance gain isn't Dictionary vs ReadOnlyDictionary. The performance gain is from consistently referencing the values as Dictionary instead of IDictionary.

Using Dictionary avoids interface overhead and allows foreach over the dictionary without allocating.

src/Grpc.Net.Client/Balancer/Internal/BalancerAddressEqualityComparer.cs

src/Grpc.Net.Client/Balancer/Subchannel.cs

mgravell · 2023-08-22T10:43:30Z

@kalduzov please blame me for that; review comments now added

kalduzov · 2023-08-22T10:47:40Z

@kalduzov please blame me for that; review comments now added

All OK. It’s just that the previous PRs were quickly reviewed, and this is just silence. We are very much looking forward to this change. Without it, all our written balancing code does not work :(

mgravell · 2023-08-22T11:42:41Z

because I like data, I did this: https://gist.github.com/mgravell/61c3a892020860a87a3b6cb3ad1687a5

short version: for key lookup (FindKeys), it really doesn't make much difference what you choose - for enumerator, there is a cost delta, and FrozenDictionary<,> is bad for that

amcasey

Initial notes. I'll take a higher level view tomorrow.

src/Grpc.Net.Client/Balancer/BalancerAddress.cs

src/Grpc.Net.Client/Balancer/BalancerAttributes.cs

src/Grpc.Net.Client/Balancer/Subchannel.cs

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [Google.Protobuf](https://togithub.com/protocolbuffers/protobuf) | nuget | patch | `3.24.2` -> `3.24.3` | | [Grpc.Net.Client](https://togithub.com/grpc/grpc-dotnet) | nuget | minor | `2.56.0` -> `2.57.0` | | [Testcontainers](https://dotnet.testcontainers.org/) ([source](https://togithub.com/testcontainers/testcontainers-dotnet)) | nuget | minor | `3.4.0` -> `3.5.0` | --- ### Release Notes <details> <summary>protocolbuffers/protobuf (Google.Protobuf)</summary> ### [`v3.24.3`](https://togithub.com/protocolbuffers/protobuf/compare/v3.24.2...v3.24.3) </details> <details> <summary>grpc/grpc-dotnet (Grpc.Net.Client)</summary> ### [`v2.57.0`](https://togithub.com/grpc/grpc-dotnet/releases/tag/v2.57.0) #### What's Changed - Start 2.57.x development cycle by [@jtattermusch](https://togithub.com/jtattermusch) in [grpc/grpc-dotnet#2227 - Validate Windows version when using WinHttpHandler by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2229 - Support infinite idle connection timeout values by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2231 - Improve BalancerAttributes debugging by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2235 - Update Grpc.Net.Client to remove ValueTask usage by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2222 - Update Newtonsoft.Json to 13.0.3 by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2219 - Consistently don't log message errors on cancellation by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2221 - Fix load balancing flaky test by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2223 - Improve comment in GrpcChannel for WinHttpHandler + OS validation by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2237 - Update OS version detection to get version directly from Windows by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2239 - Update implementation_comparison.md by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2240 - Update to use .NET 8 by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2021 - Bump semver from 6.3.0 to 6.3.1 in /testassets/InteropTestsGrpcWebWebsite/Tests by [@dependabot](https://togithub.com/dependabot) in [grpc/grpc-dotnet#2243 - Force yielding after awaiting CallTask to avoid holding onto locks by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2245 - Update Grpc.Net.Client to use ActivitySource by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2244 - Clear IAsyncStreamReader<T>.Current value before reading next value by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2248 - Upgrade QpsWorker to net8 by [@jtattermusch](https://togithub.com/jtattermusch) in [grpc/grpc-dotnet#2250 - Change subchannel ID to include a channel ID by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2253 - Update Grpc.Tools dependency to 2.57.0 by [@jtattermusch](https://togithub.com/jtattermusch) in [grpc/grpc-dotnet#2257 - Add transport status to subchannel picked log by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2261 - Reduce logger allocations by not using generic CreateLogger by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2256 - Update call debugger display to show status code by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2259 - Log socket lifetime when closing unusable sockets by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2258 - Fix unobserved exceptions with retries by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2255 - Change subchannel BalancerAddress when attributes change by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2228 - Fix connection bugs from BalancerAddress changes by [@JamesNK](https://togithub.com/JamesNK) in [grpc/grpc-dotnet#2265 - \[v2.57.x] Update version to 2.57.0-pre1 by [@jtattermusch](https://togithub.com/jtattermusch) in [grpc/grpc-dotnet#2266 - \[2.57.x] Update version to 2.57.0 by [@jtattermusch](https://togithub.com/jtattermusch) in [grpc/grpc-dotnet#2272 **Full Changelog**: grpc/grpc-dotnet@v2.56.0...v2.57.0 </details> <details> <summary>testcontainers/testcontainers-dotnet (Testcontainers)</summary> ### [`v3.5.0`](https://togithub.com/testcontainers/testcontainers-dotnet/releases/tag/3.5.0) [Compare Source](https://togithub.com/testcontainers/testcontainers-dotnet/compare/3.4.0...3.5.0) ### What's Changed #### 🚀 Features - feat: Allow MongoDb module configuration without credentials ([#983](https://togithub.com/testcontainers/testcontainers-dotnet/issues/983)) [@the-avid-engineer](https://togithub.com/the-avid-engineer) - feat: Add support for RSA private key (RsaPrivateCrtKeyParameters) TLS authentication with protected Docker daemon sockets ([#978](https://togithub.com/testcontainers/testcontainers-dotnet/issues/978)) [@zuntio](https://togithub.com/zuntio) - feat: Add InfluxDb module ([#975](https://togithub.com/testcontainers/testcontainers-dotnet/issues/975)) [@MelomanG](https://togithub.com/MelomanG) #### 🐛 Bug Fixes - fix: Do not pre pull Dockerfile build stages that do not correspond to base images ([#979](https://togithub.com/testcontainers/testcontainers-dotnet/issues/979)) [@HofmeisterAn](https://togithub.com/HofmeisterAn) #### 📖 Documentation - docs: Add documentation on enabling debug log messages for the default logger ([#991](https://togithub.com/testcontainers/testcontainers-dotnet/issues/991)) [@HofmeisterAn](https://togithub.com/HofmeisterAn) - docs: Add global Testcontainers header ([#967](https://togithub.com/testcontainers/testcontainers-dotnet/issues/967)) [@leocross](https://togithub.com/leocross) #### 🧹 Housekeeping - chore: Improve error message when Docker is not running ([#987](https://togithub.com/testcontainers/testcontainers-dotnet/issues/987)) [@0xced](https://togithub.com/0xced) - chore: Update BouncyCastle.Cryptography to 2.2.1 (previous Portable.BouncyCastle) ([#985](https://togithub.com/testcontainers/testcontainers-dotnet/issues/985)) [@jcmrva](https://togithub.com/jcmrva) - chore: Add User-Agent HTTP header to Docker.DotNet client ([#970](https://togithub.com/testcontainers/testcontainers-dotnet/issues/970)) [@eddumelendez](https://togithub.com/eddumelendez) - chore: Remove `CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT` env var ([#971](https://togithub.com/testcontainers/testcontainers-dotnet/issues/971)) [@eddumelendez](https://togithub.com/eddumelendez) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/cerbos/cerbos-sdk-net).  Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Signed-off-by: Oğuzhan Durgun <oguzhandurgun95@gmail.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

JamesNK force-pushed the jamesnk/address-change-on-attributes branch from 7e405bd to a2a88d2 Compare August 7, 2023 22:11

JamesNK requested review from mgravell, ReubenBond, BrennanConroy and amcasey August 14, 2023 01:09

JamesNK force-pushed the jamesnk/address-change-on-attributes branch from a2a88d2 to 2ed833e Compare August 15, 2023 02:21

mgravell reviewed Aug 22, 2023

View reviewed changes

JamesNK force-pushed the jamesnk/address-change-on-attributes branch 2 times, most recently from ff270fc to db9a001 Compare August 27, 2023 05:00

amcasey reviewed Aug 29, 2023

View reviewed changes

Change subchannel BalancerAddress when attributes change

830ff72

JamesNK force-pushed the jamesnk/address-change-on-attributes branch 2 times, most recently from 6032318 to d02d7e9 Compare August 29, 2023 02:16

Clean up

7908e8b

mgravell approved these changes Aug 29, 2023

View reviewed changes

amcasey approved these changes Aug 29, 2023

View reviewed changes

JamesNK merged commit 311f878 into grpc:master Aug 30, 2023
5 checks passed

JamesNK deleted the jamesnk/address-change-on-attributes branch August 30, 2023 03:56

JamesNK mentioned this pull request Aug 30, 2023

Fix connection bugs from BalancerAddress changes #2265

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change subchannel BalancerAddress when attributes change #2228

Change subchannel BalancerAddress when attributes change #2228

JamesNK commented Aug 4, 2023 •

edited

kalduzov commented Aug 22, 2023

mgravell Aug 22, 2023

JamesNK Aug 24, 2023

mgravell commented Aug 22, 2023

kalduzov commented Aug 22, 2023

mgravell commented Aug 22, 2023

amcasey left a comment

Change subchannel BalancerAddress when attributes change #2228

Change subchannel BalancerAddress when attributes change #2228

Conversation

JamesNK commented Aug 4, 2023 • edited

kalduzov commented Aug 22, 2023

mgravell Aug 22, 2023

Choose a reason for hiding this comment

JamesNK Aug 24, 2023

Choose a reason for hiding this comment

mgravell commented Aug 22, 2023

kalduzov commented Aug 22, 2023

mgravell commented Aug 22, 2023

amcasey left a comment

Choose a reason for hiding this comment

JamesNK commented Aug 4, 2023 •

edited