Add service-loaded extension points for channel initialization

[chrisvest]

Motivation:

Netty is used by many libraries and frameworks that (reasonably) hide the fact that they use Netty under the covers. When Netty is hidden, it is hard, or sometimes impossible, to modify the channel pipelines, attributes, or options, from the outside. It might not even be clear if a framework or library makes use of Netty at all. However, we sometimes want to apply some changes, or make some checks, to most or all channels that are initialized by Netty, regardless of the framework or library that is using Netty in the given case.

Some examples of use-cases are:

• Web application firewalls.
• Server-side request forgery filters.
• Intrusion detection.
• Metrics gathering.

To address these use-cases in a way that don't require integrators to somehow find every Netty usage in a process, we introduce a service-loaded extension point that hooks into the channel initialization process.

Modification:

A new abstract class, ChannelInitializerExtension, is added. Implementations of this interface can be found and service-loaded at runtime by AbstractBootstrap, with the help of the ChannelInitializerExtensions utility class.

The ChannelInitializerExtension class offers three callback methods, one for the initialization of each of the different "kinds" of channels:

• Server listener channels.
• Server child channels.
• Client channels.

The extensions are disabled by default for security reasons, and require opt in by setting the io.netty.bootstrap.extensions system property to serviceload.
To see what extensions are available, the system property can be set to log, and we will load and log (at INFO level) what extensions are available, but not actually run them.

Result:

We have extension points for use cases that are cutting-across different Netty uses, and require low-level Netty access. This makes it relatively easy to implement such use cases without requiring deep access to the Netty internals of arbitrary libraries and frameworks, and without even needing to know which libraries and frameworks are actually using Netty. The only restriction to this, is the use of shading with class-relocation, which in each case would require their own META-INF/services file. Thankfully, most libraries and frameworks that shade Netty, also offer non-shading versions.

[chrisvest]

FYI @vietj

[chrisvest]

@mostroverkhov @normanmaurer Addressed your comments.

[chrisvest]

@mostroverkhov Did some more cleanup based on your comments.
@normanmaurer I've addressed your comments, please take a look.

[idelpivnitskiy]

1. Curious why double instead of int/long? To allow users sneaking in any position, even if other extensions use 0 and 1.0?
2. Usually, the recommended approach for ServiceLoader providers to implement them in a way that order doesn't matter. Even if we add priority, users can easily abuse it by always specifying highest/lowest priority for their extensions, resulting in the same undefined ordering as ServiceLoader provides by default. Consider adding it later, if there is a use-case. As we learn how users use extensions, there might be a better approach instead of a double. For example, in the future we can add smth like before(Class)/after(Class) if there is a need to execute one extension before/after another.

[chrisvest]

Usually, the recommended approach for ServiceLoader providers to implement them in a way that order doesn't matter.

That is still the recommended approach, hence a default implementation is provided, returning a default value.

For example, in the future we can add smth like before(Class)/after(Class) if there is a need to execute one extension before/after another.

Such methods can create ambiguous partial orderings. A value with inherent total ordering cannot.

[normanmaurer]

@idelpivnitskiy does this work for you ?

[idelpivnitskiy]

Don't have a strong opinion here, we can defer adding priorities or keep it as-is.

Such methods can create ambiguous partial orderings. A value with inherent total ordering cannot.

The total ordering requires developers to know what all other providers exist and even after they assign a priority value for their provider, it doesn't mean that the other provider can not change its priority later. With before/after it's easier to target explicit providers. For example, a security provider can always be after logging provider. On the other hand, before/after required explicit dependency on another provider to access its class. So, each approach has pros and cons.

[chrisvest]

I had a discussion about this with @idelpivnitskiy and we realized that the whole deal with isApplicable and ApplicableInfo can be removed, because the post* methods have the same information and more, and at that point they can choose if they want to do something or not. Removing the "applicability check" simplified the code a fair bit.

I have also extended the use of the system property to add support for a "log" value that loads and logs what extensions are available, without actually running them. This way, operators can do dry-runs and make more informed decisions about wether to enable extensions or not.

[idelpivnitskiy]

Thank you!

[chrisvest]

I have added a commit so the DnsNameResolver and OCSP validator both add a marker attribute to the pipelines they create.
It turns out that in practice, extensions likely want to recognize pipelines and give them special treatment. At least the DNS one.

[chrisvest]

@ejona86 I followed what gRPC does with the class loader, while also having what I think is a reasonable default. Please take a look.

[chrisvest]

@ejona86 @normanmaurer Addressed your comments. Please take a look, thanks!

[chrisvest]

@normanmaurer @ejona86 Rebased this. Please take a look.

[normanmaurer]

LGTM... Let's wait and see if @ejona86 has more comments

[chrisvest]

Doesn't look like Eric has more comments, so I'm merging this.

[ejona86]

LGTM