Feature: Remove support for common names in certificates

[sigmavirus24]

We should follow the lead of Chromium. We should deprecate support for use of common names when a certificate doesn't provide a subjectAltName.

This will bring us into compliance with the >10 year old RFC 2818

[sigmavirus24]

Probably also of interest: https://bugzil.la/1088998

[shazow]

I'm in a very remove-things mood, +1.

[alex]

We should try to collect some numbers on how common this is before we move too quickly IMO, but I like the idea obviously!

[sigmavirus24]

I suspect @Lukasa will be more conservative, but yeah, it'd be nice have some numbers about this.

[shazow]

These numbers you speak of, where do we acquire them?

[alex]

We might be able to ask the Chromium folks for theirs, we could survey the
top 1 million sites or something like that, we could put a warning in and
see if anyone complains. I'm open to other ideas.

On Fri Oct 31 2014 at 7:04:18 PM Andrey Petrov notifications@github.com
wrote:

These numbers you speak of, where do we acquire them?

—
Reply to this email directly or view it on GitHub
#497 (comment).

[sigmavirus24]

Deprecation Warning for the next release and see how many people report it as a bug?

[shazow]

I like the DeprecationWarning, let's do that.

I assume Chromium's threshold for deprecating is much more conservative than ours, so if they're doing it... On the other hand, our demographics are fairly different.

[GrahamDumpleton]

Note that false SSL certificate warnings may be seen if using Python 2.7.2 or older.

This is because in older Python versions the _ssl module does not return 'subjectAltName' in the set of fields in the SSL certificate. That the field isn't returned even if present means that even if the SSL certificate is valid, the check will fail and the warning logged.

To eliminate the warnings you have two choices. The first is to surround the API call to urllib3 which generates it with a warnings.catch_warnings() context manager and override the warnings filter for that context.

        import warnings

        with warnings.catch_warnings():
            warnings.simplefilter("ignore")
                r = http.request('GET', 'https://example.com/')

The alternative is to disable urllib3 warnings globally using:

import urllib3
urllib3.disable_warnings()

Note that if you are using the requests module, which contains a bundled version of urllib3 you will most likely need to use:

import requests.packages.urllib3
requests.packages.urllib3.disable_warnings()

The summary of this related problem has been copied over from #523 given that the warning generated refers to this ticket. For more information see #523.

[alex]

@GrahamDumpleton perhaps it would make sense to change the warning text on 2.7.2, I think that should definitely still emit a warning though -- it's increasingly not practical to do TLS on the web without SANs, you can't even access pypi without them AFAIK /cc @dstufft

[shazow]

Definitely open to having different warning text for the affected versions. @GrahamDumpleton, what do you think? Proposals for such text/PRs are welcome.

[dstufft]

You cannot access PyPI without SAN support that is correct, or most of the sites on Python's TLS.

[BenjamenMeyer]

@GrahamDumpleton better solution is to use Python's logging module and then redirect the information to the log:

import logging
logging.captureWarnings(True)

Works without having to do anything specific to urllib3 or Python Requests; and keeps the messages off the console so the user doesn't get annoyed by things that are out-of-their-control.

[GrahamDumpleton]

@BenjamenMeyer Having it fill the logs is no better for us. We use with warnings.catch_warnings() around the one call in our code and make it go away entirely. No need to modify requests/urllib3 for that either.

[BenjamenMeyer]

@GrahamDumpleton another good resolution too.

I wasn't so much as referring to modifying requests/urllib3 as having to call:

requests.packages.urllib3.disable_warnings()

or

urllib3.disable_warnings()

Code specific to either library and probably removes more warnings than desired.

[shazow]

@BenjamenMeyer Note that disable_warnings accepts a category that allows you to filter by whatever you wish. https://github.com/shazow/urllib3/blob/master/urllib3/__init__.py#L62

[BenjamenMeyer]

@shazow And I would have to have intricate knowledge of urllib3 to figure out what was the right filter to apply. Even so, it's better to capture it in the logger which can also be filtered as desired at the application level.

Of course, I come from a school of thought of toss everything in the log and then filter it all together. I'd rather having a single setting for that than a dozen; and certainly not on a per-API level when an application may use multiple APIs to provide its functionalities.

[jcea]

I am getting this warning. I would love to have a recipe to create X.509 certificates with OpenSSL usable without "requests" warnings.

I the meantime I just downgraded to "requests" 2.4.3 :-(.

[sigmavirus24]

A quick search leads to a StackOverflow answer and other apparently similar resources. It isn't the role of requests or urllib3 to educate people on how to generate X.509 certificates.

[sethmlarson]

Code Changes

• Additional option on SSLContext needs to be set if available hostname_checks_common_name
• Since we're already keeping _match_hostname around even if we're trying to rely more on SSLContext hostname verification more
• Need to verify what is available for pyOpenSSL here
• SecureTransport not sure if any options are supported, likely will need to continue using _match_hostname

Documentation

• Use a URL in the error message
  • Might need to introspect getpeercert to generate a more helpful warning here by detecting missing/incorrect subjectAltName field
• Maybe small example of how to generate a certificate with subjectAltName

[pquentin]

Closing since #2113 was merged.

urllib3 2.0 won't support common names at all.

[pquentin]

If you need to generate new certificates compatible with urllib3 2.0 (that is, with subject alternative names), I would recommend using trustme: https://github.com/python-trio/trustme. This is what urllib3 uses internally.