Drop support for OpenSSL<1.1.1

[pquentin]

Warning
Did you see this URL in an error message? Please read our migration guide which covers what you should do to continue using the latest version of urllib3.

There were some integration issues that have been resolved or documented in the latest versions of urllib3, requests, botocore, and other packages. To make that apparent to casual readers we will be minimizing the comments in the discussion that are related to these issues or have been covered in the migration guide. Please follow the official migration guide and if the case you're experiencing isn't covered please open a new issue or ask in our community Discord.

Context

The TLS situation in Python has considerably improved since the early years of urllib3, thanks to the hard work of persons like Christian Heimes and Cory Benfield. urllib3 took advantage of new features even when only a subset of users could use it, and still accepts OpenSSL versions that don't have SNI, for example.

Here's what OpenSSL currently supports:

• Version 1.1.1 will be supported until 2023-09-11 (LTS).
• Version 1.0.2 is no longer supported. Extended support for 1.0.2 to gain access to security fixes for that version is available.
• Versions 1.1.0, 1.0.1, 1.0.0 and 0.9.8 are no longer supported.

RHEL 6 supports 1.0.1e+ and RHEL 7 only supports 1.0.2k and beyond.

We also know that Python 3.10+ will require OpenSSL 1.1.1+ thanks to PEP 644.

Given this the only operating systems that would be in a tough spot if we decide to drop support for OpenSSL <1.1.1 are OSes who:

• Still support OpenSSL <1.1.1 as their default OpenSSL
• Use Python >=3.7 but <3.10 for their default Python
• Are likely to upgrade their system package for urllib3 to v2.0

The combination of the above three is very unlikely. We've identified a few OSes we'd like to evaluate to make sure before we release v2.0:

• Amazon Linux 2
• Gentoo

Minimum requirements

💰 You can get paid to complete this issue! Please read the docs for more information.

• Evaluate the above OSes to see if they'd be impacted by dropping support for OpenSSL <1.1.1. Leave this in a comment in this issue.
• Raise an ImportError if not OpenSSL or ssl.OPENSSL_VERSION < (1, 1, 1) with a message about urllib3 v2.0 requiring OpenSSL 1.1.1+
• Remove work-arounds for conditional features around the ssl module that are due to OpenSSL <1.1.1 (minimum_version, HAS_SNI, _is_openssl_gt_v1_1_1, more examples below)
• Add documentation for urllib3 requiring OpenSSL 1.1.1+
• Add a newfragment

[sethmlarson]

• I'm in favor of dropping support for OpenSSL <1.0.2
  • However I'm not sure if we can stop supporting ssl without SNI, might require a bit more research (can you disable SNI at compile time of OpenSSL? Might be something that microcontrollers do?)
• If PEP 644 goes through we can discuss dropping OpenSSL <1.1.1, but I'm not sure beyond tighter support matrices what we gain from doing that? Does our code rely on detecting OpenSSL 1.1.1 features beyond TLS 1.3?
• Definitely remove notOpenSSL098

[pquentin]

• SNI can be disabled in 1.0.2, but no longer in 1.1.0: openssl/openssl@e481f9b. We'll keep the check for now.
• We currently rely on detection of OpenSSL/Python versions to:
  • Enable SNI (see SNIMissingWarning)
  • Use OpenSSL ciphers for OpenSSL 1.1.1+
  • Call context.set_alpn_protocols(ALPN_PROTOCOLS)
  • Enable post-handshake authentication
  • Use SSLKEYLOGFILE
  • Call load_default_certs
• Will remove notOpenSSL098, thanks Done

[pquentin]

For reference, PEP 644 was accepted for Python 3.10.

[graingert]

how about requiring OpenSSL 1.1.0g or newer? This would fix #2636

[pquentin]

I think that requiring 1.1.1+ for v2 is reasonable, which is what RHEL 8 and Debian stable support. While RHEL 7 is still supporting 1.0.2k+, it is end of life, and for context they still ship urllib3 1.10! Also, while we have CI for 1.1.1 and 3.0.0, we don't have anything for 1.0.2 or 1.1.0.

[graingert]

requiring 1.1.1+ will also allow dropping of pyopenssl inject_into_urllib3 https://github.com/psf/requests/search?q=inject_into_urllib3&type=code and if urllib3 drops ssl 1.0.1 then SecureTransport can go too: https://github.com/pypa/pip/blob/bf91a079791f2daf4339115fb39ce7d7e33a9312/src/pip/_internal/utils/inject_securetransport.py#L24

[sethmlarson]

It'd be interesting to see which flavors of Linux are both using OpenSSL <1.1.1 and likely to update to urllib3 v2.0 as a part of their system libraries specifically. I suspect the list is extremely small. Given that information I'd like to consider only supporting OpenSSL 1.1.1+ for v2.0 unless we find an example that's still using OpenSSL <1.1.1.

Does anyone have such an example?

[sethmlarson]

From this link: https://repology.org/project/openssl/versions

Linux flavors of interest:

• AlmaLinux 8 (OpenSSL 1.0.2 / 1.1.1)
• AlmaLinux 9 (OpenSSL 3.0)
• Alpine 3.8 (OpenSSL 1.0.2)
• Alpine 3.9+ (OpenSSL 1.1.1+)
• Amazon Linux 1 (OpenSSL 1.0.2)
• Amazon Linux 2 (OpenSSL 1.0.2/1.1.1+) Drop support for OpenSSL<1.1.1 #2168 (comment)
• Arch (OpenSSL 1.1.1)
• CentOS 6 (OpenSSL 1.0.1)
• CentOS 7 (OpenSSL 1.0.2)
• CentOS 8+ (OpenSSL 1.1.1+)
• Debian 10+ (OpenSSL 1.1.1/3.0)
• Fedora 29+ (OpenSSL 1.1.1+)
• Gentoo stable (OpenSSL 1.1.1o - Drop support for OpenSSL<1.1.1 #2168 (comment))
• Raspbian stable (OpenSSL 1.1.1+)
• Ubuntu 18.04+ (OpenSSL 1.1.1+)

Would be great to fill in the ⚠️ with details. Done

[sethmlarson]

@pquentin Brought up the case where a downstream packaged pip unbundles urllib3, if the user were to upgrade their system's installation of urllib3 they'd essentially brick their system and not be able to downgrade urllib3 using pip alone.

I'm not sure there's a way we can both use static build system and disallow users from installing urllib3 v2.0 on a system with OpenSSL <1.1.1 so any guard rails here would be for cleaning up the mess afterwards?

Perhaps we can point to documentation on how to unmangle your system if you've upgraded urllib3 after running pip install with system Python?

[graingert]

a downstream packaged pip unbundles urllib3

The system unbundled system pip is usually on a very old python version

[sethmlarson]

True, so the system would also have to have Python 3.7-3.9 installed to have this issue since urllib3 v2.0 requires 3.7+ and 3.10 requires OpenSSL 1.1.1+.

[graingert]

Are there any distributions with an unbundled pip with openssl < 1.1.1 running on python 3.7+?

[sethmlarson]

Probably not. Outside of anyone else bringing a good reason for us to support OpenSSL 1.1.0 or earlier we're going to drop support of OpenSSL <1.1.1. Going to update this issue appropriately.

[pquentin]

@mgorny Is requiring OpenSSL 1.1.1+ for urllib3 2.0 going to be a problem for Gentoo?

[mgorny]

Not at all, Gentoo stable is at 1.1.1o already.

[mgorny]

(Thanks for asking)

[pquentin]

Amazon Linux 2 has an openssl11 package but comes with OpenSSL 1.0.2 preinstalled:

$ docker run -ti amazonlinux:2 yum list | grep openssl | grep installed
openssl-libs.x86_64                    1:1.0.2k-24.amzn2.0.3          installed 

But then it's also preinstalled with Python 2.7:

$ docker run -ti amazonlinux:2 python
Python 2.7.18 (default, May 25 2022, 14:30:51) 
[GCC 7.3.1 20180712 (Red Hat 7.3.1-15)] on linux2

Updated the list above with that information. I don't think that changes our plans.

(I'm also not claiming that issue even if I completed the first item.)

[thibaudcolas]

In case it helps others – official guidance on what to do with this error is here: urllib3 - Migrating from 1.x to 2.0 – Common upgrading issues. In particular this details good options for users of AWS Lambda, RHEL 7, Read the Docs, Amazon Linux 2 (for example Vercel).

[pquentin]

This only applies to LibreSSL, and I believe @gohmc uses OpenSSL.

[denkasyanov]

TLDR: if on Ubuntu, reinstall Python having openssl >= 1.1.1

Had similar problem on our own Ubuntu server.

We wanted to keep dependecies updated, so pinning urllib was suboptimal.

It appears you don't need to install newer version of python. You can install the same version.

The solution was

1. Update openssl in Ubuntu to >= 1.1.1. For example with sudo apt upgrade
2. Remove pyenv's virtualenv AND version of python completely (not just virtualenv!)
3. Install required python version again
4. Create virtualenv

You can check what version of openssl is "baked into" your python distribution:

import ssl
print(ssl.OPENSSL_VERSION_INFO)

For us it worked with Python 3.9 managed by pyenv.
But it seems it should work with almost any combination as long as you reinstall Python.

[sethmlarson]

Hey folks, thanks to everyone who's given feedback on this issue. We're going to lock the discussion for now and point people to our migration guide which covers what you should do to continue using the latest version of urllib3.

There were some integration issues that have been resolved or documented in the latest versions of urllib3, requests, botocore, and other packages. To make that apparent to casual readers we will be minimizing the comments in the discussion that are related to these issues or have been covered in the migration guide. Please follow the official migration guide and if the case you're experiencing isn't covered please open a new issue or ask in our community Discord.