raise FullPoolError if we try to _put_conn an extra connection in blocking mode #2197
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dmazin
commented
Apr 12, 2021
dmazin
force-pushed
the
dmazin/connection-pool-tests
branch
from
28bdd64
to
12a5232
Compare
Codecov Report
@@ Coverage Diff @@
## main #2197 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 24 25 +1
Lines 2219 2288 +69
=========================================
+ Hits 2219 2288 +69
Continue to review full report at Codecov.
|
In blocking mode, if you get a connection from a pool and then try to get another connection, EmptyPoolError is raised. Right now, it somewhat confusingly says that the pool has reached maximum size, but in reality, something else is happening. The point of blocking mode is to guarantee that at most maxsize connections are open at any time. Therefore, if the pool is empty, under normal operation that means that a connection is currently in use and we should not create a new connection. I have updated the EmptyPoolError message to make this more clear.
dmazin
changed the title
dmazin
commented
Apr 16, 2021
dmazin
commented
Apr 16, 2021
| conn and conn.close() | ||
|
|
||
| if self.block: | ||
| raise FullPoolError( |
There was a problem hiding this comment.
This block originally had a "This should never happen if self.block == True" comment. If this should never happen then if it does happen we should raise an exception.
dmazin
commented
Apr 16, 2021
src/urllib3/connectionpool.py
Outdated
| except AttributeError: | ||
| # self.pool is None. | ||
| pass | ||
| # Connection never got put back into the pool, close it. | ||
| conn and conn.close() |
There was a problem hiding this comment.
I recommend viewing these changes in side-by-side diff instead of unified.
The only reason there are so many lines moved around is because I need to make sure to close the conn before raising FullPoolError.
I wish try/finally did not call finally if you return in the try, oh well.
sethmlarson
requested changes
Apr 16, 2021
@sethmlarson That makes way more sense than my initial approach. This PR is ready for another look. |
|
@sethmlarson ping <3 |
sethmlarson
requested changes
Apr 28, 2021
dmazin
force-pushed
the
dmazin/connection-pool-tests
branch
from
006fb23
to
4820d2c
Compare
sethmlarson
approved these changes
May 1, 2021
|
Thanks @dmazin for your first contribution and being patient through the review cycles 🙇 Hope you're interested in contributing more, there's a lot of cool issues in the v2.0 milestone. If not that's totally fine, thanks again! |
@sethmlarson thank you so much for going out of your way to write such a welcoming message! now THAT'S how you foster community! I am definitely wondering how to get move involved with 2.0 -- will check out the milestone |
27 tasks
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
May 5, 2023
2.0.0 (2023-04-26) ================== Read the `v2.0 migration guide <https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html>`__ for help upgrading to the latest version of urllib3. Removed ------- * Removed support for Python 2.7, 3.5, and 3.6 (`#883 <https://github.com/urllib3/urllib3/issues/883>`__, `#2336 <https://github.com/urllib3/urllib3/issues/2336>`__). * Removed fallback on certificate ``commonName`` in ``match_hostname()`` function. This behavior was deprecated in May 2000 in RFC 2818. Instead only ``subjectAltName`` is used to verify the hostname by default. To enable verifying the hostname against ``commonName`` use ``SSLContext.hostname_checks_common_name = True`` (`#2113 <https://github.com/urllib3/urllib3/issues/2113>`__). * Removed support for Python with an ``ssl`` module compiled with LibreSSL, CiscoSSL, wolfSSL, and all other OpenSSL alternatives. Python is moving to require OpenSSL with PEP 644 (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__). * Removed support for OpenSSL versions earlier than 1.1.1 or that don't have SNI support. When an incompatible OpenSSL version is detected an ``ImportError`` is raised (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__). * Removed the list of default ciphers for OpenSSL 1.1.1+ and SecureTransport as their own defaults are already secure (`#2082 <https://github.com/urllib3/urllib3/issues/2082>`__). * Removed ``urllib3.contrib.appengine.AppEngineManager`` and support for Google App Engine Standard Environment (`#2044 <https://github.com/urllib3/urllib3/issues/2044>`__). * Removed deprecated ``Retry`` options ``method_whitelist``, ``DEFAULT_REDIRECT_HEADERS_BLACKLIST`` (`#2086 <https://github.com/urllib3/urllib3/issues/2086>`__). * Removed ``urllib3.HTTPResponse.from_httplib`` (`#2648 <https://github.com/urllib3/urllib3/issues/2648>`__). * Removed default value of ``None`` for the ``request_context`` parameter of ``urllib3.PoolManager.connection_from_pool_key``. This change should have no effect on users as the default value of ``None`` was an invalid option and was never used (`#1897 <https://github.com/urllib3/urllib3/issues/1897>`__). * Removed the ``urllib3.request`` module. ``urllib3.request.RequestMethods`` has been made a private API. This change was made to ensure that ``from urllib3 import request`` imported the top-level ``request()`` function instead of the ``urllib3.request`` module (`#2269 <https://github.com/urllib3/urllib3/issues/2269>`__). * Removed support for SSLv3.0 from the ``urllib3.contrib.pyopenssl`` even when support is available from the compiled OpenSSL library (`#2233 <https://github.com/urllib3/urllib3/issues/2233>`__). * Removed the deprecated ``urllib3.contrib.ntlmpool`` module (`#2339 <https://github.com/urllib3/urllib3/issues/2339>`__). * Removed ``DEFAULT_CIPHERS``, ``HAS_SNI``, ``USE_DEFAULT_SSLCONTEXT_CIPHERS``, from the private module ``urllib3.util.ssl_`` (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__). * Removed ``urllib3.exceptions.SNIMissingWarning`` (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__). * Removed the ``_prepare_conn`` method from ``HTTPConnectionPool``. Previously this was only used to call ``HTTPSConnection.set_cert()`` by ``HTTPSConnectionPool`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). * Removed ``tls_in_tls_required`` property from ``HTTPSConnection``. This is now determined from the ``scheme`` parameter in ``HTTPConnection.set_tunnel()`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). Deprecated ---------- * Deprecated ``HTTPResponse.getheaders()`` and ``HTTPResponse.getheader()`` which will be removed in urllib3 v2.1.0. Instead use ``HTTPResponse.headers`` and ``HTTPResponse.headers.get(name, default)``. (`#1543 <https://github.com/urllib3/urllib3/issues/1543>`__, `#2814 <https://github.com/urllib3/urllib3/issues/2814>`__). * Deprecated ``urllib3.contrib.pyopenssl`` module which will be removed in urllib3 v2.1.0 (`#2691 <https://github.com/urllib3/urllib3/issues/2691>`__). * Deprecated ``urllib3.contrib.securetransport`` module which will be removed in urllib3 v2.1.0 (`#2692 <https://github.com/urllib3/urllib3/issues/2692>`__). * Deprecated ``ssl_version`` option in favor of ``ssl_minimum_version``. ``ssl_version`` will be removed in urllib3 v2.1.0 (`#2110 <https://github.com/urllib3/urllib3/issues/2110>`__). * Deprecated the ``strict`` parameter as it's not longer needed in Python 3.x. It will be removed in urllib3 v2.1.0 (`#2267 <https://github.com/urllib3/urllib3/issues/2267>`__) * Deprecated the ``NewConnectionError.pool`` attribute which will be removed in urllib3 v2.1.0 (`#2271 <https://github.com/urllib3/urllib3/issues/2271>`__). * Deprecated ``format_header_param_html5`` and ``format_header_param`` in favor of ``format_multipart_header_param`` (`#2257 <https://github.com/urllib3/urllib3/issues/2257>`__). * Deprecated ``RequestField.header_formatter`` parameter which will be removed in urllib3 v2.1.0 (`#2257 <https://github.com/urllib3/urllib3/issues/2257>`__). * Deprecated ``HTTPSConnection.set_cert()`` method. Instead pass parameters to the ``HTTPSConnection`` constructor (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). * Deprecated ``HTTPConnection.request_chunked()`` method which will be removed in urllib3 v2.1.0. Instead pass ``chunked=True`` to ``HTTPConnection.request()`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). Added ----- * Added top-level ``urllib3.request`` function which uses a preconfigured module-global ``PoolManager`` instance (`#2150 <https://github.com/urllib3/urllib3/issues/2150>`__). * Added the ``json`` parameter to ``urllib3.request()``, ``PoolManager.request()``, and ``ConnectionPool.request()`` methods to send JSON bodies in requests. Using this parameter will set the header ``Content-Type: application/json`` if ``Content-Type`` isn't already defined. Added support for parsing JSON response bodies with ``HTTPResponse.json()`` method (`#2243 <https://github.com/urllib3/urllib3/issues/2243>`__). * Added type hints to the ``urllib3`` module (`#1897 <https://github.com/urllib3/urllib3/issues/1897>`__). * Added ``ssl_minimum_version`` and ``ssl_maximum_version`` options which set ``SSLContext.minimum_version`` and ``SSLContext.maximum_version`` (`#2110 <https://github.com/urllib3/urllib3/issues/2110>`__). * Added support for Zstandard (RFC 8878) when ``zstandard`` 1.18.0 or later is installed. Added the ``zstd`` extra which installs the ``zstandard`` package (`#1992 <https://github.com/urllib3/urllib3/issues/1992>`__). * Added ``urllib3.response.BaseHTTPResponse`` class. All future response classes will be subclasses of ``BaseHTTPResponse`` (`#2083 <https://github.com/urllib3/urllib3/issues/2083>`__). * Added ``FullPoolError`` which is raised when ``PoolManager(block=True)`` and a connection is returned to a full pool (`#2197 <https://github.com/urllib3/urllib3/issues/2197>`__). * Added ``HTTPHeaderDict`` to the top-level ``urllib3`` namespace (`#2216 <https://github.com/urllib3/urllib3/issues/2216>`__). * Added support for configuring header merging behavior with HTTPHeaderDict When using a ``HTTPHeaderDict`` to provide headers for a request, by default duplicate header values will be repeated. But if ``combine=True`` is passed into a call to ``HTTPHeaderDict.add``, then the added header value will be merged in with an existing value into a comma-separated list (``X-My-Header: foo, bar``) (`#2242 <https://github.com/urllib3/urllib3/issues/2242>`__). * Added ``NameResolutionError`` exception when a DNS error occurs (`#2305 <https://github.com/urllib3/urllib3/issues/2305>`__). * Added ``proxy_assert_hostname`` and ``proxy_assert_fingerprint`` kwargs to ``ProxyManager`` (`#2409 <https://github.com/urllib3/urllib3/issues/2409>`__). * Added a configurable ``backoff_max`` parameter to the ``Retry`` class. If a custom ``backoff_max`` is provided to the ``Retry`` class, it will replace the ``Retry.DEFAULT_BACKOFF_MAX`` (`#2494 <https://github.com/urllib3/urllib3/issues/2494>`__). * Added the ``authority`` property to the Url class as per RFC 3986 3.2. This property should be used in place of ``netloc`` for users who want to include the userinfo (auth) component of the URI (`#2520 <https://github.com/urllib3/urllib3/issues/2520>`__). * Added the ``scheme`` parameter to ``HTTPConnection.set_tunnel`` to configure the scheme of the origin being tunnelled to (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). * Added the ``is_closed``, ``is_connected`` and ``has_connected_to_proxy`` properties to ``HTTPConnection`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). * Added optional ``backoff_jitter`` parameter to ``Retry``. (`#2952 <https://github.com/urllib3/urllib3/issues/2952>`__) Changed ------- * Changed ``urllib3.response.HTTPResponse.read`` to respect the semantics of ``io.BufferedIOBase`` regardless of compression. Specifically, this method: * Only returns an empty bytes object to indicate EOF (that is, the response has been fully consumed). * Never returns more bytes than requested. * Can issue any number of system calls: zero, one or multiple. If you want each ``urllib3.response.HTTPResponse.read`` call to issue a single system call, you need to disable decompression by setting ``decode_content=False`` (`#2128 <https://github.com/urllib3/urllib3/issues/2128>`__). * Changed ``urllib3.HTTPConnection.getresponse`` to return an instance of ``urllib3.HTTPResponse`` instead of ``http.client.HTTPResponse`` (`#2648 <https://github.com/urllib3/urllib3/issues/2648>`__). * Changed ``ssl_version`` to instead set the corresponding ``SSLContext.minimum_version`` and ``SSLContext.maximum_version`` values. Regardless of ``ssl_version`` passed ``SSLContext`` objects are now constructed using ``ssl.PROTOCOL_TLS_CLIENT`` (`#2110 <https://github.com/urllib3/urllib3/issues/2110>`__). * Changed default ``SSLContext.minimum_version`` to be ``TLSVersion.TLSv1_2`` in line with Python 3.10 (`#2373 <https://github.com/urllib3/urllib3/issues/2373>`__). * Changed ``ProxyError`` to wrap any connection error (timeout, TLS, DNS) that occurs when connecting to the proxy (`#2482 <https://github.com/urllib3/urllib3/pull/2482>`__). * Changed ``urllib3.util.create_urllib3_context`` to not override the system cipher suites with a default value. The new default will be cipher suites configured by the operating system (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__). * Changed ``multipart/form-data`` header parameter formatting matches the WHATWG HTML Standard as of 2021-06-10. Control characters in filenames are no longer percent encoded (`#2257 <https://github.com/urllib3/urllib3/issues/2257>`__). * Changed the error raised when connecting via HTTPS when the ``ssl`` module isn't available from ``SSLError`` to ``ImportError`` (`#2589 <https://github.com/urllib3/urllib3/issues/2589>`__). * Changed ``HTTPConnection.request()`` to always use lowercase chunk boundaries when sending requests with ``Transfer-Encoding: chunked`` (`#2515 <https://github.com/urllib3/urllib3/issues/2515>`__). * Changed ``enforce_content_length`` default to True, preventing silent data loss when reading streamed responses (`#2514 <https://github.com/urllib3/urllib3/issues/2514>`__). * Changed internal implementation of ``HTTPHeaderDict`` to use ``dict`` instead of ``collections.OrderedDict`` for better performance (`#2080 <https://github.com/urllib3/urllib3/issues/2080>`__). * Changed the ``urllib3.contrib.pyopenssl`` module to wrap ``OpenSSL.SSL.Error`` with ``ssl.SSLError`` in ``PyOpenSSLContext.load_cert_chain`` (`#2628 <https://github.com/urllib3/urllib3/issues/2628>`__). * Changed usage of the deprecated ``socket.error`` to ``OSError`` (`#2120 <https://github.com/urllib3/urllib3/issues/2120>`__). * Changed all parameters in the ``HTTPConnection`` and ``HTTPSConnection`` constructors to be keyword-only except ``host`` and ``port`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). * Changed ``HTTPConnection.getresponse()`` to set the socket timeout from ``HTTPConnection.timeout`` value before reading data from the socket. This previously was done manually by the ``HTTPConnectionPool`` calling ``HTTPConnection.sock.settimeout(...)`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). * Changed the ``_proxy_host`` property to ``_tunnel_host`` in ``HTTPConnectionPool`` to more closely match how the property is used (value in ``HTTPConnection.set_tunnel()``) (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__). * Changed name of ``Retry.BACK0FF_MAX`` to be ``Retry.DEFAULT_BACKOFF_MAX``. * Changed TLS handshakes to use ``SSLContext.check_hostname`` when possible (`#2452 <https://github.com/urllib3/urllib3/pull/2452>`__). * Changed ``server_hostname`` to behave like other parameters only used by ``HTTPSConnectionPool`` (`#2537 <https://github.com/urllib3/urllib3/pull/2537>`__). * Changed the default ``blocksize`` to 16KB to match OpenSSL's default read amounts (`#2348 <https://github.com/urllib3/urllib3/pull/2348>`__). * Changed ``HTTPResponse.read()`` to raise an error when calling with ``decode_content=False`` after using ``decode_content=True`` to prevent data loss (`#2800 <https://github.com/urllib3/urllib3/issues/2800>`__). Fixed ----- * Fixed thread-safety issue where accessing a ``PoolManager`` with many distinct origins would cause connection pools to be closed while requests are in progress (`#1252 <https://github.com/urllib3/urllib3/issues/1252>`__). * Fixed an issue where an ``HTTPConnection`` instance would erroneously reuse the socket read timeout value from reading the previous response instead of a newly configured connect timeout. Instead now if ``HTTPConnection.timeout`` is updated before sending the next request the new timeout value will be used (`#2645 <https://github.com/urllib3/urllib3/issues/2645>`__). * Fixed ``socket.error.errno`` when raised from pyOpenSSL's ``OpenSSL.SSL.SysCallError`` (`#2118 <https://github.com/urllib3/urllib3/issues/2118>`__). * Fixed the default value of ``HTTPSConnection.socket_options`` to match ``HTTPConnection`` (`#2213 <https://github.com/urllib3/urllib3/issues/2213>`__). * Fixed a bug where ``headers`` would be modified by the ``remove_headers_on_redirect`` feature (`#2272 <https://github.com/urllib3/urllib3/issues/2272>`__). * Fixed a reference cycle bug in ``urllib3.util.connection.create_connection()`` (`#2277 <https://github.com/urllib3/urllib3/issues/2277>`__). * Fixed a socket leak if ``HTTPConnection.connect()`` fails (`#2571 <https://github.com/urllib3/urllib3/pull/2571>`__). * Fixed ``urllib3.contrib.pyopenssl.WrappedSocket`` and ``urllib3.contrib.securetransport.WrappedSocket`` close methods (`#2970 <https://github.com/urllib3/urllib3/issues/2970>`__)
github-merge-queue bot
pushed a commit
to owntracks/android
that referenced
this pull request
Aug 29, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [urllib3](https://togithub.com/urllib3/urllib3) ([changelog](https://togithub.com/urllib3/urllib3/blob/main/CHANGES.rst)) | `==1.26.16` -> `==2.0.4` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### ⚠ Dependency Lookup Warnings ⚠ Warnings were logged while processing this repo. Please check the logs for more information. --- ### Release Notes <details> <summary>urllib3/urllib3 (urllib3)</summary> ### [`v2.0.4`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#204-2023-07-19) [Compare Source](https://togithub.com/urllib3/urllib3/compare/2.0.3...2.0.4) \================== - Added support for union operators to `HTTPHeaderDict` (`#​2254 <https://github.com/urllib3/urllib3/issues/2254>`\__) - Added `BaseHTTPResponse` to `urllib3.__all__` (`#​3078 <https://github.com/urllib3/urllib3/issues/3078>`\__) - Fixed `urllib3.connection.HTTPConnection` to raise the `http.client.connect` audit event to have the same behavior as the standard library HTTP client (`#​2757 <https://github.com/urllib3/urllib3/issues/2757>`\__) - Relied on the standard library for checking hostnames in supported PyPy releases (`#​3087 <https://github.com/urllib3/urllib3/issues/3087>`\__) ### [`v2.0.3`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#203-2023-06-07) [Compare Source](https://togithub.com/urllib3/urllib3/compare/2.0.2...2.0.3) \================== - Allowed alternative SSL libraries such as LibreSSL, while still issuing a warning as we cannot help users facing issues with implementations other than OpenSSL. (`#​3020 <https://github.com/urllib3/urllib3/issues/3020>`\__) - Deprecated URLs which don't have an explicit scheme (`#​2950 <https://github.com/urllib3/urllib3/pull/2950>`\_) - Fixed response decoding with Zstandard when compressed data is made of several frames. (`#​3008 <https://github.com/urllib3/urllib3/issues/3008>`\__) - Fixed `assert_hostname=False` to correctly skip hostname check. (`#​3051 <https://github.com/urllib3/urllib3/issues/3051>`\__) ### [`v2.0.2`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#202-2023-05-03) [Compare Source](https://togithub.com/urllib3/urllib3/compare/2.0.1...2.0.2) \================== - Fixed `HTTPResponse.stream()` to continue yielding bytes if buffered decompressed data was still available to be read even if the underlying socket is closed. This prevents a compressed response from being truncated. (`#​3009 <https://github.com/urllib3/urllib3/issues/3009>`\__) ### [`v2.0.1`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#201-2023-04-30) [Compare Source](https://togithub.com/urllib3/urllib3/compare/2.0.0...2.0.1) \================== - Fixed a socket leak when fingerprint or hostname verifications fail. (`#​2991 <https://github.com/urllib3/urllib3/issues/2991>`\__) - Fixed an error when `HTTPResponse.read(0)` was the first `read` call or when the internal response body buffer was otherwise empty. (`#​2998 <https://github.com/urllib3/urllib3/issues/2998>`\__) ### [`v2.0.0`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#200-2023-04-26) [Compare Source](https://togithub.com/urllib3/urllib3/compare/1.26.16...2.0.0) \================== Read the `v2.0 migration guide <https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html>`\__ for help upgrading to the latest version of urllib3. ## Removed - Removed support for Python 2.7, 3.5, and 3.6 (`#​883 <https://github.com/urllib3/urllib3/issues/883>`**, `#​2336 <https://github.com/urllib3/urllib3/issues/2336>`**). - Removed fallback on certificate `commonName` in `match_hostname()` function. This behavior was deprecated in May 2000 in RFC 2818. Instead only `subjectAltName` is used to verify the hostname by default. To enable verifying the hostname against `commonName` use `SSLContext.hostname_checks_common_name = True` (`#​2113 <https://github.com/urllib3/urllib3/issues/2113>`\__). - Removed support for Python with an `ssl` module compiled with LibreSSL, CiscoSSL, wolfSSL, and all other OpenSSL alternatives. Python is moving to require OpenSSL with PEP 644 (`#​2168 <https://github.com/urllib3/urllib3/issues/2168>`\__). - Removed support for OpenSSL versions earlier than 1.1.1 or that don't have SNI support. When an incompatible OpenSSL version is detected an `ImportError` is raised (`#​2168 <https://github.com/urllib3/urllib3/issues/2168>`\__). - Removed the list of default ciphers for OpenSSL 1.1.1+ and SecureTransport as their own defaults are already secure (`#​2082 <https://github.com/urllib3/urllib3/issues/2082>`\__). - Removed `urllib3.contrib.appengine.AppEngineManager` and support for Google App Engine Standard Environment (`#​2044 <https://github.com/urllib3/urllib3/issues/2044>`\__). - Removed deprecated `Retry` options `method_whitelist`, `DEFAULT_REDIRECT_HEADERS_BLACKLIST` (`#​2086 <https://github.com/urllib3/urllib3/issues/2086>`\__). - Removed `urllib3.HTTPResponse.from_httplib` (`#​2648 <https://github.com/urllib3/urllib3/issues/2648>`\__). - Removed default value of `None` for the `request_context` parameter of `urllib3.PoolManager.connection_from_pool_key`. This change should have no effect on users as the default value of `None` was an invalid option and was never used (`#​1897 <https://github.com/urllib3/urllib3/issues/1897>`\__). - Removed the `urllib3.request` module. `urllib3.request.RequestMethods` has been made a private API. This change was made to ensure that `from urllib3 import request` imported the top-level `request()` function instead of the `urllib3.request` module (`#​2269 <https://github.com/urllib3/urllib3/issues/2269>`\__). - Removed support for SSLv3.0 from the `urllib3.contrib.pyopenssl` even when support is available from the compiled OpenSSL library (`#​2233 <https://github.com/urllib3/urllib3/issues/2233>`\__). - Removed the deprecated `urllib3.contrib.ntlmpool` module (`#​2339 <https://github.com/urllib3/urllib3/issues/2339>`\__). - Removed `DEFAULT_CIPHERS`, `HAS_SNI`, `USE_DEFAULT_SSLCONTEXT_CIPHERS`, from the private module `urllib3.util.ssl_` (`#​2168 <https://github.com/urllib3/urllib3/issues/2168>`\__). - Removed `urllib3.exceptions.SNIMissingWarning` (`#​2168 <https://github.com/urllib3/urllib3/issues/2168>`\__). - Removed the `_prepare_conn` method from `HTTPConnectionPool`. Previously this was only used to call `HTTPSConnection.set_cert()` by `HTTPSConnectionPool` (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Removed `tls_in_tls_required` property from `HTTPSConnection`. This is now determined from the `scheme` parameter in `HTTPConnection.set_tunnel()` (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Removed the `strict` parameter/attribute from `HTTPConnection`, `HTTPSConnection`, `HTTPConnectionPool`, `HTTPSConnectionPool`, and `HTTPResponse` (`#​2064 <https://github.com/urllib3/urllib3/issues/2064>`\__). ## Deprecated - Deprecated `HTTPResponse.getheaders()` and `HTTPResponse.getheader()` which will be removed in urllib3 v2.1.0. Instead use `HTTPResponse.headers` and `HTTPResponse.headers.get(name, default)`. (`#​1543 <https://github.com/urllib3/urllib3/issues/1543>`**, `#​2814 <https://github.com/urllib3/urllib3/issues/2814>`**). - Deprecated `urllib3.contrib.pyopenssl` module which will be removed in urllib3 v2.1.0 (`#​2691 <https://github.com/urllib3/urllib3/issues/2691>`\__). - Deprecated `urllib3.contrib.securetransport` module which will be removed in urllib3 v2.1.0 (`#​2692 <https://github.com/urllib3/urllib3/issues/2692>`\__). - Deprecated `ssl_version` option in favor of `ssl_minimum_version`. `ssl_version` will be removed in urllib3 v2.1.0 (`#​2110 <https://github.com/urllib3/urllib3/issues/2110>`\__). - Deprecated the `strict` parameter of `PoolManager.connection_from_context()` as it's not longer needed in Python 3.x. It will be removed in urllib3 v2.1.0 (`#​2267 <https://github.com/urllib3/urllib3/issues/2267>`\__) - Deprecated the `NewConnectionError.pool` attribute which will be removed in urllib3 v2.1.0 (`#​2271 <https://github.com/urllib3/urllib3/issues/2271>`\__). - Deprecated `format_header_param_html5` and `format_header_param` in favor of `format_multipart_header_param` (`#​2257 <https://github.com/urllib3/urllib3/issues/2257>`\__). - Deprecated `RequestField.header_formatter` parameter which will be removed in urllib3 v2.1.0 (`#​2257 <https://github.com/urllib3/urllib3/issues/2257>`\__). - Deprecated `HTTPSConnection.set_cert()` method. Instead pass parameters to the `HTTPSConnection` constructor (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Deprecated `HTTPConnection.request_chunked()` method which will be removed in urllib3 v2.1.0. Instead pass `chunked=True` to `HTTPConnection.request()` (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). ## Added - Added top-level `urllib3.request` function which uses a preconfigured module-global `PoolManager` instance (`#​2150 <https://github.com/urllib3/urllib3/issues/2150>`\__). - Added the `json` parameter to `urllib3.request()`, `PoolManager.request()`, and `ConnectionPool.request()` methods to send JSON bodies in requests. Using this parameter will set the header `Content-Type: application/json` if `Content-Type` isn't already defined. Added support for parsing JSON response bodies with `HTTPResponse.json()` method (`#​2243 <https://github.com/urllib3/urllib3/issues/2243>`\__). - Added type hints to the `urllib3` module (`#​1897 <https://github.com/urllib3/urllib3/issues/1897>`\__). - Added `ssl_minimum_version` and `ssl_maximum_version` options which set `SSLContext.minimum_version` and `SSLContext.maximum_version` (`#​2110 <https://github.com/urllib3/urllib3/issues/2110>`\__). - Added support for Zstandard (RFC 8878) when `zstandard` 1.18.0 or later is installed. Added the `zstd` extra which installs the `zstandard` package (`#​1992 <https://github.com/urllib3/urllib3/issues/1992>`\__). - Added `urllib3.response.BaseHTTPResponse` class. All future response classes will be subclasses of `BaseHTTPResponse` (`#​2083 <https://github.com/urllib3/urllib3/issues/2083>`\__). - Added `FullPoolError` which is raised when `PoolManager(block=True)` and a connection is returned to a full pool (`#​2197 <https://github.com/urllib3/urllib3/issues/2197>`\__). - Added `HTTPHeaderDict` to the top-level `urllib3` namespace (`#​2216 <https://github.com/urllib3/urllib3/issues/2216>`\__). - Added support for configuring header merging behavior with HTTPHeaderDict When using a `HTTPHeaderDict` to provide headers for a request, by default duplicate header values will be repeated. But if `combine=True` is passed into a call to `HTTPHeaderDict.add`, then the added header value will be merged in with an existing value into a comma-separated list (`X-My-Header: foo, bar`) (`#​2242 <https://github.com/urllib3/urllib3/issues/2242>`\__). - Added `NameResolutionError` exception when a DNS error occurs (`#​2305 <https://github.com/urllib3/urllib3/issues/2305>`\__). - Added `proxy_assert_hostname` and `proxy_assert_fingerprint` kwargs to `ProxyManager` (`#​2409 <https://github.com/urllib3/urllib3/issues/2409>`\__). - Added a configurable `backoff_max` parameter to the `Retry` class. If a custom `backoff_max` is provided to the `Retry` class, it will replace the `Retry.DEFAULT_BACKOFF_MAX` (`#​2494 <https://github.com/urllib3/urllib3/issues/2494>`\__). - Added the `authority` property to the Url class as per RFC 3986 3.2. This property should be used in place of `netloc` for users who want to include the userinfo (auth) component of the URI (`#​2520 <https://github.com/urllib3/urllib3/issues/2520>`\__). - Added the `scheme` parameter to `HTTPConnection.set_tunnel` to configure the scheme of the origin being tunnelled to (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Added the `is_closed`, `is_connected` and `has_connected_to_proxy` properties to `HTTPConnection` (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Added optional `backoff_jitter` parameter to `Retry`. (`#​2952 <https://github.com/urllib3/urllib3/issues/2952>`\__) ## Changed - Changed `urllib3.response.HTTPResponse.read` to respect the semantics of `io.BufferedIOBase` regardless of compression. Specifically, this method: - Only returns an empty bytes object to indicate EOF (that is, the response has been fully consumed). - Never returns more bytes than requested. - Can issue any number of system calls: zero, one or multiple. If you want each `urllib3.response.HTTPResponse.read` call to issue a single system call, you need to disable decompression by setting `decode_content=False` (`#​2128 <https://github.com/urllib3/urllib3/issues/2128>`\__). - Changed `urllib3.HTTPConnection.getresponse` to return an instance of `urllib3.HTTPResponse` instead of `http.client.HTTPResponse` (`#​2648 <https://github.com/urllib3/urllib3/issues/2648>`\__). - Changed `ssl_version` to instead set the corresponding `SSLContext.minimum_version` and `SSLContext.maximum_version` values. Regardless of `ssl_version` passed `SSLContext` objects are now constructed using `ssl.PROTOCOL_TLS_CLIENT` (`#​2110 <https://github.com/urllib3/urllib3/issues/2110>`\__). - Changed default `SSLContext.minimum_version` to be `TLSVersion.TLSv1_2` in line with Python 3.10 (`#​2373 <https://github.com/urllib3/urllib3/issues/2373>`\__). - Changed `ProxyError` to wrap any connection error (timeout, TLS, DNS) that occurs when connecting to the proxy (`#​2482 <https://github.com/urllib3/urllib3/pull/2482>`\__). - Changed `urllib3.util.create_urllib3_context` to not override the system cipher suites with a default value. The new default will be cipher suites configured by the operating system (`#​2168 <https://github.com/urllib3/urllib3/issues/2168>`\__). - Changed `multipart/form-data` header parameter formatting matches the WHATWG HTML Standard as of 2021-06-10. Control characters in filenames are no longer percent encoded (`#​2257 <https://github.com/urllib3/urllib3/issues/2257>`\__). - Changed the error raised when connecting via HTTPS when the `ssl` module isn't available from `SSLError` to `ImportError` (`#​2589 <https://github.com/urllib3/urllib3/issues/2589>`\__). - Changed `HTTPConnection.request()` to always use lowercase chunk boundaries when sending requests with `Transfer-Encoding: chunked` (`#​2515 <https://github.com/urllib3/urllib3/issues/2515>`\__). - Changed `enforce_content_length` default to True, preventing silent data loss when reading streamed responses (`#​2514 <https://github.com/urllib3/urllib3/issues/2514>`\__). - Changed internal implementation of `HTTPHeaderDict` to use `dict` instead of `collections.OrderedDict` for better performance (`#​2080 <https://github.com/urllib3/urllib3/issues/2080>`\__). - Changed the `urllib3.contrib.pyopenssl` module to wrap `OpenSSL.SSL.Error` with `ssl.SSLError` in `PyOpenSSLContext.load_cert_chain` (`#​2628 <https://github.com/urllib3/urllib3/issues/2628>`\__). - Changed usage of the deprecated `socket.error` to `OSError` (`#​2120 <https://github.com/urllib3/urllib3/issues/2120>`\__). - Changed all parameters in the `HTTPConnection` and `HTTPSConnection` constructors to be keyword-only except `host` and `port` (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Changed `HTTPConnection.getresponse()` to set the socket timeout from `HTTPConnection.timeout` value before reading data from the socket. This previously was done manually by the `HTTPConnectionPool` calling `HTTPConnection.sock.settimeout(...)` (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Changed the `_proxy_host` property to `_tunnel_host` in `HTTPConnectionPool` to more closely match how the property is used (value in `HTTPConnection.set_tunnel()`) (`#​1985 <https://github.com/urllib3/urllib3/issues/1985>`\__). - Changed name of `Retry.BACK0FF_MAX` to be `Retry.DEFAULT_BACKOFF_MAX`. - Changed TLS handshakes to use `SSLContext.check_hostname` when possible (`#​2452 <https://github.com/urllib3/urllib3/pull/2452>`\__). - Changed `server_hostname` to behave like other parameters only used by `HTTPSConnectionPool` (`#​2537 <https://github.com/urllib3/urllib3/pull/2537>`\__). - Changed the default `blocksize` to 16KB to match OpenSSL's default read amounts (`#​2348 <https://github.com/urllib3/urllib3/pull/2348>`\__). - Changed `HTTPResponse.read()` to raise an error when calling with `decode_content=False` after using `decode_content=True` to prevent data loss (`#​2800 <https://github.com/urllib3/urllib3/issues/2800>`\__). ## Fixed - Fixed thread-safety issue where accessing a `PoolManager` with many distinct origins would cause connection pools to be closed while requests are in progress (`#​1252 <https://github.com/urllib3/urllib3/issues/1252>`\__). - Fixed an issue where an `HTTPConnection` instance would erroneously reuse the socket read timeout value from reading the previous response instead of a newly configured connect timeout. Instead now if `HTTPConnection.timeout` is updated before sending the next request the new timeout value will be used (`#​2645 <https://github.com/urllib3/urllib3/issues/2645>`\__). - Fixed `socket.error.errno` when raised from pyOpenSSL's `OpenSSL.SSL.SysCallError` (`#​2118 <https://github.com/urllib3/urllib3/issues/2118>`\__). - Fixed the default value of `HTTPSConnection.socket_options` to match `HTTPConnection` (`#​2213 <https://github.com/urllib3/urllib3/issues/2213>`\__). - Fixed a bug where `headers` would be modified by the `remove_headers_on_redirect` feature (`#​2272 <https://github.com/urllib3/urllib3/issues/2272>`\__). - Fixed a reference cycle bug in `urllib3.util.connection.create_connection()` (`#​2277 <https://github.com/urllib3/urllib3/issues/2277>`\__). - Fixed a socket leak if `HTTPConnection.connect()` fails (`#​2571 <https://github.com/urllib3/urllib3/pull/2571>`\__). - Fixed `urllib3.contrib.pyopenssl.WrappedSocket` and `urllib3.contrib.securetransport.WrappedSocket` close methods (`#​2970 <https://github.com/urllib3/urllib3/issues/2970>`\__) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/owntracks/android). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi41Ni4wIiwidXBkYXRlZEluVmVyIjoiMzYuNjQuOCIsInRhcmdldEJyYW5jaCI6Im1hc3RlciJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have been looking at ConnectionPool's behavior in blocking mode and was confused by EmptyPoolError's message (it said the pool was full, which is sort of true, but needs a bit of explanation, which I added.
I was also confused by the absence of a FullPoolError if you try to _put_conn a conn in blocking mode. I added a FullPoolError.
I also added a couple of tests for blocking mode and one test for non-blocking mode.