Dealing with requests 2.28+ and SSL

[ashic]

We're in a corporate network using zscaler on dev machines, and various ci servers using self signed certs. For non-prod environments, we've been using CURL_CA_BUNDLE='' to bypass ssl verification from requests. However, this is now deemed a bug (which it arguably should be) and from requests=2.28 onwards, this is no longer supported. As such, we can't download models using the huggingface_hub library.

Requests still supports disabling verification, but that's through an explicit verify parameter. While the _request_wrapper function of file_download.py does have a **params argument that's not passed from things like hf_hub_download and cached_download. As a result, if the latest version of requests is installed, there is no way to download a model using the libraries available.

In terms of resolution, I can think of a few approaches:

1. Add a verify parameter to top level functions that explicitly set the verify parameter to request.
2. Take a requests Session (or a session factory) as a parameter to top level functions, and use a passed in session instead of creating new ones automatically.
3. Support an environment variable that will be used to set verify on requests.

(For those experiencing this problem now, forcing a prior version via pip install "requests<2.28" will allow bypassing with the env var.)

[Wauplin]

Hi @ashic, I have looked a bit at your issue. I'm quite unsure yet if we want to add a verify argument to our top-level functions hf_hub_download and snapshot_download. Maybe I'm mistaken but the use case seems quite niche to me. As a solution already exists (pin version to requests<2.28), I'd prefer not to add complexity to huggingface_hub (that would need to be integrated in dependent libraries as well).

[Wauplin]

I'm closing this issue as "won't fix" as no news since 2 months (see above for reasons). Please open it again if needed.