add proxy support

[dojiong]

support HTTP_PROXY/HTTPS_PROXY/ALL_PROXY environments

[Narsil]

Do you mind giving a bit more of motivation ?

This looks like just emulating Python requests which is definitely not desired.
IMHO proxy behavior belongs in user code, not manipulating env variables.

sync and async path need to be in sync too.

[dojiong]

I think hf-hub is not a simple request library, it's more like a tool to manage huggingface models.
It is more convenient to transparently support proxy via env in hf-hub than provide a proxy-api to downstream user.

async version will add soon(missed)

[Narsil]

Do you mind trying to make this land upstream: algesten/ureq#322

Seems reqwest already has support: seanmonstar/reqwest#1856

[dojiong]

Do you mind trying to make this land upstream: algesten/ureq#322

Seems reqwest already has support: seanmonstar/reqwest#1856

I'll try.

[Narsil]

TBH I didn't know this variable existed. Seems like a great way to exfiltrate traffic for any attacker..

I don't think env variables should be used for this IMHO, even if it does seems quite widespread.

[dojiong]

it's safe enough, because we use tls.

[dojiong]

upstream pr opened: algesten/ureq#649

[Narsil]

tls doesn't seem like a great justification. You're still intercepting all traffic.

requests doesn't seem to have a great record .... psf/requests#6071

[dojiong]

requests's CURL_CA_BUNDLE issuse tells about a empty-string-bug, it's not about env is insecure.

attackers could modify your traffic in many ways if they can change your env. proxy env or ca-bundle env is in the same security level.

[Narsil]

Really ? How ?

[dojiong]

If attackers can change your env, that would be the one of ways below

1. have your system's permission to write file (e.g. shell config file, scripts) or inject machine code in your process
2. you download a injected script/app to run
3. someone told you to run the command

all the ways above indicate that the attacker have got the file-write permission, then they could change your script/app to do everything, even got the root privilege.

[Narsil]

even got the root privilege.

This is called privilege escalation, it's not an easy feat on a decently setup machine.

user space global proxy seems like a pretty bad vector to me. In any case it's definitely not the purpose of this lib to do this.

[dojiong]

This is called privilege escalation, it's not an easy feat on a decently setup machine.

There's lots of things can do at normal user privilege, e.g. read $HF_HOME/token.

when your system is injected, that's not important your env would be changed, there be lots of ways to get/change data they want

[dojiong]

It's like a knife, just a tool. just like browser permits javascript be executed.