hyper v1 upgrade #2059

seanmonstar · 2023-12-07T16:55:52Z

The upgrade process to hyper v1, which is tracked in #2039. This branch isn't compiling yet, which is why it's marked as a draft. But I'm doing it in pieces, and if someone wants to grab some while I'm distracted with other things, it's here.

Closes #2039

loispostula · 2024-01-12T20:04:32Z

@seanmonstar What still needs to be done, can we help?

With https://github.com/rustls/hyper-rustls/releases/tag/v%2F0.26.0 it should permit to progress

jakubadamw · 2024-01-28T13:39:26Z

@seanmonstar I added a bunch of commits to this fork: https://github.com/grafbase/reqwest/commits/hyper-v1-and-wasm-from-parts-build-split/. All but the newest one ("WIP") might be useful. Some might need adjustment, but the fork works for us. 🙂 The test suite isn't passing yet, but a lot of tests were green after the adjustment in the "WIP" commit.

seanmonstar · 2024-01-28T21:22:55Z

Thanks! If you wanna open a PR against this branch with the working commits, I'll merge it in ♥️

jakubadamw · 2024-01-29T10:03:40Z

@seanmonstar sure, of course! Here it is: #2115. They're a somewhat minimal set of changes that I had to make in order to get reqwest working for us with hyper v1, but they may be neither sufficient, nor necessarily entirely correct. They're also dependent on hyperium/hyper-util#95, which I opened separately. Kindly let me know if I can be of more help. 🙂

couchand · 2024-02-12T19:33:39Z

@seanmonstar Thanks for working on this! I'd like to draw your attention to the fact that the current draft commit seems to have omitted @jakubadamw's e-mail from the Co-authored-by: line.

seanmonstar · 2024-02-12T20:01:53Z

Yes you're right, I was fiddling trying to get all the authors, and was confused by the unverified badge, but got interrupted before settling it. It's sitting in an uncommitted state locally. Thanks for the reminder!

seanmonstar · 2024-02-19T15:16:57Z

Alright, everything else passes, I disabled the http3 feature. I could be convinced otherwise, but I'm currently leaning towards moving forward without http3, since it was "unstable" for exactly that reason. We can re-enable once it works again.

BiagioFesta · 2024-02-20T15:56:47Z

src/tls.rs

-                    rustls_pemfile::Item::X509Certificate(cert) => {
-                        certs.push(rustls::Certificate(cert))
+                    rustls_pemfile::Item::X509Certificate(cert) => certs.push(cert.into()),
+                    rustls_pemfile::Item::PKCS8Key(key) => {


upgrading rustls_pemfile would simplify this: https://github.com/seanmonstar/reqwest/pull/2139/files#diff-f229bc4b874c8edcd2c19a11fe9871e47d51519cf51ed829058f0bdbb1e4b27bR323

BiagioFesta · 2024-02-20T15:58:50Z

src/tls.rs

+
+    fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {
+        vec![
+            SignatureScheme::RSA_PKCS1_SHA1,


nit: sort alphabetically?

BiagioFesta · 2024-02-20T16:02:22Z

src/async_impl/client.rs

-                            });
-
-                        root_cert_store.add_trust_anchors(trust_anchors);
+                        root_cert_store.extend(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());


nit: probably better extend_with_slice: https://github.com/seanmonstar/reqwest/pull/2139/files#diff-0df5342f97493f9f55d6f43a14268f327ea9e791c7fdb8df6d88ec81f6baa721R469

Avoiding clone and thus additional moves I guess

src/connect.rs


    let mut buf = [0; 8192];
    let mut pos = 0;

    loop {
-        let n = conn.read(&mut buf[pos..]).await?;
+        let n = tokio_conn.read(&mut buf[pos..]).await?;


jakubadamw · 2024-03-07T12:28:19Z

@seanmonstar thanks for your work on this. Out of curiosity, is there any blocker remaining to this getting merged? 🙂

seanmonstar · 2024-03-07T13:15:31Z

The last thing before merging I need to figure out: releasing what's merged since the last release (and do I bother with deprecating/renaming builder options that have inconsistent names), and potential support plan for the "old" version. But some other higher urgency things have my attention this week.

This is a big and breaking change. A lot of the internals were worked on. But the breakage shouldn't be that visible to public API. The main differences are: - Publicly exposes `http` v1, instead of v0.2. - Integration with `hyper::Body` has changed. Co-authored-by: Sean McArthur <sean@seanmonstar.com> Co-authored-by: =?UTF-8?q?Lo=C3=AFs=20Postula?= <lois@postu.la> Co-authored-by: Jakub Wieczorek <jakub@grafbase.com>

seanmonstar · 2024-03-08T22:07:03Z

Alright, v0.11.25 is out, maybe there were some other things that could have been squeezed in, but oh well. Let's get this merged! 🚀

Venryx · 2024-03-09T16:40:16Z

To clarify for others: It looks like v0.11.25 was released prior to the merging of the pull-request that updates hyper to v1.

Reason this came to my attention: I tried updating to the "hyper v1 compatible" version of reqwest by adding this to my Cargo.toml:

reqwest = "0.11.25"

I waited for rust-analyzer to finish processing, then inspected the Cargo.lock file, and noticed these contents:

[[package]]
name = "reqwest"
version = "0.11.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0eea5a9eb898d3783f17c6407670e3592fd174cb81a10e51d4c37f49450b9946"
dependencies = [
 [...]
 "hyper 0.14.28",                <-- the unexpected line
 "hyper-rustls 0.24.2",
 "hyper-tls",
 [...]
]

So I guess a version of reqwest that depends on hyper v1 has just not been released yet?

If so, I'll just continue using GitHub as the crate source, but figured I'd mention this here for others who might have thought that v0.11.25 was supposed to support hyper v1.

For others: This is how you can include the "hyper v1 compatible" version of reqwest in your project: (add this to your Cargo.toml)

reqwest = {git = "https://github.com/seanmonstar/reqwest.git", rev = "2c11ef000b151c2eebeed2c18a7b81042220c6b0", <set your features here> }

This is a big and breaking change. A lot of the internals were worked on. But the breakage shouldn't be that visible to public API. The main differences are: - Publicly exposes `http` v1, instead of v0.2. - Integration with `hyper::Body` has changed. Co-authored-by: =?UTF-8?q?Lo=C3=AFs=20Postula?= <lois@postu.la> Co-authored-by: Jakub Wieczorek <jakub@grafbase.com>

michaelsproul mentioned this pull request Dec 8, 2023

Update dependencies to get rid of the yanked deps sigp/lighthouse#4994

Merged

dyc3 mentioned this pull request Dec 13, 2023

upgrade hyper to 1.0.0 dyc3/opentogethertube#1159

Closed

fredr mentioned this pull request Dec 15, 2023

Upgrade http crate to 1.0.0 EmbarkStudios/tame-oauth#68

Closed

seanmonstar force-pushed the hyper-v1 branch from 0fd22ac to 0dca943 Compare December 20, 2023 22:28

Xuanwo mentioned this pull request Dec 27, 2023

Tracking issues of upgrade hyper and its ecosystem to 1.0 apache/opendal#3643

Closed

2 tasks

seanmonstar mentioned this pull request Dec 29, 2023

Upgrade to hyper 1.0 #2039

Closed

5 tasks

jakubadamw mentioned this pull request Dec 30, 2023

Upgrade the http dependency to 1.0.0 cloudflare/workers-rs#429

Closed

seanmonstar mentioned this pull request Jan 1, 2024

Reading the body of a request #2078

Open

loispostula mentioned this pull request Jan 13, 2024

feat: bump rustls to 0.22 for upgrading to hyper-v1 #2091

Merged

seanmonstar force-pushed the hyper-v1 branch 6 times, most recently from cc21f90 to d187492 Compare January 17, 2024 16:58

dignifiedquire mentioned this pull request Jan 19, 2024

Update dependencies (rustls and axum-server) n0-computer/tokio-rustls-acme#5

Open

seanmonstar force-pushed the hyper-v1 branch 4 times, most recently from 9c3ce5a to 74dc685 Compare January 24, 2024 21:28

seanmonstar force-pushed the hyper-v1 branch 4 times, most recently from 4354bde to 68a683d Compare January 31, 2024 21:13

seanmonstar force-pushed the hyper-v1 branch 4 times, most recently from 4d0d28b to a819add Compare February 19, 2024 15:04

seanmonstar marked this pull request as ready for review February 19, 2024 15:15

seanmonstar changed the title ~~wip: hyper v1 upgrade~~ hyper v1 upgrade Feb 19, 2024

BiagioFesta reviewed Feb 20, 2024

View reviewed changes

yujincheng08 suggested changes Feb 21, 2024

View reviewed changes

src/connect.rs

let mut buf = [0; 8192];

let mut pos = 0;

loop {

let n = conn.read(&mut buf[pos..]).await?;

let n = tokio_conn.read(&mut buf[pos..]).await?;

This comment was marked as resolved.

Sign in to view

This comment was marked as off-topic.

Sign in to view

j0lol mentioned this pull request Mar 3, 2024

Upgrade hyper and http dependencies to 1.0 LemmyNet/activitypub-federation-rust#87

Open

wooden-worm mentioned this pull request Mar 6, 2024

Wasm build by adding default-client feature gate XAMPPRocky/octocrab#591

Merged

seanmonstar force-pushed the hyper-v1 branch from ef53459 to e14f972 Compare March 8, 2024 21:16

seanmonstar force-pushed the hyper-v1 branch from e14f972 to 539d952 Compare March 8, 2024 21:25

seanmonstar merged commit 2c11ef0 into master Mar 8, 2024
32 checks passed

seanmonstar deleted the hyper-v1 branch March 8, 2024 22:07

qthree mentioned this pull request Mar 12, 2024

revert: hyper v1 upgrade (for now) #2170

Merged

DDtKey mentioned this pull request Mar 15, 2024

[BUG][Rust][client][reqwest] Wrong representation of binary body (octet-stream) in request/response OpenAPITools/openapi-generator#18117

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hyper v1 upgrade #2059

hyper v1 upgrade #2059

seanmonstar commented Dec 7, 2023 •

edited

loispostula commented Jan 12, 2024

jakubadamw commented Jan 28, 2024

seanmonstar commented Jan 28, 2024

jakubadamw commented Jan 29, 2024

couchand commented Feb 12, 2024

seanmonstar commented Feb 12, 2024

seanmonstar commented Feb 19, 2024

BiagioFesta Feb 20, 2024

BiagioFesta Feb 20, 2024

BiagioFesta Feb 20, 2024

This comment was marked as resolved.

This comment was marked as off-topic.

This comment was marked as off-topic.

jakubadamw commented Mar 7, 2024

seanmonstar commented Mar 7, 2024

seanmonstar commented Mar 8, 2024

Venryx commented Mar 9, 2024

hyper v1 upgrade #2059

hyper v1 upgrade #2059

Conversation

seanmonstar commented Dec 7, 2023 • edited

loispostula commented Jan 12, 2024

jakubadamw commented Jan 28, 2024

seanmonstar commented Jan 28, 2024

jakubadamw commented Jan 29, 2024

couchand commented Feb 12, 2024

seanmonstar commented Feb 12, 2024

seanmonstar commented Feb 19, 2024

BiagioFesta Feb 20, 2024

Choose a reason for hiding this comment

BiagioFesta Feb 20, 2024

Choose a reason for hiding this comment

BiagioFesta Feb 20, 2024

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as off-topic.

This comment was marked as off-topic.

jakubadamw commented Mar 7, 2024

seanmonstar commented Mar 7, 2024

seanmonstar commented Mar 8, 2024

Venryx commented Mar 9, 2024

seanmonstar commented Dec 7, 2023 •

edited