add Sigv4A support #2939

Velfi · 2023-08-22T21:18:38Z

Motivation

Description

This PR adds Sigv4a support along with many tests that were ported from the CRT Sigv4a project. It does not include:

An E2E test. We should create one, but doing so requires us to set up MRAP which is a good amount of work.
A fuzz test. While I wrote and ran one on my machine, I found the fuzz testing tool to be really difficult to use and it would take some work to include fuzz testing in CI.

Additionally, test Sigv4a is annoying because Sigv4a signing is non-deterministic. Because of this, the tests test signature generation by first signing a request, and then by verifying that signature (This is how the CRT tests this feature).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

github-actions · 2023-08-24T18:18:04Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

Velfi · 2023-08-24T19:26:24Z

~~currently blocked on #2948~~

github-actions · 2023-08-24T20:43:35Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-08-25T18:11:29Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-08-25T18:56:43Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-08-25T20:48:54Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-08-29T16:20:22Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-08-29T17:03:18Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

jdisanti

Super awesome that you got this working! Great work. I have a number of questions/suggestions, and it looks like there might have been some merge issues that resulted in deleted tests.

rust-runtime/aws-smithy-runtime/src/client/orchestrator/auth.rs

aws/sdk-codegen/src/test/kotlin/software/amazon/smithy/rustsdk/EndpointBuiltInsDecoratorTest.kt

aws/sdk-codegen/src/test/kotlin/software/amazon/smithy/rustsdk/TestUtil.kt

aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/SigV4AuthDecorator.kt

aws/rust-runtime/aws-sigv4/src/http_request/test.rs

aws/rust-runtime/aws-sigv4/src/http_request/sign.rs

aws/rust-runtime/aws-sigv4/Cargo.toml

aws/rust-runtime/aws-runtime/src/auth/sigv4a.rs

github-actions · 2023-08-31T14:55:16Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

ysaito1001

LGTM! I'll wait to approve until a changelog entry is available for the PR, which I suspect is not so short.

aws/rust-runtime/aws-sigv4/src/http_request.rs

Co-authored-by: ysaito1001 <awsaito@amazon.com>

github-actions · 2023-09-25T14:58:17Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-09-25T15:34:51Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-09-26T15:07:30Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-09-26T20:40:14Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-09-27T16:11:48Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

use fixed date in MRAP signature test

github-actions · 2023-09-27T18:32:37Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

Velfi · 2023-09-27T19:10:23Z

nice work on this! I think we're really close. I want to make sure some level of EC2 test / canary is tracked and prioritized.

#3006

rcoh · 2023-09-27T20:18:21Z

aws/rust-runtime/aws-sigv4/Cargo.toml

@@ -27,7 +27,7 @@ hmac = "0.12"
 http = { version = "0.2", optional = true }
 num-bigint = { version = "0.4", optional = true }
 once_cell = "1.8"
-p256 = { version = "0.13.2", features = ["ecdsa"], optional = true }


intentional downgrade?

Yeah, b/c it would otherwise conflict with the version of digest required by hmac

aws/rust-runtime/aws-types/src/region.rs

aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/SigV4AuthDecorator.kt

rcoh

approved — see inline comments, I don't think they're blocking but we should get to the bottom of them at some point.

rcoh · 2023-09-27T20:52:44Z

you need to add the breaking-change label and rerun semver checks

…pt-2

update feature gate helper to work on blocks instead of lines

github-actions · 2023-09-28T17:35:55Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

Velfi force-pushed the zhessler-sigv4a-attempt-2 branch from 26b9f64 to e8daee5 Compare August 24, 2023 16:48

Velfi added the blocked label Aug 24, 2023

Velfi force-pushed the zhessler-sigv4a-attempt-2 branch from e67e1a4 to 294ebe2 Compare August 25, 2023 19:22

Velfi removed the blocked label Aug 25, 2023

Velfi marked this pull request as ready for review August 25, 2023 20:20

Velfi requested review from a team as code owners August 25, 2023 20:20

Velfi added the needs-sdk-review label Aug 29, 2023

jdisanti reviewed Aug 30, 2023

View reviewed changes

implement sigv4a signing

b411f99

Velfi force-pushed the zhessler-sigv4a-attempt-2 branch from 1e540e4 to b411f99 Compare September 18, 2023 20:43

ysaito1001 reviewed Sep 19, 2023

View reviewed changes

aws/rust-runtime/aws-sigv4/src/http_request.rs Outdated Show resolved Hide resolved

aws/rust-runtime/aws-sigv4/src/http_request.rs Outdated Show resolved Hide resolved

Apply suggestions from code review

5fb2c38

Co-authored-by: ysaito1001 <awsaito@amazon.com>

garryod mentioned this pull request Sep 22, 2023

[targeting] GetObject pre-signed url is incompatible with ONTAP S3 API DiamondLightSource/xchemlab#108

Open

fix unused import

465e4f5

Velfi removed the needs-sdk-review label Sep 25, 2023

Velfi added 2 commits September 25, 2023 08:55

Merge branch 'main' into zhessler-sigv4a-attempt-2

b27e254

fix missing return

1c5d331

sync some formatting with main

9f92425

update code based on PR comments

a6476da

Velfi added 4 commits September 27, 2023 10:38

fix clippy lints

f354eac

re-wrap the benchmark black box

48f52df

add CHANGELOG entry

2260a36

Merge branch 'main' into zhessler-sigv4a-attempt-2

ed5cf3e

fix doc link

3f3ea58

use fixed date in MRAP signature test

rcoh reviewed Sep 27, 2023

View reviewed changes

aws/rust-runtime/aws-types/src/region.rs Outdated Show resolved Hide resolved

rcoh reviewed Sep 27, 2023

View reviewed changes

aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/SigV4AuthDecorator.kt Outdated Show resolved Hide resolved

rcoh approved these changes Sep 27, 2023

View reviewed changes

Velfi added 2 commits September 28, 2023 11:20

Merge remote-tracking branch 'origin/main' into zhessler-sigv4a-attem…

3284915

…pt-2

remove SigningRegionSet::from_static

99d00ce

Velfi added the breaking-change This will require a breaking change label Sep 28, 2023

Velfi added 3 commits September 28, 2023 12:07

update AuthSchemeLister for smithy endpoint lib changes

3c72bd6

update feature gate helper to work on blocks instead of lines

Merge branch 'main' into zhessler-sigv4a-attempt-2

0e02f56

run formatter

9801294

Velfi enabled auto-merge September 28, 2023 17:45

Velfi added this pull request to the merge queue Sep 28, 2023

Merged via the queue into main with commit bc41828 Sep 28, 2023
40 of 41 checks passed

Velfi deleted the zhessler-sigv4a-attempt-2 branch September 28, 2023 18:16

Velfi added pending-release and removed needs-sdk-review labels Sep 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add Sigv4A support #2939

add Sigv4A support #2939

Velfi commented Aug 22, 2023 •

edited

github-actions bot commented Aug 24, 2023

Velfi commented Aug 24, 2023 •

edited

github-actions bot commented Aug 24, 2023

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

jdisanti left a comment

github-actions bot commented Aug 31, 2023

ysaito1001 left a comment

github-actions bot commented Sep 25, 2023

github-actions bot commented Sep 25, 2023

github-actions bot commented Sep 26, 2023

github-actions bot commented Sep 26, 2023

github-actions bot commented Sep 27, 2023

github-actions bot commented Sep 27, 2023

Velfi commented Sep 27, 2023

rcoh Sep 27, 2023

Velfi Sep 28, 2023

rcoh left a comment

rcoh commented Sep 27, 2023

github-actions bot commented Sep 28, 2023

add Sigv4A support #2939

add Sigv4A support #2939

Conversation

Velfi commented Aug 22, 2023 • edited

Motivation

Description

github-actions bot commented Aug 24, 2023

Velfi commented Aug 24, 2023 • edited

github-actions bot commented Aug 24, 2023

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

jdisanti left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 31, 2023

ysaito1001 left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 25, 2023

github-actions bot commented Sep 25, 2023

github-actions bot commented Sep 26, 2023

github-actions bot commented Sep 26, 2023

github-actions bot commented Sep 27, 2023

github-actions bot commented Sep 27, 2023

Velfi commented Sep 27, 2023

rcoh Sep 27, 2023

Choose a reason for hiding this comment

Velfi Sep 28, 2023

Choose a reason for hiding this comment

rcoh left a comment

Choose a reason for hiding this comment

rcoh commented Sep 27, 2023

github-actions bot commented Sep 28, 2023

Velfi commented Aug 22, 2023 •

edited

Velfi commented Aug 24, 2023 •

edited