Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement request ID access for SDK clients RFC #2129

Merged
merged 53 commits into from Feb 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
60afad8
Add `RequestId` trait
jdisanti Dec 19, 2022
c761516
Implement `RequestId` for generated AWS client errors
jdisanti Dec 19, 2022
1250e65
Move `RustWriter.implBlock` out of `StructureGenerator`
jdisanti Dec 20, 2022
ab14c6e
Create structure/builder customization hooks
jdisanti Dec 20, 2022
1ce44f8
Customize `_request_id` into AWS outputs
jdisanti Dec 20, 2022
b133cc9
Set request ID on outputs
jdisanti Dec 21, 2022
d1468c5
Refactor SDK service decorators
jdisanti Dec 22, 2022
d3d201d
Refactor S3's extended request ID implementation
jdisanti Dec 22, 2022
bd9d166
Update changelog
jdisanti Dec 22, 2022
3c21d82
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Jan 11, 2023
fa82661
Fix `codegen-core` tests
jdisanti Jan 11, 2023
8c09f45
Combine `Error` and `ErrorKind`
jdisanti Jan 12, 2023
70547bf
Add test for service error conversion
jdisanti Jan 11, 2023
65367e0
Fix request ID impl for service error
jdisanti Jan 11, 2023
7eb87a7
Fix event stream marshall/unmarshall generators
jdisanti Jan 18, 2023
e9eee00
Fix `codegen-core` tests
jdisanti Jan 18, 2023
d3c9c80
Refactor error generation to fix server tests
jdisanti Jan 18, 2023
b6d1ebb
Move error generators into `codegen-client` and fix tests
jdisanti Jan 20, 2023
a1a426b
Re-export `ErrorMetadata`
jdisanti Jan 20, 2023
daf7a38
Update changelog
jdisanti Jan 20, 2023
6438cc8
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Jan 20, 2023
ef0614f
Incorporate feedback
jdisanti Jan 25, 2023
5a7f1c1
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Jan 25, 2023
2338074
Add request IDs to trace logs
jdisanti Jan 25, 2023
cbff721
Fix doc link
jdisanti Jan 27, 2023
b038943
Fix the canary build
jdisanti Jan 27, 2023
f11225f
Simplify some error trait handling
jdisanti Jan 27, 2023
4ca4a92
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Jan 27, 2023
87e0fef
Fix tracing statement
jdisanti Jan 27, 2023
b4680d2
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Jan 27, 2023
6578643
Rename `ClientContextParamDecorator` to `ClientContextConfigCustomiza…
jdisanti Jan 27, 2023
f41fbae
Fix S3Control endpoint tests
jdisanti Jan 27, 2023
af52cd3
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Jan 27, 2023
69f367d
temp: Check if Gradle caching is causing the server codegen issue
jdisanti Jan 31, 2023
9e345fe
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Jan 31, 2023
164b2f0
Revert "temp: Check if Gradle caching is causing the server codegen i…
jdisanti Jan 31, 2023
e667dda
Fix Python server errors
jdisanti Jan 31, 2023
1bce442
Add deprecated alias to guide customers through upgrading
jdisanti Feb 1, 2023
63ca8b0
Rename `_meta` to `meta`
jdisanti Feb 1, 2023
44c1dd0
Rename the `ErrorMetadata` trait to `ProvideErrorMetadata`
jdisanti Feb 1, 2023
fe1ec5a
Rename `aws_smithy_types::Error` to `ErrorMetadata`
jdisanti Feb 1, 2023
9a5b47b
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Feb 1, 2023
afd7dde
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Feb 2, 2023
13297be
Fix doc link
jdisanti Feb 3, 2023
44ab428
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Feb 3, 2023
dd7c7ee
Minor fixes
jdisanti Feb 3, 2023
fbf08d7
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Feb 3, 2023
4d92c09
Fix test compile
jdisanti Feb 3, 2023
1805cee
Fix another doc link
jdisanti Feb 3, 2023
ea72017
Incorporate feedback
jdisanti Feb 9, 2023
41dff69
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Feb 9, 2023
f5f31e5
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Feb 9, 2023
9d74fcf
Merge remote-tracking branch 'origin/main' into jdisanti-request-id-a…
jdisanti Feb 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
164 changes: 164 additions & 0 deletions CHANGELOG.next.toml
Expand Up @@ -28,3 +28,167 @@ message = "Adds jitter to `LazyCredentialsCache`. This allows credentials with t
references = ["smithy-rs#2335"]
meta = { "breaking" = false, "tada" = false, "bug" = false }
author = "ysaito1001"

[[aws-sdk-rust]]
message = """Request IDs can now be easily retrieved on successful responses. For example, with S3:
```rust
// Import the trait to get the `request_id` method on outputs
use aws_sdk_s3::types::RequestId;
let output = client.list_buckets().send().await?;
println!("Request ID: {:?}", output.request_id());
```
"""
references = ["smithy-rs#76", "smithy-rs#2129"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "jdisanti"

[[aws-sdk-rust]]
message = """Retrieving a request ID from errors now requires importing the `RequestId` trait. For example, with S3:
```rust
use aws_sdk_s3::types::RequestId;
println!("Request ID: {:?}", error.request_id());
```
"""
references = ["smithy-rs#76", "smithy-rs#2129"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "jdisanti"

[[smithy-rs]]
message = "Generic clients no longer expose a `request_id()` function on errors. To get request ID functionality, use the SDK code generator."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems too harsh. Whitelabel clients should also be able to easily opt into retrieving request IDs without having to onboard onto the full SDK code generator --- perhaps the decorator could be enabled via a codegen config key?

Copy link
Collaborator Author

@jdisanti jdisanti Feb 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, a whitelabel client would already be using other parts of sdk-codegen for things like credentials, right? So those would be able to also use request ID after these changes. If there is a use case for pure/generic Smithy clients (generated by codegen-client) to have request ID, then it really should be in the Smithy spec.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use the terms whitelabel and generic interchangeably, to refer to clients produced by codegen-client, without the SDK decorators in sdk-codegen.

There are use cases for generic clients to retrieve request IDs from responses, services return unmodeled request IDs frequently.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add these kinds of features to sdk-codegen, but make them consumable in isolation via the smithy-build.json configuration. Essentially we want varying degrees of the AWS SDK.

references = ["smithy-rs#76", "smithy-rs#2129"]
meta = { "breaking" = true, "tada" = false, "bug" = false, "target" = "client"}
author = "jdisanti"

[[aws-sdk-rust]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably teach our changelog format a way so that an entry is meant to be duplicated for the AWS SDK.

message = "The `message()` and `code()` methods on errors have been moved into `ProvideErrorMetadata` trait. This trait will need to be imported to continue calling these."
references = ["smithy-rs#76", "smithy-rs#2129"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "jdisanti"

[[smithy-rs]]
message = "The `message()` and `code()` methods on errors have been moved into `ProvideErrorMetadata` trait. This trait will need to be imported to continue calling these."
references = ["smithy-rs#76", "smithy-rs#2129"]
meta = { "breaking" = true, "tada" = false, "bug" = false, "target" = "client"}
author = "jdisanti"

[[aws-sdk-rust]]
message = """
The `*Error` and `*ErrorKind` types have been combined to make error matching simpler.
<details>
<summary>Example with S3</summary>
**Before:**
```rust
let result = client
.get_object()
.bucket(BUCKET_NAME)
.key("some-key")
.send()
.await;
match result {
Ok(_output) => { /* Do something with the output */ }
Err(err) => match err.into_service_error() {
GetObjectError { kind, .. } => match kind {
GetObjectErrorKind::InvalidObjectState(value) => println!("invalid object state: {:?}", value),
GetObjectErrorKind::NoSuchKey(_) => println!("object didn't exist"),
}
err @ GetObjectError { .. } if err.code() == Some("SomeUnmodeledError") => {}
err @ _ => return Err(err.into()),
},
}
```
**After:**
```rust
// Needed to access the `.code()` function on the error type:
use aws_sdk_s3::types::ProvideErrorMetadata;
let result = client
.get_object()
.bucket(BUCKET_NAME)
.key("some-key")
.send()
.await;
match result {
Ok(_output) => { /* Do something with the output */ }
Err(err) => match err.into_service_error() {
GetObjectError::InvalidObjectState(value) => {
println!("invalid object state: {:?}", value);
}
GetObjectError::NoSuchKey(_) => {
println!("object didn't exist");
}
err if err.code() == Some("SomeUnmodeledError") => {}
err @ _ => return Err(err.into()),
},
}
```
</details>
"""
references = ["smithy-rs#76", "smithy-rs#2129", "smithy-rs#2075"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "jdisanti"

[[smithy-rs]]
message = """
The `*Error` and `*ErrorKind` types have been combined to make error matching simpler.
<details>
<summary>Example with S3</summary>
**Before:**
```rust
let result = client
.get_object()
.bucket(BUCKET_NAME)
.key("some-key")
.send()
.await;
match result {
Ok(_output) => { /* Do something with the output */ }
Err(err) => match err.into_service_error() {
GetObjectError { kind, .. } => match kind {
GetObjectErrorKind::InvalidObjectState(value) => println!("invalid object state: {:?}", value),
GetObjectErrorKind::NoSuchKey(_) => println!("object didn't exist"),
}
err @ GetObjectError { .. } if err.code() == Some("SomeUnmodeledError") => {}
err @ _ => return Err(err.into()),
},
}
```
**After:**
```rust
// Needed to access the `.code()` function on the error type:
use aws_sdk_s3::types::ProvideErrorMetadata;
let result = client
.get_object()
.bucket(BUCKET_NAME)
.key("some-key")
.send()
.await;
match result {
Ok(_output) => { /* Do something with the output */ }
Err(err) => match err.into_service_error() {
GetObjectError::InvalidObjectState(value) => {
println!("invalid object state: {:?}", value);
}
GetObjectError::NoSuchKey(_) => {
println!("object didn't exist");
}
err if err.code() == Some("SomeUnmodeledError") => {}
err @ _ => return Err(err.into()),
},
}
```
</details>
"""
references = ["smithy-rs#76", "smithy-rs#2129", "smithy-rs#2075"]
meta = { "breaking" = true, "tada" = false, "bug" = false, "target" = "client"}
author = "jdisanti"

[[smithy-rs]]
message = "`aws_smithy_types::Error` has been renamed to `aws_smithy_types::error::ErrorMetadata`."
references = ["smithy-rs#76", "smithy-rs#2129"]
meta = { "breaking" = true, "tada" = false, "bug" = false, "target" = "client"}
author = "jdisanti"

[[aws-sdk-rust]]
message = "`aws_smithy_types::Error` has been renamed to `aws_smithy_types::error::ErrorMetadata`."
references = ["smithy-rs#76", "smithy-rs#2129"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "jdisanti"
8 changes: 4 additions & 4 deletions aws/rust-runtime/aws-config/src/sts/assume_role.rs
Expand Up @@ -7,7 +7,7 @@

use aws_credential_types::cache::CredentialsCache;
use aws_credential_types::provider::{self, error::CredentialsError, future, ProvideCredentials};
use aws_sdk_sts::error::AssumeRoleErrorKind;
use aws_sdk_sts::error::AssumeRoleError;
use aws_sdk_sts::middleware::DefaultMiddleware;
use aws_sdk_sts::model::PolicyDescriptorType;
use aws_sdk_sts::operation::AssumeRole;
Expand Down Expand Up @@ -266,9 +266,9 @@ impl Inner {
}
Err(SdkError::ServiceError(ref context))
if matches!(
context.err().kind,
AssumeRoleErrorKind::RegionDisabledException(_)
| AssumeRoleErrorKind::MalformedPolicyDocumentException(_)
context.err(),
AssumeRoleError::RegionDisabledException(_)
| AssumeRoleError::MalformedPolicyDocumentException(_)
) =>
{
Err(CredentialsError::invalid_configuration(
Expand Down
5 changes: 4 additions & 1 deletion aws/rust-runtime/aws-http/src/lib.rs
Expand Up @@ -3,7 +3,7 @@
* SPDX-License-Identifier: Apache-2.0
*/

//! Provides user agent and credentials middleware for the AWS SDK.
//! AWS-specific middleware implementations and HTTP-related features.

#![allow(clippy::derive_partial_eq_without_eq)]
#![warn(
Expand All @@ -28,3 +28,6 @@ pub mod user_agent;

/// AWS-specific content-encoding tools
pub mod content_encoding;

/// AWS-specific request ID support
pub mod request_id;
182 changes: 182 additions & 0 deletions aws/rust-runtime/aws-http/src/request_id.rs
@@ -0,0 +1,182 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0
*/

use aws_smithy_http::http::HttpHeaders;
use aws_smithy_http::operation;
use aws_smithy_http::result::SdkError;
use aws_smithy_types::error::metadata::{
Builder as ErrorMetadataBuilder, ErrorMetadata, ProvideErrorMetadata,
};
use aws_smithy_types::error::Unhandled;
use http::{HeaderMap, HeaderValue};

/// Constant for the [`ErrorMetadata`] extra field that contains the request ID
const AWS_REQUEST_ID: &str = "aws_request_id";

/// Implementers add a function to return an AWS request ID
pub trait RequestId {
/// Returns the request ID, or `None` if the service could not be reached.
fn request_id(&self) -> Option<&str>;
}

impl<E, R> RequestId for SdkError<E, R>
where
R: HttpHeaders,
{
fn request_id(&self) -> Option<&str> {
match self {
Self::ResponseError(err) => extract_request_id(err.raw().http_headers()),
Self::ServiceError(err) => extract_request_id(err.raw().http_headers()),
jdisanti marked this conversation as resolved.
Show resolved Hide resolved
_ => None,
}
}
}

impl RequestId for ErrorMetadata {
fn request_id(&self) -> Option<&str> {
self.extra(AWS_REQUEST_ID)
}
}

impl RequestId for Unhandled {
fn request_id(&self) -> Option<&str> {
self.meta().request_id()
}
}

impl RequestId for operation::Response {
fn request_id(&self) -> Option<&str> {
extract_request_id(self.http().headers())
}
}

impl<B> RequestId for http::Response<B> {
fn request_id(&self) -> Option<&str> {
extract_request_id(self.headers())
}
}

impl<O, E> RequestId for Result<O, E>
where
O: RequestId,
E: RequestId,
{
fn request_id(&self) -> Option<&str> {
match self {
Ok(ok) => ok.request_id(),
Err(err) => err.request_id(),
}
}
}

/// Applies a request ID to a generic error builder
#[doc(hidden)]
pub fn apply_request_id(
builder: ErrorMetadataBuilder,
headers: &HeaderMap<HeaderValue>,
) -> ErrorMetadataBuilder {
if let Some(request_id) = extract_request_id(headers) {
builder.custom(AWS_REQUEST_ID, request_id)
} else {
builder
}
}

/// Extracts a request ID from HTTP response headers
fn extract_request_id(headers: &HeaderMap<HeaderValue>) -> Option<&str> {
headers
.get("x-amzn-requestid")
.or_else(|| headers.get("x-amz-request-id"))
.and_then(|value| value.to_str().ok())
}

#[cfg(test)]
mod tests {
use super::*;
use aws_smithy_http::body::SdkBody;
use http::Response;

#[test]
fn test_request_id_sdk_error() {
let without_request_id =
|| operation::Response::new(Response::builder().body(SdkBody::empty()).unwrap());
let with_request_id = || {
operation::Response::new(
Response::builder()
.header(
"x-amzn-requestid",
HeaderValue::from_static("some-request-id"),
)
.body(SdkBody::empty())
.unwrap(),
)
};
assert_eq!(
None,
SdkError::<(), _>::response_error("test", without_request_id()).request_id()
);
assert_eq!(
Some("some-request-id"),
SdkError::<(), _>::response_error("test", with_request_id()).request_id()
);
assert_eq!(
None,
SdkError::service_error((), without_request_id()).request_id()
);
assert_eq!(
Some("some-request-id"),
SdkError::service_error((), with_request_id()).request_id()
);
}

#[test]
fn test_extract_request_id() {
let mut headers = HeaderMap::new();
assert_eq!(None, extract_request_id(&headers));

headers.append(
"x-amzn-requestid",
HeaderValue::from_static("some-request-id"),
);
assert_eq!(Some("some-request-id"), extract_request_id(&headers));

headers.append(
"x-amz-request-id",
HeaderValue::from_static("other-request-id"),
);
assert_eq!(Some("some-request-id"), extract_request_id(&headers));

headers.remove("x-amzn-requestid");
assert_eq!(Some("other-request-id"), extract_request_id(&headers));
}

#[test]
fn test_apply_request_id() {
let mut headers = HeaderMap::new();
assert_eq!(
ErrorMetadata::builder().build(),
apply_request_id(ErrorMetadata::builder(), &headers).build(),
);

headers.append(
"x-amzn-requestid",
HeaderValue::from_static("some-request-id"),
);
assert_eq!(
ErrorMetadata::builder()
.custom(AWS_REQUEST_ID, "some-request-id")
.build(),
apply_request_id(ErrorMetadata::builder(), &headers).build(),
);
}

#[test]
fn test_error_metadata_request_id_impl() {
let err = ErrorMetadata::builder()
.custom(AWS_REQUEST_ID, "some-request-id")
.build();
assert_eq!(Some("some-request-id"), err.request_id());
}
}
4 changes: 2 additions & 2 deletions aws/rust-runtime/aws-inlineable/src/lib.rs
Expand Up @@ -25,8 +25,8 @@ pub mod no_credentials;
/// Support types required for adding presigning to an operation in a generated service.
pub mod presigning;

/// Special logic for handling S3's error responses.
pub mod s3_errors;
/// Special logic for extracting request IDs from S3's responses.
pub mod s3_request_id;

/// Glacier-specific checksumming behavior
pub mod glacier_checksums;
Expand Down