Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow expected content-type to ignore parameters #3576

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

djedward
Copy link
Contributor

@djedward djedward commented Apr 11, 2024

Motivation and Context

An issue was raised about a mobile client that appends "; charset=utf-8" to the Content-Type when using restJson1. The latest RFC for "application/json" does not register a charset parameter, but indicates it is reasonable to accept it.

Description

This change loosens the validation of the expected content type to allow all parameters. I figured if the type and subtype match, we can leave it up to the parsing of the data to figure out whether it can read it. it is just as easy for a client to specify "application/json" and then provide invalid json data.

Testing

  • Add an additional test for the use case
  • ran the runtime and codegen tests
  • Tested with the example server, passing requests with/without the charset set.
  • Added a test for each protocol

Checklist

  • I have updated CHANGELOG.next.toml if I made changes to the smithy-rs codegen or runtime crates
  • I have updated CHANGELOG.next.toml if I made changes to the AWS SDK, generated SDK code, or SDK runtime crates

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@djedward djedward requested review from a team as code owners April 11, 2024 04:45
@djedward djedward force-pushed the main-contenttype branch 3 times, most recently from 109da20 to fdf218b Compare April 11, 2024 05:05
@@ -10,6 +10,15 @@
# references = ["smithy-rs#920"]
# meta = { "breaking" = false, "tada" = false, "bug" = false, "target" = "client | server | all"}
# author = "rcoh"

[[aws-sdk-rust]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be [[smithy-rs]]. Could you please also add "target" = "server" to the meta tags. The change log tool will automatically list changes for the server under a separate heading in the release CHANGELOG.md.

Thanks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I missed that. Should have picked up on it when adding the "target" had failed validation.

Copy link
Contributor

@david-perez david-perez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a protocol test for each of Smithy's protocols, given that content_type_header_classifier affects all protocols (i.e. Content-Type header comparison will now only take into account essence strings in all protocols).

We should contribute the test upstream to https://github.com/smithy-lang/smithy so all code generators benefit; to test it works against smithy-rs you can add it to codegen-core/common-test-models/rest-json-extras.smithy and confirm it passes locally with ./gradlew codegen-server-test:build -P modules='rest_json_extras'.

CHANGELOG.next.toml Outdated Show resolved Hide resolved
message = """
Content-Type header validation now ignores parameter portion of media types.
"""
references = ["smithy-rs#3576"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drganjoo found this old issue; apparently someone had identified this bug previously and we didn't notice because they adjusted the client-side! We should link it too: #3471

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added this and the tests requested above. I would still need to upstream to smithy-rs, but figured I'd get them here first and these can be the model for upstream.

@@ -257,6 +257,13 @@ mod tests {
assert!(matches!(result.unwrap_err(), MissingContentTypeReason::ToStrError(_)));
}

#[test]
fn valid_params_content_type() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
fn valid_params_content_type() {
fn valid_content_type_header_classifier_params() {

just to match the naming pattern of the other tests that also contain the name of the function being tested

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I matched it to the ones above that were doing something_something_content_type, but happy to rename.

@@ -107,7 +107,7 @@ fn content_type_header_classifier(
.parse::<mime::Mime>()
// `expected_content_type` comes from the codegen.
.expect("BUG: MIME parsing failed, `expected_content_type` is not valid. Please file a bug report under https://github.com/smithy-lang/smithy-rs/issues");
if expected_content_type != found_mime {
if expected_content_type != found_mime && expected_content_type != found_mime.essence_str() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I didn't know about essence_str(). Can't we simplify this to just the latter condition though?

Suggested change
if expected_content_type != found_mime && expected_content_type != found_mime.essence_str() {
if expected_content_type != found_mime.essence_str() {

We control expected_content_type and it's a &str with just the essence string. So if the latter pair are not equal the first pair should also not be equal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that originally, but was wondering if someone would ever want to force a specific check. It seemed unlikely, but added the original check back in, just in case. Given the discussion on it today, I think just the essence_str check is enough.

@david-perez david-perez added bug Something isn't working server Rust server SDK labels Apr 11, 2024
Copy link
Contributor

@david-perez david-perez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make a note / reach out to Smithy team about upstreaming the protocol tests.

@djedward
Copy link
Contributor Author

djedward commented May 8, 2024

I've written the upstream tests. I just need to reach out to the smithy team to figure out how the validate those tests across the implementations.

Edit: Discussed the process. Will get a PR for the tests and update this PR accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working server Rust server SDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants