Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource/aws_cloudformation_stack_set_instance: Prevents conflicting creation retries #41388

Merged
merged 5 commits into from
Feb 13, 2025

Conversation

gdavison
Copy link
Contributor

Description

The resource aws_cloudformation_stack_set_instance was incorrectly retrying creation operations that were already in progress. This was due to a combination of using the tfresource.RetryWhen function with a wait for completion in the nested function as well as a Retryer on the AWS service client itself.

The function tfresource.RetryWhen calls retry.StateChangeConf, which makes its calls based on a timer even if the previous call is still running. This led to subsequent calls returning an OperationInProgress error.

However, the AWS service client has a Retryer handler for OperationInProgress errors, which retries internally to the AWS client. If the CloudFormation stack takes a long time to complete, this could lead to these internally handled OperationInProgress errors to exceed the internal retry limit, leading to an error such as

Instance: operation error CloudFormation: CreateStackInstances, exceeded maximum number of attempts, 25

Relations

Closes #28675

References

Output from Acceptance Testing

In non-Organization account

% make testacc PKG=cloudformation TESTS=TestAccCloudFormationStackSetInstance_

--- PASS: TestAccCloudFormationStackSetInstance_Disappears_stackSet (83.88s)
--- PASS: TestAccCloudFormationStackSetInstance_basic (88.40s)
--- PASS: TestAccCloudFormationStackSetInstance_disappears (93.07s)
--- PASS: TestAccCloudFormationStackSetInstance_parameterOverrides (155.97s)

In Organization management account

% make testacc PKG=cloudformation TESTS=TestAccCloudFormationStackSetInstance_

--- PASS: TestAccCloudFormationStackSetInstance_Disappears_stackSet (76.12s)
--- PASS: TestAccCloudFormationStackSetInstance_basic (80.80s)
--- PASS: TestAccCloudFormationStackSetInstance_disappears (99.88s)
--- PASS: TestAccCloudFormationStackSetInstance_parameterOverrides (151.49s)
--- PASS: TestAccCloudFormationStackSetInstance_deploymentTargets (179.48s)
--- PASS: TestAccCloudFormationStackSetInstance_DeploymentTargets_emptyOU (214.29s)
--- PASS: TestAccCloudFormationStackSetInstance_concurrencyMode (241.26s)
--- PASS: TestAccCloudFormationStackSetInstance_operationPreferences (337.12s)

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
… create stack instances

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@gdavison gdavison requested a review from a team as a code owner February 13, 2025 18:19
Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added service/cloudformation Issues and PRs that pertain to the cloudformation service. prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. labels Feb 13, 2025
jar-b
jar-b previously approved these changes Feb 13, 2025
Copy link
Member

@jar-b jar-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: Jared Baker <jar-b@users.noreply.github.com>
@gdavison gdavison merged commit 7c61b60 into main Feb 13, 2025
35 checks passed
@gdavison gdavison deleted the b-cloudformation-retries-exceeded branch February 13, 2025 19:09
@github-actions github-actions bot added this to the v5.87.0 milestone Feb 13, 2025
terraform-aws-provider bot pushed a commit that referenced this pull request Feb 13, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Copy link

This functionality has been released in v5.87.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Feb 14, 2025
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 17, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
service/cloudformation Issues and PRs that pertain to the cloudformation service.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: aws_cloudformation_stack_set_instance resource failing while operation still in progress
2 participants