Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix test iteration bug #39915

Merged
merged 10 commits into from
May 1, 2024

Conversation

alzimmermsft
Copy link
Member

@alzimmermsft alzimmermsft commented Apr 25, 2024

Description

Fixes a bug in azure-core-test where the test iteration may not be reset between test runs resulting in a case where a non-parameterized test is believed to have a test iteration that is based on a previously ran parameterized test. This change WILL break test recordings that were given the wrong iteration number. The From Source test run should catch these and will need to be fixed on this PR before it can be merged.

There is also a change to plug in a JUnit ParameterResolver so that TestBase can be instantiated with TestContextManager instead of TestInfo, which means that each test ran will have a completely unique state. Rather than the existing behavior where some information for the test run is global and others it is for the specific test instance.

Lastly, while reviewing test configurations I found a lot of tests inspecting for LIVE or PLAYBACK to determine if they should run. LiveOnly and PlaybackOnly annotations (copied from azure-storage-common) were added to act as condition evaluations to determine if a given test should run for a given testing mode.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@alzimmermsft alzimmermsft enabled auto-merge (squash) May 1, 2024 19:32
@alzimmermsft alzimmermsft merged commit 3ca785f into Azure:main May 1, 2024
148 checks passed
@alzimmermsft alzimmermsft deleted the AzTest_FixTestIterationTracking branch May 1, 2024 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Status: Done
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants