Test case failure #3093
Merged
Test case failure #3093
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixing failure for the test test.retryAnalyzer.RetryAnalyzerTest.ensureRetryDoesntRunEndlesslyForDataDrivenTests Github actions failure on `master` branch: https://github.com/testng-team/testng/actions/runs/8307524777/job/22736819681 Gradle scan results link: https://scans.gradle.com/s/wap5a5bvq3qgq/tests/task/:testng-core:test/details/test.retryAnalyzer.RetryAnalyzerTest/ensureRetryDoesntRunEndlesslyForDataDrivenTests?top-execution=1 Prerequisites for failure: The below JVM arguments should be used: * `-Duser.timezone="America/New_York”` * `-Duser.country=RU` * `-Duser.language=ru` * `-XX:+UnlockExperimentalVMOptions` * `-XX:hashCode=2` Root cause: We are using hashCode() to create Key that is used to determine if a RetryAnalyzer instance should be created or not. When we run the tests using `-XX:hashCode=2` it causes the JVM to generate the same hashcode (value 1) And this messes up the `RetryAnalyzer` object creation. Fix: Addressed this by wrapping the parameters into an object which will contain a unique id that can be used to represent the same set of parameters.
WalkthroughThe recent modifications focus on enhancing efficiency and reliability in handling test parameters within TestNG. By introducing a Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (2)
- testng-core/src/main/java/org/testng/internal/BaseTestMethod.java (3 hunks)
- testng-core/src/main/java/org/testng/internal/IObject.java (2 hunks)
Additional comments: 3
testng-core/src/main/java/org/testng/internal/IObject.java (1)
- 122-148: The introduction of the
IdentifiableArrayObjectclass is a strategic move to address the issue with hash code manipulation under specific JVM settings. This class ensures that each set of parameters is uniquely identified, which is crucial for the correct operation of theRetryAnalyzerin data-driven tests. The implementation ofequalsandhashCodemethods based on the parameters array is appropriate for this use case. However, consider documenting the potential impact of usingArrays.hashCode(parameters)inhashCodeandArrays.equals(parameters, that.parameters)inequalsfor large arrays, as these operations could become performance bottlenecks in certain scenarios.testng-core/src/main/java/org/testng/internal/BaseTestMethod.java (2)
- 836-836: The modification to use a
ConcurrentHashMapfor cachingIRetryAnalyzerinstances based on a unique parameter ID is a robust solution to the problem of hash code manipulation under specific JVM settings. This change ensures that retries are handled correctly for parameterized tests, addressing the core issue of endless retries in data-driven tests. The use ofparameterIdas a key is a clever approach to uniquely identify test method invocations.- 849-852: The introduction of the
parameterIdmethod, which generates a unique ID for test parameters, is a key part of the solution to ensure the correct operation of theRetryAnalyzerin the face of hash code manipulation. By cachingIdentifiableArrayObjectinstances in aConcurrentHashMap, this method efficiently handles parameter uniqueness. This approach significantly improves the reliability of retries in parameterized tests.
juherr
approved these changes
Mar 17, 2024
krmahadevan
deleted the
fix_test/RetryAnalyzerTest_ensureRetryDoesntRunEndlesslyForDataDrivenTests
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixing failure for the test
test.retryAnalyzer.RetryAnalyzerTest.ensureRetryDoesntRunEndlesslyForDataDrivenTestsGithub actions failure on
masterbranch:https://github.com/testng-team/testng/actions/runs/8307524777/job/22736819681
Gradle scan results link:
https://scans.gradle.com/s/wap5a5bvq3qgq/tests/task/:testng-core:test/details/test.retryAnalyzer.RetryAnalyzerTest/ensureRetryDoesntRunEndlesslyForDataDrivenTests?top-execution=1
Prerequisites for failure:
The below JVM arguments should be used:
-Duser.timezone="America/New_York”-Duser.country=RU-Duser.language=ru-XX:+UnlockExperimentalVMOptions-XX:hashCode=2Root cause:
We are using
hashCode()to create Key that is used to determine if aRetryAnalyzerinstance should be created or not.When we run the tests using
-XX:hashCode=2it causes the JVM to generate the same hashcode (value 1) And this messes up the
RetryAnalyzerobject creation.Fix:
Addressed this by wrapping the parameters into an object which will contain a unique id that can be used to represent the same set of parameters.
Summary by CodeRabbit