Skip to content
This repository has been archived by the owner on May 27, 2024. It is now read-only.

[trainer] Sanitize both identifier and test case to allow spaces in test name #1

Merged
merged 3 commits into from
Aug 2, 2023

Conversation

tahirmt
Copy link

@tahirmt tahirmt commented Aug 2, 2023

Dupe of fastlane#21432 so while upstream is updated we can unblock ourselves

Checklist

  • I've run bundle exec rspec from the root directory to see all new and existing tests pass
  • I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
  • I see several green ci/circleci builds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.

Motivation and Context

Description

If the test case name has spaces or any special characters, the reported number_of_failures will always be 0. This will also not include any failed tests inside the junit report.

The reason this was happening was because during the comparison to check the test failure there was a sanitization step that's not applied to the identifier itself so it never matched.

This is from the log output of such a test.

** TEST FAILED **
[15:11:03]: Exit status: 65
+--------------------+---+
|      Test Results      |
+--------------------+---+
| Number of tests    | 3 |
| Number of failures | 0 |
+--------------------+---+

[15:11:04]: Test execution failed. Exit status: 65

Testing Steps

The easiest way to reproduce it was to use https://github.com/Quick/quick because Quick 7 produces test case names with spaces if the spec names had spaces in it. And when you run scan the output will always say the number of failures is 0.

I have attached a sample project.

  1. Download project
  2. Run bundle install
  3. Run bundle exec fastlane test

When it runs the output will show number of failures as 0.

Then you can uncomment the branch line in the Gemfile and run bundle install and switch to this branch. Then you will see that the number of failure is 1.

SpaceTests.zip

@tahirmt tahirmt merged commit e28cb1c into master Aug 2, 2023
@tahirmt tahirmt deleted the fix-trainer-not-reporting-failures branch August 2, 2023 20:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
3 participants