Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use message based TestNG formatter #2863

Merged
merged 12 commits into from
Apr 5, 2024

Conversation

mpkorstanje
Copy link
Contributor

@mpkorstanje mpkorstanje commented Mar 25, 2024

🤔 What's changed?

Replace the TestNGFormatter with the message based implementation from
cucumber-junit-xml-formatter.
This achieves several goals:

  • Move the internal code base away from the events from the plugin module.
  • Extract common parts of Cucumber into modules that can be shared.

This results in several changes/improvements:

  1. Use the long naming strategy to include the scenario name, examples group and example numbers.
Feature: Examples Tables
  Scenario Outline: Eating cucumbers
    Given there are <start> cucumbers
    When I eat <eat> cucumbers
    Then I should have <left> cucumbers
    Examples: These are passing
      | start | eat | left |
      |    12 |   5 |    7 |
      |    20 |   5 |   15 |
    Examples: These are failing
      | start | eat | left |
      |    12 |  20 |    0 |
      |     0 |   1 |    0 |

This feature previously contained the examples

 - Eating cucumbers
 - Eating cucumbers_2
 - Eating cucumbers_3
 - Eating cucumbers_4

And will now contain the examples:

 - Examples Tables - Eating cucumbers - These are passing - Example #1.1
 - Examples Tables - Eating cucumbers - These are passing - Example #1.2
 - Examples Tables - Eating cucumbers - These are failing - Example #2.1
 - Examples Tables - Eating cucumbers - These are failing - Example #2.2
  1. Rename the suite from io.cucumber.core.plugin.TestNGFormatter to Cucumber
<suite name="Cucumber" duration-ms="61">
  <test name="Cucumber" duration-ms="61">
  1. When a scenario is skipped or failed the executed steps are always included in the
    message element.
<test-method name="Eating cucumbers - These are failing - Example #2.1" status="FAIL" duration-ms="7" started-at="1970-01-01T00:00:00.017Z" finished-at="1970-01-01T00:00:00.024Z">
  <exception class="AssertionError">
    <message><![CDATA[
Given there are 12 cucumbers................................................passed
When I eat 20 cucumbers.....................................................passed
Then I should have 0 cucumbers..............................................failed
]]></message>
  </exception>
</test-method>
<test-method name="Skipping from a step causes the rest of the scenario to be skipped" status="SKIP" duration-ms="5" started-at="1970-01-01T00:00:00.013Z" finished-at="1970-01-01T00:00:00.018Z">
  <exception class="The scenario has skipped step(s)">
    <message><![CDATA[
Given I skip a step.........................................................skipped
And a step that is skipped..................................................skipped
]]></message>
    <full-stacktrace>
      <![CDATA[The scenario has skipped step(s)]]>
    </full-stacktrace>
  </exception>
</test-method>
  1. The empty scenario and empty report are considered passing

To be consistent with the exit code of the Cucumber process empty scenarios
and test runs are considered passing.

🏷️ What kind of change is this?

  • ⚡ New feature (non-breaking change which adds new behaviour)

📋 Checklist:

@mpkorstanje mpkorstanje force-pushed the write-message-based-testng-formatter branch from 8c49089 to 66c56d9 Compare April 4, 2024 19:47
@mpkorstanje mpkorstanje changed the title Write message based testng formatter Use message based TestNG formatter Apr 5, 2024
@mpkorstanje mpkorstanje force-pushed the write-message-based-testng-formatter branch from b40f619 to b4319ea Compare April 5, 2024 01:22
@mpkorstanje mpkorstanje marked this pull request as ready for review April 5, 2024 13:47
@mpkorstanje mpkorstanje merged commit cf80f51 into main Apr 5, 2024
7 checks passed
@mpkorstanje mpkorstanje deleted the write-message-based-testng-formatter branch April 5, 2024 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant