can not set custom test method name when using data provider

[motivatedmind]

TestNG Version

7.10.2

Expected behavior

I should be able use a value from data provider and use it as a custom test name

Actual behavior

The Test method name appears in emailable report but not the custom name

Is the issue reproducible on runner?

Yes

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

package com.example.tests;

import org.testng.ITest;
import org.testng.ITestResult;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import java.lang.reflect.Method;

public class MyTest2 implements ITest {
    private String name;

    @BeforeMethod(description = "Set test case name", alwaysRun = true)
    public void beforeMethod(Method method, ITestResult result, Object[] data) {
        name = (String) data[0];
    }

    @DataProvider(name = "data-provider")
    public Object[][] dpMethod() {
        return new Object[][]{
                {"NYC"},
                {"Chicago"}
        };
    }

    @Test(dataProvider = "data-provider")
    public void myTest(String city) {
        System.out.println("city = " + city);
    }

    @Override
    public String getTestName() {
        return name;
    }

}

Emailable report:

Contribution guidelines

Incase you plan to raise a pull request to fix this issue, please make sure you refer our Contributing section for detailed set of steps.

[juherr]

This is supposed to work and it is covered by a test: https://github.com/testng-team/testng/blob/master/testng-core/src/test/java/test/name/ITestSample.java

Could you try with a ThreadLocal like the sample?

[motivatedmind]

I tried ThreadLocal as well but no difference. I also tried running ITestSample.java but same problem

[juherr]

That sounds more like a reporter issue. We need to check.

[krmahadevan]

@motivatedmind - Quick question. Have you ever seen this work in TestNG?

I ask because this has been like this for over 6 years now. Please see here

[krmahadevan]

@motivatedmind Also please can you point me to any place where there's a reference of this expectation. Just trying to ensure that if we deviated from a documented behaviour then we know about where its mentioned so in the documentation, so that when we fix the problem, we can point people to the documentation source, as a reason why the behaviour was changed.

[motivatedmind]

@motivatedmind - Quick question. Have you ever seen this work in TestNG?

I ask because this has been like this for over 6 years now. Please see here

@krmahadevan I am not aware if this has worked. But when I was looking for solutions on the internet, I saw people suggesting implementing ITest to achieve this.

[motivatedmind]

@motivatedmind Also please can you point me to any place where there's a reference of this expectation. Just trying to ensure that if we deviated from a documented behaviour then we know about where its mentioned so in the documentation, so that when we fix the problem, we can point people to the documentation source, as a reason why the behaviour was changed.

@krmahadevan Again I am not aware of reference. However in case of data provider same method is getting iterated over different data. So having same method name would make it difficult to understand which data the test failed for. It really puzzles me how people are handling this currently probably third party plugins like allure, extent etc. But at this point I am only looking basic report. It would be great if this gets fixed.

Alternately is there is any work around to get going?

[krmahadevan]

@motivatedmind - The emailable report that you are referring to, was never enhanced to make it aware of this capability.

But if you look at the report, you would notice that each of the methods are hyper links and when clicked they will take you to the section that contains the details of the parameter. See below screenshot. Would that help ?

Ideally speaking the ITest implementation IMO was meant to be used with factory powered test classes, because there would be many instances. I dont think it was meant to work properly with a data driven test.

The TestNG provided reports are pretty basic and perhaps don't have all the bells and whistles in terms of features. It would be good to adopt some of the 3rd party reports for reporting purposes or maybe use https://github.com/testng-team/reportng

[motivatedmind]

@krmahadevan I noticed that test method names are hyperlinks, and they take me to actual test case. But just that for non-technical users it sounds little confusing hence I am looking to change to custom name.

Also, I tried to alter name using reflection (I am not sure if this is right way). It works well in case of distinct test methods as below. I can also see custom name in index.html. But it doesn't work with single test method + data provider.

package com.example.tests;

import org.testng.ITest;
import org.testng.ITestResult;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
import org.testng.internal.BaseTestMethod;

import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.UUID;

public class MyDistinctTestMethods implements ITest {
    private String name;

    @BeforeMethod(description = "Set test case name", alwaysRun = true)
    public void beforeMethod(Method method, ITestResult result) throws NoSuchFieldException, IllegalAccessException {
        name = method.getName() + "_" + UUID.randomUUID();
        BaseTestMethod baseTestMethod = (BaseTestMethod) result.getMethod();
        Field field = baseTestMethod.getClass().getSuperclass().getDeclaredField("m_methodName");
        field.setAccessible(true);
        field.set(baseTestMethod, name);
    }

    @Test
    public void myTestMethod1() {
        System.out.println("myTestMethod1");
    }

    @Test
    public void myTestMethod2() {
        System.out.println("myTestMethod2");
    }

    @Override
    public String getTestName() {
        return name;
    }

}

Is there any further workaround I can do just so the names are fixed.

[krmahadevan]

@motivatedmind - Easiest would be for you to build your own reporting implementation (either from the scratch or you can extend the EmailableReporter2.java from TestNG and add the customisation to suite your needs.

[motivatedmind]

I created clone of EmailableReporter2.java. When I run in debug, I notice that IResult map receives the actual method name but not the altered name. Here what object would hold the altered name? Probably it is never getting stored?

[krmahadevan]

@motivatedmind - You would need to be changing that part, wherein instead of trying to alter the name of the test method from within your test/configuration, you basically construct the test method name using whatever parameters.

So in a nutshell, you would be doing:

• Is test a data driven test? If yes, then construct the method name by clubbing the parameter names along with the method name or choose something else as your naming strategy
• If test is a regular test, then go ahead and just use the method name as is.