Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Regress] Rest of the assertion/code in async test case is not executed when verify() fails #2794

Closed
pawlakmaly opened this issue Jul 9, 2021 · 1 comment · Fixed by #2797
Closed

[Regress] Rest of the assertion/code in async test case is not executed when verify() fails #2794

pawlakmaly opened this issue Jul 9, 2021 · 1 comment · Fixed by #2797

Comments

@pawlakmaly
Copy link

Describe the bug

When any verify() assertion fails in async test case then the rest of the code in case is not executed at all. It used to work on NW 1.6.4.

Sample test

sampleTest.js

import { NightwatchBrowser } from "nightwatch";

module.exports = {
  after(browser: NightwatchBrowser) {
    browser.end();
  },
  "Browser Context": async function (browser: NightwatchBrowser) {
    await browser.url("https://www.ecosia.org/");
    await browser.waitForElementVisible("body");
    await browser.verify.titleContains("Ecosia");
    await browser.verify.visible("input[type=search]");
    await browser.setValue("input[type=search]", "nightwatch");
    await browser.verify.visible("button[type=submit]");
    await browser.click("button[type=submit]");
    await browser.verify.containsText(".mainline-results", "Nightwatch.js");
    await browser.verify.visible("SomeNotExistingElement");
    await browser.verify.containsText(".mainline-results", "Nightwatch.js")
  }
};

Run with command

$ nightwatch test/sampleTest.js --your-other-arguments-here

Verbose output

debug.log

In NW 1.7.7

√ Running Browser Context:

√ Element <body> was visible after 18 milliseconds.
√ Testing if the page title contains 'Ecosia' (4ms)
√ Testing if element <input[type=search]> is visible (22ms)
√ Testing if element <button[type=submit]> is visible (34ms)
√ Testing if element <.mainline-results> contains text 'Nightwatch.js' (105ms)
   NoSuchElementError: An error occurred while running .isVisible() command on <SomeNotExistingElement>: 
   {"sessionId":"33cb638a93d5496a81634dcb30963805","status":0,"value":[]}
   
× Testing if element <SomeNotExistingElement> is visible in 20000ms - expected "is visible" but got: "element could not be located" (20315ms)
    at Object.Browser Context (\playground\nightwatch\tests\dummyVerifyTest.js:16:30)


FAILED: 1 assertions failed, 1 errors and  5 passed (23.154s)
_________________________________________________

TEST FAILURE: 1 error during execution; 1 assertions failed, 5 passed (24.107s)

 × dummyVerifyTest
 – Browser Context (23.154s)
   Testing if element <SomeNotExistingElement> is visible in 20000ms - expected "is visible" but got: "element could not be located" (20315ms)
       at Object.Browser Context (\playground\nightwatch\tests\dummyVerifyTest.js:16:30)

  NoSuchElementError: An error occurred while running .isVisible() command on <SomeNotExistingElement>: 
   {"sessionId":"33cb638a93d5496a81634dcb30963805","status":0,"value":[]}

In NW 1.6.4

Running:  Browser Context

√ Element <body> was visible after 18 milliseconds.
√ Testing if the page title contains 'Ecosia' (20ms)
√ Testing if element <input[type=search]> is visible (29ms)
√ Testing if element <button[type=submit]> is visible (32ms)
√ Testing if element <.mainline-results> contains text 'Nightwatch.js' (104ms)
× Testing if element <SomeNotExistingElement> is visible in 20000ms - expected "is visible" but got: "element could not be located" (20255ms)
    at Object.Browser Context (\playground\nightwatch\tests\dummyVerifyTest.js:16:30) 

√ Testing if element <.mainline-results> contains text 'Nightwatch.js' (89ms)

FAILED: 1 assertions failed, 1 errors and  6 passed (23.355s)
_________________________________________________

TEST FAILURE: 1 error during execution; 1 assertions failed, 6 passed (24.215s)

 × dummyVerifyTest
 – Browser Context (23.355s)
   Testing if element <SomeNotExistingElement> is visible in 20000ms - expected "is visible" but got: "element could not be located" (20255ms)
       at Object.Browser Context (\playground\nightwatch\tests\dummyVerifyTest.js:16:30)

  NoSuchElementError: An error occurred while running .isVisible() command on <SomeNotExistingElement>: 
   {"sessionId":"3412b1881dac0580f11146fcd762fb46","status":0,"value":[]}

Configuration

nightwatch.json

nunit3_reporter = require('nightwatch-nunit3-reporter');

const reporter = nunit3_reporter.createReporter({
    output_folder: './reports/nunit3'
  });

module.exports = {

    reporter,
    'default': {
        isLocal: true
    },
    'integration': {
        isLocal: false
    },
    waitForConditionTimeout: 20000,
    retryAssertionTimeout: 20000,
    asyncHookTimeout: 60 * 1000,

    before: function (browser, done) {
        done();
    },
    // This will be run before each test suite is started
    beforeEach: function (browser, done) {
        browser.status(function (result) {
            done();
        });
    },
    // This will be run after each test suite is finished
    afterEach: function (browser, done) {
        browser.globals.baseUrl = undefined;
        done();
    }
};

Your Environment

Executable Version
nightwatch --version 1.7.7
npm --version VERSION
yarn --version VERSION
node --version VERSION
Browser driver Version
NAME VERSION
OS Version
NAME VERSION
@gravityvi gravityvi mentioned this issue Jul 12, 2021
Merged
@pawlakmaly
Copy link
Author

Can't wait for new version of NN with fix for that <3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

remove node promise rejection in case of error gravityvi/nightwatch
1 participant
@pawlakmaly