Outdated NVDA Bot responses: "Open a modal dialog" (Modal Dialog Example, Test 1, V24.05.02)

[IsaDC]

Description of Behavior

The bot is currently operating on an older version of NVDA, which is known to display several undesirable behaviors. These issues have been resolved in the latest version (2024.1). I recommend that we either execute the test plan manually or update the bot to the most recent version.

Test Setup

• Test File: test-03-openModalDialog-nvda.collected.html
• AT: NVDA (Version 2023.3.3)
• Browser: Chrome (Version 124)

[css-meeting-bot]

The ARIA-AT Community Group just discussed Feedback: "Open a modal dialog" (Modal Dialog Example, Test 1, V24.05.02).

The full IRC log of that discussion

<jugglinmike> Subtopic: Feedback: "Open a modal dialog" (Modal Dialog Example, Test 1, V24.05.02)
<jugglinmike> github: https://github.com//issues/1062
<jugglinmike> IsaDC: the output that was reported didn't match the version of NVDA that I was using
<jugglinmike> Matt_King: Did you change the output?
<jugglinmike> IsaDC: No, I wanted to bring it up here to learn what would be the most appropriate response
<jugglinmike> Matt_King: Maybe changing it isn't a good idea because the version is going to be the version that the NVDA Bot uses
<jugglinmike> Matt_King: So then we would have to analyze the output based on that older version
<jugglinmike> Matt_King: Is the newer output going to cause the test outcomes to be different from the older output?
<jugglinmike> IsaDC: Yes because NVDA fixed many issues that we observed before
<jugglinmike> IsaDC: For instance, it used to say "dialog" twice, but now, it does not
<jugglinmike> IsaDC: So the change in behavior is significant
<jugglinmike> Matt_King: jugglinmike: Can we have an update on the NVDA version? And if so, when?
<jugglinmike> jugglinmike: Yes, if all goes well, we should be able to do this update in less than a day.
<jugglinmike> jugglinmike: With any change to NVDA, there is a risk that the new version interferes with the functionality of the NVDA AT Driver Server which we are maintaining
<jugglinmike> jugglinmike: Since this is the first time we've updated NVDA for automation, we don't yet have a good idea of how likely it is for the new version to cause problems
<jugglinmike> jugglinmike: Over time, we will develop an intuition on that.
<jugglinmike> jugglinmike: For the moment, it will be wise to allow for delays that would come from troubleshooting problems like that
<jugglinmike> Matt_King: Okay. So, what should we do in the mean time?
<jugglinmike> IsaDC: The changes are quite significant
<jugglinmike> Matt_King: Okay, then maybe we should put the NVDA testing on hold for one week and then make a decision based on where we stand with the new version's availability
<jugglinmike> Matt_King: Let's use this issue as a base for asynchronous decision-making. James_Scholes can report on the new version of NVDA's compatibility with the NVDA AT Driver server, and jugglinmike can post an estimate on the timeline for updating ARIA-AT App with the new NVDA version
<jugglinmike> IsaDC: Sounds good

[gnarf]

The backend was updated to use 2024.1 on May 9: https://github.com/bocoup/aria-at-automation-nvda-builds/releases/tag/2024.1

[mcking65]

Thank you for the update!