You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the "Test Plan Run" page presents Testers uses a two-state checkbox to convey the state of each assertion verdict, e.g.
Which statements are true about the response to f (browse mode on)?
Role 'group' is conveyed
The value of the assertion verdict is actually indeterminate in two cases, making a two-state checkbox insufficient:
When human Testers begin manually running a test and have not yet reported whether or not the assertions are satisfied.
When a "bot" tester has collected AT responses for the Test Plan Run and a Test Admin has re-assigned the Test Plan Run to a human Tester
In effect, the "unchecked" state ambiguously describes two different states: "this assertion is not satisfied," and "this assertion has not yet been evaluated."
To date, this ambiguity has been tolerable because the human Tester is the sole arbiter of assertion verdicts. They tacitly track their own progress through the Test Plan Run.
However, the ambiguity will become more confusing as the system gains the ability to automatically re-use verdicts when it recognizes historic test results. In those cases, a human Tester who receives a Test Plan Run from a Bot will not be able to recognize which assertions have been recognized as "failed" and which have not been automatically judged by the system.
The text was updated successfully, but these errors were encountered:
Please implement using Yes/No radios like we have for undesirable behaviors. This will give all the questions the same format.
If neither radio is checked, the verdict is not assigned. On submit, make sure all verdicts are assigned.
For accessibility, ensure that each assertion is a radio group that is labeled by the assertion. Also, please don't use a table element in the layout of the radio buttons and assertions.
Currently, the "Test Plan Run" page presents Testers uses a two-state checkbox to convey the state of each assertion verdict, e.g.
The value of the assertion verdict is actually indeterminate in two cases, making a two-state checkbox insufficient:
In effect, the "unchecked" state ambiguously describes two different states: "this assertion is not satisfied," and "this assertion has not yet been evaluated."
To date, this ambiguity has been tolerable because the human Tester is the sole arbiter of assertion verdicts. They tacitly track their own progress through the Test Plan Run.
However, the ambiguity will become more confusing as the system gains the ability to automatically re-use verdicts when it recognizes historic test results. In those cases, a human Tester who receives a Test Plan Run from a Bot will not be able to recognize which assertions have been recognized as "failed" and which have not been automatically judged by the system.
The text was updated successfully, but these errors were encountered: