Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request] Provide a way for .a11ytest files to be retained even if a scan produces no errors #935

Closed
renishnair-viper opened this issue Jun 2, 2023 · 6 comments
Closed

[Feature Request] Provide a way for .a11ytest files to be retained even if a scan produces no errors #935

renishnair-viper opened this issue Jun 2, 2023 · 6 comments
Labels
status: resolved This issue has been merged into main.

Comments

@renishnair-viper
Copy link

In IScanner.Scan() method output file of .a11ytest is only created when there is an error.
There is a note added (Note: no output files will be written if no errors were found.)
It should be based on some settings if there is no error but we want to review the file later it is not possible at the moment.
So it should be based on some settings if the user wants a file each time to be generated whether there is an error or not they can set the setting and get the file.
image
@microsoft-github-policy-service microsoft-github-policy-service bot added the status: new This issue is new and requires triage by DRI. label Jun 2, 2023
@marcelwgn
Copy link
Contributor

Out of curiosity, do you mind me asking what processing you are doing with the a11ytest result file and how it would help having a result even if there were no errors? Is it about the screenshot that is part of it?

@renishnair-viper
Copy link
Author

We have created an automation unit test case for our project using axe-windows. Suppose we are running the automation unit test every week after development changes and sometimes we encounter an error if we are having the last working file without any error we can compare and keep track of what minor or major changes have affected so we are getting error.

@DaveTryon
Copy link
Contributor

I have a diagnostic branch (see changes) that adds this functionality, but it's on the ConfigBuilder instead of IScanner.Scan. It's not production-ready (it would still need unit tests, documentation, and CLI support), but we'll triage this and see how it lands with respect to other work that our team owns.

@DaveTryon DaveTryon added status: ready for triage This issue is ready to be triaged by the Accessibility Insights team. and removed status: new This issue is new and requires triage by DRI. labels Jun 5, 2023
@microsoft-github-policy-service
Copy link
Contributor

This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!

@DaveTryon DaveTryon added status: ready for work This issue is ready to be worked on. and removed status: ready for triage This issue is ready to be triaged by the Accessibility Insights team. labels Jun 5, 2023
@DaveTryon
Copy link
Contributor

We agree that this makes sense. Plan is to control the setting via the ConfigBuilder, as implemented in the branch mentioned above. Relatively low priority compared to other things currently on our backlog. A community contribution might help it to get implemented more quickly.

@DaveTryon DaveTryon changed the title [Feature Request] In IScanner.Scan method always create a output file or it should be based on some settings. [Feature Request] Provide a way for .a11ytest files to be retained even if a scan produces no errors Jun 5, 2023
@DaveTryon DaveTryon self-assigned this Jul 4, 2023
@DaveTryon DaveTryon added the active (Dev) The assigned dev is working to perform the work. label Jul 4, 2023
@DaveTryon DaveTryon mentioned this issue Jul 4, 2023
Merged
1 task
DaveTryon added a commit that referenced this issue Jul 5, 2023
@DaveTryon
feat: Add ability to always save test file (#948)
11982cb 
#### Details

#935 requests that we provide a way for CLI users to create an
`.a11ytest` file even when no errors are reported. This adds a
`--alwayssavetestfile` option to the CLI, then plumbs it through to take
effect. People using the automation interface can access this directly
via the `Config.Builder.WithAlwaysSaveTestFile` method.

##### Motivation

Address #935 

##### Context

<!-- Are there any parts that you've intentionally left out-of-scope for
a later PR to handle? -->

<!-- Were there any alternative approaches you considered? What
tradeoffs did you consider? -->

#### Pull request checklist
<!-- If a checklist item is not applicable to this change, write "n/a"
in the checkbox -->
- [x] Addresses an existing issue: #935

---------

Co-authored-by: Dave Tryon <45672944@DaveTryon@users.noreply.github.com>
@DaveTryon DaveTryon added status: resolved This issue has been merged into main. and removed status: ready for work This issue is ready to be worked on. labels Jul 5, 2023
@DaveTryon DaveTryon removed their assignment Jul 5, 2023
@DaveTryon DaveTryon removed the active (Dev) The assigned dev is working to perform the work. label Jul 5, 2023
@codeofdusk
Copy link
Contributor

Released as part of 2.2.0. Closing as complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status: resolved This issue has been merged into main.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@codeofdusk @marcelwgn @DaveTryon @nang4ally @renishnair-viper