Run accessibility audit on CI

[jelbourn]

Tools to explore:
Chrome tools: https://github.com/GoogleChrome/accessibility-developer-tools
aXe: http://www.deque.com/products/axe/

The audit should run on the e2e examples.

[sendilkumarn]

We can use Protractor-accessibility-plugin ?

adding this to protractor.conf.js

[sendilkumarn]

if this is okay. I will do a PR for this :) @jelbourn

[devversion]

I think that makes sense. I'm not in position to confirm this, but I think a PR is always welcome, if it's not too much effort for you.

btw: I'm working a PR which uses Axe Core. So our work will not interfere :)

[sendilkumarn]

Okay then 👍

[jelbourn]

In the long term we probably just want to go with one a11y assertion system, but it could be good to compare them initially.

[devversion]

@jelbourn I think both a11y assertion systems are powerful.

When I see the simplicity of the protractor plugin, I think this one is way easier to handle.

I already got Axe Core running on the e2e app as well.
Those A11Y checks can be triggered / started manually by adding a line to the given jasmine spec.

To get Axe Core running, we have to

• Copy it to the vendors for e2e and load it in the e2e
  / Or inject it manually on the given spec (recommended)
• Manually log our violations to the console or file (really difficult to show all necessary information)

Maybe @marcysutton can help in this case, and explain whats different about axe :)

[marcysutton]

The Protractor plugin will only run the A11y checks once at the end of a test and it won't let you specify an HTML fragment to test against (it tests the whole page). aXe, on the other hand, will allow you to run checks whenever the UI has changed within a set of tests and even test a DOM subtree.

The Chrome A11y Developer tools are also a smaller, noisier ruleset than aXe, and Tenon requires an API key and paid account.

Here are a few questions you should consider:

• What should be done with the log output? (We could easily write a utility for this)
• How will developers respond to accessibility issues? (for example, would developers run the tests and look at the debugger?)
• Would the issues go into an issue tracker?
• If you are analyzing whole pages, how do you plan to de-dupe the issues that are common from test-to-test? (None of the mentioned tools do this, but it's worth thinking about.)

[jelbourn]

I'd actually like to have tests fail when the a11y audit encounters an issue.

[marcysutton]

You can easily do that, it just might take some more logging to make the failure understandable. Here is an example: https://github.com/marcysutton/axe-webdriverjs-demo/blob/master/spec/test.js#L54

[jelbourn]

Yeah, that seems like a good area for some custom jasmine matchers.

[devversion]

@jelbourn I already got that working without the custom webdriver of the axe-core library.

It as bit messy at the moment, because it's just a WIP branch (on vacation)

• devversion@d2af0f8

I'm looking for some good designs, to represent the violations.

Colors would be appropriated, but will require libraries like chalk

[sendilkumarn]

Protractor tests fail when there are any accessibility issues.

[devversion]

Closing this, because #2010 just landed in master.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}