I write this on 25th of December, so I think I am allowed to wish for something that I would like to think will help me and others that do accessibility audits. I am too old to believe that this tool will just wait me under the decorated tree next year at the same date, but I can still wish, can’t I?
Pun aside – I will try to describe the properties of a tool that would made my accessibility audits easier and more effective. Then, maybe, I will create a tool like that. Maybe I am already working on it, nevertheless, let’s go into specs.
Scope definition helper tool
When we do audits we need to start with a scope. Top sites from our user analytics solution, user journeys that site owners value most, some common sites like contact and careers and then add also some random sites to add some diversity. We can also parse a sitemap if the site has a public link to it or if we are a bit more advanced we can decide to crawl the site recursively and collect absolutely all links that have the same domain in them.
So WCAG EM (evaluation methodology) as a start plus some smart choices would give us a pretty decent scope. We would also need some links to documents like PDF to go a bit further (especially if working in Europe due to the Web Accessibility Directive (EN 301 549)).
This would be a good start. But to be even more efficient I would also like to collect more details than just URLs, like for example all the images, forms, tables, buttons, links, elements with ARIA attributes, elements that are generated within CSS, canvas elements, iframes and get simple statistics about their occurrence. Adding third party URLs to the picture would also be usable, especially if we would have to audit third parties as well (not so often but can happen). That would save me some time when I would like to check for some distinct problems for distinct success criteria.
With this huge list of links and some “weights” to make priorities simpler I would be way more efficient than when I have to check for things randomly. Top sites, top user journey links would of course come at the top, then sites with most elements that have potential for most problems. And elements that don’t have a semantic value but have JavaScript events on them.
With a tool that would produce a refined and prioritized scope like this I would be way more confident about capturing the most of testing surface. I think it would be the second best compared to being familiar with design style guides, components, site architecture and technical debt. So excellent for external audits but if we are a part of a team working in the known environment then such a tool would not be as valuable.
Automatic testing tool
I wrote a lot about automatic accessibility testing tools and will not repeat myself here, but I still do appreciate running automatic tools to discover obvious problems. So this audit helping tool’s second phase (or maybe interconnected with crawling to save time) would for sure be a helpful part.
Manual accessibility testing helper tool
Next step would be manual checks based on prioritized scope. A tool that would lead me through all the checks relevant, providing methods for testing, to make a consistent audit and with acceptance criteria for results would make auditing most effective.
WCAG on levels A and AA sum up to 50 success criteria at the time of writing this post, and some of them may go quite deep in the testing details, so it’s crucial to be consistent. It’s also crucial to document decisions made when auditing, to properly describe problems enriched with screenshots and meta data, sometimes videos or animated gifs to demonstrate problems.
So a tool would need to make sure I am consistent when testing and that I also document problems consistently and properly. Impact on end users, evaluating priorities so that project managers know what needs to be fixed first, code formatting, multimedia management, meta data needed, room for proper error remediation suggestions.
Another important part of this tool would be to re-check same things efficiently when we do another audit or check that things were fixed. Properly documenting them in the first place would make me more efficient when checking that they were fixed. The tool would also keep the history of previous audits as it is possible to have regressions and other, new problems when re-auditing.
Helper tool for reporting of results
We need to have quality data that can be used for multiple audiences;
- End users (impact of failures for them, to be used in accessibility statement and/or VPAT),
- Leaders (to get a birds view on the problems and to be informed quickly about the state),
- Project managers (priorities, estimations, any deadlines etc.),
- Designers,
- Developers,
- Content providers,
- Compliance personnel (level of compliance, deadlines,…),
- Sometimes even lawyers (depends on the goal of audit).
It depends on the wanted outcome, but we must at least guarantee that we document errors as much as possible and also prepare suggestions for remediation of them. That part would then also need meta data so that we could be able to export it to project management tools. Spellchecking, code checking. Possible assistive technology test details and similar improvements would also be appreciated.
The next part should help us with automatically prefilling accessibility statements and/or filling out our VPAT (Voluntary Product Accessibility Template).
Tool would also need to help with re-auditing, providing and safekeeping versions, and in some cases it would also need to provide special exports to different formats, for example password protected tagged (accessible) PDFs. Or maybe some kind of digital certification based on the content to prevent unwanted modifications.