I have not been doing full-blown accessibility audits until recently. More about the details in some other post. But I did some audits for WCAG conformance of websites and native mobile apps and will share some views of mine about them from customer perspective.
SWOT analysis (Strengths, Weaknesses, Opportunities, and Threats, opens in new window) can be quite an useful tool in business, but in this blog post I will only concentrate on strengths and weaknesses, and leave opportunities and threats for other occasions. This is only my personal opinion and not an opinion of my employees or their clients.
Weaknesses of accessibility audits
I will go against my own title and start with the weaknesses first. What are the weak or even bad sides, what can be negative and what impact can it have on the business side will be the main points.
I will also try to describe how to make things better or at least what I would suggest to improve them.
So let’s dive in the weaknesses of accessibility audits.
It’s difficult and often impossible to cover everything
When doing a manual audit it’s often impossible to check absolutely everything. Imagine a large or extra large website with hundreds or thousands of URLs and mix of content. Even if we run an automatic test that crawls all of it’s URLs we would only get a part of the situation. To test everything manual is not possible for such sites.
That’s why we need to think smart when defining scope and that’s why we have the WCAG-EM (WCAG evaluation methodology) that can help us with scope definition.
Accessibility audit is only relevant for a short time
When we audit a website or app we detect it’s current state. It is possible and sometimes also very likely that situation changes the day after the audit. Depends on our content changes, feature and bug release cycles and so on. The point I want to make is that audit results only “measure” state of accessibility in a short time period. So it’s possible that the results are not the same the day after we got it. Not only possible but quite often also a fact.
It’s therefore key to act on audit at once. Share it, plan around it, delegate tasks, prioritize. The longer you wait the more out-of-sync the audit will become.
Audit will often find a lot of basic fails that could easily be prevented
I’ve seen this a lot – a lot of very trivial but serious accessibility failures that could easily be prevented with knowledge and proper routines. For example alternative texts, wrong ARIA, missing semantics, no captions for pre-recorded videos and so on. So paying for an audit just to see that you fail on the basics is not very efficient. It takes auditors time to document failures and fix suggestions. Time that would be better spent auditing advanced interactions and writing good suggestions for them.
So before you get yourself an audit you should really fix the so called “low hanging fruits” and do your best to make things as accessible as you can. It takes time to learn and time to test and implement, but it is an investment that will return huge interests over time.
Complex patterns will often fail on accessibility
Sometimes we need to make advanced user experiences and can quickly make things complex. Complex widgets, interactions, filters, forms are often the source of barriers and often lead to accessibility failures. Even well known patterns like search with autocomplete, carousels, parallaxes, tab navigation and so on can be made extremely inaccessible. Sometimes we need to release things fast and just take a well known dependency and implement it without testing it properly.
Making things less complex is good for everybody and makes accessibility simpler as well. So next time you want your custom styled drop-down just reach for the native select and voila – it will be most probably accessible out of the box.
Third parties are often not a part of the audit but can still make your product inaccessible
I get the build versus buy discussion. It can be way more efficient to outsource some parts of solutions and just buy / lease / subscribe to use them. But if we don’t ask the providers we may get totally inaccessible solutions that then destroy all of our accessibility efforts. Cookie consents, chat solutions, payment and logistic solutions, contact forms, image and video galleries and so on are now quite often only a third party that is integrated to our website or app.
We must take full responsibility for everything that we have on our website or in our app. This means that we must require third parties to be accessible. And we must not trust them when their sales or marketing says so. Ideally they must have an accessibility statement or VPAT (Voluntary Product Accessibility Template) that is easy to acquire or even better – publicly available. Then we have to check that what is written in it also makes sense and is up to date. Don’t trust, check.
Audits can’t always test end to end user journeys
Audits often don’t include whole user journeys as it’s not possible to test some things in production. Imagine taking a loan for a car or a house. Or even buying some expensive things. This should therefore always be agreed upon by both customer and auditor. Access to test environment with “full access” can easily solve this, but it has to be planned for in advance. At the same time test environments can be unstable and sometimes even broken, so it’s best to define a time period with stability for the time of audit.
Sometimes there are third parties involved here that do not allow testing and it can make auditing of the whole journey even more difficult. I suggest that it isn’t just assumed and that both customer and auditor agree on details before audit start.
Beware of quick and cheap audits
I’ve stumbled upon extremely cheap audits with next day results. They sound too good to be true and they are most likely exactly that – over-promising. Anybody can run a free accessibility test on a public website and they will probably find some issues. Adding some free performance and SEO report would look even better to customers that are not aware of free tools that can do all of that.
Running an automatic tool and presenting the results as absolute truth can often do more harm than good to our accessibility efforts. I’ve done quite some automatic testing with multiple tools, even made a tool that includes multiple tools (aXeSiA) and please believe me that false positives are a reality. At the same time anybody can run such audits for free and even get free advice on how to fix reported errors. So please don’t even think about them – they are free for everybody, sometimes wrong and what is most important – they don’t reveal the whole accessibility situation. Automatic testing can only discover up to approximately 30% of WCAG errors, needs human interpretation and is not enough without manual auditing done by an accessibility specialist.
Manual auditing of all 78 of WCAG 2.1 success criteria on both levels A and AA takes time and can’t be done in a day. That’s the reality. Depending on the complexity it takes weeks or sometimes even months to do a full audit based on WCAG evaluation methodology (WCAG-EM).
Strengths of accessibility audits
It was intentional that I added some strengths of accessibility audits when I wrote about the weaknesses, but now I will only focus on the good stuff.
I will try to be objective and add some value, but please let me know if you don’t agree or even if you agree. Accessibility field is also “forever learning”, so some veterans will probably have other views to add here.
External audits often discover important issues
I’ve done some product development for years and everybody involved were treating the product as a fact. It was always refreshing to get new folks with different views that improved the things we took for granted. It’s only natural, and user testing is probably the best demonstration of how “new brains” find things that were hidden before.
The same goes with external accessibility audits. They often find problems in parts of user interaction that can span over accessibility to plain old usability. It’s not just how it works for people with disabilities, it’s how it works for everybody. So having an external accessibility specialist test user journeys will almost certainly add value to the product and potentially make it simpler to use for everybody.
Accessibility audit report can be used for making better priorities and products
When we get the results of accessibility audit we usually also get some sort of prioritization – what seems to be most important and what must be fixed. This should be integrated into our product development lifecycle as soon as possible, before audit gets out of sync with the state of our website or app.
Having clear priorities of accessibility issues, together with business priorities will make the product better for everybody. Stakeholders can also benefit from better knowledge about accessibility and provide enough resources so that issues are actually resolved.
Getting stakeholders on board can be easier with good accessibility audit report
As mentioned in previous sections – external audits can also be used to open the minds of stakeholders – when used correctly. I will not go into the change management and psychology behind this, but it should be easier to convince stakeholders to dedicate resources and get things fixed.
Accessibility audit report can reveal deeper problems that span out of only websites and applications – for example customer center, snail mail, social media channels and similar – all effecting customer contact. And when we know that up to a quarter of customers may have a disability we should be able to show business oriented stakeholders the benefits of fixing accessibility and staying accessible.
Good accessibility audit reports can teach designers, developers and content providers a lot
If we only get a report with errors then it’s difficult. But a good accessibility audit report will also provide information about why errors are problematic for people with disabilities and also important – what are the potential solutions.
So a good accessibility audit report will not only list errors and what success criteria of WCAG they fail, but also how to fix them. And as we know – accessibility is multi-disciplinary – so whole team will benefit from reading it. Some examples come to mind:
- Designers should also learn about the HTML semantics and ARIA, they don’t need to go into details, but they should understand what is possible and what they can use for free when using HTML.
- Developers can get some content based issues and try to find good solutions for them – for example when content providers need to use additional language on a page that has another main language.
- Content providers can learn that sometimes they need to ask designers and developers for a feature to be able to make their content accessible, for example adding transcripts to a podcast.
A good accessibility audit can be also a learning tool, especially when whole team reads it and when they can try to map results to their roles and responsibilities.
It can also be used as a check-list for future work, so that different people take responsibilities based on their roles and the items in the audit.
Making accessibility statements is easy when we have a fresh accessibility audit report
Sometimes we order an accessibility audit so that we can then make the accessibility statement. In Europe we have the Web Accessibility Directive (WAD) that demands accessibility statements for all public sector websites and applications and if organizations take it seriously they will either have internal accessibility specialists do an audit or order an audit externally.
But I think that also private sector should make an accessibility statement for their products. It’s not only nice to have and can be good for business.
The first reason are of course people with disabilities that can check what kind of barriers they can expect. They will appreciate honesty and if accessibility efforts are there barriers will be gradually removed for them.
The second reason is that if we want to sell services (and products) to public sector we need to be accessible, to simplify it. Especially in Europe with the European Accessibility Act (EAA) soon to be implemented, also private sector will need to make accessibility integral. Those who sell to public sector need to provide accessibility in their services already.
Having a good and up-to-date accessibility audit report will make accessibility statements quite a simple task as we already have the report on problems and we only need to describe them in a way that accessibility statement wants – user friendly.
Conclusion – get an accessibility audit
You should get an accessibility audit too. But – first make sure that you are ready for it. Try to learn more about accessibility, try to implement it early in the lifecycle of your products and services. Educate designers, developers and content providers. Educate customer service, educate sales, educate marketing, HR, practically whole organization.
Fix problems as best as you can, test products and services with people with disabilities.
And then after you have done all you can – get an accessibility audit to catch what you have missed. If you are only starting with accessibility and don’t have the time for the education, implementation and testing – get an accessibility audit as well.
It can be also a good entry to finally fix what you didn’t even know is broken.
