There is a lot of writing going on on the subject of modern – machine learning and artificial intelligence powered – scripts that add an overlay on top of existing page. This overlays will then resolve all accessibility issues and even guarantee total WCAG 2.1 conformance on level AA. According to their marketing that is.
Install a simple script – in same way as for example external cookie consent provider – and get all of your accessibility issues fixed automatically…
this is the marketing promise for some accessibility overlays. But is it true?
I’ve read and seen videos about overlays from both sides – let us say classical accessibility experts on one side and overlay producers on the other side, so I would like to post some thought of mine and try to be objective as much as I can.
What is needed for a website to conform to WCAG actually?
I think this is the first question we must ask ourselves – and the answer should be acceptance criteria for the overlay. Logical, right – we have a problem, we have a potential solution and we must just check if solution addresses the problem correctly…
WCAG on level AA requires quite a lot, so if overlay should really fix all accessibility issues it should actually respect all of the WCAG requirements (success criteria), right?
An overlay should make all WCAG success criteria to pass for it to guarantee accessibility. So can it do it?
main thesis for overlay acceptance criteria.
We can also think about the overlay from the opposite direction – if overlay can fix all accessibility issues then it should also be able to detect them, right? When comparing different automatic tools for detection of accessibility errors we get an approximate level of their detection and it is between 20 to 40%. This basically means that there is currently not possible to even detect all errors automatically.
This is a potential “show stopper” for the overlays then. How can they fix all errors if it is not possible to even detect them? Maybe they use some more advanced and proprietary methodology that artificial intelligence and machine learning can bring to the table? It would be possible in theory, but very likely not possible at the moment. But let’s continue with analysis anyways.
Automatic tools can detect from approx. 20 to max. 40% of all breaches of WCAG success criteria. So an overlay solution would have to detect 100% of them if it wanted to fix them. Maybe they use a hidden methodology much more advanced that is known to the public?
first show-stopper is automatically detecting all (!) breaches of WCAG. If a tool is not aware of the problem – how will it fix it?
There are some WCAG success criteria that are very easy to be checked for by a script. For example missing language tag, “onClick” event on a div or span that should really be a button and the list goes on and on. But there are also success criteria that are not easily detectable by automatic tools and static code analysis. And if we again return to best automatic accessibility tools out there we can quickly see what is possible and what not.
Machine learning and artificial intelligence to the rescue
When we start to discuss machine learning and artificial intelligence and their practical usage in the field of accessibility we must first mention alternative text generation. It is quite easy to train models that detect motives in almost any image today. It is also one of the most evolved machine learning usages today – object detection, face recognition, the list goes on and on. So – we can argue that a modern solution can provide a description for a image that can then be used as alternative text. So editors do not have to make their own alternative texts any more? Or do they?
To be able to answer this question we have to understand what a good alternative text is. Does it help to describe a image in same way if it is and if it is not wrapped in a link? No, it doesn’t. But OK, it is easy to check if image is wrapped in a link and then use appropriate image alternative text. Sure. But – does the algorithm take surrounding context into consideration or does it just describe the image as an independent content? Because placement of images withing the context is required for a good alternative text. Otherwise we can totally miss the point of the image. There are also text sentiment analysis possibilities that can help, but can they really replace the human author? I’ve seen it quite often – they fail totally if we are sarcastic for example. So I do not believe machine learning can really provide correct alternative text reliably. Sure – it can help to a point, but it is the author – a human – that needs to choose best alternative text for the image and think of the context and maybe even further.
An image can tell a thousand words. And a computer can recognize what is in the image. But can it also provide an alternative text that is suitable for the context as well? Can it really replace the authors intention with the image? I am certain that it is to fail often. More often that it should, at least.
current state of machine learning is still a long way from providing relevant alternative texts for the images. They do a marvelous job for image recognition but they can not provide author’s intention with enough confidence.
There are some advances in the machine learning field that can be used to detect common design patterns and therefore also test their accessibility. The field is quite young now but I’ve tested some beta tools that use new methodologies and they for sure look promising. But we must be honest – they are still in an early phase. So maybe after enough reinforced learning and so on. But it is very early now. Too early I would say. There is a lot of progress and I will not estimate when it will be possible but I can establish that we need quite a leap if we look at the current possibilities.
WCAG requires also captions and transcripts for audio-video. That means that if you embed a video and captions are not provided by default or added manually by author – and overlay must provide captions to assure your video is accessible. I think that we can agree that – again – automatic, computer generated, captioning works very poor, especially for non-english languages and if you have a dialect then it is even worse. This means again that an overlay can not guarantee accessible alternative texts for a video or audio. And to my knowledge they are not even trying to do so.
Another important aspect of the web accessibility are forms. I have developed and tested a lot of forms they can get very complex very soon. Think dependent fields, inline validation that works for assistive technologies, asynchronous messages from backend and so on. If overlay is to make the site accessible then it should also fix the broken forms. Can it do that? I do not think so. It can maybe fix some labeling issues automatically, for example add labels that were previously missing and were possible to extract from for example placeholders or maybe names of the elements. But it can not fix broken functionality – like for example wrong validation and I doubt it can automatically detect correct focus management.
To conclude – it is not possible to fix all WCAG breaches that can occur
I think I have made some solid points with this analysis. I can go on and on and find abundance of examples of requirements that can not be fixed with automatic tools, even with machine learning and artificial intelligence – at least not on their current levels. Maybe some super-company can invest billions and use reinforced learning and teach tools to fix some common patters but it will for sure take many years it will be robust enough to really fix problems.
Overlay that can guarantee WCAG compliance is therefore far from possible when we look at the details and compare requirements against possible solutions.
I must be objective and confess that it is theoretically possible to fix some common errors automatically and automatic testing is also making progress but we can conclude that present overlays can not guarantee 100% WCAG compliance, not even on level A!
So – as I wrote on Twitter – we now have a new dimension of lies that can upgrade Mark Twains original quote (opens in new window):
Accessibility overlays may add a new dimension to lying: Lies, damned lies, statistics and overlies.
I made up a simple word game where overlay is misspelled as overlie because it is more than a lie.