{"id":1519,"date":"2024-02-04T19:43:00","date_gmt":"2024-02-04T18:43:00","guid":{"rendered":"https:\/\/cerovac.com\/a11y\/?p=1519"},"modified":"2024-02-02T20:29:27","modified_gmt":"2024-02-02T19:29:27","slug":"testing-and-monitoring-accessibility-with-automatic-tools-will-get-less-effective","status":"publish","type":"post","link":"https:\/\/cerovac.com\/a11y\/2024\/02\/testing-and-monitoring-accessibility-with-automatic-tools-will-get-less-effective\/","title":{"rendered":"Testing and monitoring accessibility with automatic tools will get less precise"},"content":{"rendered":"\n

It seems that accessibility overlays seem more and more viable for some business owners, even if majority of accessibility specialists and lots of people with disabilities doubt they help (opens in new window)<\/a>.<\/p>\n\n\n\n

I still believe that they can fix some marginal, basic, things but often brake others, so accessibility overlays are often doing more damage than good<\/a>. And when some accessibility veterans merge or join accessibility overlay companies – it may mean that our efforts with automatic accessibility testing tools will get less precise and effective because of that. This may sound strange, especially when we consider the potentials with new technologies, but I really think that it can actually get worse even if it should get better.<\/p>\n\n\n\n

First and most important – Accessibility Conformance Testing (ACT) Rules group (opens in new window)<\/a> is doing an amazing work to add more and more rules that can be automated. As they are available to anybody they can also be used for bad purposes. It’s very easy to make a script that target all known accessibility tests and make the webpage look like there are not errors. This basically means that if somebody wants – they can use the same rules to trick automatic testing tools into thinking that there are no errors on the page<\/strong>. Adrian Roselli proved WebAIM’s WAVE got spoofed (opens in new window)<\/a> and I totally see that as a potential future practice in the overlay world.<\/p>\n\n\n\n

So we can expect that with time (and increased use of accessibility overlays) we might get much less realistic value from our automatic accessibility testing and monitoring<\/strong>. If we take for example WebAIMs Million project<\/a> – what if lot’s of pages on their list decide to use an accessibility overlay and what if this overlay tries to trick accessibility tests? It would appear that accessibility is drastically improved, even if that would be far from reality. The same can be written about other similar monitoring projects like European Union eGovernment pilot<\/a> and many other Web Accessibility Directive<\/a> and European Accessibility Act<\/a> driven monitoring.<\/p>\n\n\n\n

\n

Automatic accessibility testing and monitoring can get fooled and we might believe the statistics that will communicate improvements in accessibility on a large scale if we don’t know better.<\/p>\nMy reflection over the situation.<\/cite><\/blockquote>\n\n\n\n

It is for sure very simple to trick automatic accessibility testing tools when you know exactly what and how they can test for accessibility issues. So, instead of reality we might be served with twisted data in the near future<\/strong>, or even worse – we might already have false feeling of accessibility progress based on automatic tooling if sites use overlays that already spoof the situation as better than it really is.<\/p>\n\n\n\n

Not to mention all those automatically generated alternative texts (and “craptions” (captions that are no good)). Automatic tools can’t tell us if those are proper either. They can only tell us – “passed, we found that image has an alternative text and that it isn’t image.jpg or similar”.<\/p>\n\n\n\n

What can be done to prevent this?<\/h2>\n\n\n\n

Well, monitoring should actually check if a site has an overlay installed and either block it in a way or skip the testing, to prevent false results polluting the statistics.<\/p>\n\n\n\n

Some months ago, I made a simple tool to check for multiple overlays, it’s technically totally doable and not so difficult. It works well and it discovers over 20 accessibility overlays and widgets, and I run regular tests to make sure the logic is working. I am not open-sourcing it for obvious reasons<\/strong> – open sourcing detection methodology could be used to break the detection.<\/p>\n\n\n\n

The downside of this tool is that it takes a lot of time to detect if overlay is actually running as they often load asynchronously and our tools need to wait for them to be loaded before we can be certain.<\/p>\n\n\n\n

Nonetheless, it will be even more important to know how to test accessibility manually<\/strong>, and unfortunately creating more accessibility testing rules will potentially also mean that more of the failures will be hidden from automatic accessibility testing tools, if somebody decides so.<\/p>\n\n\n\n

Once again – we need even more accessibility awareness, knowledge and culture. When more people will be able to find (and prevent) accessibility issues we will also have more accessible digital products, and we won’t need shady “automagic” tools that often make things worse at the end.<\/p>\n","protected":false},"excerpt":{"rendered":"

Automatic testing, although limited, is useful for quick and bulk test of webpages. With current progress I would expect it to be more efficient, but such tests could easily be bypassed and we can get bad data.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[614,123,65],"tags":[285,248,549,376,128,282],"class_list":["post-1519","post","type-post","status-publish","format-standard","hentry","category-accessibility-innovation","category-accessibility-testing","category-future-of-a11y","tag-accessibility-audit","tag-accessibility-overlay","tag-automatic-accessibility-testing","tag-automatic-testing","tag-overlay","tag-webaims-million"],"_links":{"self":[{"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/posts\/1519","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/comments?post=1519"}],"version-history":[{"count":0,"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/posts\/1519\/revisions"}],"wp:attachment":[{"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/media?parent=1519"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/categories?post=1519"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cerovac.com\/a11y\/wp-json\/wp\/v2\/tags?post=1519"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}