Web Accessibility Directive (WAD) is kind of a big thing for digital accessibility in the European Union (EU) and actually also beyond (for example Liechtenstein, Norway, United Kingdom and Switzerland). Part of it’s requirements is also regular reporting about the state of accessibility of the public sector websites and mobile apps. For the first time after WAD was incorporated we can now check the first official monitoring reports (opens in new window). I invite you to check the countries of your interest, ideally in their original languages as they did not managed (yet?) to officially translate them to English. There are a lot of indicators and lessons learned in the reports and with this blog post I will try to give you a quick overview.
I am not a lawyer and my opinions are of my own, please read the reports for yourself to make a conclusion.
me, not accepting any legal consequences
23 countries have submitted their first time accessibility reports
I’ve used quite some time to read all of the reports, just to quickly find the core information that I think can be useful for anybody that is interested in the state of accessibility of public sector in the EU.
At the moment of writing this post there are only 23 countries that managed to publish their reports, others are still pending, but nevertheless I think these reports provide us with quite useful information. I’ve managed to read reports from the following countries;
- Austria
- Belgium
- Bulgaria
- Croatia
- Czechia
- Denmark
- Estonia
- Finland
- Germany
- Greece
- Hungary
- Italy
- Latvia
- Lithuania
- Luxembourg
- Malta
- Netherlands
- Poland
- Romania
- Slovakia
- Slovenia
- Spain
- Sweden
I am not sure what will be the final number as some countries do not belong to same deadlines (EFTA countries for example).
Scope and methods of monitoring
EU’s council on the accessibility defined the scope and baseline of monitoring (opens in new window) and also the sampling of the websites and mobile applications (opens in new window).
We should know the difference between an automatic accessibility audit and manual one to be aware of the needed effort. I write a lot about limitations of automatic accessibility testing on this blog, but I also try to make it clear that when you want to check some basics, especially code based problems, automatic testing is the first step and can be much more efficient than doing it manually (my open source aXeSiA tool of tools tries to do exactly that – automatically catch the code-based state of accessibility). So I totally understand that first time WAD will be monitored it will have much more “simplified” audits than “in-depth” audits. It has to do with resources that countries dedicated to WAD and it’s monitoring and I really hope that next report will include more “in-depth” audits.
My comment on scope and methods
I will not comment on sampling, it seems like a valid and statistically relevant approach but I must say that some countries failed to deliver due to limited resources. I will not call out any specifically as they are in minority, but that is an indication that can tell us quite a lot – some countries need more people to monitor accessibility of it’s public sector and that means that stakeholders (probably primarily politicians) need more awareness on the matter. Accessibility is not a checklist and a bureaucratic task!
I’ve seen that almost each country uses different methodologies and that makes me wonder why did they choose to do so. WCAG offers excellent evaluation methodology that is there to be used and trying to make it simpler may mean that we will have a difficulty comparing results between countries. It’s good that we can base the test criteria on standards like EN 301 549 and WCAG, at least.
The other thing that is not optimal is that countries use different automatic tools that give different results. I know that it is difficult to choose a single tool for all but it would make comparison much more viable. I know there were some projects that tries to map different tools and their positive and negative sides (like for example WAI-Tools) but it can be quite difficult to compare audits that are based on different tools. I tried to find more about this in a webinar with EU accessibility experts but did not get an concrete answer, only explanation that EU must not define the tools but only prescribe the methodology. I can understand that politically neutrality but at the same time as a accessibility specialist I still think it would make things much easier and more comparable. Time will show if maybe they will also listen do some member countries that would like to have a uniform tools as well.
Common accessibility failures
I did not do a comparation analysis but only collected all failures to establish the order of most failing issues that can be established for all of them at the same time. Based on automatic and manual tests I think we can conclude that situation is very similar as throughout the web;
- Failing contrasts (1.4.3)
- Missing alternative texts (1.1.1)
- Information and relations (1.3.1)
- Headings and labels (2.4.6)
- Parsing (4.1.1)
As the majority of tests were made with automatic tools and only a small part of audits were manual I don’t think this is surprising at all. Some countries reported them separately and some didn’t so I wonder… I would like to conclude that this is again a confirmation that accessibility is a “team sport” where designers, developers and content providers need to cooperate and help each other to succeed.
A developer should not fix poor contrast before consulting designer, designer can not fix poor code and content providers can break the most accessible pages by not knowing how to make their content accessible.
Lessons learned and between the lines
Multiple countries report that they are not having enough knowledge and/or personnel. This is maybe normal for first time of reporting after WAD was incorporated into legislation. It is also unfortunately even more common in private sector. They all have plans to do something about awareness and knowledge but I did not detect details on the personnel solutions.
Some countries also reached out to external companies that specialize in digital accessibility, and some even included different non-government organizations of people with disabilities. This is in my opinion the right step towards better accessibility – to include end users and experts and cooperate and learn from each-other.
There were quite a lot of public pages that were totally missing the required accessibility statement. That means again that the need for awareness and knowledge is crucial. Some of accessibility statements were there but non-compliant. That seems to again be a problem of missing knowledge and possibly not enough personnel.
Some countries reported on end user complaints and some did not. It is negatively surprising to me that reporting is not universal and that some countries did not include end user complaints. I think that one of the strategical goals of WAD is to give end-users answers and help. So to not include statistics of their questions and demands seems to totally miss on one of the most important aspects. To make things even more interesting – some complaints were reported to be not answered at all. I hope there are processes to improve on that.
Very few stakeholders make efforts to achieve good technical quality. This may also mean that their providers and agencies need to improve their deliveries. So I can detect that there is too little awareness and knowledge on both sides – stakeholders not requiring accessibility and providers not delivering it.
Almost no audited websites or apps were totally compliant. Larger organizations performed better than small and that has probably to do with resources and knowledge again. At the same time larger organizations probably also got some extra attention and probably also more demands from end users and coverage in media.
To no surprise I’ve even detected mentions about investing much more importance into graphic design than semantics, logical order and keyboard navigation. There were even conclusions that few actors really test their digital presence with assistive technologies and even keyboard alone. Again an indication about poor knowledge and awareness.
Again not a surprise – documents are way less accessible than mobile applications and websites. There are reports about abundance of scanned documents that are not converted to text form and therefore inaccessible to most assistive technologies. Some countries invest a lot into educating stakeholders to create more accessible documents and that is a good thing. I am not surprised about the fact that when personnel is used to scan documents directly to PDF and then attach them to the website it makes it very expensive to go back and run the same documents against some optical character recognition (OCR). Content providers must therefore get the crucial knowledge and improve their processes. At the same time supply chain needs to be involved – especially when document management and design is outsourced – the suppliers need to provide accessible documents before they can be distributed.
Some countries struggle with abundance of information that is still not translated to local languages. I guess that is a common problem with segmentation of languages in the EU that spans beyond WAD and accessibility and can sometimes even introduce problems when translating parts of texts to native languages which may not support the identical meanings.
As WCAG are really more web oriented but try to cover also technically independent aspects there are some confusions around what applies to apps and what to documents. EN 301 549 does try to solve that but when it comes to testing all the tools are mostly based on the WCAG, so auditors must map the relevant success criteria and it can for sure be time consuming if they do not have a verified system in place. This is again a knowledge problem that can and must be resolved with proper trainings and expert support.
Universities are getting involved in the local WAD and testing and that is a very positive impact. I’ve detected this practice is only done in some countries and I think this deserves to be a best practice that all countries will follow.
Some reports include obsolete language like “people with special needs”. This is an indication of poor knowledge to me. They should be familiar with different models of disabilities and I hope the people with disabilities would be used instead.
It was reported that some websites removed content after the WAD as to save the burden of making it accessible. This could be an interesting research in itself and will again have to be categorized to missing knowledge and personnel.
Some countries argued that they do not have the capacity to do all the work they are required to do, so I wonder if EU will try to map how many people is working with WAD per whole population. I hope there are plans to map this and maybe set a common ground for personnel as well.
Report included some statements that argued it is too time-consuming to comply with all the requirements and I guess that can make total sense for some existing organizations with thousands or even millions of urls and documents. When doing accessibility like this, reactively, it does seem an impossible task. Sometimes it is maybe clever to invest into risk assessment and maybe investigate cost-benefits toward new solutions that would make new content accessible.
Multiple languages in the EU also mean that there are almost no possibilities for machine learning powered automatic transcripts and captions. This is a big problem for some that makes it difficult to provide text alternatives for video and audio content. We know that automatic captioning is not good enough even for native English speakers but it would probably help a lot with manual interventions.
Manual audits presented a huge financial cost and some reported that simplified method auditing took from 12 to 29 hours for one website. That means that there should be enough qualified personnel to do the auditing consistently and I guess that tendency for automatic tests will rise even more. Maybe it will give good impacts on the development of ACT-rules.
Centralized content management systems and centralized design manuals and style guides were indicated as a potential solution and best practice and I can not agree more. For sure it would be positive from multiple aspects – unified branding, simpler maintenance and governance and of course – scalability. Time will show if countries will embrace this kind of thinking but personally I think it would potentially solve a lot of existing problems, even beyond accessibility.
Partnership between multiple countries (mostly countries that share their language) and is planning to develop common approaches like trainings, methodology and centralized platforms. This seems to be a initiative that could be led from the EU level down but looks like some countries went the other way.
WCAG success criteria tries to be technology agnostic and that can sometimes add to confusion. Some reports mentioned that current rules on compliance can often be perceived as very difficult to understand. Hopefully future WCAG versions (WCAG 3) will do more about that.
I’ve detected that some countries decided to cooperate with International Association of Accessibility Professionals and think that that is amazing. Inviting people with disabilities and accessibility experts to cooperate will for sure make end results most accessible possible and also bring positive effects for sustainable accessibility.
Final thoughts
Some additional conclusions and summary;
- Involving organizations of people with disabilities, accessibility experts and third parties soon and often will give best effects on the long run. I am also referring to the new user based evaluations that will for sure also be needed for WCAG 3 and are in a way also a part of the EN 301 549. Accessibility is for the users, so it makes total sense to me.
- I would love to see the unification of reporting, that would make comparison easier and also add to data quality for further statistics. I am currently not aware of such plans for the future but I guess that there are tendencies towards that.
- Countries will have to invest more into accessibility awareness and knowledge. And especially invest into experts that will be experienced enough in accessibility, so that they will be able to effectively monitor and counsel public sector needs. With European Accessibility Act soon in place it will be even more important to have enough resources.
- Common methodology and common testing tools would make comparison easier and cheaper. I am really hoping that unification will come in such a way that it will be possible to use the best ACT rules that have least false positives and that would also make possible to invest into even more ACT rules. Common automatic testing platform could also make a lot of sense and maybe even help to cover larger samples. I would maybe even go so far to say that we could have a central registry of all public sector domains (maybe they would deserve it’s own, reserved top level domain) so that there would not be any more doubts what is a public sector website and what is not.
