Let’s consider the following, quite modern situation, shall we:
Hey AI, make sure to make no mistakes and that it conforms to WCAG 2.2 on levels A and AA.
Verified design system in place and ready for AI consumption. Updated verified skills with all required resources, including bad and good examples – ready Progressive disclosure verified and working. Deterministic multilevel testing ready. Browser controlling command line interface ready and waiting for orders. Waiting for model to plan and adjust the results towards the goal.
Agents wildly writing to each-other, orchestrator dividing and conquering, using the best models for though thinking and more effective ones for smaller tasks. Getting real test coverage done and double verified.
Confirming Axe-core test passed, no issues, even when user interface control agent stress tested and rage clicked the interface, tried multiple different screen sizes, keyboard navigation simulation, went through all possible known product states in the app, across different pages and finally checked the happy path. 0 reported issues.
Screen reader simulation and accessibility tree snapshot analysis successful. Agent reports – we are done, everything works, code is conforming.
We are confident, harness seems to work. We ship. Stakeholders amazed. Competition behind us, a bit jealous, but sending likes.
We don’t know what we don’t know
But we didn’t know that there are monsters in the machine – digital monsters, immune to such silver bullets, monsters that laugh at harnesses. Hidden in the shadows of different versions of assistive technologies, accessibility APIs, browsers – and even more – in the combinations of all these. Hidden in the exceptions, the unwritten rules, the “it depends” situations.
Monsters that usually appear long after our agents confidently say we are good. Monsters that stop real people, or at least steal their time and energy. Monsters that can often mean people just go away, us thinking we don’t have any monsters at all.
Some of the monsters are very well known and when we use the good book of Web Content Accessibility Guidelines, we often get rid of most of them. When we know that harnesses and tools can only cover a smaller part of the Guidelines, that is.
When we know that Guidelines often fall into a context that can only be comprehensive to people that have experiences with the real world, the real machines, and not the statistical summarization of it.
But even the Guidelines itself are not enough for some monsters. Some monsters are always a step before the monster hunters that gather and write the Guidelines. And even a step before the new Guidelines that go way beyond the original Guidelines in some places.
So please, dear reader. Don’t be fooled by the promises of tools that say there are no monsters in the machine, even if they are expensive and very confident. When the tools are always steps behind the monsters and when the tools just know what they know and do not know what they do not know – we – humans need to step together.
The specialists among us, investigating monsters for years, and the people that fight the monsters each and every day are essential for us to have a chance of winning. Tools can be used for good, but we must know what good is in the first place. We can embrace the new tools and their possibilities to scale us in a good way. But we need to understand and know. Especially where tools have gaps. And that we still need to learn from each-other, not just rely on the tools.
