Machine learning and artificial intelligence still can’t deliver accessible code when asked

(Read 444 times)

I wanted to check for myself it ChatGPT can help delivering more accessible code. And after multiple trials I gave up. The reason for it’s confident but wrong example code is clear to me and when you read this post it will also be clear to you.

With all the attention around ChatGPT (opens in new window) I must say I wanted to check it as well. I got some code from a coworker and saw immediate problems with it. It was looking extremely good until I checked what it really does and how it works though. To make it work I needed to fix a lot of things, but at first glance it seemed correct.

And that is the scary part. I would be really happy if it would work, don’t get me wrong. But I think that the plausible answers it offers, plausible at first glance, will potentially do more harm than good.

It is obvious that so called large language models have a lot of potential. I will not deny this fact. I loved the more theoretical answers on accessibility it provided. But code related answers were problematic. I even tested it with simpler algorithms for sorting arrays and needed dozens of prompt improvements to get correct answer. Again – not a problem if we are aware of it. But if we aren’t aware of it’s over-confidence, then we may do a lot of harm and get even more problems.

It’s confidence is amazing, then you check the code, line by line, run it and it wasn’t doing what you wanted. I used some hours to redefine my prompts and on the end I got a simple sorting algorithm to work. It helped me. But that was a simple and closed algorithm for sorting an array. When I tried to get an accessible widget by prompting ChatGPT I got the same confidence but quite poor or even totally wrong answers.

As mentioned – this can be very dangerous if people can’t really know the effects of the produced code. I totally understand the reasons though.

With 98% of all websites out there having accessibility issues it’s not weird that a lot of code examples out there have the same problems. And when we use the same pool of poor examples to teach artificial intelligence it should not come as a surprise when they can’t really deliver.

my reflection about the causes behind poor delivery by ChatGTP.

I don’t have a clue about what sources did ChatGPT consume, but the reason for it’s problems with wrong answers is for sure that – poor sources. When humans will know how to write code that will make things accessible the robots will be able to learn it as well.

Author: Bogdan Cerovac

I am IAAP certified Web Accessibility Specialist (from 2020) and was Google certified Mobile Web Specialist.

Work as digital agency co-owner web developer and accessibility lead.

Sole entrepreneur behind IDEA-lab Cerovac (Inclusion, Diversity, Equity and Accessibility lab) after work. Check out my Accessibility Services if you want me to help your with digital accessibility.

Also head of the expert council at Institute for Digital Accessibility A11Y.si (in Slovenian).

Living and working in Norway (🇳🇴), originally from Slovenia (🇸🇮), loves exploring the globe (🌐).

Nurturing the web from 1999, this blog from 2019.

More about me and how to contact me: