What if your accessibility issue isn't a WCAG failure?
Spoiler: Nearly everyone logs them. The argument is about where, and whether logging them does harm.
I asked LinkedIn what auditors do with problems that don’t fail a WCAG 2.2 AA criterion. 73 people voted, 18 commented, and the split was lopsided. Here’s what they said, where they disagreed, and what it means for how we build the report.
I'm very curious about the way that other people test, largely because I think it's easy to get stuck in a certain way of working without really considering that there may be better ways of doing things. So I've started asking the community how they go about their accessibility work.
Last week I asked a question on LinkedIn about what you do about accessibility issues that can potentially obstruct or inhibit a person from using an interface but might not strictly be WCAG conformance issue, such as a skipped heading level on every page in the sample. Or maybe there's something which is a minor bug but appears so frequently that it's very annoying for users. What do you do with it?
There were four options and 73 people voted:
- Log it, marked as best practice: 78%.
- Put it in a separate section of the report: 17%.
- Mention it outside the audit: 1%.
- It’s out of scope, leave it: 2%.
So 95% of the people who answered record these issues somewhere in the audit, and nearly four in five put them inline with everything else, flagged so that it's clear that it falls outside WCAG AA. The poll reached just over a thousand people, mostly accessibility specialists, consultants, testers and engineers, with a third of them senior.
Usability is accessibility
The majority view in the comments was short and unapologetic. One accessibility lead said they'd flag the skipped heading and mark it as a recommendation, because usability is accessibility. A consultant said missing headings create conceptual difficulties when you can’t see the content, so of course they flag them, in the report. A specialist who also designs interfaces said skipped headings turn up in just about every test case they've run, WCAG or not, and they flag them every time.
One commenter widened it beyond accessibility. Testing a page, they find broken links, typos and duplicated content, and have to decide what to do with those too. Their system is built on WCAG, so they add them as best practice so they get fixed along with the accessibility work (assuming that everything in the report gets fixed). That pretty much matches what I do: anything I’d be uncomfortable overlooking goes in the report, and sometimes I’ll email the client about it while I’m looking at it to make sure it doesn't get missed. To be fair, clients often deal with critical and high issues first and work down through the list, and "advisory" issues sometimes don't get a look in. But logging them means that they are recorded and if something comes up in the future such as a complaint, you know you've done your job.
The case against
The most-liked dissent came from a well-known specialist: if it’s not a WCAG failure, it’s not a WCAG failure. Whether it’s worth reporting depends on the project, and those side quests are best tackled once the product actually conforms. When I asked whether this was about not distracting from the criteria, he said he’d seen teams prioritise fixes that weren’t failures over real barriers that WCAG had caught. Another specialist simply replied “agreed”.
It's quite a strong argument, but I'd prefer to log something that is a potential user issue rather than take the risk that a user is adversely affected by it and not have said anything about it. To be fair, I don't know if the argument is really against logging a non-WCAG issue. I think it’s against logging it in a way that lets a team mistake it for a failure. If the best-practice flag is clear, filterable and never counts towards a conformance figure, the risk should go away. If it’s just another row in the table just like all the other rows, then I can see how it could be a problem.
None of the above
One in-house accessibility lead said none of my four options fit. For their team an accessibility issue is an issue regardless of WCAG. A barrier, a bug or a usability problem is a problem by their internal standards or by the judgement of the accessibility, UX and content people building the product. It gets logged, given a severity and a priority, and worked on. They aim for 2.2 but it isn’t the framework that defines what they do, and they wouldn’t call what they do an audit.
I caught up with them on a call afterwards as I was curious as to their approach. They lead testing for an internal team at a financial organisation, and accessibility is so well integrated to their process that audits aren't really a thing. Accessibility is built in from the start and usability issues are tackled with the same vigour as official WCAG issues. This sounds like the holy grail, and something that every organisation should work towards.
Is a skipped heading a failure anyway?
I didn't intend this when I used skipped headings as an example, but in the comments a side discussion broke out. One commenter would call a skipped heading level a failure of 1.3.1 Info and Relationships. While I respect that it is open to some interpretation, personally, I’d be careful with that. 1.3.1 asks that structure conveyed visually is also available programmatically; it says nothing about the numbers. If a page goes from an H1 to an H3 and the H3 reads as a direct subsection, screen reader users get the same structure sighted users do.
The technique that recommends properly nested headings, G141, is only advisory under 1.3.1, its test checks that sections have headings and not how they’re numbered, and the criterion it’s sufficient for; 2.4.10 Section Headings, is AAA. That’s why automated tools mark skipped levels as best practice. Where a skipped level does become a failure is when the design plainly shows a heading that isn’t marked up as one, which is a missing heading, not a numbering problem. Either way it’s often worth noting, which is what I was trying to get to in the poll.
What this means for the report
For us it settles a question we’d been circling. Labrador will get a best-practice flag that’s separate from severity, visible on each issue, filterable in the report and excluded from any conformance count. That’s what 78% asked for, and it’s the version of “log it” that answers the objection from the other 17%.
If you audit for a living and your practice is different, I’d like to hear why. The last poll has affected how we’re building the tool. This one has too, and if you would like to see how it can help with your testing, you can try Labrador for free.