Accessibility Testing: Tools, Manual Checks and a Test Plan

You’ve run Lighthouse, the accessibility score says 100 and everything is green. Done? Not quite. An automated test can tell you that every image has an alt attribute. It has no idea whether that attribute says “IMG_0042.jpg” or something genuinely useful. And it certainly won’t check whether your checkout can be completed with a keyboard.

That’s why testing for accessibility means combining several methods: automated tools for quick, repeatable findings, manual checks for anything that needs human judgement, testing with disabled people for the real user experience, and a formal evaluation when you need one. In this article you’ll learn what each method can do, which tools have proven themselves, and how to turn it all into a test plan that survives contact with real projects.

Why one tool is never enough

WCAG is made up of success criteria, and many of them can’t be judged by a machine alone. A tool can measure whether the contrast between text and background reaches 4.5:1. It can’t decide whether an error message makes sense, whether the order in which focus moves through the page is logical, or whether a video’s captions are accurate.

How much automated testing catches overall depends heavily on the site and the tool. A rule of thumb you’ll often hear in the field is that it finds roughly 30 to 40 percent of accessibility issues. Treat that as a ballpark rather than a precise figure: the bigger share of the work lies in manual testing.

That said, automated tests are essential. WebAIM’s annual Million report, which analyses the home pages of the top one million websites, has found automatically detectable WCAG failures on more than 94 percent of them year after year. The most common one is low text contrast. These issues are easy to find and usually easy to fix. Not catching them automatically is a missed opportunity.

What automated tools catch – and what they miss

Tools reliably detect things like:

• insufficient contrast for regular text (Contrast (Minimum))
• images without an alt attribute
• form fields without an associated label
• links and buttons without an accessible name, such as icon-only buttons
• a missing language attribute on the html element
• invalid ARIA attributes or roles
• duplicate IDs that break label associations

They can’t judge, or can only partly judge:

• whether alt text actually describes the image meaningfully (Non-text Content)
• whether every feature can be reached and operated with a keyboard
• whether the focus order makes sense
• whether error messages explain the problem clearly
• whether headings accurately structure the content
• whether captions and audio descriptions are complete and correct
• whether contrast holds up over images, gradients or hover states

A typical real-world example: a tool reports zero issues for a product filter. Every element has a label and all ARIA attributes are valid. Then a keyboard test reveals that you can tab to the dropdown but can’t use the arrow keys to change it, and that focus jumps back to the top of the page when it closes. Technically clean, practically unusable.

Browser extensions and other tools

axe DevTools

axe DevTools by Deque is an extension for your browser’s developer tools. It’s built on axe-core, the open-source testing engine that also powers many other tools. Results are sorted by severity, point to the affected element and explain how to fix the issue. axe is deliberately designed to keep false positives to a minimum.

WAVE

WebAIM’s WAVE overlays its results as icons directly on the page. That makes it especially approachable for designers and content teams, because you can see exactly where each problem is. Besides errors, WAVE shows structural information such as headings and landmarks, which is handy for getting a quick picture of how a page is organised.

Lighthouse

Lighthouse is built into Chrome’s developer tools and checks accessibility alongside performance and SEO. For accessibility, it uses rules from axe-core too, but only a subset. A score of 100 therefore means the automated rules it runs have passed, not that the page is accessible.

Automated tests in your pipeline

Automated checks really pay off when they run on every build. With axe-core, for example, you can extend your Playwright tests:

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('basket has no automatically detectable issues', async ({ page }) => {
  await page.goto('https://shop.example/basket');
  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
    .analyze();
  expect(results.violations).toEqual([]);
});

This stops issues you’ve already fixed from creeping back in with the next release. Don’t just test the home page, though: cover states too, such as open menus, dialogs and forms showing error messages.

Manual checks any team can do

Many important checks need no special expertise, just a bit of time and a system. Run these five on every new page and component.

1. Keyboard test

Put the mouse aside and use only Tab, Shift+Tab, Enter, Space, the arrow keys and Escape. Can you reach every control (Keyboard)? Can you get back out again, for instance from an embedded video or a dialog (No Keyboard Trap)? Is the order logical (Focus Order), and can you always see where focus is (Focus Visible)? Our article on keyboard and focus goes deeper.

2. Zoom and reflow

Zoom the page to 200 percent, then to 400 percent. At 400 percent in a typical desktop browser window, the viewport is roughly 320 CSS pixels wide. Content should remain readable without horizontal scrolling, and nothing should be cut off or overlap (Reflow).

3. Content and structure

Look at the heading structure, for example with WAVE or an extension that lists headings. Does it make sense as an outline on its own? Check alt text for meaning, not just presence, and read your link text out of context: would “More” or “Click here” tell you where you’re going?

4. Forms and errors

Submit forms incomplete or with wrong input on purpose. Is the error detected, described in text and tied to the right field (Error Identification)? Is what you already entered kept? Can you spot the error without relying on colour?

5. Quick screen reader check

A screen reader instantly shows you whether buttons have sensible names, whether status messages are announced and whether dialogs manage focus properly. You don’t need to be an expert. Our quick screen reader test walks you through finding the most important issues in 15 minutes with NVDA, VoiceOver or TalkBack.

Testing with disabled people

Even a site that meets every criterion can feel clumsy in everyday use. The most reliable way to find out how usable your product really is: ask people who use assistive technology every day. They notice things no checklist covers, like a long trek to the main content, confusing announcements, or a flow that’s technically possible but exhausting.

You don’t need a large study. Even a handful of sessions with participants who use, say, a screen reader, screen magnification or voice control will give you valuable insights. Give them realistic tasks, such as finding and ordering a product, and let them use their own devices and settings. You can find participants through disability organisations or specialist recruitment agencies. Paying people fairly for their time should go without saying.

User testing doesn’t replace a systematic evaluation against WCAG; it complements it. The evaluation tells you whether the requirements are met. User testing tells you whether the result actually works well.

Formal evaluation methods

When you need a solid conformance statement, for instance as the basis for an accessibility statement or the accessibility information required under the European Accessibility Act, a structured evaluation method is worth it.

An internationally recognised framework is the W3C’s WCAG-EM (Website Accessibility Conformance Evaluation Methodology). It describes five steps: define the scope, explore the website, select a representative sample, evaluate the sample and report the findings. The sample matters, because on a large site you can’t test every single page.

In Germany, a widely used method is the BIK BITV-Test. It breaks down the requirements of Germany’s public sector accessibility regulation (BITV 2.0) and the European standard EN 301 549 into individual test steps with detailed instructions, and there are variants for WCAG 2.2 and for native apps. Even if you never commission an official test, the published test step descriptions are a great resource for testing more thoroughly yourself, as long as you can read German.

An external audit by experienced evaluators makes particular sense before a relaunch, when legal requirements apply, and when your team doesn’t yet have much experience. Make sure the report doesn’t just list failures but also gives the location, the criterion that failed and a suggested fix.

Building a test plan

A test plan doesn’t have to be a thick document. It answers four questions: what do we test, when, how and who does it?

What: follow WCAG-EM and pick a sample. Include the home page, every page template (for example article, category, product detail), all key user journeys such as sign-up, search and checkout, and recurring components like navigation, dialogs and forms.

When: accessibility doesn’t start with finished code. Check contrast, focus states and labels at the design stage. Run automated tests on every build. Do manual checks for every new component and before every major release. Schedule a full evaluation at least once a year, and after any major change.

How: agree on the benchmark, usually WCAG 2.1 AA or, better, 2.2 AA, and on your tools: for example axe in the pipeline, WAVE for spot checks, and keyboard and screen reader tests following a fixed routine.

Who: assign owners. Developers handle automated and keyboard tests, designers check contrast and states, and content editors look after alt text and headings. Bring in outside help for full evaluations when needed.

Log findings in your regular issue tracker, not in a separate spreadsheet nobody looks at. Give every ticket the relevant WCAG criterion, the location and the impact on users. And prioritise by impact: a checkout that can’t be completed with a keyboard outweighs missing alt text on a decorative image.

Quick checklist

• Automated tests (for example axe) run on every build
• Every new component tested with a keyboard
• Zoom checked at 200 and 400 percent
• Headings, alt text and link text reviewed for meaning
• Forms tested with deliberate errors
• Quick screen reader test for key user journeys
• Testing with disabled people planned
• Regular full evaluation following a defined method

Conclusion

Automated tools are a good start, but only a start. They find common technical issues quickly and reliably, so they belong in your development process. Most barriers, though, only surface through manual checks, screen reader testing and conversations with people who rely on accessible products.

The most important step is to test regularly at all. A simple plan your team actually follows beats a perfect concept gathering dust in a drawer. When you want to look up the criterion behind a finding, the WCAG guide has you covered.

Sources

• WCAG-EM Overview: Website Accessibility Conformance Evaluation Methodology – W3C WAI (opens in a new tab)
• Easy Checks – A First Review of Web Accessibility – W3C WAI (opens in a new tab)
• The WebAIM Million – WebAIM (opens in a new tab)
• WAVE Web Accessibility Evaluation Tools – WebAIM (opens in a new tab)
• axe-core – Deque Systems on GitHub (opens in a new tab)
• BIK BITV-Test (German) (opens in a new tab)