How to Run a Chatbot Accessibility Audit
A Step-by-Step Testing Guide (2026)
Quick answer: A chatbot accessibility audit is three passes, not one. First, an automated scan catches the mechanical stuff — missing labels, low contrast, empty buttons — in minutes, with free tools. Second, a keyboard-only pass (no mouse) catches focus traps and unreachable controls automated scanners cannot see. Third, a screen reader pass — NVDA on Windows, VoiceOver on Mac, TalkBack on Android — catches whether the bot's replies are actually announced, which is the failure automated tools and keyboard tests both miss. None of the three passes substitutes for the others; a bot that scores clean in an automated scan can still be unusable with a screen reader on.
The three passes, in order
Run these in sequence — each one catches failures the previous one cannot.
Pass 1 — automated scan. Fast, free, and catches roughly a third of common issues: missing alt text, unlabeled buttons, insufficient color contrast, missing form labels. It will not tell you whether a screen reader user can actually complete a conversation.
Pass 2 — keyboard-only. Unplug the mouse. Tab, Shift+Tab, Enter, arrow keys, Escape — that is the full input vocabulary. This is where focus traps, an unreachable launcher, or a widget that swallows Escape shows up. It takes ten minutes and needs no software.
Pass 3 — screen reader. The pass that actually tells you whether a blind or low-vision user can use the bot, because it is the only one that surfaces whether new messages are announced. A widget can pass Pass 1 and Pass 2 perfectly and still be silent the moment the bot replies — the W3C's own guidance on this exact failure mode is blunt about what it is for: a status message is meant to "make users aware of important changes in content that are not given focus, and to do so in a way that doesn't unnecessarily interrupt their work," per the W3C Web Accessibility Initiative's Understanding Success Criterion 4.1.3 (Status Messages) in WCAG 2.2. A chat reply that appears silently, with no aria-live region announcing it, fails exactly this criterion — the companion WCAG guide covers the standard; this section is about catching it.
Pass 1: the automated scan, with free tools
You do not need a paid audit tool to run a first pass. The U.S. federal government's own accessibility program — which every federal agency and contractor uses to test procured software — relies on tools that are free to install:
- ANDI (Accessible Name & Description Inspector) — a free, open-source bookmarklet built by the U.S. Social Security Administration. Section508.gov describes it plainly: "ANDI... is a free open-source bookmarklet. This means the tool does not require installation as a plugin and can be added to multiple browsers as a bookmark." It is one of the tools "used in the Trusted Tester Test Process," the federal government's own manual testing methodology, chosen for "ease of use, ease of teaching, and accuracy of results."
- Color Contrast Analyzer (CCA) — a free, downloadable desktop tool (Windows and macOS) that checks two colors against WCAG's contrast minimums directly — useful for your chat bubble text and quick-reply chip labels, which brand-colored themes routinely fail.
- Deque University's Web Accessibility Checklist — a free, structured reference organized by topic (structure and semantics, links and navigation, images and visual design, multimedia and motion) that doubles as a scan checklist even without Deque's paid
axetooling.
Run one of these against the page hosting your widget before you touch a keyboard or a screen reader. Fix what it flags — usually contrast and missing labels — then move to Pass 2, because an automated tool genuinely cannot exercise your bot's actual conversation flow.
Pass 2: the keyboard-only test
No install required. Put the mouse aside and work through your bot's main flow using only these keys:
| Key | What to check |
|---|---|
| Tab | Does focus reach the chat launcher, then move logically through the widget once open? |
| Shift+Tab | Can you move backward without getting stuck? |
| Enter / Space | Does it activate the launcher, send a message, and select a quick reply? |
| Arrow keys | Do they move between quick-reply chips where the widget presents them as a group? |
| Escape | Does it close the widget and return focus to the launcher — not somewhere else on the page? |
The failure to watch for is the keyboard trap: focus enters the widget and Tab never lets you leave it, or Escape does nothing. If you cannot open the widget, read a reply, choose a quick reply, and close the widget again using only this table, a real percentage of your visitors cannot either — not just screen-reader users, but anyone using switch access, voice control, or a keyboard by preference.
Pass 3: the screen reader test
This is the pass most audits skip, because it needs software most non-technical testers have never opened. All three options below are free.
Windows — NVDA. NVDA (NonVisual Desktop Access) is, in Deque University's description, "a screen reader for Microsoft Windows that is totally free, yet fully functional and portable" — built by NV Access and downloadable at nvaccess.org. Two commands get you through a widget test: Insert+Space toggles between Browse Mode (reading the page) and Focus Mode (typing into a field) — if NVDA does not switch into Focus Mode when you land on the bot's text input, that is a real defect; and NVDA's Elements List (Insert+F7) lets you jump straight to a filtered list of links, headings, or form fields on the page, which is a fast way to confirm the launcher and input actually appear as real controls with names, not anonymous divs.
Mac — VoiceOver. Built into macOS; turn it on and off with Command+F5. Its modifier combination, Control+Option (referred to as "VO"), prefixes most commands — VO+U opens the Rotor, a menu that lists headings, links, and form controls on the page, the fastest way to confirm your launcher and input show up as real, named controls rather than silent decoration.
Android — TalkBack. Google's own description: "TalkBack is the Google screen reader included on Android devices. TalkBack gives you eyes-free control of your device." If your chat widget renders inside a mobile web view or app, this is the pass that catches mobile-specific silence — a common one is a widget that announces fine in a desktop browser with NVDA but goes quiet inside an app's embedded view because the underlying markup differs.
For every platform, run the same script: turn the screen reader on, navigate to the launcher and confirm it announces a name (not "button, button"), open the widget, send a message, and listen. The single most important thing to listen for is whether the bot's reply is announced automatically — if you hear nothing until you manually navigate to find the new text, the message area is missing a live region, which is the same WCAG 4.1.3 failure the W3C's status-messages guidance above describes. This one failure is, by a wide margin, the most common chat-specific accessibility defect, and it is invisible in Pass 1 and Pass 2 alike.
Per-component checklist
Work through each part of the widget individually rather than testing "the bot" as one unit — most audits miss a component because they never isolated it.
| Component | What to check | Test with |
|---|---|---|
| Launcher button | Real button (not a div), reachable by Tab, announces a name like "Open chat with [Brand] support" | Keyboard + screen reader |
| Text input | Screen reader switches into Focus/edit mode; has a visible or accessible label, not just placeholder text | Screen reader |
| Send control | Reachable by Tab or Enter from the input; announces its function, not just "button" | Keyboard + screen reader |
| Quick-reply chips | Each chip has its own accessible name; arrow keys or Tab move between them; see quick reply vs. button for how these should be marked up | Keyboard + screen reader |
| Message area / new replies | Announced automatically via a live region — this is the WCAG 4.1.3 failure above | Screen reader only |
| Typing indicator | Either announced briefly or not read at every micro-update (both are acceptable; constant re-announcement is not) | Screen reader |
| Human handoff | The path to a person — live chat, phone, or email — is reachable without completing a bot flow first; see human handoff | Keyboard + screen reader |
| Cards / carousels | Each card's content and any embedded link or button is reachable and labeled individually, not read as one undifferentiated block | Keyboard + screen reader |
Logging what you find
An audit that lives only in your head does not get fixed. Keep a simple log — a spreadsheet works fine — with these columns, and reuse it every re-audit:
| Component | Finding | Test used | Severity (blocker / major / minor) | Owner (you or vendor) | Status | Re-test date |
|---|
Severity matters for triage: a keyboard trap or a silent message area is a blocker — it stops a user cold and should hold up launch. Low contrast on a secondary label is usually minor — worth fixing, not worth delaying for. Route widget-code findings (the launcher's markup, live-region wiring, focus handling) to your vendor as a support ticket citing the specific component and test; route content and theme findings (contrast, chip labels, timeout length) to yourself, since those are configuration, not code, in nearly every platform we cover.
How often to re-audit
Re-run all three passes whenever you change the widget's theme, add a new flow (especially one using cards, carousels, or a new input type), or switch vendors — a new platform means new underlying markup, even if the conversation design looks identical. Absent a trigger like those, a full re-audit every quarter catches drift from platform updates you did not request. The keyboard-only pass alone (Pass 2) is cheap enough to run after every significant flow change, even between quarterly audits.
Frequently asked questions
How long does a chatbot accessibility audit take?
The three passes above take about an afternoon for a typical SMB widget: 15-20 minutes for an automated scan and fixes, 10 minutes for the keyboard-only pass, and 30-45 minutes for a careful screen reader pass across NVDA or VoiceOver plus a mobile TalkBack check. Logging findings and filing vendor tickets is the variable part, depending on how many blockers you find.
Do I need to test with all three screen readers — NVDA, VoiceOver, and TalkBack?
Test with whichever matches where your customers actually are. If your widget is desktop-only, NVDA (Windows) or VoiceOver (Mac) alone covers the most common failure modes, since the underlying live-region and focus-handling defects are usually platform-independent. If any meaningful share of traffic is mobile, add a TalkBack pass — mobile web views sometimes render markup differently than desktop browsers, and that gap is exactly the kind automated tools miss.
What is the single most important thing to check?
Whether the bot's replies are announced automatically to a screen reader. This one item — a missing aria-live region on the message area, WCAG's Status Messages criterion — is both the most common chat-specific accessibility failure and the one that neither an automated scanner nor a keyboard-only pass can catch. Only a screen reader test finds it.
Can I use a free tool for the whole audit, or do I need to pay for something?
Every tool named in this guide — ANDI, the Color Contrast Analyzer, Deque's Web Accessibility Checklist, NVDA, VoiceOver, and TalkBack — is free. Paid tools (Deque's axe product line among them) add automated coverage and reporting, but the three-pass procedure above does not require one.
What do I do with what I find?
Log it with severity, route widget-code findings to your vendor and content/theme findings to yourself, fix blockers before launch, and set a re-test date. The logging section above has a reusable column set.
Related guides
- Chatbot accessibility and WCAG — why this is a compliance question, the WCAG criteria mapped to chat-widget parts, and the vendor-vs-operator split.
- Chatbot QA testing protocol — the broader 10-step pre-launch pass this audit slots into.
- Quick reply vs. button — the tap targets the per-component checklist above tests.
- Typing indicator — the live-region component most often over- or under-announced.
- Human handoff and live chat — the escape hatch every accessible bot needs, and what to test on it.
- Conversation design — writing bot content that a screen reader delivers well.
- Best AI chatbot platforms 2026 — if an audit turns up a widget-code defect your current vendor won't fix, a ranked look at alternatives.
Sources
- W3C Web Accessibility Initiative, Understanding Success Criterion 4.1.3: Status Messages (WCAG 2.2) — w3.org/WAI/WCAG22/Understanding/status-messages.html, read 24 September 2026: "the intent of this success criterion is to make users aware of important changes in content that are not given focus, and to do so in a way that doesn't unnecessarily interrupt their work"; "status messages can be programmatically determined through role or properties such that they can be presented to the user by assistive technologies without receiving focus."
- W3C Web Accessibility Initiative, Easy Checks – A First Review of Web Accessibility — w3.org/WAI/test-evaluate/preliminary/, read 24 September 2026: "These checks cover just a few accessibility issues and are designed to be quick and easy, rather than definitive. A web page could seem to pass these checks, yet still have significant accessibility barriers."
- Section508.gov, Tools for Testing Information and Communications Technology — section508.gov/tools/tools-for-testing-ict/, read 24 September 2026: "The tools used in the Trusted Tester Test Process (and ICT Testing Baseline tests) have been chosen based on several factors including ease of use, ease of teaching, and accuracy of results. They are also free to install and use"; "ANDI (Accessible Name & Description Inspector), developed by the Social Security Administration, is a free open-source bookmarklet"; "Color Contrast Analyzer (CCA) is a free open-source tool that displays the contrast ratio for two selected colors."
- Deque University, Web Accessibility Checklist — dequeuniversity.com/checklists/web/, read 24 September 2026 (categories: Structure and Semantics; Links and Navigation; Images and Visual Design; Multimedia, Animations, and Motion).
- Deque University, Screen Reader Keyboard Shortcuts — NVDA — dequeuniversity.com/screenreaders/nvda-keyboard-shortcuts, read 24 September 2026: "NVDA (Non-Visual Desktop Access) is a screen reader for Microsoft Windows that is totally free, yet fully functional and portable"; Browse/Focus mode toggle "Insert + Space Bar"; Elements List "Insert + F7."
- Deque University, VoiceOver Keyboard Shortcuts on a Mac — dequeuniversity.com/screenreaders/voiceover-keyboard-shortcuts, read 24 September 2026: "Start (or stop) VoiceOver — Command + F5"; "VoiceOver Activation keys (or VO keys) — Control + Option"; "Open Rotor — VO + U."
- Google Accessibility Help, Get started on Android with TalkBack — support.google.com/accessibility/android/answer/6283677, read 24 September 2026: "TalkBack is the Google screen reader included on Android devices. TalkBack gives you eyes-free control of your device."
- Chatbotscape evaluation methodology. /methodology (continuously updated).
About this guide
Chatbotscape launched in 2026 as an independent review site for chatbot platforms. This guide is part of our SMB chatbot Academy and is written for the owner or operations lead who needs to actually run an accessibility test, not just understand why one matters — that context lives in the companion WCAG guide. It reads the W3C's, Section508.gov's, Deque University's, and Google's own published pages as of 24 September 2026. Editorial guidance, not legal advice; for your specific compliance obligations, consult a qualified professional.
Methodology
Every tool description and keyboard/gesture command above was verified against the page named in Sources on 24 September 2026, not carried over from general knowledge of these products. The three-pass framework and the per-component checklist are editorial working analysis, informed by the WCAG 4.1.3 status-messages criterion and by which chat-widget failures recur across published accessibility audits and practitioner write-ups (see the fan-out research note above); they are not the output of a controlled usability study. Corrections to editorial@chatbotscape.com.
Last updated
25 September 2026 — Initial publication. Next scheduled refresh: 25 December 2026.