What is the difference between real user monitoring and synthetic accessibility testing?
Synthetic accessibility testing checks pages and tasks you choose, in a browser you control — automated scans and scripted journeys. Real user monitoring checks for accessibility barriers inside real visitors' sessions on the live site. Synthetic testing tells you what can go wrong before release; real user monitoring tells you what did go wrong, for how many people.
A primer: lab and field data in web performance
Performance teams measure a site in two ways. Lab data, in web.dev's definition, “is determined by loading a web page in a controlled environment with a predefined set of network and device conditions.” Field data “is determined by monitoring all users who visit a page and measuring a given set of performance metrics for each one of those users' individual experiences” — and “is also commonly known as Real User Monitoring (RUM) data; the two terms are interchangeable” ( web.dev, 2022).
Synthetic monitoring is the lab side run on a schedule: a scripted browser loading known pages from known conditions. It is repeatable and catches regressions before users do. Real user monitoring is the field side: it sees the devices, networks and paths people actually used, which is why a page can look fast in the lab and still be slow for many of its visitors. Neither replaces the other.
The same split, applied to accessibility
Most accessibility testing is synthetic. An automated scan loads chosen pages and runs rules against them. A scripted or simulated journey attempts chosen tasks. A manual audit samples chosen flows. All of them work on a version of the site someone decided to test, in a state someone set up.
Real user monitoring for accessibility moves the checks into the field. A small script on the live site runs accessibility rules and behavioural checks inside real visitors' sessions and reports the barriers they met. Nobody chooses the pages, the states or the content; the visitors do.
| Synthetic testing | Real user monitoring | |
|---|---|---|
| Where it runs | A browser and environment you control | Your visitors’ own browsers |
| What it tests | Pages and paths you chose | Whatever real visitors loaded and did |
| When problems surface | Before release, on a schedule or on deploy | After release, as sessions happen |
| Repeatability | High — same inputs, same result | Low — every session differs |
| Coverage of states | The states you set up | The states people actually reached |
| Tells you | What can go wrong on a known path | What did go wrong, and how often |
| Cannot tell you | What real visitors encounter | Whether a task is completable, before anyone tries |
What only real user sessions surface
These are the findings a synthetic test can miss not because its rules are weaker, but because it was never in the situation.
States nobody scripted
Signed-in accounts, half-completed forms, the validation error a visitor actually triggered, the empty cart, the search with no results. Synthetic tests reach the states you set up; real sessions reach the rest.
Content that differs per visitor
Third-party chat and consent widgets, recommendations, A/B variants and personalized pages render differently from one session to the next. A check that runs on the page each visitor saw covers the versions they got.
Focus behaviour under real use
Keyboard focus trapped inside a widget, focus dropped to the page body after a single-page-app route change, focus that is not visible, and repeated Tab presses that make no progress — observed as they happen.
The conditions a barrier appeared in
Viewport size, browser zoom and root font size recorded with each finding, so a reflow failure at high zoom is tied to the visitors who were zoomed.
How many people a barrier affects
The share of sampled sessions that met each barrier, with a confidence interval, which turns a list of violations into an order of work.
What real user monitoring cannot do
Field data is uncontrolled by design, and that cuts both ways.
- It widens where checks run far more than what a machine can decide. Rules in a real session answer the same questions as rules in a scan, plus a few behavioural ones about focus. Neither can judge whether alternative text is accurate or a heading meaningful.
- It cannot tell you a task is completable. It sees barriers visitors met, not whether anyone could have finished — that takes a journey.
- It only sees what ships and gets traffic. A broken page nobody visits produces no findings, and nothing is found before release.
- It must not identify people. Responsible monitoring records behaviour and conditions, never whether a visitor uses assistive technology or has a disability.
None of it replaces testing by people. As the W3C puts it, evaluation tools “can not determine accessibility, they can only assist in doing so” (W3C Web Accessibility Initiative, 2024).
How to use both
- Run synthetic tests before release. Scan every page and run journeys through the critical tasks on deploy, so known paths are protected before anyone meets a regression.
- Monitor real sessions after release. Let monitoring surface the states, content and conditions the tests never covered.
- Feed the field back into the lab. When monitoring finds a barrier on an untested route, add that route to scanning and turn the path into a journey, so it is protected on the next release.
- Prioritize by exposure. Fix the barriers the most sessions meet first, and confirm each fix with a re-scan or journey run.
Accessible does both halves on one domain: real user monitoring in the field, and scanning and journeys in the lab. For where each layer fits, read static scans vs. simulated journeys vs. real user monitoring.
Sources
- Why lab and field data can be different (and what to do about it) (opens in a new tab). web.dev, July 18, 2022.
- Selecting Web Accessibility Evaluation Tools (opens in a new tab). W3C Web Accessibility Initiative, May 13, 2024.