How a shop is tested, and against which standards

The W3C publishes the Web Content Accessibility Guidelines (WCAG 2.1, WCAG 2.2). ETSI publishes EN 301 549. Its clause 9 adopts the WCAG success criteria at levels A and AA as requirements for web content, numbered 9 followed by the criterion’s number: criterion 1.1.1 is clause 9.1.1.1. A criterion added by WCAG 2.2, such as 2.5.8 Target Size (Minimum), has a clause only in V4.1.1, the version whose web requirements are aligned with WCAG 2.2.

Validiscan works to WCAG 2.2 AA because it is the newer version and the one that EN 301 549 V4.1.1, published by ETSI in September 2026, is aligned with. The W3C’s Accessibility Guidelines Working Group, which writes WCAG, recommends that sites adopt WCAG 2.2 as their new conformance target, even if formal obligations mention previous versions (WCAG 2.2). The results are also stated against WCAG 2.1 AA, because that is the level EN 301 549 V3.2.1 adopts, the version whose reference is published in the Official Journal under the Web Accessibility Directive.

The two versions overlap almost completely. Every WCAG 2.1 success criterion at levels A and AA is also in WCAG 2.2, except 4.1.1 Parsing, which WCAG 2.2 removed. WCAG 2.2 adds six criteria at those levels:

• 2.4.11 Focus Not Obscured (Minimum), level AA
• 2.5.7 Dragging Movements, level AA
• 2.5.8 Target Size (Minimum), level AA
• 3.2.6 Consistent Help, level A
• 3.3.7 Redundant Entry, level A
• 3.3.8 Accessible Authentication (Minimum), level AA

Each criterion is evaluated once. The Remediation Evidence Package then shows the results twice, in one table for WCAG 2.1 AA and one for WCAG 2.2 AA. For 4.1.1 Parsing, which only the WCAG 2.1 table lists, the package records the criterion as supported, with the basis “Always satisfied for HTML content (WCAG 2.1 errata)”.

The package describes what it covers in this sentence: “Conformance with WCAG 2.1 AA and WCAG 2.2 AA success criteria as evaluated on the listed pages and dates.” It also says: “This package is evidence of an evaluation and of remediation work. It is not a certificate and does not state legal compliance.”

Validiscan’s scanner loads each page in a real browser, Chromium driven by Playwright, and runs two kinds of rules on it: the rules of axe-core, an open-source accessibility testing engine, and Validiscan’s own checks for barriers typical of online shops. Each finding names its rule, the elements and pages where it fails, and the WCAG criteria and EN 301 549 clauses of that rule.

Many WCAG success criteria need a person’s judgement. Validiscan’s manual checklist has 35 items, each for one success criterion at level A or AA that automated checks cannot fully prove. Of the 55 WCAG 2.2 success criteria at levels A and AA, 22 are tested by no rule that runs and are covered only by this checklist. The other 33 are tested by at least one running rule. For some of them, such as keyboard operation, focus order and captions, the checklist has an item as well, because a rule tests them only in part. For the rest, it has none.

The checklist asks, among other things, for these checks:

• the way to the basket with a keyboard alone: the menus, a product page, choosing a variant, adding to the basket, then changing a quantity and removing an item;
• every modal and the cookie banner by keyboard: focus moves in, Escape closes, focus comes back;
• search, product variants, the basket and the checkout fields with a screen reader;
• form errors: announced when they appear, naming the field and saying how to correct it;
• the page zoomed to 400 percent;
• videos: captions and audio description.

The checklist names the screen readers for these passes: NVDA with Chrome on Windows, and VoiceOver with Safari on a Mac. For each item, Validiscan records pass, fail or not applicable, with notes and evidence. The result decides how the criterion is reported in the Remediation Evidence Package.

A full scan, which is the baseline scan of a project, loads a sample of each kind of page in the shop, up to 150 sampled pages by default: the home page, product lists, product pages and the other kinds of page it finds, with the cart, the first checkout step, search results, the login or registration page, the contact page and the legal pages on top when the scan finds them. By default, every page is loaded at desktop size (1440 by 900 pixels) and at phone size (390 by 844 pixels, with touch input), and all checks run at both sizes.

To reach the cart and the checkout, the scan adds one product that is in stock to the cart, opens the cart and opens the first checkout step. It never completes a purchase and never enters payment data.

A cookie banner is tested as part of the page first. By default, the scan then accepts it, so that the rest of the page can be tested. Known analytics, advertising, chat and review services are blocked during a scan by default, so that the results do not depend on them. Consent tools are not blocked, because the cookie banner is part of what is tested.

Validiscan scans a shop in full only with its owner’s permission. A full scan, a verification scan and a regression protection run need the shop owner’s written permission, recorded in Validiscan’s application with the time and the name of the person who recorded it. Without that record, the application refuses to queue the scan, and a scheduled regression protection run is skipped.

A full scan follows the shop’s robots.txt by default. It can be set to ignore robots.txt only for a shop whose owner’s permission is recorded.

The free check runs without that permission, within fixed limits. It loads at most 15 pages, never adds anything to a cart and never opens the checkout. It follows the robots.txt of the shop’s host and leaves at least 1 s between the requests it starts itself, with known exceptions: its first request, for the home page, is made before robots.txt is read, and when the shop answers a page load in the browser with a redirect, the browser follows it at once, without that gap and without checking robots.txt. The free check page describes every limit and exception.

Automated checks find only part of all accessibility barriers, and a scan describes the pages it loaded on the dates it ran. The Remediation Evidence Package prints its own limitations. The list below also names gaps that Validiscan has recorded in its scanner and not yet closed.

• Automated checks prove only part of WCAG. A rule cannot judge whether a text alternative describes its image, whether an error message makes sense when it is read aloud, or whether captions are right. The manual checklist covers the last two. The quality of text alternatives has no checklist item today, so a result of supports for 1.1.1 Non-text Content rests on automated checks alone. The failures a scan reports are a floor, not a total.
• Pages are sampled. A scan checks a sample of each kind of page. Other pages built from the same templates are expected to behave alike, but they are not checked one by one.
• A scan is a point in time. The results describe the shop on the dates of the scans. Later changes to the theme, apps or content are not covered until the next scan; regression protection re-scans the shop on a schedule.
• The checkout ends at its first step. No order is placed and no payment data is entered, so the later steps of the checkout are not tested automatically.
• Other frames and closed shadow DOM. Content inside cross-origin frames and closed shadow DOM, such as payment fields and some consent tools and widgets, is not evaluated automatically. Validiscan’s cookie banner check does not evaluate a banner that sits in shadow DOM or in a cross-origin frame.
• Blocked services are not tested. The analytics, advertising, chat and review services that a scan blocks are not tested. Fixing third-party embeds, such as payment and chat, is not part of a project either.
• Results that need review are not reported yet. When axe-core or one of Validiscan’s checks cannot decide a result, it marks the result for review instead of reporting a failure. The scanner does not keep these results today, so they do not appear as findings. This affects links to PDF and other documents, which Validiscan’s documents check only marks for review, and cookie banner, focus, keyboard and form results that a check could not settle. A few rules that run therefore cannot produce a finding today. Until the scanner keeps these results, Validiscan checks the cookie banner by hand and records the result in the manual checklist. PDFs themselves are not part of a project.
• Screenshots show the first screen. The scanner captures each page at the size of the screen, not its full length. A screenshot for a finding further down a page shows the top of the page. Until the scanner captures full pages or single elements, Validiscan checks the figures of each Remediation Evidence Package by hand before the package is issued.