Skip to main content

Testing with Playwright

This guide explains how to integrate automated accessibility (a11y) testing into your Playwright test suite using @axe-core/playwright. The goal is to ensure WCAG compliance as part of your CI/CD pipeline.

📌 Overview​

This setup:

  • Runs accessibility scans using Axe
  • Supports WCAG 2.0, 2.1, and 2.2 rules
  • Allows excluding parts of the DOM
  • Fails tests when violations are detected
  • Outputs detailed logs for debugging

🧪 Example Test Case​

import { test } from '@playwright/test';
import { runA11yCheck } from './runA11yCheck';

test('The page should be Accessible', async ({
  page,
}) => {
  const params = {
    xxx: 'xxxx',
  };

  // load the page as implemented in a normal intergration-test
  await Page.goto(params);

  // if there are timing issues
  await Page.someTitle.waitFor({ state: 'visible' });

  // for exclude different classes or html elements
  const excludeList = ['.classname', '.classname-2'];

  // run the test with the helper function
  await runA11yCheck({ page, excludeList });
});

⚙️ Accessibility Helper Function​

import type { Page } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
import { expect } from './expect';

export async function runA11yCheck(arg: {
  page: Page;
  excludeList?: string | Array<string>;
}) {
  const page = (arg as { page: Page }).page ?? (arg as unknown as Page);

  const excludeList =
    (arg as { excludeList?: string | Array<string> }).excludeList ?? [];

  const excludeSelector = Array.isArray(excludeList)
    ? excludeList.join(', ')
    : excludeList;

  /*
  For all tags see this site to apply them to the withTags section
  www.deque.com/axe/core-documentation/api-documentation/
 */
  
  const axe = new AxeBuilder({ page }).withTags([
    'wcag2a',
    'wcag2aa',
    'wcag21a',
    'wcag21aa',
    'wcag22a',
    'wcag22aa',
    'best-practice',
    'experimental',
    'cat.aria',
    'cat.color',
    'cat.forms',
    'cat.keyboard',
    'cat.language',
    'cat.name-role-value',
    'cat.parsing',
    'cat.semantics',
    'cat.sensory-and-visual-cues',
    'section508',
    'cat.structure',
    'cat.tables',
    'cat.text-alternatives',
    'cat.time-and-media',
  ]);

  if (excludeSelector && excludeSelector.length > 0) {
    axe.exclude(excludeSelector);
  }

  const accessibilityScanResults = await axe.analyze();

  accessibilityScanResults.violations.forEach((violation) => {
    console.warn(
      JSON.stringify(
        {
          id: violation.id,
          impact: violation.impact ?? 'no-impact',
          help: violation.help,
          nodesCount: violation.nodes.length,
          helpUrl: violation.helpUrl,
          nodes: violation.nodes.map((node) => ({
            html: node.html,
            failureSummary: node.failureSummary,
          })),
        },
        null,
        2,
      ),
    );
  });

  expect(accessibilityScanResults.violations.length).toBe(0);
}

📊 Output & Debugging​

Example output:

{
  "id": "color-contrast",
  "impact": "serious",
  "help": "Elements must have sufficient color contrast",
  "nodesCount": 2,
  "helpUrl": "https://dequeuniversity.com/rules/axe/4.0/color-contrast",
  "nodes": [
    {
      "html": "<button class=\"low-contrast\">Click</button>",
      "failureSummary": "Fix contrast ratio"
    }
  ]
}

🔁 CI/CD Integration​

- name: Run Playwright Tests
  run: npm run test:e2e

✅ Best Practices​

  • Run a11y tests on key user flows
  • Keep your exclude list minimal
  • Fix violations instead of ignoring them
  • Treat accessibility failures like bugs
  • Monitor trends over time

⚠️ Limitations of Automated Testing​

While automated tools like Axe are excellent for catching many common accessibility issues, they cannot detect all WCAG violations.

The following still require manual testing:

  • Keyboard Navigation: Verifying logical tab order and ensuring no keyboard traps exist.
  • Screen Readers: Testing how content is announced by screen readers (e.g., NVDA, JAWS, VoiceOver) to ensure it makes sense.
  • Meaningful Alt Text: Checking if alternative text for images actually describes the content accurately in context.
  • Focus Management: Confirming that focus is moved correctly after dynamic changes or navigation.

📚 Resources​

🧠 Summary​

  • Automated accessibility validation: Catch common issues early in the development cycle.
  • CI/CD Integration: Enforce compliance and prevent regressions in every PR.
  • Hybrid Approach: Use automation for speed and scale, but always supplement with manual testing for full WCAG compliance.