Accessibility Is Just Another Functional Test

Editor’s Note: The following post has been provided by Evinced. Insight Partners is an investor in both Evinced and The New Stack.

Accessibility advocates have been demanding more accessible websites and mobile apps for years. And, with an estimated 15% of the global population living with disabilities, most businesses are striving to make their digital experiences accessible to differently-abled people.

So, if everyone wants accessibility, what’s taking so long to make it happen?

And why are companies checking for accessibility at the end of a development cycle — or worse, months after a release?

At Evinced, we’ve been asking the question: Why can’t accessibility be just another automated functional test?

The truth is that it’s been too hard to do, for far too long.

It’s Too Hard to Set up and Maintain Tests

Legacy tools only take static scans, forcing developers to program the scans to run at specific points in the test to ensure they check the application thoroughly.

There are two approaches to this, and frankly, both are bad:

Option 1: Add the scans directly. Going through each line of a test and adding a static scan at every interaction could produce many more scans that need to be added per test. It’s not practical for hundreds and thousands of tests.

//Original Test
@Test
public void trvlHomePage() {
driver.get(demoPage);
// Click "Accommodation Type" dropdown
driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1)")).click();
// Click "City" dropdown
driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2)")).click();
// Click "When" date picker
driver.findElement(By.cssSelector(".react-date-picker")).click();
}

//Test with legacy accessibility scans
@Test
public void trvlHomePageLegacy() {
driver.get(demoPage);
scanForAccessibility();
// Click "Accommodation Type" dropdown
driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1)")).click();
scanForAccessibility();
// Click "City" dropdown
driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2)")).click();
scanForAccessibility();
// Click "When" date picker
driver.findElement(By.cssSelector(".react-date-picker")).click();
scanForAccessibility();
}

Option 2: Abstract away the static scans of individual tests into your page objects or helper methods. This approach only works until the page changes. Maintaining pages and ensuring static scans are in the correct places to sustain accessibility coverage creates too much work.

 public void homePageLogin(String username, String password) {
driver.findElement(By.id("username")).sendKeys(username);
driver.findElement(By.id("password")).sendKeys(password);
driver.findElement(By.id("loginBtn")).click();
scanForAccessibility();
}

There’s a lot of noise — reports with hundreds and thousands of issues and many duplicates — when using legacy tools.

For example, when testing for accessibility using a legacy tool like aXe, developers need to add an accessibility check and a resulting report for each element in the test. Afterward, they must manually deduplicate the issues across the reports. While a single-page site could be manageable, if you like de-duplicating reports, it’s not practical for tests involving multiple elements, user interactions, and pages.

Manual Checks Aren’t Scalable and Can’t Keep up with Build Cycles

Functional testing has become automated in the last decade, but accessibility testing still relies heavily on manual audits and reviews at the end of a development cycle. These reviews can interrupt production cycles causing expensive delays. Audits occurring after builds are released can result in live bugs.

A New Approach

We need solutions to these problems and a cohesive approach that helps developers shift accessibility testing left and treat it like any other round of functional testing.

Simply put: What would make accessibility testing easier?

Here’s a thought: What if you could set up an accessibility SDK within your company’s test framework before running the first test?

For instance, perhaps you could simply instantiate and call a test automation SDK during setup, just once, with a few lines of code even if you were working with 10 or 10,000 tests. Wouldn’t accessibility testing be easier if it were this simple to implement?

 
import com.evinced.EvincedWebDriver;


@BeforeClass
private void driverSetup() {
driver = new EvincedWebDriver(new ChromeDriver());
driver.evStart();
}

Support for Most Frameworks

Of course, the best accessibility testing solutions would include built-in support for the most commonly used web and mobile automation frameworks — Selenium, Cypress, Playwright, WebdriverIO, Ruby, XCUITest, Espresso, Appium, and others.

And, you’d want an SDK that integrates directly with continuous integration systems like Jenkins, CircleCI, etc., and supports various output formats — JSON, HTML, SARIF etc. (It also raises the possibility of conditional gating based on accessibility for each build.)

Two More Requirements

To really bring it together, it would be important to continuously scan the document object model (DOM) for changes as your UI tests run and exercise various UI elements.

Instead of asking you to code up a single test and report for every single page and interaction, the accessibility SDK could discover issues automatically. Ideally, there would be no per-test code at all. And then you’d receive a single, de-deduplicated report for your whole test suite.

That’s accessibility testing made simple, and what’s simple is sustainable.

Toward Automation-Empowered Expertise

Accessibility testing isn’t new, but it does need to change.

The model we’ve suggested above is the key. Shifting accessibility testing left, making testing easier and more sustainable will ensure more users are served, developers stop throwing their hands up in frustration, and experts can focus where experts are truly needed.