Accessibility testing

Accessibility testing

This resource was created by our Technology Taskforce, a group of senior IT accessibility individuals from leading UK and global organisations. For more information, including how to join, see our website. 

Introduction

Accessibility testing is a critical part of your organisation’s roll-out of new software, along with upgrades and updates to existing software. Every IT department is aware of the benefits of testing software before implementation and this guide will outline some of the best practice, the different options available to you, and whether automated tools are as accurate as manual testing.

What is accessibility testing?

According to W3C (World Wide Web Consortium):

“Web accessibility testing is a subset of usability testing where the users under consideration have disabilities that affect how they use the web. The end goal, in both usability and accessibility, is to discover how easily people can use a web site and feed that information back into improving future designs and implementations.”

In essence, the purpose of web accessibility testing is to ensure that all your users / customers have access to your systems, regardless of a disability or impairment.

Any technology department will be aware of the benefits of testing new applications before rolling out onto a network or system, and accessibility testing is no different. If testing is not carried out, many your users may find your website or systems inaccessible to them.

What should organisations test for accessibility?

All websites, systems and other digital platforms should be tested for accessibility. This includes new products being procured, existing products, and products being updated.

Accessibility testing during procurement

During procurement, it is vital that a website, system or other product is tested for accessibility before it is purchased. Organisations should include an accessibility testing phase in the procurement process, in order that this does not cause delays or unexpected issues, or that testing is rushed or skipped.

Further information

Our resource, ‘Inclusive technology procurement – Where to start,’ has more information.

BDF Partners can access our Procurement Toolkit for more information on procurement.

Testing existing or updated systems for accessibility

It is important to test websites, systems and platforms regularly for accessibility issues. While something can appear accessible at launch, issues can go unnoticed, or they can arise due to updates or upgrades – either to the product itself, or to AT that disabled people use to access it.

We recommend:

• Building accessibility testing into the lifecycle of the product, irrespective of scheduled updates or upgrades
• Testing for accessibility before major updates or upgrades are rolled out.

Are there automated tools that I can use?

There are many automated tools available to you, many of them are free to use. These have a limited application and should be used with caution (see manual testing).

W3C has created a list of automated accessibility testing tools.

The Paciello Group has also created a list of free website accessibility testing tools.

The advantage of using automated tools means that it can be easier to identify any issues quite early in the testing process, and pick up any of the major bugs and issues. They are also quick and easy to use, so as soon as a screen is drafted or changed it can be tested ‘little and often.’

However, there are also limitations to automated accessibility testing. We explore these below.

Manual testing

Manual testing is always the preferred option of accessibility experts for a number of reasons.

Many of the automated tools follow WCAG guidance, but there is no substitute for human intelligence. For example, the automated tool may test to see if a picture/photo/graphic has a label on it, to ensure that it can be used by screen reader software, but it would be unable to identify if the label was accurate or correct.

Manual and automatic testing do complement each other, and automated tools have their uses. However, they are only recommended as a starting point, rather than a full assessment.

Testing also needs to include a range of assistive technology (AT) to ensure it is compatible with a wide range of software. Websites must work for as wide a range of individuals as possible, using as wide a range of AT as possible. To find out more about the AT you may want to test against, please see our resource, ‘Assistive technology catalogues.’

In the UK, the Government Digital Services (GDS) carried out some research outlining the benefits of user testing vs. automated testing. During their research, they created the ‘world’s least-accessible website’ and then tested it. You can see more about their findings on the GDS Accessibility blog.

What options are there for manual testing?

Organisations have different approaches for manual testing depending on several factors. There are businesses that specialise in accessibility testing who will come into your organisation and test your applications, websites or software. They are generally very effective, but there is a cost associated.

Some organisations have internal accessibility teams who can carry out the testing. The advantage of this is that they may already be familiar with the software they are testing, along with the AT used within the organisation. Some of these people may have disabilities themselves, although this is generally the minority.

Other organisations utilise their staff disability networks and AT user groups, although this is purely optional as it is often in addition to their day job. These individuals have the option to opt in or opt out, depending on their availability. The benefits for individuals can vary – such as being rewarded for additional work, or being recognised at their appraisals. Alternatively, they may be one of the end users of the product, and it’s in their best interest to be involved. They may even just want to raise the profile of accessibility.

Conclusion

Accessibility testing is a vital part of the lifecycle of any website, system or platform. It must be done when it is procured or developed, regularly during its lifecycle, and before major updates or upgrades are rolled out.

Often, the most productive and cost-effective approach is a flexible one using a combination of automated and manual testing, using both external organisations and internal testers.