WAI-Tools Documentation of WCAG Interpretation and Test Rule

Preface

As a part of the WAI-Tools project, the Norwegian Digitalisation Agency (previously the Agency for Public Management and eGovernment – Difi) constitute the documentation of their internal interpretation of the Web Content Accessibility Guidelines (WCAG) and Accessibility Conformance Testing (ACT) Rules. This documentation is the part of deliverable “D2.4 Working Instance of Digdir Observatory” under Work Package 2 in the WAI-Tools project.

The purpose is to document the Agency’s interpretation of the WCAG success criteria and coverage of these success criteria by ACT Rules. In addition, the gap analysis between WCAG success criteria and ACT Rules has also been documented, such as aspects of the success criteria that are not covered by ACT Rules in WAI-Tools and aspects of the ACT Rules that go beyond the scope of the success criteria. This documentation is subdivided into one chapter per WCAG success criteria. Each chapter defines:

  • specific details relevant to WCAG success criteria,
  • purpose of success criteria,
  • user accessibility needs (functional performance statements (fps)) (based on EN 301 549),
  • interpretation and specification of the success criteria (Norwegian Digitalisation Agency),
  • coverage of Success Criteria by ACT rules developed in WAI-Tools,
  • gap analysis, and ACT Rules related to each WCAG success criterion.

The Norwegian Digitalisation Agency’s existing interpretation and specification of WCAG success criteria are limited to WCAG 2.0 with a brief overview of associated test procedures. The Norwegian Digitalisation Agency also has a fundamental interpretation and specification of the new WCAG 2.1 success criteria but does not have associated test procedures. Our contributions to the test rules development were based on our existing interpretation of the WCAG 2.0 and 2.1 success criteria.

Each chapter of this documentation has been written using a template specially designed for this documentation. This template contains two modules, one for WCAG success criteria and the other one for ACT Rule. To understand how these two modules have been filled out by the authors, the interested readers are referred to the template which can be found in Chapter 55 at the end of this documentation.

Background

This documentation has been written between 01 July 2020 and 31 October 2020 as one of the main results of deliverables in a European Commission (EC) co-funded project named Web Accessibility Initiative - Advanced Decision Support Tools for Scalable Web Accessibility Assessments (WAI-Tools) Project under Horizon 2020 Program (780057).

WAI-Tools Project

WAI-Tools is an Innovation Action project that has been active for 39 months between 1 November 2017 and 31 January 2021.

In WAI-Tools Work Package 1 the objective is “to build, implement, and validate a set of open test rules, developed through the international consensus process of W3C”. These test rules are developed by project staff through the W3C consensus process, implemented in open source engines developed and maintained by Deque, FCID (University of Lisbon), and Siteimprove.

The Norwegian Digitalisation Agency (previously the Agency for Public Management and eGovernment – Difi) is responsible for deliverable “Documentation of Digdir’s WCAG interpretation and ACT Rules” under deliverable “D2. Deployment of Test Rules” under Work Package 2. In this documentation, interpretation, and mapping of test rules developed under this project to WCAG 2.0/2.1 have been documented.

Web Accessibility Directive (WAD)

The Web Accessibility Directive (WAD), Directive (EU) 2016/2102, in force since 22 December 2016, aims to provide people with disabilities with better access to the public services websites and mobile applications. The Directive covers websites and applications of the public sector with a limited number of exceptions e.g. broadcasters, live streaming.

It is also required in WAD and its corresponding, that the member states are required to monitor the conformity of websites and mobile applications of public sector bodies with the accessibility requirements provided for in Article 4 of Directive (EU) 2016/2102 using a simplified monitoring method to detect non-compliance and an in-depth monitoring method to verify compliance.

Article 6 of Directive (EU) 2016/2102 refers to European standard EN 301 549. The content of websites and mobile applications meets harmonised standards and shall ensure at least a level of accessibility equivalent to that ensured by European standard EN 301 549, which in turn refers to WCAG 2.1 success criteria. In other words, it has been statutory throughout Europe for public sector bodies websites to be designed in accordance with WCAG 2.1.

In addition, we know from our experience and with reference to paragraph 56 of Directive (EU) 2016/2102 that there are different interpretations of WCAG success criteria across member states which requires harmonisation to monitor conformity. WAI-Tools is an initiative of a harmonized interpretation of WCAG, with associated test rules. This interpretation of WCAG is implicit in the test rules developed in WAI-Tools.

The terms compliance and non-compliance are not defined in Directive (EU) 2016/2102. The directive refers to European standard EN 301 549, where the determination of compliance is defined in the following way:

“Compliance is achieved either when the pre-condition is true and the corresponding test (in Annex C in EN 301 549) is passed, or when the pre-condition is false (i.e. the pre-condition is not met or not valid).”

Directive

For each of the requirements regarding web pages, the pre-conditions and tests are stated in the following way:

Type of assessment Inspection
Pre-conditions 1. The ICT is a web page.
Procedure 1. Check that the web page does not fail WCAG 2.1 Success Criteria X.Y.Z [Name of Success Criteria]
Result

Pass: Check 1 is true

Fail: Check 1 is false

Web Content Accessibility Guidelines (WCAG) 2.0/2.1)

The Web Content Accessibility Guidelines (WCAG) standard is developed through a W3C process in cooperation with individuals and organisations around the world. WCAG defines guidelines to make web content more accessible for people with disabilities. These accessibility guidelines cover a wide range of disabilities such as visual, auditory, physical, auditory, speech, cognitive, language, learning, and neurological disabilities.

WCAG 2.0 was published on 11 December 2008. WCAG 2.1 was published on 5 June 2018. There are 13 guidelines organised under 4 principles: perceivable, operable, understandable, and robust. For each guideline, there are testable success criteria, which are at three levels of conformance: A, AA, and AAA. All requirements (“success criteria”) from 2.0 are included in 2.1. The 2.0 success criteria are exactly the same (verbatim, word-for-word) in 2.1.

The accessibility requirements in WCAG 2.0/2.1 are mostly written in a technology-neutral language which requires interpretation of what exactly is required to comply with these requirements especially when it comes to specific technology.

Accessibility Conformance Testing (ACT) Rules

An ACT Rule is a plain language explanation of how to test a specific type of content for specific aspects of WCAG accessibility requirements (“success criteria”). An ACT Rule describes the expected results of accessibility testing tools and/or methodologies when running a conformance test against a specific part of the applicable test subject. In the context of WCAG, ACT Rules test for failures in satisfying Success Criteria. ACT rules are written for the testing process and usually more specific and limited to a single aspect of a WCAG accessibility requirement. Each ACT Rule contains a description of the type of content under test, the test to perform, and the expected results.

The ACT Rules were written and developed by the ACT Rules Community Group (ACT-R) using W3C ACT Rules Format. One of the objectives of this project was to establish an open community by bringing together key partners from industry, government, and research, who can develop authoritative resources on accessibility conformance testing (ACT).

The ACT Rules Community Group (ACT-R) is an open forum with the aim to document and harmonise the interpretation of W3C accessibility standards such as WCAG and WAI-ARIA. ACT-R is an initiative to keep working on the development of ACT Rules and harmonising the interpretation of W3C accessibility standards even though the WAI-Tools project has been ended. This long-term process of development of ACT Rules will continuously increase the coverage of WCAG success criteria requirements which later can be used not only in WAD simplified but in in-depth monitoring as well.

According to W3C ACT Rules Format, there are two types of ACT Rules:

  • Atomic rules describe how to test a specific type of solution such as website content. It contains a precise definition of what elements, nodes or other “parts” of a test subject are to be tested, and when those test targets are considered to pass or fail the rule. Atomic rules are to be kept small and specific. These rules test a single “condition” and do so without using the outcomes from other rules.
  • Composite rules describe how the outcomes of multiple atomic rules can be combined into a single outcome for each test target. In short, a composite rule can have multiple “conditions”, each of these tested in separate atomic rules.

The implementation of ACT Rules is based on test mode. The Test Mode of a rule describes how a test was carried out, in the case of ACT Rule, implementation can be automated, semi-automated, or manual. In an automated implementation, the test is carried out automatically means such as software tools and without any human intervention. In a manual implementation, the test is carried out by human evaluators, it not just includes actual test procedure carried out by the evaluators but also where the evaluators helped by instructions or guidance provided by software tools. An implementation is considered as semi-automated, where the test is carried out with the help of software tools, but human evaluation or judgment is still required to decide the outcome of the test.

The outcome of evaluation using ACT Rule on a test subject or one of its constituent test targets can be one of the three following types:

  • Inapplicable: No part of the test subject matches the applicability of the rule
  • Passed: A test target meets all expectations of the rule
  • Failed: A test target does not meet all expectations of the rule

The outcome of an ACT Rule “Passed” does not mean that a test subject is compliance with all the applicable requirements of WCAG success criteria but only compliance with a specific aspect of WCAG success criteria covered by that ACT Rule. There are a few manual ACT Rules that possibly fully satisfy some of the WCAG success criteria. However, in general, “Passed” and “Inapplicable” outcomes of ACT Rules require further testing or manual checks.

ACT Rules are open source and can be used by member states and other organisations to implement these rules in the development of manual testing procedures, semi-automatic and/or automatic testing tools.

Summary of our findings

The WCAG success criteria are generic, complex, and require a wide range of various aspects and/or situations to be compliant. It is hard to cover all the aspects and/or situations required by WCAG success criteria, ACT Rules developed in the WAI-Tools project are very specific and atomic to a specific aspect and/situation that cover a small portion of WCAG success criteria. There can be several ACT Rules to cover all applicable aspects of WCAG success criteria accessibility requirements, which means that there could still be a gap in compliance with all aspects of a WCAG success criterion.

Currently, the ACT Rules developed in WAI-Tools can be used in and limited to testing contents created using web technologies, such as Hypertext Markup Language (HTML), Cascading Style Sheets (CSS-2018), Accessible Rich Internet Applications (WAI-ARIA), Scalable Vector Graphics (SVG2).

There have been 07 Composite ACT Rules (uses outcomes from 26 Input Rules that are atomic ACT Rules) and 49 Atomic ACT Rules developed until October 2020 in the WAI-Tools project. In total including both composite and atomic, there are 56 ACT Rules that have been documented in this documentation and covers compliance and/or non-compliance of certain accessibility requirements of 30 WCAG success criteria (SC). The documentation includes 4 atomic ACT Rules that are not related to any WCAG success criteria conformance.

The WAI-Tools project is moving forward, and the goal is to develop 70 ACT Rules by the end of January 2021. The overview of ACT Rules that have been documented in this documentation along with WCAG success criteria covered by these rules are shown in Table 1 below where N/A stands for not applicable:

NOTE: We documented each ACT Rule in this documentation when the rule has been completed after the intensive development process established by the project. There have been many changes along the way until the end of the project period. We may not update all the changes that were made after the ACT Rules are documented in this documentation. Therefore, the dates when the ACT Rule was last updated (completed) and documented in this documentation are also added in each ACT Rule module.

Note: Links in the "ACT-R ID" column of the following Table 1 navigates to the external webpage (ACT Rules website) and links in the "ACT Rules Name" column of the following Table 1 navigates to the internal section of contents (ACT Rules) of this document.

Table 1. Overview of ACT Rules and WCAG success criteria
Rule No. ACT-R ID ACT Rule Name WCAG SC Type
1 5c01ea ARIA state or property is permitted N/A Atomic
2 73f2c2 autocomplete attribute has valid value 1.3.5 Atomic
3 97a4e1 Button has non-empty accessible name 4.1.2 Atomic
4 e086e5 Form field has non-empty accessible name 4.1.2 Atomic
5 2779a5 HTML page has non-empty title 2.4.2 Atomic
6 b5c3f8 HTML has lang attribute 3.1.1 Atomic
7 5b7ae0 HTML page lang and xml:lang attributes have matching values 3.1.1 Atomic
8 23a2a8 Image has non-empty accessible name 1.1.1 Atomic
9 c487ae Link has non-empty accessible name 4.1.2, 2.4.4, 2.4.9 Atomic
10 bc659a meta element has no refresh delay 2.2.1, 2.2.4, 3.2.5 Atomic
11 674b10 Role attribute has valid value 4.1.2 Atomic
12 4e8ab6 Element with role attribute has required states and properties 4.1.2 Atomic
13 de46e4 Element with lang attribute has valid language tag 3.1.2 Atomic
14 bf051a HTML page lang attribute has valid language tag 3.1.1 Atomic
15 6cfa84 Element with aria-hidden has no focusable content 1.3.1, 4.1.2 Atomic
16 cae760 iframe element has non-empty accessible name 4.1.2 Atomic
18 3ea0c8 id attribute value is unique 4.1.1 Atomic
19 5f99a7 aria-* attribute is defined in WAI-ARIA N/A Atomic
20 6a7281 ARIA state or property has valid value N/A Atomic
21 c3232f video element visual-only content has accessible alternative 1.2.1 Composite
fd26cf video element visual-only content is media alternative for text N/A Atomic
ee13b5 video element visual-only content has transcript N/A Atomic
ac7dc6 video element visual-only content has description track N/A Atomic
d7ba54 video element visual-only content has audio track alternative N/A Atomic
22 e7aa44 audio element content has text alternative 1.2.1 Composite
2eb176 audio element content has transcript N/A Atomic
afb423 audio element content is media alternative for text N/A Atomic
23 eac66b video element auditory content has accessible alternative 1.2.2 Composite
f51b46 Video element auditory content has captions N/A Atomic
ab4d13 Video element content is media alternative for text N/A Atomic
24 c5a4ea video element visual content has accessible alternative 1.2.3 Composite
1a02b0 Video element visual content has transcript 1.2.8 Atomic
1ea59c Video element visual content has audio description N/A Atomic
ab4d13 Video element content is media alternative for text N/A Atomic
f196ce Video element visual content has description track N/A Atomic
25 1ec09b Video element visual content has strict accessible alternative 1.2.5 Composite
1ea59c Video element visual content has audio description N/A Atomic
ab4d13 Video element content is media alternative for text N/A Atomic
f196ce Video element visual content has description track N/A Atomic
26 80af7b Focusable element has no keyboard trap 2.1.2 Composite
ebe86a Focusable element has no keyboard trap via non-standard navigation N/A Atomic
a1b64e Focusable element has no keyboard trap via standard navigation N/A Atomic
27 e6952f Attribute is not duplicated 4.1.1 Atomic
28 c4a8a4 HTML page title is descriptive 2.4.2 Atomic
29 cc0f0a Form control label is descriptive 2.4.6 Atomic
31 59796f Image button has non-empty accessible name 1.1.1, 4.1.2 Atomic
32 ff89c9 Aria required context role 1.3.1 Atomic
33 bc4a75 Aria required owned element 1.3.1 Atomic
34 ffd0e9 Heading has non-empty accessible name 1.3.1, 2.4.6 Atomic
35 7d6734 svg element with explicit role has non-empty accessible name 1.1.1 Atomic
36 9eb3f6 Image filename is accessible name for image 1.1.1 Atomic
38 b20e66 Links with identical accessible names have equivalent purpose 2.4.9 Atomic
39 4b1c6c iframe elements with identical accessible names have equivalent purpose 4.1.2 Atomic
40 e88epe Image not in the accessibility tree is decorative 1.1.1 Atomic
41 80f0bf audio or video avoids automatically playing audio 1.4.2 Composite
4c31df audio or video that plays automatically has a control mechanism N/A Atomic
aaa1bf audio or video that plays automatically has no audio that lasts more than 3 seconds N/A Atomic
42 b33eff Orientation of the page is not restricted using CSS transform property 1.3.4 Atomic
44 a25f45 headers attribute specified on a cell refers to cells in the same table element 1.3.1 Atomic
45 d0f69e Table header cell has assigned data cells 1.3.1 Atomic
47 b4f0c3 meta viewport allows for zoom 1.4.4 Atomic
48 36b590 Error message describes invalid form field value 3.3.1 Atomic
49 fd3a94 Links with identical accessible names and context serve equivalent purpose 2.4.4 Atomic
50 59br37 Zoomed text node is not clipped with CSS overflow 1.4.4 Atomic
51 5effbb Link in context is descriptive 2.4.4,
2.4.9
Atomic
53 ffbc54 No keyboard shortcut uses only printable characters 2.1.4 Atomic
54 c249d5 Device motion based changes to the content can be disabled 2.5.4 Atomic
55 efbfc7 Text content that changes automatically can be paused, stopped or hidden 2.2.2 Atomic
57 46ca7f Element marked as decorative is not exposed N/A Atomic
58 0ssw9k Scrollable element is keyboard accessible 2.1.1 Atomic
62 8fc3b6 Object element rendering non-text content has non-empty accessible name 1.1.1 Atomic
65 ucwvc8 HTML page language subtag matches default language 3.1.1 Atomic
67 7677a9 Device motion based changes to the content can also be created from the user interface 2.5.4 Atomic
68 qt1vmo Image accessible name is descriptive 1.1.1 Atomic

Further use of this documentation

This section describes the value of this documentation and suggests how it can be used and further developed. After WAD was introduced and adopted by the member states, the accessibility of websites, applications, and ICT services is becoming one of the legal requirements around the member states in Europe.

According to WAD and the corresponding implementing act, all member states are required to perform monitoring of websites and mobile applications. This monitoring requirement of WAD made it more important for the member states to have a harmonized testing methodology to perform consistent monitoring.

ACT Rules can be used to monitor the degree of compliance with WAD accessibility requirements, which will make it possible for the organisations to know how to fulfil accessibility requirements required by the directive in a consistent manner. Monitoring bodies in member states can use this documentation of ACT Rules and interpretation of accessibility requirements to provide guidelines to businesses and perform controls, audits, and monitoring in their countries.

The Norwegian Agency tasks also include to perform a pilot for simplified and in-depth monitoring in line with Directive (EU) 2016/2102 on a limited set of websites under Work Package 2, to establish a “demonstrator”. The testing of websites was performed using testing tools and/or open-source engines developed by Deque, FCID (University of Lisbon), and Siteimprove that have adopted the test rules developed under the project. Note that the pilot monitoring documentation which includes a description of the sampling approach, testing, analysis, data model, and reporting results is delivered as a separate deliverable in the project.

These efforts provide valuable input and documentation on how a monitoring body enforces the Web Accessibility Directive through the use of the ACT Rules and deliverables from the WAI-Tools project. The complete documentation of WCAG interpretation and test rule is documented in a PDF document.

References

  • W3C, Web Accessibility Initiative (2020). Web Accessibility Initiative - Advanced Decision Support Tools for Scalable Web Accessibility Assessments (WAI-Tools) Project. Retrieved 15.10.2020, from https://www.w3.org/WAI/about/projects/wai-tools/

  • European Commission (2017). Grant Agreement number 780057 – WAI-Tools.

  • How WAI Develops Accessibility Standards through the W3C Process: Milestones and Opportunities to Contribute. Retrieved 15.10.2020, from https://www.w3.org/WAI/standards-guidelines/w3c-process/

  • Interpretation and overview of test procedures for WCAG 2.0 A and AA. Retrieved 15.10.2020, from https://uu.difi.no/om-oss/english/interpretation-and-overview-test-procedures-wcag-20-and-aa

  • Accessibility Conformance Testing (ACT) Rules Format 1.0. Retrieved 15.10.2020, from https://www.w3.org/TR/act-rules-format/

  • ACT Rules Community (2020). Retrieved 15.10.2020, from https://act-rules.github.io/pages/about

  • ACT Rules Community (2020). Rules. Retrieved 15.10.2020, from https://act-rules.github.io/rules/

  • W3C, Web Accessibility Initiative (2020). Evaluation and Report Language (EARL) 1.0 Schema. Retrieved 15.10.2020, from https://www.w3.org/TR/EARL10-Schema/

  • European Commission (2016). Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies.

  • European Commission (2018). Commission Implementing Decision (EU) 2018/1523 of 11 October 2018 establishing a model accessibility statement in accordance with Directive (EU) 2016/2102 of the European Parliament and of the Council on the accessibility of the websites and mobile applications of public sector bodies.

  • European Commission (2018). Commission Implementing Decision (EU) 2018/1524 establishing a monitoring methodology and the arrangements for reporting by Member States in accordance with Directive (EU) 2016/2102 of the European Parliament and of the Council on the accessibility of the websites and mobile applications of public sector bodies.

  • European Commission (2018). Commission Implementing Decision (EU) 2018/2048 of 20 December 2018 on the harmonised standard for websites and mobile applications drafted in support of Directive (EU) 2016/2102 of the European Parliament and of the Council.

  • ETSI, CEN & CENELEC (2018). European standard EN 301 549 V2.V2.1.2 (2018-08) Accessibility requirements for ICT products and services.