Recommendations for a monitoring methodology

Along with Empirica and the European Disability Forum, W3C and COWI, Funka has, on behalf of the European Commission, developed recommendations for a monitoring methodology on web accessibility.

When the Web Directive, the European legislation in the field of accessibility, enters into force, the monitoring will be made with a Europe-wide methodology.

Within the framework of the mission we have:

  • made an overview of the national monitoring methodologies used in EU Member States today
  • reviewed and compared them in terms of quality and reliability, meaning that it must be possible to repeat a test and get the same results
  • compared the results with the W3C evaluation methodology to ensure that the methods test the right thing
  • conducted user testing with persons with disabilities to validate the user experience

Overall, we found 47 European methods of monitoring, made by public bodies in countries with national laws or regulations, made by public bodies in countries without laws or strict rules, but also other types of monitoring, for example, implemented by nonprofit organizations or commercial entities.

Despite some differences, there are many common features

Most current monitoring methodologies use a combination of manual and automated testing. Few countries carry out regular monitoring. End users are very rarely involved in the tests.

Several of the methods are poorly documented, making test results unreliable. Some of the methods we checked in detail had big problems with quality and reliability.

None of the tested methods would, in our view, serve to monitor compliance of the Webdirective.

Funkas recommendations

There are many considerations that must be taken into this kind of recommendations, but based on the results of the study, we have concluded that the most effective is to do several things in parallel. The key is to not only measure the accessibility, but to ensure that the controls at the same time serves to educate website owners.

In brief, our recommendations are to

  • combine self declaration and awareness raising with expert evaluation and the involvement of end users
  • Base the selection of websites to test on the W3C evaluation methodology WCAG-EM
  • Base the tests on the Norwegian method indicators
  • Ensure that the method is constantly evolving

Monitoring methodologies for Web Accessibility in the European Union, opens in new window

Funka develops recommendations for a monitoring methodology on web accessibility for the European Commission


Susanna Laurin

Title: Chief Research and Innovation Officer (Susanna Laurin)

+46 8 555 770 61 (Susanna Laurin)