Recommendations for a monitoring methodology
Along with Empirica and the European Disability Forum, W3C and COWI, Funka has, on behalf of the European Commission, developed recommendations for a monitoring methodology on web accessibility.
When the Web Directive, the European legislation in the field of accessibility, enters into force, the monitoring will be made with a Europe-wide methodology.
Within the framework of the mission we have:
- made an overview of the national monitoring methodologies used in EU Member States today
- reviewed and compared them in terms of quality and reliability, meaning that it must be possible to repeat a test and get the same results
- compared the results with the W3C evaluation methodology to ensure that the methods test the right thing
- conducted user testing with persons with disabilities to validate the user experience
Overall, we found 47 European methods of monitoring, made by public bodies in countries with national laws or regulations, made by public bodies in countries without laws or strict rules, but also other types of monitoring, for example, implemented by nonprofit organizations or commercial entities.
Despite some differences, there are many common features
Most current monitoring methodologies use a combination of manual and automated testing. Few countries carry out regular monitoring. End users are very rarely involved in the tests.
Several of the methods are poorly documented, making test results unreliable. Some of the methods we checked in detail had big problems with quality and reliability.
None of the tested methods would, in our view, serve to monitor compliance of the Webdirective.
Funkas recommendations
There are many considerations that must be taken into this kind of recommendations, but based on the results of the study, we have concluded that the most effective is to do several things in parallel. The key is to not only measure the accessibility, but to ensure that the controls at the same time serves to educate website owners.
In brief, our recommendations are to
- combine self declaration and awareness raising with expert evaluation and the involvement of end users
- Base the selection of websites to test on the W3C evaluation methodology WCAG-EM
- Base the tests on the Norwegian method indicators
- Ensure that the method is constantly evolving
Monitoring methodologies for Web Accessibility in the European Union, opens in new window