Evaluation Tools to Promote Web Accessibility Implementation : The EVAL Research and Development Pro

1
Evaluation Tools to Promote Web Accessibility
Implementation The EVALResearch and
Development Project
BrailleNet Association Pierre GUILLOU, Denis
BOULAY, Sylvie DUCHATEAU, Dominique
BURGER Watchfire Michael COOPER
2
Partners

• BrailleNet Association
• Association founded in 1997, France
• With regard to online Accessibility, training
  information sessions and certifications programs
  to promote the international recommendations of
  the W3C/WAI (Web Accessibility Initiative)
• Certification methodology called AccessiWeb
  (www.accessiweb.org)
• www.braillenet.org pierre.guillou_at_accessiweb.org
  
• Watchfire
• Company founded in 1996, Canada
• Online Business Management software and services
• Protection of brand, reduction of risks
  associated with privacy and accessibility
  breaches and enable regulatory compliance.
• www.watchfire.com michaelc_at_watchfire.com

3
Contents

• European situation on Web Accessibility
• EC level
• National levels
• EVAL Program
• Results

4
WCAG at the EC level

• From WCAG .
• E-Europe 2002 2005 Information Society for
  All
• 2001 adoption of WCAG by the EC
• to local WCAGs.
• Vote of national laws on Web Accessibility
  obligations for Public Web sites
• Different interpretations of WCAG

5
WCAGs at the national levels !
6
The current situation in Europe
7
How to implement national policies
8
How to implement national policies

• National Agency for Digital Accessibility.
• Public procurement.
• Web observatory (benchmarking, monitoring, ).
• Working Groups.
• Budget for assistive technologies.
• Budget for helping public Web sites
  (development, training, control).

9
How to implement national policies

• National Agency for Digital Accessibility.
• Public procurement.
• Web observatory (benchmarking, monitoring, ).
• Working Groups.
• Budget for assistive technologies.
• Budget for helping public Web sites
  (development, training, control).
• Set of guidelines with an evaluation
  methodology.
• Accessibility toolkit (Guides, call of tenders,
  requirements, ).
• Online information on Web accessibility,
  sensibilisation and advice.
• Training courses for webmasters, developers,
  students.

10
How to implement national policies

• National Agency for Digital Accessibility.
• Public procurement.
• Web observatory (benchmarking, monitoring, ).
• Working Groups.
• Budget for assistive technologies.
• Budget for helping public Web sites
  (development, training, control).
• Set of guidelines with an evaluation
  methodology.
• Accessibility toolkit (Guides, call of tenders,
  requirements, ).
• Online information on Web accessibility,
  sensibilisation and advice.
• Training courses for webmasters, developers,
  students.
• Conformant tools (evaluation, authoring,
  repairing) for a Quality Framework for
  developers
• Quality Mark for Web accessibility verification.

11
How to implement national policies

• National Agency for Digital Accessibility.
• Public procurement.
• Web observatory (benchmarking, monitoring, ).
• Working Groups.
• Budget for assistive technologies.
• Budget for helping public Web sites
  (development, training, control).
• Set of guidelines with an evaluation
  methodology.
• Accessibility toolkit (Guides, call of tenders,
  requirements, ).
• Online information on Web accessibility,
  sensibilisation and advice.
• Training courses for webmasters, developers,
  students.
• Conformant tools (evaluation, authoring,
  repairing) for a Quality Framework for
  developers
• Quality Mark for Web accessibility verification.

12
The EVAL Program (1/5)

• Main objective to verify that evaluation tools
  can be used to effectively implement national
  accessibility laws.
• Started on February, 2004
• ? First experimentation in France
• implementation methodology of AccessiWeb criteria
• in the Watchfire WebXM platform

13
The EVAL Program (2/5)

• 3 phases
• Analysis of requirements
• Automatic Tests
• Tool-Assisted Manual Evaluation Support

14
The EVAL Program (3/5)

• Phase 1 Analysis of requirements
• common understanding of national standards
• creation of analysis algorithm for each
  evaluation
• evaluation supports different services (audit,
  certification, repair, development, ) with
  different needs (aggregated results, selected
  checkpoints, manual evaluation for non automatic
  testable checkpoints, )

15
The EVAL Program (4/5)

• Phase 2 Automatic Tests
• Watchfire analysed which requirements could be
  automatically evaluated or only manually
• Review by BrailleNet
• Implementation by Watchfire (retrieve HTML pages,
  parse them and analysis them)
• Product was put on a test server and many public
  Web sites evaluations were performed by the 2
  partners to compare results and make some
  improvements

16
The EVAL Program (5/5)

• Phase 3 Tool-Assisted Manual Evaluation Support
• Not all Web accessibility criteria are
  automatically testable
• Collection of data to support manual evaluation
  process was added to the WebXM platform

17
New features in WebXM

• WebXM version 4.0 performs AccessiWeb evaluations
  and new needs as manual evaluation.
• (http//www.watchfire.com/resources/webxm-accessib
  ility.pdf)
• ? Screen shots on next slides

18
1. Select a set of guidelines
19
2. Customize a set of guidelines
20
3. Evaluation results types

• Tool raises
• Error
• Warning
• Passed
• Not Applicable

21
4. Manual evaluation Wizards (1/3)
22
4. Manual evaluation Wizards (2/3)
23
4. Manual evaluation Wizards (3/3)

• Open manual evaluation support by choosing
  Evaluate button by an issue on a page
• Two types
• listsa list of suspect items on the page with
  relevant data in a table
• indicativetransformation of the original page
  to simulate a condition
• Question at bottom whether page passes or fails

24
5. Report formatslist of issues,
guideline-sorted list, checklist
25
5. Report formatslist of issues, list of pages
(1/2), checklist
26
5. Report formatslist of issues, list of pages
(2/2), checklist
27
5. Report formatslist of issues, list of pages,
checklist (1/2)
28
5. Report formatslist of issues, list of pages,
checklist (2/2)

• Developed for external manual evaluation and
  per-page reporting
• Includes the best data WebXM knows about,
  including results of manual evaluation via issue
  classification
• Aggregates scores to higher levels

29
Results of the EVAL program (1/2)

• Cooperation between tool manufacturers and
  standards authors is necessary
• Implementation of new sets of guidelines
• Customisation of set of guidelines
• Manual Evaluation support by us of wizards
• Sets of reports
• Guidelines-oriented report
• Issue Classification
• Checklist report

30
Results of the EVAL program (2/2)

• Automatic WCAG evaluations and standard reports
  are not sufficient to answer Web accessibility
  policies issues.
• Evaluation tools must
• use new sets of guidelines,
• support manual evaluation
• and support specific needs regarding to audit,
  certification, authoring tool, training and
  repair.

31
Thank you !

• Questions are welcome.