Towards Automated Quality Assurance for Socially Created Information Assets on the Web

1
Towards Automated Quality Assurance for Socially
Created Information Assets on the Web

• David Davenport Markus Schaal
• Computer Engineering Dept.,
• Bilkent University, 06800 Ankara.
• david_at_cs.bilkent.edu.tr schaal_at_cs.bilkent.edu.
  tr

2
Introduction

• Quality in collaborative web-based projects
• considered good
• no guarantees!

• Outline
• Quality Quality Assurance
• Info. Quality User Expertise
• From Actions to Assessments
• A Research Testbed

3
Quality is

• something good, desirable!
• hard to definemeet or exceed user expectations
• easy to spot lack offails to meet spec.,poorly
  designed,difficult to use, unreliable, etc.

4
Quality is

• an open-ended quest for perfection

5
Quality Assurance

• Assuring improvements requires a process

System/product/service
Make measurements compare
Vision/goal
Implement changes
6
What is Information Quality?

• Criteria
• accessibility, coverage, correctness,
  consistency, recency, reliability, cost, etc.
• Information must inform / educate
• success depends on recipient too!
• Measuring information quality
• either observe user behaviour (individual
  groups)
• or ask their opinion but users are not all equal.

7
Users are not all equal
Modulate user opinion accordingly

• experts
• novices
• subversives!

But, on the web, how can you know your users?
8
From Actions to Assessments
assessmentsof quality
users
System model of user
modulates
contents
User
updates
Systemmodel ofcontent
viewupdaterate
Action
9
Content User Assessments

• Update content user models using evidence
• gathered from implicit observations by
  explicitly asking users, but
• modulated by current assessment of user
  expertise/trust!
• Everything depends on users, encourage by
• making it easy to contribute and rate content
• providing incentives to do so,
• showing them the results!
• What evidence should be gathered how?
• What content user models should be used?
• How should the models be updated?
• How should the results be reported?

Research questions
10
A Research Testbed

• Target students taking CS101/2 Java course
• Lots of Java / computing resources on the
  web(some old, some too complex, some plain
  wrong!)
• Task is to create a quality wiki-like resource.
• Module for Moodle, Drupal, MediaWiki
• Enable distributed users to collaboratively
  build quality information assets on the web.
• Of note
• Log all interactions
• Try out various user, content update models
• Show overall system quality its history

11
To Conclude

• Web is increasingly important info. resource
• Improvements in Quality benefit all
• Need way to evaluate quality,identify
  opportunities for improvements, track changes
• Our research aims to
• produce a clear understanding common quality
  framework
• add-on quality modules usable by others.

12
Thank you for your time attention
contactdavid_at_cs.bilkent.edu.tr /
schaal_at_cs.bilkent.edu.tr