Where all the items are above average: Assessing information quality in contemporary digital setting

1
Where all the items are above average Assessing
information quality in contemporary digital
settings

• Miles Efron
• School of Information
• University of Texas, Austin
• miles_at_ischool.utexas.edu
• http//www.ischool.utexas.edu/miles

2
Modern Information Literacy

• An information literate individual is able to
• Determine the extent of information needed
• Access the needed information effectively and
  efficiently
• Evaluate information and its sources critically
• Incorporate selected information into ones
  knowledge base
• Use information effectively to accomplish a
  specific purpose
• Understand the economic, legal, and social issues
  surrounding the use of information, and access
  and use information ethically and legally

(ACRL, 2000)
3
Modern Information Literacy
The uncertain quality and expanding quantity of
information pose large challenges for society.
The sheer abundance of information will not in
itself create a more informed citizenry without a
complementary cluster of abilities necessary to
use information effectively.
(ACRL, 2000)
4
Evaluating Online Info Quality

• Online information is increasingly important in
  contemporary research, but also increasingly
  difficult to evaluate.
• This difficulty is especially keen at
  institutions like ASU with large proportions of
  millennials in the user population.
• Reference librarians stand to excel in the job of
  teaching evaluation skills.

5
Evaluating Online Info Quality
This is not to say that teaching students to
evaluate information quality in traditional,
print-based environments is not crucial it
is. For students in the liberal arts and
sciences, clearly a broad palette of resources is
essential for good work.

• Online information is increasingly important in
  contemporary research, but also increasingly
  difficult to evaluate.
• This difficulty is especially keen at
  institutions like ASU with large proportions of
  millennials in the user population.
• Reference librarians stand to excel in the job of
  teaching evaluation skills.

6
Evaluating Online Info Quality

• Online information is increasingly important in
  contemporary research, but also increasingly
  difficult to evaluate.
• This difficulty is especially keen at
  institutions like ASU with large proportions of
  millennials in the user population.
• Reference librarians stand to excel in the job of
  teaching evaluation skills.

7
Evaluating Online Info Quality

• Online information is increasingly important in
  contemporary research, but also increasingly
  difficult to evaluate.
• This difficulty is especially keen at
  institutions like ASU with large proportions of
  millennials in the user population.
• Reference librarians stand to excel in the job of
  teaching evaluation skills.

1. (Sullivan, 2003 Janes, 2003 p. 38)
8
Evaluating Online Info Quality

• Online information is increasingly important in
  contemporary research, but also increasingly
  difficult to evaluate.
• This difficulty is especially keen at
  institutions like ASU with large proportions of
  millennials in the user population.
• Reference librarians stand to excel in the job of
  teaching evaluation skills.

2. (Janes, 2003, p. 39 ARL, 2004)
9
Current Info Literacy Pedagogy

• A strong body of literature and best practices
  concerning teaching online information evaluation
  already exists3.
• Emphasis on interrogating author credentials,
  motivation, competence.
• Combines evaluation of non-Internet information
  with techniques aimed specifically at Web
  resources.

3. (cf. Bopp Smith, 2000)
10
Maturing Technologies, new Literacy Challenges

• Contributor-run digital libraries (e.g. Wikis,
  Amazon) author incentives.
• Blogs information commons4 but also insulators
  and incubators
• Adversarial Information Retrieval how long will
  computational models of trust remain useful?

4. (ALA, 2005).
11
Amazon Incentive and Objectivity
12
Amazon Incentive and Objectivity
Mean stars per book
Mode stars per book
13
Political Blogs Style, Authority, and Bias
14
Adversarial Information Retrieval

• Computational models of authority (such as
  Google' PageRank) figure importantly into our
  ability to filter large bodies of online
  information.
• Fraud against such models has spawned a new
  research area 'adversarial info retrieval.'
• Authority models are now subject to an unnerving
  arms race dynamic.

15
Example Separating the wheat from the chaff

• As especially vibrant news and current events
  forum slashdot.org.
• Slashdot allows readers to moderate the
  discussion cooperatively.
• Part of this process each reader must define a
  "quality threshold" that governs what he or she
  reads on the site.
• Defining this threshold (and similar choices)
  constitutes an important information skill.

16
Example Separating the wheat from the chaff
17
Information Literacy and the Information Commons

• What new roles will reference librarians play in
  this domain?
• Instruction in interpreting not only the quality
  of resources, but the incentives and mechanisms
  for ensuring quality that shape them.
• Demonstration (quantitative and qualitative) of
  the need for interrogation of information
  quality.
• Positioning ourselves as guides to interpreting
  and participating in the information commons.

18
Sources

• American Library Association ALA. (2005).
  info-commons.org an online publication
  advocating access to ideas. Retrieved October 10,
  2005 from http//www.info-commons.org/.
• Association of College and Research Libraries
  ACRL. (2000). Information Literacy Competency
  Standards for Higher Education Electronic
  Version. Retrieved October 10, 2005 from
  http//www.ala.org/ala/acrl/acrlstandards/informat
  ionliteracycompetency.htm.
• Association of Research Libraries ARL. (2004).
  ARL Statistics Interactive Edition Electronic
  Version. Retrieved October 10, 2005 from
  http//fisher.lib.virginia.edu/arl.
• Bopp, R. E. Smith, L. C. (2000). Reference and
  Information Services an Introduction, 3rd
  Edition. Libraries Unlimited.
• Efron, M. (2003). The liberal media and
  right-wing conspiracies Using cocitation
  information to estimate political orientation in
  Web documents. In Proceedings of CIKM 2004, pp.
  390-398. New York ACM Press.
• Janes, J. (2003). Introduction to Reference Work
  in the Electronic Age. Neal-Schuman Publishers.
• Sullivan, D. (2003). Searches Per Day. Retrieved
  October 10, 2005 from http//searchenginewatch.com
  /reports/article.php/2156461.