Karen Calhoun Janet Hawk

1
Online catalogs What users and librarians wantA
review of market research data
OCLC Data Quality Research 2008

• Karen CalhounJanet Hawk
• Prepared for PCC Participants Meeting
• ALA Midwinter, Denver CO25 January 2009

2
With thanks to Joanne Cantrell, OCLC Market
Research Analyst Peggy Gallagher, OCLC Market
Analysis Manager
Photo by allw3ndyhttp//flickr.com/photos/allw3nd
y/2757149584/
3
Where does the library professions definition of
catalog quality come from?
Charles A. Cutter. Rules for a dictionary catalog.
4
30-second summary of online catalog user studies

• Keyword searching reigns
• The default search is chosen most often
• Number of terms in a query 1 to 3
• Search failure rate (zero hits) is very high 20
  to 40 percent

The latest study Moulaison, Heather L. 2008.
OPAC queries at a medium-sized academic library
a Transaction log analysis. LRTS 52 (4)
230-237.
5
Will Google Books usurp the library catalog?
Ludwig, Mark J. and Margaret R. Wells. Google
Books vs. BISON. Library Journal, July 15, 2008.
http//www.libraryjournal.com/article/CA6566451.ht
ml
6
Assumptions and mindsets Where do subject-rich
index terms come from?
LCSH from 3 to 7 wordsper record
Markey, Karen and Karen Calhoun.1987. Unique
words contributed by MARC records with summary
and/or contents notes. Proceedings of the 50th
ASIS Annual Meeting (Medford NJ Learned
Information), p. 153162.
7
What factors influence an end users definition
of catalog quality today?

• Dewey arranged books by subject, but Amazon
  tries to find every way we might want to get from
  the A of a book we know to the B, C, and Z of
  books we dont know, including the fact that lots
  of other people bought Z.
• --David Weinberger, Everything is Miscellaneous,
  p. 62.

8
Assumptions and mindsets What is full?
Product description and purchase information Mor
e like this Editorial reviews and author
info Inside the book tags, ratings, customer re
views, lists and more
Bibliographic information Library
holdings Details Subjects Editions Reviews
Bibliographic information Australian library
holdings
3 more screens
With thanks to David Lankes http//quartz.syr.ed
u/rdlankes/Presentations/2007/ALCTS.pdf
9
You need more book descriptions. Telling me the
author name and book title does not tell me what
a book is about. -High school student-
I would like to preview actual pages from the
books. This would greatly help me educate myself
on the subject matter presented and get a sense
of what the book actually offers. -College
student-
Please link me to the item i'm searching for.
-Graduate student-
10
Objectives of our metadata quality research

• Start over with a blank page
• Identify and compare metadata expectations
• End users
• Librarians
• Compare expectations of types of librarians
• Determine end-user satisfaction with WorldCat.org
• Define a new WorldCat quality program
• Considering the perspectives of all
  constituencies of WorldCat
• End users (and subgroups of end users)
• Librarians (and subgroups of librarians)

11
How did we conduct the research?Research
methodologies

• Focus groups
• Conducted by Blue Bear, LLC
• Pop-up survey on WorldCat.org
• Conducted by ForeSee Results
• Librarian survey
• Conducted by Marketing Backup

12
End-user focus groups

• Three focus groups
• College students, ages 1824
• General public, ages 2559
• Scholars, including academic faculty and graduate
  students
• Format
• Individual usability tests captured comments
  on-screen
• Facilitator-led, group discussion

13
(No Transcript)
14
What did we learn?End-user focus group results

• Key observations
• Delivery is as important, if not more important,
  than discovery.
• Seamless, easy flow from discovery through
  delivery is critical.
• Improved search relevance is necessary.

15
Pop-up survey

• Live on WorldCat.org May 12, 2008
• 11,151 responses through July 9, 2008
• 7,583 were from end users (68)
• Evaluates the metadata most helpful in
  identifying a needed item

16
End user respondents

• Students 28
• Teacher/professor 22
• Business professionals 19
• Retired 9
• Other 21
• End-user country
• USA 56
• Canada 4
• Mexico 3
• United Kingdom 3

• Types of students
• Grad/post-grad 54
• Undergrad 30
• High school 10
• Other 6

• End-user age
• 18 younger 5
• 1930 24
• 31-40 17
• 41-50 20
• 51-60 20
• 61 13

17
What did we learn?Pop-up survey results

• Information most essential in identifying the
  item needed?
• End users (n7535)

Delivery
Delivery
Delivery
Discovery
Discovery
Discovery
Discovery
Discovery
Discovery
Discovery
Discovery
Delivery
Discovery
Discovery
18
Item details in WorldCat.org
The World Is Flat
Lots of detail
Not Quite the Diplomat
Not much detail
19
Subject information in WorldCat.org
6 subject-rich wordsBarackObamaTravelAfricaP
residentialCandidates
20
What did we learn?Pop-up survey suggestions
Changes to help identify an item? End users
(n7535)
21
End-user recommendations

• Improve search relevance
• Add more links to online full text (and make
  linking easy)
• Add more summaries/abstracts Make summaries more
  prominent
• Add more details in the search results (e.g.,
  cover art and summaries)

22
Librarian survey

• In the field September-November 2008 (U.S. and
  non U.S.)
• 1,397 responses North America (64) and outside
  North America (36)
• Evaluates
• The metadata most helpful in identifying a needed
  item
• Attributes liked most about WorldCat
• Recommended enhancements to WorldCat

23
Librarian survey
Current areas of responsibility

• Acquisitions 28
• Cataloging 64
• Collection development or selection 31
• Resource sharing 24
• Reference/public service 44
• Library director/administration 21

58 play more than one role in their library.
24
What did we learn?Librarian survey results
Reactions to WorldCat.orgcompared to end users
DISCOVERYMost essential information
25
What did we learn?Librarian survey results
compared to end-user results
DISCOVERYRecommended enhancements
26
What did we learn?Librarian survey results
TOP 5 recommended enhancements to WorldCatTotal
librarian responses
27
What did we learn?Librarian survey results
Enhancements to WorldCat Top 10 cataloger and
acquisitions responses
28
What did we learn?Librarian survey results
Enhancements to WorldCat Top 10 library director
and public services responses
29
What did we learn?Librarian survey results

• Enhancements for WorldCatTOP 10 responses by
  academic librarians

30
What did we learn?Librarian survey results

• TOP 10 responses inside and outside North America

31
What did we learn?Librarian survey results
32
Recommendations from librarian survey

• Merge duplicates
• Make it easier to make corrections to records
  (fix typos do upgrades) social cataloging
  experimentWikipedia
• More emphasis on accuracy/currency of library
  holdings
• EnrichmentTOCs, summaries, cover artwork with
  content suppliers, use APIs, etc.
• Education about what users say they want

33
A few ideas to discuss

• Catalogs have many audiences, inside and outside
  the library
• With respect to metadata quality, librarians
  and end users definitions generally differ
• Different groups of end users have different
  priorities, but there are some commonalities
  across groups
• The end users delivery experience is as
  important, if not more important than the
  discovery experience
• Most important for analog materials summaries,
  TOCs, etc.
• Most important for licensed e- and digital
  materials the ability to link easily and
  conveniently to the online content itself
• Different groups of librarians have different
  priorities, but there are some commonalities
  across groups
• Merge duplicate records
• Add TOCs

34
Thank You!

• Publication of report expected February 2009
• Download free PDF or purchase printed version
• Many thanks to our reviewers!
• Your comments and suggestions welcome
• Karen Calhoun, calhounk_at_oclc.org
• http//community.oclc.org/metalogue
• Janet Hawk, hawkj_at_oclc.org