Crunching Numbers: OPAC Log Analysis of WebVoyage

1
Crunching Numbers OPAC Log Analysis of WebVoyage
2007 SCVUGM, Stillwater, OK

• Bennett Claire Ponsford
• Digital Services Librarian
• Texas AM University Libraries

2
Overview

• Why analyze your log files
• How to do it
• What we found
• The changes we made
• What the latest logs say
• What next?

3
Why Analyze?

• To see how your users search when youre not
  watching
• To resolve internal disagreements over default
  searches, limits, etc.
• To see whether changes to WebVoyage really
  improved search results
• As a counterpoint to task-based user testing

4

• C.S. Lewis
• Lewis, C. S. (Clive Staples)
• LION WITCH?
• LION, WITCH?
• LION, WITCH, AND WARDROBE?
• Lewis, C. S. (Clive Staples)

5
(No Transcript)
6
Issues to Think About

• Does Voyager capture the data you need?
• Privacy concerns
• Does your network organize data the way you need?
• Staff vs. public IP addresses
• Do you want all searches or a sample?

7
How To

• Read the documentation
• Technical Manual, Chapter 15, Popacjob
• Begin logging your data
• Extract data into Access database
• Clean up data as needed
• Run queries
• Scratch head and contact Tech Support

8
(No Transcript)
9
Data Fields
10
Data Fields (cont.)
11
Data Fields (cont.)
12
(No Transcript)
13
SQL for Count of Search Type

• SELECT Fall_2007_OPAC_log.Search_type,
  Count(Fall_2007_OPAC_log.Search_type) AS
  CountOfSearch_type1
• FROM Fall_2007_OPAC_log
• WHERE (((Fall_2007_OPAC_log.Hyperlink)"N") AND
  ((Fall_2007_OPAC_log.Search_tab)"1") AND
  ((Fall_2007_OPAC_log.Client_type)"W"))
• GROUP BY Fall_2007_OPAC_log.Search_type
• ORDER BY Fall_2007_OPAC_log.Search_type

14
Results
15
June 2006 (Voyager 5)
16
September 2006 (Voyager 5)

• Changed interface
• Defaults
• Kept Tab at Simple Search
• Changed Search to Keyword (CMD with javascript)
• Changed result sort to by relevance

17
Fall 2006

• Preparing to upgrade to Voyager 6.1
• New keyword searches with to automatically AND
  words together
• Some people unhappy with recent changes
• Default search
• Search results sort order
• Decided to look at the data

18
Decisions upgrading to V6

• Basic data
• Where are our searchers
• What search tab are they using
• How are they searching
• Default search
• Order of title searches
• Simple limits

19
Where Are Our Searchers?
20
What Search Tab Used?
21
Default Search Discussion

• Title search (TALL)
• What we traditionally had used
• References preference
• General keyword search (new GKEY)
• What users are used to in a Google world
• More forgiving search

22
(No Transcript)
23
(No Transcript)
24
Default Search Decision

• General keyword search (new GKEY)
• User preference
• Fewer No Hit results

25
First Title Search Discussion

• Left anchored title (TALL)
• Preferred by Reference
• Title keyword (new TKEY)
• More forgiving

26
Title Search (TALL) Problems
27
Title Search Decision

• Title keyword
• Left-anchored title had too many problems

28
Simple Limits

• Several additional location limits requested
• Concern that too many would be confusing

29
Search Limits Used
30
Simple Limits Decision

• Added new limits and will evaluate with more data

31
(No Transcript)
32
Analysis of Voyager 6 Logs

• Improved differentiation library staff and public
  IP addresses

33
Where are our users?
34
Keyword and Subject Searches
35
Author Searches
36
Title Searches
37
Location Limits Used
38
Comparison of Limits
39
Have Changes Helped?

• Search frequency
• No hits percentage

40
Search Frequency
41
No Hit Percentages
42
Detailed No Hit Percentages
43
Analysis of No Hit Searches

• Do we have the title?
• Why did the search not find it?
• What can we do to help?

44
No Hit Title Searches Do We Own Them?
45
No Hit Title Searches Problems
46
What Next?

• Continued analysis of searches with no hits
• Analysis of search repair strategies
• Word counts

47
Improvements Spelling

• Spellchecking
• Automatic searching of variant spellings
• or and
• British vs. American spellings
• Numbers
• Abbreviations
• Did you mean? Suggestions based on field
• Working on using ASPELL to create spellchecker

48
Improvements Help

• More granular no hits help
• Specific search types
• Any search with conference or proceedings in
  it
• Journal title searches including vol., no.,
  or a number
• Searches with more than 4 or 5 words
• More granular help for too many hits

49
Improvements Specific Searches

• Keyword searches
• Automatic stemming
• Ignore punctuation and spacing
• Ignore stop words
• Title searches
• Ignore initial article

50
More Information

• Jansen, Bernard J. Search log analysis What it
  is, whats been done, how to do it, Library
  Information Science Research, 28 (2006) 407-432.
• Yu, Holly and Margo Young, The impact of Web
  search engines on subject searching in OPAC,
  Information Technology and Libraries, 23 (2004)
  168-180.

51
Contact Information

• Bennett Claire Ponsford
• bennett.ponsford_at_tamu.edu
• 979/845-0877
• https//libcat.tamu.edu (production)
• http//surprise-am.tamu.edu (test)