DisIntegrated Library Systems and the Future of Searching

1
Dis-Integrated Library Systems and the Future of
Searching
Andrew K. Pace NCSU Libraries October 23, 2006
2
Disclaimers
Who would claim to be that who was not?

• I am a librarian
• I am a department head
• I used to be employed by a library automation
  company
• I was a product manager for OPAC, Z39.50,
  electronic course reserves, and Web
  authentication
• I am an iconoclast and rabble-rouser, but remain
  cynically optimistic about the future of libraries

3
Whats in store?

• How did this happen?
• The wrath of the dis-integrated systems
• The Library Systems Puzzle
• Putting thought into action (NCSU cases)
• The future of ILS vendors
• Integration vs. (and?) Interoperability and the
  future of library automation

4
The Timeline
Integration sophistication
First library automation punchcard circulation
at University of Texas
Mainframe and dumb terminals the birth of
integration
Coming of age era the best of the text-based
library systems
GUI, WWW, RDBMS the great distraction, or the
beginning of legacy status
The dance with the one you brung era. Making
the best of a bad situation.
Stand-alone modules, separated by department or
function
Peril or Promise? XML, Java, OSS development.
The new ILS
1936
1936
1960
1970
1980
1990
2000
2005
1960
1970
1980
1990
2000
2006
5
Where we are

• The RFP has not evolved
• The traditional ILS is a legacy system
• New innovation requires new technology

If we can be competitive with our ILS offerings
and then truly differentiate ourselves through
excellent customer service, new products, and a
powerful vision, we win." Patrick Sommers,
SirsiDynix
Innovation will have to come via integration
with new technology. Trevor Dykstra InfoVision
Amlib
Incremental functionality improvements in the
ILS are more and more expensive. Roland
Deitz Endeavor Information Systems
6
What hath disintegration wrought?

• CONTAINERS
• Federated/Meta Searching
• Reference Linking
• Digital Asset Management
• Institutional Repository
• Learning Management Systems
• Statistics Workstations
• Portals
• Electronic Resource Management
• New online catalogs

• CONTENT
• Cultural heritage collections
• Digital assets
• Pre-print services
• Open Web
• Research data
• Learning objects
• Textual primary sources
• Graphical images
• Conference websites
• Datasets
• Software simulations
• Course resources

7
What hath disintegration wrought?
Muse Global
8
(No Transcript)
9
The library system puzzle
Web
Serials
Catalog
AI / FT DBs
10
Serials
11
Classic Integrated System
MARC Records
Patron Records
Patron self-service
WEBPAC
circ transactions

• websites (856)
• e-books
• e-journals
• databases
• datasets

reserve records
item holdings
serial holdings
Acquisitions Records
Serials Control Records
12
Dis-integrated Library System

• Licensing Files
• ILL Files
• Collection Management Files
• Helpdesk Files
• Statistical Files

alpha list of databases
subject list of databases
web subject guides
Authentication Authorization

• websites (856)
• e-books
• e-journals
• databases
• datasets

e-journal finder
institutional repository
Serials Solutions
TDNet
SFX
alert services
MyLibrary
13
E-MATRIX
ADMINISTRATIVE METADATA
licensing
I L S
Website Catalog E-resources Alert
Services Local DBs Collections Digital
Archives
Data Repos- itories
subscript-ion info
statistics
E-MATRIX
DATA HOOKS
PRESENTATION LAYER
technical support
remote access
Other Databases E-journal finder ETDs Instnl
Repository Etc.
evaluative data
Evaluative Tools
vendor data
14
E-matrix Challenges

• Public interface is secondary concern
• Leveraging existing ILS data
• Adherence to emerging standards
• Avoid solutions looking for problems
• Get rid of the E-

15
(No Transcript)
16
Collection Mgmt evaluative data
SJERMs Journals / Serials Electronic
Resources Databases
Bib Data
Acquisitions and licensing data
Local subjects
Statistical Data
Search / Browse
17
MyLibrary
I L S
Patron Database
MyTOCs
MyAccount
Collection Mgmt evaluative data
SJERMs Journals / Serials Electronic
Resources Databases
My Courses
Bib Data
Acquisitions and licensing data
Course Reserves
Local subjects
Statistical Data
Search / Browse
Bib Data
I L S
18
Catalog
How could I not find an excuse to talk about
NCSUs new Endeca-powered catalog?
19
OPACs
"Most integrated library systems, as they are
currently configured and used, should be removed
from public view." - Roy Tennant, CDL
20
OPACs
E-books
Movies Videos
Media Files
Books
21
OPAC Complainers

• There is certainly no dearth of OPAC
  complainers. You have Andrew Pace (OPACs suck),
  and Roy Tennant (You Cant Put Lipstick on a Pig)
  writing and presenting about the need for change
  (more simplicity) in the OPAC world. I can
  appreciate their arguments for a simpler OPAC
  (not to mention the rest of the system) but other
  then present their arguments, neither has much in
  the way of suggestions nor have they sparked a
  movement among librarians or the automation
  vendors to do anything about the situation.
• -ACRL Blog entry
• Fall 2005

22
What ILS Catalogs Do Well(liberally stolen from
Roy Tennant)

• Inventory control What and where
• Known item searching

To enable a person to find a book of which
either is known author, title, or subject.
- Charles Cutter, Rules for Catalogs
23
What ILS Catalogs Dont do Well(liberally
stolen from Roy Tennant)

• Any search other than known item
• Anything other than books and journals
• Logical groupings of results (e.g. FRBR)
• Faceted browsing
• Relevance ranking
• Sideways searching (suggestions, expansion of
  searches and search targets)

24
NextGen Library Search Tools

• The Next Generation catalog more than just a
  facelift
• WorldCat.org (Beta)
• RedLightGreen (RLG), subsumed by WorldCat
• OCLC Fictionfinder
• Vivisimo clustered search
• Aquabrowser visual context
• Endeca Guided Navigation
• Innovative Interfaces OPAC Pro and Encore
• Ex Libris Primo
• Polaris, AJAX-Enabled OPAC
• SirsiDynix Enterprise Portal System, FAST
• Talis, et al?Web Services
• EBSCO Research Databases
• Georgia PINES, Koha, and the Library 2.0
  Bandwagon
• And of course the entire commercial web

25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
See also http//search.athenscounty.lib.oh.us
30
Existing OPACs are hard to use

• Known item searching works pretty well
  (sometimes), but
• Lots of topical searches and poor subject access
• keyword gives too many or too few results leads
  to general distrust among users
• authority searching is under-utilized and
  misunderstood
• Relevance system sort order
• Unforgiving on spelling errors, stemming
• Response time doesnt meet expectations of
  web-savvy users

31
Valuable metadata is buried

• Subject headings are not leveraged in keyword
  searching
• they should be browsed or linked from, not
  searched
• Data from the item record is not leveraged
• should be able to easily filter based on users
  changing requirements using item type, location,
  circulation status, popularity

32
What is Endeca?

• Software company based in Cambridge, MA
• Search and information access technology provider
  for a number of major e-commerce websites
• Developers of the Endeca Information Access
  Platform

33
Why Endeca?

• Customized relevance ranking of results
• Better subject access by leveraging available
  metadata through facets
• Improved response time
• Enhanced natural language searching through spell
  correction, etc.
• Browse

34
Endeca ProFind Overview
Endeca ProFind
Always Online
NCSU exports and reformats
Data Foundry
Navigation Engine
Parse text files
Raw MARC data
Indices
Flat text files
HTTP
HTTP
NCSU Web Application
Client browser
35
Quick Demo

• http//catalog.lib.ncsu.edu
• Or
• Video

36
Some User Reaction

• This is absolutely the coolest thing I've seen
  all century.
• Will Owen, Head of Systems (UNC Libraries)
• Also, I'm really digging the new NCSU library
  catalog. Very nice."
• - Educause staff (non-librarian)
• The new Endeca system is incredible. It would be
  difficult to exaggerate how much better it is
  than our old online card catalog (and therefore
  that of most other universities). I've found
  myself searching the catalog just for fun,
  whereas before it was a chore to find what I
  needed.
• - NCSU Undergrad, Statistics

37
Usage statistics
38
Usability testing

• For students, relevance ranking is key.
• March 2006 13 continue to page 2
• Faceted navigation is intuitive, even for
  students who dont use it.
• Beware of library jargon
• keyword anywhere, keyword in subject
• User behavior is influenced by previous
  experience.

39
Interesting tidbits (March 2006)

• Keyword searching up 230
• Authority searching down 45
• 3.6 of all searches had spelling corrected
  automatically
• 2.6 of all searches had alternate spelling
  suggestions

40
Future directions

• Experiment with FRBR search/display through
  partnership with OCLC.
• Update circulation status throughout the day.
• Integrate catalog w/other tools through web
  services
• OpenSearch, RSS
• Enrich catalog through external web services
• book jackets, reviews, etc. Amazon/OCLC
• Build modular shopping cart functionality.
• Use Endeca to index local collections.

41

• Metasearch / Federated Search
• Locally mounted databases
• Open Archives
• Image databases
• Institutional repositories
• Search inside, Google Print, Million Book
  project, Microsoft Live Search
• Open Content Alliance
• The 800 pound gorilla

AI / Full-text databases
Web
42
Rumsfelds Law of Metasearch
You metasearch with the standard you have, not
the standard you wish you had.
43
Strong to our finish

• Too often, we have an "eat your spinach" message
  about the library come to the library, it is
  good for you.
• Lorcan Dempsey, OCLC

44
Reversal of fortune
OLD SEARCH MODEL
NEW SEARCH MODEL
45
Our de facto ambassador?

• As we shake hands with Google, Yahoo!, MSN,
  and Ask, it is essential that we create a
  critical mass for libraries. The OCLC RLG
  merger clarifies that there is one central
  organization representing libraries, museums, and
  other cultural organizations in the Web space.
• -Jay Jordan, OCLC

46
The library system puzzle
Serials
AI / FT DBs
Metasearch
ERM Systems
GS
Digital Repositories
Guided Navigation
Catalog
Web
IR
Legacy ILS
47
Where were headed

• Sticker shock.get used to it
• Interoperability
• Web Services
• 3rd parties

We spend too much time fighting over small
market share points, bludgeoning each other only
to result in no significant profit and certainly
a loss of opportunity to move the profession
forward. Vendors and librarians together must
recognize this. Carl Grant, President of VTLS
48
Where were headed, part deux

• Re-integration
• More market consolidation
• More vertical integration
• More Open Source, local development, and
  3rd-party partnerships

25 Public libraries
25 Academic libraries
25 Govt / Special libraries
25 Consortial libraries
25 Financial Inventory Apps
25 Learning Mgmt Systems
25 Portals, Fed. Search, ref linking
25 Serials content and Mgmt
Library systems are changing because library
assets are changing, but re-integration is
inevitable. Verne Coppi, VP for Development
Endeavor Information Systems
49
Eyes on the ball.
The worst level of Internet service that users
will accept is the best level of service they
have ever seen.
-paraphrased from Ron Dunn, Thomson
Learning National Online 1999
50
Building a Dirt Bike
51
Thank You.
http//www.lib.ncsu.edu/staff/presentations
Andrew K. Pace Head, Information Technology NCSU
Libraries andrew_pace_at_ncsu.edu