The Search for the Holy Grail: Why OneStop Searching is Both Essential and Hopeless

1
The Search for the Holy Grail Why One-Stop
Searching is Both Essential and Hopeless

• Roy Tennant

escholarship.cdlib.org/rtennant/presentations/2002
ala/mars/
2
The Issue

• Most users do not care where the information they
  need comes from, or who provides it (remember,
  only librarians like to search)
• Nor should they have to
• But our systems presently require them to know
  these things
• How can we create systems that minimize what the
  user needs to know to get what they want?

3
Local Catalog
Vendor Dbs
Local Web Site
Remote Catalogs
Remote Digital Content
Local Digital Content
Remote Web Sites
4
Local Catalog
Vendor Dbs
Local Web Site
Remote Catalogs
Remote Digital Content
Local Digital Content
Remote Web Sites
5
Local Catalog
Vendor Dbs
Local Web Site
Remote Catalogs
Remote Digital Content
Local Digital Content
Remote Web Sites
6
Local Catalog
Vendor Dbs
Local Web Site
Remote Catalogs
Remote Digital Content
Local Digital Content
Remote Web Sites
7
Local Catalog
Vendor Dbs
Local Web Site
Remote Catalogs
Remote Digital Content
Local Digital Content
Remote Web Sites
8
(No Transcript)
9
One-Stop SearchingThe Vision
query
10
One-Stop SearchingThe Vision
query
query
query
11
One-Stop SearchingThe Vision
response

• response

response
12
One-Stop SearchingThe Vision

• processed response

13
(No Transcript)
14
What Most Users Want
15
How We Can Give it To Them
The User Interface
Online Reference
The Integration Engine
OAI- Compliant Archives
Google
WorldCat on Steroids
Serial Databases
Digital Library Collections
Local Circulation Systems
16
The Integration Engine

• Parse the query for each database, formulating
  the best possible search for that resource
• Sort, organize, and de-dup the results
• Rank according to perceived relevance
• Be fault-tolerant (does the best it can with what
  its given)
• Offer additional ways to filter or display the
  results

17
Challenges

• Target selection and weighting
• Response time
• Unresponsive targets
• Screen scraping
• Presentation of results
• Merging
• De-duping
• Ranking
• Further options
• Search within results
• Filtering

18
(No Transcript)
19
http//searchlight.cdlib.org/cgi-bin/searchlight
20
Strategies to Consider

• Focused cross-database searching
• A few good things
• By subject area
• Cooperative development (e.g., the ARL Scholars
  Portal project)
• Press vendors for an option to receive search
  results as an XML data stream

21
Why Will it Never be Fully Achieved?

• Market forces
• The pace of change
• Complexity of systems and diversity of data,
  metadata, terminologies, etc.
• Organizational, political, and cultural realities
• Therefore, our search for the one-stop search
  solution is destined to fail as did the knights
  in their quest for the Holy Grail

22
But is it Worth the Quest?
Certainly! Even minor gains will be better than
what we have now! Godspeed!