The Perfect Search Engine Is Not Enough

1
The Perfect Search Engine Is Not Enough

• Jaime Teevan, MIT

with Christine Alvarado, Mark Ackerman and David
Karger
2
Let Me Interview You!

• Web

• Whats the last Web page you visited? How did
  you get there?
• Have you looked for anything on the Web?

• Email

• Whats the last email you read? What did you do
  with it?
• Have you gone back to an email youve read
  before?

• Files

• Whats the last file you looked at? How did you
  get to it?
• Have you looked for a file?

3
OverviewUnderstanding
Search
Directed

• Introduction
• Related work
• Methodology
• What we learned
• How?
• Why?
• Who?
• So what?

Prefer to search in steps Because its
easier Step size varies by person
4
HaystackPersonal Information Storage
Web pages
Email
Files
Calendar
Contacts
5
Directed Search in Haystack
What was that paper I read last week about
Information Retrieval?
Haystack
6
Directed Search in Haystack
Ah yes! Thank you.
Haystack
7
Or Elsewhere
Ah yes! Thank you.
Perfect Search Engine
8
Related Work

• Directed search
• Lab studies Capra03, Maglio97
• Log analysis Broder02, Spink01
• Observational studies Malone83
• Information Seeking
• Marchionini, ODay and Jeffries, Bates, Belkin,
• Evolving information need

9
Modified Diary Study

• Subjects 15 CS graduate students
• Ten interviews each (2/day x 5 days)
• Two question types
• Last email/file/Web page looked at
• Last email/file/Web page looked for
• Supplemented with direct observation and an
  hour-long semi-structured interview

10
OverviewUnderstanding
Directed
Search

• Introduction
• Related work
• Methodology
• What we learned
• How?
• Why?
• Who?
• So what?

11
Directed Search Today

• Target Connie Monroes office number

? Type into a search engine Connie Monroe,
office number
12
What We Observed
Interviewer Have you looked for anything on the
Web today? Jim I had to look for the office
number of the Harvard professor.
I So how did you go about doing that? J I
went to the homepage of the Math department at
Harvard
13
What We Observed
I So you went to the Math department, and then
what did you do over there? J It had a place
where you can find people and I went to that page
and they had a dropdown list of visiting faculty,
and so I went to that link and I looked for her
name and there it was.
14
What We Observed

• J I knew that she had a very small Web page
  saying, Im here at Harvard. Heres my contact
  information.

15
Strategies Looking for Information
Teleporting
Orienteering
16
Why Do People Orienteer?

• The tools dont work
• Easier than saying what you want
• You know where you are
• You know what you find

17
Easier Than Saying What You Want

• Describing the target is hard
• Cant
• Prefer not to
• Habit
• Whichever way I remember first.
• Search for source
• E.g., Your last email search

18
You Know Where You Are

• Stay in known space
• URL manipulation
• Bookmarks
• History
• Backtracking
• Following an information scent
• Never end up at a dead end

19
You Know What You Find

• Context gives understanding of answer
• I was looking for a specific file. But even
  when I saw its name, I wouldnt have known that
  that was the file I wanted until I saw all of the
  other names in the same directory
• Understanding negative results
• I basically clicked on every single button
  until I was convinced I dont think that it
  exists

20
Individual Strategies

• Search strategies varied by individual
• People who pile information take small steps
• People who file information take big steps

• Where was the last email you found?

• Inbox?
• Elsewhere?

21
File or Pile Email
Filer
Piler
22
How Individuals Search For Files
Filers
Big steps
Pilers
Small steps
23
Applying What We Learned

• ? Support orienteering

• Advantages to orienteering
• Easier than saying what you want
• You know where you are
• You know what you find
• Individual differences in step size

• Meta-info, source, flag sources with info

• URL manipulation, paths apparent, all steps

• Answer context, trusted sources, exhaustive

• Allow for different step sizes

24
Structural Consistency Important
All must be the same to re-find the information!
25
Preserve What User Remembers

• Supports orienteering for re-finding
• Allows access to new information

26
More to Learn from the Data

• Differences in finding v. re-finding
• How organization relates to search
• Importance of type (email, files and Web)
• Looked at v. looked for
• ? Keep in mind population

27
Questions?
Teevan, J., Alvarado, C., Ackerman, M. S. and
Karger, D. R. (2004). The Perfect Search Engine
is Not Enough A Study of Orienteering Behavior
in Directed Search. To appear in Proceedings of
CHI 2004. (Linked from http//www.teevan.org)
28
Relating How and What

• People only keyword search 39 of the time
• What people look for related to how they look

Orienteer to specific information

• Surprise

29
Relating How and Corpus

• Email and files Almost never keyword searched
• Easy to associate information with document
• Web Used keyword search much more often

30
Relating What and Corpus

• Email searches were primarily for specific
  information
• File searches were primarily for documents
• Web searches were more evenly distributed