Flexible Search and Navigation using Faceted Metadata

1
Flexible Search and Navigation using Faceted
Metadata

• Prof. Marti Hearst
• Dr. Rashmi Sinha, Ame Elliott, Jennifer English,
  Kirsten Swearingen, Ping Yee
• February, 2002
• University of California, Berkeley
• http//bailando.sims.berkeley.edu/flamenco.html
• Research funded by
• NSF CAREER Grant, NSF9984741

2
Outline

• Motivation
• Approach
• Integrate Search into Information Architecture
  via Faceted Metadata
• Definitions
• Information Architecture
• Faceted Metadata
• Recipe Interface and Usability Study
• Image Interfaces and Usability Studies
• Conclusions

3
Motivation and Background
4
Claims

• Web Search is OK
• Gets people to the right starting points
• Web SITE search is NOT ok
• The best way to improve site search is
• NOT to make new fancy algorithms
• Instead improve the interface

5
The Philosophy

• Information architecture should be designed to
  integrate search throughout
• Search results should reflect the information
  architecture.
• This supports an interplay between navigation and
  search
• This supports the most common human search
  strategies.

6
An Important Search Strategy

• Do a simple, general search
• Gets results in the generally correct area
• Look around in the local space of those results
• If that space looks wrong, start over
• Akin to Shneidermans overview details
• Our approach supports this strategy
• Integrate navigation with search

7
Following Hyperlinks

• Works great when it is clear where to go next
• Frustrating when the desired directions are
  undetectable or unavailable

8
An Analogy
hypertext
9
Main Idea

• Use metadata to show where to go next
• More flexible than canned hyperlinks
• Less complex than full search
• Help users see and return to what happened
  previously

10
Search Usability Design Goals

1. Strive for Consistency
2. Provide Shortcuts
3. Offer Informative Feedback
4. Design for Closure
5. Provide Simple Error Handling
6. Permit Easy Reversal of Actions
7. Support User Control
8. Reduce Short-term Memory Load

From Shneiderman, Byrd, Croft, Clarifying
Search, DLIB Magazine, Jan 1997. www.dlib.org
11
Information Architecture
12
A Taxonomy of WebSites
Catalog Sites Web-based Information Systems
Web-Presence Sites Service-Oriented Sites
high
Complexity of Data
low
low
high
Complexity of Applications
From The (Short) Araneus Guide to Website
development, by Mecca, et al, Proceedings of
WebDB99, http//www-rocq.inria.fr/cluet/WEBDB/pr
ocwebdb99.html
13
An Important IA Trend

• Generating web pages from databases
• Implications
• Web sites can adapt to user actions
• Web sites can be instrumented

14
Faceted Metadata
15
Metadata data about dataFacets orthogonal
categories
16
Faceted Metadata Biomedical MeSH (Medical
Subject Headings)www.nlm.nih.org/mesh
17
Mesh Facets (one level expanded)
18
Questions we are trying to answer

• How many facets are allowable?
• Should facets be mixed and matched?
• How much is too much?
• Should hierarchies be progressively revealed,
  tabbed, some combination?
• How should free-text search be integrated?

19
How NOT to do it

• Yahoo uses faceted metadata poorly in both their
  search results and in their top-level directory
• They combine region other hierarchical facets
  in awkward ways

20
Yahoos use of facets
21
Yahoos use of facets
22
Yahoos use of facets
23
Yahoos use of facets

• Where is Berkeley?
• College and University gt Colleges and
  Universities gtUnited States gt U gt University of
  California gt Campuses gt Berkeley
• U.S. States gt California gt Cities gtBerkeley gt
  Education gt College and University gt Public gt UC
  Berkeley

24
Problem with Metadata Previews as Currently Used

• Hand edited, predefined
• Not tailored to task as it develops
• Not personalized
• Often not systematically integrated with search,
  or within the information architecture in general

25
Recipe Collection Examples
26
From soar.berkeley.edu (a poor example)
27
(No Transcript)
28
From www.epicurious.com (a good example)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
Epicurious Metadata Usage

• Advantages
• Creates combinations of metadata on the fly
• Different metadata choices show the same
  information in different ways
• Previews show how many recipes will result
• Easy to back up
• Supports several task types
• Help me find a summer pasta,'' (ingredient type
  event type),
• How can I use an avocado in a salad?''
  (ingredient type dish type),
• How can I bake sea-bass'' (preparation type
  ingredient type)

33
Metadata usage in Epicurious
Recipe
34
Metadata usage in Epicurious
35
Metadata usage in Epicurious
36
Metadata usage in Epicurious
gt
I
37
Metadata usage in Epicurious
gt
I
Select
Prepare
Cuisine
I
38
Recipe Information Architecture

• Information design
• Recipes have five types of metadata categories
• Cuisine, Preparation, Ingredients, Dish, Occasion
• Each category has one level of subcategories

39
Recipe Information Architecture

• Navigation design
• Home page
• show top level of all categories
• Other pages
• A link on an attribute ANDS that attribute to the
  current query results are shown according to a
  category that is not yet part of the query
• A change-view link does not change the query, but
  does change which categorys metadata organizes
  the results

40
Epicurious Basic Search

• Lacks integration with metadata

41
(No Transcript)
42
Usability Study epicurious
43
Epicurious Usability Study

• 9 participants
• Three interfaces
• Simple search form
• Enhanced search form
• Browse
• Two task types
• known-item search
• browsing for inspiration

44
Epicurious Usability Study

• 9 participants
• Independent Variables
• 1) Epicurious Interface (Basic vs. Enhanced vs.
  Browse)
• 2) Task type (known-item search vs. browsing for
  inspiration)
• 3) Degree of constraint of query
• 4) Number of results required (1 vs. many)
• Dependent Variables
• 1) Time to find satisfactory recipe(s)
• 2) Navigation path (backtracking, starting over,
  revising queries)
• 3) Satisfaction with results of search
• 4) Satisfaction with individual system features
  (e.g. breadcrumbs, query previews, refine by
  hyperlinks)
• 5) Likelihood of using each interface in the
  future.

45
Epicurious Usability Study

• Participants were asked to
• Do 3 pre-specified searches in advance
• In the lab
• Specify a cooking scenario of interest to them
• Search for 3 recipes for this recipe
• Search for each recipe using each of the
  interfaces
• Complete several structured tasks
• Along the way, answer questions about
• Getting closer or farther away from goal
• Satisfaction with search results
• Satisfaction with the interface

46
Epicurious Usability Study Preference Data
47
Epicurious Usability StudyInterface Preference
48
Epicurious Usability StudyFeature Preference
49
Epicurious Usability StudyConstraint-based
Preferences
50
Usability Study Results Summary

• People liked the browsing-style metadata-based
  search and found it helpful
• People sometimes preferred the metadata search
  when the task was more constrained
• But zero results are frustrating
• This can be alleviated with query previews
• People dis-prefer the standard simple search

51
Missing From Epicurious

• How to scale?
• Hierarchical facets
• Larger collection
• How to integrate search?
• How to allow expansion in addition to refinement?

52
Application to Image Search
53
Current Approaches to Image Search

• Visual Content and Cues, e.g.,
• QBIC (Flickner et al. 95)
• Blobworld (Carson et al. 99)
• Body Plans (Forsyth Fleck 00)
• Color, texture, shape
• Move through a similarity space
• Keyword based
• Piction (Srihari 91)
• WebSeek (Smith and Jain 97)
• Google image search

54
A Commonality Among Current Content-based
Approaches

• Emphasis on similarity
• Little work on analyzing the search needs

55
The Users

• Architects and City Planners

56
The Collection

• 40,000 images from the UCB architecture slide
  library
• The current database and interface is called
  SPIRO
• Very rich, faceted, hierarchical metadata

57
Architects Image Use

• Common activitie
• Use images for inspiration
• Browsing during early stages of design
• Collage making, sketching, pinning up on walls
• This is different than illustrating powerpoint
• Maintain sketchbooks shoeboxes of images
• Young professionals have 500, older 5k
• No formal organization scheme
• None of 10 architects interviewed about their
  image collections used indexes
• Do not like to use computers to find images

58
Development Timeline

• Needs assessment.
• Interviewed architects and conducted contextual
  inquiries.
• Lo-fi prototyping.
• Showed paper prototype to 3 professional
  architects.
• Design / Study Round 1.
• Simple interactive version. Users liked metadata
  idea.
• Design / Study Round 2
• Developed 4 different detailed versions
  evaluated with 11 architects results somewhat
  positive but many problems identified. Matrix
  emerged as a good idea.
• Metadata revision.
• Compressed and simplified the metadata
  hierarchies
• Design / Study Round 3.
• New version based on results of Round 2
• Highly positive user response

59
The Interface

• Nine hierarchical facets
• Matrix
• SingleTree
• Chess metaphor
• Opening
• Middlegame
• Endgame
• Tightly Integrated Search
• Expand as well as Refine
• Intermediate pages for large categories

60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
Usability Study on Round 3

• 19 participants
• Architecture/City Planning background
• Two versions of the interface
• Tree (one hierarchical facet at a time)
• Matrix (multiple hierarchical facets)
• Several tasks
• Subjective responses
• All highly positive
• Very strong desire to use the interface in future
• Will replace the current SPIRO interface

69
Study Tasks

• High Constraint Search
• Find images with metadata assigned from 3 facets
• (e.g., exterior views of temples in Lebanon)
• 1.1)       Start by using a Keyword Search
• 1.2)       Start by Browsing (clicking a
  hyperlink) 1.3)       Start by using method of
  choice
• Low Constraint Search
• Find a low-constraint set of images (metadata in
  one facet)
• Specific Image Search
• Given a photograph and no other info, find the
  same image in the collection
• Browse for Images of Interest

70
Interface Evaluation

• Users rated Matrix more highly for
• Usefulness for design work
• Seeing relationships between images
• Flexibility
• Power
• On all except find this image task, users also
  rated the Matrix higher for
• Feeling on track during search
• Feeling confident about having found all relevant
  images

71
Overall Preferences Matrix vs. Tree
Simple search (e.g. images of deserts) Complex search (e.g. exteriors of temples in Lebanon) Find images like this one OVERALL PREFERENCE
Matrix 13 14 16 16
Tree 5 4 3 3
72
User Comments - Matrix

• Easier to pursue other queries from each
  individual page
• Powerful at limiting and expanding result sets.
  Easy to shift between searches.
• Keep better track of where I am located as well
  as possible places to go from there.
• Left margin menu made it easy to view other
  possible search queries, helped in
  trouble-shooting research problems.
• Interface was friendlier, easier, more helpful.
• I understood the hierarchical relationships
  better.

73
User Comments Tree

• Pro
• Simple
• More typical of other search engines Id use
• Visually simpler and more intuitiveMatrix a bit
  overwhelming with choices.
• Con
• I found SingleTree difficult to use when I had
  to refine my search on a search topic which I was
  not familiar with. I found myself guessing.
• SingleTree required more thought to use and to
  find specific images.
• I do not trust my typng and spelling skills. I
  like having categories.

74
Task Completion Times
(Find Image is an artificial task given a photo
and no other info, find it in the collection.)
75
When Given A Choice

• For each interface, one task allowed the user to
  start with either a keyword search or the
  hyperlinks.
• 3 chose to search in both interfaces
• 11 chose to browse in both interfaces
• 4 chose to search in Matrix, browse in Tree
• 1 chose to browse in Matrix, search in Tree

76
Precision and Recall
Computed for tasks 1.1-1.3 Pooling used for
determining relevant set Precision based on what
was visible on screen
77
Feature Usage Percentages
(Dark bars show subtotals)
78
Feature Usage () Types of Actions
79
Feature Usage () Refining
80
Feature Usage Expanding / Starting Over
81
Interface Evaluation

• Users rated Matrix more highly for
• Usefulness for design work
• Seeing relationships between images
• Flexibility
• Power
• On all except find this image task, users also
  rated the Matrix higher for
• Feeling on track during search
• Feeling confident about having found all relevant
  images

82
Application to Medline
83
Summary and Conclusions
84
Summary

• A new approach to web site search
• Use hierarchical faceted metadata dynamically,
  integrated with search
• Many difficult design decisions
• Iterating and testing was key

85
Summary

• Two Usability Studies Completed
• Recipes 13,000 items
• Architecture Images 40,000 items
• Conclusions
• Users like and are successful with the dynamic
  faceted hierarchical metadata, especially for
  browsing tasks
• Very positive results, in contrast with studies
  on earlier iterations
• Note it seems you have to care about the
  contents of the collection to like the interface

86
Summary

• We have addressed several interface problems
• How to seamlessly integrate metadata previews
  with search
• Show search results in metadata context
• Disambiguate search terms
• How to show hierarchical metadata from several
  facets
• The matrix view
• Show one level of depth in the matrix view
• How to handle large metadata categories
• Use intermediate pages
• How to support expanding as well as refining
• Still working on it to some extent

87
Advantages of the Approach

• Supports different search types
• Highly constrained known-item searches
• Open-ended, browsing tasks
• Can easily switch from one mode to the other
  midstream
• Can both expand and refine

88
Advantages of the Approach

• Honors many of the most important usability
  design goals
• User control
• Provides context for results
• Reduces short term memory load
• Allows easy reversal of actions
• Provides consistent view

89
Advantages of the Approach

• Allows different people to add content without
  breaking things
• Can make use of standard technology

90
Some Unanswered Questions

• How to integrate with relevance feedback (more
  like this)?
• Would like to use blobworld-like features
• How to incorporate user preferences and past
  behavior?
• How to combine facets to reflect tasks?

91
Thank you!
For more information

• bailando.sims.berkeley.edu/flamenco.html