Title: Ideas for Integrating Browsing and Search in the CDL
1Ideas for Integrating Browsing and Search in the
CDL
Marti Hearst SIMS, UC Berkeley
http//www.sims.berkeley.edu/hearst
2Goals
- Seamlessly integrate browsing and searching
- Give users a browsing the shelves feeling
- Allow them to discover new things
- Mix and match different concepts in the query
- Do this in an intuitive, unconfusing interface
- Avoid empty search results
3Faceted Metadata
4There are many ways to do it wrong
- Examples
- Melvyl online catalog
- no way to browse enormous category listings
- Audible.com, BooksOnTape.com, and
BrillianceAudio - no way to browse a given category and
simultaneosly select unabridged versions - Amazon.com
- has finally gotten browsing over multiple kinds
of features working this is a recent development - but still restricted on what can be added into
the query
5(No Transcript)
6(No Transcript)
7(No Transcript)
8(No Transcript)
9(No Transcript)
10(No Transcript)
11(No Transcript)
12(No Transcript)
13(No Transcript)
14(No Transcript)
15(No Transcript)
16(No Transcript)
17(No Transcript)
18(No Transcript)
19The Flamenco Project
- Incorporating Faceted Hierarchical Metadata into
Interfaces for Large Collections - Key Goals
- Support integrated browsing and keyword search
- Provide an experience of browsing the shelves
- Add power and flexibility without introducing
confusion or a feeling of clutter - Allow users to take the path most natural to them
- Method
- User-centered design, including needs assessment
and many iterations of design and testing
Yee, Swearingen, Li, Hearst, Faceted Metadata for
Image Search and Browsing, Proceedings of CHI
2003.
20Some Challenges
- Users dont like new search interfaces.
- How to show lots more information without
overwhelming or confusing? - Our approach
- Integrate the search seamlessly into the
information architecture. - Use proper HCI methodologies.
- Use faceted metadata
21Example of Faceted MetadataMedical Subject
Headings (MeSH)
- Facets
- 1. Anatomy A
- 2. Organisms B
- 3. Diseases C
- 4. Chemicals and Drugs D
- 5. Analytical, Diagnostic and Therapeutic
Techniques and Equipment E - 6. Psychiatry and Psychology F
- 7. Biological Sciences G
- 8. Physical Sciences H
- 9. Anthropology, Education, Sociology and
Social Phenomena I - 10. Technology and Food and Beverages J
- 11. Humanities K
- 12. Information Science L
- 13. Persons M
- 14. Health Care N
- 15. Geographic Locations Z
22Each Facet Has Hierarchy
- 1. Anatomy A Body Regions A01
- 2. B
Musculoskeletal System A02 - 3. C Digestive
System A03 - 4. D Respiratory
System A04 - 5. E Urogenital
System A05 - 6. F
- 7. G
- 8. Physical Sciences H
- 9. I
- 10. J
- 11. K
- 12. L
- 13. M
-
23Descending the Hierarchy
- 1. Anatomy A Body Regions A01
Abdomen A01.047 - 2. B
Musculoskeletal System A02 Back
A01.176 - 3. C Digestive
System A03 Breast A01.236 - 4. D Respiratory
System A04 Extremities A01.378
- 5. E Urogenital
System A05 Head A01.456 - 6. F
Neck
A01.598 - 7. G
. - 8. Physical Sciences H
- 9. I
- 10. J
- 11. K
- 12. L
- 13. M
24Descending the Hierarchy
- 1. Anatomy A Body Regions A01
Abdomen A01.047 - 2. B
Musculoskeletal System A02 Back
A01.176 - 3. C Digestive
System A03 Breast A01.236 - 4. D Respiratory
System A04 Extremities A01.378
- 5. E Urogenital
System A05 Head A01.456 - 6. F
Neck
A01.598 - 7. G
. - 8. Physical Sciences H Electronics
- 9. I
Astronomy - 10. J
Nature - 11. K
Time - 12. L
Weights and Measures - 13. M .
25The Approach
- Assign faceted metadata to content items
- Allow users to navigate through the faceted
metadata in a flexible manner - Organize search results according to the faceted
metadata so navigation looks similar throughout - Give previews of next choices
- Allow access to previous choices
26The Flamenco Interface
- Hierarchical facets
- Chess metaphor
- Opening
- Middle game
- End game
- Tightly Integrated Search
27(No Transcript)
28(No Transcript)
29(No Transcript)
30(No Transcript)
31(No Transcript)
32(No Transcript)
33(No Transcript)
34(No Transcript)
35(No Transcript)
36What is Tricky About This?
- It is easy to do it poorly
- Yahoo directory structure
- It is hard to be not overwhelming
- Most users prefer simplicity unless complexity
really makes a difference - It is hard to make it flow
- Can it feel like browsing the shelves?
37Using HCI Methodology
- Identify Target Population
- Architects, city planners
- Needs assessment.
- Interviewed architects and conducted contextual
inquiries. - Lo-fi prototyping.
- Showed paper prototype to 3 professional
architects. - Design / Study Round 1.
- Simple interactive version. Users liked metadata
idea. - Design / Study Round 2
- Developed 4 different detailed versions
evaluated with 11 architects results somewhat
positive but many problems identified. Matrix
emerged as a good idea. - Metadata revision.
- Compressed and simplified the metadata
hierarchies
38Using HCI Methodology
- Design / Study Round 3.
- New version based on results of Round 2
- Highly positive user response
- Identified new user population/collection
- Students and scholars of art history
- Fine arts images
- Study Round 4
- Compare the metadata system to a strong,
representative baseline
39Most Recent Usability Study
- Participants Collection
- 32 Art History Students
- 35,000 images from SF Fine Arts Museum
- Study Design
- Within-subjects
- Each participant sees both interfaces
- Balanced in terms of order and tasks
- Participants assess each interface after use
- Afterwards they compare them directly
- Data recorded in behavior logs, server logs,
paper-surveys one or two experienced testers at
each trial. - Used 9 point Likert scales.
- Session took about 1.5 hours pay was 15/hour
40The Baseline System
- Floogle
- Take the best of the existing keyword-based image
search systems
41Comparison of Common Image Search Systems
42sword
sword
43(No Transcript)
44(No Transcript)
45(No Transcript)
46Evaluation Quandary
- How to assess the success of browsing?
- Timing is usually not a good indicator
- People often spend longer when browsing is going
well. - Not the case for directed search
- Can look for comprehensiveness and correctness
(precision and recall) - But subjective measures seem to be most
important here.
47Hypotheses
- We attempted to design tasks to test the
following hypotheses - Participants will experience greater search
satisfaction, feel greater confidence in the
results, produce higher recall, and encounter
fewer dead ends using FC over Baseline - FC will perceived to be more useful and flexible
than Baseline - Participants will feel more familiar with the
contents of the collection after using FC - Participants will use FC to create multi-faceted
queries
48Four Types of Tasks
- Unstructured (3) Search for images of interest
- Structured Task (11-14) Gather materials for an
art history essay on a given topic, e.g. - Find all woodcuts created in the US
- Choose the decade with the most
- Select one of the artists in this periods and
show all of their woodcuts - Choose a subject depicted in these works and find
another artist who treated the same subject in a
different way. - Structured Task (10) compare related images
- Find images by artists from 2 different countries
that depict conflict between groups. - Unstructured (5) search for images of interest
49Other Points
- Participants were NOT walked through the
interfaces. - The wording of Task 2 reflected the metadata not
the case for Task 3 - Within tasks, queries were not different in
difficulty (tslt1.7, p gt0.05 according to
post-task questions) - Flamenco is and order of magnitude slower than
Floogle on average. - In task 2 users were allowed 3 more minutes in FC
than in Baseline. - Time spent in tasks 2 and 3 were significantly
longer in FC (about 2 min more).
50Results
- Participants felt significantly more confident
they had found all relevant images using FC (Task
2 t(62)2.18, plt.05 Task 3 t(62)2.03, plt.05) - Participants felt significantly more satisfied
with the results - (Task 2 t(62)3.78, plt.001 Task 3 t(62)2.03,
plt.05) - Recall scores
- Task2a In Baseline 57 of participants found all
relevant results, in FC 81 found all. - Task 2b In Baseline 21 found all relevant, in
FC 77 found all.
51Post-Interface Assessments
All significant at plt.05 except simple and
overwhelming
52Perceived Uses of Interfaces
Baseline
FC
53Post-Test Comparison
FC
Baseline
Which Interface Preferable For
Find images of roses Find all works from a given
period Find pictures by 2 artists in same media
Overall Assessment
More useful for your tasks Easiest to use Most
flexible More likely to result in dead
ends Helped you learn more Overall preference
54Facet Usage
- Facets driven largely by task content
- Multiple facets 45 of time in structured tasks
- For unstructured tasks,
- Artists (17)
- Date (15)
- Location (15)
- Others ranged from 5-12
- Multiple facets 19 of time
- From end game, expansion from
- Artists (39)
- Media (29)
- Shapes (19)
55Qualitative Observations
- Baseline
- Simplicity, similarity to Google a plus
- Also noted the usefulness of the category links
- FC
- Starting page well-organized, gave ideas for
what to search for - Query previews were commented on explicitly by 9
participants - Commented on matrix prompting where to go next
- 3 were confused about what the matrix shows
- Generally liked the grouping and organizing
- End game links seemed useful 9 explicitly
remarked positively on the guidance provided
there. - Often get requests to use the system in future
56Study Results Summary
- Overwhelmingly positive results for the faceted
metadata interface. - Somewhat heavy use of multiple facets.
- Strong preference over the current state of the
art. - This result not seen in similarity-based image
search interfaces. - Hypotheses are supported.
57New Features
- Save groups of images and searches
- Find Similar Images
58Advantages
- Users have a feeling of control
- Users can predict what will happen
- Not true of statistical ranking or clustering
- Adding new items to the system changes the
behavior in understandable ways - Users have flexibility
- In ordering of operations
- In combining of operations
59Thank you!
- Marti Hearst
- flamenco.berkeley.edu
- www.sims.berkeley.edu/hearst