Flamenco Image Browser: Using Metadata to Improve Image Search During Architectural Design

1
Flamenco Image BrowserUsing Metadata to Improve
Image Search During Architectural Design

• Ame Elliott
• Group for User Interface Research (GUIR) Dept.
  of Architecture
• University of California, Berkeley

2
Problem Statement

• Architects begin new projects by looking at
  images of prior projects
• How are elementary schools put together?
• Whats new in patio materials?
• Current image search interfaces are not helpful
  for image browsing in the early phase of design
• Rely on well-formulated textual queries
• Ignore context of architecture

3
Scope of the Project

• The Flamenco Image Browser is one customization
  of a toolkit for domain-specific search (the
  Flamenco projects)
• The Flamenco Image Browser is one component of
  my dissertation

Understanding Architects Work Practices (ethnogra
phy, task analysis, interviews)
Prototyping Tools .
Hardware Digital Desk
Software Image Search UI
Evaluation
Quantitative
Qualitative
4
Scope of the Project

• The Flamenco Image Browser is one customization
  of a toolkit for domain-specific search (the
  Flamenco projects)
• The Flamenco Image Browser is one component of
  my dissertation

Understanding Architects Work Practices (ethnogra
phy, task analysis, interviews)
Prototyping Tools .
Hardware Digital Desk
Software Image Search UI
Evaluation
Quantitative
Qualitative
5
Flamenco Image Browser

• Interface to a collection of architectural
  images using metadata about the images to support
  search
• Metadata about the images provides
• Query previews giving hints about possible next
  actions
• Ways of broadening or refining search terms

6
Why Use Metadata?

• Open questions about its utility
• How to use metadata not well understood
• Not an obvious thing to do for images
• Contrasts with automated content analysis
• Defies conventional wisdom
• If a picture is worth 1,000 words, let the
  pictures speak for themselves
• Professional architects have different needs than
  curators
• The thumbnail is the whole document, not like
  text search using a title to stand in for a long
  article

7
Faceted Metadata
Planalto Palace Parti Communiste
Francais Pantheon Oscar Neimeyer
Oscar Neimeyer Jacques-Gabriel
Soufflot 20th Century 20th Century
17th 18th C. Brasilia Paris
Paris Stone Curvilinear Stone
Image Architect Period Location Concept
8
Faceted Metadata
Planalto Palace Parti Communiste
Francais Pantheon Oscar Neimeyer
Oscar Neimeyer Jaques-Gabriel
Soufflot 20th Century 20th Century
17th 18th C. Brasilia Paris
Paris Stone Curvilinear Stone
Image Architect Period Location Concept
9
Evaluation Methodology

• Solicit feedback from architects to determine if
  faceted metadata is helpful and how to present it
• Not evaluating if the current metadata in the
  system is the right metadata
• Lo-Fi evaluation of paper prototype
• 1 hour one-on-one with 3 professional architects
• Walk-through interactions on a paper computer,
  users think-aloud (audio recorded), questions
  about the experience
• Informal study of live version
• 1 hour one-on-one with 9 architects /grad
  students, 2 tasks (audio recorded) and a survey

10
(No Transcript)
11
100 images of France in the 1,000 results for
stone
734 images of France in the entire collection
12
Results of Lo-Fi Study

• The difference between searching within results
  and changing to a new search confused users
• The breadcrumb did not help users orient
  themselves when browsing results
• Home gtgt Gehry, Frank gtgt Berlin
• Home gtgt Berlin
• Users were surprised at the outcome and could not
  anticipate which they were getting (refine or
  expand).
• Image search is exploratory, not reductive
• Specific textual queries not important
• If I knew what I wanted to type in the box Id
  just go get it myself.
• Expanding search more important than refining

is different than
13
Goals for Study of Live Web Site

• Make iterative changes to the alternatives to
  identify useful aspects of the UI not
  statistical analysis
• Learn to what extent the metadata is useful for
  searching
• How much text is too much?
• What kinds of queries will the users do?
• Explore how to clarify searching within results
  vs. starting a new search

14
Informal Study of Live Version

• Comparison- a form-based UI to the same
  collection of images
• Task 1 Free form search
• Participants told they are helping to test an
  image search engine
• Asked to talk about a project theyre working on
  and something theyd like to find
• Let them go at it and try to find images theyre
  interested in they can ask any questions they
  want but no formal instructions given
• Task 2 Treasure hunt
• Participants given paper copies of 3 images (but
  no metadata) and told to find them (range from
  easy to hard)
• Exit survey

15
(No Transcript)
16
(No Transcript)
17
About the System

• Web browser used to view Cold Fusion generated
  pages, queries made to a mySQL database
• 36,000 images from the UC Berkeley Architecture
  Slide library
• 5 facets of metadata about each image
• Image title, architect, period, location, and
  concept
• Shallow hierarchy for results in architect,
  period, and location
• Concept facet is from the Getty Art and
  Architecture thesaurus - a 40,000 term
  hierarchical thesaurus of descriptors of the
  built environment this hierarchy not tested in
  the live version

Building Materials
Masonry Finishing
Materials
Stonework Brickwork
Shingles
18
Results Metadata is Helpful

• Very positive feedback about Flamenco
• All 9 participants named the metadata in the
  search results area as their favorite aspect of
  Flamenco
• Metadata was successful at giving hints about
  where to go next
• Perceived as useful These are places I can go
  from here.

19
Results More Metadata Please

• Participants asked for more metadata
• Although there were complaints about the contents
  of the metadata, users still wanted more
• Longer lists of options (more hints)
• Users wanted more control to make very specific
  searches
• Half the participants requested the ability to
  control order of results with metadata
• Juxtapose visible images 2 different ways
• Overview (one image from each project) vs. like
  together ( all images of a project next to each
  other)
• Different than ranking for text retrieval
  (precision, recall), but ordering does matter

20
Results Complaints

• The UI was not successful at clarifying searching
  within results vs. starting a new search
• Only 2 of the 9 participants understood the
  distinction without discussion but they want to
  do both
• The 1/3 of the participants who couldnt find a
  treasure hunt image felt that Flamenco was slow
• Corroborates findings that perceived system speed
  is about finding what you want (Spool 00)

21
Enthusiasm for Metadata

• Contradicted my suspicions that a sledgehammer
  and pick- through-the-rubble approach would be
  preferable
• No-one thought there was too much text
• Adding more text preferable to adding another row
  of images

22
Conclusions

• Metadata is useful for exploratory tasks
• Good at giving hints about where to go next
• Architects want to use metadata to get more
  control
• Of results display
• Build complex queries
• Sometimes the right word is worth more than many
  pictures

23
Contributions

• Increase understanding of what makes a good
  search interface
• Provide insights about how image search differs
  from text search

24
Acknowledgements

• Thanks to
• Marti Hearst my advisor
• Andy Edmonds for excellent coding
• Andy Chou for help with the database
• the Berkeley Museum Informatics Project for
  access to the image collection
• the Flamenco project team for search ideas
• and the members of GUIR for helpful feedback

25
Thanks!
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
Related Work

• Berrypicking model of information seeking
  (Bates)
• Information scent (Pirolli)
• Query previews (Plaisant Shneiderman et al)
• QBIC Blobworld (content analysis)

30
Architects Image Use Practices

• Architects rely on personal image collections
  and libraries of print media
• Images emerge as relevant due to their
  juxtaposition with other images

31
(No Transcript)