Image Metadata: What users will want from mature interoperable image retrieval systems

1
Image Metadata What users will want from mature
interoperable image retrieval systems

• Howard Besser
• UCLA School of Education Information
• http//www.gseis.ucla.edu/howard

2
Image Metadata What users will want from mature
interoperable image retrieval systems-

• Developmental Stages
• Metadata background
• Merging Images from 7 Museums (MESL)
• Structural and Administrative Metadata (MOA2)
• Image Technical Information (NISO/DLF)
• Finding Image Origins
• Other Metadata Issues (IPR, Moving Images,
  Complex Objects)

3
Developmental Stages

• Experiment with methods
• Build real operational systems
• Build interoperable operational systems

4
Traditional Digital Library Model
5
Ideal Digital Library Model
6
Developmental Stages

• Experiment with methods
• Build real operational systems
• Build interoperable operational systems
• For DL Initiatives
• For OPACs
• For I A Services
• For Image Retrieval

7
Metadata is not just indexing terms

• CBIR attributes used for retrieval on color,
  shape, texture, etc.
• Structural attributes used for page-turning
• Administrative attributes used for managing a
  digital work over time
• IPR attributes to limit unauthorized use
• Identification attributes to determine what
  application software is needed to view a
  particular digital work
• Can be located anywhere

8
Merging Images from 7 Museums (MESL)

• Project Description
• Inconsistent Metadata Issues-
• Strange Search Results-
• User Needs Assessment-

9
Samples from a MESL Site
10
Samples from a MESL Site
11
Creating New Image Sets (Views)
12
Fields in MESL Data Dictionary (1.1)

• 1. data agreement number
• 2. holding institution
• 3. accession number
• 4. accession method
• 5. credit line
• 6. label
• 7. object type/ objectclass/ object name
• 8. object title/caption
• 9. creator/maker - name
• 10. creator/maker - culture/nationality
• 11. creator/maker - role
• 12. creation place
• 13. creation begin date
• 14. creation end date
• 15. creation technique/method/process
• 16. material/medium

• 17. support
• 18. dimension/extent-quantity-unit
• 19. parts/pieces
• 20. marks/inscriptions
• 21. edition/state
• 22. associated events, people, organizations,
  places
• 23. concepts/subject
• 24. concepts/style-period
• 25. concepts/function
• 26. description
• 27. accompanying image - file name
• 28. accompanying image - caption
• 29. accompanying image - capture data
• 30. accompanying document - file name
• 31. accompanying document - type
• 32. version identification

13

• MESL Data Dictionary
• 1. data agreement number
• 2. holding institution
• 3. accession number
• 4. ccession method
• 5. credit line
• 6. label
• 7. object type/ objectclass/ object name
• 8. object title/caption
• 9. creator/maker - name
• 10. creator/maker - culture/nationality
• 11. creator/maker - role
• 12. creation place
• 13. creation begin date
• 14. creation end date
• 15. creation technique/method/process
• 16. material/medium

• Museum Collectn Mgmt System
• 1. object title/caption
• 2. accession method
• 3. accession number
• 4. label
• 5. credit line
• 6. creation end date
• 7. object type/ objectclass/ object name
• 8. holding institution
• 9. ...
• 10. ...
• ...
• ...
• 99. creation begin date
• 100. data agreement number
• 101. creation
• ...

14
Authority control over artist name

• Goya y Lucinetes, Francisco de (Houston)
• Goya y Lucientes, Francisco Jose de (Harvard)
• Goya, Francisco de (NGA)

15
MESL Technical Info
16
Search Discrepencies
17
Query for surreal
18
Query for haystack
19
Query for oil portraits of children
20
Madonna Query

• columbia (99)
• michigan (66)
• virginia (66)
• cornell (65)
• illinois (65)
• maryland (0)

21
Africa Query

• illinois (273)
• virginia (249)
• cornell (195)
• michigan (104)
• columbia (99)
• maryland (0)

22
Search Discrepancy -- What Happened?

• different mapping btwn original data fields and
  perceived user needs
• different ways in which the various search
  engines work

23
Fields indexed from MESL Data Dictionary (1)

• 1. data agreement number
• 2. holding institution
• 3. accession number
• 4. ccession method
• 5. credit line
• 6. label
• 7. object type/ objectclass/ object name
• 8. object title/caption
• 9. creator/maker - name
• 10. creator/maker - culture/nationality
• 11. creator/maker - role
• 12. creation place
• 13. creation begin date
• 14. creation end date
• 15. creation technique/method/process
• 16. material/medium

• 17. support
• 18. dimension/extent-quantity-unit
• 19. parts/pieces
• 20. marks/inscriptions
• 21. edition/state
• 22. associated events, people, organizations,
  places
• 23. concepts/subject
• 24. concepts/style-period
• 25. concepts/function
• 26. description
• 27. accompanying image - file name
• 28. accompanying image - caption
• 29. accompanying image - capture data
• 30. accompanying document - file name
• 31. accompanying document - type
• 32. version identification

24
Fields indexed from MESL Data Dictionary (2)

• 1. data agreement number
• 2. holding institution
• 3. accession number
• 4. ccession method
• 5. credit line
• 6. label
• 7. object type/ objectclass/ object name
• 8. object title/caption
• 9. creator/maker - name
• 10. creator/maker - culture/nationality
• 11. creator/maker - role
• 12. creation place
• 13. creation begin date
• 14. creation end date
• 15. creation technique/method/process
• 16. material/medium

• 17. support
• 18. dimension/extent-quantity-unit
• 19. parts/pieces
• 20. marks/inscriptions
• 21. edition/state
• 22. associated events, people, organizations,
  places
• 23. concepts/subject
• 24. concepts/style-period
• 25. concepts/function
• 26. description
• 27. accompanying image - file name
• 28. accompanying image - caption
• 29. accompanying image - capture data
• 30. accompanying document - file name
• 31. accompanying document - type
• 32. version identification

25
Fields indexed from MESL Data Dictionary (3)

• 1. data agreement number
• 2. holding institution
• 3. accession number
• 4. ccession method
• 5. credit line
• 6. label
• 7. object type/ objectclass/ object name
• 8. object title/caption
• 9. creator/maker - name
• 10. creator/maker - culture/nationality
• 11. creator/maker - role
• 12. creation place
• 13. creation begin date
• 14. creation end date
• 15. creation technique/method/process
• 16. material/medium

• 17. support
• 18. dimension/extent-quantity-unit
• 19. parts/pieces
• 20. marks/inscriptions
• 21. edition/state
• 22. associated events, people, organizations,
  places
• 23. concepts/subject
• 24. concepts/style-period
• 25. concepts/function
• 26. description
• 27. accompanying image - file name
• 28. accompanying image - caption
• 29. accompanying image - capture data
• 30. accompanying document - file name
• 31. accompanying document - type
• 32. version identification

26
UCB Mellon GrantExamining Faculty Student Use
Usefulness

• What Faculty Do with Digital Images
• Major Issues for Faculty
• Faculty Concerns about teaching with Digital
  Images
• Faculty Concerns about Image Quality and Metadata

27
Faculty Use of Digital Images Major Issues

• technical support
• training
• tools (software and hardware)
• More than just query options
• time commitment

28
Faculty Concerns about Image Quality and Metadata

• Image quality is important, but the quality
  needed is contextual
• For these faculty, digital image quality was no
  worse than slides
• Any metadata delivered must be customizable by
  faculty member

29
MESL Follow-on Projects

• Academic Image Cooperative -- http//www.academici
  mage.org/
• AMICO Project --http//www.amn.org/AMICO/
• Museum Digital Licensing Consortium
  --http//www.digitalmuseums.org/

30
Structural and Administrative Metadata (MOA2)-

• Special Collections Material
• DLF Metadata for Interoperability Testbed
• Administrative Metadata-
• Structural Metadata-

31
Making of America II

• R D
• Distributed Repositories
• Transportation, 1869-1900
• Testbed Project
• Best Practices
• Structural and administrative metadata

32
MOA2 Goal is Interpoerability

• Book example

33
MOA II Classes of Objects

• Continuous Tone Photos
• Photo Albums
• Diaries, journals, letterpress books
• Ledgers
• Correspondence

34
MOA II Metadata

• Administrative Metadata
• for enhancing resource management
• Structural Metadata
• for reflecting internal hierarchies and
  relationships btwn parts
• Raw/Seared/Cooked

35
MOA II Behaviors

• Navigation
• Display/Print

36
NISO/DLF Image Metadata Workshop (4/99) Image
Technical Information Possible Goals

• Metadata fields
• Rules for Field Contents (authority control)
• Core set of necessary fields
• Syntax for expressing fields and contents
  (headers)

37
Image MetadataFocus on Metadata that may prove
helpful for

• management
• use
• preservation
• ...

38
Image MetadataBreak-out Groups Work Done-

• Characteristics and Features of Images
• Image Production and Reformatting Features
• Image Identification and Integrity

39
Image Metadata Elements for Data DictionaryData
Dictionary Entries

• Element Name
• Definition (short) of the element name
• Is the element required? (Identified as
  Mandatory, Mandatory if Applicable, Recommended,
  Optional)
• How is the value of the element represented?
• Examples
• When is this data collected?
• What is the purpose of this data?
• Who would the identified users be?
• How is the metadata used?
• What other metadata standards reference it?

40
Image Metadata Elements for Data
DictionaryCharacteristics and Features Element
List

• Format Issues
• Resolution Issues
• Encoding
• Compression
• Others

41
Image Metadata Elements for Data Dictionary
Image Production Element List (Pertaining to the
Image)

• In-image target(s)
• System target(s), associated with the object
• Responsible agent
• Rationale
• Hardware
• Software

42
Image Metadata Elements for Data Dictionary
Image Production Element List (Pertaining to the
Process)

• Format of the image
• Intrinsic characteristics of the image
• Identification
• Provides a means for defining methodology
  including documentation and rationale
• Who is involved with the file?
• Who created the image file?
• Who commissioned the creation of the image file
  (i.e., the chartering entity), as opposed to Who
  is the responsible agency? Who is the owner?
• Where
• What
• When necessary dates including capture
  date/time, modification
• Checksum
• Navigational aid
• Encoding tools

43
Image MetadataNISO/DLF Image MetadataIn
Progress

• Data Dictionary for both Characteristics
  Features and for Image Production Elements due
  end of 6/00

44
Finding Image Origins
45
Identification/Provenance (Images)-

• The number of variant forms of a work can be
  enormous
• Image Families
• A digital image frequently has many layers of
  parentage
• Information about the parentage that can indicate
  the quality and veracity of the image (Dublin
  Core "Source" and "Relation")
• how to deal with different versions derived from
  the same scan or different encoding schemes
• Vocabulary Standards to express this

46
The number of variant forms of a work can be
enormous

• different views of the same object
• different scans of the same photo
• different resolutions
• different compression schemes
• different compression ratios
• different file storage formats
• different details of the same image
• ...

47
Image Families
48
Identification/Provenance

• how to deal with different versions (browse,
  hi-res, medium res) derived from the same scan or
  different encoding schemes (TIFF, PICT, JFIF)
• Vocabulary Standards to express this
• VRA Surrogate Categories
• CIMI's "Image Elements

49
Are some of the images I retrieved actually
identical to each other?

• Canonical forms

50
Other Metadata Issues-

• For IPR management
• Approaches to Indexing Moving Image Materials
• Structural Metadata for Complex Objects

51
ltIndecsgt

• formal structure for describing and uniquely
  identifying intellectual property itself, the
  people and businesses involved in its trading,
  and the agreements which they make about it
  (primarily for publishing, music, and visual
  arts)
• will develop high-level specifications for the
  services that will be required to implement a
  global IP trading system based on this ltindecsgt
  generic data model
• focus is on encoding rights at a high level, not
  on resource discovery
• likely to involve metadata schma registration and
  directory to allow interoperation of personal
  identifiers for rightsholders and users
• supported by EEC DG-13
• First meeting July 1999
• http//www.indecs.org/

52
Indexing ofMoving Image Materials

• Whole works vs. parts of Works
• MPEG 7
• Approaches to segmentation thumbnail
  representation
• Closed caption indexing
• Audio description indexing
• Semiotics

53
Structural Metadata for Complex Objects-

• MPEG 4
• SMIL

54
Synchronized Multimedia Integration Language
(SMIL)

• For repurposing and reuse in different ways
• Use XML to reference various pieces in different
  ways
• Supported by Realmedia but not Microsoft or
  Macromedia

55
MPEG 4

• Object-oriented
• Very low level of granularity (even objects vs
  backgrounds)
• Scaleable bandwidth use
• Binary Format for Scenes (BIFS) borrows concepts
  from VRML

56
What will we need in mature CBIR systems?

• Look at what users want
• Hooks to allow users to DO something with the
  images
• Interoperability
• Image Metadata Standards

57
Image Metadata What users will want from mature
interoperable image retrieval systems

• http//www.gseis.ucla.edu/howard
• http//sunsite.berkeley.edu/Imaging/Databases/1998
  mellon
• Spring 1999 special issue of Visual Resources
• http//www.gseis.ucla.edu/howard/image-meta.html
• http//www.niso.org/image.html
• http//sunsite.Berkeley.EDU/Imaging/Databases/sta
  ndards
• http//sunsite.Berkeley.EDU/moa2/
• http//www.gseis.ucla.edu/howard/Classes/287-movi
  ng.html
• http//sunsite.Berkeley.EDU/Longevity/
• http//www.nlc-bnc.ca/ifla/II/metadata.htm
• http//www.gseis.ucla.edu/howard/Classes/287-mov-
  index-bib.html

58
(No Transcript)
59
Problems of Rich Media

• Complexity of formats (storage compression)
• Synchronicity between media/streams
• Pieces and Boundaries
• Persistent IDs
• Interactivity
• Historical context
• Content
• Recontextualization (Postmodernism)

60
Opportunities--a scenario

• Huge stable online DB of rich media (Prelinger
  Archives)
• Creators create new works that consist mainly of
  links to and transitions btwn pieces of the rich
  media DB
• Works are not really assembled until run-time
• Securing IP permission may shift from
  capital-intensive producer to end-user
• Economics of media production may change
  drastically

61
(No Transcript)