A New Standard for Controlled Vocabularies NISO Updates Z39.19

1
A New Standard for Controlled Vocabularies NISO
Updates Z39.19

• Emily Gallup Fayen
• Muse Global, Inc.
• June 8, 2004

2
In the beginning

• ANSI/NISO Z39.19,Guidelines for the Construction,
  Format, and Management of Monolingual Thesauri
  1993
• The most frequently requested NISO Standard
• In spite of its age the Standard is still relevant

3
The 1999 NISO Workshop on Electronic Thesauri

• Identified limitations of Z39.19
• Difficult for non-lexicographers to understand
• Focused on construction and maintenance
• Document indexing applications
• Post-coordinated retrieval
• Printed products
• Outdated technology
• http//www.niso.org/news/events_workshop/thes99rpt
  .html

4
The Need

• A Standard that would provide a better, more
  inclusive way to represent content
• A changing audience
• A changing information environment
• Interoperability and sharing

5
NISO makes plans

• Questions
• Should there be a new Standard?
• Should Z39.19 be revised?
• Should there be a supplement?
• Is there an International Standard we can use?

6
In 2002

• NISO initiates revision of Z39.19
• Generous support from
• The Getty Foundation
• H.W. Wilson Company
• The National Library of Medicine

7
Scope

• Expand beyond thesaurus
• Make more user-friendly
• Explain important concepts
• Explain principles of vocabulary control
• Include electronic information environment

8
Scope (cont.)

• Include additional user search methods
• Browse
• Navigate
• Keyword searching
• Expand beyond A I services
• Include Web applications

9
The Team

• Vivian Bliss Microsoft
• Carol Brent ProQuest
• John Dickert DTIC
• Lynn El-Hoshy Library of Congress
• Marjorie Hlava Access Innovations
• Stephen Hearn ALA
• Sabine Kuhn Chemical Abstracts Service
• Pat Kuhr H.W. Wilson Company
• Diane McKerlie DMA Consulting
• Peter Morville -- Semantic Studios
• Stuart Nelson National Library of Medicine
• Allan Savage National Library of Medicine
• Diane Vizine-Goetz OCLC
• Marcia Lei-Zeng Special Libraries Association

10
Planned Extensions to Z39.19

• Coverage
• Types of vocabularies
• Post-coordinated vs. Pre-coordinated
• Display formats
• Multilingual vocabularies
• Interoperability

11
Coverage

• Expand the scope from documents to content
  objects
• A content object is any item that can be
  described for inclusion in an information
  retrieval system

12
Content objects

• Documents
• Electronic documents and their metadata
• Maps
• Music
• Paintings
• Sculpture
• . . .

13
Types of vocabularies

• Expand from thesaurus to controlled vocabularies
• A slippery slope
• How to determine what is in and out of scope

14
Boundaries

• Not limited to thesauri
• Applicable to areas other than A I
• Suitable for pre-coordinated retrieval
• but probably not for use in
• Patient records
• Lab tests
• Classification schemes
• Other encoding schemes

15
Post-coordinated vs. Pre-coordinated

• Old
• Designed for use with post-coordinated retrieval
  systems
• New
• Extend the Standard to include pre-coordinated
  terms
• Especially for use in web navigation systems

16
Controlled vocabularies

• Lists
• Synonym rings
• Taxonomies
• Thesauri

17
Lists

• A list is a simple group of terms
• Example
• Alabama
• Alaska
• Arkansas
• California
• Colorado
• . . . .
• Frequently used in Web site pick lists and
  pull down menus

18
Synonym Ring

• A synonym ring is a list of synonyms or near
  synonyms that are used interchangeably for
  retrieval purposes
• Example
• Speech disorders
• Speech defects
• Speech, disorders of
• Defective speech
• Frequently used in systems where
• the content is not indexed or
• the indexing vocabulary is not controlled

19
Taxonomies

• A taxonomy is a set of preferred terms, all
  connected by a hierarchy or polyhierarchy
• Example
• Chemistry
• Organic chemistry
• Polymer chemistry
• Nylon
• Frequently used in web navigation systems

20
Thesaurus

• A thesaurus is a controlled vocabulary with
  multiple types of relationships
• Example
• Rice
• UF paddy
• BT Cereals
• BT Plant Products
• NT Brown rice
• RT Rice straw

21
Thesaurus (cont.)

• Relationship types
• Use/Used For indicates preferred term
• Hierarchy indicates broader and narrower terms
• Associative almost unlimited types of
  relationships may be used
• The most complex format for controlled
  vocabularies and widely used

22
Display formats

• Old formats Printed lists
• Alphabetical listing of terms
• Hierarchical listing of terms
• Permuted or rotated listing

23
Display formats

• New formats
• Electronic (online) formats
• PDF version of printed products
• Web-enabled formats
• Browse and navigate (Go to)
• Keyword searching
• Hyperlinks to/from hierarchy, term detail record,
  scope notes, history notes, facets, etc.

24
Multilingual vocabularies

• Old Standard
• limited to monolingual vocabularies
• Highly skewed towards English
• New Standard
• Must include a discussion of multilingual
  vocabularies
• Special retrieval problems

25
Interoperability

• One of the most important issues from the 1999
  workshop
• Question How to
• compare indexes
• perform searches
• merge databases that have been developed using
  different controlled vocabularies?

26

• Most controlled vocabularies are domain specific
• Few have been developed with an idea that anyone
  else might want to use them
• No thought that anyone would want to share them

27
What will help?

• Vocabulary designers should follow the Standard
• Search developers must understand the
  implications of searching using a controlled
  vocabulary
• Valuable lessons to be learned from metasearch
  activities

28
Progress to date

• Working drafts covering
• Introduction
• Scope
• Referenced Standards
• Definitions
• Controlled vocabularies
• Term choice
• Compound terms
• Relationships

29
Under development

• Display Formats
• Interoperability
• Construction and Maintenance
• Vocabulary Management Systems
• Testing and Evaluation

30
Target

• Draft for review by November 2004
• For more information
• Go to http//www.niso.org and search for Z39.19
• Contact me at emily_at_museglobal.com