Thesaurus Federations: Construction, Applications, and Experiences Open Forum on Metadata Registries

1
Thesaurus FederationsConstruction
Applications and ExperiencesOpen Forum on
Metadata Registries January 19 2000

• Ralf Nikolai
• Forschungszentrum Informatik (FZI)Karlsruhe
  Germanye-mail nikolai_at_fzi.dehttp//www.fzi.de/d
  bs/dbs.html

2
Outline

• Motivation
• Basic Definitions
• Thesaurus Integration Challenges
• Thesaurus Federation
• Architecture
• Representing Relationships and Conflicts
• Analyzing Component Thesauri
• Phase Model for Concept Integration
• Conclusions

3
Motivation

• Emerging Market Information Brokering
• comprehensive searchescovering several
  autonomous information systems
• information exchange between different user
  groups and systems
• basis common understanding of meaning
• Example European Environmental Information
  Services (EEIS)
• General Environmental Information
• Catalogue of Data Sources (CDS)
• thesaurus General Multilingual Environmental
  Thesaurus(GEMET)
• Earth Observation Information
• INFEO
• thesaurus GCMD valids

4
Scenario EEIS
5
Basics Thesaurus

• Thesaurus Ordered set of concepts and terms
  which are usually used in one domain for
  indexing storing and retrieval of information
  bearing objects.
• Base types of relations
• equivalent relationrelates terms representing
  the same concepts
• hierarchic relationrelates sub-/superordinated
  concepts (subtypes generic partitive
  instance-of)
• associative relationexpresses additional
  manifold relations between concepts(e.g.
  antonyms)
• optional definitions thematic grouping

6
Basics Thesaurus vs. Ontology

• Ontology Definitions Explicit specification of
  a conceptualization (Gruber)and many more
  definitions
• A definitional thesaurus can be seen as one kind
  of ontology!
• no axioms rules and functions
• Our work concentrates on thesauri
• use of thesauri in many information systems
• small set of clearly defined relations
• good common understanding

7
Thesaurus Integration Challenges

• thesaurus integration has frequently to be done
• requirements reference model semi-automatic
  integration
• thesauri are distributed
• basis common communication protocols
• requirements overcome distance distributed
  query processing
• thesauri are autonomous
• just as the information systems are
• requirements handling of inconsistencies change
  management
• software systems for thesauri are heterogeneous
• data model
• data schema
• interfaces
• requirements overcome heterogeneity of systems

8
Thesaurus Integration Challenges

• overlapping content of different thesauri
• e.g. (partial) equivalencies of concepts
• complex semantic relationships between concept
  hierarchies
• requirement integrated view on overall set of
  concepts
• objective and quality very different
• e.g. use of relationsdefinition of concepts
  thematic classification of conceptsdensity of
  network of concepts
• requirements analysis before integration
  adequate integration
• evaluation of integration result
• requirement methodology for judging the quality
  of integration

9
Architecture for Federated Information Systems

• state of the art
• federated database management systems (fDBMS)
• autonomous component systems federated schema
• federation services for global applications
• I3 reference architecture (Wiedehold Stanford)
• wrapper mediator facilitator
• deficiencies
• fDBMS data sources must be databases
• I3 very generic architecture transfer has to be
  done
• thesauri and thesaurus based value adding
  services are not considered

10
Architecture for Federated Information Systems
Reference Architecture with Thesaurus Federation
11
Classical Multi-Thesaurus Systems

• multi-thesaurus environments
• simultaneous access to several thesauriwithout
  inter-thesaurus relationships
• thesaurus switching systems
• mapping of terms (exact match nearest match)
• sometimes use of interlingua
• common for thesauri covering the same domain
  (e.g. medicine)
• thesaurus compounds
• thesaurus couplings (micro/macro thesauri)
• relate top-term of specific micro thesaurus to
  specific term of macro thesaurus
• thesaurus unions
• merging of thesauri results in a new consistent
  thesaurus(inconsistencies are eliminated by
  restructuring procedures)

12
Classical Multi-Thesaurus Systems

• deficiencies
• equivalence relationships not adequate for
  different types of inter-thesaurus relations
• huge manual effort to resolve all conflictsresp.
  loss of semantic by ignoring all inconsistencies
• missing consistency rules and constraints(undefin
  ed design space)lead to difficulties by
  detecting and handling inconsistencies

13
Multi-Thesaurus Systems

• Our Approach Thesaurus Federations
• loosely integrated autonomous heterogeneous
  thesauri
• distinction between indexing/non-indexing
  thesauri
• transfer types of intra-thesaurus relations to
  inter-thesaurus relations
• semantic richness as in component thesauri
• common to users
• enables re-use of software systems by simple
  enhancements
• store rejected inter-thesaurus relationships
• avoid to suggest them again
• enables marking of polysems/homonyms
• allow to add concepts
• explicit consistency rules and constraints
• enables algorithms for detection and handling of
  inconsistencies
• situation dependent resolving of inconsistencies
• coherent overall view for user

14
Thesaurus Federation Inconsistency
e.g. invariant freedom of redundancy
15
Inter-Thesaurus-Relations
How to fill the thesaurus federation database
Concept integration!
16
Concept Integration

• Tasks
• find classify and establish inter-thesaurus-rela
  tionships
• consider semantic of concepts and
  relations(concept hierarchies)
• consider thesaurus-specific characteristics
• find and mark inconsistencies
• judge quality of integration result
• Approaches
• document oriented
• needs set of documents indexed with different
  thesauri
• thesaurus oriented
• several approaches but e.g.no quality and
  compatability analysis of component thesaurino
  explicit statement how integration process is
  influencedno computer support for handling
  inconsistencies

17
Model for Analyzing Component Thesauri

• Independent Analysis
• human expert needed for objective statements
  about quality
• useful indicators computable numbers easy to
  detect facts
• General Criteria (Costs Languages according to
  norms)
• Functional Criteria (presentation maintenance
  software interfaces)
• Content Criteria
• terms (concepts equivalence quota foreign
  words quota pre-coordination level
  definitions)
• relations (connectivity accessibility ratio
  polyhierarchy polydimensionality)
• structure (thematic groups (number size) width
  height)
• Compatibility Analysis
• Formal Criteria (use of relations last revision
  form of definitions)
• Content Criteria (domain compatibility of
  groups lexical identitiesconflicts comparison
  of results of independent analysis)

18
Analyzing Component Thesauri

• Example GEMET 1.5 and AGROVOC 3.0

19
Analyzing Component Thesauri Result
Interpretation

• basis to configure integration procedures e.g.
  
• consider different semantic of relations
• AGROVOC GEMET no distinction of subtypes of
  hierarchy relations (generic partitive
  instance-of)
• pre-processing necessary
• AGROVOC hierarchic relationships not valid in
  any context are stored as associative
  relationships
• more emphasis on associative relationships
• GEMET synonyms are reliable AGROVOC synonyms
  may point to several descriptors
• additional procedure for AGROVOC 1n synonyms
• foreign words in AGROVOC Latin animal/plant
  names
• if integration is needed special knowledge
  required
• general decision use more scientific reference
  system GEMET to integrate retrieval thesaurus
  AGROVOC

20
Analyzing Component Thesauri Result
Interpretation

• specification of integration expectations
• improved division of GEMET concepts
• decrease number of horizontal polydimensionalities
  
• improved connectivity for GEMET concepts
• improved equivalence quota
• discovering of different abstraction levels of
  siblings in AGROVOC
• improved hierarchical structure for AGROVOC
  concepts
• integration of AGROVOC top terms which could be
  assigned to GEMET thematic groups (improved
  accessibility ratio for top terms)
• without Latin top terms 96

21
Term Integration Phase Model
nomalize terms enrich information structure
analysis of component thesauri analysis of
compatabilityspecification of expectations
configuration of process detection of potential
relationships iterative improvement detection
of inconsistencies
acceptance/rejection establishing add concepts
evaluation of federationuser feedback
improvementsenhancements change management
22
Conclusions

• integration of retrieval vocabulary important
  challenge
• interoperability on a semantic level
• classical thesauri still most important kind of
  retrieval vocabulary
• handling of vague semantic necessary
• common understanding very helpful
• norms are one step
• quality criteria additional required
• clearly defined consistency rules and constraints
  required
• for the thesaurus developer/maintainer
• to enable computer support for consistency checks
• reference model simplifies individual integration
  process
• avoid reinvention of the wheel again and again

23
Thesaurus Federations
For further information please contact
Ralf Nikolai Forschungszentrum Informatik FZI Kar
lsruhe Germany Tel. [phone] e-mail
nikolai_at_fzi.de http//www.fzi.de/dbs.html