Capturing and Modeling NeuroRadiological Knowledge on a Community Basis: The Head Injury Scenario.

1
Capturing and Modeling Neuro-Radiological
Knowledge on a Community Basis The Head
Injury Scenario.

• Alexander Garcia, Zhuo Zhang,
• Menaka Rajapakse, Christopher J. O. Baker,
• and Suisheng Tang
• Data Mining Department
• Institute for Infocomm Research
• Singapore

2
Outline

• Motivation
• MiBank Head Injury Database
• Ontology Development
• Collective Intelligence
• The facebook approach
• Medical Image Annotator
• Discussion and Conclusions

3
Motivation

• National Neurological Institute Singapore (NII)
  has 500 head injury patients each year with
  Brain, Scalp, Skull, Internal bleeding requiring
  rapid diagnosis.
• Clinical radiology reports comprise of multiple
  series of Computed Tomography (CT) Images with
  unstructured text associated to images)
• Computationally a weak association between images
  and words, cannot retrieve similar images.
• Conceptually a tightly coupled association
  between Image and Diagnosis
• MiBank database of DICOM files,
• (http//dicom.i2r.a-star.edu.sg/pacsone/)

4
Features of MiBank

• Browser
• Search by category, patient, report, note, study
• Annotation with free-text
• Forum discussion
• DICOM viewer
• Image upload download

5
MiBank Medical Image Databank Head Injury
Web site http//dicom.i2r.a-star.edu.sg/pacsone/
505 studies, 1775 series, 31561 images. Pass word
protect, DICOM viewer, searchable
6
Possible query in MiBank
Show me all cases who have skull fracture with
acute subdural hematoma But do not have brain
edema.
Details in next page
Impossible query with no-predefined terms
Show all cases who have skull fracture with
midline shift and acute subdural hematoma But do
not have brain edema.
7
Current Limitations of MiBank

• Can not query based on image features
  explicitly
• Can not associate the description in R-report
  to specific instance of an image.

Need to see all instances for bone fracture
Sample Radiology Report

• A fracture of the right frontal bone.
• Mild midline shift to the left is present.
• An acute extradural hematoma, measuring
• 1.9 cm in maximal thickness, is noted.
• A 1 cm thick acute subdural hematoma is
• also present over the right cerebral
• hemisphere.

8
What do we want What do we need?

• Properly annotated data images, radiology
  reports
• Meaningful associations between reports, images,
  and across images
• an ontology .

• Retrieve patients with right midnight shifts of
  less than 3mm for whom there has been no reported
  haematoma
• Retrieve all images similar to this one

9
Header InfoMining
Semantic Query
Ontology
OriginalDICOM data
Categorization
Image retrieve
Web Interface
Head Injurydatabase(Relational)
Indexing
Customized online report
Reportdata
Statistic report
Text mining
Search Engine
Discussion forum
Visualization
10
The Role of the Ontology

• Community defined controlled vocabulary for
  annotation of radiology images.
• Hierarchical descriptions of medical terms
  relevant to anatomy, pathology and head injury
  specific features found in medical images.
• Consensus model of head injury terminology
  generated through community engagement for
  knowledge reuse in medical information systems.
• Query model for semantic search

11
Ontology Development
Garcia et al
12
Ontology Development
P h a s e 1
P h a s e 2
13
Text Processing / Baseline Ontology
FMA
Non FMA

• Plain scans were acquired. Note is made of the
  MRI dated 2/3/2004 and CT dated
  18/2/2004.Evidence of previous left high parietal
  craniectomy noted. Hypodensity in the left
  parietal-occipital region is compatible with
  gliosis at site of previous surgery. A large
  left-sided scalp hematoma is seen. Underlying
  linear radiolucency in the left frontal bone was
  seen. This suggests an undisplaced fracture.
  Underlying acute subdural hematoma is seen with a
  maximal depth of 1.2 cm. Acute subarachnoid blood
  is also noted collecting mainly in the
  ipsilateral cerebral hemisphere, sylvian fissure
  as well as tentorium. There is diffuse cerebral
  edema. Mass-effect is seen with midline shift to
  the right, and developing hydrocephalus. Basal
  cisterns are effaced.

FMA / Galen / R-report terms anatomy, pathology,
trauma, injury
14
Capturing Knowledge Phase 1
Not an easy task
Disadvantages

• Requires excessive amount of time
• Experts easily bored no short term result.
• Results in the creation of unstructured knowledge
  stores that are difficult to reuse and maintain.
• Skimping on validation may include errors,
  omissions, inconsistencies irrelevances
• Experts are not always capturing the evidence
  rather explaining context
• Storing the knowledge that is not
  machine-readable

• Inside experts head
• Difficult to describe
• concepts and relations
• Difficult for non-
• experts to understand.

15
Ontology Development
Maintenance
Evolution
P h a s e 1
P h a s e 2
16
Capturing Knowledge Phase 2

• Collective Knowledge Resources
• intelligent collection?
• collaborative bookmarking, searching
• database of intentions
• clicking, rating, tagging, buying - Amazon
• what we all know but hadnt got around to saying
  in public before
• blogs, wikis,
• discussion lists -

• Knowledge Elicitation via Collective Intelligence
• The capacity to provide useful information based
  on human contributions which gets better as more
  people participate.
• Data Types
• mix of structured, machine-readable data and
  unstructured data from human input

17
Tags Make The Difference !
facebook

• The Premise
• From unstructured and unrelated annotation to
  structured meaningful annotation
• Simple tagging it possible to derive meaningful
  associations
• Need to have a tool to gather knowledge that is
  directly linked to supporting evidence.

18
Medical Image Annotator MIA

• Main challenge in medical image retrieval is that
  it heavily depends on experts knowledge of data
  structures and annotation is poor. So the
  objective of MIA is knowledge capture.
• MIA is designed for medical image annotation and
  its users are domain experts who require a
  consistent vocabulary for annotation tasks,
  knowledge sharing and machine automation.
• User community consists of Radiologists,
  Neurosurgeons (specifically, NNI doctors).
  Medical students, junior doctors, image
  processing researchers.
• MIA is a designed to both facilitate the
  building of appropriate ontology by domain
  experts and effective maintenance and evolution
  of the ontology, given new use cases
  /images.

MIA User Interface Our contribution the use of
WEB 2.0 technology to support knowledge capture,
and the approach to community engagement in the
development of the ontology more concretely in
the maintenance and evolution
19
MIA Platform Architecture
Easy to extend, any OWL file can be loaded
Ontologies can be edited online add node
rename node delete node
Ajax to update ontologies on server side to
provide dynamic content on a web page so no
page-refresh, no re-loading
Image
.owlfile
OntologyEditor
Database
AJAX
Owl Parser
Java script (DHTML)
Ontology Image Management Console
Tree Constructor
OntologyViewer
Client-side browser
Server-side processors
OWL files can be loaded dynamically OWL ?
relational database ? OWL

• Users can keep their own version of ontology
• Consolidated ontology will be generated based on
  community inputs.

20
Knowledge Capture in Action
21
Knowledge Capture in Action
22
Knowledge Capture in Action
23
Medical Image Annotator MIA

• Advantages
• Fast and easy
• Domain experts lead the process
• Always rooted in reality or a medical use case
• Maintenance and evolution of the controlled
  vocabulary is assured.
• Excellent training for new doctors / radiologists
• Facilitates Data Mining of Radiology reports

24
Ontology Evolution

• Different trainee and clinical doctors building
  ontologies with extensions on
• different sub trees
• Consolidated ontology is currently manually
  curated
• Goal is automatically align merge ontologies

25
Query with the Head Injury Ontology

• Simple ontology-term assisted query
• Search for images based merely on simple
  combination of ontology terms (and / or)
• Form based interface linked to SQL Queires
• Ontology reasoning (A-box)
• Content navigation over R-reports using defined
  object properties (Knowlegtor)
• Use of subsumption and object properties

26
Head Injury Ontology
27
Find patient records for Fracture
28
Discussion and Conclusions

• Medical images should be better annotated in
  order to facilitate information retrieval
• Collective knowledge is real FAQ-o-Sphere
• Controlled vocabularies (CVs) and/or ontologies
  are being developed by communities
• Simple tagging combined with knowledge
  elicitation methods supports ontology development
• Collective knowledge capture requires dedicated
  infrastructure that supports specific tasks
• Querability can be improved through the use of
  explicit tags and CVs/ontologies

29
Challenges for the Community

• How to get knowledge from all those intelligent
  people on the Internet
• How to give everyone the benefit of everyone
  elses experience
• How to leverage and contribute to the ecosystem
  that has created todays web.

Social Semantic Web
Social Web
Life Science
30
Acknowledgments

• Bonarges Aleman-Meza Social Web
• Tom Gruber - Semantic-Social Web
• MIA Developers - Zhang Zhuo and Menaka Rajapakse
• Suisheng Tang M.D. and Project PI, - Coordinator
  of domain experts and builder of baseline
  ontology
• Tchoyoson Lim Radiologist NNI (National
  Neuroscience Institute, Singapore)