Subdueing RHSEG: A Report on the Marriage of Graph Based Knowledge Discovery Subdue with Image Segme

1
National Aeronautics and Space Administration
Subdueing RHSEG A Report on the Marriage of
Graph Based Knowledge Discovery (Subdue) with
Image Segmentation Hierarchies (from RHSEG) for
Data Analysis, Mining and Knowledge Discovery
James C. TiltonComputational Information
Scienceand Technology OfficeNASA Goddard Space
Flight CenterGreenbelt, MD 20771, USA
Diane J. Cook, Huie-Rogers Professor Nikhil
Ketkar, Graduate StudentSchool of Elect. Eng.
Comp. ScienceWashington State UniversityPullman,
WA 99164, USA
2
Objective

• Improve our ability to extract and/or discover
  relevant information from image or image-like
  data.

Approach

• Utilize RHSEG to produce a hierarchical
  segmentation of the image or image-like data.
• Develop a graphical description of the
  hierarchical segmentation suitable for input into
  the Subdue knowledge discovery system.
• Utilize Subdue to extract and/or discover
  relevant information in the data.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
3
RHSEG Background HSEG

• The Hierarchical Segmentation (HSEG) algorithm
  produces a hierarchical set of segmentations of
  image or image-like data.

What is a Hierarchical Set of Segmentations?

• It is a set of image segmentations that
• consist of segmentations at different levels of
  detail, in which
• the coarser segmentations can be produced from
  merges of regions from the finer segmentations,
  and
• the region boundaries are maintained at the full
  image spatial resolution.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
4
Advantages of Image Segmentation and Segmentation
Hierarchies

• Image Analysis is transformed from pixel-based
  analysis into region-based or object-based
  analysis.
• A hierarchy of segmentations allows dynamic
  selection of the appropriate level of
  segmentation detail for each object of interest.
• A composite best segmentation may be adaptively
  extracted from the segmentation hierarchy.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
5
Hierarchical Segmentation (HSEG)
HSEG is a hybrid of Hierarchical
Step-Wise Optimization region growing together
with a form of spectral clustering controlled
by a spclust_wght parameter. J. M. Beaulieu
and M. Goldberg, Hierarchy in picture segmentatio
n A stepwise optimal approach,IEEE Transactions
on Pattern Analysis and Machine Intelligence,
vol. 11, no. 2, pp. 150-163, 1989.
National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
6
RHSEG Recursive Hierarchical Segmentation

• A recursive approximation of HSEG, called RHSEG,
  is much more computationally efficient
  (especially for spclust_wght gt 0.0).
• RHSEG recursively subdivides the image data and
  then recombines the results such that the number
  of regions handled at any point in the program is
  restrained.
• The recombination step requires special
  blending code to avoid processing window
  artifacts. This special blending code is the
  subject of a current patent application.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
7
Parallel RHSEG

• Recursive HSEG (RHSEG) facilitates a highly
  efficient parallel implementation a full
  Landsat TM scene (6500x6500 by 6 bands) can be
  processed in two to eight minutes with 256 2.1
  GHz CPUs (Thunderhead Beowulf Cluster).
• Aspects of the parallel implementation of RHSEG
  have been awarded a patent by the United States
  Patent and Trademark Office.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
8
Dissimilarity Criteria include

• Vector Norm
• Mean Squared Error
• Minimizing Entropy Change
• Spectral Information Divergence (SID),
• Spectral Angle Mapper (SAM),
• Normalized Vector Distance, and
• SAR Speckle Criteria.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
9
Graph-based Relational Learning

• Most data mining algorithms deal with linear
  attribute-value data
• Need to represent and learn relationships between
  attributes
• Finding patterns in graph(s)
• Discovery
• Clustering
• Supervised learning

10
Graph Representation

• Input is a labeled (vertices and edges) directed
  graph
• A substructure is a connected subgraph
• An instance of a substructure is an isomorphic
  subgraph of the input graph
• Input graph compressed by replacing instances
  with vertex representing substructure

Input Database
Substructure S1 (graph form)
Compressed Database
T1
shape
C1
S1
R1
R1
on
shape
T2
T3
T4
S3
S2
S4
11
Graph Representation
S2
S1
S1
S1
S1
S1
S2
S2
12
RHSEG/Subdue combination First Steps


A true color rendition of a 768x768 pixel section
of Ikonos data from the Patterson Park area of
Baltimore, MD.
Goal for RHSEG/Subdue combination A labeling in
terms of generalized labels such as residential,
park and harbor.
National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
13
Graph Coding Example
Graph vertices and edges (v region object
region class) v 1 3, v 2 1, v 3 1, v 4 1, v 5
1, v 6 1, v 7 1, v 8 2, v 9 4, V 10 4, v 11 1, v
12 5, v 13 5, v 14 1 (u 1st region 2nd
region relationship) u 1 8 link, u 1 9 link,
u 1 11 link, u 2 8 link, u 3 8 link, u 4 8 link,
u 5 8 link, u 5 10 link, u 6 8 link, u 7 8
link, u 8 9 link, u 8 10 link, u 8 11 link, u 8
12 link, u 8 13 link, u 8 14 link, u 9 11 link


Color coded map of a 16x16 pixel section of the
RHSEG segmentation of the northeast corner of
Patterson Park.
National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
14
Unsupervised Learning
Pattern captures patches of greenery adjacent to
residential buildings.
15
Unsupervised Learning
Pattern seems to capture characteristics of
residential blocks but does have many false
positives.
16
AISR Program Project Work Plan

• Determine the most appropriate parameter settings
  for RHSEG for creating segmentation
  hierarchies appropriate for analysis by Subdue.
• Determine the most appropriate manner in which to
  abstract the RHSEG hierarchical segmentations
  before conversion to a graph form for input into
  Subdue.
• Determine the most useful features to include in
  the input graph for Subdue, and the manner in
  which to present this feature information to
  Subdue. The preliminary work used only the ground
  cover type number as a feature. We plan to also
  consider such features as region size, mean
  values and texture.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
17
AISR Program Project Work Plan

• Determine the most effective graph representation
  of RHSEG abstractions. Different graph
  representations may highlight alternative
  relationships between regions in the data.
• Devise an automatic approach for transforming an
  appropriately abstracted segmentation hierarchy
  from RHSEG into a graph structure understandable
  by Subdue. Subdue currently requires a graph
  description from an ASCII format file. Processing
  efficiency may require a binary format interface.
• Determine the most appropriate parameter settings
  for Subdue to conduct a productive analysis of
  input graphs derived from the RHSEG segmentation
  hierarchies.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
18
AISR Program Project Work Plan

• Modify Subdue to handle weighted edges that
  reflect the relationship strength between
  regions, and providing this information to
  Subdue.
• Develop effective approaches for presenting the
  Subdue analysis results in the context of image
  analysis, and data mining and knowledge discovery
  for imagery.
• Demonstrate results for selected remote sensing
  applications.

National Aeronautics and Space Administration
RHSEG and Subdue AISR PI Workshop May 5-7,
2008
19
www.nasa.gov