Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data

1
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
James C. Tilton Mail Code 606 NASA
GSFCGreenbelt, MD 20771 James.C.Tilton_at_nasa.gov
William T. Lawrence Natural Sciences Bowie State
University Bowie, MD 20715 wlawrence_at_bowiestate.ed
u
Computational Information Sciences and
Technology Office
2
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Proposal To develop tools and methods for
automated change detection from remotely sensed
imagery utilizing a previously developed approach
for creating segmentation hierarchies from
imagery data. Step-1 Proposal has been submitted
to the ROSES-2005 NRA, Land-Cover/Land-Use Change
Element
3
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
What is a Segmentation Hierarchy?

• A set of image segmentations that
• consist of segmentations at different levels of
  detail, in which
• the coarser segmentations can be produced from
  merges of regions from the finer segmentations,
  and
• the region boundaries are maintained at the full
  image spatial resolution.

4
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Advantages of a Segmentation Hierarchy

• Image Analysis is transformed from pixel-by-pixel
  analysis into object-by-object analysis, allowing
  the utilization of object shape, texture and
  context for a more robust and accurate analysis.
• A hierarchy of segmentations allows dynamic
  selection of the appropriate level of
  segmentation detail for each object of interest.

5
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Example Ikonos Data

• Collected May 17, 2000 over Baltimore, MD.
• Four meter spatial resolution.
• Four spectral bands blue, green, red and nir.
• 384x384 pixel sub-section.
• Twelve-level hierarchical segmentation.

6
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Original Image
7
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Region Mean Image with 30 Regions
8
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Region Mean Image with 18 Regions
9
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Region Mean Image with 11 Regions
10
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Region Mean Image with 8 Regions
11
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Region Mean Image with 6 Regions
12
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Region Mean Image with 4 Regions
13
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Original Image
14
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Twelve Level Hierarchical Boundaries
15
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
RHSEG and HSEGViewer

• Hierarchical Segmentations produced by RHSEG
• RHSEG is a hybrid of Hierarchical Step-Wise
  Optimization region growing with spectral
  clustering controlled by spclust_wght
  parameter.
• J. M. Beaulieu and M. Goldberg, Hierarchy in
  picture
• segmentation A stepwise optimal approach,
• IEEE Transactions on Pattern Analysis and Machine
  
• Intelligence, vol. 11, no. 2, pp. 150-163, 1989.

16
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
RHSEG and HSEGViewer

• Recursive implementation facilitates a highly
  efficient parallel implementation a full
  Landsat TM scene (6500x6500 by 6 bands) can be
  processed in under 10 minutes with 256 CPUs.
• The HSEGViewer program provides a convenient,
  user-friendly, tool for visualizing and
  interacting with the image segmentation
  hierarchies produced by the RHSEG program.

17
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
RHSEG and HSEGViewer

• HSEGViewer and demo version of RHSEG are
  available through http//tco.gsfc.nasa.gov/RHSEG/i
  ndex.html
• More information on RHSEG available at
  http//cisto.gsfc.nasa.gov/TILTON/index.html

18
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Monitoring Change First Steps

• Assembled a multi-season, multi-year test data
  set from
• MODIS Terra AM1 platform for initial tests
• Bands 1-7 (aggregated to 1KM)
• Twelve dates 31 JAN 2003, 19 APR 2003, 09 AUG
  2003, 21 OCT 2003, 28 OCT 2003, 18 NOV 2003, 01
  FEB 2004, 20 MAR 2004, 11 JUN 2004, 24 SEP 2004,
  29 NOV 2004, 28 FEB 2005.
• 1002x1002 pixels at 1km spatial resolution
  centered roughly over the Salton Sea.
• Southern California fires visible in 28 OCT 2003
  scene.

19
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
21 OCT 2003 Bands 7, 2 1
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
20
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
28 OCT 2003 Bands 7, 2 1
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
21
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
01 FEB 2004 Bands 7, 2 1
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
22
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
01 FEB 2004 Hierarchical Boundary Map 15 regions
23
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
01 FEB 2004 Hierarchical Boundary Map 9 regions
24
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
01 FEB 2004 Bands 7, 2 1
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
25
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Normalized Dissimilarity vs. Region Mean at
Finest Hierarchical Level.
26
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Sum of Masks from Minimum Mean Regions
Histogram
0 644777 1 2211 2 1041 3
872 4 1127 5 4703 6
20366 7 63794 8 109386 9 87193
10 57075 11 26905 12 29126
27
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Land vs. Water Mask (0-3 designated as land and
4-12 as water.)
28
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Cloud and Snow Detection/Masking
31 JAN 2003 Data Set (A2003031.1815) Brightest
Region
Region 22 is clearly a cloud at all hierarchical
levels (by inspection)
29
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Bands 7, 2 1 (water masked)
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
30
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Cloud Region (water masked)
31
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Cloud and Snow Detection/Masking
31 JAN 2003 Data Set (A2003031.1815) Selected
Region
Region 51//55 is mountain snow at hierarchical
levels 0 through 33 (by inspection and change
in normalized dissimilarity)
32
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Bands 7, 2 1 (water masked)
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
33
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Cloud and Snow Regions (water
masked)
34
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Cloud and Snow Detection/Masking
31 JAN 2003 Data Set (A2003031.1815) Selected
Region
Region 64 is mountain snow through hierarchical
level 4 (by inspection and change in normalized
dissimilarity)
35
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Bands 7, 2 1 (water masked)
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
36
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Cloud and Snow Regions (water
masked)
37
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Cloud and Snow Detection/Masking
31 JAN 2003 Data Set (A2003031.1815) Selected
Region
Region 61 is mountain snow through hierarchical
level 22 (by inspection and change in
normalized dissimilarity)
38
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Bands 7, 2 1 (water masked)
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
39
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Cloud and Snow Regions (water
masked)
40
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Bands 7, 2 1 (water, clouds snow
masked)
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
41
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
31 JAN 2003 Bands 7, 2 1 (water masked)
Band bandwidth
1 620- 670nm 2 841- 875nm 7 2105-2155nm
42
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Monitoring Change Next Steps

• Obtain Cloud Mask data product (MOD 35) for
  pertinent data set dates and compare with RHSEG
  results.
• If available, obtain Snow Cover data product (MOD
  10) and compare with RHSEG results.
• Obtain other pertinent MODIS data products (e.g.
  MOD 12 Land Cover/Land Cover Change, MOD 14
  Thermal Anomalies, Fires Biomass Burning, MOD
  13 Gridded Vegetation Indices, ) for analysis
  and comparison.

43
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Monitoring Change Next Steps

• Develop more flexible tools for analyzing RHSEG
  segmentation hierarchies, including improvements
  to HSEGViewer.
• Implement other dissimilarity criteria in RHSEG,
  such as the Spectral Angle Mapper criterion.
• Implement tools to evaluate various spatial
  features for use in analyzing the RHSEG
  segmentation hierarchies, such as convex_area,
  solidity, and extent, as well as texture and
  fractal measures.
• Implement tools to find and track corresponding
  regions across multi-temporal data sets.

44
Monitoring Change Through Hierarchical
Segmentation of Remotely Sensed Image Data
Monitoring Change Future Plans

• Automate process to flag areas with intra-data
  change
• Create a rule-based automated classification
  system to label regions
• Create a system to evaluate change as expected
  or unexpected
• Use a rules-based system to flag areas of change
  that are not expected
• Automated evaluation of change would facilitate
• (human) follow-up for change mediation/interventio
  n