Multitemporal SAR image segmentation and classification based on a semanticbased stratified SAR and - PowerPoint PPT Presentation

1 / 33
About This Presentation
Title:

Multitemporal SAR image segmentation and classification based on a semanticbased stratified SAR and

Description:

Multi-temporal SAR image segmentation and classification based on a semantic ... Optical and SAR data feature complementary surface type investigation properties. ... – PowerPoint PPT presentation

Number of Views:799
Avg rating:3.0/5.0
Slides: 34
Provided by: franois1
Category:

less

Transcript and Presenter's Notes

Title: Multitemporal SAR image segmentation and classification based on a semanticbased stratified SAR and


1
Multi-temporal SAR image segmentation and
classification based on a semantic-based
stratified SAR and optical data fusion approach
  • François Aspert, Ecole Politechnique Federale de
    Lausanne (EPFL) -SPI
  • Francesco Holecz, Sarmap
  • Andrea Baraldi, JRC
  • Stefano Natali, MEEO
  • Jean-Philippe Thiran, EPFL-SPI

2
Content
  • Technological scenario in SAR remote sensing
    imagery
  • Image segmentation problem definition in
    biological vision
  • Goal of this work
  • Methodology
  • SAR and optical data pre-processing requirements
  • SAR data edge preserving smoothing Edge
    detection Standard region growing segmentation
    employing an edge map (Buhmans).
  • Preliminary spectral-based classification as an
    auxiliary source of semantic information.
  • Multi-source semantic-based stratified
    segmentation (applied to the standard Buhmans
    segmentation algorithm for comparison purposes).
  • Conclusion and future works

3
Technological scenario in SAR remote sensing
imagery
  • Currently, 8 spaceborne SAR systems are
    operational. By the end of 2008, 3 additional SAR
    sensors are planned to be launched, thereby
    allowing to obtain an almost continuous
    monitoring of the Earth coverage at large scale.
  • In order to transform the huge amounts of
    multi-temporal multi-source SAR and Optical data
    into information, automated (either prior
    knowledge-based or unsupervised) data
    pre-processing and image understanding
    (classification) techniques are mandatory, as
    ground truth data and ancillary information are
    typically tedious, expensive, and either
    difficult or impossible to gather.

4
Biological vision
  • Biological vision consists of a set of ill-posed
    problems, such as (Victor, 1994)
  • Shape-from-shading finding boundaries between
    regions of uniform but different
    intensity/spectral values.
  • Shape-from-texture finding boundaries between
    regions of uniform but different textures.
  • Structure-from-motion.
  • Figure-from-ground.
  • The visual system necessarily makes inferences
    fom these partial information, and the discovery
    of how these inferences work, is what the study
    of bological vision is all about.

5
Image primal sketch in computer vision
  • Image segmentation (simplification) Preliminary
    image mapping Image primal sketch (Marr,
    Vision, 1980).
  • Provide an image partition into asemantic areas
    (regions, segments, objects, e.g., segment1,
    segment2, etc.) of uniform either i)
    intensity/spectral values (shape-form-shading),
    or ii) texture (shape-from-texture).
  • The intrinsic vagueness of the abovementioned
    term  uniform  reflects the inherent
    ill-posedness of the image segmentation problem.
  • The dual problem of an ill-posed image
    segmentation task is the ill-posed edge
    detection, either i) intensity/spectral, or ii)
    textural.

6
Image segmentation in computer vision
  • In practice, i.e., in real-world commercial
    software toolboxes, image segmentation algorithms
    provide multi-scale solutions in the hope that
    the target phenomenon will appear correctly
    segmented at some scale.
  • Unfortunately, this operational approach is
    contrary to the principle of the visual
    perception of texture (Julesz, 1983)   The
    aperture of attention changes in spatial scale
    according to the size of the feature being
    sought.  ? Each picture primitive component is
    represented only once in its true form to be
    combined into complex objects as spatial
    arrangements of basic component parts
    (recognition-by-components).

7
Goal of this work
  • To propose an original set of algorithms aiming
    at the automated multi-temporal SAR and optical
    image data fusion, simplification (information
    aggregation, e.g., asemantic segmentation, if
    any), and classification.

8
Comparison of a traditional SAR Optical
segmentation data fusion algorithm vs. a
stratified segmentation approach
Conceptual inconsistency???!!! Optical
data-driven information should be employed only
once at a symbolic (semantic) level in an early
stage of the SAR segmentation workflow. Goal of
this experiment comparison purposes!!!!
Preliminary classification means preliminary to
(morphological, tetxural, geometric, etc.)
feature extraction!!!!!!
SAR data pre-processing
- Focusing - Geometric Calibration -
Radiometric Calibration - Radiometric
Normalization
Digital Elevation Model
Radiometrically calibrated optical data
Multi-temporal Anisotropic Diffusion Iterative
edge preserving smoothing of fused data at the
asemantic pixel level
Data fusion at the asemantic pixel level
(stacking) Edge preserving smoothing Edge
detection
Brightness image generator
Edge Detection
Standard region growing segmentation employing
an edge map
Seg.
Classification map
Object-based classification
Object-based classif.
9
SAR/optical data pre-processing requirements
  • Due to the increasing number of SAR and optical
    sensors, it is conditio sine qua non that
    automated pre-processing supports a pletora of
    different sensor imagery acquired in different
    geometries, polarization, etc. As a consequence,
    temporal, spatial, (and spectral) project
    requirements can be optimized.
  • To allow automated data understanding, the input
    dataset must be well behaved, i.e., SAR and
    optical images must be i) radiometrically
    calibrated and ii) geometrically corrected (not
    the other way around!!).

10
ENVISAT ASAR Pre-processing
ENVISAT ASAR image mosaic of Senegal consisting
of about 180 multi-temporal images geometrically
and radiometrically pre-processed in a fully
automated way.
11
Intensity/Color-based Multi-temporal Anisotropic
Non-Linear Diffusion (MANLD) for non-parametric
SAR speckle removal
Perona and Malik, 1990 Acton and Landis, 1997
  • No texture model is adopted, i.e., no texture
    segmentation ispursued, but intensity/color-based
    region growing (diffusion).
  • Use the heat diffusion equation for iterative
    edge-preserving smoothing employing no speckle
    (multiplicative noise) model.
  • The diffusion coefficient c 1 in the interior
    of each region (where gradient ?I 0), whereas c
    -gt 0 at each edge point.
  • k image-wide data-driven free parameter (edges
    with gradient lt K are not smoothed).

12
Multi-temporal Anisotropic Non-Linear Diffusion
(MANLD)
Original ENVISAT ASAR HH image
MANLD smoothed 4-temporal ENVISAT ASAR HH image
sequence
13
Multi-temporal Anisotropic Non-Linear Diffusion
(MANLD)
MANLD smoothed 4-temporal ENVISAT ASAR HH image
sequence stacked with one Optical image
(pixel-based data fusion)
Original ENVISAT ASAR HH image
14
MANLD for edge-preserving smoothing
  • Advantages
  • Only one free parameter k which is image-wide
    data-driven (Acton and Landis, 1997).
  •  Edge-preserving  smoothing is achieved.
  • Advantage using multi-sensor time-series (SAR and
    Optical) vs. single-date image diffusion.
  • Limitations
  • Inability to reject outliers (sensitive to noisy
    data).
  • Creation of artifical edges (called staircase
    artefacts).

15
Edge detection in cascade to edge-preserving
smoothing
  • First-stage edge-preserving smoothing by MANLD
    allows in cascade
  •  Effective  edge detection (high sensitivity).
  •  Good  edge localization.
  • Clear (1!) response per edge.
  • Existing edge detection techniques
  • Canny, Prewitt, Sobel, Marr-Hildreth, etc.
  • Canny edge detector is optimal with respect to
    the abovementioned criteria.

16
Edge Detection
F. Aspert et al., ENVISAT Conference, March 2007
  • Use Canny to get optimal edge detector
  • Smoothing
  • Edge magnitude
  • Edge direction
  • Non-maximum suppression
  • Hysteresis thresholding
  • Extends Canny edge detector
  • Use First Fundamental Form multi-temporal
    gradient definition
  • Find multi-temporal gradient magnitude
  • Find multi-temporal gradient direction

17
Edge detection - Multi-temporal vs Single-date
edge detection
Single-date edge map, Input dataset 1 ENVISAT
ASAR image
Multi-temporal edge map, Input dataset 4
ENVISAT ASAR and 1 Optical image
First-stage edge preserving diffusion algorithm
Automatic diffusion coefficient estimation, 60
iterations. Second-stage Canny edge detection
Automatic threshold computation.
18
Standard region growing segmentation employing an
edge map
l. Hermes and J. Buhman, IEEE TGRS, Sept 2006
  • Buhmans region growing is a two-stage image
    segmentation approach
  • Minimize an objective function to find seed
    segments in the whole image sequence.
  • Use statistical properties to promote large
    regions bounded by edges belonging to an edge
    map.
  • User-defined parameter
  • Region merging stopping criterion maximum size
    of a region (set equal to 1500 pixels, based on
    heuristics!!!).

19
Standard region growing segmentation employing an
edge map
Multi-temporal segmentation featuring 200 output
regions. Maximum region size (stopping criterion
based on heuristics) 1500 pixels.
MANLD filtered image with automatic sensitivity
thresholding and 60 iterations.
20
Standard region growing segmentation employing an
edge map
  • Disadvantages
  • Stopping criterion based on heuristics.
  • Large amount of memory required.
  • Large processing time.
  • Advantages
  • Edge map improves final segmentation

21
Multi-source semantic-based stratified
segmentation approach
  • Rationale
  • Optical and SAR data feature complementary
    surface type investigation properties. For
    example, in vegetation land cover detection
  • Optical data is suitable for separating
    vegetation, ranging from low (e.g., rangeland) to
    large biomass (e.g., forests), from
    non-vegetation.
  • SAR data is suitable for detecting forest stands
    (based on image texture properties), but it is
    little sensitive to grassland and rangeland.
  • Preliminary spectral classification is adopted to
    pursue semantic-based stratification of any
    standard segmentation algorithm.
  • Segmentation is performed stratum specific,
    according to a well-known divide-and-conquer
    problem solving approach capable of making the
    inherently ill-posed image segmentation problem
     easier  to solve.

22
Fully automated preliminary spectral
classification - Optical single-date imagery
A. Baraldi et al., IEEE TGARS, Sept 2006
WR DR PBHNDVI PBMNDVI PBLNDVII BBBHTIRF BBBHTIRNF
BBBLTIRF BBBLTIRNF SBBHTIRF SBBHTIRNF SBBLTIRF SB
BLTIRNF ABBHTIRF ABBHTIRNF
ABBLTIRF ABBLTIRNF DBBHTIRF DBBHTIRNF DBBLTIRF DBB
LTIRNF SHB WS TW TKCL TNCL SHCL SN ICSN TWSHSN SU
Unclassified SVHNIR SVLNIR AVHNIR
AVLNIR WVHNIR WVLNIR WE SHV SSRHNIR SSRLNIR ASR
HNIR ASRLNIR SHR AHR
Single-date optical Data
Preliminary spectral knowledge-based classifier
SOIL MAPPER
Classification map
SOIL MAPPER by MEEO provides an exhaustive and
mutually exclusive semantic image partition into
six spectral categories i) Vegetation, ii) Bare
soil/Built-up, iii) Water/Shadow, iv) Snow/Ice ,
v) Clouds, and vi) Outliers.
23
Fully automated preliminary spectral
classification - Optical single-date imagery
24
Fully automated bi-temporal post-classification
land cover change detection
Sarmap, 2007
Optical, Time 1
Optical, Time 3
Optical, Time 4
Optical, Time 2
25
Fully automated bi-temporal post-classification
land cover change detection
Data Landsat-5 TM
26
Traditional multi-source segmentation vs
Semantic-based stratified multi-source
segmentation - Model comparison
  • Dataset
  • One SPOT-4 image acquired on September 2006
  • Four ENVISAT ASAR HH/HV images acquired between
    May and August 2006
  • Stratified segmentation is performed exclusively
    on agriculture candidate areas. Objective
    identify the development of agricultural fields
    through the May-August 2006 time period.
  • Stratified parameter setting in the standard
    region growing algorithm (Buhmans)
  • Edge preserving diffusion algorithm Automatic
    sensitivity thresholding, 60 iterations.
  • Edge detection Automatic threshold computation.
  • Region merging The maximum size of a region has
    been set to 1500 pixels.

27
Multi-source semantic-based stratified
segmentation approach
Preliminary spectral classification map generated
from the single-date SPOT-4 image. Only the
agriculture candidate area (as an OR combination
of 3 spectral classes) is considered for the
stratified segmentation.
28
Multi-source semantic-based stratified
segmentation approach
Stratified segmentation (Buhmans) where data
fusion occurs at the semantic pixel level
Traditional segmentation (Buhmans) where data
fusion occurs at the asemantic pixel level
29
Multi-source semantic-based stratified
segmentation approach
  • Advantages
  • Consistent with any traditional segmentation
    approach. New combination of well-known image
    processing algorithms!!!!!!!
  • Less memory occupation.
  • Less processing time.
  • Stratum-specific parameter setting
    (divide-and-conquer approach).
  • Increased sensitivity (level of genuine but small
    image details detected) .
  • Increased accuracy.
  • Encourage data fusion at the pixel-specific
    semantic (symbolic) level rather than at the
    level of unlabeled asemantic digital numbers.
  • Future developments
  • Clever exploitation of the stratified
    segmentation approach, i.e., employ spectral
    driven semantic information only once in the
    early stages of the SAR data procesing workflow.
  • Preliminary rule-based classification of SAR
    imagery.

30
Comparison of a traditional SAR Optical
segmentation data fusion algorithm vs. a
stratified segmentation approach
SAR data pre-processing
Optical data pre-processing
- Focusing - Geometric Calibration -
Radiometric Calibration - Radiometric
Normalization
- Radiometric calibration into TOA
reflectance - Orthorectification -
Topographic correction
Digital Elevation Model
Digital Elevation Model
Preliminary spectral classifier SOIL MAPPER
Preliminary rule-based classifier
Primal sketch in the Marr sense
Primal sketch in the Marr sense
Data fusion at the semantic pixel level
Multi-temporal multi-source Data Fusion at the
semantic pixel level (fuzzy
rule-based) classification
Preliminary spectral-based classification map
Preliminary rule-based classification map
SAR Optical data classification map
31
Preliminary prior knowledge-based classification
- Interferometric SAR
Sarmap, 2008 A. Roth and T. Esch, DLR, 2007???
Interferometric Dual Polarization SAR Data
Unclassified Stable Smooth Surface - Water
Road Unstable Smooth Surface - Water Road Dense
Vegetation Average Vegetation Short Dry
Vegetation Unstable Short Vegetation -
Decrease Unstable Short Vegetation - Strong
decrease Unstable Short Vegetation -
Increase Unstable Short Vegetation - Strong
increase Average Vegetation Change -
Decrease Strong Vegetation Change -
Decrease Flooded Vegetation Average Vegetation
Change - Increase Strong Vegetation Change -
Increase Weak Vegetation Change - Vegetation
loss Weak Vegetation Change - Vegetation
growth Stable Short Vegetation Stable Bare
Soil Unstable Bare Soil Stable Very Dry Bare
Soil Stable Very Wet Bare Soil Strong Scatterer -
Urban Rock Drying Flooding
Knowledge based Classifier
25 Classes
32
Preliminary prior knowledge-based classification
- Interferometric SAR
Data Interferometric ALOS PALSAR HH/HV Dual
Polarization (June-Aug 2007)
Preliminary classification map
33
Preliminary spectral classification - Optical
single-date imagery
Data Landsat-5 TM (October 2000)
Preliminary spectral classification map
Write a Comment
User Comments (0)
About PowerShow.com