Saliency-guided Enhancement for Volume Visualization

1
Saliency-guided Enhancement for Volume
Visualization

• Youngmin Kim and Amitabh Varshney
• Department of Computer Science
• University of Maryland at College Park

2
Motivation

• The volume datasets have grown in complexity
• Visible Human Project
• 13GB 60GB
• National Library of Medicine (NIH)
• Richtmyer-Meshkov Instability Simulation
• 2 TB ( 7.5GB 273 time steps)
• Lawrence Livermore National Laboratory
• Human visual capabilities remain fixed
• The need to draw visual attention to appropriate
  regions in their visualization

3
Motivation

• We can draw viewer attention in several ways
• Obtrusive methods like arrows or flashing pixels
• Distracts the viewer from exploring other regions
• Principles of visual perception used by artists
  and illustrators
• Gently guide to regions that they wished to
  emphasize

4
Contributions

• A new saliency-based enhancement operator
• Guides visual attention in volume visualization
  without sacrificing local context
• Considers the influence of each voxel at multiple
  scales
• Augments the existing visualization pipeline
• Enhances regional visual saliency
• Validation by eye-tracking-based user study
• Our method elicits greater visual attention

5
Related Work - Saliency

• Computation and Evaluation
• Computational models for image Itti et al. PAMI
  98 and mesh Lee et al. SIGGRAPH 05
• Evaluation by predicting eye movements Parkhurst
  et al. 02, Privitera and Stark PAMI 00

Mesh Saliency

• Use of eye movements
• Volume composition Lu et al. EuroVis 06
• Abstractions of photographs DeCarlo and Santella
  SIGGRAPH 02, NPAR 04
• Use of Saliency
• Progressive visualization Machiraju et al., 01
• Importance-based enhancement Rheingans and Ebert
  TVCG 01
• Interior and exterior visualization Viola et al.
  TVCG 05
• Generalizing focuscontext Hauser Dagstuhl 03
• Saliency has not been used for guiding visual
  attention

6
Related Work Transfer Functions

• Transfer Functions map the physical appearance to
  the local geometric attributes such as
• Gradient magnitude Levoy CGA 88
• First and second derivatives Kindlmann and
  Durkin Volume Rendering 98
• Multi-dimensional transfer functions Kindlmann
  et al. Vis 03, Kniss et al. TVCG 02, Kniss et
  al. Vis 03, Machiraju et al. 01
• Have played a crucial role in informative
  Visualization
• Difficult to emphasize (or deemphasize) regions
  specified exclusively by locations in a volume

7
Overview

• Saliency Field
• Enhancement Operators
• Emphasis Field
• Saliency Enhancement
• Saliency-enhanced Volume Rendering
• Validation by eye-tracking based user study

8
Basic idea from Saliency Computation
C Mean curvature

• Saliency map is
• Mesh saliency based on curvature values
• Image saliency based on intensity and color
• In general, saliency may be defined on a given
  scalar field

S (v) G(C, v, s) G(C, v, 2s)
9
Emphasis Field Computation
Given a saliency field, can we design some scalar
field that will generate it?

• Mesh Saliency S (v) G(C, v, s) G(C, v, 2s)
• We introduce the concept of an Emphasis Field E
  to define a Saliency Field S in a volume
• S (v) G(E, v, s) G(E, v, 2s)

10
Emphasis Field Computation

• Expressible as simultaneous linear equations
• Saliency Enhancement Operator (C-1)
• CE S , which implies E C-1S
• Given a saliency field S , the enhancement
  operator C-1 will generate the emphasis field E

where cij is the difference between two
Gaussian weights at scale s and at scale 2s for a
voxel vj from the center voxel vi
11
Emphasis Field Computation

• We like to use enhancement operators at multiple
  scales si
• Let E i be the emphasis field at scale si
• Compute this by applying the enhancement operator
  Ci-1 on the saliency field S
• Final emphasis field is computed as the summation
  of E i

12
Emphasis Field in Practice

• A system of simultaneous linear equations in n
  variables
• Generally, can handle arbitrary saliency regions
  and values
• Computationally expensive O(kn2) or O(n3)
• Alleviate this by solving a 1D system of equations

• Given a saliency field
• Solve 1D system of equations at multiple scales
  and sum them up
• Approximate results using piecewise polynomial
  radial functions Wendland 1995

• Interpret results to be along the radial
  dimension
• Assume spherical regions of interest (ROI)

13
Visualization Enhancement

• Emphasis Fields can alter visualization
  parameters in several ways
• Various rendering stylizations and effects
  possible
• We outline a couple of possibilities
• Brightness
• Widely used to elicit visual attention by artists
  
• Modulate the Value parameter in the HSV model as
  follows
• Vnew(v) V(v)(1E (v)), where ?- E (v) ?
• Used 0.4 ? 0.6 and 0.15 ?- 0.35
• Saturation
• Can modulate Saturation instead of Value if the
  latter is not effective (for instance, in regions
  already very bright)

14
Gaussian-based vs. Saliency-guided Enhancement

• Previous Gaussian-based Enhancement of a Volume
• Volume Illustration Rheingans and Ebert TVCG 01
• Importance-based regional enhancement
• We use a Gaussian fall-off from the boundary of
  ROI

15
Visualization Enhancement - Brightness
Traditional Volume Rendering
Gaussian-based Enhancement
Saliency-guided Enhancement
Traditional Volume Rendering
Gaussian-based Enhancement
Saliency-guided Enhancement
16
Visualization Enhancement - Saturation

• Increasing brightness diminishes the appearance
  of blood vessels at the center of the Sheep Heart
  model

Traditional Volume Rendering
Saliency-guided Enhancement
17
User Study

• Validated results by an eye-tracking-based user
  study
• Hypotheses The eye fixations increase over the
  region of interest (ROI) in a volume by the
  saliency-guided enhancement compared to
• the traditional volume visualization (Hypothesis
  H1)
• the Gaussian-based enhancement (Hypothesis H2)

18
User Study Experimental Design

• Eye-tracker and General Settings
• ISCAN ETL-500
• Records eye movements at 60Hz
• 17-inch LCD monitor
• With a resolution of 1280x1024
• Placed at a distance of 50cm (19.7) from the
  subjects
• Eye-tracker Calibration
• Desired accuracy of 30 pixels
• Two-step calibration process
• Standard calibration with 5 points
• Look and click on 13 points
• Triangulation and interpolation
• with 4 corner points
• Accuracy test on 16 random points

19
User Study Experimental Design

• Extracting fixations from raw points
• Raw points all points from the eye-tracker
• Saccade Removal
• Velocity gt 15/sec
• Fixation combining
• Filter out the points which stay less than 100ms
  within 15 pixels
• Average eye locations within 15 pixels and 100ms

20
User Study Experimental Design

• Image Ordering
• 10 users (who passed the accuracy tests)
• Total of 20 images 4 models (1 original 2
  regions 2 different enhancement methods
  (Gaussian, Saliency))
• Each user saw 12 images out of these 20 images
• 4 models (1 original 2 altered))
• Enhanced different regions with different methods
• Placed similar images far apart to alleviate
  differential carryover effects
• Randomized the order of regions and the order of
  enhancement types (Gaussian and saliency-based)
  to counterbalance overall effects
• Duration
• 12 trials (images), each of which takes 5 seconds

21
User Study Result I
Traditional Volume Rendering
Traditional Volume Rendering With Fixation Points
Saliency Field
Gaussian-based Enhancement
Gaussian-based Enhancement With Fixation Points
Saliency-guided Enhancement With Fixation Points
Saliency-guided Enhancement
22
User Study Result II
Traditional Volume Rendering
Traditional Volume Rendering With Fixation Points
Saliency Field
Gaussian-based Enhancement
Gaussian-based Enhancement With Fixation Points
Saliency-guided Enhancement With Fixation Points
Saliency-guided Enhancement
23
Data Analysis I

• The percentage of fixations on the ROI for the
  original, Gaussian-enhanced, and
  Saliency-enhanced visualizations

24
Data Analysis II

• A two-way ANOVA on the percentage of fixations
  for two conditions, regions and enhancement
  methods for each volume
• For regions, no statistically significant results
  as expected
• F(1,34) 0.2827 3.3336, p gt 0.05
• For enhancement methods, statistically
  significant results
• F(2,34) 7.2668 31.479, p 0.01

25
Data Analysis III

• Carried out a pairwise t-test on the percentage
  of fixations before and after we applied
  enhancement techniques for each model
• Found a statistically significant difference in
  the percentage of fixations with saliency-guided
  enhancement for all the models

Hypothesis H1 More fixations than the traditional
Hypothesis H2 More fixations than the Gaussian
26
Conclusions

• Introduced the concept of the Emphasis Field for
  selective visual emphasis (or de-emphasis)
• Developed the computational framework to generate
  the Emphasis Field from a given Saliency Field
• Illustrated the use of the Emphasis Field in
  Visualization
• Validated its ability to successfully guide
  visual attention to desired regions
• Saliency-guided Enhancement provides a powerful
  tool to help scientists, engineers, and medical
  researchers explore large visual datasets

27
Future Work

• Measure comprehensibility of the volume rendered
  images
• Explore other appearance attributes such as
  opacity and texture detail
• Generalize to handle time-varying datasets with
  multiple superposed scalar and vector fields
• Identify the relative importance of various scales

28
Acknowledgments

• Datasets Stefan Roettger (University of
  Erlangen) and Dirk Bartz (University of
  Tuebingen)
• Discussions David Jacobs, François Guimbretière,
  Derek Juba, and Robert Patro (University of
  Maryland)
• Eye-tracker François Guimbretière
• The Anonymous Referees
• Supported by NSF grants CCF 05-41120, CCF
  04-29753, CNS 04-03313, and IIS 04-14699

29
Questions ??
www.cs.umd.edu/gvil www.cs.umd.edu/gvil/projects/
sevv.shtml Supplemental material in the DVD-ROM
LabProjectImages