Mod

1
Modélisation de linteraction avec objets
déformables en temps-réel pour des simulateurs
médicaux

• Diego dAulignac
• GRAVIR/INRIA Rhone-Alpes
• France

2
Medical Simulators

• Motivations
• danger to patients
• cost
• certification
• Objectives
• Geometric Models
• Physical Models
• deformation
• interaction

3
Problems

• Simulation MUST be real-time!
• deformation
• resolution
• Simulation MUST be realistic!
• model
• identification of parameters
• Simulation MUST be interactive!
• collision detection
• haptic interaction

4
Plan

• Deformation Models
• Mass-Spring vs. FEM
• Real-time Resolution Techniques
• Static
• Dynamic
• Echographic Simulator
• parameter identification
• Liver Model
• interactive deformation

5
Deformable Object

• Geometry
• Elements
• Springs TW90
• Tetrahedra FEM OH99
• Comparison
• Realism
• Speed

6
Geometrical Model

• 56 surface points
• 108 triangles
• 57 total points
• 120 tetrahedra
• 230 edges

7
Mass-Spring Model
Initial length
Deformed length
8
Finite Element Method (FEM)
Deformation tensor
Deformed configuration
Initial configuration
x
a
Greens strain
displacements
Small strain
Cauchy Strain
9
Strain-Stress
Deformation Energy
Lamé coefficients
force per unit area
10
Mass-Spring Model

• Springs are placed along the edges (230)
• Not very realistic modeling a volume with
  springs!
• The force of each spring relatively cheap to
  evaluate
• globally fast

11
Finite Element Method (FEM)

• 120 tetrahedra using Greens strain tensor
• Continuum is modeled with volumetric element.
• Dilatation may be controlled
• Approximately four times slower than mass-spring
  network

12
Deformable Models (conclusions)

• Mass-Spring
• One dimentional elements
• Unrealistic to model volume
• Tetrahedral FEM
• Good realism for 3D continuum
• Control of dilatation
• Approximately 4 times slower to evaluate forces

13
Contributions

• Quantitative and qualitative comparison of
  mass-springs and tetrahedral elements
• Interactive non-linear static resolution
• Formal analysis of the real-time stability of
  integration methods
• based on parameters
• Identification of the parameters of a model from
  experimental data
• Relevant medical applications

14
Plan

• Deformation Models
• Mass-Spring vs. FEM
• Real-time Resolution Techniques
• Static
• Dynamic
• Echographic Simulator
• parameter identification
• Liver Model
• interactive deformation

15
Real-time Resolution

• Static Resolution
• linear resolution Cotin97
• small displacements
• Our approach non-linear resolution
• large displacements
• Dynamic resolution
• explicit Picinbono01
• implicit BW98

16
Linear Static Resolution

• Linear case
• Pre-inversion (if enough space)
• No large strain
• No rotation
• No material non-linearity

Principle of virtual work internal and external
forces are balanced
17
Nonlinear Static Resolution

• Non-linear case
• Stiffness matrix changes with displacement
• geometric
• material

18
Newton Iteration

• Full Newton-Rapson method
• Reevaluation of Jacobian
• Faster convergence

• Modified Newton-Rapson method
• Constant Jacobian
• Slower Convergence

19
Dynamic Analysis
2nd order non-linear differential equation
Convert to 1st order system
20
Explicit Integration
Runge-Kutta method with s stages
s
Order of consistency (accuracy) vs. stages
precision
21
Explicit Integration Stability
linearizing
Im
Timestep is limited by the the physical
parameters!
Re
22
Implicit Integation
If you know your history, then you would know
where you are coming from. Bob Marley
Over-damped case
B-stable implicit euler
linearisation
Semi-implicit euler
Stable for linear case (A-stable) any
timestep any physical parameters
23
Resolution (conclusions)

• Static analysis
• non-linear resolution for large displacements
• Dynamic
• explicit
• strict stability criteria
• implicit
• no limit on timestep, but resolution of
  non-linear system

24
Contributions

• Quantitative and qualitative comparison of
  mass-springs and tetrahedral elements
• Interactive non-linear static resolution
• Formal analysis of the real-time stability of
  integration methods
• based on parameters
• Identification of the parameters of a model from
  experimental data
• Relevant medical applications

25
Plan

• Deformation Models
• Mass-Spring vs. FEM
• Real-time Resolution Techniques
• Static
• Dynamic
• Echographic Simulator
• parameter identification
• Liver Model
• interactive deformation

26
Thigh Echography
27
Echographic Simulator

• Data Acquisition
• Model of the thigh
• Mass-Spring
• Neural
• Interaction
• collision
• haptics
• Generation of echographic image

28
Data Acquisition
(at LIRMM, Montpellier)
64 sample points are marked on the thigh. For
each, the forces for some given penetrations are
measured
Two different probes (a) Indentor shaped probe
for punctual force-penetration data (b) Probe
with surface equal to that of a typical
echographic probe
1- The end effector advances in small steps
(2mm) in the direction normal to the surface of
the thigh. 2- The force depending on the
penetration distance is measured
29
Data Acquisition Experimental Results
dAulignac et al. MICCAI 99
Indentor probe
Surface probe

• The two probes do not offer the same resistance
• difference in surface area
• Different curves for different points
• different depth of soft tissue
• Highly non-linear behaviour

30
Echographic Simulator

• Data Acquisition
• Model of the thigh
• Mass-Spring
• Neural
• Interaction
• collision
• haptics
• Generation of echographic image

31
Dynamic Model of the thigh
Incompressibility of the tissue
Elasticity of the epidermis

• Why mass-spring model?
• computationally efficient
• interior NOT discretized into tetrahedra

32
Identification of the Parameters of aDynamic
Model
Optimization Algorithm
New parameters (elasticity, plasticity, collision
stiffness ...)
Error
-
Behaviour
Resolution
Model
Desired behaviour
Measurements
For each sample point, 10-12 deformation/force
values with each probe gt Total of 1200
measurements.
33
Parameter Estimation
(in collaboration with UC Berkeley) dAulignac
et al., IROS 99
Least-squares minimisation 1. find (a,b)
for each non-linear spring 2. find (a,b) for
each non-linear spring, and (a) for all linear
springs
gt Avoid local minima

• Error of the model with respect to the
  experimental data

gt Overall error less than 5
Error (N)
34
Dynamic Analysis

• Explicit integration
• Euler stability
• too small timesteps
• no real-time
• ...or large mass
• slow movement
• no gravity
• Implicit integration
• Semi-Implicit Euler
• constant Jacobian
• 100 steps per second
• h1/100 (i.e. real time)

35
Dynamic Resolution
100 Hz using semi-implicit integration
36
Neural Networks
Displacement of particles u

• Static Analysis
• Multi-layer perceptron is a general approximizer
• Network is trained directly on experimental data
• back-propagation

Forces acting on particles f
64 inputs and outputs
37
Neural Networks
Displacement (mm)
Force (N)
Neural Model
Experimental data
38
Mass-Spring vs. Neural Model

• Mass-spring
• topology chosen
• based on measurements
• dynamic resolution
• semi-implicit (100 Hz)
• Neural model
• no assuption on topology
• static resolution
• very fast
• no change of topology

39
Echographic Simulator

• Data Acquisition
• Model of the thigh
• Mass-Spring
• Neural
• Interaction
• collision
• haptics
• Generation of echographic image

40
Interaction

• Collision Detection
• Collision Response
• Force Feedback

41
Collision Detection

• Finds polygons in the OpenGL viewing frustrum
• Detects collision between simple rigid body and
  any other object quickly

42
Collision Response
Penalty forces Hunt and Crossley 1975

• Inter-penetration distance must be computed
• Generates large forces (bad for haptics)

43
Haptics

• Haptic devices require high update frequency
• typically around 1kHz
• .which the simulation normally cant meet
• 100 Hz (dynamic model)

44
Haptic Interaction

• Local approximation of the contact
• simple local model running in a separate thread
• fast collision detection
• fast force computation

Haptic loop (1kHz) collision detection and
response with local model
Balaniuk 99
Local model update
position
Simulation Loop (100Hz) deformation global
collision detection and response
45
Haptic Feedback
dAulignac et al. , ICRA, 2000
With local model
force
time
Without local model
46
Echographic Simulator

• Data Acquisition
• Model of the thigh
• Mass-Spring
• Neural
• Interaction
• collision
• haptics
• Generation of echographic image

47
Echographic Image Generation
Vieira01 (in collaboration with TIMC-IMAG,
France)

• 64 images aquired
• on each sample point
• Voxel Map
• 120 Mb
• Interpolation
• fill in the blanks
• Provide image
• any rotation
• any position

48
Echographic Image Deformation

• Problem
• structures deform differently
• vein
• bone, etc.
• segmentation
• Linear deformation
• Possible extension precalculated deformation
  maps Troccaz et al, 2000

49
A first Prototype
50
Echographic Simulator (conclusions)

• Data Acquisition
• Model of the thigh
• Mass-Spring
• Neural
• Interaction
• local model
• Generation of echographic image
• linear deformation

51
Contributions

• Quantitative and qualitative comparison of
  mass-springs and tetrahedral elements
• Interactive non-linear static resolution
• Formal analysis of the real-time stability of
  integration methods
• based on parameters
• Identification of the parameters of a model from
  experimental data
• Relevant medical applications

52
Plan

• Deformation Models
• Mass-Spring vs. FEM
• Real-time Resolution Techniques
• Static
• Dynamic
• Echographic Simulator
• parameter identification
• Liver Model
• interactive deformation

53
Keyhole Surgery
Surgery involves soft tissues
simulation
Need to model deformation in real-time!
54
Human Liver

• Interior composed of parenchyma
• Surounded by elastic skin or Glissons capsule
• Venous network
• Approximate weight 1.5 kg

55
Liver Model

• Geometry
• Physical Model
• Dynamic Analysis
• explicit integration stability
• Static Analysis
• non-linear resolution

56
Geometrical Model

• 187 Vertices
• 370 Triangles
• 299 Particles
• 1151 Tetrahedra
• 1634 Edges

GHS3D
57
Physical Model
Boux et al., ISER, 2000
Heterogenous
Non-linear
Stress
Strain
skin
Parenchyma
Weight distributed equaly on all particles (i.e.
approximately 5g each)
58
Explicit Integration
280 steps per second mass 5 grams
59
Stability Analysis
Im
Re
60
Simulation
Achitecture SGI Onyx2 Compexity 370 facets 1151
tetrahedra 3399 springs Frequency 150Hz

• Explicit not stable!
• ...large mass

61
Static Resolution
The large deformations of the organ during
operation require non-linear resolution
techniques.
62
Iterative Solution
Calculate forces on nodes Evaluate stiffness
matrix K? (analytically) Iteratively solve
linear system for displacements u Ku f by
successive over- relaxation (SOR) until residual
forces lt epsilon through Newton-Rapson iteration
63
Modified Newton-Raphson

• Accurate solution (many SOR iterations) does not
  allow faster solution
• Inexact Jacobian limits convergence speed
• Of special importance for strong nonlinearities

residual
iterations
64
Newton-Raphson

• Less iteration to converge then modified NR
• Exact Jacobian allows faster convergence
• Global time gain when solving linear system
  accurately

residual
iterations
65
Pseudo-Dynamic
Interactive resolution of the non-linear system.
66
Result
Pseudo-dynamic
1157 tetrahedra Iterative non-linear resolution
Rotational invarience (N.B. Real-time animation)
60 NR iterations/sec on SGI Octane 175Mhz
67
Liver Model (conclusions)

• Physical Model
• mass-springs
• Dynamic Analysis
• explicit integration unstable
• Static Analysis
• interactive non-linear resolution

68
Summary

• Physical Models
• Mass-Spring or FEM?
• Resolution
• Static
• linear or non-linear?
• Dynamic
• explicit or implicit?
• Medical Simulators
• The choice of numerical methods must be guided by
  the application!

69
Contributions

• Quantitative and qualitative comparison of
  mass-springs and tetrahedral elements
• Interactive non-linear static resolution
• Formal analysis of the real-time stability of
  integration methods
• based on parameters
• Identification of the parameters of a model from
  experimental data
• Relevant medical applications

70
Local Model
71
Explicit Integration
Dynamic equations solved by Eulers method
Linearizing by assuming constant matrices we can
calculate derivative analytically
The absolute value of (1z) must be smaller than 1
72
Backwards engineering
Geometrical description
Geometrical description
elasticity
elasticity
Physical Model
Physical Model
forces displacement
forces displacements
Results
Results