Inertial and Magnetic Tracking of Limb Segment Orientation for Inserting Humans into Virtual Environ

1
Inertial and Magnetic Tracking of Limb
SegmentOrientation for Inserting Humans into
Virtual Environments Eric BachmannMarch 2001
2
INTRODUCTION
To be fully immersed in a virtual environment
subjects must interact with the virtual world
using their physical bodies. Being able to
accurately track the posture and position of the
human body is a basic requirement for an
immersive VE. Source based technologies are
not capable of accurately determining body
posture and position throughout a large working
volume without occlusion and interference
problems. Multi-axis magnetic/inertial sensors
can be used to determine orientation and body
posture without depending on an artificially
generated source.
3
INTRODUCTION - SOURCELESS TRACKING
Sourced tracking systems require maintenance of a
link between a tracked rigid body and one or
more fixed stations via a transmitted
source Interference, noise, or occlusion may
cause the link to be distorted or broken Distance
over which the link can be maintained is often
limited Update rate and accuracy may be limited
by the physical characteristics of the source
4
INERTIAL / MAGNETIC ORIENTATION TRACKING

• Inertial / Magnetic Orientation Tracking is based
  on the passive measurement of physical
  quantities directly related to motion and
  attitude. (Sourceless)
• Body posture can be ascertained by attaching
  inertial / magnetic sensors to human limbs
• Each segment is orientated independently
• Segments are positioned relative to each other by
  adding rotated limb-translation vectors

5
ORIENTATION USING GRAVITY AND MAGNETIC FIELD

• Orientations of individual segments can be
  determined by sensing
• the gravity vector
• the local magnetic field vector
• There is rotation which relates the vectors
  sensed in a body based reference frame to their
  known values in an earth-fixed reference frame
• The rotation can be expressed using a unit
  quaternion
• The rotation or orientation represented by the
  quaternion is the orientation of the limb segment

6
ORIENTATION USING ANGULAR RATE
Orientations of individual segments can be
determined by integrating angular rates The
angular rates p, q, and r may be used to find the
time derivative of an orientation quaternion This
derivative is given by where the quaternion q
relates the body-fixed reference frame to the
earth fixed frame Drift and bias in the angular
rate sensors will limit the period of time over
which q can be accurately estimated
7
MAGNETIC ANGULAR RATE GRAVITY (MARG) SENSORS

• 3-DOF MARG sensors have nine-axes
• Three-axis rate sensor
• Three-axis accelerometer
• Three-axis magnetometer
• Three-axis accelerometer senses the three
  components of the gravity vector in body
  coordinates
• sensed accelerations must be averaged over time
• Three-axis magnetometer senses three components
  of the magnetic field vector in body coordinates
• Rate sensor data gives velocity of rotation
• quickens response by providing short-term
  orientation estimates

8
A COMPLEMENTARY QUATERNION ATTITUDE FILTER-
Block Diagram
The quaternion-based complementary estimation
filter estimates attitude Combining of filter
inputs is treated as a parameter optimization
problem Error minimization is accomplished by
adjusting the derivative of the estimated
orientation quaternion, q-hat
9
A COMPLEMENTARY QUATERNION ATTITUDE FILTER-
measurement vector

• Accelerometer readings return an approximation to
  the local vertical
• represented by the pure unit quaternion
• Magnetometers provide an approximation to the
  local magnetic field vector
• represented by the pure unit quaternion
• Combining the vector parts of h and b produces a
  6 x 1 measurement vector

10
A COMPLEMENTARY QUATERNION ATTITUDE FILTER-
Computed Measurement Vector
Gravity in an earth-fixed reference frame is
always down It can be expressed as the down
unit quaternion The direction of the local
magnetic field vector can be expressed as a pure
unit quaternion These may be expressed in body
coordinates using the estimated q as a rotation
operator Combining the vector parts of h-hat
and b-hat produces a 6 x 1 computed measurement
vector
11
A COMPLEMENTARY QUATERNION ATTITUDE FILTER-
Error Vector

• The difference between the actual measurements
  and the computed measurements is an error vector
• The square of the modeling error is termed the
  criterion function

12
A COMPLEMENTARY QUATERNION ATTITUDE FILTER-
Gauss-Newton Iteration
Modeling error correction is done by Gauss-Newton
iteration Computed measurement vector is
linearized about the estimated orientation
quaternion using the first two terms of the
Taylor series expansion The criterion function
becomes Setting the error to zero and solving
for delta q produces
13
A COMPLEMENTARY QUATERNION ATTITUDE FILTER -
Derivation of the X matrix
The X matrix is defined as The elements are
the vector parts of partial derivatives of the
computed measurement vector with respect to each
component of the estimated orientation
quaternion By the product rule
14
A COMPLEMENTARY QUATERNION ATTITUDE FILTER -
Selection of Filter Gain

• When dealing with noise corrupted data it is
  necessary to scale the correction step
• where
• In the absence of noise alpha would be set to
  unity
• Noisy or inaccurate measurements require a scalar
  closer to zero
• Crossover frequency for the complementary filter
  in Hertzis given by

15
A COMPLEMENTARY QUATERNION ATTITUDE FILTER - Max
Filter Gain
The linearized estimated rate quaternion can be
written Discretization gives nth estimate
as which will converge for values of Maximum
value at which the filter is stable
16
A COMPLEMENTARY QUATERNION ATTITUDE FILTER -
Reduced Order Filter
For a delta-q of arbitrary length, X is not of
full rank and the regression matrix S is
singular. By the orthogonal quaternion theorem,
if delta-q is assumed to be orthogonal to q the
dimensionality of the matrix to be inverted can
be reduced to 3 x 3. The full correction step
becomes
17
A COMPLEMENTARY QUATERNION ATTITUDE FILTER -
Weighted Least Squares

• Magnetometer data is susceptible to interference
  from magnetic sources within the working
  environment.
• Effects of magnetic noise can be reduced through
  weighting of the error vector.

18
A COMPLEMENTARY QUATERNION ATTITUDE FILTER -
Reduced Drift Correction

• Calculating a drift correction requires two
  quaternion rotations, vector multiplication,
  matrix multiplication and inversion, numerous
  scalar multiplications and the performance of
  rate sensor update calculations
• Updates using only rate sensor data require only
  a quaternion multiplication and a single
  integration
• Depending on the rate sensor time constant, drift
  correction is not required on each filter cycle
• Significantly reduces the computational cost of
  the filter
• Effectively increases the number of filters which
  may be operated using a given amount of
  processing power

19
ANIMATION WITH WORLD REFERENCE FRAME ORIENTATION

• Orientation data expressed in earth fixed
  coordinates are applied directly to each
  individual segment of a simple human model.
• Each segment is orientated independently
• Segments are positioned through addition of
  rotated limb-translation vectors
• Orientations are expressed using quaternions
• Computational expense of numerous transformations
  is avoided
• No inverse kinematics, rotation matrices or joint
  angles

20
ANIMATION WITH WORLD REFERENCE FRAME ORIENTATION

Each limb segment has an associated orientation
(quaternion) and a position (vector) Suppose the
location of the root is given by If v is a
vertex associated with the torso, its translated
and rotated position is given by The location
and position of a vertex on the upper arm is
given by
21
HUMAN MODEL CALIBRATION
In general body segment frames are not actually
aligned with the associated sensor frame This
relation is expressed by an offset
quaternion. In the reference position, the body
segment and earth frames are aligned At this time
the sensor gives the offset The offset
quaternion can be calculated as
22
PROTOTYPE BODY TRACKING SYSTEM- Overview and
Goals

• Implementation Goals
• Orientation tracking of any three or more limb
  segments
• Sufficient dynamic response and update rate (100
  Hz) to capture fast body motions
• Calibration of sensors without specialized
  equipment
• Automatic compensation for misalignments between
  sensor and limb segment axes
• Adjustable human model to account for
  anthropometric variations

23
PROTOTYPE BODY TRACKING SYSTEM - System Diagram
24
PROTOTYPE BODY TRACKING SYSTEM - Sensor
Calibration

• Calibration of inertial sensors requires
  determination of
• Sensor Nulls
• Sensor Scale Factors
• Magnetic Reference Vector
• MARG Sensor calibration is achieved through
• a sequence of six 90 or 180 degree rotations
• six predefined positions
• and
• two 360-degree rotations
• Calibration requires approximately 60 seconds
• No specialized equipment needed

25
EXPERIMENTAL RESULTS- Overview

• Experimental results document the accuracy and
  robustness of the prototype system under various
  dynamic and static conditions
• Static Stability
• Static Convergence
• Dynamic Response and Accuracy
• Weighted Least Squares
• Posture Estimation
• Magnetic/Gravity Orientation Tracking
• Reduced Drift Correction

26
EXPERIMENTAL RESULTS- Weighted Least Squares
Test the stability of the orientation estimate
under exposure to strong magnetic fields using
various magnetometer weighting factors
Estimated Orientation During Exposure To
Magnetic Source p 1.0, k 4.0
Estimated Orientation During Exposure To
Magnetic Source p 0.25, k 4.0
27
EXPERIMENTAL RESULTS- Dynamic Response and
Accuracy

• Sensors cycled through various rotation angles
• Sensor moved and positioned using a precision
  rotary tilt table

Roll Angle vs. Time45 Degree Roll Excursions at
10 deg/sec.
28
EXPERIMENTAL RESULTS- Video Appendix

• Video recordings of the system in various
  operating regimens demonstrate accurate minimal
  lag posture estimation
• Sensor Calibration
• Body Model Calibration
• Posture Tracking
• Magnetic/Gravity Tracking
• Reduced Drift Correction
• InterSense InertiaCube Comparison

29
FUTURE WORK

• This Research lays the groundwork for a system
  capable of sourceless tracking of the entire
  human body.
• Suggested directions for work include
• Improved Smaller MARG Sensors
• More Realistic Human Model
• Integration of RF Positioning
• Wireless Communication
• Further Filter Research
• Prototype Body Suit
• Networked Virtual Environment Integration

30
SPONSORS

• U. S. Army Research Office (ARO) under Proposal
  No. 40410-MA
• U. S. Navy Modeling and Simulation Management
  Office (N6M)
• Naval Postgraduate School

31
SUMMARY

• Inertial / magnetic (MARG) body tracking avoids
  the shortcomings associated with current
  technologies.
• It is capable of providing wide area tracking of
  multiple users for networked synthetic
  environment applications
• Nine-axes sensor data is processed by a
  quaternion attitude filter
• Avatar posture is determined using only
  orientation data
• Posture is accurately tracked with minimal lag
• The technology is practical, robust and easy to
  use

32
Background - Quaternions

• Quaternions are an extension of complex numbers
• Define a four dimensional volume using three
  imaginary parts and one real
• Four dimensional vector with an associated
  quaternion product
• Represent an orientation using an arbitrary axis
  and a rotation about that axis
• The rotation of a vector, p, by a quaternion, q,
  is defined as
• Quaternions can be used to describe any
  orientation without singularities
• Trigonometric functions are not required

33
Background - MARG Sensor

• 10.1 x 5.5 x 2.5 cm
• Output range 0 - 5 vdc
• Power 12 vdc, 50 ma
• Crossbow Accelerometers
• Tokin Rate Sensors
• Honeywell Magnetometers
• Incorporates set/reset circuit for magnetometers
• Manually adjustable magnetometer null points

34
Background - Derivation of the X matrix

• From the product rule of differential calculus
• thus
• The general form for each column is

35
PROTOTYPE BODY TRACKING SYSTEM - Software

• Implements
• Quaternion attitude filter
• Sensor calibration algorithm
• Body model based on earth fixed reference frame
  orientation data
• Body model calibration Algorithm
• Object oriented design written in C

36
Background - Static Stability

• Stability of orientation measurement monitored
  for extended periods

One Hour Static Test Nine-axes Enabledk 1.0
One Hour Static Test No Rate Sensorsk 1.0
37
Background - Static Convergence

• Test convergence of the filter following the
  introduction of transient orientation errors

Criterion Error vs. TimeFollowing30 Degree
Transient Errork 1.0
Criterion Error vs. TimeFollowing30 Degree
Transient Errork 4.0
38
Background - References

• Bachmann, E., Duman, I., Usta U., McGhee, R., and
  Zyda, M., Orientation Tracking for Humans and
  Robots Using Inertial Sensors, International
  Symposium on Computation Intelligence in Robotics
  Automation (CIRA 99), Monterey, CA, November
  1999, pp, 187-194.
• McGhee, R., Bachmann, E., Yun, X., and Zyda, M.,
  Real-Time Tracking and Display of Human Limb
  Segment Motions Using Sourceless Sensors and a
  Quaternion-Based Filtering Algorithm Part I
  Theory, to be submitted to Presence
  Teleoperators and Virtual Environments, March
  2001.
• Bachmann, E., McGhee, R., Yun, X., and Zyda, M.,
  Real-Time Tracking and Display of Human Limb
  Segment Motions Using Sourceless Sensors and a
  Quaternion-Based Filtering Algorithm Part II
  Implementation, to be submitted to Presence
  Teleoperators and Virtual Environments, March
  2001.