Paper presentation: Spherical matching

1

• Context-aware Sensing
• with Hidden Markov Models
• Supervisor Prof Guang Zhong Yang
• Second Supervisor Prof Duncan Gillies
• Aziah Ali
• Department of Computing
• Imperial College London

2
Presentation Overview

• Laparoscopic MIS
• Research Aims
• Body Sensor Networks
• Context Aware Sensing
• Hidden Markov Models
• Multiple Eigenspaces
• Experimental Procedure
• Results
• Conclusion
• Future work

3
Laparoscopic MIS

• MIS (minimally invasive surgery) surgical
  method using smaller / no skin incision compared
  to open surgery
• Laparoscopic surgery - a form of MIS performed on
  abdominal cavity, by inserting laparoscopic
  instruments through ports into the cavity
• Surgeon views the site of operation displayed on
  a monitor via a camera inserted through one of
  the ports

4
Advantages of Laparoscopy

• Reduced surgical trauma due to the small size of
  incisions faster recovery, less pain
• Time taken for patients to resume normal
  activities and work is greatly reduced
• Reduced duration of post-operative bowel
  paralysis
• Can be performed as an outpatient procedure
  requiring only one hospital day less cost to
  the hospital

5
Challenges of Laparoscopy

• Tool ergonomics muscle stiffness, temporary
  damage to hands nerve, fatigue
• Visual perception surgeon views 3D operation
  site in 2D, causing harder depth and spatial
  judgment
• Tactile feedback and cognitive factor lack of
  tactile feedback, fulcrum effect
• Environmental factors ambient noise

6
Training and Assessment of Laparoscopic Skills (1)

• Training
• Box trainers cheap and simple to setup but only
  for simple procedures
• Virtual Reality simulators and training
  platforms procedures level of complexity can be
  varied, complications and emergencies can be
  simulated but these can be expensive and hardly
  portable
• Skills assessment
• Examinations does not test the practical aspect
  of performing laparoscopy
• Assessment by an expert watching the trainee
  performs a procedure very subjective and
  theres a risk of personal bias
• Objective assessment OSAT (Objective Structured
  Assessment of Technical Skill) has fixed
  structured criteria that needs to be assessed

7
Training and Assessment of Laparoscopic Skills (2)
ICSAD
Box trainer
LapSim
8
Limitations of current methods

• Subjective method
• Expensive
• Elaborate setup
• Demands a lot of human intervention
• Hardly portable

9
Research Aims

• Development of a context-aware system for
  efficient monitoring and classification of
  laparoscopic tasks to provide an objective method
  for training and assessing skills among surgeons
  using context recognition techniques that is
  cheap, simple to use, portable and requires less
  human intervention
• Major challenges
• Laparoscopic tasks classification
• Data segmentation detecting relevant segments
• Laparoscopic skills assessment

10
Body Sensor Networks

• Body Sensor Network a collection of wearable
  and implantable sensor nodes used to collect data
  from human subject for further analysis
• Major applications
• Monitoring of patients with chronic disease
  (cardiovascular diseases, high blood pressure,
  diabetes)
• Hospital patients monitoring
• Daily activity monitoring of the elderly
• Post-operative monitoring
• BSN nodes small wireless platform developed to
  provide a standard integrated hardware and
  software platform for body sensor networks

11
BSN Node
BSN node components (top left) top side of main
board, (top right) bottom side of main board,
(bottom left) battery board and (bottom right)
prototype board.
12
Technical Challenges in BSN

• Improved sensor design
• Biocompatibility
• Energy supply and demand
• System security and reliability
• Context aware sensing

13
Context-aware Sensing

• Definition of context Any information that can
  be used to characterize the situation of an
  entity. Entity is a person, place or object that
  is considered relevant to the interaction between
  a user and an application, including the user and
  the application themselves
• In BSN environment, main considerations for a
  context-aware system is the interpretation of the
  acquired body signals from wearable and
  implantable sensor and their association with the
  ambient environment (mainly the users
  activities, the physiological states and the
  environment in which the user in)
• Context recognition can be formulated as a
  classification problem and can be solved using
  classification algorithms

14
Hidden Markov Model

• A statistical model where the system being
  modelled is assumed to be a Markov Process with
  unknown parameters, and the challenge is to
  determine the hidden parameters from the
  observable parameters. The extracted model
  parameters can then be used to perform further
  analysis, for example for pattern recognition
  application.

Simple left-to-right HMM.
15
Advantages of Hidden Markov Models

• Robust to temporal changes
• Precise and sound probabilistic modeling
• Allow incorporation of prior knowledge to the
  model trained
• Modular
• Have been used successfully in many
  classification applications

16
Limitations of previous applications of HMM

• Some of the sensors proved to be a hindrance to
  the user
• Manual activity segmentation requires huge
  resources in isolated recognition
• Most experiments carried out in controlled
  settings, not natural environment
• Limited size of datasets for training

17
Multiple Eigenspaces (MES)

• An unsupervised method to extract and represent
  highly correlated low-dimensional structure from
  high-dimensional input data based on PCA
• PCA finds a single eigenspace (linear subspace of
  the feature space) that best represents the input
  data, while MES determines multiple of such
  eigenspaces
• Each eigenspace represents a highly correlated
  subset of the input data
• Advantages dimensionality of the multiple
  eigenspaces is much lower and each eigenspace can
  serve as a model to describe the subset of data
  they represent

18
Multiple Eigenspaces (2)

• Initialization of eigenspaces small subsets of
  data vectors (segments) are created from the
  data, and the eigenspace for each segment is
  calculated with the dimension set to zero
• Eigenspaces growing the initial sets are
  iteratively enlarged by adding segments not yet
  in the set. The new data vectors are then
  accepted or rejected based on calculated
  reconstruction error. If there is a change in the
  initial set, new eigenspaces and dimension is
  determined
• Eigenspaces selection - the result of
  eigenspaces growing usually consists of redundant
  and overlapping eigenspaces. In this stage, the
  eigenspace that best represent the data with
  minimal redundancy is selected

19
Multiple Eigenspaces (3)
Eigenspaces growing
Eigenspaces selection
20
Experimental Procedure

• With the sensing unit developed using BSN nodes,
  data was collected from users performing
  activities in three different settings and
  environments.
• recognizing a sequence of simple activities in a
  kitchen
• recognizing simple gestures using laparoscopic
  gripper in a box trainer
• recognizing basic daily activities

21
Experimental Procedure (2)

• Kitchen activities a) opening the door, b)
  turning the tap on or off, c) opening or closing
  the cupboard, d) making coffee, e) adding milk
  and f) drinking coffee ()
• Laparoscopic gripper movements - a) three times
  of simulation of port placement, b) three times
  of right rotation of tool, c) three times of left
  rotation of tool, d) rotation of the roticulator,
  and finally e) transferring objects from a point
  to another
• Daily activities - a) walk on a treadmill at the
  speed of 5km per hour for 3 minutes, b) sit down
  and read magazine for 3 minutes, c) lie down for
  three minutes, d) cough while lying down for
  three times, e) sit on a chair and read magazine
  for three minutes and finally e) cough while
  sitting down for three times

22
Sensing Units
Ear sensor- with sensors to measure heart rate,
oxygen saturation and temperature (Experiment 3)
Sensor glove 2 accelerometers and a bend
sensor (Experiment 1 and 2)
23
Sample data
Sample data for kitchen activities
Sample data for gripper movements
Sample data for daily activities
24
Data Analysis

• HMM training was performed on the manually
  segmented raw data collected by the sensing units
• One HMM was trained for each activity (class) to
  be recognized
• The same set of data using glove sensor is used
  as input to MES algorithm, and the output of
  segmented data is then used as input to HMM for
  classification

25
Results (HMM)
26
Results (HMM)
27
Results (MES)
Eigenspaces growing result for synthetic dataset
Eigenspaces growing result for kitchen dataset
Y-axis eigenspaces X-axis data vectors
(segments)
Eigenspaces growing result for laproscopic
gripper dataset
28
Results (MES)
29
Results (HMM with MES)
30
Discussion

• HMM classification for experiments with sensor
  glove give better result compared to experiment
  with ear sensor
• Accelerometer data
• Sensor position
• Generally, accuracy increased when more datasets
  used for training
• Segmentation results using MES is acceptably
  close to ground truth
• Classification results of HMM using MES-segmented
  data is comparable to classification results
  using HMM with ground truth markers

31
Conclusion

• Context recognition using HMM is successfully
  applied to recognize simple activities using BSN
  sensing units
• MES is capable of discovering useful segments in
  the raw data for glove sensor, allowing reliable
  unsupervised data segmentation
• Combination of MES and HMM will eliminate the
  need to manually segment the data before being
  input to HMM
• Preliminary results are encouraging but there are
  still much room for improvement

32
Future Work

• MDL implementation for eigenspaces selection
  phase
• Online data segmentation using MES to find
  relevant segments to send as input to HMM
  algorithm
• Investigation on types and numbers of features
  for classification
• Integration of MES and HMM algorithm as a
  classification system with automatic segmentation
  for laparoscopic training and skills assessment
• Systematic design of experimental setup in a
  natural setting to validate the use of proposed
  system to recognize laparoscopic tasks for
  training and skills assessment purpose

33
PhD Plan
34
Thank you.