Speech and Gesture Corpus From Designing to Piloting - PowerPoint PPT Presentation

Loading...

PPT – Speech and Gesture Corpus From Designing to Piloting PowerPoint presentation | free to view - id: 69b354-ZTVkM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Speech and Gesture Corpus From Designing to Piloting

Description:

Speech and Gesture Corpus From Designing to Piloting Gheida Shahrour Supervised by Prof. Martin Russell Dr Neil Cooke Electronic, Electrical and Computer Engineering – PowerPoint PPT presentation

Number of Views:15
Avg rating:3.0/5.0
Slides: 32
Provided by: GheidaS8
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Speech and Gesture Corpus From Designing to Piloting


1
Speech and Gesture Corpus From Designing to
Piloting
  • Gheida Shahrour
  • Supervised by
  • Prof. Martin Russell
  • Dr Neil Cooke
  • Electronic, Electrical and Computer Engineering
  • University of Birmingham

2
Motivation
  • Our research focuses on modelling human behaviour
    from body motion.
  • No dataset which could serve our research focus.

3
Dataset Specification
  • We need data that
  • Contains the motion of peoples head, arms and
    hands
  • Captured from people come from different cultural
    backgrounds
  • Contains spontaneous speech
  • Captured using a marker-based tracking technique

4
Why Marker-based Tracking Technique?
  • Capturing peoples gestures is mainly based on
    computer vision techniques
  • skin colour- peoples skin light in images.
  • contour of people- objects may overlap/occluded
  • tracking from sequence of frames-may not be
    accurate
  • images are from 2D- accuracy issues.
  • To Avoid these problems
  • We will capture gestures using marker-based
    optical motion tracking
  • data obtained from 3D coordinate system
  • less occlusion recovered easily
  • tracking the object accurately- good calibration
  • tracking the light-reflective markers- accuracy.

5
Qualisys Track Manager (QTM)
  • The Balance and Posture Laboratory in the School
    of Psychology equipped with QTM system
    (http//www.qualisys.com)
  • 12 cameras with LED strobes which emits a beam of
    infrared light which is not visible to the naked
    eye.
  • QTM Software Analogue Interface for recording
    speech
  • passive markers- different sizes
  • calibration Kit axis L shape wand T shape.

http//www.qualisys.com
6
Camera Strobe
http//www.qualisys.com
7
How it works?
.
  • The spherical markers are coated with a material
    to amplify their brightness.
  • The strobes project light towards the markers and
    the markers reflect it back to the camera
  • Then the camera system measures a 2-dimensional
    position of the reflective target by combining
    the 2-D data from several cameras.
  • The camera uses the reflected data from multiple
    cameras to calculate the 3D position of the
    markers with high spatial resolution.

8
How it works?
9
The Process of Capturing data
.
  • Attach markers on the objects of interest- how?
  • Define the measurement area where subjects will
    stand
  • Test the area
  • Calibrate the area
  • Capture your data
  • Save your data

10
Reprocess Data Files
  • Reprocess the files you captured to construct the
    3D view-how?

11
Labeling Data
  • Label your data how?
  • Create a text file- Unique name
  • Unique colour
  • Upload the file
  • Drag drop
  • Play the motion data
  • Play it again
  • Fill the gap
  • Play it again
  • Save the file
  • Export the data

12
Experiments (1)_Methods Materials
  • 2 volunteers each wears 36 7mm flat-based half
    spherical markers on
  • - head(4)
  • - elbows(2)
  • - waist(4)
  • - golf gloves(26).
  • 12 cameras measurement volume is not specified
  • frame rate 200 frames per second
  • speech is not recorded.

13
  • Experiments (1)3D View

14
  • Experiments (1)Best Result

15
Experiment (2)_Motivation
  • To improve the quality of data.
  • 1. Quantity number of unidentified markers
    trajectories should be the same number of the
    markers used in the experiment.
  • 2. Quality No loss of markers, ghost markers
  • The technique the reduction both the number of
    markers the measurement volume

16
Typical 3D Data Cameras Position
17
Low Vs High 3D Tracker Parameters
  • Prediction error
  • Residual the remaining of the trajectory set to
    low
  • Filling gaps between frames

http//www.qualisys.com
18
Markers Trajectories Filling the Gap
19
Missing Data
20
How to Fill these Gaps?
21
Experiments (2)_Methods Materials
  • 3 volunteers each wears 28 7mm flat-based half
    spherical markers on
  • - head(4)
  • - elbows(2)
  • - shoulders(2)
  • - waist(4)
  • - golf gloves(16)

22
Experiments (2)_ Measurement Volume
23
Experiments (1)_Cameras Position
24
Experiments (2)_Cameras Position
25
Experiments (2)_Sessions
26
Experiments (2)_Result
27
Experiments (2)_Result
28
Conclusion
  • We will track motion of head, arms and hand
  • Leave 3 fingers out middle, ring and pink.
  • Occlusion of the markers on fingers is not only
    due to the cameras set up, but also due to the
    degree of freedom of the hands
  • Finding unidentified trajectories of markers is
    laborious and time consuming.
  • Tracking all fingers is very useful for many
    applications such as Sign Language but this is
    not our focus.

29
Data collection_ assignment
  • Each volunteer will wear not less than 12mm
    passive markers on head(4), elbows(2), waist(4),
    shoulder(3) and gloves(10)

30
Group Setup
  • Put yourselves into groups of 3.
  • The members of each group should be from the same
    first language, same gender same country of
    birth
  • Each member in British group (country of birth is
    Britain first language is English) will record
    2 sessions. Each session will last 15 minutes
    captured in 5 stages. Each stage lasts for 3
    minutes.
  • Each member in the cultural group (country of
    birth is not Britain first language is not
    English) will record 4 sessions. 2 sessions in
    English as a Second Language and 2 in their first
    language. Each session will last 15 minutes
    captured in 5 stages. Each stage lasts for 3
    minutes.

31
  • Any Question?
About PowerShow.com