Using Robots in Autism Therapy: A Survey of Ongoing Research

1
Using Robots in Autism Therapy A Survey of
Ongoing Research

• Marjorie Skubic
• Associate Professor
• Electrical and Computer Engineering Dept.
• Computer Science Dept. (joint apt.)

2
Outline

• Motivation How I got interested
• Autistic disorders
• A survey of the research
• Why robots might help
• The field of researchers
• Conclusions

3
How I got interested

• Research in Human-Robot Interaction
• Looking for a killer application
• Better How can we use robots to help people?
• Talks at the IEEE RO-MAN 2005 Workshop

4
Autistic Disorders

• 1 of 300 children diagnosed with autism with
  rates rising
• 1 of 800 children diagnosed with Down syndrome
• 1 of 450 children diagnosed with juvenile
  diabetes
• 1 of 333 children will develop cancer by age 20
• Diagnosis currently made through behavioral
  observation
• No blood test or genetic screening is available
  although there is evidence of a genetic link

5
Autistic Disorders Characteristics

• Inability to relate to other people
• Little use of eye contact with other people
• Difficulty understanding gestures and facial
  expressions
• Difficulties with verbal non-verbal
  communication
• Difficulty understanding others intentions,
  feelings, and mental states

6
Why Use Robots?

• Most children, including children with autism,
  are attracted to robots.
• This natural affinity is exploited, and the robot
  is used as an interactive toy.
• Robots may provide a less threatening environment
  than interacting with people.
• Robots can provide a repetitive and more
  predictable environment.
• This safe environment can gently push a child
  with autism towards human interaction.

7
The Connection to Imitation

• One theory Autism may be caused by early
  impairments in imitation and shared attention
  (Rogers Pennington, 1991) (Baron-Cohen, 1995)
• Imitation is a format of communication, a means
  to express interest and engage others in
  interaction (Nadel, 1999)
• Idea Use a doll-like robot to engage children
  with autism and teach basic imitative interaction
  skills
• From K. Dautenhahn, and A. Billard, Games
  Children with Autism Can Play With Robota, a
  Humanoid Robotic Doll, Proc. 1st Cambridge
  Workshop on Universal Access and Assistive
  Technology, 2002

8
Robota
A six-year old autistic boy playing with Robota.
He seemed curious about Robota's head movements
and so he touches the doll. From K. Dautenhahn,
and A. Billard, Games Children with Autism Can
Play With Robota, a Humanoid Robotic Doll, Proc.
1st Cambridge Workshop on Universal Access and
Assistive Technology, 2002
9
Imitation Using Robota
Robota allows the child to understand that the
dolls movement originates from his own movement
(sense of agency) and is limited to a restricted
category of movement (enhances intentional
action) From J. Nadel, Early Imitation and a
Sense of Agency, Proc. 4th Intl. Workshop on
Epigenetic Robots, 2004
10
An autistic child playing chasing games with
the mobile robot From K. Dautenhahn, and A.
Billard, Games Children with Autism Can Play With
Robota, a Humanoid Robotic Doll, Proc. 1st
Cambridge Workshop on Universal Access and
Assistive Technology, 2002
11
Joint Attention Using Robota
Robota is controlled via teleoperation by the
investigator.
From B. Robins, P. Dickerson, and K. Dautenhahn,
Robots as Embodied Beings Interactionally
Sensitive Body Movements In Interactions Among
Autistic Children and a Robot, Proc. RO-MAN 2005
12
Two autistic children Note Andys gaze at Jack.
The investigator encourages the children to show
each other how they can interact with the
robot. The robot will not move unless the
children show the same movement, i.e., they must
work together.
13
Andy and Jack touch each other to balance
themselves while each raising a leg.
14
Adam shows no interest in his classmates and
usually tries to avoid the rest of the children.
But Adam is interested in Robota.
Adam takes Robs hand to show him how to interact
with Robota.
15
Interacting with Keepon
Keepon is controlled via teleoperation.
From H. Kozima, C. Nakagawa, and Y. Yasuda,
Interactive Robots for Communication-Care A
Case Study in Autism Therapy, Proc. RO-MAN 2005
16
Views from Keepons camera eyes
17
Attentive action
Emotive action
Keepon's kinematic mechanism. Two gimbals are
connected by four wires the lower gimbal is
driven by two motors. Another motor rotates the
whole inner-structure yet another drives the
skull downward for bobbing.
18
Enabling Interaction
Joint attention Sharing the perceptual
information
Eye-contact Referring to each other's mental
states
Enables people to exchange intention and emotion
toward a target.
19
(No Transcript)
20
Using Robots for Autism Diagnosis
ESRA
Playtest
From B. Scassellati, Quantitative Metrics of
Social Response for Autism Diagnosis, Proc.
RO-MAN 2005
21
Autism Diagnosis Methods

• Reaction to the ESRA robot with and without the
  face configuration

Can generate facial expressions using 5 servo
motors
22
Autism Diagnosis Methods

• Measure listening preferences to speech sounds

23
Autism Diagnosis Methods

• Vocal prosody, i.e., how something is said

Features F24 vs. F1 Mean pitch energy vs. mean
pitch
Separation of two features used in a Bayesian
classifier distinguishes low energy categories
(neutral and soothing) from high energy
categories (approval, attention, and prohibition).
24
Autism Diagnosis Methods

• Position tracking relative to another person

25
Autism Diagnosis Methods

• Gaze direction and focus of attention

Red adolescents with autism Blue typical
adolescents
26
Linear discriminant analysis of autistic (au) and
typical (nc) gaze patterns. Linear filters F(x)
are trained to reproduce the gaze pattern G(x) of
each individual x and then applied to predict the
gaze patterns of any other individual.
For example, F(au)G(self) indicates a filter
trained on an individual with autism and tested
on that same individual while F(nc)G(au)
indicates a filter trained on a control
individual and tested on an individual with
autism. The mean performance of this data
(y-axis) is a function of the response percentile
of individual pairings. Significant differences
(all plt0.01 for a two-tailed t-test) are seen
between the following classes (1) F(nc)G(self),
(2) F(au)G(self), (3) F(nc) G(other nc), and
(4) the three other conditions.
27
University of Sherbrooke

• Project for engineering students
• Design a robotic toy for an autistic child
• Educational value
• Real world problem
• Students work together in a team
• Students must first investigate autistic disorders

28
University of Sherbrooke
Pushing Jumbo around the play area.
Rolling game with Roball.
From Michaud, F., Théberge-Turmel, C. (2002),
"Mobile robotic toys and autism", Socially
Intelligent Agents - Creating Relationships with
Computers and Robots, Kluwer, pp. 125-132.
29
University of Sherbrooke
Girl showing signs of interest toward Bobus.
Assembling the arms and tail of C-Pac.
From Michaud, F., Théberge-Turmel, C. (2002),
"Mobile robotic toys and autism", Socially
Intelligent Agents - Creating Relationships with
Computers and Robots, Kluwer, pp. 125-132.
30
(No Transcript)
31
The Field of Researchers

• Francois Michaud
• University of Sherbrooke, Canada
• Kerstin Dautenhahn Ben Robbins
• University of Hertfordshire, UK
• Aude Billard
• Swiss Federal Institute of Technology (EPFL)
• Jacqueline Nadel
• French National Centre of Scientific Research

32
The Field of Researchers

• Brian Scassellati and Bob Schultz
• Yale University
• Javier Movellan
• University of California San Diego
• Hideki Kozima
• National Institute of ICT, Japan
• Michio Okada
• ATR, Kyoto, Japan

33
Conclusions

• The use of robots for autism therapy and
  diagnosis is just beginning.
• There is anecdotal evidence that robot therapy
  can help children with autism
• How can we start here at MU with the new Thompson
  Family Center for Autism and Neurodevelopmental
  Disorders?

34
Maybe the Tiger Kitty
The iCat by Philips Research
35
Acknowledgements

• Thanks to Brian Scassellati, Francois Michaud,
  Ben Robins, and Hideki Kozima for helpful
  discussions.