A project to develop software to assist people with Autism to understand facial expressions

1
A project to develop software to assist people
with Autism to understand facial expressions
emotions
www.ascinclusion.eu
2
Introduction

• A project to develop software to assist people
  with Autism Spectrum Disorders to recognise and
  understand facial expressions and emotions
• Primarily for children aged 5 to 10
• Project by 7 organisations under a European
  Commission funded 3-year cooperative research
  project (FP7 programme)
• First version will be available in late 2014 (in
  English)

www.ascinclusion.eu
3
Autism Spectrum Conditions (ASC)

• Autism Spectrum Conditions (ASC) are
  neuro-developmental conditions, characterised by
  difficulties in 3 main areas communication,
  social interaction and understanding information.
• Common symptoms include restricted interests,
  repetitive behaviors, lack of eye contact.
• Current studies suggest 1 of the population
  might fit an ASC diagnosis (Baron-Cohen et al.,
  2009).
• Individuals with ASC lack the sense of social
  reciprocity and fail to develop and maintain age
  appropriate peer relationships.
• As a consequence, children with ASC are at high
  risk of social exclusion (Frith, 2003).
• Studies have shown that in ASC, the earlier
  intervention is provided, the more effective it
  is (Howlin Rutter, 1987 ).

www.ascinclusion.eu
4
Emotion recognition in ASC

• Children and adults with ASC experience
  significant difficulties recognising others, as
  well as their own, emotions and mental states
  (Baron-Cohen, 1995 Hobson, 1993).
• Including facial expressions, vocal intonation,
  gestures and body language, and their integration
  in context (Golan, Baron-Cohen Golan, 2008).

www.ascinclusion.eu
5
Emotional expression in ASC

• Individuals with ASC also experience difficulties
  in expressing their own emotions non-verbally,
  including the ability to
• Direct appropriate facial expressions to others
  (Kasari et al, 1993)
• Modulate their vocal intonation appropriately
  when expressing emotion (McCann Peppe, 2003)
• Using appropriate gestures and body language
  (Attwood, 1998).
• Integrating non-verbal communicative cues with
  speech (De Marchena Eigsti, 2010).

www.ascinclusion.eu
6
Emotional understanding and expression in ASC

• In summary, individuals with ASC have a limited
  ability to interpret other people's intentions,
  emotions and mental states, and to express their
  own.
• These skills are not learned spontaneously in
  ASC, and therefore need to be taught explicitly.
• Without filling these gaps, the risk of exclusion
  for individuals with ASC is significantly
  increased.

www.ascinclusion.eu
7
Systematic strengths in ASC

• Individuals with ASC are hyper-attentive to
  detail and prefer predictable, rule-based
  environments and systems.
• Within such environments, individuals with ASC
  show good and sometimes even superior systemising
  skills, compared to the general population
  (Baron-Cohen, 2003).
• If provided with a system of emotions, it is
  plausible that systemising skills could be
  harnessed to help individuals with ASC learn to
  recognise emotions (Golan Baron-Cohen, 2006).
• The computer provides individuals with ASC with
  such a predictable, rule-based environment,
  enabling them to capitalise on their systematic
  skills.
• Computerised socio-emotional training in ASC has
  been shown to change behavior, as well as brain
  functioning (Bölte et al., 2006)

www.ascinclusion.eu
8
Current needs and issues

• Studies attempting to teach emotion and mental
  state recognition to people with Autism have
  shown mixed results, reporting limited
  generalisation from taught curriculum to
  situations not included in the training program.
• Training of individuals with ASC requires
  facilitation by trained specialists, which is not
  always available, and may be quite expensive.
• In view of the growing prevalence of ASC, there
  is an urgent need to seek new innovative methods
  for supporting integration of children with ASC
  into society.
• Interactive Communication Technologies (ICT), may
  provide an easily available solution, enabling
  users everywhere to enjoy state-of-the-art
  professional support online.

www.ascinclusion.eu
9
Target Audience

• Children of age 5-10, with High-Functioning
  Autism Spectrum Conditions (ASC)
• Parents of children with High-Functioning ASC
• Therapists who specialise in ASC
• NGOs and governmental institutes that support the
  ASC population

www.ascinclusion.eu
10
Project main objectives

• Create an internet-based service that will assist
  children with ASC and their carers to improve
  their socio-emotional communication skills.
• Attend both to the recognition and to the
  expression of socio-emotional cues.
• Attend to individual sensory channels (facial
  expressions, prosody, body language) separately,
  and to their integration in context.
• Provide carers with professional information,
  principles to enhance generalisation, and tools
  to monitor childs progress.  

www.ascinclusion.eu
11
Means to attain objectives

1. Create a personalised experience, adjusted to
   individual skills, motivations, and challenges
2. Use serious games,  simulations, animation, video
   and audio clips.
3. Increase motivation through virtual world
   motivational elements.
4. Forums, information sheets, QA section, and
   professional information and reports on the
   childs progress for carers.

www.ascinclusion.eu
12
Product concept

• The product consists of several subsystems that
  are integrated together on an online virtual
  world platform to provide the user with a
  comprehensive, motivating experience.
• Subsystems included in the platform
• Facial expression analysis.
• Body language and gesture analysis.
• Vocal intonation analysis.

www.ascinclusion.eu
13
Product concept
www.ascinclusion.eu
14
Expected Project Impact

• Increased motivation and interest of children in
  socio-emotional phenomena.
• Improvement in childrens expressive emotional
  repertoire (including facial expressions, vocal
  intonation, body language and vocabulary).
• Improvement in childrens emotion recognition
  skills (from facial expressions, voice
  intonation, body language, and context).
• Improvement in parents socio-emotional awareness
  and socio-emotional mediation skills.
• Improvement in childrens social skills and
  adaptive behavior at home and in school.

www.ascinclusion.eu
15
Future impact

• Improvement in childrens social inclusion in and
  out of the educational system.
• Decrease in reliance on professional support
  (e.g. speech and language pathologists,
  psychologists, special education teachers,
  tutors).
• Increase in up-to-date evidence based
  professional service availability in rural areas
  in European countries.

www.ascinclusion.eu
16
Status and milestones
Estimated start point Milestones
1.11.2011 Project kick off
end of year 1 First version of the basic systems
end of year 2 Experiments with system start
end of year 2 First Integrated system
end of project (year 3) Final project results
www.ascinclusion.eu
17
Project partners
18
Acknowledgements

• The research leading to this software product has
  received funding from the European Communitys
  Seventh Framework Programme (FP7, 2007-2013),
  under grant agreement n 289021.