Video Enhanced VR for Teaching Restaurant

1

• Video Enhanced VR for Teaching Restaurant
• Skills to
• Children with Autism

2
Abstract While Virtual
reality has been used for various training
applications, few studies have measured its
effectiveness in teaching social interactions.
This research used web delivered gaming
technology to create virtual worlds where
children with Autism Spectrum Disorder (ASD)
interacted with avatars to practice restaurant
social skills. Videos of similar real world
situations within the virtual sequences
reinforced the lessons and aided in
generalization, a known problem for individuals
with ASD. Before training, two unknown restaurant
social skills were identified for each of five
children with ASD, aged 7 to 16. All children who
completed VR training correctly performed two new
social skills in a virtual restaurant. In
post-training real world restaurants, these
children exhibited at least one new appropriate
social interaction and 75 demonstrated 2 new
social skills.
3
Learning Steps for Autism

• Define a common social skill deficit-eating in a
  restaurant
• Divide social skill into discreet steps(1) Wait
  to be seated, (2) Answer hostess questions, (3)
  Follow hostess to table, (4) Wait for server and
  read menu, (4) Order food, (5) Table conversation
• Define measurable actions in each stepEx Step 1
  includes determining if there is a hostess or you
  seat yourself, waiting in the appropriate place,
  recognizing the hostess, and waiting for others
  already in line
• Identify meaning behind each step - Theory of
  Mind
• Show a range of appropriate responses for each
  step
• Use non-threatening practice of steps with
  avatars in VR
• Display video of real people in same step within
  VR lesson
• Reward or correct at each step based on childs
  actions
• Vary scene/avatar actions to teach variations
• Allow user control of steps for different
  learning abilities
• Vary avatar responses to avoid patterns and teach
  concepts

4
Program Design

• Text with video explanation of each stepWhat to
  do (Question/How to Respond) Explain the social
  action or exchangeWhy? -Explain the meaning
  behind the actionsSee it! -See a video of real
  people doing the stepOops What to do if
  something goes wrongShow possible variations of
  What to do
• Virtual world practice in pizza restaurantThree
  windows on screen Interactive VR world with
  avatars Question/answer and video/correction
  window Displays responses child can pick
  If child picks correct response, shows video
  If child picks wrong response,
  corrects/repeats step What to do/Why? Guidance
• Multi-player practice with other people over web

5
Program Details

• Player moves in virtual worlds with mouse or
  keyboard directional keys
• Avatar program tracks and verbally responds to
  different actions of the player
• Avatars in virtual world include player, a friend
  the player is eating with, a hostess, a waitress,
  and other patrons at different tables
• Visual cues. Example Colors visually separate
  window functions Error/reward statements in red,
  questions with choices in blue, inactive window
  in black
• Videos at each step are repeatable with button
  control
• Avatar spoken words have both text displayed at
  window bottom and audio
• Print out of step information available for
  guidance if child prefers hard copy
• Text with video explanation program was developed
  with Flash
• VR practice world was created and delivered over
  web with Wild Tangent Platform
• Multi-player VR practice was created and
  delivered over web with Adobe Atmosphere
• VR actions are controlled with JAVA programs
• Graphics used Wild Tangent world creation
  toolkit, Atmosphere world creation toolkit, 3d
  StudioMax, Poser, and Photoshop
• Video production was done in a real a restaurant
  during lunch hour. Waitress, hostess, and friend
  were scripted actors. The regular restaurant
  clientele was part of the filming background.
  Fourteen action and seventeen avatar phrase
  variations were used
• Video processing was done with Premier

6
Study Design

• Six children ages 7 to 16 with ASD chosenAll in
  special education programs and recommended by
  therapists. Child 01(Age 11, Male, Asian)-Child
  02(Age 7, Male, White)-Child 03(Age 9, Female,
  White)-Child 04(Age 16, Male, Black)-Child 05(Age
  9, Male, White)-Child 06(12, Female, Black).
• Pre-training testing in real restaurants on six
  skill steps Individual testing was done in
  multiple lunch or dinner visits. Restaurants
  varied and depended on parent recommendation.
  Interactions were not scripted. Actions and
  latency were recorded by 2 observers. Correct
  response required that 90 of all parts of a step
  be done correctly. Child had to display
  inappropriate behavior at more than chance level
  for 2 or more steps to be included in study.
• Five children practiced correct social actions on
  computerTraining was approximately one hour, two
  to three times over a 2 week period. Training
  first involved a treasure hunt game to practice
  navigation in virtual space. All children
  mastered control in no more than two plays. Each
  child then completed the Text, VR Pizza world,
  and web training programs. After training, each
  child assessed his knowledge with a written
  questionnaire. All children came back
  approximately one week later and repeated the
  three training programs and questionnaire.
• Post-training testing in real restaurants on six
  skillsPre-training testing was repeated in the
  same restaurants and data collected as before.
  Appropriate skill responses varied between visits
  because hostess, waitress, patrons, and other
  details varied, creating different required
  social interactions. Two restaurant visits were
  done per child with an average of 3-day
  separation between visits.

7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
Conclusions

• 4 children completed studyChild 03 knew all
  skills in pre-tests. Child 06 was removed because
  of family conflicts
• All learned 2 new skills in virtual worlds at
  100 accuracyFour subjects mastered all 6 skill
  steps to 100 accuracy on all computer training
• All did 1 new skill in real world with 90
  accuracy75 did 2 new skills in real world with
  90 accuracySkill retention was strongest when
  the restaurant visit was close in time frame to
  computer practice. Child 05 did a third computer
  training between restaurant visits after missing
  Step 1 in first post-training and subsequently
  did all skills correctly
• Mixed results for one child who generalized only
  1 skillChild 04 was severely autistic (CARS 34)
  and missed all skills in pre-training. In first
  post-training, he demonstrated 100 accuracy in 5
  of 6 steps. In second restaurant visit, music and
  the unexpected presence of teacher at next table
  distracted child. Computer refresher and retest
  could not be completed before child moved to
  another state
• ProblemsReal world actions did not always match
  discreet step divisions. Example child might
  order correctly but stare at ceiling rather than
  waitress (was given 50 accuracy). Observers
  required that each action be completely
  acceptable by normal social standards
• Graphs of results and details of pilot study
  available from do2learn_at_do2learn.com

11
(No Transcript)
12
(No Transcript)
13
(No Transcript)