Probabilistic Control of Human Robot Interaction: Experiments with a Robotic Assistant for Nursing Homes

1
Probabilistic Control of Human Robot
InteractionExperiments with a Robotic
Assistant for Nursing Homes
Joelle Pineau Michael Montemerlo Martha Pollack
Nicholas Roy Sebastian Thrun Carnegie Mellon
University University of Michigan
2
Introducing Pearl A mobile robotic assistant
for elderly people and nurses
ROLE
Reminding to eat, drink, take meds
cameras
Providing info (TV, weather)
LCD mouth
Monitoring Rx adherence safety
Calling for help in emergencies
microphone speakers
Supporting communication
touchscreen
Remote health services
handle bars
Providing physical assistance
carrying tray
Management support of ADLs
laser
Linking caregiver and resources
sonars
Moving things around
mobile base
3
The Nursebot project in its early days
4
Architecture
High-level controller
Cognitive support
Navigation
Communication
5
Architecture
High-level controller
Cognitive support
Navigation
Communication

• Localization and map building
• (Burgard et al., 1999)
• People detection and tracking
• (Montemerlo et al., 2002)

6
Architecture
High-level controller
Cognitive support
Navigation
Communication

• Autominder system
• (Pollack et al., 2002)

7
Architecture
High-level controller
Cognitive support
Navigation
Communication

• Speech recognition Sphinx system
• (Ravishankar, 1996)
• Speech synthesis Festival system
• (Black et al., 1999)

8
The role of the top-level controller
High-level controller

• Établir les priorités parmi les objectifs des
  différents modules
• Négocier entre plusieurs objectifs ayant des
  coûts/gains variés

Cognitive support
Navigation
Communication
ACTION SELECTION - based on the trade-off
between - goals from different modules - goals
with varying costs / rewards - reducing
uncertainty versus accomplishing goals.
9
Speech recognition with Sphinx
10
Robot control under uncertainty
Belief State P(stweather-today)0.5 P(stappointm
ent-today )0.5
Speechtoday
State weather-today
USER
Action say-weather, update-appointment, clarif
y-query
11
Robot control using Partially Observable Markov
Decision Processes (POMDPs)
Belief state
Observations Costs / Rewards
P(s1)
P(s2)
State
USER ENVIRONMENT WORLD
Actions
Problem Which action allows the robot to
maximize its reward?
12
Methods to solve POMDPs

• Objective Find a policy, ?(b), which maximizes
  reward.

POMDP
New methods?
Performance
AMDP
FIB
QMDP
UMDP
MDP
O(S2AT)
O(S2AO )
O(S2A)
O(S2AO)
O(S2AB)
T
Complexity
13
New approach A hierarchy of POMDPs

• Idea Exploit domain knowledge to divide one
  POMDP into many smaller ones.
• Motivation Complexity of POMDP solving grows
  exponentially with of actions.
• Assumption We are given POMDP M
  S,A,?,b,T,O,R and hierarchy H

subtask
Act
abstract action
ExamineHealth
Navigate
Move
ClarifyGoal
VerifyPulse
VerifyMeds
primitive action
North
South
East
West
14
PolCA Planning with a hierarchy of POMDPs

• Step 1 Select the action set

Navigate
AMove N,S,E,W
Move
ClarifyGoal
South
East
West
North
ACTIONS North South East West ClarifyGoal VerifyPu
lse VerifyMeds
15
PolCA Planning with a hierarchy of POMDPs

• Step 1 Select the action set
• Step 2 Minimize the state set

Navigate
AMove N,S,E,W SMove X,Y
Move
ClarifyGoal
South
East
West
North
ACTIONS North South East West ClarifyGoal VerifyPu
lse VerifyMeds
STATE FEATURES X-position Y-position X-goal Y-goal
HealthStatus
16
PolCA Planning with a hierarchy of POMDPs

• Step 1 Select the action set
• Step 2 Minimize the state set
• Step 3 Choose parameters

Navigate
AMove N,S,E,W SMove X,Y
Move
ClarifyGoal
South
East
West
North
ACTIONS North South East West ClarifyGoal VerifyPu
lse VerifyMeds
STATE FEATURES X-position Y-position X-goal Y-goal
HealthStatus
PARAMETERS bh,Th,Oh,Rh
17
PolCA Planning with a hierarchy of POMDPs

• Step 1 Select the action set
• Step 2 Minimize the state set
• Step 3 Choose parameters
• Step 4 Plan task h

Navigate
AMove N,S,E,W SMove X,Y
Move
ClarifyGoal
South
East
West
North
ACTIONS North South East West ClarifyGoal VerifyPu
lse VerifyMeds
STATE FEATURES X-position Y-position X-goal Y-goal
HealthStatus
PARAMETERS bh,Th,Oh,Rh
PLAN ?h
18
PolCA in the Nursebot domain

• Goal A robot is deployed in a nursing home,
  where it provides reminders to elderly users and
  accompanies them to appointments.
• Domain S512, A20, O19
• Hierarchy

19
Sample scenario
20
Results for dialogue system
POMDP policy MDP policy
0.18
0.1
0.1
21
Summary

• We have developed a first prototype robot able to
  serve as a mobile nursing assistant for elderly
  people.
• The top-level controller uses a hierarchical
  variant of POMDPs to select actions.
• This allows it to acquire necessary information
  and successfully complete assigned tasks.
• Probabilistic techniques have been found to be
  very useful to flexibly model and track
  individuals.

22
The Nursebot team
CMU - Robotics Greg Armstrong Michael
Montemerlo Joelle Pineau Nicholas Roy Jamie
Schulte Sebastian Thrun CMU - HCI/Design Francin
e Gemperle Jennifer Goetz Sarah Kiesler Aaron
Powers
U. of Pittsburgh - Nursing Jacqueline
Dunbar-Jacobs Sandra Engberg Judith Matthews U.
of Pittsburgh - CS Don Chiarulli Colleen
McCarthy U. of Freiburg - CS Maren
Bennewitz Wolfram Burgard Dirk Schulz
U. of Michigan - CS Laura Brown Dirk
Colbry Cheryl Orosz Bart Peintner Martha
Pollack Sailesh Ramakrishnan Standard
Robotics Greg Baltus
For more details www.cs.cmu.edu/nursebot