A Biologically Motivated Software Architecture for an Intelligent Humanoid Robot

1
A Biologically Motivated Software Architecture
for an Intelligent Humanoid Robot

• Richard Alan Peters II, D. Mitchell Wilkes,
• Daniel M. Gaines, and Kazuhiko Kawamura

Center for Intelligent Systems
Vanderbilt University
Nashville, Tennessee, USA
2
Intelligence
The ability of an individual to learn from
experience, to reason well, to remember important
information, and to cope with the demands of
daily living. (R. Sternberg, 1994).
Intelligence has emerged through evolution and is
manifest in mammals. The processes of designing
an intelligent robot might be facilitated by
mimicking the natural structures and functions of
mammalian brains.
3
Topics

• Mammalian Brains
• ISAC, the Vanderbilt Humanoid Robot
• A Control System Architecture for ISAC
• A Partial Implementation
• Research Issues

4
The Structure of Mammalian Brains
Krubitzer, Kass, Allman, Squire

• The evolution of structure
• Common features of neocortical organization
• Species differences
• Memory and association
• Attention
• Implications for robot control architectures

5
The Evolution of Structure
Figure Leah Krubitzer
6
Common features of Neocortical Organization

• Somatosensory Cortex (SI, SII)
• Motor Cortex (M)
• Visual Cortex (VI, VII)
• Auditory Cortex (AI)
• Association Cortex
• Size differences in cortical modules are
  disproportionate to size differences in cortex

7
Common features of Neocortical Organization
Figure Leah Krubitzer
8
Species Differences

• Sizes and shapes of a specific cortical field
• Internal organization of a cortical field
• Amount of cortex devoted to a particular sensory
  or cognitive function
  
• Number of cortical fields
• Addition of modules to cortical fields
• Connections between cortical fields

9
Memory a Functional Taxonomy
Squire

• Immediate memory data buffers for current
  sensory input holds information for about 0.1s
  
• Working memory scratch-pads, e.g. phono-logical
  loop, visuospatial sketch pad the representation
  of sensory information in its absence
  
• Short term memory (IM WM) is a collection of
  memory systems that operate in parallel
  
• Long-term memory can be recalled for years
  different physically from STM

10
Memory Biological Mechanisms

• Immediate memory chemicals in synapse
• Working memory increase in presynaptic
  vesicles intra-neuron and inter-neuron protein
  release and transmitter activity
  
• Long-term memory growth of new synapses
  requires transcription of genes in neurons.

11
Association

• The simultaneous activation of more than one
  sensory processing area for a given set of
  external stimuli
  
• A memory that links multiple events or stimuli
• Much of the neocortex not devoted to sensory
  processing appears to be involved in association

12
Memory and Sensory Data Bandwidth

• Bandwidth of signals out of sensory cortical
  fields is much smaller than input bandwidth
  
• Sensory cortical fields all project to areas
  within association cortex
  
• Suggests Environment is scanned for certain
  salient information, much is missed. Previous
  memories linked by association fill in the gaps
  in information.

13
Attention a Definition

• An apparent sequence of spatio-temporal events,
  to which a computational system or subsystem
  allocates a hierarchy of resources.

In that sense, the dynamic activation of
structures in the brain is attentional .
14
Attention Some Types
Visual where to look next Auditory sudden
onset or end of sound Haptic bumped into somet
hing Proprioceptic entering unstable position
Memory triggered by sensory input Task
action selection Conscious recallable event se
quence
15
Attention Executive Control
Figure Posner Raichle
16
Mammalian Brains

• Have sensory processing modules that work
  continually in parallel
  
• Selectively filter incoming sensory data and
  supplement that information from memory through
  context and association
  
• Exhibit dynamic patterns of activity through
  local changes in cellular metabolism shifts in
  activation

17
ISAC, a Two-Armed Humanoid Robot
18
Physical Structure of ISAC

• Arms two 6 DOF actuated by pneumatic McKibben
  artificial muscles
  
• Hands anthropomorphic, pneumatic with proximity
  sensors and 6-axis FT sensors at wrists
  
• Vision stereo color PTV head
• Audition user microphone
• Infrared motion sensor array

19
ISAC Hardware under Construction

• Hybrid pneumatic / electric anthropomorphic hand
• Head mounted binaural microphone system
• Finger tip touch sensors

20
Computational Structure of ISAC

• Network of standard PCs
• Windows NT 4.0 OS
• Special hardware limited to device controllers
• Designed under Vanderbilts Intelligent Machine
  Architecture (IMA)

21
Low-Level Software Architecture IMA

• Software agent (SA) design model and tools
• SA 1 element of a domain-level system descr.
• SA tightly encapsulates all aspects of an
  element
  
• SAs communicate through message passing
• Enables concurrent SA execution on separate
  machines on a network
  
• Facilitates inter-agent communication w/ DCOM
• Can implement almost any logical architecture

22
Primitive Agent Types

• Hardware provide abstractions of sensors and
  actuators, and low level processing and control
  (e.g., noise filtering or servo-control loops).
  
• Behavior encapsulate tightly coupled sensing -
  actuation loops. May or may not have runtime
  parameters .
  
• Environment process sensor data to update an
  abstraction of the environment. Can support
  behaviors such as move-to'' or fixate'' which
  require run-time parameters.
• Task encapsulate decision-making capabilities,
  and sequencing mechanisms for hardware, behavior,
  and environment agents.

23
Agent Object Model

• The agent object model describes how an agent
  network, defined by the robot-environment model,
  is constructed from a collection of component
  objects

24
IMA Component Objects

• Agent Comp. agent interfaces to manager and to
  persistent streams
  
• Policy Comp. encapsulates an OS thread
• Representation Comp. a DCOM object that
  communicates an agents state to other agents
  
• Mechanism Comp. configurable objects that can
  be invoked to perform one of a set of
  computations
  
• Agent Links interfaces defined by
  representations
  
• Relationship Comp. manage a set of agent links
  to selectively update and / or use each
  participant link

25
Properties of IMA

• Granularity multiple logical levels
• Composition agents can incorporate agents
• Reusable can be combined for new
  functionalities
  
• Inherently Parallel asynchronous, concurrent
  op.
  
• Explicit Representation sensor info is
  ubiquitous
  
• Flat Connectivity all agents are logically
  equal w.r.t. sensory info access and actuator
  commands
  
• Incremental All modules that produce commands
  for the hardware work in an incremental mode

26
Correspondence of IMA Agents to System-Level
Entities
27
A Bio-Inspired Control Architecture

• IMA can be used to implement almost any control
  architecture.
  
• Individual agents can tightly couple sensing to
  actuation, and incorporate each other a la
  subsumption (Brooks).
  
• IMA Inter-agent communications facilitate motor
  schemas (Arkin).
  
• Composition of agents which have flat
  connectivity enables hybrid architectures

28
ISAC Control System Architecture

• Primary Software Agents
• Sensory EgoSphere
• Attentional Networks
• Database Associative Memory
• Attentional Control via Activation
• Learning
• System Status Self Evaluation

29
Example Primary Software Agents
Vision
Audition

• Visual attention
• Color segmentation
• Object recognition
• Face recognition
• Gesture recognition

Aural attention Sound segmentation Speech recogn
ition
Speaker identification Sonic localization
30
Example Primary Software Agents
Motor
Others

• L R Arm control
• L R Hand control
• PTV motion

Infrared motion det. Peer agents Object agents
Attention agent
Sensory data recds.
31
Higher Level Software Agents

• Visual tracking
• Visual servoing
• Move EF to FP
• Dual arm control
• Person Id
• Interpret V-Com
• Reflex control

Robot self agent Human agent Object agents (vari
ous) Visually guided grasping Converse Reflex c
ontrol
Memory association
32
Agents and Cortical Fields

• Agents can be designed to be functionally
  analogous to the common field structure of the
  neocortex.
  
• Visual, auditory, haptic, proprioceptic,
  attentional, and memory association agents remain
  on constantly and always transform the current
  sensory inputs from the environment

33
Atlantis a Three Layer Architecture
Erran Gat

• Deliberator world model, planner
• Sequencer task queue, executor, monitors
• Controller sensor / actuator coupled behaviors

34
Atlantis General Schematic
Figure Erran Gat
35
Three-Layer Control with IMA

• Elements of control layer agents.
• Sequencing through links depending on
  activation vectors. (Due to flat connectivity.)
  
• Deliberation Various agents modify the links
  and activation levels of others. (Due to
  composability.)

36
Virtual Three-Layer Architecture
Deliberative Agent
Sn
IMA Agent
S1
S3
...
S2
activation link
max activation
37
3-Layer Control through Schemas

• Agents compete with each other for control of
  other agents and the use of hardware resources.
  
• They do this by modifying activation vectors
  associated with agent relationships.
  
• Thus, the sequencer is equivalent to a motor
  schema

38
Simple Task Agent Operation
39
Current Implementation

• ISAC is being designed primarily as a human
  service robot to interact smoothly, naturally
  with, with people.
  
• Several high level agents mediate the
  interaction robot self agent, human agent,
  object agents.

40
Two High-Level Agents

• Human Agent encapsulates what the robot knows
  about the human
  
• Robot Self Agent maintains knowledge about the
  internal state of the robot and can communicate
  this with the human

41
Human-Robot Interaction Desiderata

• The robot is user-friendly it has a humanoid
  appearance and can converse with a person.
  
• A person can discover the robots abilities
  through conversation.
  
• The person can evoke those abilities from the
  robot.
  
• The robot can detect its own failures and report
  them to the person.
  
• The person and the robot can converse on the
  robots internal state for failure diagnosis.

42
Human-Robot Interaction
A
IMA Primitive Agent
Hardware Interface
DBAM
Human Interaction
Robot Self Agent
Human Agent
Human
Robot
A
A
A
A
A
A
A
Software System
43
Robot Self Agent

• Human Interaction maps the person's goals into
  robot action
  
• Action Selection activates primitive agents
  based on information from the human interaction
  component
  
• Robot Status Evaluation detects failure in
  primitive agents and maintains information about
  the overall state of the robot

44
Task Agent 1
Task Agent 2
Task Agent N
45
Human Interaction Agents

• Conversation Module
• Interprets the human's input and generates
  responses to the human
  
• Based upon the Maas-Neotek robots developed by
  Mauldin
  
• Increases or decreases the activation level of a
  primitive agent
  
• Pronoun Module
• Resolves human phrases and to environment
  primitive agents.
  
• acts as a pointer or reference for use by task
  primitive agents
  
• points to other modules of the Robot Self Agent
  or to primitive agents for purposes of failure
  description
  
• Interrogation Module
• handles discussions about the robot's status
  involving questions from the human
  

46
Status Evaluation Agents

• Emotion Module
• artificial emotional model used to describe the
  current state of the robot
  
• provide internal feedback to the robot's software
  agents
  
• fuzzy evaluator is used to provide a textual
  description for each emotion
  
• Description Module
• contains names and a description of the purpose
  of primitive agents
  
• information about the status of primitive agents
  active or inactive, and successful or
  unsuccessful

47
Action Selection Agents

• Activator Module is a clearing-house for
  contributing to the activation state of primitive
  agents
  
• Other Robot Self Agent modules can adjust the
  activation level of primitive agents through the
  Activator Module

48
Details of Agents IMA Component Level
Emotion Agent
Conversation Agent
Description Agent
Fuzzy
Text
Text In
Config
PA Abilities
Rep N
Rep 1
Rep 2
Description Relationship
PA Names
Text Out
Interpreter
Rel 1
Rel 2
Rel N
Activator Agent
Pronoun Agent
Interrogation Agent
Rep 1
Rep 2
Rep N
Why
Text In
Config
Activator
Text In
What
Pointer
Binding Mechanism
Link1
Link2
LinkN
Where
49
Human Agent
50
Testbed System
51
Visual Attention FeatureGate

• A model of human visual attention, developed by
  Kyle R. Cave, that is consistent with empirical
  observation and experimental results.
  
• Activates locations in the visual field based on
  their local salience and discriminability with
  respect to sets of features.
  
• A pyramid structure where info is gated to the
  next (smaller) level as a function of local
  activations

52
Decision Making with FeatureGate

• FeatureGate must determine if no target objects
  are present, one target object is present or
  multiple target objects are present.
  
• The Bayes Test for Multiple Hypotheses is adapted
  to FeatureGate to decide which of the above
  hypotheses has occurred. The object of this test
  is to determine which hypothesis has the minimum
  cost for a given result.
• Probability density functions (pdfs), a priori
  probabilities and cost functions determine the
  total cost for each hypothesis.
  
• Since a priori probabilities and cost functions
  are held constant, the pdfs are the only unknowns.

53
Information Used in Hypothesis Testing

• The most efficient data to create pdfs for
  hypothesis testing is contained in levels 2, 3
  and 4 out of levels 0-9.
  
• At these three levels, the top ten activation
  values that are at least 65 of the maximum
  activation value are stored. The Euclidean
  distances between the units containing all of
  these stored activation values are computed.
• The pdfs at each level for single target object
  presence and for multiple target object presence
  are formulated from the Euclidean distances
  measured in test trials for each hypothesis.
• The pdf for absence of target objects is a
  Gaussian distribution calculated with a
  user-defined deviation.

54
FeatureGate Test for Multiple Hypotheses

• Choose Hj for which Cj is a minimum
• Hj is hypothesis j.
• Cj is the total cost for hypothesis j.
• Cj,i is the cost of choosing hypothesis j given
  hypothesis i is true.
  
• pi,l(y) is the probability density function for
  hypothesis i at level l.
  
• P(Hi) is the probability that hypothesis i occurs.

55
Agent Coupling and Composition

• Agents
• can subsume others, suppress or inhibit, or
  initiate, terminate, send messages,
  
• can be sequenced deterministically or
  stochastically,
  
• can interrupt each other, or
• can possess activation vectors that can be
  modified by other agents.
  
• Through agent relationships weighted links can be
  forged or removed.

56
Camera Head Control
Image Sensory
Visual Attention Network
Retinal Motion Signal
Left
Eye Motion Center
Saccade
Right
VOR
Smooth Pursuit
Vergence
OKR
Pan
Motor Signal
Motor Command
Tilt
Camera Head (Eyes) Controller
57
Features
Vergence
Fixating on same object Clues for target Segmen
ting
Saccade
Quickly shifting gaze at once
Smooth Pursuit
Smoothing movement Minimizing target slip
OKN
VOR
Stabilizing target on visual field
Keeping eyes on target while moving head
58
Methods
Vergence
Saccade
Neural Net-based Saccadic Map
Color Clue providing Error for
Online Weights Adjustment
Edge-based Zero Disparity Filter (Coombs
Brown) Logical AND of left/right vertical-edge i
mage
Smooth Pursuit
World Coordinates ?-?-? Predictive Control
(Bar-Shalom Fortmann) Image Coordinates RB
F Net Tracking Control (Looney)
OKN
VOR
Compensation for Head Velocity and Acceleration
Compensation for Target Movement in Visual Flo
w
Field
Under investigation/design
59
How do humans reach to grasp?
First, humans fixate on the target
Fixation creates an object-centered frame
Finally, humans move the hand relative to the
object-centered frame. This approach is adopted i
n our Humanoid
60
FPS Fixation Point Servoing

• The manipulation frame is fixated at the object
• Servoing IF A1A2 THEN (Down, Right, Back

61
Schema Control (L)
Actv.
Agent Policy (Rule)
Eval.
Activation Relationship
Agent Engine
Object. Geom..
State Rep.
Eval.
Eval.
Actv.
Eval.
Eval.
Force Sensor (L)
Force Sensor (R)
Actv.
Eval.
Eval.
Actv.
Right Arm Position
Left Arm Position
Schema Control (R)
Left Gripper
Right Gripper
Resource Comps.
Evaluators/Activators
Visual Servoing in IMA
62
Schema Control/ Arm
FPS
Actv.
Agent Policy (Rule)
Agent Engine
State Rep.
Eval.
Eval.
Actv.
Actv.
Eval.
Eval.
Activation Relationship
Track Object
Arm Position
Actv.
Grasping an Object
Schema Control/ P.T.
Pan Tilt Positions
Gripper
Evaluators/Activators
Resource Comps.
63
PneuHand
3 Fingers Opposable Thumb Pnuematic Low Cost
Applications Grasping Objects Gestures

64
In Current Development

• Sensory EgoSphere (SES)
• Database Associative Memory (DBAM)
• Learning Algorithms
• System Status Evaluation

65
A Working Memory Sensory EgoSphere

• "a two-dimensional spherical surface that is a
  map of the world as seen by an observer at the
  center of the sphere. Visible points on regions
  or objects in the world are projected on the
  EgoSphere wherever the lines of sight from a
  sensor at the center of the EgoSphere to points
  in the world intersect the surface of the
  sphere.
• J. S. Albus

66
Sensory EgoSphere
67
Sensory EgoSphere

• Store for sensory data that can be localized,
  (e.g., visual imagery, localized sound, motion
  vectors, distance to closest surface,etc.)
  
• Data is time stamped.
• SES is a multiresolution, 2Dx1D map of the
  robot's environment indexed by elevation,
  azimuth, and time.
  
• Tessellated as a geodesic dome.

68
The SES as Working Memory

• Sensory data can be copied to the SES by any
  agent that finds it important.
  
• Simple descriptors (e.g. Fourier magnitude
  components of edge map, LPC coefficients)
  computed for each input can be stored. New input
  from the same location is compared to detect
  changes.
• Associations with long term memory records can be
  formed.
  
• Long term memories are consolidated by the
  repetition of such associations.

69
Database Associative Memory

• The set of agents form a database of records.
• The links between records are associations.
• Records of sensory data can be associated by
  space-time proximity or sequence, co-occurrence
  during a specific task.
  
• Includes all SES records.
• Long term memories indicated by strengths of
  links to associated agents and data records.

70
Learning

• ISAC should modify programmed skills through
  experience and acquire new skills by example or
  trial and error.
  
• Learning includes modifying the links between
  agents in the DBAM (modification of sequences in
  logical layer 2).
  
• When stuck or in a learning mode, non-maximally
  activated agents could be initiated. The results
  analyzed (by robot or teacher) and activations
  altered accordingly through a Reinforcement
  Learning protocol.
• Could use, e.g., spreading activation.

71
System Status Self Evaluation

• Most agents have a vector that indicates their
  current status and/or a measure of confidence in
  their most recent results.
  
• Calling agents can use these to detect faults.
• A System Status Evaluation Agent keeps track of
  current problems and biases agent activations if
  necessary.
  
• Could be used by RL / SA network.