Symbol Anchoring in a Grounded Layered Architecture with Integrated Reasoning

1
Symbol Anchoring in aGrounded Layered
Architecturewith Integrated Reasoning

• Stuart C. Shapiro
• Department of Computer Science and Engineering
• and Center for Cognitive Science
• University at Buffalo
• shapiro_at_cse.buffalo.edu

2
Based on

• Stuart C. Shapiro Haythem O. Ismail, Anchoring
  in a grounded layered architecture with
  integrated reasoning, Robotics and Autonomous
  Systems 43, 2-3 (May 2003) 97-108.

3
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

4
Definition

• Anchoring is the problem of connecting, inside
  an artificial system, symbols and sensor data
  that refer to the same physical objects in the
  external world.
• Silvia Coradeschi Alessandro Saffiotti

5
Embodied Cassie

• A computational cognitive agent
• Embodied in hardware
• or Software-Simulated
• Based on SNePS and GLAIR.

6
Cassie, the FEVAHR(Foveal ExtraVehicular
Activity Helper-Retriever)
7
FEVAHR/Cassie in the Lab
8
Crystal Cassie
First Person Perspective Views
9
Patofil and Filopatfrom The Trial, The Trail
A VR drama by Josephine Anstey et al.
10
DeliveryAgent
Using Byron Weber Beckers Java version of Rich
Pattis Karel the Robot
11
Magellan ProTM Mobile RobotfromiRobot
12
SNePS

• Knowledge Representation and Reasoning
• Intensional Representation
• Propositions as Terms
• SNIP SNePS Inference Package
• Specialized connectives and quantifiers
• SNeBR SNePS Belief Revision
• SNeRE SNePS Rational Engine (Acting Language)
• Interface Languages
• SNePSUL Lisp-Like
• SNePSLOG Logic-Like
• GATN for Fragments of English.

13
Interaction with Cassie
(Current) Set of Beliefs SNePS
English (Statement, Question, Command)
Reasoning Clarification Dialogue Looking in World
GATN Parser
(Updated) Set of Beliefs SNePS
Actions SNeRE
(New Belief) SNePS
Answer SNIP
GATN Generator
Reasoning
English sentence expressing new belief
answering question reporting actions
14
Entities, Terms, Symbols, Objects

• Cassies mental entity a person named Stu
• SNePS term B4
• Object in world

15
GLAIR Architecture
Grounded Layered Architecture with Integrated
Reasoning
Knowledge Level
Mind
SNePS
Perceptuo-Motor Level
IP Sockets
B o d y
NL
Sensory-Actuator Level
Vision
Sonar
World
Motion
Proprioception
16
Alignment
KL
SNePS term
PML structure
Body (PML/SAL)
World
Object/Phenomenon
Action
17
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

18
World Objectsto Feature Tuples
World
PML/SAL
ltHeight, Width, Texture, .
. . gt
19
Feature Tuplesto KL Terms
PML/SAL
KL
ProperName(B4, Stu)
ltHeight, Width, Texture, .
. . gt
Alignment
20
Incomplete PML-Descriptions
PML/SAL
KL
Height(B4, B12)
ltHeight, nil, nil, .
. . gt
21
Unifying PML-Descriptions
PML/SAL
KL
ltx, nil, nil, ... gt
B20
ltnil, y, nil, ... gt
B30
Prop
Isa
ltnil, nil, z, ... gt
B31
Prop
ltx, y, z, ... gt
B6
22
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

23
Deictic Modality Registersfor being situated
in the world
PML
KL Terms denoting
I You Now Vision . . .
Cassie Addressee Current time Current state(s) of
looking at x . . .
24
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

25
Primitive Actions Alignedwith PML/SAL Functions
PML
KL
SAL
World
Find(B4)
M2(B4)
lt... gt
Modality Registers
A KL symbol is a pivot coordinating different
modalities.
26
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

27
Aligning NOW using MTF
Maximal Temporal Frame based on NOW
KL
PML
NOW
Semi-lattice of times, all of which contain
NOW, any of which could be meant by
now Finite---only conceptualized times of
conceptualized states
28
Moving NOW with MTF
KL
NOW
PML
Moves when Cassie acts, newly observes a
state, or is informed of a new state. Always
includes times of states in modality registers.
29
Providing a Feel for Time
q
t1
t2
KL
PML
hom
COUNT
n
NOW
0
30
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

31
Aligning Lexemes/NL
PML/SAL
KL
World
ProperName(B4, Stu)
npr
ctgy
Stu
Isa(B4, person)
n
ctgy
person
sing
num
Grammar
NL
32
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

33
Acting 1
34
Acting 2
Follow a red robot.
I found a red robot. I am looking at a red robot.
35
Acting 3
Follow a red robot.
I found a red robot. I am looking at a red robot.
I went to a red robot. I am near a red robot. I
am following a red robot.
36
Use of Indexicals 1
Come here.
I found you. I am looking at you.
37
Use of Indexicals 2
Come here.
I found you. I am looking at you.
I came to you, Stu. I am near you.
38
Use of Indexicals 3
Who am I?
Your name is Stu and you are a person.
Who have you talked to?
I am talking to you.
Talk to Bill.
I am talking to you, Bill.
Come here.
39
Use of Indexicals 4
Come here.
I found you. I am looking at you.
40
Use of Indexicals 5
Come here.
I found you. I am looking at you.
I came to you. I am near you.
41
Use of Indexicals 6
Who am I?
Your name is Bill and you are a person.
Who are you?
I am the FEVAHR and my name is Cassie.
Who have you talked to?
I talked to Stu and I am talking to you.
42
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

43
Summary

• KL terms denote mental entities.
• KL terms aligned to PML structures.
• PML variables/registers contain KL terms.
• PML variables/registers/structures grounded in
  world via sensors effectors.
• PML attentional structures anchor symbols by
  changing only when motivated.

44
Summary from Jackendoff
Ray Jackendoff, Foundations of Language, Oxford
University Press, 2002, Fig. 11.1, p.
348. (Independent work)
45
For More Information

• Personnel
• Manual
• Tutorial
• Bibliography
• ftpable SNePS source code
• etc.
• http//www.cse.buffalo.edu/sneps/