Reactive Systems

1
Reactive Systems

• Yolanda Gil
• CS 541, Fall 2003
• (Thanks to Karen Myers from SRI International)

2
The problem with plans (I)

• Attack Goliath
• Gather pile of rocks
• Grasp slingshot
• Fire at giant
• Hit on the head

3
The problem with plans (II)

• Unknown how many stones
• Unknown if stones
• Unknown how many attempts
• Conditions for termination
• What if failure
• Check state

• Attack Goliath
• Gather pile of rocks
• Grasp slingshot
• Fire at giant
• Hit on the head

4
Reactive Systems

• Embedded in the real world
• Have sensors and effectors
• Actively test the external environment
• Need to respond to events in dynamic environments
• Failure may require aborting and generating new
  response
• Do we need deliberate reasoning (planning)?

5
Outline and Informal Roadmap

• Control systems
• Networks of variables (arcs) and functions
  (nodes)
• Reactive Action Packages (RAPs)
• Networks of conditions and tasks
• Task Control Architecture (TCA)
• Network arranged according to vertical
  capabilities
• Procedural Reasoning System (PRS)
• Integrates planning, BDI, and reactive techniques
• Anytime algorithms
• When time is short, managing what you think about
• Other approaches and issues

6
Readings

• RAP (http//people.cs.uchicago.edu/firby/raps)
• Firby, J Task Networks for Controlling
  Continuous Processes, Proceedings of Artificial
  Intelligence Planning conference, 1994.
• TCA (http//www-2.cs.cmu.edu/afs/cs/project/TCA/re
  lease/tca.orig.html, http//www-2.cs.cmu.edu/afs/c
  s/project/TCA/release/tca.html)
• Simmons, R. Structured Control for Autonomous
  Robots, IEEE Transactions on Robotics and
  Automation, Feb 1994.
• PRS (http//www.ai.sri.com/prs)
• Reactive reasoning and planning an experiment
  with a mobile robot, M. Georgeff and A. Lansky,
  in Proceedings of AAAI, 1987.
• Anytime algorithms
• Zilberstein, S. Using Anytime Algorithms in
  Intelligent Systems, AI Magazine, 1996.

7
Control Systems An Example (I)Control of
temperature profile for a spray deposition
process. Jones, P.D.A. Duncan, S.R. Rayment,
T. Grant, P.S. IEEE transactions on control
systems technology special issue on control of
industrial spatially distributed processes, Sept
2003.
8
Control Systems An Example (II)Control of
temperature profile for a spray deposition
process. Jones, P.D.A. Duncan, S.R. Rayment,
T. Grant, P.S. IEEE transactions on control
systems technology special issue on control of
industrial spatially distributed processes, Sept
2003.
9
Beyond Stimulus-Response

• Address problems that require a combination of
• Coordinated activity to accomplish tasks
• Reactivity to world dynamics
• Situate control decisions within an explicit,
  persistent decision-making framework

10
Reactive Action Packages (RAP)
11
A Symbolic Discrete Task
12
Waiting for a signal to proceed
13
Concurrent tasks
14
More Complex Task Networks
15
Task Control Architecture (TCA)

• Vertical task decomposition several
  task-specific modules communicate through a
  central control module
• Deliberation top-down task-subtask, resolve
  constraints
• Central control routes messages
• Inform, query, command, monitor

16
Ambler Walking Robot
17
Ambler Modules
18
Ambler Task Tree
19
TCA Monitoring

• Central control traverses tree and handles
  messages
• asks gait planner to traverse arc,
• gait planner asks terrain mapper for elevation
  map in order to take steps
• Gait planner asks leg recovery planner to place
  leg, move leg, move body,
• Gait planner activates monitor whether achieved
  position

20
TCA Control

• Ordering and temporal constraints
• Delay planning constraint goal cannot be issued
  until previous task achieved
• Can do place leg planning while monitoring
  achieve position
• Exception handling error recovery modules
  examine and modify task trees
• Eg if position not achieved, add take steps
  subtask

21
Ambler Planning and Execution
22
An Alternative to TCAs Vertical
CapabilitiesHorizontal Layered Control

• Reason about behavior of objects
• Plan changes to the world
• Identify objects
• Monitor changes
• Build maps
• Explore
• Wander
• Avoid objects

23
Procedural Reasoning System (PRS)

• Framework for symbolic reactive control systems
  in dynamic environments
• Eg Mobile robot control
• Eg diagnosis of the Space Shuttles Reaction
  Controls System

24
PRS Main Features

• Pre-compiled procedural knowledge
• BDI (Belief, Desires, Intentions) foundation
• Combines deliberative and reactive features
• Plan selection, formation, execution, sensing
• Plans dynamically and incrementally
• Integrates goal-directed and event-driven
  behavior
• Can interrupt plan execution
• Meta-level reasoning
• Multi-agent planning

25
PRS Architecture
User
Tasks
Procedures
Interpreter
Intentions
Database
World
26
PRS Architecture Database

• Contains beliefs or facts about the world
• Includes meta-level information
• Eg goal G is active

User
Tasks
Procedures
Interpreter
Intentions
Database
World
27
PRS Architecture Tasks

• Represent desired behavior
• Conditions over some time interval
• eg (walk a b) set of behaviors in which agent
  walks from a to b)

User
Tasks
Procedures
Interpreter
Intentions
Database
World
28
Expressing Tasks in a Dynamic Environment

• (! P) -- achieve P
• (? P) -- test P
• ( P) -- maintain P
• ( C) -- wait until C
• (-gt C) -- assert C
• (gt C) -- retract C

29
PRS Architecture Intentions

• Currently active procedures
• Procedure currently being executed

User
Tasks
Procedures
Interpreter
Intentions
Database
World
30
PRS Architecture Procedures

• Pre-compiled procedures
• Express actions and tests to achieve goals or to
  react to conditions

User
Tasks
Procedures
Interpreter
Intentions
Database
World
31
Representing Procedures with Act Formalism

• Environment conditions
• Purpose (goal or condition)
• applicability criteria
• Plot
• directed graph
• partially ordered conditional parallel actions,
  loops
• Successful node execution by achievement of
  nodes goals
• If no body primitive action
• Metapredicates
• Achieve Achieve-By proc
• Test Conclude effects
• Wait-Until Use-Resource
• Require-Until

Cross-Country Delivery
Cue


(ACHIEVE (DELIVER CUSTOMER.1 GOODS.1))
(ACHIEVE
(RECORD-INVOICE
Preconditions

CUSTOMER.1

GOODS.1
(TEST
INVOICE.1) )
(AND
(LOCATED CUSTOMER.1 CITY.2)
(LOCATED GOODS.1 CITY.1)
(DISTANCE CITY.1 CITY.2 DISTANCE.1)
(gt DISTANCE.1 1000) ) )
(ACHIEVE-BY
(ACHIEVE-BY
(LOCATED
(LOCATED
Setting

AIRCARGO.1
LANDCARGO.1

CITY.2)
CITY.2)
(TEST
SHIP-BY-AIR) )
SHIP-BY-RAIL) )
(AND
(AIR-SHIPMENT AIRCARGO.1 GOODS.1)
(LAND-SHIPMENT LANDCARGO.1 GOODS.1) ) )
Resources

(ACHIEVE

(LOCAL-DELIVERY
- no entry -
CUSTOMER.1
GOODS.1) )
(CONCLUDE
Propertities

(COMPLETED-INVOICE

INVOICE.1) )
(AUTHORING-SYSTEM ACT-EDITOR)
Comment


Long distance delivery of goods to customers
32
PRS Interpreter
Execution Cycle 1. New information arrives that
updates facts and/or tasks 2. Acts are triggered
by new facts or tasks 3. A triggered Act is
intended 4. An intended Act is selected 5. That
intention is activated 6. An action is
performed 7. New facts or tasks are posted 8.
Intentions are updated
33
Meta-Reasoning

• Can include meta-level procedures
• eg choose among multiple applicable procedures
• eg evaluate how much more reasoning can be done
  within time constraints
• eg how to achieve a conjunction or disjunction
  of goals

34
Shuttles RCS Malfunction Handling
RCS Controls

• Automates specification and execution of RCS
  malfunction procedures.
• Reacts to changes in RCS. Ensures safe operation
  while carrying out diagnosis and remediation
  procedures.

RCS Jets
Jet Fail - On
Achieve

Position
valve.ox
closed,
Cue

Position
valve.fu
closed
Test

Alarm sounding,
Shuttle
MESSAGES

RCS warning light on,
GPC
Achieve

Status RCS
jet.1
is failed-on,

Notify "Thruster
jet.1
failed-on"

GPC displays
dir.1
for
jet.1
for
rcs.1



Preconditions
Test
External
External
Test
Test

Direction
jet.1
is
dir.1

TASKS
FACTS

Not high-usage of
jet.1

High-usage of
jet.1


Regulator Test
Setting
Test
Jet Fail - On

Connected
manifold.ox
to
jet.1,
FACTS
Test
Test
Dump Propellant


Connected
manifold.fu
to
jet.1,
TASKS

Not type
jet.1
vernier

Type
jet.1
vernier
BELIEFS

Connects
valve.fu
by
leg.fu



to
manifold.fu,

Connects
valve.ox
by
leg.ox

Achieve


to
manifold.ox
,

Pressure
manifold.ox
is
pres.ox
,

Oxidizer-subsystem
ox.1
of
rcs.1
,
Executing

Pressure
manifold.fu
is
pres.fu
procedures can post

Fuel-subsystem
fu.1
of
rcs.1
,
Procedure
GOALS, FACTS,

Part
valve.ox
of
ox.1
,
Library
BELIEFS

Part
valve.fu
of
fu.1

or


Test
Test
send MESSAGES
Determine new
Achieve

gt pres.ox 130,

pres.ox 130,
procedures

Notify "Thruster
jet.1
failed-on

gt pres.fu 130

pres.fu 130
that are eligible
Jet Fail - On


ELECTRICALLY"
for execution
Regulator Test
Achieve
Achieve

Notify "Thruster
jet.1
failed-on

Notify "TURN-OFF rcs.1 manifold.ox


INPUT CARD"


manifold.fu DRIVER"
Select procedures
for execution
35
Multiple Tasks, Multiple Agenst

• Multithreaded operation multiple tasks being
  performed, runtime stacks where tasks are
  executed, suspended, and resumed
• Supports distributed planning several PRS agents
  run asynchronously and communicate through
  message passing

36
Anytime Algorithms

• Time to deliberate about events varies
• Algorithms to compute the best answers they can
  in the time available
• Anytime algorithms
• Can be suspended and resumed with little overhead
• Can be terminated at any time and return some
  answer
• The answers returned improve with time

37
A time-dependent planning problem

• Observe (O)
• React (E) time required to carry out reaction of
  type E
• Herald (C) earliest observation time that
  enables prediction of condition C requiring a
  response
• Utility (C,E) utility of reacting to with E to C
• Response (C) time between having information to
  predict C and C occurring

38
When Time is Short

• Prediction time time required to predict event
  given info available
• Deliberation time max time for committing to a
  reaction (if reaction is needed)
• Reaction time time required to react to event
• React(E) Response(C)

39
Deliberation

• Decision procedure D for each C given t time to
  deliberate, D returns best guess E about how to
  react
• Utility(C, D(C,T))
• Deliberation scheduling
• Given several deliberation procedures, determine
  how to best allocate deliberation time

40
Utility versus time
Linear improvement, bounded utility
One-shot improvement
Linear improvement, unbounded utility
Diminishing returns
41
Other Approaches and Issues

• Blackboard architectures (Guardian)
• Universal plans
• Related issues covered in the course
• Reasoning about uncertainty
• Learning
• from the environment
• Becoming increasingly reactive

42
Summary

• Control systems
• Networks of variables (arcs) and functions
  (nodes)
• Reactive Action Packages (RAPs)
• Networks of conditions and tasks
• Task Control Architecture (TCA)
• Network arranged according to vertical
  capabilities
• Procedural Reasoning System (PRS)
• Integrates planning, BDI, and reactive techniques
• Anytime algorithms
• When time is short, managing what you think about
• Learning and uncertainty reasoning