Workshop on High Performance, Fault-Adaptive Large Scale Real-Time Systems Vanderbilt University

1
Workshop on High Performance, Fault-Adaptive
Large Scale Real-Time SystemsVanderbilt
University

• The SRTA Agent Architecture as a Basis for
  Building Soft Real-Time Multi-Agent Systems
• Victor R. Lesser
• Computer Science Department
• University of Massachusetts, Amherst
• November 15, 2002

2
Acknowledgements

• Byran Horling
• Dr. Regis Vincent (SRI)
• Dr. Tom Wagner (Honeywell Research)
• URLhttp//mas.cs.umass.edu/bhorling/papers/02-14
  .ps.gz

3
Outline

• Background/Motivation
• EW Challenge Problem
• Approach to Soft Real-Time
• Approach to Building MAS
• SRTA Agent Architecture
• Experimental Evaluation
• Summary

4
Long Term Motivation

• Development of domain-independent techniques
  (toolkits) for coordinating the soft real-time
  activities of teams of cooperative agents?
• Ease the construction of complex, multi-agent
  applications that operate in a coherent manner
• Avoid reproducing for each application the
  complex reasoning involved in soft real-time
  control and in coordinating the activities of
  agents

5
DARPA EW Challenge Problem Distributed Sensor
Network

• Small 2D Doppler radar units
• Scan one of three 120? sectors at a time
• Commodity Processor associated with each radar
• Communicate short messages using radio
• Triangulate radars to do tracking

6
Approach to Soft Real-Time Design-to-Time

• Given a time bound, dynamically construct and
  execute a problem-solving procedure which will
  (probably) produce a reasonable answer with
  (approximately) the time available. (DAmbrosio)
• Involves elements of planning (deciding what to
  do) and scheduling (deciding when to perform
  particular actions).

7
TÆMS A Domain Independent Framework for
Modeling Agent Activities

• The top-level goals/objectives/abstract-tasks
  that an agent intends to achieve
• One or more of the possible ways that they could
  be achieved
• abstraction hierarchy (HTN) whose leaves are
  basic action instantiations, called methods
• A precise, quantitative definition of the
  performance (Qafs)
• solution quality, cost, time and resource usage.

8
Soft Real Time Control -- Different Paths for
Achieving Task -- BUILD PRODUCT OBJECTS

• Schedule A - Client has no resource limitations
  maximize quality
• Query-and-Extract-PC-Connection,
  Query-and-Extract-PC-Mall, Search-and-Process-ZDne
  t, Query-and-Process-Consumers-Report (Expected
  Q55.3,C2, D11.5)
• Schedule B - Client is interested in a free
  solution
• Query-and-Extract-PC-Connection,
  Query-and-Extract-PC-Mall, Search-and-Process-Zdne
  t (Expected Q33.2,C0, D8.4)
• Schedule C - Client request an even trade-off
  between quality, cost and duration
• Query-and-Extract-PC-Connection,
  Search-and-Process-Zdnet (Expected Q22.4,C0,
  D5.6)
• Schedule D - Client wishes to maximize quality
  while meeting a hard deadline of 7 minutes
• Query-and-Extract-PC-Mall, Query-and-Process-Consu
  mers-Report (Expected Q25.9,C2, D6)
• Examples of Schedules Produced by the
  Design-To-Criteria (DTC) Scheduler

9
Representing Coordination Patterns Among Agents
10
Approach to Decomposing A Problem into Agents

• Sophisticated/Highly Competent Agents
• Concurrent goals, goals are time and resource
  sensitive, goals have varying utilities
• Goals have alternative ways of being solved that
  produce differing levels of utility and consume
  differing amounts of resources
• Not all goals necessarily need to be solved
• (Sub)Goals spread across agents are
  interdependent
• Contention for scarce resources
• Contributing towards the solution of a
  higher-level goal
• Hard and soft constraints
• Sufficient computational/communication resources
  to do some reasoning about coordination
• Medium granularity domain tasks

11
What about simpler agents?

• Activities of simple,single-threaded agents
  become the goals of sophisticated agents with
  dedicated processing resources
• Sophisticated agents do the selection,
  multiplexing, scheduling, coordination and
  distribution of goals
• Contrast with O.S. doing the scheduling without
  context

12
Approach to Soft, Real-Time Distributed
Coordination/Resource Allocation

• Structured as a distributed optimization problem
  with a range of satisficing solutions
• Adaptable to available time and communication
  bandwidth
• Responsive to dynamics of environment
• Organizationally constrained range of agents
  and issues are limited
• Can be done at different levels of abstraction
• Does not require all issues to be resolved to be
  successful resource manager agents able to
  resolve some issues locally

13
Layered Agent Architecture
Problem Solver / Negotiation Soft Real Time
Architecture Java Agent Framework

• Domain analysis and goal formulation
• Organization-level resource allocation
• Agent-level resource allocation
• Constraint discovery and satisfaction
• Intra-agent organization and communication
• Environmental access points

14
JAF Java Agent Architecture

• Component-based agent design
• Attempt to maximize code reuse.

Execute
Control
Communicate
Pulse Actions
Scan Scheduler
Problem Solver
Execute
Control
Communicate
Resource Modeler
Directory Service
Log
Scheduler
State
Sensor
Observe

• Interfaces are hidden by JAF.
• Radsim/ RF communication/ sensor

15
SRTA Soft Real-Time Agent Architecture

• Facilitates creation of multi-resource management
  agents
• Basis of building complex virtual agent
  organizations
• Allows for abstract negotiation maps abstract
  assignment into detailed resource allocations
• Ability to resolve conflicts locally that are not
  resolved through negotiation
• These are key to building soft real-time
  distributed allocation policies

16
Soft Real-Time Control Architecture
Schedule Failure
Problem solver Periodic Task Controller
Negotiation (e.g. SPAM)
Commitments/ Decommitments
Goal Description/Objective
Update Expectations
TÆMS Library
Schedule failure/ Abstract view
Other Agents
TAEMS-Plan Network/Objective
Learning
Update Cache
Cache Check
Resource Modeler
Cache Hit
DTC-Planner
Resource Uses
Linear Plan
Schedule
Conflict Resolution Module
Partial Order Scheduler
Schedule Failure
Parallel Schedule
Multiple Structures
Results
Parallel Execution Module
Task Merging
17
Characteristics of Soft Real-Time Control
Architecture

• Operates at 50 to 100ms cycle time
• Written in JAVA except for Planner in C
• Uses domain-independent, quantitative
  representation of agent activities -- TÆMS
• Scheduling of multiple activities that have
  deadlines and are resource sensitive
• Can choose among alternative ways of achieving
  activities that trade off decreased utility for
  lower resource consumption
• Responds to uncertain conditions without the need
  for complete re-planning/scheduling of activities

18
Addressing Real Time Direct

• Direct technologies - making it possible
• DTC (Design-To-Criteria) planner
• TÆMS HTN for representing alternative plan
  options using quantitative information
• Create appropriate plan given time, resource
  costs and quality constraints
• Partial order scheduling creates loose
  schedules which can be quickly shifted to
  real-time constraints.
• Avoids constant re-planning
• Allows parallel execution and resource usage.
• Modeling of some meta-level activities (e.g.
  negotiation) permit more direct reasoning of time
  allocation.
• We do not model scheduling costs yet.
• Learning component discerns actual execution
  characteristics so future actions can be better
  modeled.

19
Addressing Real Time Indirect

• Indirect technologies - making it easier
• Periodic commitments reduce the need for
  re-negotiation.
• Scheduled caching reduces the need to call DTC
• Piecemeal addition and removal of tasks
  eliminates the need for constant dramatic
  rescheduling and re-planning.

20
Scheduling

• Partial-order Scheduler uses a sliding
  mechanism, coupled with a resource modeler, to
  quickly shift scheduled tasks.
• Action start time uncertainty - real time.
• Duration uncertainty.
• Commitments have a window of time in which the
  agent can perform them.
• Precise action scheduling is left to the
  discretion of the performing agent.

21
First task to achieve
enables3
enables2
Send Results
Track Low
Track Medium
Track High
lock
RF
Sensor
Deadline 3000
22
2 Other tasks to achieve
Send-Message 1
Send-tracking-Info
Enables4
Enables1
lock1
RF
Sensor
23
Reacting to Unexpected Changes
Set-Parameters
Track-Medium
Send-Results
Calibrate
Init
Send Msg
Send-Info-Tracking
Negotiate-Tracking
time
1000
1500
2000
2500
3000
3500
4000
500
Set-Parameters
Track-Medium
Send-Results
Calibrate
Init
Send Msg
Send-Info-Tracking
Negotiate-Tracking
time
1000
1500
2000
2500
3000
3500
4000
500
Set-Parameters
Send-Results
Track-Medium
Calibrate
Init
Send Msg
Negotiate-Tracking
Send-Info-Tracking
time
1000
1500
2000
2500
3000
3500
4000
500

• Analogous reactions also take place within the
  periodic task controller
• Slot-based scheduler used to facilitate
  repetitive actions

24
Meta-Level Costing

• Typical scheduling reasons about primitive
  actions.
• This only accounts for some percentage of the
  agents time.
• So called meta-level activities (e.g.
  negotiation, scheduling, planning) use
  significant resources but are usually not
  accounted for directly.
• Without accountability, these activities can
  interfere with the actions and commitments
  currently scheduled over.

25
Meta-Level Costing (contd)

• To completely reason about all the agents
  actions, we must
• Directly and Indirectly incorporate the
  activities in plans.
• Derive expected costs for these activities.
• Use this information when generating schedules.
• We are currently using a representation of
  negotiation activities in some our task
  structures
• Future goal is to more directly account for
  activities like planning and scheduling.

26
Reducing Scheduling Overhead

• Activity parallelism learning
• Anticipation of converting linear plan to
  parallel plan if resources are available
• Schedule caching
• Adjustable time granularity
• Responsiveness vs. meta-level overhead

27
Activity Parallelism Learning

• The plan is first scheduled
• The schedule is analyzed for parallel actions
• These are used to form sets of hints,
  associated with the current resource context
• A hint is actually just a mutual facilitates
  relationship
• These hints are later applied to the task
  structure before planning
• The facilitates tells DTC that if A is run first,
  B will have a duration of zero (or vice versa)
• This technique required no changes to DTC
• This can result in better plan selection.

28
Schedule Caching

• Agents in repetitive environments must frequently
  address goals which have previously been seen.
• Ex In sensor environment, Scan-Sector or
  Perform-Track-Measurement
• Results from prior planning can be reused
• A key is generated for each task structure
• Incorporates method names and expectations,
  interrelationships, normalized deadlines, etc.
• Works correctly with activity parallelism hints
• If results exist from a task structure with the
  same key, that plan is used instead of calling
  DTC
• DTC is an external C binary, requiring file
  reads and writes, so savings are significant

29
Adjustable Time Granularity

• The millisecond time line is divided into coarser
  increments.
• Does not significantly degrade the agents
  ability to meet wall clock time deadlines
• (assuming success within a reasonable grain size)
• It does decrease the number of rescheduling
  events which are needed
• Since the agent was already operating effectively
  at a coarse timeline, this technique has no new
  drawbacks.

Deadlines
Deadlines
Real Time
Real Time
1
2
3
4
5
Perceived Time
Perceived Time
Actions
Actions
û
û
û
û
ü
ü
30
Effects of Schedule Caching

• Periodic tasks and methods with deadlines are
  more achievable
• As a result, track updates increase

31
Effects of Changing Granularity
Rescheduling attempts decrease, reducing
overhead Methods with deadlines and periodic
tasks are satisfied more often 30-40 seems ideal
above that, the coarse granularity decreases
agent responsiveness
32
Summary

• SRTA architecture is a powerful tool for building
  soft real time agent organizations
• Sophisticated soft real time agent control is
  practical by exploiting a variety of mechanisms
• to speed up the planning, scheduling and
  rescheduling cycles

33
Future SRTA Work

• Continue to speed up architecture
• Simple Planner for time-critical situations
• Additional work on conflict-resolution strategies
• Meta-level control component to balance control
  and coordination costs and domain problem solving