ONR July 21, 1998

1
ONR UCAV Project Overview
Exploration of Hybrid and Intelligent Control
Architectures in Conjunction with Probabilistic
Verification

• S. Shankar Sastry
• July 21, 1998
• Electronics Research Laboratory
• University of California, Berkeley

2
Problem Design of Intelligent Control
Architectures for Distributed Multi-Agent Systems

• An architecture design problem for a distributed
  system begins with specified safety and
  efficiency objectives for each of the system
  missions (surveillance, reconnaissance, combat,
  transport) and aims to characterize control,
  observation and communication.
• Mission decomposition among different agents
• Task decomposition for each agent
• Inter-agent and agentmother ship coordination
• Continuous control and mode switching logic for
  each agent
• Fault management
• This research attempts to develop fundamental
  techniques, theoretical understanding and
  software tools for distributed intelligent
  control architectures with UCAV as an example.

3
Fundamental Issues for Multi-Agent Systems

• Central control paradigm breaks down when dealing
  with distributed multi-agent systems
• Complexity of communication, real-time
  performance
• Risk of single point failure
• Completely decentralized control
• Has the potential to increase safety, reliability
  and speed of response
• But lacks optimality and presents difficulty in
  mission and task decomposition
• Real-world environments
• Complex, spatially extended, dynamic, stochastic
  and largely unknown
• We propose a hierarchical perception and control
  architecture
• Fusion of the central control paradigm with
  autonomous intelligent systems
• Hierarchical or modular design to manage
  complexity
• Inter-agent and agentship coordination to
  achieve global performance
• Robust, adaptive and fault tolerant hybrid
  control design and verification
• Vision-based control and navigation

4
Autonomous Control of Uninhabited Combat Air
Vehicles

• UCAV missions
• Surveillance, reconnaissance, combat, transport
• Problem characteristics
• Each UCAV must switch between different modes of
  operation
• Take-off, landing, hover, terrain following,
  target tracking, etc.
• Normal and faulted operation
• Individual UCAVs must coordinate with each other
  and with the mothership
• For safe and efficient execution of system-level
  tasks surveillance, combat
• For fault identification and reconfiguration
• Autonomous surveillance, navigation and target
  tracking requires feedback coupling between
  hierarchies of observation and control

5
Research Objectives Design and Evaluation of
Intelligent Control Architectures for Multi-agent
Systems such as UCAVs

• Research Thrusts
• Intelligent control architectures for
  coordinating multi-agent systems
• Decentralization for safety, reliability and
  speed of response
• Centralization for optimality
• Minimal coordination design
• Verification and design tools for intelligent
  control architectures
• Hybrid system synthesis and verification
  (deterministic and probabilistic)
• Perception and action hierarchies for
  vision-based control and navigation
• Hierarchical aggregation, wide-area surveillance,
  low-level perception
• Experimental Testbed
• Control of multiple coordinated semi-autonomous
  DV8 helicopters

6
Methods
Methods

• Semi-Formal Methods
• Architecture design for distributed autonomous
  multi-agent systems
• Hybrid simulation
• Structural and parametric learning
• Real-time code generation
• Modularity to manage
• Complexity
• Scalability
• Expansion

• Formal Methods
• Hybrid systems (continuous and discrete event
  systems)
• Modeling
• Verification
• Synthesis
• Probabilistic verification
• Vision-based control

7
Thrust 1 Intelligent Control Architectures
Research Thrust 1 Intelligent Control
Architectures

• Coordinated multi-agent system
• Missions for the overall system surveillance,
  combat, transportation
• Limited centralized control
• Individual agents implement individually optimal
  (linear, nonlinear, robust, adaptive) controllers
  and coordinate with others to obtain global
  information, execute global plan for
  surveillance/combat, and avoid conflicts
• Mobile communication and coordination systems
• Time-driven for dynamic positioning and stability
• Event-driven for maneuverability and agility
• Research issues
• Intrinsic models
• Supervisory control of discrete event systems
• Hybrid system formalism

8
Decentralized Observation, Communication and
Control for Multi-Agent Systems

• Given a strategic objective and local observation
• What are the required information protocols with
  Human-centered system and other autonomous agents
  to command tactical control?
• Given a distributed control problem and the local
  observation at each site, what is the inter-site
  communication (minimal) or coordination protocols
  required to solve this problem?
• Given a cooperative mission
• What is the strategic objective (possibly
  dynamic) of each autonomous agent?
• How to distribute among the available agents a
  specified centralized control problem?

9
Decentralized Observation and Communication for
Discrete Event Systems
Agent Communication Channels
A1
A2
A3
Plant (Lp)

• The agents have partial observation but can
  exchange messages.
• The plant has a set of unobservable
  distinguished events (failures).
• OBJECTIVE Design the inter-agent communication
  scheme required
• to detect and isolate the distinguished events

10
Synthesis of Inter-agent Communication for
Decentralized Observation

• Theorem 1 (Lp, áåpoi, åfiñiÎI) is decentrally
  diagnosable if there exists
• n Î N such that for all sf Î åf, usfv Î Lp, v
  n, implies
• (w Î Lp) Ù ( i, Påpoi (w) Påpoi (usfv)) Þ (sf
  Î w).
• If any two sufficiently long plant traces look
  the same to all the agents, then either they have
  no failures or have all the same failures.
• Synthesis The communicate all plant
  observations solution works.
• General drawback Redundant information is
  communicated.
• L(f) may not be regular even though Lp is
  regular.
• Current focus Minimal communication, protocol
  synthesis, trace abstraction
• Documentation Draft paper available and sent to
  WODES98

11
Decentralized Control of Discrete Event
SystemsProblem Formulation

• Each agent has a set of controllable events
• Controllable events are a subset of the set of
  observable events
• The next event is either an uncontrollable event
  from the plant, a controllable event enabled by
  an agent, or a message event scheduled by an
  agent
• A control objective is specified by a language
• Investigate the existence and synthesis of
  coordination protocols

12
Communication and Control Synthesis for DES models

• Advantages
• Will synthesize symbolic, event-driven,
  inter-agent communication over a finite message
  set
• Very simple models permitting logical or
  combinatorial analysis and insights
• AHS example Worked for most coordinating
  maneuvers other than stability properties for
  vehicle following
• Limitation No formal way to capture continuous
  dynamics
• The semantics of an event is generally some
  alignment or safety conditions in velocity,
  position, and euler angles with respect to
  targets or other agents
• SOLUTION Distributed control of hybrid systems
  systems

13
Intelligent Control Architecture
UCAV Control Architecture

• Mission Planning
• Resource Allocation

Mission Control
Strategic Objective

• Generating Trajectory
• Constraints
• Fault Management

Strategic Layer
Inter-UCAV Coordination
Trajectory Constraints

• Flight Mode Switching
• Trajectory Planning

Sensor Info on Targets, UCAVs
Tactical Layer
Replan
Trajectory

• Trajectory Tracking
• Set Point Control

Regulation Layer
Environmental Sensors
Actuator Commands
Tracking errors
UCAV Dynamics
14
Preliminary Control Architecture for Coordinating
UCAVs

• Regulation Layer (fully autonomous)
• Control of UCAV actuators in different modes
  stabilization and tracking
• Tactical Layer (fully autonomous)
• Safe and efficient trajectory generation, mode
  switching
• Strategic Layer (semi-autonomous)
• Generating trajectory constraints and influencing
  the tasks of other agents using UCAV-UCAV and
  UCAV-ship coordination for efficient
• Navigation, surveillance, conflict avoidance
• Fault management
• Weapons configuration
• Mission Control Layer (centralized)
• Mission planning, resource allocation, mission
  optimization, mission emergency response, pilot
  interface

15
Thrust 2 Verification and Design Tools
Research Thrust 2 Verification and Design Tools

• The conceptual underpinning for intelligent
  multi-agent systems is the ability to verify
  sensory-motor hierarchies perform as expected
• Difficulties with existing approaches
• Model checking approaches (algorithms) grow
  rapidly in computational complexity
• Deductive approaches are ad-hoc
• We are developing hybrid control synthesis
  approaches that solve the problem of verification
  by deriving pre-verified hybrid system.
• These algorithms are based on game-theory, hence
  worst-case safety criterion
• We are in the process of relaxing them to
  probabilistic specifications.

16
Thrust 2 Verification and Design Tools
Hybrid Control Synthesis and Verification

• Approach
• The heart of the approach is not to verify that
  every run of the hybrid system satisfies certain
  safety or liveness parameters, rather to ensure
  critical properties are satisfied with a certain
  safety critical probability
• Design Mode Verification (switching laws)
• To avoid unstable or unsafe states caused by mode
  switching (takeoff, hover, land, etc.)
• Faulted Mode Verification (detection and
  handling)
• To maintain integrity and safety, and ensure
  gradual degraded performance
• Probabilistic Verification (worst case vs. the
  mean behavior)
• To soften the verification of hybrid systems by
  rapprochement between Markov and Bayesian
  decision networks

17
Controller Synthesis for Hybrid Systems

• The key problem in the design of multi-modal or
  multi-agent hybrid control systems is a synthesis
  procedure.
• Our approach to controller synthesis is in the
  spirit of controller synthesis for automata as
  well as continuous robust controller synthesis.
  It is based on the notion of a game theoretic
  approach to hybrid control design.
• Synthesis procedure involves solution of Hamilton
  Jacobi equations for computation of safe sets.
• The systems that we apply the procedure to may be
  proven to be at best semi-decidable, but
  approximation procedures apply.

18
Thrust 3 Perception and Action Hierarchies
Research Thrust 3 Perception and Action
Hierarchies

• Design a perception and action hierarchy centered
  around the vision sensor to support surveillance,
  observation, and control functions
• Hierarchical vision for planning at different
  levels of control hierarchy
• Strategic or situational 3D scene description,
  tactical target recognition, tracking, and
  assessment, and guiding motor actions
• Control around the vision sensor
• Visual servoing and tracking, landing on moving
  platforms

19
What Vision Can Do for Control

• Global situation scene description and assessment
  
• Estimating the 3D geometry of the scene, object
  and target locations, behavior of the objects
• Allows looking ahead in planning, anticipation of
  future events
• Provides additional information for multi-agent
  interaction
• Tactical target recognition and tracking
• Using model-based recognition to identify targets
  and objects, estimating the motion of these
  objects
• Allows greater flexibility and accuracy in
  tactical missions
• Provides the focus of attention in situation
  planning

20
Relation between Control and Vision
Higher level
Task decomposition for each agent
Inter-agent, agentmother ship coordination
Lower level

• Higher level visual processing precise, global
  information, computational intensive
• Lower level visual processing local information,
  fast, higher ambiguity

21
Key Issues in Vision and Control Deliver the
Right Information at the Right Time

• How to coordinate the planning stage with sensing
  stage
• The planner should adjust to the speed and
  uncertainty of the vision system
• The vision system should optimize its information
  flow from the lower level to the higher level,
  given the need of the planner
• How to adjust the focus of attention
• Selecting attention of visual processing in terms
  of the object locations, as well as level of
  abstraction
• Fine tuning lower-level vision-motor control loop
• A well-designed lower-level vision-motor control
  alleviates computation requirements of
  higher-level visual processing

22
Approach for Hierarchical Vision Processing

• Use grouping to extract a compact description of
  the scene from lower processing
• Reduces the computation complexity of
  higher-level reasoning, provides a basis for
  attention selection
• Information estimated from big picture of the
  scene is less likely to be affected by noise in
  the sensor
• Efficient computation algorithm which is able to
  capture the big picture of a scene has been
  developed
• General results reported in CVPR97, results on
  motion reported in ICCV98

23
Approach for Hierarchical Vision Processing

• Applying higher-level reasoning on the groups
  extracted
• Model-based object recognition
• Matching image groups to object models
• 3D scene geometry estimation
• Based on the motion correspondence found
• Tracking and behavior analysis of objects
• Applying Bayesian theory in selecting the right
  level of visual processing

24
Approach for Lower-level VisionMotor Control

• Vision-guided motor control
• Use low-level image, motion flow information in
  formulating motor control law
• Tracking in the 3D coordinates
• Use optical flow equations to build a model of
  the scene in 3D space
• Look-ahead control law to allow for visual
  processing time
• Tracking in the image plane (2D)
• Track objects (such as the landing pad) in image
  frame
• Relate image measurement (such as image location
  of the pad, curvature of the lane marker) to
  motor control law

25
Research Contributions

• Fundamental Research Contributions
• Design of hybrid control synthesis and
  verification tools that can be used for a wide
  range of real-time embedded systems
• Design of vision and control hierarchies for
  surveillance and navigation
• Hierarchical vision for planning at different
  levels of control hierarchy
• Control around the vision sensor
• Our multi-agent control architecture can be used
  for many applications
• ONR applications
• UCAVs, simulated battlefield environment,
  distributed command and control, automatic target
  recognition, decision support aids for
  human-centered systems, intelligent telemedical
  system
• General engineering applications
• Distributed communication systems, distributed
  power systems, air traffic management systems,
  intelligent vehicle highway systems, automotive
  control

26
Research Schedule
FY 98
FY 99
FY 00
O N D J F M A M J J A S
O N D J F M A M J J A S
O N D J F M A M J
Intelligent Control Architectures
Multi-Agent Decentralized Observation System
Performance Evaluation of UCAV Architecture
Preliminary UCAV Architecture
Final UCAV Architecture
Verification Tools
Software Tools for Synthesis and Verification
Probabilistic Verification Theory
Hybrid Control Synthesis Methods
Perception
Smart Aerobots 3D Simulation
Label Recognition C QNX Real-Time
Visual Situation Assessment
Terrain Following Control Scheme
Vision System for Autonomous Takeoff/Landing
Integrated System for Target Recognition and
Terrain Following for Multiple UCAVs
Public Tests
Cal Day Demo April 17
Robotic Helicopter Competition Aug 12-13,
Richland, WA
Cal Day Demo
UCAV Architecture Demo
Robotic Helicopter Competition
27
Deliverables

• Task Duration Deliverables
• Intelligent Control Architectures
• Specification Tools 7/97 - 7/98 software,
  technical reports
• Design Tools 7/97 - 9/99 software,
  technical reports
• Architecture Evaluation Environment 7/97 -
  7/00 software, technical reports
• UCAV Application 7/97 - 7/00 experiments,
  technical reports
• Verification Tools
• Design Mode Verification 7/97
  -12/98 software, technical reports
• Faulted Mode Verification 7/97 -
  9/99 software, technical reports
• Probabilistic Verification 9/97 - 9/99
  technical reports
• Perception
• Surveillance 7/97 - 9/99 software,
  experiments
• Hierarchical Vision 7/97 - 7/00 software,
  technical reports
• Visual Servoing 9/97 - 7/00 experiments,
  technical reports
• UCAV Application 7/97 - 7/00 experiments,
  technical reports

28
Measures of Program Success

• FY97-98
• Design of preliminary UCAV architeture
• Design of hybrid control synthesis methods
• Design of multi-agent decentralized observation
  system
• FY98-99
• Development of probabilistic verification theory
• Final UCAV architecture design
• Vision and control for terrain following,
  take-off and landing for single UCAV
• FY99-00
• Performance evaluation of UCAV architecture
• Integration of vision and control for multiple
  coordinating UCAVs
• Final version of the software tools for
• Hybrid control synthesis and verification, and
• Decentralized observation and control
• Demonstration of UCAV architecture using the
  helicopter testbed

29
FY97-98 Accomplishments

• Controller synthesis for hybrid systems.
• Developed algorithms and computational
  procedures for
• designing verified hybrid controllers optimizing
  multiple
• objectives
• Multi-agent decentralized observation problem.
• Designed inter-agent communication scheme to
  detect and
• isolate distinguished events in system dynamics
• SmartAerobots. 3D virtual environment
  simulation.
• Visualization tool for control schemes and
  vision
• algorithmsbuilt on top of a simulation based on
  mathematical
• models of helicopter dynamics
• Label recognition prototype in Matlab, then in
  C (QNX real-time)

30
Berkeley Team

• Name Role Tel E-mail
• Shankar Sastry Principal (510)
  642-7200 sastry_at_robotics.eecs.berkeley.edu
• Investigator [phone]
• [phone]
• Jitendra Malik Co-Principal (510)
  642-7597 malik_at_cs.berkeley.edu
• Investigator
• Datta Godbole Research [phone] godbole_at_rob
  otics.eecs.berkeley.edu
• Engineer [phone]
• John Lygeros Postdoc [phone] lygeros_at_robot
  ics.eecs.berkeley.edu
• Jianbao Shi Postdoc [phone] jshi_at_cs.berke
  ley.edu
• Omid Shakernia Graduate Student (510)
  643-2383 omids_at_robotics.eecs.berkeley.edu

31
Teaming and Interdependency

• Collaboration with Prof. Varaiya (Berekeley) in
  designing a hierarchical control architecture for
  coordinating UCAVs
• Collaboration with Prof. Russell (Berkeley) in
  developing probabilistic design and analysis
  tools
• Collaboration with Prof. Zadeh (Berkeley) on soft
  computing tools for control of UCAVs and mode
  transition methods for DV 8 developed using fuzzy
  control
• Collaboration with Prof. Speyer (UCLA) on fault
  detection and handling methods
• Collaboration with Prof. Morse (Yale) on
  vision-guided navigation
• Informal conversations with Prof. Anderson (ANU),
  Prof. Hyland (Michigan) and visit to Naval Post
  Graduate School
• Pending more formal collaborations with Profs.
  Narendra, Morse