Adaptive Coordinated Control of Intelligent Multi-Agent Teams

1
Adaptive Coordinated Control of Intelligent
Multi-Agent Teams

• Shankar Sastry, Ruzena Bajcsy, Peter Bartlett,
  Laurent El Ghaoui, Mike Jordan, Jitendra Malik,
  Stuart Russell, Pravin Varaiya (Berkeley)
• Vijay Kumar, Kostas Danillidis, Ali Jadbabaie,
  George Pappas, C. J. Taylor (Penn)
• Howie Choset, Alfred Rizzi, Chuck Thorpe (CMU)

2
Team

• Berkeley Bajcsy, Bartlett, El Ghaoui, Jordan,
  Malik, Russell, Sastry, Varaiya Drs. Shim,
  Geyer
• Penn Danillidis, Jadbabaie, Kumar, Pappas, Shi,
  Taylor
• CMU Choset, Rizzi
• Focus this review on students postdocs

3
Technology Challenges

1. The world and national security threats are
   different mobile operations in urban terrain,
   perimeter protection, convoy protection, anti
   terrorism operations.
2. Use of robotic and mixed initiative forces, the
   need for coordination of manned and unmanned
   forces
3. The need for dynamic strategies and tactics for
   dealing with a determined and flexible adversary.
4. Exploitation of the 3rd dimension by organic UAVs
   designed for use by individual dismounts

4
New Technical Innovations

• Control of the 3 D Digital battlefield need to
  use 3rd dimension, aerial forces, robotic and
  mixed initiative forces, untethered
  communications
• Adaptive Coordinated Control of Multiple Agents
  reconfiguration of teams dynamically in response
  to adversarial action
• Intelligent coordination of multiple agents
  ability to discover intent and reconfigure
  strategies adaptively
• Fusion of Action Perception Learning for
  humans embedded in the midst of automation
  Embedded Humans

5
Intellectual OrganizationThrust Areas

• Architecture Design for Adaptive, Dynamic
  Planning of air ground assets, robotic manned
  forces
• Integration of Rich Multi-Sensor Information into
  Virtual Environments incorporating human
  intervention
• Handling Uncertainty and Adversarial Intent in
  Adaptive Planning

6
Challenge Scenarios

• Reconaissance and Intelligence robotic ranger
  force for scouting fixed area for time critical
  targets (demos this afternoon).
• Mixed Initiative Engagement in urban environments
  using small UAVs. Live fire exercises at Ft.
  Hunter Liggett. Landing of A-160 and UCAR UAVs.
  Emphasis on sensor webs fixed and mobile.
• Recognition and Tracking of Unfriendlies
  emphasis on networked low bandwidth and high
  bandwidth sensors (cameras) for tracking. Demo at
  scale of 557 sensor nodes (tomorrows
  presentation)

7
Hierarchical Architectures for Dynamic Adaptive
Planning

• Progess to date in hierarchical architectures for
  decision making in normal modes of operation.
  Main emphasis is on replanning in fault or
  degraded modes of operation including
  deviations from hierarchical operation.
• Key technical issues
• Abstractions of Hybrid Systems for Architecture
  Design
• Hierarchical abstractions
• Assume-guarantee reasoning for abstractions

8
Thrust I continued

• Toolboxes for Design of Hybrid, Adaptive Control
  Systems
• Principled Embedded Systems Design for UAVs using
  time triggered architectures
• Model Predictive Controllers for adaptive
  obstacle avoidance
• Control of Hybrid Systems
• Numerical Solutions for Controller Synthesis
• Hierarchical Solutions of Synthesis Procedures
• Liveness and other acceptance conditions
• Controller Libraries
• Many world semantics and hierarchy semantics
• Modal decomposition

9
hyperA Useful Toolbox for Hybrid Control
Systems Design

• Shankar Sastry, Jonathan Sprinkle,
• Mike Eklund, Ian Mitchell

10
What Are Hybrid Systems?

• Dynamical systems with interacting continuous and
  discrete dynamics

11
Why are Hybrid Systems Hard?

• Zeno behavior lack of existence of solutions
• Lack of continuous dependence on initial
  conditions

Slide by Rafael Garcia
12
Tool Integration

• No one tool exists for all of these
• Several academic toolmakers created HSIF (Hybrid
  Systems Interchange Format)
• Hybrid Systems Interchange Format
• Failed to mature for a few reasons
• Tool-specific, and Tool-driven, not capability
  driven
• Not enough programming power behind it
• No thought to coverage of corner cases

13
Tool Integration
Example correct output

• What is needed is a framework that utilizes
  interchange format
• Must support what we know about hybrid systems
  semantics, encourage tools integration
• Implicit tool semantics makes fully meaningful
  translation impossible, or impractical
• The proper specification of the semantics of an
  interchange format would ease this difficulty
• Leverage HSIF as a learning tool for the semantic
  specification of the hyper core

RK 2
-
3 variable
-
step solver and
breakpoint solver determine
sample times
Note two values at
Note two values at
the same time
the same time
Incorrect output
Slide by Edward Lee
14
Example Simulation Tool HyVisual

• Ptolemy IIs HyVisual
• http//ptolemy.eecs.berkeley.edu/hyvisual/

Slide by Edward Lee
15
What we are doing Hyper Framework

• A new toolbox/toolsuite called hyper with the
  following characteristics
• High performance simulation
• High robustness factor
• High level modeling (with refinement)
• High number of interacting tools
• Provide a formal interchange between tools
• Low-level fundamental model specifications (a
  core)
• Requires a set of implementable functions to
  call
• Add a base package with interfaces for
  interoperability, and a lightweight editor
• Include industrial-strength solvers through
  transformations

16
Hyper Framework
Interoperability Interfaces
17
Hyper Framework

• Extensible to other tools
• Existing examples for integration through
  HyVisual/LSM
• A more focused, useful, core interchange format
• When integrated, allows persistence of legacy
  models in industry (Matlab/Simulink), now with
  advantage(s) of synthesis/verification
• Newer/faster tools can be tested against known
  true
• Check for same behavior
• Can be used for regression testing

18
UAV Research Test bed at Berkeley

• Architecture for multi-level rotorcraft UAVs
  1996- to date
• Pursuit-evasion games 2000- 2002
• Vision Based landing on pitching decks 2001- to
  date (transitioning to Army/Socom Maverick/A-160
  program, Oct 2005)
• Multi-target tracking 2001- to date (transitioned
  to Raytheon (shooter localization), Northrup
  (pipeline monitoring), Lockheed (ballistic
  missile defense), demo August 2005)
• Formation flying and formation change 2002- to
  date (transition begun to Socom, 160th SOAR,
  Sikorsky, United Technology)
• NMPC Based Acrobatic Flying, Conflict Resolution
  2003 (transitioned to DARPA/Army UCAR, December
  2004 and and Northrup Grumann UCAV-N Program, )
• Aerial Pursuit Evasion Games 2003 (transitioned
  to to Boeing UCAV program, demo at Edwards AFB,
  June 2004, transition to Army/DARPA UCAR, Feb
  2005)
• Sensor Webs (low bandwidth air dropped sensor
  webs demonstrated at China Lake, Feb 2004) now
  Smart Bird personal UAVs
• Personal back pack sized UAVs (Smart Bird)
  taskable through PDAs and cell phones, Convoy
  training, Ft. Hunter Liggett, April 05-ongoing

19
Berkeley BEAR Team Fleet Line-up
Ursa Minor 3 (1999March 2000) Basic
navigationcontrol system, algorithm, software
development and test platform
Ursa Major 1 (Nov. 2002 ) Low-cost, high-payload
platform Aggressive Maneuver, Vision-based
landing Multi-agent scenarios, Model-predictive
control
Ursa Magna 1,2 (June 1999present) Advanced
navigationcontrol algorithm development
platform Multi-agent scenarios, formation
flight, Vision-based landing
Ursa Maxima 2 (July 2000present) High-payload
platform for Multi-agent scenarios, formation
flight
20
Time Triggered Control system architecture

• GPS and INS write navigation data to a buffer
• Controller accesses and reads the buffer with
  10ms period
• Controller writes control outputs to servos with
  20ms period

21
Test Results Hovering and Cruising
22
Ursa Electra1
Length 1.8 m Width 0.39m Height 0.54m
Weight 7.8 kg Rotor Diameter 1.8m Lithium
Polymer Battery Pack 38V, 8000mAh Flight time 20
min System operation time 80 min
23
Transition to 160th SOAR and Ft. Rucker

• Technology to be transitioned Autonomous
  Helicopter Formation Flight
• Previous work
• Mesh stability and experimental results
• New suggestion Model Predictive Control
• MPC for heterogeneous formation
• Simulation results
• Implementation issues
• Communication
• Pilot/controller interaction
• Initiation/Termination of a formation
• Interrupted by hostile event

24
Integration of Multi-Sensor Information Into
Virtual Environments

• Adaptive Hierarchial Networks for Acquiring and
  providing information
• Networked sensors
• Bandwidth utilitzation
• Extraction of 3 D Models from Distributed Sensors
• 3 D models from video data
• Integration of real and virtual environments
• Environments for Human Intervention Decision
  Making
• Situational awareness
• Display of uncertain data
• Triaging of data for decision making

25
ACCLIMATE MURI Platform Test Organic Air
Vehicles and UAV Mobile Ground Station in Fort
Hunter-Liggett Live-Fire Convoy Training
Exercises April 4-9,2005 Perry Kavros, Travis
Pynn, Peter Ray and Shankar Sastry University of
California, Berkeley
26

• Research Context Embedded Intelligence
• Organic air vehicles (OAVs) provide the vital 3rd
  dimension in supporting the changing nature of
  combat
• Swarms controlled efficiently
• formation flying to reach targets
• change of formation in response to threats
• Conflict detection and resolution among OAVs
• Terrain avoidance and path planning to avoid
  threats
• determination of adversarys tactics based on
  geo-temporal situation
• Autonomous tasking of resource concentration
• autonomous negotiation of targets, logistics and
  reinforcement
• Communication by ad-hoc or peer-to-peer
  networking

27

• RESEARCH AGENDA Developing Trusted, Intelligent
  UAVs
• Goal Develop robust autonomous systems that
  react intelligently within the mission context,
  interact with other autonomous systems and human
  operators to achieve mission objectives
• Majority of currently fielded UAVs are
  teleoperated or fly preprogrammed missions
• Intelligent autonomy cannot be achieved without a
  complete understanding of the mission and
  warfighters needs
• human-centered design approach from warfighters
  perspective
• tests in a realistic environment
• Integration into warfighting tactics, techniques
  and procedures
• ACCLIMATE Platform Tests at Fort Hunter-Liggett
  Live-Fire Exercises Context
• UAV teleoperation from bunkers, vehicles along
  route or UAV mobile ground control station
• OAV terrain recon at 1000 ft and extremely low
  altitudes along route
• OAV monitor convoy route outside the surface
  danger zones

28
SMART BIRD Single-Man Aerial Reconnaisance
Tool Battlefield Information Recon Deployment UC
Berkeley, S. Shankar Sastry, PI
Single operator, stealth, back-pack size, hand
launch/recovery, modular, 48 electric-powered
wing, 2.5 lb UAV (1 lb payload), 2-mi. range,
loitering ability (day/night) without GPS,
autopilot (wing leveler altitude hold), 2.4 GHz
video/data downlink, requires no tools for
assembly/disassembly
29
The intelligence provided by tactical UAVs might
be multiplied considerably if they can flit about
undetected. Unweaving the Web
Deception and Adaptation in
Future Urban Operations (Rand 2003)
Altitude-reduced signature
Concealment in vegetation, behind terrain or in
ground clutter.
Vertical takeoff from the ground or out of
hiding into a near-ground hover for a quick look
30

• BEAR Organic UAVs for FHL LFX
• Electric JOKER
• Radio-controlled
• Manufactured by Minicopters
• (Vellmar, Germany)
• Stabilized by a Carvec flight control
• system
• Pan-tilt-zoom camera control
• 1.4 m main rotor diameter
• 2 kg payload
• 8.0 kg total weight
• 15 min maximum flight time
• Autonomous JOKER (Aug 2004)
• Electric SmartBAT
• Radio-controlled
• Hand-launched/hand-recovered
• Embedded camera, antenna/receiver
• 48-inch foam wing with EPP leading edge

31

• Electric Joker with flight and camera
  stabilization system
• Designed for reliable performance at high speed
  with aggressive aerial techniques
• Easy transport, launch, recovery, small
  signature, quiet
• Controlled from inside vehicle via video
  downlink to monitor

• BEAR autonomous Electric Joker
• Designed for perch and stare operations
• Autonomous launch and landing
• Waypoint navigation
• Vision-based landing (Dec 2005)

32
Smart Bird takes flight
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
Ft. Hunter-Liggett Live Fire Exercise command
staff and trainees critique the video from the
BEAR organic UAV.
37
Adaptive Coordinated Control in the Multi-Agent 
3D Dynamic Battlefield ADAPTIVE COORDINATED
CONTROL OFINTELLIGENT MULTI-AGENT TEAMS
(ACCLIMATE)
Vision Based Landing of UAVs

• Dr. Christopher GeyerCarnegie Mellon
  University(formerly U.C. Berkeley)
• Todd Templeton, Marci Meingast, Mike Eklund,
  Prof. Shankar SastryU.C. Berkeley
• Supporting cast David Shim, Hoam Chung, Peter
  Ray, Travis Pynn
• November 1, 2005

38
UAV Landing Problem

• Focus was on detection How do you do obstacle
  detection gt500ft AGL?
• Greedy approach
• Explore terrain in spiral starting at Point F at
  500ft AGL until potential site found, descend to
  investigate
• Other possible modes
• E.g. constantly keep list of landing sites during
  flight

Aerial map of test area in Victorville, CA
39
3D Terrain from Parallax Results

• Terrain elevation and appearance recovered from
  flight simulated near Victorville, CA airport
• Path of vehicle super-imposed on map
• 5 meter average error at 1500m AGL

40
Uncertainty and Adversarial Intent

• Models of Uncertainty
• Environmental non deterministic and
  probabilistic
• Adversarial
• Guarantees of Success in the face of uncertainty
• Decision making in the presence of uncertainty
• Learning of Adversarial Strategy
• Probing strategies
• Games, partial information solution concepts
• Adaptation to changing utility functions of
  adversary

41
Moores Law 2x stuff per 1-2 yr
42
Bells Law new computer class per 10 years
log (people per computer)
streaming information to/from physical world

• Enabled by technological opportunities
• Smaller, more numerous and more intimately
  connected
• Ushers in a new kind of application
• Ultimately used in many ways not previously
  imagined

year
43
Instrumenting the world
Great Duck Island
Redwoods
Elder Care
Factories
Soil monitoring
44
The Sensor Network Challenge

• Monitoring Managing Spaces and Things

applications
Store
Comm.
uRobots actuate
MEMS sensing
Proc
Power
technology
Miniature, low-power connections to the physical
world
45
Traditional Systems

• Well established layers of abstractions
• Strict boundaries
• Ample resources
• Independent Applications at endpoints communicate
  pt-pt through routers
• Well attended

Application
Application
User
System
Network Stack
Transport
Threads
Network
Address Space
Data Link
Files
Physical Layer
Drivers
Routers
46
by comparison ...

• Highly Constrained resources
• processing, storage, bandwidth, power
• Applications spread over many small nodes
• self-organizing Collectives
• highly integrated with changing environment and
  network
• communication is fundamental
• Concurrency intensive in bursts
• streams of sensor data and network traffic
• Robust
• inaccessible, critical operation
• Unclear where the boundaries belong
• even HW/SW will move

47
Mote Evolution
48
Sensor Networks Testbed for Urban and Special
Force Operations Smart Dust, Dot Motes, MICA
Motes

• Dot motes, MICA motes and smart dust

49
PEG Software Overview

• New routing protocols to relax dependency on
  localization service
• Remote configuration interface
• Solution to a problem, not an original goal
• Network reprogramming
• System layer for remotely invoking disparate
  services
• Standard services
• Sleep, ping, RF power, blink, network reprogram
• MATLAB command-line interface to network
• Strong decoupling between sensor networks and
  clients

50
PEG Demo from July 03
51
Networked Personnel Detection Sensors

• Drop Experiment at 29 Palms, March 2001
• Bald Camel Experiments Feburary 17th, 2004, China
  Lake, Ca
• NEST Final Demo with 577 unattended ground
  sensors, August 2005

52
Last 2 of 6 motes are dropped from MAV, March 2001
53
Expendable Microradar Sensor Network Reports
Unauthorized Entry
Utilize the Worlds Smallest Radar to Detect
Adversary Penetration
TECHNOLOGY
OPERATIONAL CUEING
100 Sensor
GPS
Message Hopping Radio
Satellite Link
54
BALD CAMEL Overview
Goal End to end System demonstration of
networked personnel sensors

• Sensors Livermore micro-radars (wideband
  pulsed, 5.8 GHz) 20 m range
• Target Individuals, pack animals, vehicles
• Sensor field 50 sensors 2 data exfiltration
  nodes
• Network Peer to peer adhoc network running on
  TinyOS operating system
• Radio 900 MHz spread spectrum, 80m range
• Exfiltration Satcom data link over commercial
  system (Iridium) to Internet
• Packaging Polyurethane foam rocks air
  droppable self orienting antennas
• Localization GPS on every node patch antenna
• Drop tube 8 foot long, 9 in. tube - 100 lbs.
  max (Hellfire-C bomb rack)
• Re use existing arming and fuzing system
• Lifetime 10 days to 3 months
• Emplacement Predator, Helicoptor
• User Interface Powerscene (PredatorView) -
  simple upgrade to CAOC system. ADSI messaging.
• Price goal 300 / node
• Response lt2 seconds delay
• Participants Berkeley, AEPTech, Livermore,
  DARPA, Crossbow/Cambridge, Advanteca,
• MLB, EgLin

55
Applications

• 24/7 monitoring of trails and remote areas
• - Alert on any activity
• - Monitor high/low activity and direction of
  travel
• - Examples Guerilla activity Anticipate
  ambush, Drug Interdiction, Pipeline
  protection, border protection
• - Directly cue Predator sensor operator for
  target validation or satcom exfiltration
• Perimeter security
• - Detect lurkers outside perimeter
• - Detect infiltrators inside perimeter
• - Cue imager

56
Packaged Unit (April 2002)
Camouflage Packaging
57
BALD CAMEL UAV Deployed Ground Sensor
Dispensing SystemFeb 2004

• Design, Development and Flight Test of UAV
  Carriage/Dispensing System For BALD CAMEL Ground
  Sensor System
• Design Compatible With Predator and Hunter UAVs

UAV Carriage
Demonstration Test Helicopter (UAV Surrogate)
58
Sensors Being Dispensed
59
Ground Pattern
Dry Creek Bed
Road
60
Shooter Localization Using SensorWebs, November
2004 at Mc Kenna MOUT site
Berkeley motes and Vanderbilt algorithm
61
NEST Final Experiment MTT Demo, August 2005

• Goal
• Track an unknown number of multiple targets using
  a sensor network of binary sensors without
  classification information
• Coordinate multiple pursuers to chase and capture
  multiple evaders in minimum time using a sensor
  network
• Done in simulation due to physical and time
  constraints

62
NEST Final Experiment Summer 2005
63
NEST Final Experiment Sensor Node

• Telos B mote
• 8MHz TI MSP430 microcontroller
• RAM 10kB Flash 48kB
• Chipcon CC2420 Radio 250kbps, 2.4GHz, IEEE
  802.15.4 standard compliant
• Radio range of up to 125 meters

• Trio Sensor Board
• Features a microphone, a piezoelectric buzzer,
  x-y axis magnetometers, and four passive infrared
  (PIR) motion sensors
• Solar-power charging circuitry

Trio Node
64
NEST Final Experiment System

• Software
• TinyOS
• Deluge
• Network reprogramming
• Drip and Drain (Routing Layer)
• Drip disseminate commands
• Drain collect data
• DetectionEvent
• Multi-moded event generator
• Multi-sensor fusion and multiple-target tracking
  algorithms

65
Indra Camera Network Testbed

• Cory hall 3rd floor
• DVR on 1st floor operations room
• 4 omnicams/12 perspective cameras

directional
omni
66
Distributed Tracking, May 2005

• Issues
• Distributed multiple-target tracking and identity
  management Oh, Hwang, Roy, Sastry, 2005
• Representation (low communication cost, high
  performance)

67
Technical Cooperation

• Army Labs and Sites
• ARL (POC Emmerman, Kolodny)
• 160th SOAR, Ft. Campbell
• SOCOM, Chris Whitaker
• Fort Benning, Ga MOUT site
• Other Government Labs
• AFRL Wright Patterson (Banda, Bortner, Koenig)
• SOCOM (Secunda)
• USMC bases, Quantico, 29 Palms (POC Col. (retd)
  Kiers, Brig Gen (retd.) Holcomb),
• NASA (Meyer, Tobias)

68
Technology Transfer

• Industrial Partners
• Honeywell, Minneapolis (Datta Godbole, Tariq
  Samad)
• Boeing Phantom Works, St. Louis (Dave Corman, Jim
  Paunicka, Jared Rossom)
• Northrup Grumann, Los Angeles (Robert Miller,
  Omid Shakernia)
• Lockheed Missiles Space, Palo Alto (Jim Ryder,
  Prasanta Bose)
• Raytheon, Fairfax (Bob Berzedevin)
• Sikorsky (Clas Jacobsen, Mihai Huzmezan)
• Aerospace Corporation (Kirstie Bellman)

69
Third Year Review Program November 1st , 2005

• Thrust I Architectures for Multi-Vehicle
  Collaboration 955-1135 am
• Pappas, Overview 30 minutes
• Agung 20 minutes
• Chitta 20 minutes
• Thrust II Multi-Media Environments for aiding
  decision making 1130 300 pm (including
  working lunch)
• Kumar Overview 30 minutes
• Taylor 25 minutes
• Cowley 15 minutes
• Geyer 30 minutes
• Daniilidis, 30 minutes
• Thrust I and II Demonstrations and Posters 310
  530 pm

70
Third Year Review Program November 2nd

• Thrust III. Learning and Adaptation in the
  Presence of Uncertainty June 8th 900-1130
• Sastry and Oh, Overview 40 minutes
• Shim 30 minutes
• Chung 30 minutes
• Ahammad and Meingast 30 minutes
• Government Caucus 1125 to 1 pm.
• Wrap Up and Feedback 100 130 pm.

71
Backups