Lecture VI: Collective Behavior of Multi-Agent Systems II: Intervention

1
Lecture VICollective Behavior of Multi-Agent
Systems II Intervention

• Zhixin Liu
• Complex Systems Research Center,
• Academy of Mathematics and Systems Sciences, CAS

2
In the last lecture, we talked about

• Collective Behavior of Multi-Agent Systems I
  Analysis

3
In the last lecture, we talked about

• Introduction
• Model Vicsek model

4
Multi-Agent System (MAS)
Autonomy capable of autonomous action
Interactions capable of interacting with other
agents

• MAS
• Many agents
• Local interactions between agents
• Collective behavior in the population level
• More is different.---Philp Anderson, 1972
• e.g., small-world, swarm intelligence, panic,
  phase transition, coordination, synchronization,
  consensus, clustering, aggregation,
• Examples
• Physical systems
• Biological systems
• Social and economic systems
• Engineering systems

5
Vicsek Model (T. Vicsek et al. , PRL, 1995)
http//angel.elte.hu/vicsek/
xi(t) position of agent i in the plane
6
Vicsek Model
http//angel.elte.hu/vicsek/
7
In the last lecture, we talked about

• Introduction
• Model
• Theoretical analysis
• Concluding remarks

8
The Linearized Vicsek Model

A. Jadbabaie , J. Lin, and S. Morse, IEEE Trans.
Auto. Control, 2003.
9
Theorem 2 (Jadbabaie et al. , 2003)
Joint connectivity of the neighbor graphs on each
time interval th, (t1)h with h gt0
Synchronization of the linearized Vicsek model
Related result J.N.Tsitsiklis, et al., IEEE
TAC, 1984
10
Random Framework

• Random initial states
• 1) The initial positions of all agents are
  uniformly and independently distributed in the
  unit square
• 2) The initial headings of all agents are
  uniformly and independently distributed in -?e,
  ?-e with e? (0, ?). The initial headings and
  positions are independent.

11
Theorem 7 High Density Implies Synchronization

• For any given system parameters
• and when the number of agnets n
• is large, the Vicsek model will synchronize
  almost surely.

This theorem is consistent with the simulation
result.
12
Theorem 8 High density with short distance
interaction
Let
and the velocity satisfy Then
for large population, the MAS will synchronize
almost surely.
13
Three Categories of Research on Collective
Behavior
14
Three Categories of Research on Collective
Behavior

• Analysis
• Given the local rules of the agents, what is the
  collective behavior of the overall system ?
  
  (Bottom Up)
• Design
• Given the desired collective behavior, what are
  the local rules for agents ?
  
  (Top Down)
• Intervention
• Given the local rule of the agents, how we
  intervene the collective behavior?

J.Han, M.Li, L.guo, JSSC,2006
15
Example 1 Synchronization
Q Under what conditions such a system can reach
consensus?
16
Example 2 Escape Panic
D. Helbing, et al., Nature, Vol. 407, 2000
Fire, panic
Normal, no panic
17
Three Categories of Research on Collective
Behaviors

• Analysis
• Given the local rules of the agents, what is the
  collective behavior of the overall system ?
  
  (Bottom Up)
• Design
• Given the desired collective behavior, what are
  the local rules for agents ?
  
  (Top Down)
• Intervention
• Given the local rule of the agents, how we
  intervene the collective behavior?

J.Han, M.Li, L.guo, JSSC,2006
18
Example 1 Formation control

• How we design the control law of each plane to
  maintain the form ?

19
Example 2 Swarm Intelligence
(Marco Dorigo et al., 2001-2004)
www.answers.com/topic/s-bot-mobile-robot
20
Example 3Distributed Control in Boid Model

• Each agent is described by a double integrator
  (Newton's second law of motion )

where xi, vi and ui represent the position,
velocity and the control input of the agent i.
Goal 1) Avoid collision 2) Alignment
3) Cohension
What information can be used to design the
controller? The position and velocity of neighbors
R. Olfati-Saber, IEEE Trans. Auto. Control ,2006.
21
Algorithm

• Controller design

where Aaij(q) is the adjacency matrix,
() is the action function,
iss-norm, and
Neighbor graph

• Theorem 1
• If the neighbor graphs are connected at each time
  instant. Then
• The group will form cohesion.
• All agents asymptotically move with the same
  velocity.
• No interagent collisions occur.

22
Three Categories of Research on Collective
Behaviors

• Analysis
• Given the local rules of the agents, what is the
  collective behavior of the overall system ?
  
  (Bottom Up)
• Design
• Given the desired collective behavior, what are
  the local rules for agents ?
  
  (Top Down)
• Intervention
• Given the local rule of the agents, how we
  intervene the collective behavior?

J.Han, M.Li, L.guo, JSSC,2006
23
Intervention
Example 1 Can we guide the birds flight if
we know how they fly ?
24
Example 2 Leadership by Numbers
Couzin, et al., Nature, Vol. 433, 2005
The larger the group is, the smaller the leaders
are needed.
25
Example 3 CockroachJ.Halloy, et al., Science,
November 2007
26
III. Intervention Given the
local rule of the agents, how
we intervene the collective behavior?

• The current control theory can not be applied
  directly, because
• It is a many-body self-organized system.
• The purpose of control aims to collective
  behavior.
• Not allowed to change the local rules of the
  existing agents
• Distributed Control special task of formation,
• Pinning Control Networked system, imposed
  controllers on selected nodes

27

• Intervention Via
• Soft Control

28
Soft Control

• The multi-agent system
• Many agents
• Each agent follows the local rules Autonomous,
  distributed
• Agents are connected, the local effect will
  affect the whole.

From Jing Hans PPT
29
Soft Control
an associate of a person selling goods or
services or a political group, who pretends no
association to the seller/group and assumes the
air of an enthusiastic customer.

• The Control
• No global parameter to adjust
• Not to change the local rule of the existing
  agents
• Put a few shill agents to guide (seduce)
• Shill is controlled by us, not following the
  local rules,
• is treated as an ordinary agent by
  other ordinary agents
• The power of shill seems limited The control
  is soft and seems weak

From Jing Hans PPT
30
Soft Control

• Key points
• Different from distributed control approach.
  Intervention to the distributed system
• Not to change the local rule of the existing
  agents
• Add one (or a few) special agent called shill
  based on the system state information, to
  intervene the collective behavior
• The shill is controlled by us, but is treated
  as an ordinary agent by all other agents.
• Shill is not leader, not leader-follower type.
• Feedback intervention by shill(s).

This page is very important!
From Jing Hans PPT
31
There Are Lots of Questions

• What is the purpose/task of control here?
• Synchronization/consensus
• Group connected / Dissolve a group
• Turning (Minimal Circling)
• Lead to a destination (in a shortest time)
• Avoid hitting an object
• Tracking
• In what degree we can control the shill?
  (heading, position, speed, )
• How much information the shill can observe ?
  (positions, headings, )

From Jing Hans PPT
32
A Case Study

• Problem statement
• System A group of n agents with initial headings
  ?i(0)?0, ?)
• Goal all agents move to the direction of ?
  eventually.
• Soft control
• Design one shill agent based on the agents
  state information.
• Assumptions
• The local rule about the ordinary agents is known
• The position x0(t) and heading ?0(t) of the spy
  can be controlled at any time step t
• The state information (headings and positions) of
  all ordinary agents are observable at any time
  step

From Jing Hans PPT
33
Vicsek Model
http//angel.elte.hu/vicsek/
34
A Case Study

• Problem statement
• System A group of n agents with initial headings
  ?i(0)?0, ?)
• Goal all agents move to the direction of ?
  eventually.
• Soft control
• Design one shill agent based on the agents
  state information.
• Assumptions
• The local rule about the ordinary agents is known
• The position x0(t) and heading ?0(t) of the shill
  can be controlled at any time step t
• The state information (headings and positions) of
  all ordinary agents are observable at any time
  step

From Jing Hans PPT
35
The Control Law u?
Control the Shill agent
From Jing Hans PPT
36
Control the Shill agent
Theorem 4 For any initial headings and
positions ? i(0)?0, ?), xi(0)?R2, 1? i ? n, the
update rule and the control law uß will lead to
the asymptotic synchronization of the group.
It is possible to control the collective behavior
of a group of agents by a shill.
J.Han, M.Li, L.guo, JSSC,2006
37
Simulation
38
An Alternative Control Law
otherwise
where
Result The control law ut will also lead to
asymptotic synchronization of the group.
39
Simulations
Switching between u? and ur
Control Law u?
40
Remarks on Soft Control

• It is not just for the above model
• Can be applied to other MAS ,e.g.,
• Panic in Crowd
• Evolution of Language
• Multi-player Game
• Add the special agent(s) is just one way
  Should be other ways for different systems
• Remove agents
• Put obstacle

We need a theory for Soft Control !
From Jing Hans PPT
41

• Intervention Via
• Leader-Follower Model (LFM)

42
Example 1 Leadership by Numbers
Couzin, et al., Nature, Vol. 433, 2005
The larger the group is, the smaller the leaders
are needed.
43
Leader-Follower Model

• Problem statement
• System
• A group of n agents
• Goal
• All agents move with the expected
  direction eventually.
• Intervention by leaders
• Add some information agents-called
  leaders, which move with the expected direction.

44
Leader-Follower Model

• Key points
• Not to change the local rule of the existing
  agents.
• Add some (usually not very few) information
  agents called leaders, to control or
  intervene the MAS But the existing agents
  treated them as ordinary agents.
• The proportion of the leaders is controlled by us
  (If the number of leaders is small, then
  connectivity may not be guaranteed).
• Open-loop intervention by leaders.

45
Mathematical Model
Ordinary agents (labeled by 1,2,,n)
Neighbors
Position
Heading
46
Simulation Example
N1000
47

• Q How many leaders are required for
  consensus/synchronization?

48
Assumption on the initial states
Random Framework

• 1) The initial positions of all agents are
  independently and uniformly distributed in the
  unit square.
• 2) The initial headings of the agents are
  uniformly and independently distributed in -p,
  p), and the initial headings of the leaders are
  . The headings and the positions are mutually
  independent.

49
Some Notations
50
Some Notations (cont.)
Laplacian L(0)D(0) A(0)
Normalized Laplacian
Spectrum
Spectral gap
where
51
Key Steps in the Analysis of the LFM

• Analysis of the system dynamics
• Estimation of the rate of consensus
• Dealing with the matrices with increasing
  dimension
• Dealing with the inherent nonlinearity

52
Analysis of the System Dynamics

• Evolution of the distance

Lemma 1 For any two agents i and j, their
distance satisfy the following inequality
where
53
Analysis of the System Dynamics

• Evolution of the headings

• Step 1 Projection

54
Analysis of the System Dynamics

• Step 2 Analyze the stability of

Step 3 Dealing with the changing neighbor graphs
55
Estimation of Consensus Rate
The consensus rate depends on

• 1) A key lemma For any vector ff1,f2,,fnt,
  we have

2)
56
Dealing with the Matrices with Increasing
Dimension
Estimation of multi-array martingales

where
Moreover, if
then we have
57
Dealing with the Matrices with Increasing
Dimension
Using the above corollary, we have for large n
where
58
The Degree of The Initial Graph
Lemma For initial graph G0, we have for large n
59
The Degree of The Initial Graph
Corollary
60
Dealing with the Inherent Nonlinearity

• Proposition 1
• For any positive v and r, we have for large n

where
61
Main Result

• Theorem 5
• Let the velocity v gt 0 and radius r gt 0 be
  positive constants. If the proportion of the
  leaders satisfies
• where C is a constant depending on v and r,
  then the headings of all agents will converge to
  almost surely when the population size n is
  large enough.

62

• Concluding Remarks

• In this talk, we talked about intervention to the
  multi-agent systems
• Soft control
• Design the control law of the shill
• Leader-follower model
• Control the number of the leaders

63

• Concluding Remarks

These two lectures mainly focus on the collective
behavior of the MAS. In the next lecture, we
will talk about game theory.
64
Thank you!