Multiagent Negotiation for Constraint Satisfaction

1
Multiagent Negotiation for Constraint Satisfaction

• Costas Tsatsoulis
• Dept. of Electrical Engineering and Computer
  Science
• The University of Kansas

2
Problem Characterization

• Distributed constraint satisfaction (resource
  allocation) problems which are either
  over-constrained (no optimal solution exists) or
  time bound (optimal solution cannot be found in
  allocated time) or both
• Highly decentralized environments, where problem
  solving is performed on the local level making
  maximum use of local information
• Both resources and their consumers are subject to
  change, and have significant individual
  constraints on action.

3
Proposed Solution

• Bottom-up formation of problem solving coalitions
  using a multiagent system
• Allocation of resources done in a collaborative
  fashion
• Collaboration managed by negotiation
• Goal is to achieve just-in-time or
  good-enough, soon-enough solutions

4
Brief Introduction to Negotiation for Reasoning

• We use argument-based negotiation
• Assuming benign and collaborative agents, each
  agent attempts to convince others that it has a
  more urgent need for a resource than they do
• To do so an agent must present arguments to its
  negotiation partners
• If the arguments are believed and are stronger
  than the reasons an agent is using a resource,
  then it will free the resource of a percentage of
  it to the negotiating partner
• Resources may be surrendered completely,
  partially, or conditionally

5
Negotiation Strategies

• The parameters of a negotiation are its
  strategy.
• Parameters may be the types of information to
  exchange, the time spent negotiating, the
  importance an agent places on the tasks it is
  performing and the perceived need for resources,
  the number of agents it is willing to negotiate,
  with, and so on.
• Our innovation is the use of adaptive, dynamic
  negotiation strategies, which situates the
  negotiation in the current state of the world

6
(No Transcript)
7
Domain of Application

• Multiple target - multiple shooter problem

8
Sensor
Agent
9
Agent polls sensor for data at regular intervals
10
Send messages
Receive messages
Initiate Negotiations
Respond to Negotiations
Control the sensor
Read system data
Read self data
Request system resources
Perform problem solving
11
(No Transcript)
12
Instantiate negotiation neighborhood
Establish negotiation parameters
Instantiate profile
Initiate negotiations
13
Send messages
Receive messages
Initiate Negotiations
Respond to Negotiations
Read sensor data
Control the sensor
Read system data
Read self data
Request system resources
Perform problem solving
14
Send messages
Receive messages
Initiate Negotiations
Respond to Negotiations
Read sensor data
Control the sensor
Read system data
Read self data
Responding Agent
Request system resources
Perform problem solving
15
Reflective Programming Model

• Provide executing agents with a sophisticated,
  adaptable view of system state, resources, and
  competing computations
• Enable agents to monitor and control their own
  progress
• Agents can potentially select their own
  computational model
• Agents may adapt future behavior in response to
  past behavior on a system-level
• System-level resources become explicit and enter
  into the negotiation
• Soon-enough results are achieved

16
System Software Architecture
RTSS Scheduler Object
TAO
17
Scheduling

• RTSS Library API wraps RTSS Scheduler Object
  interface
• Ace ORB (TAO) Object based access
• Scheduler object
• Evaluates agent requests and responds
• Combines agent allocations into a new schedule
• Agents adjust their own scheduling constraints
• Currently CPU use pattern
• Other system resources in the future
• Negotiate with Scheduling object

18
Ping

• Enables agents to relate computational and
  temporal progress for each thread
• Proportional progress model (e.g. .2,.4,.6,.8)
• Agent code uses ping interface to
• Establish progress update points
• Query progress state
• Supports per-thread model with per process POSIX
  timers
• Per threads timers preferred

19
CPU Use

• Provides resource use information for all
  registered RT threads
• Gives each thread an agent and global view
• Can be used by threads in deciding how to request
  or adjust resource use
• Data Stream Kernel Interface based
• Extensible to other resources

20
Agent Profiles

• Agent has preference over aggregates of goods
• Agent has preference over satisfaction of goals
• Agent has preference over negotiation partners
• Parameterized preference profiles used to select
  among goals and goods
• Situation- and goal-dependent preferences
• Situation instantiates profiles, persuasion
  threshold, evaluation criteria
• Helps define which offers to accept, decline, or
  modify in negotiation

21
Instantiate Profile

• Profiles are parameterized structures that
  describe the preferences of an agent
• They are instantiated based on the world
  information the agent has, i.e. what it senses,
  what it knows about its own state, and what it
  knows about the system state
• The agent profile may indicate preferences of
  certain resources over others, preferences of
  negotiation partners, and preferences over task
  satisfaction

22
Profile
Profile Header
Agent A Profile
Profile Body
Resource Preference
If ?taskrecognition then be willing to spend
more power
If ?target-speedgt200 then ask system for smaller
and more frequent periods of execution
If ?powerlt10 then avoid ON/OFF switching
Negotiation Partner Preference
If ?time-of-daynight then prefer negotiating
with an IR sensor
If ?targettype-A and ?problt80 then prefer
negotiating with an RF sensor
23
Negotiating I. Basics

• An agent may negotiate with multiple partners
  concurrently
• An agent may be an initiator, a responder or both
• An agent estimates the future location of a
  target and when a target will enter another
  sensors lobe(s)
• This establishes who to negotiate with, when to
  start negotiating, and how long to negotiate
• Negotiation is argumentation-based.
  Argumentation is modeled as production rules in
  CLIPS

24
Negotiating II. Initiating a Negotiation Request

• Retrieve the best case to obtain a negotiation
  strategy
• Use utility theory to sort potential negotiation
  partners
• The utility is a weighted sum of three sets of
  attributes
• the time that the target will hit the coverage of
  a potential partner or the time that the target
  will leave the coverage of a potential partner
• the past experience of negotiations between the
  agent and a potential partner (have I been
  helpful to you? how many times have we had
  successful negotiations?)
• the current relationship between the agent and a
  potential partner (are we already negotiating
  about some other things? how much coverage
  overlap do we have between your sensor sectors
  and mine? have I already initiated a
  negotiation request to you?)
• For each potential partner, we obtain the most
  useful sector (highest utility) and sort the
  potential partners based on their most useful
  sectors.

25
Negotiating II. Initiating a Negotiation
Request (cont.)

• Create negotiation tasks
• Upload relevant data for each negotiation task
• Activate a negotiation thread for each
  negotiation task
• A request to negotiate consists of
• (Initiating-Agent Message-ID Request)

26
NegotiatingIII. Responding to a Request

• A request to negotiate consists of
• (Initiating-Agent Message-ID Request)
• If Request is to turn on a sector that is already
  on and measuring, then the agent agrees
• If there are no idle negotiation threads, then
  the agent declines to negotiate
• Otherwise, the agent decides to negotiate
• creates a negotiation task
• retrieves the best case to obtain a set of
  negotiation parameters
• uploads relevant data about self, other sensors
  and target
• activates a negotiation thread
• if the activation fails, the agent declines the
  request to negotiate

27
Case-Based NegotiationI. Situation Description

• tasks
• descriptions of what an agent is doing.
• partners
• describes the agents with whom an agent
  negotiates with.
• target model
• a description of the target which is being
  tracked or identified.
• constraints
• the limiting factors determining if a task can be
  accomplished.
• request
• the type of request TurnOnSectorAmpliture,
  TurnOnSectorFrequency, TurnOnSectorBoth,
  GiveUpCpuResource

28
Case-Based NegotiationII. Negotiation Strategy

• Initiator
• ranking of classes of information to send as part
  of the argument
• total time willing to spend
• total negotiation steps willing to engage in
• total cpu resources willing to allocate to the
  negotiation
• Responder
• total time willing to spend
• total negotiation steps willing to engage in
• cpu resources willing to allocate to the
  negotiation
• power required to satisfy the request
• function used in certain negotiations
  (exponential, linear)
• function parameters (kappa and beta)
• ranking of classes of information that it would
  send back to the initiator during
  negotiation/argumentation (so it defends its use
  of its resources)
• persuasion threshold used to determine if
  convinced to release or share a resource

29
Case-Based NegotiationIII. Retrieval

• Simple weighted similarity measures on the case
  features
• Best case selected - If more than one match with
  the exact same value we select the one with the
  best outcome
• Case solution contains the negotiation strategy
• Case also contains outcome of old negotiation
  (success, aborted, out-of-resources, rejected,
  channel-jammed, out-of-time)

30
Case-Based NegotiationIV. Adaptation

• Retrieved strategy is adapted to reflect
  differences in situations
• Two adaptation strategies
• 1. difference driven
• TimeAllowed and StepsAllowed modified based on
  the difference of TargetSpeed
• if old request was TurnOnSectorBoth and new
  request is TurnOnSectorAmplitude, lower
  persuasion threshold by 10
• 2. outcome driven
• if old outcome was Rejected then decrease
  TimeAllowed and StepsAllowed by 10
• if old outcome was OutOfTime then increase
  TimeAllowed and StepsAllowed by 15

31
Case-Based NegotiationV. Learning

• New negotiation strategies and their results are
  learned
• Before learning we cluster the case base to see
  if the new case should add new information and
  should be learned
• We have noticed that learned cases are used often
  in negotiation
• We have not yet studied the advantages -if any-
  of learning

32
NegotiatingIV. Negotiating by Argumentation

• Supply arguments to responder to convince (i.e.
  push argument above persuasion threshold)
• Use CLIPS production rules
• (defglobal ?evidenceSupport 0.0)
• (deftemplate self (slot _posX) (slot _posY)(slot
  _numOfTasks) )
• (defrule self-num-of-tasks
• (self (_numOfTasks ?x))
• gt
• (if ( ?x 1)then
• (bind ?evidenceSupport ( 0.1
  ?evidenceSupport))
• else
• (if ( ?x 2)then
• (bind ?evidenceSupport ( 0.2
  ?evidenceSupport)))))

33
NegotiatingIV. Negotiating by Argumentation
(cont)

• If initial argument is not sufficient, responding
  agent sends the initiator a MORE_INFO message
• Initiator sends the next ranked class(es) of
  arguments
• Process stops when
• persuasion threshold is exceeded SUCCESS
• total time or steps are exceeded by any
  participant OUT-OF-TIME
• no response is received to any message
  CHANNEL-JAMMED
• no more information can be supplied REJECTED
• low cpu resources OUT-OF-RESOURCES
• other catastrophic failure ABORTED

34
NegotiatingV. Using the RTSS for Negotiation

• During negotiation the agents need to
• know the exact time elapsed
• know the exact cpu used by various processes
  and threads
• be able to request new cpu allocation as the
  result of negotiation (give up or get cpu
  resources)
• The RTSS gives the agents this information

35
RESULTS
36
EXPERIMENTAL SET-UP
speed0.5ft/sec
37
(No Transcript)
38
(No Transcript)
39
Tracking Accuracy
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
ComparisonsNegotiation vs. No
NegotiationCase-Based vs. Static Strategy
45
Experiments

• No negotiation Each sensor senses and tracks
  independently. There is no communication between
  sensor agents
• Static negotiation strategy Agents negotiate
  using a pre-defined, static, good negotiation
  strategy

46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
Some Conclusions

• The No negotiation strategy sends almost 50
  more messages to the Tracker and almost 20 more
  total messages, yet has almost 50 worse tracking
  accuracy than negotiation strategies
• The static negotiation strategy sends almost 10
  fewer messages, but has a higher message cost
  than CBR because of excessive length of messages
• The static negotiation strategy has approximately
  35 worse tracking accuracy than CBR

51
Go to http//www.ittc.ukans.edu/ANTS for
details on software, methodology, presentations,
etc.