Agents

1
Agents their Societies

• Monojit Choudhury
• Dept. of Computer Science Engg.
• Indian Institute of Technology
• Kharagpur, INDIA

2
Is AI IA ?
Artificial Intelligence
Intelligent Systems
Intelligent Agents
Machine Learning
Searching
Neural Nets
Planning
Statistical Methods
Fuzzy Logic
Frames Nets
CBR RBS
3
What are Agents?

• An agent is a computer system that is situated
  in some environment, and that is capable of
  autonomous action in this environment in order to
  meet its design objectives.

4
Examples of Agents

• Any control system like AC
• Too cold ? heating on
• Too hot ? cooling on
• Temperature OK ? do nothing
• Most software daemons
• Background processes in UNIX
• X Windows program xbiff

5
What are Intelligent Agents?

• What is Intelligence?
• An ideal rational agent chooses actions to
  maximize its performance based on its percepts,
  knowledge, and capabilities
• Reactivity
• Pro-activeness
• Social ability
• Learning(?)

6
Examples of IA

• Real-life IAs
• Travel Agents
• Brokers
• Artificial IAs
• E-mail agent
• CAP (Calendar APprentice)
• Adele (Agent for Distance Learning Environments)

7
Agent Reasoning

• Reactive
• Table lookup, Rule based, CBR
• State-based
• Finite Automaton, Theorem proving
• Goal-directed
• Search, A, IDA, planning
• Utility maximizing
• GA, CSP

8
IA Architectures

• Reactive Agents
• Deliberative
• Belief-Desire-Intention Agents (BDI)
• Agent Oriented Programming
• Hybrid/Layered
• Horizontal Layering Touring Machines
• Vertical Layering InteRRap

9
Touring machine Architecture
Modelling Layer
Perception Subsystem
Action Subsystem
Planning Layer
Reactive Layer
Action Output
Sensor Input
Control Subsystem
10
InteRRap Architecture
Cooperation layer
Social knowledge
Plan layer
Planning knowledge
Behavior layer
World model
World Interface
Action output
Perceptual input
11
Agent Oriented Software Engineering

• Agent software design process that extend OO
  methodologies to address agent requirements
• GAIA (Wooldridge, Jennings, Kinny, Zambonelli)
• DESIRE (Treur, Jonker, Brazier)
• Agent UML (Odell et al.)
• Concurrent METATEM (Fisher)

12
Intelligence? Think again!

• Is symbolic processing key to AI? Or
• Is intelligent and rational behavior innately
  linked to the environment that an agent occupies?
• May be
• Intelligent behavior is not disembodied, but is a
  product of the interaction the agent maintains
  with its environment
• Intelligence is an emergent property of several
  simple behavior

13
Are they intelligent?
Many insect societies display significant amount
of socie-tal intelligence and sophis-tication,
even though individual insects are quite dumb
14
And what about them?
Wolves and several other animals hunt in packs.
They display bewildering hunting strategies when
in packs, but are not good hunters as an
individual.
15
And what about us?
Most of the games that we play are guided by
simple rules, yet the overall scenario becomes
extremely complex involving a lot of
coordination, cooperation, competition and
arbitration among human agents.
16
Multi Agent Systems (MAS)
Actions (Ai ), Aigt 2
The set of World states W
17
Why MAS?

• Multi Agent System Distributed AI (DAI)
• Centralized can be decentralized.
• Decentralized may not be centralizable
• Competitive environment
• Private information
• Information itself decentralized
• Faster, efficient, flexible, robust scalable
• Individual agents might be simpler to design

18
When where to use MAS?

• Environment is open and have no centralized
  designer
• Environment provides an infrastructure specifying
  communication interaction protocols
• Environment contains agents that are autonomous
  and distributed and may be self-interested or
  cooperative.

19
Challenges for MAS designers

• Foresee environmental dynamics for the system
  being designed
• Design action set to cover all foreseeable
  situations
• Provide mechanisms to choose appropriate behavior
  under all circumstances

20
Issues in MAS design
Coordination
Search
Distributed Problem Solving Planning
Distributed Rational Decision Making
Learning in MAS
21
Coordination
Coordination
Cooperation
Competition
Planning
Negotiation
Distributed
Centralized
22
Speech Act Theory

• Utterances are speech acts with
• Locution
• Physical utterance with context and reference
• Illocution
• Conveying intentions
• Perlocutions
• Actions resulting from illocutions
• Senders intention is clearly defined by the
  message

23
Communication Languages

• Knowledge Query Manipulation Language (Labrou
  Finin) KQML
• High-level protocol for information exchange
  independent of content syntax and ontology
• Performatives
• Inform, Request, Promise, Reply, Query
• Query ask-one, ask-if, ask-all, subscribe
• Facilitator Broker, Recruit, Recommend
• Knowledge Interchange Format KIF

24
Agent Interaction Protocols

• Motive
• Determine shared goals
• Determine common tasks
• Avoid unnecessary conflicts
• Pool knowledge and evidence
• Contract Net
• Blackboard System
• Negotiation

25
Contract Net
Manager announces the existence of tasks via a
multicast
26
Contract Net
Agents evaluate the announcements. Some of these
agents submit bids
27
Contract Net
The manager award the contract to the most
appropriate agent
28
Blackboard System
Blackboard
Executing Activated KS
Library of Knowledge Sources/Agents
Control Components
Pending KS Activations
Events
29
Negotiation

• Process through which parties can arrive at a
  mutually acceptable solution
• Bargaining
• Argumentation
• Continuously divisible good
• Negotiation outcome a partition of the good
• Individual preferences over different parts of
  the good represented as utility functions
• Concerns
• Fairness of division
• Efficient allocation

30
Evaluation Criteria

• Proportional each agent believes it got at least
  1/n th of the good being divided.
• Envy-free each agent believes no other agent got
  more than what it got.
• Equitable share received by each agent is
  perceived to be identical by local estimate
• Efficient the perceived share of no agent can be
  increased without decreasing the perceived share
  of some other agent.

31
Problem of the Cake
Vanilla flavor is awesome!
I like strawberry icing on my cake.
32
Concerns

• Envy-free divisions can still produce spiteful
  behavior leading to inefficient allocations
• agent A believes it can get the largest share by
  its own estimate, but the share received by B
  would be more by Bs estimate
• Agent A can act spitefully to deny B the
  corresponding share even though its own share
  decreases

33
How to cut the cake?Austins Procedure
Cutting a cake
Arbitrator moves the knife starting from the left
edge of the cake any agent can call cut to
stop the procedure.
Agent A calls cut, gets part of the cake from
left edge to current knife position agent B
gets the rest of the cake
34
Is it efficient?

• Austins procedure does not guarantee efficiency
• Theorem There does not exist an algorithm that
  guarantees an efficient division of continuously
  divisible good between two agents

35
Modeling for more

• Utility functions
• ÛB Model of Bs utility, UA As utility
  function
• Modelers goals
• maximize its utility
• to be envy-free
• Augmented knives moving procedure

Left knife
Right knife
Region allocated to B
36
Properties of the Procedure

• If a Pareto-optimal allocation for a problem
  involves a contiguous region, the procedure will
  select it
• Dominates Austins procedure, with respect to
  efficiency of allocations
• Produces Pareto-optimal allocations for rational
  agents with
• monotonic utility functions
• utility functions with a single optimum

37
Negotiation - Bayesian modeling

• Model for negotiation among agents
• capture subtle relationships among beliefs,
  actions and their significance
• progressive negotiation
• define and use favorable negotiation context
• Model of other party to negotiation, that
  gradually improves

38
Advantages

• Can effectively capture
• the causality among beliefs and actions
• direct and indirect influences of various factors
  on behaviors
• Method to update beliefs, and probability
  estimates
• Facilitate combination of domain knowledge and
  data

39
A bargaining scenario between agent A (seller)
and agent B (buyer)
40
The Approach
41
The Approach (contd.)
42
Action choices and network updates

• Negotiation context is set such that
• the desired value of the target node has maximum
  possible probability
• the I nodes are valued to satisfy the above
• the action node-values correspond to the
  negotiation actions
• Belief-node values are updated based on outcome
  of the current negotiation process
• The network may be changed with new
  nodes/node-values, or some nodes/node-values may
  be deleted.
• The updated network is used by the algorithm in
  future negotiations

43
Designing MAS
Political Science
Sociology
Ethology
Game Theory
Economics
Psychology
44
An ApplicationMovies2Go Recommending movies

• Finding movies based on user interest
• Informal feedback
• Users have preferences about actors, actresses,
  genres, etc.
• Preferences routinely conflict
• Need robust mechanism for selecting compromise
  choices
• Learning to recommend based on usage

45
Approach

• Use techniques from Voting theory
• Adapting techniques for trading off disparate
  preferences between multiple users to tradeoff
  conflicting preferences of individual users on
  multiple dimensions
• Provides desirable guarantees regarding the
  recommendations generated from stored preferences

46
VOTING FOR MOVIES

• The user can give each dimension an importance
  ranking relative to other dimensions.
• User can rate actors, actresses, genres,
  directors, etc.
• Given a set of movies, each dimension orders them
  based on the stated preferences (number of voters
  per dimension is proportional to the importance
  of that dimension)

9
7
7
11
10
47
But which Voting Scheme?

• Evaluation Criteria
• Social Welfare
• Pareto Efficiency
• Individual Rationality
• Stability Dominant Strategies
• Computational Efficiency
• Distribution Communication Efficiency

48
Perfect Voting

• Input Social Preference Ordering gti
• Output gt
• A social preference ordering should always exist
• Defined for every pair o,o ?O
• Asymmetric and transitive
• Pareto efficient
• Independent of irrelevant alternatives
• No dictator

49
Arrows impossibility theorem

• No social choice rule satisfies all of these 6
  conditions.
• Some of the conditions are relaxed in real
  life protocols

50
Standard Voting Protocols

• Plurality Protocol
• Binary Protocol
• Borda Protocol
• Clarke tax algorithm
• Every agent i ? A reveals his valuation vi(g) for
  every possible g
• g argmaxg Sivi(g)
• taxi Si?jvi(g) Si?jvj(argmaxg Si?kvk(g))

51
Recommending Movies

• Select movies from internet sites based on
  constraints
• Use voting scheme (Borda Count) to rank
  selections
• For each dimension, a movie gets votes equal to
  number of movies below it ranked by that
  dimension multiplied by the importance of that
  dimension
• For each movie, the votes received from all
  dimension are added
• Movie(s) are ordered by the total votes received

52
Recommending Movies

• Use a Naive Bayes classifier to select movies
  based on synopsis of movies rated by user
• Proactive recommendation of new movies
• Query modalities
• Unconstrained Queries Find me a movie?
• Constrained Queries
• Find me a movie similar to movie X?
• Find me a movie having actor/director/genre X?

53
Other Applications

• Buyer-seller agents in e-market
• RT monitoring of telecommunication networks
• Optimization of Industrial manufacturing
  Production processes
• Computer e-games
• Modelling optimization of traffic control
• Investigation of social aspects of intelligence

54
Economic Mechanism Auction

• Use of economic principles to design both agent
  interaction frameworks and interaction strategies
  
• Standardized procedures for allocating
  goods/tasks
• Players
• auctioneer,
• bidders

55
Auction Types Mechanisms

• Auction classes
• Basic one seller, many buyers
• Reverse one buyer, many sellers
• Double many buyers/sellers
• Mechanisms
• English
• First-price sealed bid (FPSB)
• Dutch
• Vickrey

56
Private-value auctions

• Four mechanisms have same expected revenue and
  are efficient
• Dutch strategically equivalent to FPSB
• Vickrey strategically equivalent to English both
  have dominant strategies (no speculation)
• For risk-averse bidders Dutch FPSB dominate
  Vickrey and English
• For risk-averse auctioneer Vickrey English
  dominate Dutch FPSB

57
Learning in MAS
58
Issues in DML

• Easy for a single agent to learn.
• Learned agents can teach (ATA)
• What if two agents to learn parts of a single
  system simultaneously?
• Credit Assignment Problem
• Learning through communication
• Learning to communicate
• What if agents are selfish?

59
Standard Techniques

• Reinforcement learning
• Q-learning
• Interactive
• Isolated
• Learning organizational roles
• Learning in market environments
• Improving learning by communication

60
Prisoners Dilemma
Should I cooperate with him or not??
I want to cooperate, but what if he defects??
61
Payoffs

• Reward for mutual cooperation 3
• Temptation to defect 5
• Suckers payoff 0
• Punishment for mutual defection 1

62
Payoff Matrix
B
A
63
Nash Equilibrium

• (a,b) is a Nash equilibrium for players A
  B, if regardless of any action that the opponent
  chooses, the payoff cannot decrease from
  PM(a,b)
• Existential property NE exist for all games
• Non-uniqueness There can be several NEs for a
  game
• Non-pareto optimality An NE may not be
  pareto-optimal

64
Another example
65
The Game of the Prisoners

• Prisoners dilemma
• Best option Defect (NE)
• Consider the game of repeated prisoners dilemma
• Accumulate the points of each game
• Play in a round robin fashion
• What is the best strategy?

66
Tit for Tat!

• Robert Axelrods experiment
• Anatol Rapaport suggest Tit for Tat
• Cooperate first time and do what your opponent
  did last time
• Tit for Tat emerged as an unbeatable strategy. It
  is an evolutionarily stable strategy

67
Animals caught in prisoners dilemma

• How should an animal behave towards its opponent?
• Hawk
• Attack
• Dove
• Retreat if opponent playing Hawk
• Or just display strength
• Payoffs
• Winner 50, Loser 0
• Injury -100, Display -10

68
Payoff Matrix
Attacker
Opponent
69
Bourgeois Strategy

• Hawk/Dove are not evolutionary stable strategies
• Maynard Smith introduced Bourgeois strategy
• Play dove strategy when in opponents
  territory play hawk when in own territory
• Is evolutionarily stable strategy

70
Payoff Matrix
Opponent
Attacker
71
Examples from Nature

• The Speckled wood butterflies display Bourgeois
  strategy (Nick Davies)
• Stickleback and the mirror (Manfred Milinski)
• Displays Tit for Tat
• Nice, forgiving, retaliating