James Bond and Michael Ovitz The Secret Life of Agents

1
James Bond and Michael Ovitz The Secret Life of
Agents

• Katia Sycara
• The Robotics Institute
• Carnegie Mellon University
• Pittsburgh, PA 15213
• [phone]
• katia_at_cmu.edu
• http//www.cs.cmu.edu/sycara
• Project Homepage http//www.cs.cmu.edu/softagent
  s

2
Team Members CMU

• Liren Chen
• Somesh Jha
• Rande Shern
• Dajun Zeng

• Keith Decker
• Anadeep Pannu
• Vandana Verma

Prasad Chalasani Kostya Domashnev Onn Shehory
3
Talk Outline

• Introduction to Agents
• Motivations and advantages of distributed agent
  technology
• The Retsina Approachª
• Retsina Agent Architecture
• Middle Agents
• Multi-agent interaction protocols (negotiation,
  contingent contracting)
• Retsina Applications
• Concluding Remarks
• ________________________________
• ª Retsina stands for Reusable Task Structured
  Intelligent Networked Agents.

4
The (Re)-Emergence of Agents The Marriage of
Two Holy Grails
AI
SE

• Goal-directed
• Adaptive
• Knowledge-based

• Reusability
• Robustness
• Flexibility

Ubiquitous Networked Information Access
Intelligent Software Agents!
5
What is an Agent?

• A computational system that
• has goals, sensors and effectors
• is autonomous
• is adaptive
• is long lived
• lives in a networked infrastructure
• interacts with other agents

6
Agent vs. Agent

• James Bond Michael Ovitz

7
Next Generation of Agent Technology

• Currently, agent technology is mostly single
  agent focusing on information retrieval and
  filtering according to user profile
• Multi-Agent Systems that interact with humans and
  each other
• Integrate information management and decision
  support
• Enable real-time synchronization of the tasks and
  actions of humans in teams and organizations
• Acquire and disseminate timely and relevant
  information
• Anticipate and satisfy human information and
  problem solving needs
• Notify about changes in the environment
• Adapt to user, task and situation

8
Motivation for Multi Agent Systems (MAS)
Technology

• Global Information Markets
• Increasingly networked world
• Vast quantities of unorganized information
• Diverse information sources
• Inability for human to manage information access
  process---information overloading
• Moving from locating documents to making
  decisions

9
Features of MAS

• Multiple agents connected through communication
  networks
• Coordination - no agent has sufficient
  information or capabilities to solve problem
  aloneª
• Decentralized control - no master agent
• Decentralized data - no global data storage
• Agent Coupling - balancing computation and
  communication
• Asynchronous - multiple activities operating in
  parallel
• _______________________
• ª Agents could be cooperative or self-interested.

10
MAS Basic Questions (Bond and Gasser, 88)

• Coherence in coordinated decision making
• Recognition and reconciliation of disparate
  viewpoints or conflicting intentions
• Graceful performance degradation in face of
  missing information and resources
• Satisfaction of system-wide criteria (e.g.,
  optimality of solution, hard real-time deadlines,
  etc.)
• How to recognize workload imbalances and
  appropriately redistribute activities and
  responsibilities among agents
• Modeling other agents
• Synthesizing different views and results

11
MAS Concepts and Tools

• Distributed Constraint Satisfaction and
  Optimization (Yokoo, et al,1991 Sycara, et al.
  1991)
• Distributed Truth Maintenance and Multi-Agent
  Search (Huhns et al. 1991, Ishida 1997)
• Organizational Structuring (Lesser and Corkill
  1991, Gasser 1993, Decker 1995)
• Multiagent Planning (Georgeff 1983, 1984, 1995
  Durfee and Lesser 1991, Jennings 1995, Grosz et
  al. 1996)
• Contracting (Smith 1980, Mueller 1993, Sandholm
  1997)
• Negotiation (Sycara 1990 Kraus 1991, Zlotkin
  1996)

12
MAS Concepts and Tools (Contd.)

• Economic and Game Theoretic Techniques
  (Rosenschein 1995 Gmytrasiewicz et al. 1991,
  Wellman 1993)
• Open systems (Hewitt 1991, Gasser 91)
• Multiagent Logics and Ecological Approaches
  (Cohen and Levesque 1987 Ferber 1990, Shoham
  1993 Huberman et al. 1996)
• Social Laws and Norms (Tenenholtz 1991,
  Castelfranchi 1993)
• Multi-Agent Learning (Sen 1993, Durfee 1994,
  Sycara 1996)

13
Retsina Approach

• Architecture that includes data- and
  knowledge-bases and a distributed collection of
  intelligent agentsª
• Reusable and composable agent components (agent
  editor, agent operating system)
• Operates in an open world
• agents, network links, and information sources
  appear/disappear
• uncertainty
• Dynamic agent team formation on-demand
• ____________________________
• ª K.Sycara, K.Decker, A.Pannu, M.Williamson, and
  D.Zeng. Distributed Intelligent Agents. IEEE
  Expert, Dec-96

14
Retsina Functional Organization
15
Main System Issues in MAS

• Single agent architecture
• Retsina agent architecture
• Agent Self-cloning
• Finding other agents
• Middle agents
• Matchmaking and brokering
• Agent interaction protocols
• Negotiation
• Contingent contracting

16
Retsina Agent Architecture

• Planning
• hierarchical task network-based formalism
• library of task reduction schemas
• alternative task reductions
• contingent plans, loops
• incremental task reduction, interleaved with
  execution
• information gathered during execution directs
  future planning
• Scheduling
• fully expanded leaf nodes executable basic
  actions
• enabled actions (all parameters and provisions in
  place)
• adjust periodic tasks with missed deadlines

17

• Communication and Coordination
• processes incoming and outgoing messages
• creates new goals/objectives
• determines coordination interactions
• addresses security issues
• Execution Monitoring
• setup execution context (parameters and
  provisions)
• action monitoring
• deadlines/timeouts
• data collection for decisions (e.g. cloning)
• complete execution (provide results to
  appropriate downstream actions)

18
Retsina Agent Architecture
19
A task Structure (Advertisement Task Structure)
20
Reusable Behaviors

• Advertising
• send agent capability model to middle-agent(s)
• shared query behavior for other agents
• Polling for messages
• Answering queries one-shot and periodic
• Monitoring for changes and notification
• Self-cloning

21
Self-Cloning Process

• Agents that perceive an overload look for other
  agents to pass tasks to (simple model to predict
  idle time using learned estimation of task
  durations)
• When other agents not found
• locate a host with resources for cloning
• create a clone on the host
• partition tasks and transfer to clone
• the old'' agent updates its advertisement the
  clone'' agent advertises
• When clone is idle -- consider self-extinction,
  and shut down if necessary.

22
Cloning Experimental Setting

• Number of agents 10 to 20.
• Number of clones allowed up to 10.
• Number of tasks arriving at the system up to
  1000.
• Task distribution with respect to the required
  capabilities for execution normal distribution,
  where 10 of the tasks are beyond the
  capabilities of the agents.
• Agent capabilities an agent can perform up to 20
  average tasks simultaneously.

23
Task Completion w/wo Cloning
24
Middle Agents

• An agent needs to have some task/service
  performed. How can it find agents able to perform
  that task?
• In an open system
• agents generally don't have knowledge of all
  other agents
• service providers are liable to come and go over
  time
• A solution middle agents that specialize in
  making connections between agentsª
• _________________________
• ª K.Decker, K.Sycara, M. Williamson.
  Middle-Agents for the Internet. IJCAI-97

25
Middle Agent Types
Capabilities Initially Known By
26
Matchmaking Agent Yellow Page Services
27
Matchmaking in Agent Coordination

• When an agent A advertises its capability, it
  Intends toª perform any task that fits the
  specification of that capability.
• In the Retsina system an agent A advertises a
  relational schema SA, i.e., agent A intends to
  answer any query on its schema.
• If an agent B finds another agent A with a
  certain capability through matchmaking, B
  believes that agent A can successfully perform
  the task.
• Matchmaking gives operational semantics to
  predicates such as Intend.to, Bel.
• _______________________________
• ª Grosz and Kraus 1996

28
Performance of Match-made System
29
Performance of Brokered System
30
Agents in Electronic Commerce
31
Adaptive Negotiation (the Bazaar Model)

• Aims at modeling multi-issue negotiation
  processesª
• Combines the strategic modeling aspects of
  game-theoretic models and single agent sequential
  decision making models
• Supports an open world model
• Addresses heterogeneous multi-agent learning
  utilizing the iterative nature of sequential
  decision making and the explicit representation
  of beliefs about other agents
• ______________________________
• ª D.Zeng and K.Sycara. Benefits of Learning in
  Negotiation. Proceedings of AAAI-97.

32
Utility of Learning Experimental Design

• The set of players N is comprised of one buyer
  and one supplier who make alternative proposals.
• For simplicity, the range of possible prices is
  from 0 to 100 units and this is public
  information
• The set of possible actions (proposed prices by
  either the buyer or the supplier) A equals to 0,
  1, 2,, 100
• Reservation prices are private information.
• Each player's utility is linear to the final
  price ( a number between 0 and 100) accepted by
  both players
• Normalized Nash product as joint utility (the
  optimal joint utility when full information is
  available is 0.25)

33
Average Performance of Three Experimental
Configurations in Bazaar

• A non-learning agent makes decisions based solely
  on his own reservation price
• A learning agents makes decisions based on both
  the agent's own and the opponent's reservation
  price

34
Contingent Contracts and Options

• Most multi-agent systems don't handle uncertainty
  effectively
• rigid task delegation mechanism (contracts are
  binding rather than contingent
• no explicit modeling of stochastic events
• no explicit mechanism for controlling agent
  performance variability
• We are exploring the use of option pricing to
  address the above issues

35
Evaluation of Contingent vs Binding Contracts

• In the experiments, we only had two kinds of
  agents
• Interface Agents Accept queries from the user.
• Information Agents Answer queries given by the
  Interface agents.
• In each cycle a new information agent with a load
  randomly distributed between L and 0.9 appears
  with probability ?.
• When a new information agent comes up, interface
  agents have the option to abort the query on the
  old information agent and restart it on the new
  one.
• Interface agents can only switch a bounded number
  of queries to the new agent. This is indicated as
  Bound in the graphs.
• In the experiments the average delay in answering
  the queries was measured. This is indicated as
  Delay in the graphs.

36
Contingent Contracts
L
37
Contingent Contracts
Bound
38
Contingent Contracting to Handle Unreliability of
Information Sources

• Uncertain waiting time in response to queries
• random network congestion
• uncertain serve congestion/breakdown

39
The Query Restart Problem

• Agent A sends query to Agent B.
• Agent B can complete the query in time X, where
• X 1 with probability p.
• X c (c gt 1) with probability 1 - p.
• Expectation EX p (1 - p) c
• If not done by time 1, should agent A abort and
  restart, or wait?
• Can restarting reduce expectation? The variance?
  Both?
• Does it help to repeatedly restart k times?

40
A Simple Scenario Single restart

• Strategy restart just after time 1, if not done
  by then.
• Let Xi completion time of i'th query, i 1,2.
• X1, X2 are independent, identically distributed.
• New completion time is Y
• Y
• New expectation
• EY p (1 - p)(1 E X2) (X1, X2 indep.)
• 1 p (1 - p) (1 - p)? c
• If (and only if) c gt 1 1 / p, EY lt X1 !

1 if X1 1, 1 X2 if X1 c.
41
A Simple Scenario k Restarts
Number of Restarts k
42
Applications

• Visitor Hoster (PLEIADES)
• Satellite Visibility (THALES)
• Portfolio Management (WARREN)

43
Characteristics of Retsina

• Open System
• Adaptivity at the agent and organization level
  provides robustness
• Service-based, economic coordination of agents
• Reusable and extensible domain-independent
  computational infrastructure
• Integrates information gathering and execution
  monitoring with decision making
• Framework for addressing uncertainty and
  strategic interactions

44
Future of Software Agents

• Agent-based software development is an emerging
  paradigm
• Agent society that parallels human society
• Implication of the emergence of agent society for
  human workplaces, institutions, and social
  relations
• Agent society as a unit of intelligence
• Opportunities and Challenges
• The WEB is a vast knowledge base presenting novel
  opportunities for AI
• Overall system (human and software agent)
  predictability
• Security, privacy, trust issues
• Integration of legacy systems

45
Overall Issues in Open MAS

• Overall agent organization
• Single agent architecture
• Retsina agent architecture
• Agent Self-cloning
• Finding other agents
• Middle agents
• Matchmaking and brokering
• Agent interaction protocols
• Negotiation
• Contingent contracting

46
Overall Issues in Open MAS

• Overall agent organization
• Single agent architecture
• Retsina agent architecture
• Agent Self-cloning
• Finding other agents
• Middle agents
• Matchmaking and brokering
• Agent interaction protocols
• Negotiation
• Contingent contracting

47
Overall Issues in Open MAS

• Overall agent organization
• Single agent architecture
• Retsina agent architecture
• Agent Self-cloning
• Finding other agents
• Middle agents
• Matchmaking and brokering
• Agent interaction protocols
• Negotiation
• Contingent contracting