Intelligent agents on the Web

1
Intelligent agents on the Web
5th International Workshop on Symbolic and
Numeric Algorithms for Scientific Computing
Timisoara, Romania, October 1-4, 2003

• Adina Magda Florea
• http//turing.cs.pub.ro/adina
• adina_at_cs.pub.ro

2
IDC (http//www.idc.com)

• IDC estimates that the global market for software
  agents grew from 7.2 millions in 1997 to 51.5
  millions in 1999
• and that it will reach 873.2 millions in 2004,
• with a compound annual growth rate of 76.2
  between 1999 and 2004.

2
3
Agents
1. Basic notions of agents and MAS

• Much discussion of what (software) agents are and
  how they differ from programs in general
• Much discussion about the difference between
  software agents and intelligent agents
• Do they bring us anything new in modelling and
  constructing our applications?

A.M. Florea, SYNASC03
3
4
Agents characteristics

• act on behalf of a user or a / another program
• operate without the direct intervention of humans
  and have control over their actions and internal
  state - autonomy
• sense the environment and acts upon it -
  reactivity
• capable purposeful action - pro-activity
• goal-directed vs reactive behaviour?
• interact with other agents and humans - social
  ability
• function continuously - persistent software
• mobility ?

A.M. Florea, SYNASC03
4
5
Motivations for agents

• Large-scale, complex, distributed systems
  understand, built, manage
• Open and heterogeneous systems - build components
  independently
• Distribution of resources
• Distribution of expertise
• Needs for personalization and customization
• Interoperability of pre-existing systems /
  integration of legacy systems

A.M. Florea, SYNASC03
5
6
Agents vs Objects

• Autonomy - stronger - agents have sole control
  over their actions, an agent may refuse or ask
  for compensation
• Flexibility - Agents are reactive, like objects,
  but also pro-active
• Higher level communication than object messages
• Agents are usually persistent
• Own thread of control

A.M. Florea, SYNASC03
6
7
Multi-agent systems

• Many entities (agents) in a common environment

Environment
Influenece area
Interactions
A.M. Florea, SYNASC03
7
8
Multi-agent systems

• High-level interactions
• Interactions for - coordination
• - communication
• - organization
• Coordination
• ? collectively motivated / interested
• ? self interested
• - own goals / indifferent
• - own goals / competition / competing for the
  same resources
• - own goals / competition / contradictory goals
• - own goals / coalitions

8
9
Multi-agent systems

• Communication
• ? communication protocol
• ? communication language
• - negotiation to reach agreement
• - ontology
• Organizational structures
• ? centralized vs decentralized
• ? hierarchical/ markets

9
10
Cognitive agents

• The model of human intelligence and human
  perspective of the world ? characterise an
  intelligent agent using symbolic representations
  and mentalistic notions
• knowledge - John knows humans are mortal
• beliefs - John took his umbrella because he
  believed it was going to rain
• desires, goals - John wants to possess a PhD
• intentions - John intends to work hard in order
  to have a PhD
• choices - John decided to apply for a PhD
• commitments - John will not stop working until
  getting his PhD
• obligations - John has to work to make a living
• (Shoham, 1993)

10
11
Premises of cognitive agents

• Such a mentalistic or intentional view of agents
  - a kind of "folk psychology" it is a useful
  paradigm for describing complex distributed
  systems.
• The complexity of a system or the fact that we
  can not know or predict the internal structure of
  all components seems to imply that we must rely
  on animistic, intentional explanation of system
  functioning and behavior.

11
12
Reactive agents

• Simple processing units that perceive and react
  to changes in their environment.
• Do not have a symbolic representation of the
  world and do not use complex symbolic reasoning.
• Intelligence is not a property of the active
  entity but it is distributed in the system, and
  steams as the result of the interaction between
  the many entities of the distributed structure
  and the environment.

12
13
Several types of information agents
2. Information agents

• Personal agents
• provide "intelligent" and user-friendly
  interfaces
• observe the user and learn users profile
• sort, classify and administrate e-mails,
• organize and schedule user's tasks
• in general, agents that automate the routine
  tasks of the users
• Web agents
• Tour guides Search engines
• Indexing agents - human indexing
• FAQ finders - spider indexing
• Expertise finders

13
14
Cooperative information retrieval systems

• Use information retrieval theory and AI
• Make information resources available by wrapping
  them with agents capabilities
• Every agent is expert with its own repository
• Agents communicate using an ACL

14
15
RETSINA Reusable Environment for Task-Structured
Intelligent Networked Agents

• RETSINA is a domain-independent and reusable
  infrastructure on which MAS systems, services,
  and components live, communicate, and interact.
• RETSINA is an architecture for developing
  distributed intelligent software agents that
  cooperate asynchronously to perform information
  management information gathering, information
  filtering, information integration
• RETSINA is project developed at the Robotics
  Institute, CMU

15
16
RETSINA MAS architecture
16
17
The agent architecture
17
18
WebMate an information search agent in RETSINA

• WebMate is a personal agent for WWW browsing that
  enhances searches and learns user interests.
• Information searching
• trigger pair model
• document similarity based on relevance feedback

18
19

• Trigger Pair Model
• If a word S is significantly correlated with
  another word T, then (S, T) is considered a
  trigger pair with S being the trigger and T being
  the triggered word
• Relevance feedback
• The user identifies relevant pages from an
  initial list of retrieved documents
• the system analyzes the page using the context of
  keyword (i.e. the words near by)
• the system finds out the relevant keywords
• enlarge the user query using the relevant
  keywords

19
A.M. Florea, Feb 2003
20
Information agents for e-communities (BTexact
Technologies)

• Personal Agent Framework (PAF)
• central profile management agent
• suite of application agents that use profiles in
  conjunction with several information sources
• Web-based agents

20
21

• Profiler Agent
• one for each user
• stores interest information in a hierarchy in
  which interests lower in the hierarchy inherit
  their parent interest characteristics
• transparent for the user
• each interest
• private
• restricted
• public

21
22

• Application agents
• Bugle uses profile information to generate a
  daily newsletter that contains articles relevant
  to the users interests
• Grapewine works in the background, periodically
  notifying members via email about other members
  who have similar interest profiles. iVine lets
  the user interactively locate members with
  similar interests. Shows the shared areas of
  interest so the use can decide.
• Pandora helps broaden users interests via
  collective filtering, suggests new interests for
  members to explore.
• Radar just-in-time information agent monitors
  the user current activity while, for example,
  authoring a document, and offers relevant
  information resources, news reports, FAQs allows
  interaction with iVine.

22
23
Agents roles in e-learning
3. Agents for e-learning

• Enhance e-learning content and experience
• give help, advice, feedback
• act as a peer learning
• participate in assessments
• participate in simulation
• personalize the learning experience
• Enhance LMSs
• facilitate participation
• facilitate interaction
• facilitate instructors activities

23
24
ADELE

• Pedagogical agents developed by Center for
  Advanced Research in Technology for Education
  (CARTE) at USC / ISI to assist students in
  working through course materials
• The lead character, an agent named Adele (Agent
  for Distance Learning Environments), is a
  pedagogical agent designed to work with Web-based
  educational simulations.

24
25

• Adele consists of a pedagogical agent and a 2D
  animated persona, which is implemented as a
  web-based Java applet.
• Adele
• adapts the presentation of the material as needed
• monitors students progress
• provides feedback, hints and rationales to guide
  student actions
• references relevant material
• evaluates student performance by probing
  questions.
• She is used in two medical education systems
  case-based diagnosis and trauma care.

25
26

• Simulations created for the course in diagnostic
  skill development presents the student with
  actual cases, including patient history, results
  of exams, lab tests, x-rays, CT scans and other
  diagnostic imaging methods.
• By questioning and examining the virtual
  "patient" and studying clinical data, the student
  is able to practice diagnostic skills.

26
27

• Trauma care is a collaborative activity -
  physicians and paramedics work with other
  emergency response personnel.
• Adele functioning includes the notion of
  "situations".
• A situation is a high-level description of an
  "interesting state" along with a description of
  steps to take in that situation.
• The animated persona is a Java applet. It can be
  used alone or with a Web page-based JavaScript
  interface, or incorporated in larger simulations.

27
28

• ADELEs architecture

Architecture of single user system. In the
multi-user system, RE is server-based, as is the
Session Manager Student model, case task plan,
initial state Student record of actions
28
29

• Task representation
• Task plan task steps and their dependencies,
  step rationale
• task steps object-oriented data structures
  processed by Adeles Java-based reasoning engine
• Reasoning engine runs in 3 modes
• restricts unsolicited input Hint, Why
• practice mode Hint
• exam Adele is not available
• Situation triggers a plan
• Situation plans are pre-authored
• Adeles reasoning ? situation-monitoring task
• ? situation-based reasoning.

29
30

• Pedagogy
• Situation-based reasoning allows the recognition
  of pedagogical opportunities
• ask questions related to a particular task
• give feedback to chosen answers
• ask follow-up questions
• give references significant to a particular task
• verify correctness of plan step order
• records the students actions
• analyze students record and provides domain
  appropriate feedback (e.g., evaluation of
  diagnosis, evaluation of diagnostics costs,
  evaluation of the steps taken).

30
31

• Adeles persona
• Uses gaze and gestures to react to students
  actions repertoire of facial expressions and
  body postures that represent emotions surprise,
  disappointment, etc.
• Senses users mouse pointing, turns her head and
  looks toward that point.
• She has also a pointer that she can use to point
  to objects in other windows.
• Animations are produced from 2-dimensional
  drawings gt makes possible to run on a variety of
  desktops (no 3D graphics needed).

31
32
STEVE

• Developed at Information Science Institute, USC
• Learning environment simulation of the naval
  training facility in Great Lakes, Illinois
• Steve a 3D pedagogical agent
• Training a 3D, interactive, simulation
  environment

32
33

• Students and Steve agents are immersed in the
  simulation environment
• Students 3D immersive view of the virtual world
  through a head-mounted display (HMD) and
  interacts with the world via data gloves
• Lockheed Martins Vista Viewer software uses data
  from a position and orientation sensor on the HMD
  to update the students view as he moves around
• Additional sensors on the glove keep track of the
  students hands and Vista sends messages when the
  student touches virtual objects

33
34

• Humans and agents communicate through spoken
  dialogue
• An agent speaks to a person by sending a message
  to the persons text-to-speech software
  broadcasts the utterance through the headphones
  mounted on the HMD
• Entropics TrueTalk for speech synthesis
• Students speak to the microphone on the HMD -
  sends the utterance to the speech recognition
  software semantic representation of the
  utterance to the agents.
• Entropics GrapHvite for speech recognition

34
35

• Steves cognitive architecture

Task knowledge
Pedagogical capabilities
Perception snapshots important events
Abstract motor commands
Soar rules
Spatial properties
Motor Control
Perception
Relevant events
Detailed motor commands
Message Dispatcher
Interface components
Simulator
Visual, audio effects
35
36

• Separation between domain independent
  capabilities and domain specific knowledge
• Perception, cognition and motor control modules
  general capabilities independent of a particular
  domain
• planning
• replanning
• plan execution
• assessment of students actions
• question answering (What should I do next?, Why?)
• episodic memory
• communication
• control of human figure

36
37

• Course author specifies the domain knowledge in a
  declarative language
• Domain knowledge
• perceptual knowledge
• knowledge about objects in the virtual world,
  objects simulation attributes and spatial
  properties
• task knowledge
• procedures for accomplishing domain tasks and
  text fragments for talking
• Tasks set of steps
• ordering constraints
• causal links
• hierarchical planning

37
38

• Steve acts as a tutor or learning companion
• Steve was extended to support team training
• Steve agents can play two roles
• tutor for an individual team member
• can substitute for missing team members

38
39

• Tasks were extended with roles for different
  participants
• Planning is extended by mapping task steps to
  team roles roles are assigned during plan
  creation
• Team task request
• each Steve agent involved in the task as a team
  member or instructor uses his task knowledge to
  construct a complete task model
• New types of actions - a speech act from one team
  member to another
• each speech act appears as a primitive action in
  task description

39
40
Learning Companion that recognizes affect

• MIT Media Lab
• Affective states significant to learning
  anxiety, worry/boredom, indifference, interest,
  curiosity, confident, etc.
• on-goal and off-goal

• Posture
• Eye-gaze
• Facial expression
• Hand movement

Affect recognition
40
41
41
42
Agents for LMSs

• Knowbots (or Knowledge Robots) created to
  automate the repetitive tasks of human
  facilitators in online workshops
• A system developed at ALN Center at Vanderbilt
  University, Nashville, TN

42
43

• System architecture
• 5 components knowbots, the learner, the
  knowledge base, the repository of assignments and
  the interface with the facilitator.
• Knowbots sit between the instructor and the
  learner, mediating the interaction.

43
44

• 3 types of knowbots
• scheduled - sends a reminder and a report to each
  participant upon completion of a scheduled check
• on-demand - invoked by the learner these
  knowbots return results immediately to the
  requesting user
• submission helper - for submission of an
  assignment that assists the user in submitting
  the assignment they also notify the facilitator
  when the submission is made.

44
45

• Knowbot structure
• user-interface agents
• checker agents (agents that check submissions)
• e-mail agents
• knowledge base modules.
• User-interface agents - graphical interface,
  web-based agents assure user interaction with
  the knowbot
• Execute the checker agents by request
• Present information to the user
• Provide appropriate interface to execute actions
  such as requests for help
• Communicate with other agents and with the
  knowledge base.

45
46

• Email agents are responsible for generating,
  composing, organizing, and sending e-mails to
  both the instructor and the participants.
• Examples of e-mails that are generated and sent
  to the participants are
• the assignment-status report
• the assignment reminder and notification
• the message responding to a request for help.
• The e-mail agents compose the content of the
  e-mail by retrieving data from the knowledge base.

46
47

• Checker agents are responsible for checking
  assignments for the participants.
• The agents can be invoked either by the scheduler
  or by the participant through the user-interface
  agents.
• determine the completion status of the assignment
  based on the pre-defined knowledge of
  requirements for assignment completion.
• record the results and access the knowledge base
  through the established Open Database
  Connectivity (ODBC) using the Cold Fusion Markup
  Language (CFML).
• determine what particular knowledge each
  participant needs in order to complete the
  assignment.

47
48

• Knowbots in the system
• Posting knowbot - looks for two types of messages
  posted in the specified forum by participants
  one is a self-introduction message, the other is
  a reply-to-another message. The knowbot then
  sends a reminder and the results of the scheduled
  check via e-mail to the participants.
• S,OD
• Course Review knowbot - looks for at least 3
  course-reviewed messages posted in 3 different
  threads by the participants and sends a reminder
  and the result of the checking by e-mail to the
  participants.
• S,OD
• Basic HTML knowbot - checks the status of each
  participant's personal homepage to determine if
  it contains the required elements such as mail-to
  tag, bulleted list, etc.
• S,OD

48
49

• Topic knowbot - is invoked by the student and
  determines if at least one message has been
  posted into the specified forum in the
  conferencing system about the required topic. The
  result is displayed to the student. OD
  only
• Multimedia knowbot - Each participant submits
  information via a knowbot. The knowbot notifies
  the workshop facilitator about the submission,
  provides a template for the facilitator to check
  the participant's work, stores the results into
  the database and sends a notification e-mail to
  report the result to the participant.
• Submission Helper
• Discussion Builder knowbot - Same functionality
  as Multimedia knowbot
• Submission Helper

49
50
Electronic commerce
4. Agents for e-commerce

• Transactions - business-to-busines (B2B)
• - business-to-consumer (B2C)
• - consumer-to-consumer (C2C)
• Difficulties of eCommerce
• Trust
• Privacy and security
• Billing
• Reliability

50
51
Consumer's buying behavior

• Consumer's Buying Behavior (CBB) research - a
  number of models of the consumer's behavior
• CBB - Guttman e.a., 1998
• Need identification
• Product brokering
• Merchant brokering
• Negotiation
• Purchase and delivery
• Product service and evaluation
• - some stages may overlap

51
52
Agents as mediators in eCommerce

• Persona Bargain
• Logic Firefly Finder
  Jango Kasbah T_at_T IntelliShoper
• Need
• identification
• Product
• brokering
• Merchant
• brokering
• Negotiation
• Purchase
• and delivery
• Product
• service

52
53
(a) Comparison shopping agents

• Search online shops to find products, merchants
  and best deals
• Product brokering
• Techniques
• feature-based filtering feature keywords
• collaborative filtering similarities between
  users profiles
• constraint-based filtering specifying
  constraints (price, date limit)

53
54

• Product brokering
• let the users create preference profiles
• allows shoppers to specify constraints on a
  product and scores the products
• CSP engine hard constraints and soft constraints
• 1988 ? AOL
• helps consumers find products (alert) (Ringo
  books, CDs)
• ACF Automated Collaborative Filtering
• identifies the shopper's "nearest neighbours" and
  offers products highly rated by them
• 1998 ? Microsoft

54
55

• Merchant brokering
• finds specifications and product reviews
• makes recommendations to the user
• submit queries to vendors sites and interpret
  results to identify lowest price items
• monitors "what's new" lists, watches for special
  offers
• automates the building of wrappers to parse
  HTML docs and extract products features
• Web pages are different exploits
• ? Navigation regularities (easy to find
  products)
• ? Corporate regularities (similar looknfeel)
• ? Vertical separation (use of white spaces)
• 1999 ? Excite

55
56

• (b) Auction bots
• Agents that can organize and/or participate in
  online auctions for goods
• Aim develop a Web-based system in which users
  can create their own agents to buy and sell goods
  on their behalf
• User options
• Create a new buying agent
• Create a new selling agent
• See currently active agents
• Create a new finding agent
• Browse the marketplace for active agents

56
57

• Selling agent parameters set by the user
• - desired date to sell the good
• - desired price to sell the good
• - minimum price to sell at
• - "decay" function of the price over time to
  determine the current offer price
• anxious - linear function
• cool headed - quadratic function
• frugal - exponential function
• Buying agent parameters set by the user
• - date to buy the item by
• - desired price
• - maximum price
• - "growth" function of price over time

57
58

• Integrates product brokering, merchant brokering,
  and negotiation
• User agents negotiate across multiple attributes
  of a transaction, e.g., warranty length and
  options, shipping time and cost, service
  contract, return policy, quantity, accessories,
  credit options, payment options
• Agents quantify those aspects using a
  multi-attribute utility function
• Today Frictionless Commerce applies the
  technology to B2B markets (e-sourcing)

58
59
IntelliShoper (U. Iowa)

• Integrates product brokering, merchant brokering,
  and negotiation
• Goals
• Customize behavior adaptively by learning users
  preferences
• Provides assistance by remaining autonomous from
  both customer and vendors
• Protect shoppers privacy by concealing their
  identities and behavior from vendors

59
60
Anonymizing server
IntelliShoper server
Privacy Agent
Monitor Agent
Learning Agent
Vendor plug-ins
Sequence of shopping assistance activities

• The user creates an account and one or more
  personae
• The user takes on a persona
• The persona initiates a shopping session by
  submitting a query to the LA
• The LA stores the users request in the database
• The LA uses vendors plug-ins to send requests to
  vendors
• Results from vendors are parsed through the
  vendors plug-ins
• IS stores the result in the database
• The LA uses the persona profile to rank the hits
• The LA presents the results to the persona
• The PA forwards the results to the user
• The user can further interact with the LA

Basic interaction loop
60
61
12. The MA loads standing queries from the
database 13. The MA uses vendor plug-ins to check
for any new results from the vendors 14. IS
parses new and updated hits 15. IS stores the
hits in the database until the users logs in again
Occurs offline

• Privacy Agent
• lets the user take a shopping persona
• hides identity user info (permutation, stripping
  of IP addresses, encryption, decription)
• Shopping Persona
• becomes the public user
• 2 aims protect user privacy multiple profiles
• Interface
• create new persona preferred sites
• see current personae (name, what to buy,
  preferences)
• submit a new shopping request via the query
  interface
• view hits

61
62

• Interface with the vendors Web sites
• submitting queries
• parsing results
• Language for specifying vendor dependent logic
  based on XML and inspired by Apples Sherlock
  engine
• Personas Profile
• Preference
• Keywords
• Relevant features numeric (discretized) and
  textual (keywords)
• Updating the user profile
• Temperatures for features
• Updates temperatures after any user action
  related to a given hit

62
63

• Temperature update
• T(t1) (1 - ?) T(t) ? ?T
• 5 possible actions
• Buy string positive feedback ?T 2
• Browse weak positive feedback ?T 1
• Skip weak negative feedback ?T -1
• Remove strong negative feedback ?T -2
• Status
• - Research project
• - Current prototypes eBay, Yahoo and Amazon
  auctions
• - Research on the development of intelligent
  wrappers that could automate submitting queries
  and parsing results.

63
64

• References
• M. Wooldrige. An Introduction to MultiAgent
  Systems, John WileySons, 2002, Ch.11, p.243-266.
• R. Guttman, A. Mokas, P. Maes. Agents as
  mediators in electronic commerce. In Intelligent
  Information Agents, M. Klush (Ed.), Springer
  Verlag 1999, p.131-152.
• P. Noriega, C. Sierra. Auctions and multi-agent
  systems. In Intelligent Information Agents, M.
  Klush (Ed.), Springer Verlag 1999, p.153-175.
• W. Brenner, R. Zarnekov, H. Witting. Intelligent
  Software Agents, Springer Verlag, 1998, Ch.6,
  p.267-299.
• K. Sycara, Massimo Paolucci, Joseph Giampapa
  The RETSINA MAS Infrastructure TechReport
  CMU-RI-TR-01-05 2001
• K. Chen, K. Sycaca WebMate A Personal Agent
  for Browsing and Searching The Robotics
  Institute, Carnegie Mellon University 1998
• K. L. Clarc, V.S. Lazarou A Multiagent System
  for Distributed Information Retrieval on the
  World Wide Web 1997
• F. Menczer, W. Street, A. Monge. Adaptive
  assistants for customized e-shopping. IEEE
  Intelligent Systems, Nov/Dec 2002, p.12-19.

64
65

• References - continued
• S. Case, N. Azarmi, M. Thint, T. Ohtami.
  Enhancing e-communities with agent-based systems.
  IEEE Computer, July 2002, p.64-69.
• R. Ganeshan, W.L. Johnson, E. Shaw, and B.P.
  Wood. Tutoring Diagnostic Problem Solving , In
  Proceedings of the Fifth Int'l Conf. on
  Intelligent Tutoring Systems, 2000.
• E. Shaw, W.L. Johnson, and R. Ganeshan.
  Pedagogical Agents on the Web. In Proceedings of
  the Third Int'l Conf. on Autonomous Agents, pp.
  283-290, May, 1999.
• C. Thaiupathump, J. Bourne, J.O. Campbell.
  Intelligent Agents for Online Learning. JALN
  Volume 3, Issue 2 - November 1999.
• ADELE http//www.isi.edu/isd/ADE/ade-body.html
• Ganeshan, R., Johnson, W.L., Shaw, E., and Wood,
  B.P. Tutoring Diagnostic Problem Solving , In
  Proceedings of the Fifth Int'l Conf. on
  Intelligent Tutoring Systems, 2000
• Shaw, E., Ganeshan, R., Johnson, W.L., and
  Millar, D. Building a Case for Agent-Assisted
  Learning as a Catalyst for Curriculum Reform in
  Medical Education, In Proceedings of the Int'l
  Conf. on Artificial Intelligence in Education,
  July, 1999
• Shaw, E., Johnson, W.L., and Ganeshan, R.,
  Pedagogical Agents on the Web. In Proceedings of
  the Third Int'l Conf. on Autonomous Agents, pp.
  283-290, May, 1999

65
66

• References - continued
• STEVE http//www.isi.edu/isd/VET/vet-body.html
• Rickel, J., Johnson, W.L., Virtual Humans for
  Team Training in Virtual Reality, in Proceedings
  of the Ninth International Conference on AI in
  Education, pp. 578-585, July 1999, IOS Press.
  (Received Best Paper award.)
• Rickel, J., Johnson, W.L., Intelligent Tutoring
  in Virtual Reality A Preliminary Report, in
  Proceedings of the Eighth World Conference on AI
  in Education, pp. 294-301, August 1997, IOS
  Press.
• Rickel, J., Johnson, W.L., Integrating
  Pedagogical Capabilities in a Virtual Environment
  Agent, in Proceedings of the First International
  Conference on Autonomous Agents, pp. 30-38,
  February 1997.
• Survey of Work on Animated Pedagogical Agents
• W.L. Johnson, J.W. Rickel, and J.C. Lester.
  Animated Pedagogical Agents Face-to-Face
  Interaction in Interactive Learning Environments.
  International Journal of Artificial Intelligence
  in Education 1147-78, 2000.
• Johnson, W.L., Pedagogical Agents, invited paper
  at the International Conference on Computers in
  Education. Also to appear in the Italian AI
  Society Magazine.

66