The Knowledge Technology of I-MASS (EC IST Research) Communication, Computing and Interactive Networks

1
The Knowledge Technology of I-MASS (EC IST
Research) Communication, Computing and
Interactive Networks

• Peter J. Braspenning
• p.braspenning_at_CRS.unimaas.nl
• Local I-MASS group
• Peter-Paul Kruijsen, Gabriel Hopmans Peter J.
  Braspenning
• Communications Research Semiotics
• University of Maastricht

2
Signs Meanings
3
Content

• Introduction
• What is (Computational) Semiotics?
• Semiotic Framework
• Logo of CRS
• Communication Process among People
• Perceive-Act Pathways
• Multi-Modal User Interface
• Semiotic Engineering
• Cognitive Engineering
• Agent-Oriented Modeling
• Systems Modeling via Multi-Agent Societies
• Computational Semiotics for Agent Technology
• Knowledge Landscape VRR
• Conclusions

4
What is Semiotics?

• Semiotics discipline of combining the theory of
  signs (representa-tions), symbols (categories),
  and meaning extraction. It is an in-clusive
  discipline which incorporates all aspects of
  dealing with symbols and symbolic systems,
  starting with encoding and ending with the
  extraction of meaning.
• Mathematical tools of semiotics include those
  used in control scien-ces, pattern recognition,
  neural networks, artificial intelligence, and
  cybernetics. Unified use within a computational
  semiotic framework leads to better treatments of
  the complexities (com-munication and computation)
  inherent in advanced intelligent systems.
  Semiotics is a strongly emerging
  multi-disciplinary field of study around a new
  paradigm for surpassing the classical mind-body
  dichotomy by focussing on all processes in which
  the triad object-sign-interpreter plays an
  essential role.
• The pervasive use of icons in the interaction
  with communicative virtual environments (CVEs),
  is also part of Semiotics. A lot of different
  kinds of signs are exchanged while communication
  takes place.

5
What is Semiotics? (continued)

• Semiotics is devoted to studying COM-MUNICATION
  representations, their interpretation and usage
• It investigates SIGNS and the processes by which
  we take them to mean something to us and expect
  them to mean something to others
• It investigates the resolution of meanings in
  conversations, collective discourse and culture
  in general
• Semiotics also covers non-human commu-nication
  processes such as that of animals and
  machines/artificial systems

6
Semiotic framework

• Peirce sign something standing for something
  else for somebody in one or more respects
• interpretant
• sign object

• Components of Symbol System
• Schuyt
• interpretation system
• groups acts events

7
Logo of CRS
8
Communication Process among People
interpretant
Unlimited semiosis
interpretant
interpretant
decoding
coding
I like guys with dark hair
message (signs)
medium
9
Perceive-Act Pathways
10
Semiotic Engineering perspective of HCI

• design intention
• interaction principles

11
Cognitive Engineering
usage model
interaction
system image
user
designer
12
Cognitive Engineering xSemiotic Engineering
Cognitive Engineering
Semiotic Engineering
context
medium
user
designer
13
Semiotic Engineering and Interface Evaluation

• Communicability Concept
• Communicability is the property of software that
  efficiently and effectively conveys to users its
  underlying design intent and interactive
  principles
• The communicability evaluation method allows
  designers to appreciate how well users are
  getting the intended messages across the
  interface and to identify communication
  breakdowns that may take place during interaction

14
Assessing Communicability
15
Agent-Oriented Modeling (issues)

• Intelligent Agent an assistant that takes care
  of many gory details of many mundane tasks
• Additional properties are autonomy, sociability,
  and a human-like communication
• Often able to adapt to user's interests, habits
  and preferences
• Enabled to communicate with other agents it is
  potentially entering role-taking behavior and
  social commitments with other agents that allow
  it to function in a society of agents

• Multi-Agent System bring such agents together in
  a kind of abstract society, wherein coordination,
  cooperation and/or collaboration are of paramount
  importance in order to solve problems that no
  single agent could handle on its own
• FIPA specifications represent a collection of
  standards, which are intended to promote the
  interoperation of heterogeneous agents and the
  services that they can represent.

16
Systems Modelingvia Multi-Agent Societies

• One has to decide how to provide efficient
  inter-agent communication support, what language
  should the agents talk, should the agents be
  stationary or mobile, and what technology should
  be used to build the architecture
• At present, there are not much experience reports
• Architecture of a multi-agent system can
  naturally be viewed as a computational
  organization
• Additional organizational concepts
• organizational rules,
• organizational structures, and
• organizational patterns

17
Systems Modelingvia Multi-Agent Societies
(continued)

• I-MASS uses MASs perspective not just as a
  framework for inter-action, but more as forming
  abstract societies consisting of agencies
  (comparable to societal institutions), complex
  agents (in the sense of consisting of simpler
  agents), and agents (roughly comparable to
  individuals in a societal context).
• We try to deal with content inter-operability
  issues at different abstraction layers of
  syntactics, semantics, pragmatics and social
  world. These layers fit into a coherent semiotics
  framework.

18
Computational Semiotics for Agent Technology

• Coordination is cen-tral to building MASs
• Coordinating behav-iors in MASs are often
  realized by forming groups in which both control
  and data are distri-buted.

• Therefore, agents have to have some auto-nomy in
  performing their actions.
• However, this autono-my may lead to
  un-coordinated activities due to uncertainty
  about the actions of each of the agents.

19
The relationship between uncertainty and the
situation that the agents have to handle

• The uncertainty lowers as the familiarity of the
  situation that needs to be handled increases!
• Therefore, it makes sense to develop a framework
  in which agents know how to handle routine,
  familiar, and unfamiliar situations

20
Co-ordination among agents guiding principles

• Coordination among agents is easier to esta-blish
  in routine than in unfamiliar situations
• In general, communi-cation between agents will be
  more needed in unfamiliar situations than in
  routine situations.

21
Needed an agent-architecturein which three
kinds of interaction are adressed

• Conceptual models J. Rasmussen, Information
  Processing and Human-Machine Interaction An
  Approach to Cognitive Engineering, 1986
• skills
• rules
• knowledge
• The knowledge representation should be adapted to
  these kinds of interactions.

22
Computational scenario

• First, perceived information from the environment
  leads the agent to execute an action if the
  correspond-ing situation is perceived in terms of
  action.
• If this is not the case, the agent tries to
  recognize the situation. It can recognize the
  considered situation in terms of an action or in
  terms of a goal. In the first case, it tries to
  execute the corresponding action, and in the
  second case it invokes the planning module.
• Finally, if the agent faces an ambiguity and
  cannot come to a decision, or faces many
  alternatives, then it invokes the decisionmaking
  module (based on a Cognitive Map) to make a
  decision in order to commit to achieve a goal or
  an action. A goal leads an agent to plan, that is
  to produce a sequence of actions that achieve the
  chosen goal.

23
Three levels of control of human behavior
perception recognition decision planning
execution perception recognition decision
execution perception recognition planning a
execution

• Knowledge

perception recognition planning b
execution perception recognition execution
Rules
Skills
perception execution
B. Chaib-draa P. Levesque, Hierarchical Model
and Communication by Signs, Signals and Symbols
in Multiagent Environments, 1998
a the planning process adapts old cases to the
new situation, and the adaptation is significant
b the planning process adapts old cases to the
new situation, and the adaptation is generally
minor
24
Signals, Signs, and Symbols

• Signals can be viewed as data representing
  timespace variables from a dynamic, spatial
  configuration in the environment and they can be
  processed directly by the agents as continuous
  varia-bles. In communication by signals, the
  signal delivered by an agent i has the end of
  simply being a releaser for the receiving agent j
  -- of simply eliciting a reaction by j. That is,
  the signal generally invokes a stimulus or a
  reaction, without passing through the memory (a
  data base in this model).

• Signs indicate a state in the environ-ment with
  reference to certain norms for acts. In the case
  of communication by signs, the sender makes a
  sign which refers to some state of environment
  and which has the end of signifying, of letting
  the receiver knows the same reference. Of course,
  the sender and the receiver should share a set of
  signs with their references in order to
  communi-cate efficiently. For instance in urban
  traffic, communication between a driver and a
  policeman at a crossroad is generally done by
  signs. The policeman makes a specific sign which
  refers to a certain action and which is addressed
  to certain driver(s). The addressee(s)
  recognize(s) the reference of this sign and
  activate(s) stored patterns of behaviors.

25
Signals, Signs, and Symbols

• Finally, agents can also communi-cate by symbols.
  Symbols repre-sent variables, relations and
  properties and can be formally processed. They
  are abstract con-structs related to and defined
  by a formal structure of relations and processes,
  which according to convention can be related to
  features of the external world.
• In urban traffic for instance, a dialogue between
  a policeman and a driver in natural language
  reflects a symbolbased communication. Another
  example of symbolic communication is honk the
  car horn'', etc.

• Information at knowledge and rule levels can act
  as symbols depending on the situation and the
  language used for com-munication. In familiar
  situa-tions corresponding to the rule level,
  agents can use a specific language (derived or
  not from a natural language). This lang-uage is
  generally constructed from repeated activities.
  When unfamiliar situations occur, agents do not
  dispose of any operative knowledge nor of any
  specialized language. They must then make use of
  a non specialized language (for example natural
  language), which is less concise but more
  flexible than their oper-ative language used in
  familiar situation.

26
Coordination in summary

• With signals and signs, agents do not force their
  cognitive control to a higher level (i.e. the
  knowledge level) than the demands of the
  situation requires.
• In contrast, agents have a propensity for
  behaviors based on skills and rules. These
  behaviors are gene-rally fast, effortless and
  propitious to a better co-ordination between
  agents.

27
Recap

• Communication and Semiosis are two sides of the
  same coin
• Knowledge Representation has tradition-ally only
  be treated in the context of solip-sistic
  systems. However, communication is constitutive
  for Knowledge (Represent-ation) Reasoning
  Reflection
• Agent Technology as a modeling metho-dology
  allows us to treat rather complex systems.
  Moreover, the semiotics perspec-tive sheds new
  light on issues concerning User Interfaces,
  Exploration Narratives, and all emerging kinds of
  New Media

28
Knowledge Landscape VRR

• A systemic approach via Agent-Oriented Modeling
  the development of agent-based tools, and
• An operational elaboration of the new concept of
  the Virtual Reference Room, by means of which
  contextualized access to heterogeneous objects
  can be realized, and more knowledge-based
  navigation by means of these contexts becomes
  feasible.

29
Virtual Reference Room

• Explanations about how to explore the
  collections, pro-ducts and services of the
  institution
• Delivery of information about actual services of
  the Reference Room that are in force
• Pointers to where bibliographical services may be
  found
• Pointers to relevant exhibitions and other
  relevant events in connection with their search
  questions and associated references
• Orientation maps to more autonomously explore the
  facilities of the Reference Room and around the
  particular institutional collections maintained

30
Virtual Reference Room (cont.)
31
Virtual Reference Room (cont.)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
The big picture
36
Conclusions w.r.t. I-MASS

• Short term research will address the precise
  kinds of enabling technologies (e.g., from
  information retrieval, ontological engineering
  and knowledge engineering) that the I-MASS system
  should incorporate to synthesize (configuring and
  presenting) pieces of information that seem to
  fulfill an apparently existing need for
  information at the side of the users
• Also research about semantical (and pragmatical)
  inter-operability will require much effort,
  especially as it contributes quite a lot to the
  systems ability to provide good answers
• Longer-term research must address how the
  cultural domain may be modeled by means of
  process models that capture the relevant insights
  of cultural processes (e.g., the rise and fall of
  the Roman Empire, or the Renais-sance).

37
Conclusions

• Leibniz (1646-1716) ambition was to awake the
  sleeping child in us all
• I-MASS forces us to use Knowledge Tech-nology to
  the utmost and to use societal metaphors as much
  as possible!