An Agent Oriented Programming Language integrating Temporal Planning and the Plan Coordination Mechanisms

1
An Agent Oriented Programming Language
integrating Temporal Planning and the Plan
Coordination Mechanisms

• Muhammad Adnan HASHMI

Thesis Supervisor Amal EL FALLAH SEGHROUCHNI
2
Objectives
AOPL Planning
Planning Coordination
(Temporal Planning, Reactivity, Dynamic
Environments)
(Temporal Planning, Different Priority Goals)
AOPL integrating Planning, Plan Repairing and
Coordination
3
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

4
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

5
AOP Languages

• Allow to program intelligent and autonomous
  agents
• Main Characteristics
• Mental State Beliefs, Goals, Commitments
• Reasoning Mechanism
• Capabilities, Services
• Communication
• Concurrence
• Some Languages
• Agent-0 Shoham 1993, 2APL Dastani 2008,
  AgentSpeak (L) Rao 1996

6
Objectives (AOP Languages)

• Current languages
• Most AOP languages follow a reactive (PRS based)
  approach
• AOP languages do not support temporal planning
• Only a few support planning
• Problems
• Execution without planning may result in the goal
  failures
• Agent can reach a dead end
• Conflicts can arise among different plans
• Real world actions take place over a timespan

7
Objectives (AOP Languages)

• Current languages
• Most AOP languages follow a reactive (PRS based)
  approach
• AOP languages do not support temporal planning
• Only a few support planning
• Our aim
• Propose an extension to a programming language to
  endow it with planning skills
• Has temporal planning
• Deals with uncertainty of the environment
• Incorporate reactivity by dealing with on the fly
  goals having different priorities

8
Related Work

• CANPLAN De Silva et al. 2005 combines JSHOP
  with JACK
• Programmers responsibility to call the planner
• Doesnt deal with uncertainty of the environment
• X-BDI Meneguzzi et al. 2004 incorporates
  classical planning in a BDI framework
• Efficiency concerns
• Loss of the domain information
• CYPRESS Myers et al. 1994 integrates SIPE-2
  with PRS

9
Plans Coordination

• Planning for single agents
• Computing plan from Initial State to Goal State
• Coordination of plans
• Removing conflicts (negative interactions)
• Utilizing help relations (positive interactions)

a b c
d e
q
A1
A2
G
I
10
Related Work (Plans Coordination)

• Coordination before planning
• Social Laws Shoham 1995, Buzzing 2006
• Coordination during planning
• Partial Global Planning Durfee and Lesser 1987
• Incremental Plan Merging Alami et al. 1994
• Recursive Petri Nets El Fallah Seghrouchni and
  Haddad 1996
• Coordination after planning
• Temporal Plan Merging Tsamardinos et al. 2000
• Merging Hierarchical Plans von Martial 1992

11
Objectives (Plans Coordination)

• Propose plans coordination mechanisms for the
  plans having different priorities
• Two different scenarios
• Coordination during planning
• Coordinated Planning Problem (CPP)
• Coordination after planning
• Proactive-Reactive Coordination Problem (PRCP)

12
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

13
Assumptions

• Two agents a and ß sharing the same environment
• Agent a having higher priority (reactive) goals
• Agent ß having normal priority (proactive) goals
• Two possible conflicts among plans
• Causal link threat
• Parallel actions interference

14
Two Possible Conflicts

• Causal Link (A1, A2, p)
• Action A1 adds an effect p
• Action A2 needs this effect
• No action between A1 and A2 adding p
• Causal Link Threat
• If an action A deletes p and lies between A1 and
  A2, then A threatens the causal link (A1, A2, p)

p
A1
A2
Threat
A
p
15
Two Possible Conflicts

• Causal Link (A1, A2, p)
• Action A1 adds an effect p
• Action A2 needs this effect
• No action between A1 and A2 adding p
• Parallel Actions Interference
• Actions A1 and A2 lie in parallel
• Either one of them deletes the preconditions or
  add effects of the other

A1
p
A1
p
p
A2
p
A2
16
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

17
Coordinated Planning Problem

• Prerequisite
• Plan Pa of Agent a
• Our Aim
• Compute a Plan Pß for Agent ß
• Has no conflict with Pa
• Avails the cooperative opportunities offered by
  Pa
• Solution
• Non Temporal Domains ? µ-SATPLAN
• Temporal Domains ? Coordinated-Sapa

18
Coordinated-Sapa

• Our extension of the well-known temporal planner
  Sapa M. Do S. Kambhampati 2001
• Input
• Initial State I
• A Set of Actions D
• A Set of Proactive Goals GP with deadlines
• A Reactive Plan PR
• Output
• A plan PP from I to GP
• PP is consistent with PR
• OR
• Deadline Violated ? Failure Goal name whose
  deadline is violated
• OR
• No Solution Possible ? Failure

19
SAPADo and Kambhampati 2001

• A Multi-Objective Heuristic Based Temporal Planner

20
Sapa
Search through the space of time-stamped states
S(P,M,?,Q,t)
21
Sapa
Search through the space of time-stamped states
S(P,M,?,Q,t)
22
Main Flow-Diagram of SAPA
S(P,M,?,Q,t)
S(P I, M, ? , Q f, t 0)
23
Some Important Concepts
S(P,M,?,Q,t)

• Goal Satisfaction
• S(P,M,?,Q,t) ? G if ?ltpi,tigt? G either
• ? ltpi,tjgt ? P, tj lt ti and no event in Q deletes
  pi
• ? e ? Q that adds pi at time te lt ti
• Action Application
• Action A is applicable in S (P,M,?,Q,t) if
• All preconditions of A are satisfied by P
• As effects do not interfere with Q
• When A is applied to S (P,M,?,Q,t)
• P is updated according to As instantaneous
  effects
• Delayed effects of A are put in Q.
• Special Advance-Time Action
• Advances time to next earliest event in the
  queue, and adds the event to P

24
Coordinated-Sapa

• An Extension of Sapa that Finds Coordinated Plans
  in Temporal Domains

25
Handling Positive Interactions

• Before starting planning for Agent ß
• Create an event e (st, et, effect)
  corresponding to every effect k of every action A
  in Pa
• e.st st(A)
• e.et et(A)
• e.effect k
• Put all these events in the event queue Q
• WHY? At every state, Coordinated-Sapa will take
  into account all the changes made by Agent a
• Advanced-Time action will add the effects
  generated by Agent a, to P

26
Handling Negative Interactions
S(P,M,?,Q,t)

• Add another action applicability condition to the
  planning mechanism of Agent ß
• Action Application
• Action A is applicable in S(P,M,?,Q,t) if
• All preconditions of A are satisfied by P
• As effects do not interfere with Q
• A does not threaten any causal link at t

27
Example Plan Generated

• Agent ß drives person P4 in Car1 to city C2
• So that P4 can board the plane Pl1
• Agent a is bringing the plane Pl1 to C2 from C3
• Agent ß makes person P4 board the plane Pl1 at C2
• Agent a flies plane Pl1 from C2 to C5

28
Experimental Results

• Domains and problems taken from 3rd International
  Planning Competition
• A multi-agent problem is generated by taking the
  original problem and dividing the goals in two
  sets, one for each agent

29
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

30
Proactive-Reactive Coordination

• Prerequisite
• Reactive plan Pa of Agent a
• Proactive plan Pß of Agent ß
• Our Aim
• Modify plan Pß such that
• It has no conflict with Pa
• Avails the cooperative opportunities offered by
  Pa
• Solution
• Plan Merging Algorithm

31
Case Study
Rescue Agent a
Analyzer Agent ß

• Tasks of Rescue Agent
• Rescue the Victims
• Tasks of Analyzer Agent
• Analyze the goal cells
• Call the central agent
• Constraints
• One agent in a cell
• Hyper energy cells
• Needs fuel or energy to enter
• Agent should have key to open door

32
Conflict Resolution

• Threat-Repair Link (A1, A2, p)
• Action A1 deletes p
• A2 is a subsequent action and adds p
• A1 is called Threat Action
• A2 is called Repair Action

p
B1
B2
Threat
A1
-p
A2
p
Repair
33
Valid and Possibly Valid Time Stamps

• Possibly Valid Time Slot for an action A
• All preconditions are met
• No parallel actions interference

P1
? a
b h
P2
? b
c -d
P3
? c
e
P4
? e
f
P5
? f
g
P6
? g
i
P7

• i
• h

g
P1
? b
d -h

• Valid Time Slot for an action A
• All preconditions are met
• No parallel actions interference
• Either
• No causal link threat
• Repair Action exist before the deadline

P1
? a
b h
P2
? b
c -d
P3
? c
e
P4
? e
f
P5
? f
g
P6
? g
i
P7

• i
• h

g
P1
? b
d -h
P2
? d
k
P3

• k

h
34
Plan Merging Algorithm

• Fix all the actions of Reactive Plan Pa on
  timeline
• For every action CA of Proactive Plan
• Search for the first Possibly Valid Time Slot T
  on timeline
• Reason about the time slot T
• There could be 5 cases at T

35
Plan Merging Algorithm

• Case 1 No causal link threat by CA at T
• Assign Time Slot T to CA
• EXAMPLE
• Current Action Move(A1, A2)
• Returned Time Slot 0 - 5
• Any Threat? No
• Assign Time Slot 0 5 to CA

36
Plan Merging Algorithm

• Case 2 CA threatens a Causal Link but Repair
  Action exist
• Assign Time Slot T to CA
• Save a Possible Threat ltThreatAction,
  RepairAction, Deadlinegt
  
• EXAMPLE
• Current Action Move(A4, A5)
• Time Slot 20 - 25
• Any Threat? Yes (Agent a needs A5 at time
  40-45)
• Repair Action Move(A5, A6)
• Assign Time Slot 20 - 25 to Move (A4, A5)
• Save ltMove(A4, A5), Move(A5, A6), 40gt

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
B
C
D
37
Plan Merging Algorithm

• Case 3 It is a Repair Action but can not meet a
  deadline of some Threat Action
• Backtrack to the Threat Action,find another time
  stamp
• EXAMPLE
• Current Action Move (A8, A9)
• Returned Time Slot 50 - 55
• Any Threat? Yes (Agent a needs A9 at 85-110)
• Repair Action Move (A9, B9)
• Save ltMove(A8, A9), Move(A9, B9), 85gt
• Next Action AnalyzeCell (A9)
• Time Slot Assigned 55 - 70

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
B
C
D
38
Plan Merging Algorithm

• Next Action CallCentral (A9)
• Time Slot Assigned 80 90
• Next Action Move (A9, B9)
• Is it a Repair Action? Yes
• Meet all deadlines? No (Agent a needs A9 at 85)
• Backtrack to action Move(A8, A9)
• Find another Time Slot
• New Time Slot 110 115 (Valid Time Slot)

Attention
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
B
C
D
39
Plan Merging Algorithm

• Case 4 All the effects of CA are already
  achieved
• WHAT TO DO?
• Mark CA as redundant
• POST PROCESSING
• Remove all redundant actions from the plan
• Recursively remove all actions which achieve only
  the preconditions of removed action

40
Plan Merging Algorithm

• EXAMPLE
• Current Action OpenDoor (C11)
• Returned Time Slot 172 - 175
• Redundant(OpenDoor(C11)) ? true
• Because openedDoor(C11) is true at time 172
• When the plan is returned
• Remove OpenDoor(C11) from plan
• Also remove TakeKey(C11, key1) from plan

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
B
C
D
41
Plan Merging Algorithm

• Case 5 Action CAs preconditions can not be
  achieved
• Remove action CA from the plan and compute a plan
  to achieve effects of CA
• I State just before CA
• G Effects (CA)
• Plan should have no conflict with Reactive Plan
  Pß and if CA is a repair action, repair effects
  must meet their deadline
• ReplacementPlan Coordinated-Sapa (I, G, Pß)
• If a plan is returned, replace the removed
  actions with the plan
• If a deadline is violated, backtrack to the
  threat action
• If no plan possible, then remove another action
  CA 1
• G G U Effects (CA 1) \ Pre (CA 1)

Use Coordinated-Sapa
42
Plan Merging Algorithm

• EXAMPLE
• Current Action TakeEnergy(B13, energy1)
• Preconditions can not be achieved
• Repair the plan

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
B
C
D
43
Plan Repair Algorithm

• Create a CPP by removing TakeEnergy(B13, energy1)
• I at(ß, B13), at(energy1, B13), at(energy2,
  B13)
• G hasEnergy(ß, energy), at(ß, B13)
• Call Coordinated-Sapa to solve this CPP
• Coordinated-Sapa returns fail
• Why? energy2 is also needed by Agent a

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
B
C
D
44
Plan Repair Algorithm

• Create another CPP by removing Move(B13, A12)
• I at(ß, B13), at(energy1, B13), at(energy2,
  C15)
• G at(ß, A12)
• Call Coordinated-Sapa to solve this CPP
• A plan is returned to enter A12 by taking the
  fuel from D14
• POST PROCESSING
• This plan will become a replacement for both
  TakeEnergy(B13, energy1) and Move(B13, A12)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
B
C
D
45
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

46
CLAIM Suna and El Fallah Seghrouchni 2005

• An AOP language having
• Cognitive aspects specific to intelligent agents
• Communication primitives
• Mobility primitives
• CLAIM Agent
• Is autonomous, intelligent and mobile
• Has a mental state containing knowledge, goals,
  and capabilities
• Is able to communicate with other agents
• Entails a reactive behaviour

47
Agent Definition in CLAIM

• defineAgent agentName
• parent null agentName
• knowledge null knowledge1
  knowledgem
• goals null goal1 goaln
• capabilities null capability1
  capabilityo
• agents null agName1, agName2, ,
  agNameq

capability name message null
message conditions null condition
do process effects null effect1
effectf
48
P-CLAIM

• An AOP language having
• Cognitive aspects specific to intelligent agents
• Communication primitives
• Mobility primitives
• Temporal planning capability (New)
• P-CLAIM Agent
• Is autonomous, intelligent and mobile
• Has a mental state containing knowledge, goals,
  and capabilities
• Is able to communicate with other agents
• Entails a planning based behaviour (New)
• Achieves goals based on their priorities (New)
• Maintains the stability of the plan in the
  dynamic environments (New)

49
Agent Definition in P-CLAIM

• Similar to CLAIM, but
• Added priorities to goals
• High Preemptive (reactive) Should be immediately
  planned for and achieved
• High and Normal (proactive) High priority goals
  should be achieved before normal priority goals
• Capabilities in CLAIM ? (Activities Actions) in
  P-CLAIM
• Activities are short plans to achieve tasks
• Actions are primitive tasks with duration

50
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

51
Agent Definition to Planning (Translator)

Agent Description File
Knowledge
Goals
Activities
Actions
Translator (JavaCC)

Initial State
Goals
Methods
Operators
Problem File
Domain File
Planner
52
Agent Life Cycle
53
Messages Handler
54
Messages Handler

• Waits for messages from other agents
• Message is a request to achieve a goal?
• Assigns priority to the goal
• High Preemptive ? puts it in Global Reactive
  Goals (GRG) queue
• High or Normal ? puts it in Global Proactive
  Goals (GPG) priority queue
• Message is an information?
• Store the information in the knowledge base of
  the agent

55
Goal Priority Assignment
Message Priority Senders Importance High Normal
High Preemptive High Preemptive High Preemptive High
High High High Normal
Normal Normal Normal Normal
56
Planner
57
2- GPG Goals are accessed only when GRG is empty
Planner
GRG
GPG
1- Fetch goals one by one from GRG and GPG and
calls Compute_Plan to compute a plan for the goal
Main Algorithm
3- Sends a suspension signal to Executor if the
goal is reactive i.e. from GRG
Compute Plan
Temporal Converter
58
Plan Computation

• JSHOP2 algorithm Nau et el. 2003 is used to
  compute a totally ordered plan for each goal
• An HTN planning algorithm
• Decomposes the task into sub-tasks by applying
  methods
• Recursively applies the same procedure on every
  composite sub-task until there are only primitive
  tasks

T1
M1
T11
T12
M12
M11
59
Temporal Converter

• Input to procedure
• A totally ordered plan
• Actions information (Add, Del, Pre, Durations)
• Output of procedure
• A position constrained parallel plan
• Every action is assigned a time stamp
• Multiple actions can possibly lie in parallel

60
Temporal Converter (Example)
I a, b, c
P1
?a
-a d e f
P2
?f
-f g h
P3
?e
i
P4
?h
-h j
P5
?g ?i
k
P6
?k
l
P7
?l
m
P8
?m
c
P9
?c ?m
-c
P10
?e
-k -c h
Dur(P1)15
Dur(P2)15
Dur(P3)15
Dur(P4)15
Dur(P5)15
Dur(P6)15
Dur(P7)15
Dur(P8)15
Dur(P9)15
Dur(P10)15
61
Temporal Converter (Example)
P10
?e
-k -c h
Action Under Consideration
P1
?a
-a d e f
P2
?f
-f g h
P4
?h
-h j
P6
?k
l
P7
?l
m
P8
?m
n c
P9
?c ?m ?n
-c
P3
?e
i
P5
?g ?i
k
P10
?e
-k -c h
Forbidden
Forbidden
6
5
7
MaxT
PreT
MinT
1
4
2
MinOrderingT
3
62
Temporal Converter (Example)
Input Plan
P1
?a
-a d e f
P2
?f
-f g h
P3
?e
i
P4
?h
-h j
P5
?g ?i
k
P6
?k
l
P7
?l
m
P8
?m
c
P9
?c ?m
-c
P10
?e
-k -c h
Output Plan
P1
?a
-a d e f
P2
?f
-f g h
P4
?h
-h j
P6
?k
l
P7
?l
m
P8
?m
n c
P9
?c ?m ?n
-c
P3
?e
i
P5
?g ?i
k
P10
?e
-k -c h
Makespan Gain 30
63
Experimental Results (Temporal Converter)
64
Merging the New Plan to Global Plan
65
Merging the New Plan to Global Plan
Planner
Proactive Goal
Reactive Goal
Append at the end of Pexec
Merge at the start of Pexec
Plan Under Execution (Pexec)
66
Schedule Handler and Executor
67
Schedule Handler and Executor
68
Plan Mender
69
Plan Mender
Compute Plan Using Sapa
Compute Plan Using Sapa
SW
ReplacementPlan
ContinuationPlan
G
SP(Pexec4)
No Plan
SP(Pexec3)
SP(Pexec2)
SP(Pexec1)
Pexecnew ReplacementPlan ContinuationPlan
70
Experimental Results (Plan Mender)

• To perform tests, we explicitly modified the
  knowledge base of the agent to cause plan failure

71
Outline

• Context and Objectives
• Plan Coordination Mechanisms
• Coordinated Planning Problem
• Proactive-Reactive Coordination Problem
• P-CLAIM AOP Language supporting Temporal
  Planning
• Language Definition
• Planning Mechanism
• Conclusion and Perspectives

72
Conclusion

• An agent oriented programming language
  supporting
• Temporal Planning
• Plan Repairing
• Dealing with different priority goals
• Coordinated Planning Problem
• Computing plan while coordinating it with another
  plan
• SATPLAN ? µ-SATPLAN
• Sapa ? Coordinated-Sapa
• Proactive-Reactive Coordination Problem
• Modifying a plan to remove conflicts with a
  higher priority plan
• Plan Merging Algorithm

73
Conclusion

• Properties of the temporal conversion algorithm
• Soundness (Proof By construction)
• Termination
• Properties of the plan merging algorithm
• Soundness (Verified by experimental evaluation)
• Computational Complexity
• Temporal Converter Quadratic
• Plan Merging Algorithm
• Worst Case Exponential
• Average Case Quadratic

74
Perspectives

• Coordination of plans for same priority goals
• Negotiation based strategy
• Planning with incomplete information
• Information gathering actions
• Improvement in the computational complexity of
  plan merging algorithm
• Propose efficient heuristics to reduce the number
  of backtracks

75
Publications

1. A. El Fallah Seghrouchni, M. A. Hashmi,
   Multi-Agent Planning, Book chapter in Software
   Agents, Agent Systems and Their Applications. M.
   Essaaidi et al. (Eds.), 2012, IOS Press.
2. Y. Dimopoulos, M. A. Hashmi, P. Moraitis,
   µ-SATPLAN Multi-Agent Planning as
   Satisfiability, Knowledge Based Systems Journal,
   2011
3. M. A. Hashmi, A. El Fallah Seghrouchni, Merging
   of Temporal Plans supported by Plan Repairing,
   Proceedings of 22nd IEEE International Conference
   on Tools with Artificial Intelligence (ICTAI
   2010), Aras, France
4. M. A. Hashmi, A. El Fallah Seghrouchni,
   Coordination of Temporal Plans for the Reactive
   and Proactive Goals, Proceedings of IEEE/WIC/ACM
   International Conference on Intelligent Agent
   Technology (IAT 10), Toronto, Canada
5. Y. Dimopoulos, M. A. Hashmi, P. Moraitis,
   Extending SATPLAN to Multiple Agents,
   Proceedings of 30th SGAI International Conference
   on Artificial Intelligence 2010, Cambridge, UK.
6. M. A. Hashmi, A. El Fallah Seghrouchni, Temporal
   Planning in Dynamic Environments for P-CLAIM
   Agents, In proceedings of Languages,
   Methodologies and Development Tools for
   Multi-Agent Systems (LADS09), Torino, Italy,
   Springer-Verlag.

76
Publications

1. M. A. Hashmi, A. El Fallah Seghrouchni, Temporal
   Planning in Dynamic Environments for Mobile
   Agents, Proceedings of International Conference
   on Frontiers of Information Technology (FIT 09),
   Abottabad, Pakistan, Publisher ACM press.
2. M. A. Hashmi, A Planning Component for CLAIM
   Agents, In the proceedings of 17th International
   Conference on Control Systems and Computer
   Science, Bucharest, Romania, Volume 2, pages
   485-492, Politehnica Press

77
Thanks for your attention!!!