Reasoning about actions, change, events and time

1
Reasoning about actions, change, events and time

• Luiz Josué
• Rafael Amorim
• Reusing slides from Jacques Robin

2
Outline

• Non-monotonic reasoning
• When and where to reason about actions, change,
  event and time?
• Reasoning services and illustrative case study
• Roadmap of approach
• The Situation Calculus (SC)
• Key ideas
• Representation
• Reasoning
• The frame problem
• The ramification problem
• The qualification problem
• Limitations

• The Event Calculus
• Key ideas
• Normal logic programs and negation as failure
• Representation
• Reasoning
• Abductive logic programs and abductive planning
• Solution to the frame problem
• Limitations and comparison with SC
• Transaction Logic
• Key ideas
• Transaction logic program syntax
• Transaction logic program semantics
• Representing action and changes in transaction
  logic
• Limitations and comparison with SC, EC

3
Non-monotonic reasoning

• Classical logic reasoning is monotonic
• If KB f, then ?g, KB ? g f
• Inference engine only performs ask and tell to
  the KB, never retract
• Non-monotonic reasoning
• Allows KB f, and then KB ? g ? f
• Previously derived facts can be retracted upon
  arrival (for example from sensors) of new,
  conflicting evidence

4
Two orthogonal sources of non-monotonicity

• Ontological in non-stationary environment E
• KB must reflect environment changes as time goes
  by
• when fact f true in E(t) no longer true in
  E(t1)
• unless KB uses a historically cumulative
  knowledge representation scheme
• sentence s in KB(t) that represents f must
  retracted from KB(t1).
• and so must all sentences in KB(t) proven using
  s! (truth-maintenance)
• Epistemological in partially observable
  environment E
• Decision making with partial knowledge requires
  using abduction in addition to deduction
• KB must reflect changes of the agents beliefs
  as new evidence becomes available through sensing
• KB(t) D(t) ? A(t,H(t)), where
• D(t) are sentences derived purely deductively
  from percept sequence and KB(0)
• A(t,H(t)) are sentences derived using at least
  one abductive step relying on some hypothesis in
  H(t)
• When ?a ? D(t) and a ? A(t-1,H(t-1)) , a must be
  retracted from KB(t)
• and so must all sentences in A(t-1,H(t-1))
  derived using a!

5
When to reason about actions, change, events and
time?

• Non-stationary environments
• Sequential (only actions and change)
• Concurrent synchronous (actions, change and
  events)

...
...
State 1
State 2
State 4
State 5
State 3
Environment
Agent
Reasoning
Reasoning
Percept
Action
Percept
Action
6
When to reason about actions, change, events and
time?

• Non-stationary environments
• Concurrent asynchronous (actions, change, events
  and time)

...
State 1
State 2
State 4
State 3
State 5
State 6
Environment
Agent
Reasoning
Percept
Action
Percept
Action
Reasoning
7
Where to reason about actions, change, events
and time?
Environment
Automaton Agent
Percept interpreter percepts(t) ? model(t-1) ?
model(t)
Sensors
Environment model updater model(t-1) ?
model(t) model(t) ? model(t) actions(t) ?
model(t) ? model(t1)
Environment model (past and present)
Action chooser model(t) ? actions(t)
Actuators
8
Where to reason about actions, change, events
and time?
Environment
DeliberativeAgent
Percept interperter percepts(t) ? model(t-1) ?
model(t)
Sensors
Model of past and current environments
Environment model updater model(t-1) ?
model(t) model(t) ? model(t) do(action(t-1)) ?
model(t)
Goal updater model(t) ? goals(t-1) ? goals(t)
Goals
Future environments predictor model(t) ?
hyp(action(t)) ? hyp(model(t1)) model(t) ?
model(t1)
Model of hypothetical future environments
Action chooser result(action(t),...,action(tn
)) hyp(model(tn))? hyp(model(tn)) ? goal(t)
? do(action(t))
Actuators
9
Reasoning services

• For all services environment model decomposed
  in
• Eternals properties and relations not affected
  by actions, events or the passing of time
• Fluents properties and relations that change as
  result of the execution of an action, the
  occurrence of an event or the passing of time
• Single action (event) consequence
• Given current environment state model
• Compute fluents resulting from the execution
  (occurrence) of a single action (event)
• Temporal projection
• Given current environment state model
• and a sequence of hypothetical actions to
  execute and events to occur
• Compute fluents of the resulting environment
  state
• Planning
• Given current environment model
• and a set of goal fluents
• Compute action sequence whose execution will
  turn these goal fluents true

10
Roadmap of the approaches situation and event
calculi
Full Classical First-Order Logic (FCFOL)
(Pure) Definite Logic Program (DLP)
11
Roadmap of the approaches transaction logic
Normal Logic Program (NLP)
Tabled Prolog Engine Negative abduction

• added construct
• connective Negation As Failure (NAF)

(Pure) Definite Logic Program (DLP)
12
Illustrative case study the block world

• Agent robot with one arm building stacks from
  blocks on a table
• Eternals
• isclear(table)
• block(B)
• Fluents
• on(Block,Loc)
• isclear(Loc)
• above(B1,B2)
• Action safeMove(B,L1,L2)
• preconditions
• on(B,L1) ? isclear(L2) ? isclear(B)
• intended effects
• ?on(B,L1) ? on(B,L2)
• side effects
• block(L2) ? ?isclear(L2)

• Action riskyMove(B,L1,L2)
• preconditions
• on(B,L1) ? isclear(L2)
• intended effects
• ?on(B,L1) ? on(B,L2)
• side effects
• ?X, above(X,B) ? (?above(X,L1) ?
  above(X,L2))
• block(L2) ? ?isclear(L2)

13
Situation Calculus

• Specific way to describe changes in first order
  logic to reason about actions and your effects
• It conceives of the world as consisting of a
  sequence of situations, each of which is a
  "snapshot of the state of the world.
• Situations are generated from previous situations
  by actions
• The world is the sequence of situations linked by
  actions.

14
Ontology of the Situation Calculus
FCFOL NonFunctionalTerm
FCFOLFormula
FCFOLAtomicFormula
FCFOLTerm
FCFOLFunctionalTerm
PosEffect Atom
Result Term
Eternal Atom
Fluent Term
Eternal Term
Action Term
Precond Atom
Effect Atom
NegEffect Atom
Situation
Effect Axiom
Precond Axiom
Fluent Atom
Frame Axiom
Successor State Axiom
Ramification Axiom
15
Precondition and effect axioms

• Precondition axioms
• ?B,L1,L2,S block(B) ? on(B,L1,S) ? isclear(L2,S)
  ? isclear(B,S)
• ? precond(safeMove(B,L1,L2)
  ,S)
• ?B,L1,L2,S block(B) ? on(B,L1,S) ?
  isclear(L2,S) ?
  precond(riskyMove(B,L1,L2),S)
• Effect axioms
• ?B,L1,L2,S,S1 precond(safeMove(B,L1,L2),S)
  ? (S1 result(safeMove(B,L1,L2
  ),S) ??(S1 S) ?
  ?on(B,L1,S1) ? on(B,L2,S1)
  ? (block(L2) ? ?isclear(L2,S1)))
• ?B,L1,L2,S,S1 precond(riskyMove(B,L1,L2),S)
  ? (S1 result(riskyMove(B,L1,
  L2),S) ? ?(S1 S)
  ? ?on(B,L1,S1) ? on(B,L2,S1)
  ? (block(L2) ? ?isclear(L2,S1)))

16
The frame problem

• In most non-stationary environments
• Most fluents remain the same most of the time
• Each action only affects a tiny percentage of
  all fluents
• Effect axioms only specify the fluents changed by
  an action
• But the execution of an action changes the
  situation
• How to specify that fluents unchanged by the last
  executed action carry over to the new situation
  that it created?
• Naive approach frame axioms
• ?B,L1,L2,S,S1,B1,B2 precond(safeMove(B,L1,L2)
  ,S) ? on(B1,B2,S) ? ?(B B1) ? ?(B B2) ?
  (S1 result(safeMove(B,L1,L2),S) ? ?(S1 S) ?
  on(B1,B2,S1)
• Leads to combinatorial number of frame axioms

17
Solving the frame problem in the situation
calculus

• Substituting implicative effect and frame axioms
  by successor state axiom using equivalence
• Successor state axiom schema
• (precond(A,S) ? (fi(result(A,S)) ?
  ((posEffect(A,fi))
  
  ? fi(S) ?
  ?negEffect(A,fi))) ? .... ? posEffect(aj,fi) ?
  ... ? negEffect(ak,fi) ? ...
• where S is a situation, A,aj,ak an action and fi
  are fluents
• Example
• (?A,B,L,S precond(A,S) ? (on(B,L,result(A,S)
  ) ? posEffect(A,on(B,L))
  ? (on(B,L,S) ? ?negEffect(A,on(B,L
  )))) ? ?A,B,L1,L2,S posEffect(safeMove(B,L1,L2),
  on(B,L2)) ? ?A,B,L1,L2,S posEffect(safeMove(B,L1,
  L2), isclear(L1)) ? ?A,B,L1,L2,S
  negEffect(safeMove(B,L1,L2), on(B,L1)) ?
  ?A,B,L1,L2,S negEffect(safeMove(B,L1,L2),
  isclear(L2)) ? ?A,B,L1,L2,S posEffect(riskyMove(B
  ,L1,L2), on(B,L1)) ...

18
The ramification problem

• Actions have
• intended effects that satisfy the agents goal
  and justify their execution
• side effects related to preconditions of other
  actions
• Modeling side effects together with intended ones
  in successor state axioms link them to actions
  leading to combinatorial explosion
• Side effects best modeled orthogonally as
  ramification axioms that relate fluents among
  themselves independently of actions in the same
  situation
• Example
• ?B1,B2,B3,S above(B1,B2,S) ?
  (on(B1,B2,S) ? (on(B1,B3,S) ? above(B3,B2,S)))
• Execution of riskyMove(b,c) in S-1 w/
  above(a,b,S-1)
• Successor state axiom allows deducing on(b,c,S)
  ? above(a,b,S)
• Ramification axiom then allows deducing
  above(a,c,S)

19
The qualification problem

• Because it is based on classical logic, the
  situation calculus requires explicitly modeling
  all preconditions and all side-effects of every
  action in every situation
• In practice, rarely occurring combinations of
  fluents are easily overlooked when specifying
  preconditions and side-effects
• This is the qualification problem think of
  everything that can go wrong
• Best solved by probabilistic knowledge
  representation, aggregating all unforeseen cases
  in one catch-all situation

20
Unique name axioms

• Being based on classical logic, a KB using the
  situation calculus must contain unique name
  axioms stating that no two members of the
  Herbrand base are equal unless explicitly
  specified
• Example
• ?a b ? ?a c ? ?a table ? ...
• ... ? (?B,L1,L2,S,B,L1,L2,S
  safeMove(B,L1,L2,S) safeMove(B1,L1,L2,S) ?
  (B B1, L1 L1, L2 L2, S S)) ? ...

21
Limitations of the situation calculus

• Expressiveness
• Punctual and implicit representation of time by
  way of situations
• All changes result from a single agent executing
  actions
• Limited to sequential, mono-agent environments
• Non-intuitive for being based on full classical
  first-order logic
• Efficiency
• Very space inefficient
• Guards the entire history of the environment
• If a fluent changed only once in a 2001 steps
  process, the knowledge base contains 2000 copies
  of it, each one w/ a different situation argument
• Time inefficient
• Reasoning relies on theorem proving in full
  classical first-order logic
• Determining the truth of a fluent requires
  regressing through the entire history of the
  environment

22
The Event Calculus (EC)

• Exists in various version of increasing
  expressiveness
• Most expressive versions overcome all the
  expressiveness limitations of the SC except the
  qualification problem
• Also more time efficient than the SC for relying
  on abductive logic programming in Horn
  first-order logic instead of theorem proving in
  classical first-order logic
• Explicitly represents time points, time
  intervals, punctual events, events with
  durations, sequential and concurrent events
• An agent executing an action is a special case of
  event

not(holds(Fluent))
holds(Fluent)
not(holds(Fluent))
t
terminates(Event,Fluent)
initiates(Event,Fluent)
23
General logic programs

• General (or normal) logic programs extend pure
  (or definite) logic programs with Negation As
  Failure (NAF) connective not that can precede
  atoms in a clause premise
• Inference engine is extended with step to prove
  premise not A
• Generally it tries to prove goal A
• if it succeeds, then not A fails
• if it finitely fails to do so, it assumes A
  false (negatively abduces A) and not A succeeds
• abducing not A from failure to prove A it is a
  completely different inference than deductively
  proving ?A in classical first-order logic
• Tabled logic programming engine can detect loops
  through NAF such as query q with program p -
  not q. q - not p.
• It computes the Well-Founded Semantics (WFS) in
  ternary logic
• When A neither succeeds nor can it finitely fail
  due to NAF loops, the WFS assigned truth value
  unknown to both A and not A

24
Representing a domain with EC

• Domain-dependent knowledge represented as
  abductive logic program clauses using EC
  predicates
• holds_at(F,T) fluent F is true at time T
• precond(E,T) precondition of event E are
  fulfilled at time T
• do(A,T) action A is executed at time T
• happens(E,T) event E occurs at time T
• initiates(E,F,T) fluent F becomes true as a
  positive effect of event E
• terminates(E,F,T) fluent F becomes false as a
  negative effect of event E
• clipped(F,T1,T2) fluent F becomes false between
  time T1 and time T2
• Domain-independent EC clauses that specify
  generic relations between EC predicates, that
  axiomatize their temporal reasoning semantics
• holds_at(F,T2) - happens(E,T1),
  initiates(E,F,T1), T1 ? T2,
  not clipped(F,T1,T2).
• clipped(F,T1,T3) - happens(E,T2),
  terminates(E,F,T2),
  T1 ? T2, T2 ? T3.
• happens(E,T) - precond(E,T), not action(E).
• happens(E,T) - precond(E,T), action(E), do(E,T).

25
Ontology of the Event Calculus

Literal
Premisse

PALPClause

Conclusion

CFOLAtomicFormula
Fact
IntegrityConstraint

Functor
CFOLTerm
ECCLause
PredicateSymbol
Holds
Happens
Abducible
Precond
CFOLNonFunctionalTerm
Initiates
Do
Terminates
Functor
Clipped
ConstantSymbol
CFOLVariable
FunctionSymbol
26
Block world EC clauses

• Initial state clauses
• holds_at(on(a,b),0). holds_at(on(b,c),0).
  holds_at(on(c,table),0). holds_at(isclear(a),0).
• Precondition clauses
• precond(safeMove(B,L1,L2),T) -
  holds_at(on(B,L1),T),
• 
  holds_at(isclear(L2),T),
• 
  holds_at(isclear(B),T).
• Positive effect clauses
• initiates(safeMove(B,L1,L2), on(B,L2),T).
• initiates(safeMove(B,L1,L2), isclear(L1), T).
• Negative effect clauses
• terminates(safeMove(B,L1,L2), on(B,L1),T).
• terminates(safeMove(B,L1,L2), isclear(L2), T).
• Ramification clauses
• holds_at(above(B1,B2),T) - holds_at(on(B1,B2),T)
  .
• holds_at(above(B1,B3),T) - holds_at(on(B1,B2),T)
  , holds_at(above(B2,B3),T).

27
CHR example

• 1. facts gt block(a),block(b),block(c),
• holds_at(on(b,l1),0),holds_at(on(c,l2),0
  ),holds_at(on(a,b),0),
• holds_at(not_on(b,l2),0),holds_at(not_on
  (a,l2),0),holds_at(not_on(c,l1),0),
• holds_at(not_on(a,c),0),holds_at(not_on(
  b,c),0),holds_at(not_on(c,a),0),
• holds_at(not_on(c,b),0),holds_at(not_on(
  b,a),0),holds_at(isclear(a),0),
• holds_at(isclear(c),0),holds_at(not_clea
  r(l1),0),holds_at(not_clear(l2),0).
• 2. precond(safe_move(B,L1,L2),T) ltgt
  holds_at(isclear(L2),T), holds_at(isclear(B),T),
• 
  holds_at(on(B,L1),T).
• 3. initiates(safe_move(B,L1,L2),on(B,L2),T) ltgt
  true.
• 4. initiates(safe_move(B,L1,L2),isclear(B,L1),T)
  ltgt true.
• 5. terminates(safe_move(B,L1,L2),on(B,L1),T) ltgt
  true.
• 6. terminates(safe_move(B,L1,L2),isclear(L2),T)
  ltgt true.
• 7. holds_at(above(B1,B2),T) ltgt
  holds_at(on(B1,B2),T) (holds_at(on(B1,B2),T),
  holds_at(above(B2,B3),T)).
• 8. happens(E,T) ltgt precond(E,T), not action(E) 
  precond(E,T), action(E), do(E,T).
• 9. clipped(F,T1,T3) ltgt happens(E,T2),
  terminates(E,F,T2), T1 ? T2, T2 ? T3.

28
Solution to the frame problem

• Predicate completion
• 1. Initiates(Load,Loaded,t)
• 2. Initiates(Shoot,Dead,t) ? HoldsAt(Loaded,t)
• 3. Terminates(Shoot,Alive,t) ? HoldsAt(Loaded,t)
• 4. Initially(Alive)
• 5. Happens(Load,T1)
• 6. Happens(Sneeze,T2)
• 7. Happens(Shoot,T3)
• 8. T1 lt T2
• 9. T2 lt T3
• 10. T3 lt T4
• The Sneeze event doesnt unload the gun
• Replace 1, 2 and 3 by
• Initiates(a,f,t) lt--gt a Load f Loaded v
• a Shoot f
  Dead HoldsAt(Loaded,t)
• Terminates(a,f,t) lt--gt a Shoot f Dead
  HoldsAt(Loaded,t)
• Replace 5, 6 na 7 by
• Happens(a,t) lt--gt a Load t T1 v a
  Sneeze t T2 v a Shoot t T3

29
Variations of the EC

• Basic EC already handles concurrent actions and
  time intervals
• Additional arguments to EC predicates and
  additional EC clauses allow extending basic EC to
  handle
• Events with durations happens(E,T1,T2)
• Multiple agents do(Agent,Action,T)

30
EC vs. SC

• Most expressive versions overcome all the
  expressiveness limitations of the SC except the
  ramification problem
• Also more time efficient than the SC for relying
  on abductive logic programming in Horn
  first-order logic instead of theorem proving in
  classical first-order logic

31
Abductive logic programming

• Extend general logic programs with
• Integrity constraints I
• A distinguished subset of predicate symbols
  called abducibles
• Atoms with an abducible functors (abducible
  atoms) are generally restricted to appear only in
  clauses premises
• they have no definitions where they appear in
  conclusion
• An abducible atom goal is assumed true (positive
  abduction) whenever
• Its assumption does not allow deducing
  conclusions that violate the integrity
  constraints
• Abduction can be used to construct plans using an
  EC representation of the domain D
• holds atoms represent the goal G to be reached
• do atoms represent the hypothesis H to abduce
• The abductive proof procedure computes H such
  that
• D ? H G
• D ? H ? I ? false

32
Transaction Logic (TL) starting point

• A set of Prolog clauses defining a predicate have
  two distinct semantics
• A declarative, logical one (Clarks completion or
  least Herbrand model)
• A procedural one (implicit)
• Example
• p - q, r.
• p - u.
• Declarative semantics p ? ((q ? r) ? u)
• Procedural semantics
• define p call q if q
  true then call r return r
  else return fail call u
  return u
• In the declarative semantics the order of q, r
  and u does not matter
• In the procedural semantics, the order is crucial
• Transaction logic unifies the two semantics by
  representing the procedural one explicitly and
  declaratively

33
Ontology of the Transaction Logic
conditionExp
0..1
conclusion
1
0..1
formula
atom
Query
LogicalPredicativeATom
1..2
1..3
0..
premise
LogicalBacktrackableUpdateAtom
0..1
Operator
0..1
connective
0..1
0..1
Clause
TransactionalConnective
Program
34
Transaction logic connectives

• Extends normal logic programs with backtrackable
  clause update predicates and two new connectives
• btinsertC1, ..., Cn premise has side effect to
  explicitly add clauses C1, ..., Cn to the logic
  program
• btdeleteC1, ..., Cn premise has side effect to
  explicitly delete all clauses that match C1 or
  ... or Cn from the logic program
• Serial conjunction ?
• Declaratively captures procedural semantics of
  Prologs conjunction
• Distinct from classical conjunction
• In TL a, b ? b, a, but a ? b ? b ? a
• Serial disjunction ?
• Declaratively captures procedural semantics of
  Prologs disjunction
• Distinct from classical disjunction
• In TL a b ? b a, but a ? b ? b ? a
• Transaction semantics
• If one element of a serial conjunctive premise
  fails
• ex, r fails in c - btinsert(a) ? btdelete(b) ?
  r ? s
• then b is put back in the KB and a is retracted
  from it

35
Ontology of the Transaction Logic Operators
TransactionalConnective
ImperativeConnective
ModalConnective
GeneralConnective
SerialConnective
SerialDisjunction
SerialConjunction
LoopConnective
ConditionalConnective
HypoteticalConnective
RetrospectiveConnective
DoUntil
RetrospectivePossibility
RetrospectiveNecessity
HypoteticalPossibility
HypoteticalNecessity
WhileDo
IfThen
UnlessDo
IfThenElse
LoopUntil
36
The block world in TL

• Initial state clauses
• block(a). block(b). block(c). on(a,b).
  on(b,table). on(c,table). isclear(table).
• Ramification clauses
• isclear(B) - block(B), not(on(C,B)).
• above(B1,B3) - on(B1,B2) (on(B1,B2),
  above(B2,B3)).
• Precondition clauses
• preconds(safeMove(B,L1,L2)) - on(B,L1),
  isclear(B), isclear(L2).
• Effect clauses
• effects(safeMove(B,L1,L2)) - btdelete(on(B,L1))
  ? btinsert(on(B,L2)).
• Action execution clauses
• do(A) - preconds(A) ? effects(A).
• Planning clauses
• achieve(Goal) - Goal.
• achieve(on(B,L)) - do(safeMove(B,_,L)).

37
TL x EC x SC

• TL far more space and time efficient than EC and
  SC
• Because it does not keep the history of the
  environment
• It only maintains its current state
• But this history is available in the trace of
  its proof mechanism that can be backtracked upon
  demand
• Like SC and unlike EC, TL does not explicitly
  represent time points and durations
• With additional arguments and clauses it does
  concisely and efficiently support planning and
  temporal projection
• Complex, chained ramifications require
  truth-maintenance which is not supported as
  built-in by available TL engines