Separating Concerns: Actors, Coordination Constraints, and Implementation Protocols in a Reflective Architecture

1
Separating Concerns Actors, Coordination
Constraints, and Implementation Protocols in a
Reflective Architecture

• Gul Agha
• University of Illinois at Urbana-Champaign
• http//osl.cs.uiuc.edu

2
Collaborators

• Mark Astley
• Svend Frolund
• YoungMin Kwon
• Ian Mason
• Jose Meseguer
• Shangping Ren
• Koushik Sen

• Scott Smith
• Dan Sturman
• Carolyn Talcott
• Prasanna Thati
• Nalini Venkatasubramanian
• Reza Ziaei

3
The Actor Model
A Model of Distributed Object Computation
State
Interface
Thread
Interface
Procedure
State
Thread
Messages
Procedure
Interface
State
Thread
Procedure
4
The Actor Model

• Actor system - collection of independent agents
  interacting via message passing
• Features
• Acquaintances - initial, created, acquired
• History Sensitive
• Asynchronous fair communication
• An actor can
• create a new actor and initialize its behavior
• send a message to an existing actor that is an
  acquaintance
• change its local state or behavior

5
Ticker Actor Specification

• States T(n) -- n a number
• Messages tick, time_at_c -- c an actor id
• Reaction Rules
• ( t T(n) ) t ? tick
• gt
• ( t T(n1) ) t ? tick
• ( t T(n) ) t ? time_at_c
• gt
• ( t T(n) ) c ? reply(n)

6
Ticker Actor Specification

• States T(n) -- n a number
• Messages tick, time_at_c -- c an actor id
• Reaction Rules
• ( t T(n) ) t ? tick
• gt
• ( t T(n1) ) t ? tick
• ( t T(n) ) t ? time_at_c
• gt
• ( t T(n) ) c ? reply(n)

7
Ticker Actor Scenario
dlv(t?tick)
... c ...
dlv(t?time_at_c)
8
The Actor Model
send
ready
create
9
Operational Semantics
Notion of equivalence
10
May versus Must Testing
11
Join Continuation
a2
a1
v1
v2

f (v1,v2)
12
Local Synchronization Constraints

• Recipient Actor may not be ready to accept a
  message in current state

13
Separate Synchronization Constraints

• Abstract Data Types separate the interface (what
  ) from the representation (how)
• Synchronization constraints express when in terms
  of what
• constrain get ( ) when empty?(buffer)
• constrain put ( ) when full?(buffer)
• Conditions are functions over the state.

14
Actor Languages

• Primitives extended by
• join continuations
• local synchronization constraints
• Synchronous communication
• Translated to primitive (kernel) actor language
• (see Mason Talcott U2K)
• Translation preserves actor semantics

15
How to Specify Actor Systems

• SpecDiagrams (Smith Talcott) graphical
  notation for specifying message passing behavior
  of open distributed object systems
• Draws upon concepts from various formalisms
• Dynamic name generation and name passing as in
  the pi-calculus
• Asynchronous communication and locality
  discipline on names as in concurrent object
  models such as Actors
• Imperative notions such as variables,
  environments, and assignments
• Logical features such as assertions and
  constraints, which are appropriate for
  specification languages.

16
SpecDiagrams
17
SpecDiagrams Syntax

• D 0 ae a(x)D D1 // D2 (? a) D
  recXD X (asynch p)
• D1D2 D1 D2 fork(D) (control)
• ? D x e
• pick(x) e (imperative)
• pick(x) D wait(Ø) (logical)
• Let rcp(D) be the set of all free names in D that
  occur as an input subject.
• Uniqueness of names If D1 D2 are two top-level
  diagrams, then rcp(D1) n rcp(D2) Ø

18
Example Memory Cell Specification
19
Example Specification of Two Tickers
20
A UML Specification and Its SpecDiagram
21
Refining Specifications
22
Reasoning about SpecDiagrams

• Trace-based equivalence relation for may-testing
• Executable Semantics in Term Rewriting
• (Thati, Agha Talcott 2004)

23
Policies as Coordination Constraints

• Local synchronization constraints express
  constraints over the history of single actor
• Synchronization between actors
• precedence of actions
• atomic actions
• real-time constraints
• Example two robots with a shared task
• Express coordination constraints modularly
• Real-time constraints

24
Coordinating Robot Arms
Synchronizer SimulRobot (actor leftHand,
rightHand) Constrain lhand.release
( ) rhand.grasp ( )
25
Transforming Constraints

• Dynamically customize schedulers

S. Frolund, Coordinating Distributed Objects,
MIT Press
26
Abstracting Interaction

• Interaction may involve more than synchronization
  constraints
• Separate interaction policy from protocols used
  to implement policy

27
What is a Protocol?

• A protocol defines a role for each participating
  actor relative to the protocol

• Primary Backup
• Periodic Checkpoint

28
Specifying Protocols

• Protocol Auragen-backup
• parameter declarations
• requirements
• protocol methods and initialization
• role definitions

29
Protocol Instances

• Protocol instances are runtime instantiatons of
  protocols
• A participant actor has a role in a protocol
• An actor may participate in multiple protocols
• Protocol methods may be invoked on instances

30
Transparent Implementation of Protocols

• Reflection enables protocol implementations to be
  independent of application implementations
• A meta-level actor describes functionality of
  actor.
• To change the applications behavior, modify the
  relevant meta-actor.

31
Dynamic Environment

• Unpredictable requirements
• Heterogeneity, e.g. Mobile Agents or Devices

32
The Two Level Actor Model

• Stratify actors into
• Base-level actors (application)
• Meta-level actors (OS, communication,
  middleware)
• Meta-level actors
• can examine/modify runtime state and annotations
  of base-level objects.
• notified when base-level events of interest to
  them occur.
• distributed groups of meta-actors cooperate to
  provide local and system wide services.
• (Venkatasubramanian Talcott)

33
TLAM structure/syntax

• TLAM lt Net, TLAS, loc gt
• Net - distributed computer network with
• processor nodes
• communication link between nodes
• TLAS - a two level actor system with
• base- and meta-level actor states
• base- and meta-level messages
• annotations on base-level actors and messages
• reaction rules for messages and events
• loc - distribution function mapping actors to
  nodes

34
TLAM semantics

• Labeled transition system
• Trans ? C ? Lab ? C
• A configuration C lt lc, nq, acgt
• a (hypothetical) global snapshot
• lc - link configuration -- messages in transit
• nq - node buffer configuration -- messages
  queued
• ac - actor configuration -- base- and meta-level
  actor states
• Two kinds of transitions
• communication -- moving messages
• execution -- computation step followed by event
  handling

35
Ticker Monitor Rules

• (tm M(t,mc,tx,rx))
• dlv(t?tick)/ gt
• (tm M(t,mc,tx1,rx))
• (tm M(t,mc,tx,rx))
• dlv(t?time_at_c)/ gt
• (tm M(t,mc,tx,rx1)) mc ? log(t,tx,rx1,c)
• (tm M(t,mc,tx,rx)) tm ? reset
• /tT(0)gt
• (tm M(t,mc,0,0)) mc ? reset-ack

36
Monitored Ticker Scenario
(tmM(t,mc,2,0))
(tT(2))
t?tick
...
dlv(t?tick)
... c ...
...
(c..t..)
(tmM(t,mc,3,0))
(tT(3))
t?tick
...
t?time_at_c
mc?log(t,3,0)
(c..t..)
(tmM(t,mc,3,1))
(tT(n))
t?tick
dlv(t?time_at_c)
c?reply(3)
...
(c..t..)
37
TLAM modeling

• System
• set of configurations closed under TLAM
  transitions
• System properties specified in terms of
  properties of
• configurations
• transitions
• admissible computations
• Restriction to base level measures meta-level
  effect on functionality

38
Log Service Example

• Logging Service
• logs messages delivered to a given set of base
  actors and when requested reports the messages
  logged since the previous request.
• To provide the service, we specify a set of
  meta-actor behaviors and configure a service
  based on these behaviors.

39
Logging Service Specification

• A system S provides logging service w.r.t.
• loggable actors A
• request GetLog ActorId ? Message and
• reply LogReply ActorId ? Message ? Message
• if every request
• GetLog(client)
• gets a unique reply,
• LogReply(client, log), s.t.
• log is a subset of the messages delivered to
  loggable actors,
• no messages in log have previously been
  reported.
• every message delivered to a loggable actor will
  eventually be reported.

40
Logging Behavior

• A system S with logging behavior has
• a logger on each node, that does the recording
  for the loggable actors on its node,
• a reporter on each node, that does the reporting
  for the loggable actors on its node, and
• a log server ls (one for the system) that
• receives log requests GetLog(c) ls ? getLog_at_c,
  
• collects reports from each node in the system,
  and
• sends the log reply LogReply(c,L) c ?
  getLogReply(L)_at_ls.

41
Logger Rule

• (la logger(A))
• deliver(a?v)/
• seta(a,Log, geta(a,Log)a?v)
  gt
• (la logger(A))
• if a in A

42
Reporter Rule

• (ra reporter(A)) (ra ? sendLog_at_c)
• /seta(a,Log,) for a in Agt
• (ra reporter(A)) (c ?
  report(L)_at_ra)
• where L Union(a in A, get(a,Log))

43
Log Server Rules

• (ls logServerI(R)) ( ls ? getLog_at_c )
• gt
• (ls logServerW(R,c,,R))
• r ? sendReport_at_ls r in R
• (ls logServerW(R,c,L,R'r)) ( ls ? report(L')_at_r
  )
• gt
• (ls logServerW(R,c,LL',R')) if r not in R
• (ls logServerW(R,c,L,))
• gt
• (ls logServerI(R)) ( c ? getLogReply(L)_at_ls )

44
Logging Non-interference Requirement

• A system S satisfies the logging non-interference
  requirement if
• non-logging meta actors do not set Log
  attributes
• the only messages sent to logging meta actors
  by non-logging meta actors are log request
  messages addressed to the log server

45
Logging Theorems

• Theorem 1 (Base-meta noninterference)
• If system S has Logging Behavior, then Log
  meta-actors of S preserve base-level behavior.
• Theorem 2 (Behavior implies service)
• If system S has Logging Behavior and satisfies
  the logging initial conditions and
  non-interference requirements, then S provides
  logging service.

46
Theorem 2 Proof -- LogOk

• LogOk holds for all system configurations
• If the log server is idle, then there are no
  undelivered internal logger system messages.
• If the log server is waiting for reports from R',
  then there are no undelivered messages to or from
  the other reporters, and for each reporter ra in
  R' there is a single undelivered message, either
• (ra ? sendLog_at_ls)
• or
• (ls ? report(L)_at_ra)

47
Theorem 2 Proof - Responsiveness

• Every request receives a unique reply
• by fairness messages will be delivered (once)
• by LogOk a waiting log server will get the
  expected replies and respond to the request it is
  handling

48
QoS Broker Case Study

• Multimedia (MM) Server Design Management
• Multimedia Load Management
• Adaptive scheduling of incoming MM requests
• Reconfigure data to optimize servicing of
  requests
• Cost-based QoS metrics
• QoS Broker
• coordinates activities in a distributed MM system
• interactions between multiple QoS sessions
• interactions across multiple system services
• manages requests -- admission control
• manages data placement activities
• (Venkatasubramanian, Talcott Agha, TOSEM 2004 )

49
Light-weight Verification Methods

• Concolic Testing
• Combine symbolic and concrete testing
• Runtime Verification
• Monitoring specifications
• Probabilistic Models
• Statistical Model Checking
• As opposed to
• Theorem Proving
• Model Checking all states

50
Decentralized Runtime Monitoring

• Distribute property
• Properties expressed with respect to a process
• Local properties at every process
• Decentralize Monitoring
• Maintain knowledge of global state at each
  process
• Update knowledge with incoming messages
• Attach knowledge with outgoing messages
• At each process check safety property against
  local knowledge

51
Decentralized Monitoring Example
If a receives a value from b then b calculated
the value after receiving request from a
valRcv ? _at_b(?(valComputed ? _at_a(?valReq)))
valComputed ? _at_a(?valReq)
?(valComputed ? _at_a(?valReq))
_at_a(?valReq)
b
valComputed
a
valReq
valRcv
?valReq
valRcv ? _at_b(?(valComputed ? _at_a(?valReq)))
52
Past time Distributed Temporal Logic (pt-DTL)

• Based on epistemic logic
• Aumann76Meenakshi et. al. 00
• Properties with respect to a process, say p
• Interpreted over a sequence of global states that
  the process p is aware of
• Each process monitors the properties local to it
• No need for extra messages to create a relevant
  portion of global state
• KnowledgeVector keeps track of relevant global
  state that can effect a property.

53
Remote Expressions in pt-DTL

• Remote expressions arbitrary expressions
  related to the state of a remote process
• Propositions constructed from remote and local
  expressions
• If my alarm is set then eventually in past
  difference between my temperature and temperature
  at process b exceeded the allowed value
• alarm ? ?((myTemp - _at_btemp) gt allowed)

54
Safety in Airplane Landing

• If my airplane is landing then the runway that
  the airport has allocated matches the one that I
  am planning to use
• landing ? (runway _at_airportallocRunway)

55
Leader Election Example

• If a leader is elected then if the current
  process is a leader then, at its knowledge, none
  of the other processes is a leader
• elected ? (stateleader ? /\i?j(_at_j(state ?
  leader)))

56
pt-DTL syntax and semantics

• Fi true false P(Ei) Fi Fi Æ Fi
  propositional
• Fi Fi ?Fi Fi S Fi temporal
• _at_jFj epistemic
• Ei c vi 2 Vi f(Ei) functional
• _at_jEj epistemic

• c constant
• vi variable at process I
• P(Ei) predicate on Ei
• f(Ei) function f applied to Ei
• _at_jEj expression Ej at process j

• Fi previously Fi
• Fi always in past Fi
• ?Fi eventually in past Fi
• Fi S Fi Fi since Fi
• _at_jFj Fj at process j

57
Interpretation of _at_jEj at process i
s31
s32
s33
p3
m4
m1
m2
s22
p2
s23
s21
m3
p1
s12
s11
Since, at s23 p2 is aware of s12 of p1 value of
_at_1E in s23 at p2 value of E in s12 at p1
58
Monitoring Algorithm

• Requirements
• Should be fast so that online monitoring is
  possible
• Little memory overhead
• Additional messages sent should be minimal
  ideally zero
• KnowledgeVector
• Motivated by Vector Clocks
• Unlike Vector Clocks size independent of number
  of processes

59
KnowledgeVector

• KV is a vector
• one entry for each process appearing in formula
• KVj denotes entry for process j
• KVj.seq is the sequence number of last event
  seen at process j
• KVj.values stores values of j-expressions and
  j-formulae

60
KnowledgeVector Algorithm

• internal event (at process i)
• store eval(Ei,si) and eval(Fi,si) for each _at_iEi
  and _at_iFi in KVii.values
• send m
• KVii.seq à KVii.seq 1. Send KVi with m as
  KVm
• receive m
• for all j, if KVmj.seq gt KVij.seq then
• KVij.seq à KVmj.seq
• KVij.values à KVmj.value
• store eval(Ei,si) and eval(Fi,si) for each _at_iEi
  and _at_iFi in KVii.values

61
Example
p3
p2
Y7
Y3
violation
p1
X5
X9
X6
KV1.seq
(Y _at_1X) at p2
KV1.values
62
DIANA Architecture
pt-DTL Monitor
63
Modeling Large Scale Distributed Systems

• Each node is modeled by a finite state machine
• Cannot represent the state of all nodes as a
  product of individual states
• 1000 nodes of 5 state machines ? 51000 states

64
Models of Time

• Global synchronous wall clock
• Synchronization is too tight
• Too detailed an execution model
• Asynchronous, distributed time
• Vector clocks are too expensive
• Application behavior is complicated
• Need a more expressive model of time
• Notion of distance and distribution.
• Space-Time cone of causal influence.

t
Future causality
R
Q
P
Not Causally related
x
S
Past causally connected
y
Light Cones
65
WSN modeling

• Statistically abstract the state of a WSN
• Represent a state as a probability mass function
  (pmf)
• e.g. 90 of the nodes are in READY state.
• Model a WSN as Discrete Time Markov Chain (DTMC)
• Periodic sampling of a real WSN ? DTMC estimation

66
Features

• With PCTL like logics, PXA gt .3 is always
  true is always false regardless of initial pmfs
• However if 50 out of 100 nodes are in A state and
  the others are in B state gt50 of the nodes are
  always in A state.
• iLTL can specify this situation because it works
  on pmf transitions

67
iLTL Formula

• Syntax

68
iLTL Formulaatomic propositions

• At least 10 more nodes are in READY state than
  in IDLE state
• PXREADY gt PXIDLE 0.1
• PXREADY - PXIDLE gt 0.1
• Expected Queue length is less than 2
• 1PQS1 2PQS2 3PQS3 lt 2
• Availability of a system X is 10 larger than
  that of a system Y
• PXREADY gt PYREADY 0.1

69
Main Theorem

• If
• Markov matrix M is diagonalizable
• Absolute value of second largest eigenvalue of M
  is strictly less than 1
• For all inequalities of an iLTL formula ?, the
  steady state expected value of LHS is not equal
  to its RHS
• Then
• There is a bound N after which all inequalities
  of ? become constants

70
Model Checking Algorithm
71
iLTLChecker

• http//osl.cs.uiuc.edu/ykwon4

72
Example Markov Reward Model

• Example
• send/ack
• Ss, ra, rX, XX, d
• Expected energy consumption

73
Model Checking an Agilla Application

• Agilla is an agent middleware for sensor networks
• Markov Chain Agilla(S,M),
• S Halt, Wait, Arrive, Leave
• M is 4x4 matrix that is estimated
• Energy consumption for each state
• r 0,1,3,10 pA for each state from 3V source
• Expected energy consumption
• Eenergy rx, where xi P Xsi

74
Model Estimation

• Sampling from 20 motes -gt Markov model estimation

75
Model Checking

• Model Markov chain Agilla has states
  H, W, A, L, transits by 0.5405,
  0.4039, 0.0175, 0.7020 0.0567,
  0.3765, 0.3444, 0.1044 0.2973,
  0.0381, 0.5593, 0.0000 0.1055,
  0.1815, 0.0788, 0.1936
• Specification
• a 1PAgillaW 3PAgillaA 10PAgillaL
  gt 2.25,
• b 1PAgillaW 3PAgillaA 10PAgillaL
  lt 2.3 ltgt (a /\ b)

Is the expected energy between 2.25 to 2.3pW?
76
Conclusions

• Programming Abstractions
• Richer models of time, state, and transitions
• Probabilistic models and properties
• Lightweight Formal Methods