CS541 Review

1
CS541 Review

• Jim Blythe

2
Planning for the Grid
3
LIGOs Pulsar Search(Laser Interferometer
Gravitational-wave Observatory)
Extract channel
Short Fourier Transform
transpose
Long time frames
30 minutes
Short time frames
Single Frame
Time-frequency Image
Extract frequency range
event DB
Construct image
Find Candidate
Store
4
Operators include data dependencies, host and
resource constraints.

• (operator pulsar-search
• (preconds
• ((lthostgt (or Condor-pool Mpi))
• (ltstart-timegt Number)
• (ltchannelgt Channel)
• (ltfcentergt Number)
• (ltright-ascensiongt Number)
• (ltsample-rategt Number)
• (ltfilegt File-Handle)
• These two are parameters for the
  frequency-extract.
• (ltf0gt (and Number (get-low-freq-from-center-and
  -band
• 
  ltfcentergt ltfbandgt)))
• (ltfNgt (and Number (get-high-freq-from-center-an
  d-band
• 
  ltfcentergt ltfbandgt)))
• (ltrun-timegt (and Number
• (estimate-pulsar-search-run-time
• ltstart-timegt ltend-timegt
  ltsample-rategt
• ltf0gt ltfNgt
  lthostgt ltrun-timegt)))
• )

• (effects
• ()
• (
• (add (created ltfilegt))
• (add (at ltfilegt lthostgt))
• (add (pulsar ltstart-timegt ltend-timegt ltchannelgt
  
• ltinstrumentgt
  ltformatgt
• ltfcentergt ltfbandgt
• ltfderv1gt ltfderv2gt
  ltfderv3gt ltfderv4gt ltfderv5gt
• ltright-ascensiongt ltdeclinationgt
  ltsample-rategt
• ltfilegt))
• )
• ))

5
(No Transcript)
6
Temporal logics for planning
7
Fahiem Bacchus
8
Fahiem Bacchus
9
Heuristic search planning
10
Derive cost estimate from a relaxed planning
problem

• Ignore the deletes on actions
• BUT still NP-hard, so approximate
• For individual propositions p
• d(s, p) 0 if p is true in s
• 1 min(d(s, pre(a))) otherwise
• min over actions a that add p

11
HSP2 overview

• Best-first search, using h
• Based on WA - weighted A
• f(n) g(n) W h(n).
• If W 1, its A (with admissible h).
• If W gt 1, its a little greedy generally finds
  solutions faster, but not optimal.
• In HSP2, W 5

12
HSPr problem space

• States are sets of atoms (correspond to sets of
  states in original space)
• initial state is the goal G
• Goal states are those that are true in s0
  (initial state in planning problem)
• Still use h. h(s) sum g(s0, p)

13
Mutexes in HSPr, take 2

• Better definition
• A set M of pairs R p, q is a mutex set if
• (1) R is not true in s0
• (2) for every op o that adds p,
• either o deletes q
• or o does not add q, and for some precond r of
  o,
• r, q is in M.
• Recursive definition allows for some interaction
  of the operators

14
Temporal reasoning and scheduling
15
Temporal planning with mutual exclusion relation

• Propositions and actions are monotonically
  increasing, no-goods monotonically decreasing

16
ASPEN

• Combine planning and scheduling steps as
  alternative conflict repair operations
• Activities have start time, end time, duration
• Maintain most-commitment approach easier to
  reason about temporal dependencies with full
  information
• C.f. TLPlan

17
Contributors for a non-depletable resource
violation
18
Contributors for a depletable resource violation
19
Learning search control knowledgeand case-based
planning
20
Using EBL to improve plan quality

• Given planning domain, evaluation function
• planners plan, a better plan
• Learn control knowledge to produce the better
  plan
• Explanation used explain why the alternative
  plan is better
• Target concept control rules that make choices
  based on the planner state and meta-state

21
Architecture of Quality system
22
Explaining better plans recursivelytarget
concept shared subgoal
23
Hamlet blame assignment
24
Probabilistic planning
25
Sources of uncertainty

• Incomplete knowledge of the world (uncertain
  initial state)
• Non-deterministic effects of actions
• Effects of external agents or state dynamics.

26
Dealing with uncertainty re-planning and
conditional planning

• Re-planning
• Make a plan assuming nothing bad will happen
• Build a new plan if a problem is found
• (either re-plan to the goal state or try to
  repair the plan)
• In some cases, this is too late.
• Deal with contingencies (plans for bad outcomes)
  at planning time, before they occur.
• Cant plan for every contingency, so need to
  prioritize
• Implies sensing
• Build a plan that reduces the number of
  contingencies requires (conformant planning)
• May not be possible

27
A Buridan plan based on SNLP
28
Computing the probability of success 2 Bayes
nets
Time-stamped literal node
Action outcome node
What is the worst-case time complexity of this
algorithm?
29
MAXPLAN

• Inspired by SATPLAN. Compile planning problem to
  an instance of E-MAJSAT
• E-MAJSAT given a boolean formula with variables
  that are either choice variables or chance
  variables, find an assignment to the choice
  variables that maximizes the probability that the
  formula is true.
• Choice variables we can control them
• e.g. which action to use
• Chance variables we cannot control them
• e.g. the weather, the outcome of each action, ..
• Then use standard algorithm to compute and
  maximize probability of success

30
Probabilistic planningexogenous events
31
Representing external sources of change

• Model actions that external agents can take in
  the same way as actions that the planner can
  take.
• (event oil-spills
• (probability 0.1)
• (preconds
• (and (oil-in-tanker ltsea-sectorgt)
• (poor-weather ltsea-sectorgt)))
• (effects
• (del (oil-in-tanker ltsea-sectorgt))
• (add (oil-in-sea ltsea-sectorgt))))

32
Computing the probability of success using a
Bayes net
33
Example the weather events and the corresponding
markov chain

• The markov chain shows possible states
  independent of time.
• As long as transition probabilities are
  independent of time, the probability of the state
  at some future time t can be computed in
  logarithmic time complexity in t.
• The computation time is polynomial in the number
  of states in the markov chain.

34
The event graph

• Captures the dependencies between events needed
  to build small but correct markov chains.
• Any event whose literals should be included will
  be an ancestor of the events governing objective
  literals.

35
Probabilistic planningstructured policy
iteration
36
Structured representation
Craig Boutilier

• States decomposable into state variables
• Structured representations the norm in AI
• STRIPS, Sit-Calc., Bayesian networks, etc.
• Describe how actions affect/depend on features
• Natural, concise, can be exploited
  computationally
• Same ideas can be used for MDPs
• actions, rewards, policies, value functions, etc.
• dynamic Bayes nets DeanKanazawa89,BouDeaGol95
• decision trees and diagrams BouDeaGol95,Hoeyetal9
  9

37
Action Representation DBN/ADD
Craig Boutilier
38
Structured Policy and Value Function
Craig Boutilier