Winning by being lazy: Hierarchy, Abstraction and Leastcommitment in the newage planning

1
Winning by being lazy Hierarchy, Abstraction
and Least-commitment in the new-age planning

• Subbarao Kambhampati
• Arizona State University
• rakaposhi.eas.asu.edu/yochan.html

Invited Talk at NIPS-98 Workshop on
Reinforcement Learning
2
--Theory of inverted reinforcement
3
Objective

• A quick overview of the ideas of abstraction,
  hierarchy, reuse and least-commitment in planning
• With special emphasis on new-age planners
• Share some (hopefully portable) lessons...

4
Overview

• Planning -- Then and Now
• Abstraction/hierarchy in planning
• Detail Abstraction
• Least Commitment
• Task decomposition
• Experiential abstraction (reuse/replay)
• Lessons

5
Planning The problem

• States are modeled in terms of (binary)
• state-variables (factored rep.)
• -- Complete initial state, partial goal
  state
• Actions are modeled as state
• transformation functions
• -- Syntax ADL language (Pednault)
• Plans are sequences of actions

At(A,M),At(B,M) In(A), In(B)
Appolo 13
Earth
Earth
At(A,E), At(B,E),At(R,E)
Effects
6
Refinement Planning The idea
Refine
All Sol
P
P
All Seq.
All Solutions
AIMAG-97
7
Existing Refinement Strategies
Extend Prefix
Add in the middle
State-Space
Plan-Space
Regression
Extend Suffix
HTN
Decompose
8
Then
Now
Conjunctive planners
Disjunctive planners

• Search in the space of conjunctive partial plans
• Disjunction split into the search space
• Solution extraction is trivial
• Examples
• STRIPS Prodigy
• SNLP UCPOP
• NONLIN SIPE

• Search in the space of disjunctive partial plans
• Disjunction handled explicitly
• Solution extraction is non-trivial
• CSP/SAT methods
• Examples
• Graphplan
• SATPLAN

AIMag-97IJCAI-97
9
Refining disjunctive plans
1 Load(A)
or

0
2 Load(B)
or
3 Fly(R)
10
Detail Abstraction

• Idea
• Abstract some details of the problem or actions.
• Solve the abstracted version.
• Extend the solution to the detailed version
• Precondition Abstraction
• Work on satisfying important preconditions first
• Importance judged by
• Length of plans for subgoals ABSTRIPS, PABLO
• Inter-goal relations ALPINE
• Distribution-based HighPoint
• Strong abstractions (with downward refinement
  property) are rare
• Effectiveness is planner-dependent
• Clashes with other heuristics such as most
  constrained first

11
Abstracting Resources(Teasing apart Planning and
Scheduling)

• Most planners thrash by addressing planning and
  scheduling considerations together
• Eg. Blocks world, with multiple robot hands
• Idea Abstract resources away during planning
• Plan assuming infinite resources
• Do a post-planning resource allocation phase
• Re-plan if needed

(with Biplav Srivastava)
12
Least Commitment (Detail Postponement)

• Postpone commitments unless forced
• Big idea in conjunctive refinement planning
• Partial-order planners UCPOP, SNLP
• Interacts with precondition abstraction
• Becomes a non-issue in disjunctive planning
• There is very little commitment to begin with
• Encodings based on partial order planning can
  actually be worse off Mali, 98
• Exception Variablized (lifted) representations

13
Task Decomposition (HTN) Planning

• Domain model contains non-primitive actions, and
  schemas for reducing them
• Reduction schemas are given by the designer
• Can be seen as encoding user-intent
• Two notions of completeness
• Schema completeness
• (Partial Hierarchicalization)
• Planner completeness

14
Modeling Action Reduction
15
Dual views of HTN planning

• Capturing hierarchical structure of the domain
• Motivates top-down planning
• Start with abstract plans, and reduce them
• Many technical headaches
• Respecting user-intent, maintaining systematicity
  and minimality AAAI-98
• Phantomization, filters, promiscuity,
  downward-unlinearizability..

• Capturing expert advice about desirable solutions
• Motivates bottom-up planning
• Ensure that each partial plan being considered is
  legal with respect to the reduction schemas
• Connection to efficiency is not obvious

Relative advantages are still unclear...

Barrett, 97
16
HTN planning in the new-age

• The ideas of top-down and bottom-up HTN planning
  can be ported to disjunctive planners AIPS-98
• Abstract actions can be seen as disjunctive
  constraints
• Add constraints to the SAT/CSP encodings of the
  planning problem to ensure that
• Abstract actions are related to primitive actions
  through the reduction schemas Top-down version
  OR
• Each primitive actions must be part of some task
  reduction schema Bottom-up version
• Puzzle How can increasing encoding sizes lead to
  efficient planning?
• New constraints support simplification of the
  original constraints

with Amol Mali
17
HTNSAT Some results
40x speedup
with Mali, AIPS-98
18
Experiential AbstractionMacrops, Reuse, Replay

• Structures being reused
• Opaque vs. Modifiable
• Solution vs. Solving process (derivation)
• Acquisition of structures to be reused
• Human given vs. Automatically acquired
• Mechanics of reuse
• Phased vs. simultaneous
• Costs
• Storage Retrieval costs Solution quality

19
Case-study DerSNLP

• Modifiable derivational traces were reused
• Traces were automatically acquired during problem
  solving
• Analyze the interactions among the parts of a
  plan, and store plans for non-interacting
  subgoals separately
• Reduces retrieval cost
• Use of EBL failure analysis to detect
  interactions
• All relevant trace fragments were retrieved and
  replayed before the control is given to
  from-scratch planner
• Extension failures are traced to individual
  replayed traces, and their storage indices are
  modified appropriately
• Improves retrieval accuracy

(with Ihrig, JAIR 97)
20
DerSNLP Results
Performance with increased Training
Solvability with increased traning
Library Size
5
3
1
(JAIR, 97)
21
Reuse in Disjunctive Planning

• Harder to make a disjunctive planner commit to
  extending a specific plan first
• Options
• Support opaque macros along with primitive
  actions
• Modify the problem/domain specification so the
  old plans constraints will be respected in any
  solution
• MAX-SAT formulations of reuse problem

with Amol Mali
22
Reachability/Relevance minimizations

• Reachability analysis
• Analyze which actions cannot be executed together
  and which propositions cannot be made together
  at particular time steps
• Graphplan mutual exclusions
• Domain invariants

• Relevance analysis
• Analyze which actions are relevant and must occur
  together
• Greedy Regression (RIFO)
• Operator Graphs
• Inseperability constraints

Explicate which parts of a disjunctive structure
cannot be part of a solution (focusing)
23
General Lessons

• Dual views Detail reduction vs. Expert advice
• Detail reduction gt hierarchical solving with
  promise of improved efficiency
• Expert advice implies further constraints on the
  solutions
• Strong abstractions are rare
• Must take abstractions as advice that can be
  overridden
• The interaction between abstraction and search
  mechanism
• Emphasis on automatic generation of abstractions
• Need to consider utility issues
• Emphasis on satisficing solutions
• Few quantitative guarantees on solution quality