Problem Space Analysis for Plan Library Generation and Algorithm Selection in Real-time Systems

1
Problem Space Analysis for Plan Library
Generation and Algorithm Selection in Real-time
Systems

• Robert H. Holder, III
• Dissertation Proposal Defense
• August 26, 2009

Committee Dr. Tim Finin, Chair Dr. Marie
desJardins Dr. Tim Oates Dr. R. Scott Cost
2
-Scott Adams, Dilbert comic, June 1, 2008
...it was not obvious what you had to do next
so you had to think two, six, ten moves ahead.
Scenario planning is about having twelve plans,
so if one does not work you go to the next. The
fun is figuring out a backup for whatever could
go wrong.' -Seth Godin, entrepreneur,
describing his strategy for success in games and
entrepreneurship. From The Red Rubber Ball At
Work by Kevin Carroll
3
Outline

• Motivation
• Related Work
• Approach
• Problem-Solution (PS) Map
• Solution-Problem-Utility (SPU) Map
• Solution Similarity (SS) Map
• Map Utilization
• Plan Library Generation
• Algorithm Selection and Configuration
• Informed Problem Decomposition
• Map Generation
• Domain-Based Hints
• Sampling
• Interpolation
• Research Directions
• Summary

4
Outline

• Motivation
• Related Work
• Approach
• Problem-Solution (PS) Map
• Solution-Problem-Utility (SPU) Map
• Solution Similarity (SS) Map
• Map Utilization
• Plan Library Generation
• Algorithm Selection and Configuration
• Informed Problem Decomposition
• Map Generation
• Domain-Based Hints
• Sampling
• Interpolation
• Research Directions
• Summary

5
Motivation

• Real-time Planning Problems
• Shipboard Computing Resource Allocation
• Mobile Sensor Scheduling
• Unmanned Vehicle Routing
• Wireless Sensor Network (WSN) Reconfiguration

6
Motivation

• For planning problems in a real-time
  environment,

A plan library is a means of rapidly adapting to
a new environment.
Hypothesis
Plan Library Problem space analysis can inform
the efficient creation of a plan library.
Problem Space Analysis A sampling of problem
instances and their solutions can lend insight
into the underlying structure of the domain.
7
Motivation
Additional benefits of problem space analysis
8
Contributions

• Framework to model and reason about the
  topological structure of the solutions of related
  plans
• Techniques to predict and leverage the effect of
  problem instance characteristics and attributes
  on the topological structure of the solution
  space
• Novel algorithms that exploit solution space
  structure to generate plan libraries, select
  configure algorithms, and decompose large
  problems
• Evaluation of these algorithms by comparison to
  competing techniques for a set of sample problems

9
Dynamic Vehicle Routing Problem
Three vehicles to visit twelve destinations
10
Dynamic Vehicle Routing Problem
Static solution for known destinations
11
Dynamic Vehicle Routing Problem
Consideration for potential additional destination
12
Dynamic Vehicle Routing Problem
X
X
X
Contingency solution for additional destination
13
Dynamic Vehicle Routing Problem
Implementation of contingency solution
14
Dynamic Vehicle Routing Problem
How to plan contingencies for arbitrary
possibilities?
15
Outline

• Motivation
• Related Work
• Approach
• Problem-Solution (PS) Map
• Solution-Problem-Utility (SPU) Map
• Solution Similarity (SS) Map
• Map Utilization
• Plan Library Generation
• Algorithm Selection and Configuration
• Informed Problem Decomposition
• Map Generation
• Domain-Based Hints
• Sampling
• Interpolation
• Research Directions
• Summary

16
Related Work Technique Comparison

• Plan Library generation
• case-based reasoning (created after runtime)
• reactive planning (subset of input, localized)
• conditional contingency planning (localized)
• decision-theoretic (relies on probabilities)
• Real-time Planning
• plan repair (runtime, works for small changes)
• anytime contract algorithms (runtime)

17
Related Work Case-Based Reasoning
Smyth McKenna (2001)
18
Related Work Domain Space Analysis

• State Space Analysis
• Bulka (2009) Backbone planning
• Kondaris (2008) Automated skill learning
• Hoffman (2001) State space benches and
  exits
• Solution Space Analysis
• Miner (2009) Solution gradient lines
• Rosen, et. al. (2005) Medical plan comparison
• Gopal Starkschall (2002) Medical plan
  topology
• Plan-Space Planning
• Trinquart (2003) Plan space reachability
• Hoffman Nebel (2001) Uses plan space
  structure to estimate distance to goal state

19
Related Work Gopal Starkschall (2002)
20
Outline

• Motivation
• Related Work
• Approach
• Problem-Solution (PS) Map
• Solution-Problem-Utility (SPU) Map
• Solution Similarity (SS) Map
• Map Utilization
• Plan Library Generation
• Algorithm Selection and Configuration
• Informed Problem Decomposition
• Map Generation
• Domain-Based Hints
• Sampling
• Interpolation
• Research Directions
• Summary

21
Approach Overview
22
Traveling Salesperson Problem
4
1
2
3

• Four fixed cities
• Central starting point
• Unknown fifth city

23
Approach Problem-Solution Map
4
1
2
3
Each point represents a potential location of the
fifth city. The color of the point represents
the optimal solution for the resulting 5-city TSP.
24
ApproachProblem-Solution Map
4
0-1-3-2-5-4
0-1-5-4-2-3
0-1-3-2-4-5
1
2
0-1-3-5-2-4
3
0-1-5-3-2-4
0-5-1-3-2-4
0-3-2-4-1-5
0-1-4-2-3-5
Each point represents a potential location of the
fifth city. The color of the point represents
the optimal solution for the resulting 5-city TSP.
25
Approach

• Problem-Solution Map (PS Map)
• contiguous regions only need to store one
  solution per region
• complexity higher interaction of regions
  indicates a more complex space

26
Approach Solution-Problem-Utility Map
Optimal library requires 8/120 6.7 of possible
solutions. Is this too many?
27
Approach Solution-Problem-Utility Map
Tolerating solution degradation and extending the
scope of neighboring solutions can reduce library
size
28
Approach Solution-Problem- Utility Map
Discrete and continuous SPU Maps showing global
competency of one solution
29
Approach

• Solution Problem Utility Map (SPU Map)
• reduce solution regions (and thus library size)
  by tolerating utility degradation
• gradient of solution degradation

30
ApproachSolution-Similarity Map
0-1-3-2-5-4
0-1-5-4-2-3
0-1-3-2-4-5
0-1-3-5-2-4
0-1-5-3-2-4
0-5-1-3-2-4
0-3-2-4-1-5
0-1-4-2-3-5
high similarity
low similarity
31
Approach

• Solution Similarity Map (SS Map)
• reduce library size by
• relying on run-time adaptation
• creating parameterized solutions
• suggests regions where regular world assumption
  does not hold

32
Outline

• Motivation
• Related Work
• Approach
• Problem-Solution (PS) Map
• Solution-Problem-Utility (SPU) Map
• Solution Similarity (SS) Map
• Map Utilization
• Plan Library Generation
• Algorithm Selection and Configuration
• Informed Problem Decomposition
• Map Generation
• Domain-Based Hints
• Sampling
• Interpolation
• Research Directions
• Summary

33
Algorithm Selection Configuration
Initial solution use fast, aggressive heuristics
34
Algorithm Selection Configuration
Initial solution use more precise heuristics,
emphasize exploration over exploitation,
use less aggressive
hill-climbing Adaptation if solutions are
similar, use hill-climbing, else use genetic
algorithm
35
Informed Problem Decomposition

• To decompose a DVRP or DTSP, a system can fix one
  of the unknown city locations.
• Choosing cities such that the subproblem yields a
  more contiguous map will be advantageous for
  planning.

less contiguous
more contiguous
36
Outline

• Motivation
• Related Work
• Approach
• Problem-Solution (PS) Map
• Solution-Problem-Utility (SPU) Map
• Solution Similarity (SS) Map
• Map Utilization
• Plan Library Generation
• Algorithm Selection and Configuration
• Informed Problem Decomposition
• Map Generation
• Sampling
• Interpolation
• Domain-Based Hints
• Research Directions
• Summary

37
Map Generation

• 5-city TSP (1 unknown city)
• small problem with two degrees of freedom
• 12k problem instances
• naive solver
• fast runtime
• 5-city TSP (2 unknown cities)
• larger problem with four degrees of freedom
• 311k problem instances
• naive solver
• 20 minutes runtime

38
Map Generation

• Complete map generation is not feasible
• (would make algorithm selection irrelevant)
• how can we approximate the map efficiently?
• Map approximation
• sampling
• interpolation
• domain-based hints
• example fixed city locations lie on regional
  borders

39
Map Generation
Initial view of solution space topology
40
Map Generation
Ideal view of solution space topology
41
Map Generation
Preliminary experiment with random sampling and
nearest neighbor classification
42
Map Generation

• Ideal PS Map

• Approximated PS Map

Preliminary experiment with random sampling and
nearest neighbor classification
43
Map Generation
Preliminary experiment with random sampling and
nearest neighbor classification demonstrates 75
accuracy from a 0.5 sample rate.
44
Sampling Classification

• Sampling
• uniform/random/NOLHS/Rapidly expanding Random
  Trees (RRT)
• strategic sampling (active learning)
• schemes biased by domain hints
• Classification
• Nearest neighbor
• k nearest neighbors vs. radius of nearest
  neighbors
• weighting neighbors by distance
• Support Vector Machine (linear, non-linear)
• Bayesian Network
• Neural Network

45
Domain-Based Hints

• All the fixed points lie on regional boundaries

46
Domain-Based Hints

• All the fixed points lie on regional boundaries

47
SPU and SS Map Generation

• SPU Map
• calculate utility of solution for sample of
  problem instances
• regression to find limits of solution competence
• does optimal region shape inform tolerated region
  shape?
• SS Map
• for each solution, find similarity to each
  neighbor
• look for similarities to non-neighbors?
• algorithm selection - can we characterize how
  quickly solution is changing?

48
Regression

• Regression depends on a function form, i.e. a
  kernel
• Can we determine the appropriate kernel based on
  problem characteristics?
• Piecemeal regression may allow local
  customization of regression
• Support Vector Regression Machines (Drucker,
  1996)

49
Approach Overview
50
Outline

• Motivation
• Related Work
• Approach
• Problem-Solution (PS) Map
• Solution-Problem-Utility (SPU) Map
• Solution Similarity (SS) Map
• Map Utilization
• Plan Library Generation
• Algorithm Selection and Configuration
• Informed Problem Decomposition
• Map Generation
• Domain-Based Hints
• Sampling
• Interpolation
• Research Directions
• Summary

51
Domains

• Dynamic Traveling Salesman Problem
• Dynamic Vehicle Routing Problem
• Wireless Sensor Network (Re)Configuration

52
Research Plan Goals

• Investigate inferring a problem space analysis
  and solution topology from a sample set of
  problem instance and solution pairs
• Apply problem space analysis and other domain
  characteristics to the creation of an efficient
  plan library
• Apply problem space analysis to the selection of
  algorithms suited for a particular region of the
  problem space
• Apply problem space analysis to strategic
  decomposition of large planning problems

53
Research Plan

• Fall 2009
• Generate ideal set of PS and SPU Maps for various
  TSP, VRP, and WSN problems
• Test PS Map generation using various sampling and
  classification schemes
• Test SPU Map generation using various sampling
  and regression schemes
• Spring 2010
• Confirm usefulness of domain-based hints
• Apply bias to sampling and classification schemes
• Summer 2010
• Library generation algorithm
• Evaluation
• Fall 2010
• Algorithm selection technique
• Problem decomposition algorithm
• Evaluation
• Spring 2011
• Final experiments and evaluation
• Writing and defense

54
Evaluation

• System performance
• Comparison to other DTSP, DVRP algorithms
• Comparison to known optimal solutions (TSP/VRP)
• Comparison to baseline communication throughput
  and network life metrics (WSN)
• Characterize performance as function of library
  size
• Offline computation time vs. online performance
• Map approximation
• Raw accuracy
• Function of proximity to optimal solution region
• Expected utility degradation due to inaccuracies
• Characterize performance as function of sampling
  and interpolation schemes

55
Contributions

• Framework to model and reason about the
  topological structure of the solutions of related
  plans
• Techniques to predict and leverage the effect of
  problem instance characteristics and attributes
  on the topological structure of the solution
  space
• Novel algorithms that exploit solution space
  structure to generate plan libraries, select
  configure algorithms, and decompose large
  problems
• Evaluation of these algorithms on a sample
  problems by comparison to competing techniques

56
Summary

• Use of problem space analysis to facilitate
  real-time planning
• plan library creation
• algorithm selection
• problem decomposition
• Use of sampling and interpolation techniques to
  create problem space maps
• traditional techniques
• domain-based hints
• informed kernel selection

57
References

• Bulka (2009) Efficient Planning Using Plan
  Libraries to Capture the Structure of the State
  Space
• Gopal Starkschall (2002) Plan space
  representation of treatment plans in
  multidimensional space
• Hoffman (2001) Local Search Topology in Planning
  Benchmarks An Empirical Analysis
• Hoffman Nebel (2001) The FF Planning System
  Fast Plan Generation Through Heuristic Search
• Onder Pollack (1996) Contingency Selection in
  Plan Generation
• Kondaris (2008) Autonomous Robot Skill
  Acquisition
• Miner (2009) Rule Abstraction Understanding
  Emergent Behavior in Swarm Systems
• Smyth McKenny (2001) Competence Models and the
  Maintenance Problem
• Rosen, et. al. (2005) Interactively exploring
  optimized treatment plans
• Trinquart (2003) Analyzing Reachability within
  Plan Space

58
Thank you

• Questions
• Suggestions
• Comments

59
Backup
60
Related Work Contingency Planning
Onder Pollock (1996)
61
Related Work Miner (2009)