Title: Problem Space Analysis for Plan Library Generation and Algorithm Selection in Real-time Systems
1Problem Space Analysis for Plan Library
Generation and Algorithm Selection in Real-time
Systems
- Robert H. Holder, III
- Dissertation Proposal Defense
- August 26, 2009
Committee Dr. Tim Finin, Chair Dr. Marie
desJardins Dr. Tim Oates Dr. R. Scott Cost
2-Scott Adams, Dilbert comic, June 1, 2008
...it was not obvious what you had to do next
so you had to think two, six, ten moves ahead.
Scenario planning is about having twelve plans,
so if one does not work you go to the next. The
fun is figuring out a backup for whatever could
go wrong.' -Seth Godin, entrepreneur,
describing his strategy for success in games and
entrepreneurship. From The Red Rubber Ball At
Work by Kevin Carroll
3Outline
- Motivation
- Related Work
- Approach
- Problem-Solution (PS) Map
- Solution-Problem-Utility (SPU) Map
- Solution Similarity (SS) Map
- Map Utilization
- Plan Library Generation
- Algorithm Selection and Configuration
- Informed Problem Decomposition
- Map Generation
- Domain-Based Hints
- Sampling
- Interpolation
- Research Directions
- Summary
4Outline
- Motivation
- Related Work
- Approach
- Problem-Solution (PS) Map
- Solution-Problem-Utility (SPU) Map
- Solution Similarity (SS) Map
- Map Utilization
- Plan Library Generation
- Algorithm Selection and Configuration
- Informed Problem Decomposition
- Map Generation
- Domain-Based Hints
- Sampling
- Interpolation
- Research Directions
- Summary
5Motivation
- Real-time Planning Problems
- Shipboard Computing Resource Allocation
- Mobile Sensor Scheduling
- Unmanned Vehicle Routing
- Wireless Sensor Network (WSN) Reconfiguration
6Motivation
- For planning problems in a real-time
environment, -
A plan library is a means of rapidly adapting to
a new environment.
Hypothesis
Plan Library Problem space analysis can inform
the efficient creation of a plan library.
Problem Space Analysis A sampling of problem
instances and their solutions can lend insight
into the underlying structure of the domain.
7Motivation
Additional benefits of problem space analysis
8Contributions
- Framework to model and reason about the
topological structure of the solutions of related
plans - Techniques to predict and leverage the effect of
problem instance characteristics and attributes
on the topological structure of the solution
space - Novel algorithms that exploit solution space
structure to generate plan libraries, select
configure algorithms, and decompose large
problems - Evaluation of these algorithms by comparison to
competing techniques for a set of sample problems
9Dynamic Vehicle Routing Problem
Three vehicles to visit twelve destinations
10Dynamic Vehicle Routing Problem
Static solution for known destinations
11Dynamic Vehicle Routing Problem
Consideration for potential additional destination
12Dynamic Vehicle Routing Problem
X
X
X
Contingency solution for additional destination
13Dynamic Vehicle Routing Problem
Implementation of contingency solution
14Dynamic Vehicle Routing Problem
How to plan contingencies for arbitrary
possibilities?
15Outline
- Motivation
- Related Work
- Approach
- Problem-Solution (PS) Map
- Solution-Problem-Utility (SPU) Map
- Solution Similarity (SS) Map
- Map Utilization
- Plan Library Generation
- Algorithm Selection and Configuration
- Informed Problem Decomposition
- Map Generation
- Domain-Based Hints
- Sampling
- Interpolation
- Research Directions
- Summary
16Related Work Technique Comparison
- Plan Library generation
- case-based reasoning (created after runtime)
- reactive planning (subset of input, localized)
- conditional contingency planning (localized)
- decision-theoretic (relies on probabilities)
- Real-time Planning
- plan repair (runtime, works for small changes)
- anytime contract algorithms (runtime)
-
17Related Work Case-Based Reasoning
Smyth McKenna (2001)
18Related Work Domain Space Analysis
- State Space Analysis
- Bulka (2009) Backbone planning
- Kondaris (2008) Automated skill learning
- Hoffman (2001) State space benches and
exits - Solution Space Analysis
- Miner (2009) Solution gradient lines
- Rosen, et. al. (2005) Medical plan comparison
- Gopal Starkschall (2002) Medical plan
topology - Plan-Space Planning
- Trinquart (2003) Plan space reachability
- Hoffman Nebel (2001) Uses plan space
structure to estimate distance to goal state
19Related Work Gopal Starkschall (2002)
20Outline
- Motivation
- Related Work
- Approach
- Problem-Solution (PS) Map
- Solution-Problem-Utility (SPU) Map
- Solution Similarity (SS) Map
- Map Utilization
- Plan Library Generation
- Algorithm Selection and Configuration
- Informed Problem Decomposition
- Map Generation
- Domain-Based Hints
- Sampling
- Interpolation
- Research Directions
- Summary
21Approach Overview
22Traveling Salesperson Problem
4
1
2
3
- Four fixed cities
- Central starting point
- Unknown fifth city
23Approach Problem-Solution Map
4
1
2
3
Each point represents a potential location of the
fifth city. The color of the point represents
the optimal solution for the resulting 5-city TSP.
24ApproachProblem-Solution Map
4
0-1-3-2-5-4
0-1-5-4-2-3
0-1-3-2-4-5
1
2
0-1-3-5-2-4
3
0-1-5-3-2-4
0-5-1-3-2-4
0-3-2-4-1-5
0-1-4-2-3-5
Each point represents a potential location of the
fifth city. The color of the point represents
the optimal solution for the resulting 5-city TSP.
25Approach
- Problem-Solution Map (PS Map)
- contiguous regions only need to store one
solution per region - complexity higher interaction of regions
indicates a more complex space
26Approach Solution-Problem-Utility Map
Optimal library requires 8/120 6.7 of possible
solutions. Is this too many?
27Approach Solution-Problem-Utility Map
Tolerating solution degradation and extending the
scope of neighboring solutions can reduce library
size
28Approach Solution-Problem- Utility Map
Discrete and continuous SPU Maps showing global
competency of one solution
29Approach
- Solution Problem Utility Map (SPU Map)
- reduce solution regions (and thus library size)
by tolerating utility degradation - gradient of solution degradation
30ApproachSolution-Similarity Map
0-1-3-2-5-4
0-1-5-4-2-3
0-1-3-2-4-5
0-1-3-5-2-4
0-1-5-3-2-4
0-5-1-3-2-4
0-3-2-4-1-5
0-1-4-2-3-5
high similarity
low similarity
31Approach
- Solution Similarity Map (SS Map)
- reduce library size by
- relying on run-time adaptation
- creating parameterized solutions
- suggests regions where regular world assumption
does not hold
32Outline
- Motivation
- Related Work
- Approach
- Problem-Solution (PS) Map
- Solution-Problem-Utility (SPU) Map
- Solution Similarity (SS) Map
- Map Utilization
- Plan Library Generation
- Algorithm Selection and Configuration
- Informed Problem Decomposition
- Map Generation
- Domain-Based Hints
- Sampling
- Interpolation
- Research Directions
- Summary
33Algorithm Selection Configuration
Initial solution use fast, aggressive heuristics
34Algorithm Selection Configuration
Initial solution use more precise heuristics,
emphasize exploration over exploitation,
use less aggressive
hill-climbing Adaptation if solutions are
similar, use hill-climbing, else use genetic
algorithm
35Informed Problem Decomposition
- To decompose a DVRP or DTSP, a system can fix one
of the unknown city locations. - Choosing cities such that the subproblem yields a
more contiguous map will be advantageous for
planning.
less contiguous
more contiguous
36Outline
- Motivation
- Related Work
- Approach
- Problem-Solution (PS) Map
- Solution-Problem-Utility (SPU) Map
- Solution Similarity (SS) Map
- Map Utilization
- Plan Library Generation
- Algorithm Selection and Configuration
- Informed Problem Decomposition
- Map Generation
- Sampling
- Interpolation
- Domain-Based Hints
- Research Directions
- Summary
37Map Generation
- 5-city TSP (1 unknown city)
- small problem with two degrees of freedom
- 12k problem instances
- naive solver
- fast runtime
- 5-city TSP (2 unknown cities)
- larger problem with four degrees of freedom
- 311k problem instances
- naive solver
- 20 minutes runtime
38Map Generation
- Complete map generation is not feasible
- (would make algorithm selection irrelevant)
- how can we approximate the map efficiently?
- Map approximation
- sampling
- interpolation
- domain-based hints
- example fixed city locations lie on regional
borders
39Map Generation
Initial view of solution space topology
40Map Generation
Ideal view of solution space topology
41Map Generation
Preliminary experiment with random sampling and
nearest neighbor classification
42Map Generation
Preliminary experiment with random sampling and
nearest neighbor classification
43Map Generation
Preliminary experiment with random sampling and
nearest neighbor classification demonstrates 75
accuracy from a 0.5 sample rate.
44Sampling Classification
- Sampling
- uniform/random/NOLHS/Rapidly expanding Random
Trees (RRT) - strategic sampling (active learning)
- schemes biased by domain hints
- Classification
- Nearest neighbor
- k nearest neighbors vs. radius of nearest
neighbors - weighting neighbors by distance
- Support Vector Machine (linear, non-linear)
- Bayesian Network
- Neural Network
45Domain-Based Hints
- All the fixed points lie on regional boundaries
46Domain-Based Hints
- All the fixed points lie on regional boundaries
47SPU and SS Map Generation
- SPU Map
- calculate utility of solution for sample of
problem instances - regression to find limits of solution competence
- does optimal region shape inform tolerated region
shape? - SS Map
- for each solution, find similarity to each
neighbor - look for similarities to non-neighbors?
- algorithm selection - can we characterize how
quickly solution is changing?
48Regression
- Regression depends on a function form, i.e. a
kernel - Can we determine the appropriate kernel based on
problem characteristics? - Piecemeal regression may allow local
customization of regression - Support Vector Regression Machines (Drucker,
1996)
49Approach Overview
50Outline
- Motivation
- Related Work
- Approach
- Problem-Solution (PS) Map
- Solution-Problem-Utility (SPU) Map
- Solution Similarity (SS) Map
- Map Utilization
- Plan Library Generation
- Algorithm Selection and Configuration
- Informed Problem Decomposition
- Map Generation
- Domain-Based Hints
- Sampling
- Interpolation
- Research Directions
- Summary
51Domains
- Dynamic Traveling Salesman Problem
- Dynamic Vehicle Routing Problem
- Wireless Sensor Network (Re)Configuration
52Research Plan Goals
- Investigate inferring a problem space analysis
and solution topology from a sample set of
problem instance and solution pairs - Apply problem space analysis and other domain
characteristics to the creation of an efficient
plan library - Apply problem space analysis to the selection of
algorithms suited for a particular region of the
problem space - Apply problem space analysis to strategic
decomposition of large planning problems
53Research Plan
- Fall 2009
- Generate ideal set of PS and SPU Maps for various
TSP, VRP, and WSN problems - Test PS Map generation using various sampling and
classification schemes - Test SPU Map generation using various sampling
and regression schemes - Spring 2010
- Confirm usefulness of domain-based hints
- Apply bias to sampling and classification schemes
- Summer 2010
- Library generation algorithm
- Evaluation
- Fall 2010
- Algorithm selection technique
- Problem decomposition algorithm
- Evaluation
- Spring 2011
- Final experiments and evaluation
- Writing and defense
54Evaluation
- System performance
- Comparison to other DTSP, DVRP algorithms
- Comparison to known optimal solutions (TSP/VRP)
- Comparison to baseline communication throughput
and network life metrics (WSN) - Characterize performance as function of library
size - Offline computation time vs. online performance
- Map approximation
- Raw accuracy
- Function of proximity to optimal solution region
- Expected utility degradation due to inaccuracies
- Characterize performance as function of sampling
and interpolation schemes
55Contributions
- Framework to model and reason about the
topological structure of the solutions of related
plans - Techniques to predict and leverage the effect of
problem instance characteristics and attributes
on the topological structure of the solution
space - Novel algorithms that exploit solution space
structure to generate plan libraries, select
configure algorithms, and decompose large
problems - Evaluation of these algorithms on a sample
problems by comparison to competing techniques
56Summary
- Use of problem space analysis to facilitate
real-time planning - plan library creation
- algorithm selection
- problem decomposition
- Use of sampling and interpolation techniques to
create problem space maps - traditional techniques
- domain-based hints
- informed kernel selection
57References
- Bulka (2009) Efficient Planning Using Plan
Libraries to Capture the Structure of the State
Space - Gopal Starkschall (2002) Plan space
representation of treatment plans in
multidimensional space - Hoffman (2001) Local Search Topology in Planning
Benchmarks An Empirical Analysis - Hoffman Nebel (2001) The FF Planning System
Fast Plan Generation Through Heuristic Search - Onder Pollack (1996) Contingency Selection in
Plan Generation - Kondaris (2008) Autonomous Robot Skill
Acquisition - Miner (2009) Rule Abstraction Understanding
Emergent Behavior in Swarm Systems - Smyth McKenny (2001) Competence Models and the
Maintenance Problem - Rosen, et. al. (2005) Interactively exploring
optimized treatment plans - Trinquart (2003) Analyzing Reachability within
Plan Space
58Thank you
- Questions
- Suggestions
-
- Comments
59Backup
60Related Work Contingency Planning
Onder Pollock (1996)
61Related Work Miner (2009)