Automated Translation of Dynamic Programming Problems to JAVA Code and their Solution via an Interme - PowerPoint PPT Presentation

Loading...

PPT – Automated Translation of Dynamic Programming Problems to JAVA Code and their Solution via an Interme PowerPoint presentation | free to view - id: 11b04-ZjUyM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Automated Translation of Dynamic Programming Problems to JAVA Code and their Solution via an Interme

Description:

Automated Translation of Dynamic Programming Problems to JAVA Code and their ... a DP problem is any optimization problem that lends itself to a formulation that ... – PowerPoint PPT presentation

Number of Views:398
Avg rating:3.0/5.0
Slides: 46
Provided by: ho6
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Automated Translation of Dynamic Programming Problems to JAVA Code and their Solution via an Interme


1
Automated Translation of Dynamic Programming
Problems to JAVA Code and their Solution via an
Intermediate Petri Net Representation
ICS800 Dissertation Research
Dissertation Defense February 28, 2005
Holger Mauch
2
Motivation and Contributions
  • No general DP solver existed previously
  • DP2PN2Solver is fairly general
  • No general specification language with DP focus
    existed
  • gDPS designed exclusively for DP problems
  • Debugging a flawed DP model is difficult
  • Intermediate Bellman net representation allows
    consistency checks
  • A novel low-level Bellman net model helps

3
Overview
  • Specification of a DP problem
  • Translation to a Petri net (PN) model
  • Consistency checks on PN level
  • Translation to executable solver code
  • JAVA solver code
  • Spreadsheet
  • PNML code for simulation of PN
  • Executable code yields solution to problem

4
Overview DP2PN modules
5
Overview PN2Solver modules
6
Dynamic Programming (DP)
  • general optimization technique
  • here focus on discrete DP (as opposed to
    continuous DP)

7
Principle of Optimality
  • an optimal set of decisions has the property
    that whatever the first decision is, the
    remaining decisions must be optimal with respect
    to the outcome which results from the first
    decision (Bellman, Dynamic Programming, 1957)

8
Definition of DP
  • no unique mathematical definition
  • ... a DP problem is any optimization problem
    that lends itself to a formulation that yields an
    equation known as the DP optimality equation.
    (Sniedovich 1992)
  • DP is recursive optimization (Nemhauser 1966)

9
18 DP Problem Examples...
  • Optimal Binary Search Tree (BST)
  • Edit Distance Problem (EDP)
  • Fishery Profits Problem (FPP)
  • Integer Linear Programming (ILP)
  • 0/1 Knapsack Problem (KS01)
  • Longest Common Subsequence (LCS)
  • Longest Simple Path (LSP)
  • Matrix Chain Multiplication (MCM)
  • Optimal Financing Problem (OFP)
  • Optimal Storage on a Tape (OST)
  • Reject Allowance Problem (RAP)
  • Reliability Design Problem (RDP)
  • Shrub Covering Assortment (SCA)
  • Stagecoach Problem (SCP)
  • Shortest Path Acyclic (SPA)
  • Shortest Path Cyclic (SPC)
  • Traveling Salesman Problem (TSP)
  • Winning in Las Vegas (WLV)

10
...from these Textbooks
  • Cormen et al., Introduction to Algorithms BST,
    LCS, LSP, MCM, SCP, SPC
  • Hillier/Lieberman, Introduction to OR ILP, RAP,
    RDP, SCP, WLV
  • Horowitz et al., Computer Algorithms BST, EDP,
    KS01, RDP, SCP, SPA, TSP
  • Winston, OR Applications and Algorithms FPP,
    KS01, OFP, SCP, TSP

11
Matrix Chain Multiplication (MCM) Problem
  • Given product A1A2...An of matrices
  • Find optimal parenthesization
  • dimensions of Ai are denoted by di-1 and di
  • DP functional equation f(i,j)mini?k1,j)di-1dkdj if iopt. parenth. for Ai...Aj
  • Base f(i,j)0 if ij
  • Goal Find f(1,n)

12
MCM Problem Instance
  • n4
  • (d0,d1,d2,d3,d4)(3,4,5,2,2)
  • f(1,4)76 elementary multiplications in
  • (A1(A2A3))A4 optimal parenthesization

13
DP Technology
  • No standardized specifications (as in linear
    programming (LP)) existed before gDPS
  • No general DP solver software existed before
    DP2PN2Solver
  • needed specialized software for each problem
  • develop specialized software application or
  • develop specialized spreadsheet template
  • OR/MS people often lack software development
    skills

14
Improvement
  • Define the structure of a DP problem in a
    standardized specification
  • Many DP problems of practical relevance have
    similar structure
  • Compare to LP problem specification opt
    zc1x1...cnxn s.t. constraints
    ai1x1...ainxnbi
  • DP specification will be more complex and not
    cover all DP problems

15
DP2PN2Solver Steps
  • Step (1) requires human intelligence, all other
    steps are automated
  • Human modeler design the DP problem
    specification for the real-world problem and
    collect the input data for the problem instance
    to be solved
  • DP2PN module produces Bellman net
  • PN2Solver module produces executable code
  • Run the resulting executable solver code to get
    the solution to the problem instance

16
Elements of DP Specification
  • Multistage decision model
  • States (Stage is part of a state)
  • Decision (Actions)
  • Rewards (Stage return, costs)
  • Transformation (Transition function)
  • Goal state (Goal of computation)
  • DP functional equation
  • Recursive relationship
  • Base case(s) (Base states their function values)

17
DP Elements of MCM
  • State s(i,j) (a pair of integers)
  • Decision k?D(s)i,..,j-1
  • Reward r(s,k)di-1dkdj
  • Transformations
  • t1(s,k)(i,k)
  • t2(s,k)(k1,j)
  • Goal state (1,n). Goal of computation f(1,n)
  • DP functional equation
  • Recursive relationship f(s)mink?Df(t1(s,k))f(t2
    (s,k))r(s,k)
  • Base cases f(i,j)0 if ij (base states (i,j) if
    ij)

18
MCM example in general DP Specification Language
(gDPS)
  • BEGIN
  • NAME MCM //MatrixMultiplication
  • GENERAL_VARIABLES_BEGIN
  • //dimensions in MatMult problem
  • private static int dimension 3, 4, 5, 2,
    2
  • private static int n dimension.length-1
    //n4 matrices
  • GENERAL_VARIABLES_END
  • //a state is a vector/list of primitive or Set
    types
  • STATE_TYPE (int i, int j)
  • DECISION_VARIABLE int k
  • //the set of alternatives is a function of the
    state
  • DECISION_SPACE D(i,j)i,..,j-1

19
MCM example in gDPS (contd.)
  • GOAL f(1,n)
  • DPFE_BASE_CONDITIONS
  • f(i,j)0.0 WHEN (ij)
  • DPFE f(i,j)
  • MIN_k IN D
  • f(t1(i,j,k))f(t2(i,j,k))r(i,j,k)
  • REWARD_FUNCTION r(i,j,k)dimensioni-1
  • dimensionkdimensionj
  • TRANSFORMATION_FUNCTION t1(i,j,k)(i,k)
  • t2(i,j,k)(k1,j)
  • END

20
Elements of gDPS
  • GENERAL_VARIABLES section (optional)
  • int, double, String, etc. in C/Java style
    syntax
  • SET_VARIABLES section (optional)
  • Set variables in mathematical notation
  • GENERAL_FUNCTIONS section (optional)
  • arbitrary functions in C/Java style syntax
  • STATE_TYPE section (mandatory)
  • vector of state variables of heterogeneos types
  • DECISION_VARIABLE and DECISION_SPACE sections
    (mandatory)

21
Elements of gDPS (contd.)
  • DPFE and DPFE_BASE_CONDITIONS section (mandatory)
  • GOAL section (mandatory)
  • declare goal of the computation (start state)
  • REWARD_FUNCTION section (mandatory)
  • TRANSFORMATION_FUNCTION section (mandatory)
  • define one or more
  • TRANSITION_WEIGHTS section (optional)
  • for discounted or probabilistic DP problems

22
Petri Net Model for DP Instances
  • Observation The main operators for solving DP
    problem instances are
  • Minimization/Maximization
  • Addition (rewards add up) and multiplication
  • These operators can easily be modeled with Petri
    nets
  • Idea Transform gDPS specification into a PN
    model with the specification parser DP2PN

23
Petri Net Model for DP Instances
  • Weighted directed graph not sufficient
  • Would like exactly one edge per decision, but one
    decision might lead to multiple successor states
  • Tree model would not take advantage of
    overlapping subproblems

24
Place-Transition Nets
25
Definition of Place-Transition Net
  • A 6-tuple (P,T,F,K,M0,W) is called a P/T-net, iff
  • (P,T,F) is a finite net
  • K P???? gives a capacity for each place
  • W F??-0 attaches a weight to each arc of the
    net
  • M0 P ???? is the initial marking, respecting
    the capacities, i.e. M0(p)?K(p) ?p?P.

26
Activation of a Transition
  • A transition t is activated if
  • for every place p from the preset t the number
    of tokens at p is greater or equal to the weight
    W(p,t) of the arrow from p to t
  • for every place p in the postset t the number of
    tokens at p plus the weight W(t,p) of the arrow
    from t to p does not exceed the capacity K(p) of
    p.

27
Example Activated Transition
28
Firing of a transition
  • An activated transition t may fire by
  • decreasing the number of tokens ?p?t by W(p,t)
  • increasing the number of tokens ?p?t by W(t,p).

29
Example Firing
30
Petri Net Models for DP
  • 2 suitable PN models (Bellman nets)
  • Low-level Bellman nets (LLBN) use
    Place/Transition net semantics
  • High-level Bellman nets (HLBN)
  • based on the emerging PNML standard (Billington
    et al. 2003)
  • closely related to Colored Petri Nets (Jensen
    1992) and Predicate/Transition nets (Genrich
    1987)

31
Low-Level Bellman Net Model
32
Low-Level Bellman Net Model
33
High-Level Bellman Net Model
34
High-Level Bellman Net Model
35
Low-Level Bellman Net
  • Mauch 2004
  • Places
  • state place for each state
  • min-in place for each decision
  • Transitions
  • min-transition for each minimization operation
  • copy transition transferring tokens from a state
    place to the appropriate min-in places

36
High-Level Bellman Net
  • Lew 2002, Lew/Mauch 2003, Lew/Mauch 2004
  • real-valued tokens, at most one per place
  • Transitions
  • M-Transitions (for min/max)
  • E-Transitions (for addition, multiplication)
  • Places
  • state places for each state
  • intermediate places
  • enabling places (technicality that prevents
    transitions from firing more than once)

37
LLBN vs. HLBN (part 1/2)
  • LLBN
  • theoretical analysis easier
  • optimal policy can be determined immediately from
    final marking
  • no synchronization necessary, transitions can
    fire concurrently, any order
  • HLBN
  • more expressive than LLBN, easier to express
    complex functionality
  • more efficient transformation to solver code
  • some synchronization necessary, limited degree of
    concurrency

38
LLBN vs. HLBN (part 2/2)
  • LLBNs and HLBNs shown to be equivalent for a
    subclass Pnat,min, of DP problems where
  • functional values are natural numbers and
  • direction of optimization is minimization and
  • operator connecting functionals in DPFE is
    addition
  • Proof is constructive for Pnat,min, conversion
    algorithms have been designed
  • Existing PN software packages (e.g. Renew) can
    simulate both LLBN and HLBN
  • DP2PN2Solver uses HLBN

39
DP2PN Software Module
  • DP2PN parser created with JavaCC
    (compiler-compiler)
  • integrates seamlessly into Java environment
  • Catches syntactical errors like
  • No goal state provided
  • No or illegal DP functional equation provided
  • No base case for the DP functional equation
    provided.
  • No decision variable provided
  • No decision set provided

40
DP2PN Software Module
  • State-Decision-Reward-Transformation-Table
    produced by algorithm
  • initialize the set statesSeen to contain the goal
    state
  • initialize the queue stateQueue to contain the
    goal state
  • while stateQueue not empty
  • currentStatedequeue state from stateQueue
  • if currentState is not a base state
  • loop over all possible decisions
  • generate table entry (a row for the
  • state-decision-reward-transformation-table)
  • enqueue those next states which are not in
    statesSeen
  • into the stateQueue and also add them to
    statesSeen

41
State-Decision-Reward-Transformation Table for MCM
42
Output of DP2PN (File Format)
  • view MCMBN.csv in MS-Excel

43
Consistency Checks on PN Level
  • Challenge Certain DP specification design
    mistakes are difficult to detect.
  • Examination of Bellman net might help.
  • PN theory developed algorithms to check
  • structural (static) properties
  • circularity (e.g. SPA DP model used for SPC)
  • initially unmarked source place (e.g. a base case
    has been forgotten)
  • no sink place representing the goal state place
    (e.g. mistake in transformation function)
  • behavioral (dynamic) properties
  • premature deadlock (e.g. SPA DP model used for
    SPC)
  • liveness (e.g. invalid/omitted recursion base)
  • Algorithms verify the semantics of the DP problem
    specification
  • Syntactic validity can be checked by the DP2PN
    parser
  • DPBNs represent DP problem correctly

44
PN2Solver From PN to Executable
  • Input PN representation of DP problem instance
  • Output solver code, which solves the DP problem
    instance
  • Three PN2Solver software modules
  • PN2Java
  • PN2XML
  • PN2Spreadsheet

45
PN2Java From PN to Java
  • User is not dependent on PN simulators, but can
    use ubiquitous Java platform
  • Automatically produced Java solver code
  • is documented with comments
  • can be manually refined, if desired
  • can be used as building block for larger, problem
    specific software development project

46
PN2Java Output (part of MCM)
  • List argumentList //reusable temporary
    variable used for min/max
  • List operandList //reusable temporary
    variable for eval. (Etransitions)
  • //code to initialize base state place
    variables
  • CalculationObject I5 new CalculationObject("
    (1,1)",0.0)
  • CalculationObject I20 new
    CalculationObject("(4,4)",0.0)
  • CalculationObject I25 new
    CalculationObject("(2,2)",0.0)
  • CalculationObject I36 new
    CalculationObject("(3,3)",0.0)
  • //code that simulates the transitions firing
  • operandListnew ArrayList()
  • operandList.add(I5)
  • operandList.add(I25)
  • ECalculationObject I32 new
    ECalculationObject(
  • "k1",
  • I5.getValue()I25.getValue()60.0,
  • operandList)
  • argumentListnew ArrayList()
  • argumentList.add(I32)

47
Java Solver Output (MCM example)
  • The optimal value is 76.0
  • The solution tree is
  • State (1,4) has optimal value 76.0
  • Decision k3
  • State (1,3) has optimal value 64.0
  • Decision k1
  • Base state (1,1) has initial value 0.0
  • State (2,3) has optimal value 40.0
  • Decision k2
  • Base state (2,2) has initial value 0.0
  • Base state (3,3) has initial value 0.0
  • Base state (4,4) has initial value 0.0
  • optimal parenthesization (A1(A2A3))A4 ((A1)((A2)
    (A3)))(A4)

48
PN2XML From PN to PNML
  • PNML is XML-based interchange format for PN
  • standardization of PNML in progress
  • Import PNML file into PN software systems
  • Simulate the Bellman net
  • Graphical visualization of solution process
  • PN simulation software often allows the automated
    analysis of certain PN properties helps with
    consistency checks
  • Overhead to run PN software
  • Reconstruction of opt. policy not automated
  • Do demonstration with Renew software system

49
PN2Spreadsheet
  • Popularity of spreadsheet, especially in OR/MS
    community
  • Reconstruction of optimal policy not automated
  • link to MCM example

50
Future Work
  • Add a graphical front-end as a more convenient
    user interface
  • Extend LLBN model (e.g. for multiplicative DP
    functional equations)
  • Analysis of Bellman nets
  • automate consistency checks
  • consult PN theory to find more properties that
    can be examined
  • Potential for parallelization. Concurrency
    property in Bellman net identifies computations
    that can be performed in parallel.

51
Summary of Contributions (part 1)
  • Designed and defined gDPS source language
  • general
  • easy to learn (especially for nonprogrammers)
  • Modeled numerous DP applications
  • illustrates usefulness as educational tool
  • Theoretically defined LLBN, HLBN, DPBN
  • conform to standard conventions
  • permit analyses
  • Analyzed Bellman nets and their theoretical
    properties

52
Summary of Contributions (part 2)
  • Developed internal representation for Bellman
    nets. Flexibility allows
  • different sources
  • different execution systems
  • Implemented
  • preprocessor for ILP
  • compiler from gDPS to BN
  • solver code generators
  • from BN to Java
  • from BN to Spreadsheet
  • from BN to PNML (permits use of PN tool as
    analyzer and execution system)
  • Designed DPBN error analysis routines

53
Summary of Contributions (part 3)
  • Designed overall system structure that
  • is general, flexible, user-friendly
  • is adaptable to needs/preferences of users
  • in educational
  • and certain commercial settings
  • allows automated analyses
  • can serve as framework for a more advanced system
About PowerShow.com