UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2002

1
UMass Lowell Computer Science 91.503 Analysis
of Algorithms Prof. Karen Daniels Fall, 2002

• Lecture 1 (Part 3)
• Tuesday, 9/3/02
• Design Patterns for Optimization Problems
• Dynamic Programming Greedy Algorithms

2
Algorithmic Paradigm Context
Divide
Dynamic
Conquer
Programming
View problem as collection of
subproblems
Recursive nature
Independent subproblems
Overlapping subproblems
Number of subproblems
depends on
typically small
partitioning
factors
Preprocessing
Characteristic running time
typically log
depends on number
function of n
and difficulty of
subproblems
Primarily for optimization
problems
Optimal substructure
optimal solution to problem
contains within it optimal
solutions to subproblems
3

• Dynamic Programming

4
Example Matrix Parenthesization Definitions

• Given chain of n matrices ltA1, A2, An, gt
• Compute product A1A2 An efficiently
• Minimize cost number of scalar
  multiplications
• Multiplication order matters!

5
Example Matrix Parenthesization Step 1
Characterizing an Optimal Solution
Observation Any parenthesization of AiAi1 Aj
must split it between Ak and Ak1 for some
k. THM Optimal Matrix Parenthesization If an
optimal parenthesization of AiAi1 Aj splits at
k, then parenthesization of prefix AiAi1 Ak
must be an optimal parenthesization. Why?
If existed less costly way to parenthesize
prefix, then substituting that parenthesization
would yield less costly way to parenthesize
AiAi1 Aj , contradicting optimality of that
parenthesization.
6
Example Matrix Parenthesization Step 2 A
Recursive Solution

• Recursive definition of minimum parenthesization
  cost

0 if i j
mi,j
minmi,k mk1,j pi-1pkpj if i lt j
i lt k lt j
How many distinct subproblems?
7
Example Matrix Parenthesization Step 3
Computing Optimal Costs
s value of k that achieves optimal cost in
computing mi, j
source 91.503 textbook Cormen, et al.
8
Example Matrix Parenthesization Step 4
Constructing an Optimal Solution

• PRINT-OPTIMAL-PARENS(s, i, j)
• if i j
• then print Ai
• else print (
• PRINT-OPTIMAL-PARENS(s, i, si, j)
• PRINT-OPTIMAL-PARENS(s, si, j1, j)
• print )

source 91.503 textbook Cormen, et al.
9
Example Matrix Parenthesization Memoization
source 91.503 textbook Cormen, et al.

• Provide Dynamic Programming efficiency
• But with top-down strategy
• Use recursion
• Fill in table on demand
• Example
• RECURSIVE-MATRIX-CHAIN

10
Example Longest Common Subsequence (LCS)
Motivation

• Strand of DNA string over finite set A,C,G,T
• each element of set is a base adenine, guanine,
  cytosine or thymine
• Compare DNA similarities
• S1 ACCGGTCGAGTGCGCGGAAGCCGGCCGAA
• S2 GTCGTTCGGAATGCCGTTGCTCTGTAAA
• One measure of similarity
• find the longest string S3 containing bases that
  also appear (not necessarily consecutively) in S1
  and S2
• S3 GTCGTCGGAAGCCGGCCGAA

source 91.503 textbook Cormen, et al.
11
Example LCS Definitions
source 91.503 textbook Cormen, et al.

• Sequence is a subsequence of
  if (strictly
  increasing indices of X) such that
• example is subsequence of
  with index sequence
• Z is common subsequence of X and Y if Z is
  subsequence of both X and Y
• example
• common subsequence but not longest
• common subsequence. Longest?

Longest Common Subsequence Problem Given 2
sequences X, Y, find maximum-length common
subsequence Z.
12
Example LCS Step 1 Characterize an LCS
THM 15.1 Optimal LCS Substructure Given
sequences For any LCS of X and Y 1 if
then and Zk-1 is an LCS of Xm-1 and Yn-1 2
if then Z is an LCS of Xm-1 and Y 3
if then Z is an LCS of X and Yn-1
PROOF based on producing contradictions 1 a)
Suppose . Appending to Z
contradicts longest nature of Z. b) To
establish longest nature of Zk-1, suppose common
subsequence W of Xm-1 and Yn-1 has length gt k-1.
Appending to W yields common subsequence of
length gt k contradiction. 2 Common subsequence
W of Xm-1 and Y of length gt k would also be
common subsequence of Xm, Y, contradicting
longest nature of Z. 3 Similar to proof of (2)
source 91.503 textbook Cormen, et al.
13
Example LCS Step 2 A Recursive Solution

• Implications of Thm 15.1

no
yes
Find LCS(Xm-1, Yn-1)
Find LCS(X, Yn-1)
Find LCS(Xm-1, Y)
LCS(X, Y) LCS(Xm-1, Yn-1) xm
LCS(X, Y) max(LCS(Xm-1, Y), LCS(X, Yn-1)
LCS(X, Y)
14
Example LCS Step 2 A Recursive Solution
(continued)
source 91.503 textbook Cormen, et al.

• Overlapping subproblem structure
• Recurrence for length of optimal solution

Q(mn) distinct subproblems
Conditions of problem can exclude some
subproblems!
15
Example LCS Step 3 Compute Length of an LCS
c table ( represent b table)
source 91.503 textbook Cormen, et al.
16
Example LCS Step 4 Construct an LCS
source 91.503 textbook Cormen, et al.
17
Example LCS Improve the Code
source 91.503 textbook Cormen, et al.

• Can eliminate b table
• ci,j depends only on 3 other c table entries
• ci-1,j-1 ci-1,j ci,j-1
• given value of ci,j, can pick the one in O(1)
  time
• reconstruct LCS in O(mn) time similar to
  PRINT-LCS
• same Q(mn) space, but Q(mn) was needed anyway...
• Asymptotic space reduction
• leverage need only 2 rows of c table at a time
• row being computed
• previous row
• can also do it with space for 1 row of c table
• but does not preserve LCS reconstruction data

18
Algorithmic Paradigm Context
19

• Greedy Algorithms

20
What is a Greedy Algorithm?

• Solves an optimization problem
• Optimal Substructure
• optimal solution contains in it optimal solutions
  to subproblems
• Greedy Strategy
• At each decision point, do what looks best
  locally
• Choice does not depend on evaluating potential
  future choices or solving subproblems
• Top-down algorithmic structure
• With each step, reduce problem to a smaller
  problem
• Greedy Choice Property
• locally best globally best

21
Examples

• From 91.404
• Minimum Spanning Tree
• Dijkstras single-source shortest path
• Huffman Codes
• Fractional Knapsack
• Activity Selection

22
Example Optimization Problem Activity Selection

• Problem Instance
• Set S 1,2,...,n of n activities
• Each activity i has
• start time si
• finish time fi
• Activities i, j are compatible iff
  non-overlapping
• Objective
• select a maximum-sized set of mutually compatible
  activities

source 91.503 textbook Cormen, et al.
23
source 91.503 textbook Cormen, et al.
24
Examples Activity Selection
source 91.503 textbook Cormen, et al.

• Algorithm
• S presort activities in S by nondecreasing
  finish time
• and renumber
• GREEDY-ACTIVITY-SELECTOR(S)
• n lengthS
• A 1
• j 1
• for i 2 to n
• do if
• then
• j i
• return A

25
Another use for Greedy Algorithm

• If optimization problem does not have greedy
  choice property, greedy algorithm may still be
  useful in bounding the optimal solution
• Example minimization problem

Upper Bound (heuristic)
Solution Values
Optimal (unknown value)
Lower Bound