Loading...

PPT – Advanced Algorithms PowerPoint presentation | free to download - id: 7aa9bb-YmVjN

The Adobe Flash plugin is needed to view this content

Advanced Algorithms

(Feodor F. Dragan) Department of Computer Science

Kent State University

Textbook Thomas Cormen, Charles Leisterson,

Ronald Rivest, and Cliff Stein,

Introduction to Algorithms,

McGraw Hill Publishing Company

and MIT Press, 2001 (2nd

Edition).

- Grading
- Homework 40
- Midterm Exam 30
- Final Exam 30

All the course slides will be available

at http//www.cs.kent.edu/dragan/CS6-76101-AdvAlg

.html

- Course Outline
- We will cover the following topics (the

topics and order listed are tentative and - subject to change some topics may only be

quickly surveyed to add breadth, - while others will be covered in reasonable

depth). - Dynamic Programming
- Optimal greedy algorithms
- Amortized analysis
- Parallel and circuit algorithms
- Network flow algorithms
- Randomized algorithms
- Number theoretic cryptographic algorithms
- String matching algorithms
- Computational geometry algorithms
- Algorithms for NP-hard and NP-complete problems
- Approximation algorithms
- Online algorithms
- Linear programming algorithms

What is a computer program exactly?

Input

Output

Some mysterious processing

Programs Data Structures Algorithms

CHAPTER 15 Dynamic Programming

- We begin discussion of an important algorithm

design technique, called dynamic programming (or

DP for short). - It is not a particular algorithm, it is a

metatechnique. - Programming tableau method, not writing a

code. - The technique is among the most powerful for

designing algorithms for optimization problems.

This is true for two reasons. - Dynamic programming solutions are based on a few

common elements. - Dynamic programming problems are typical

optimization problems (find the minimum or

maximum cost solution, subject to various

constraints. - The technique is related to divideandconquer,

in the sense that it breaks problems down into

smaller problems that it solves recursively.

However, because of the somewhat different nature

of dynamic programming problems, standard

divideandconquer solutions are not usually

efficient.

The basic elements that characterize a dynamic

programming algorithm are

- Substructure Decompose your problem into

smaller (and hopefully simpler) subproblems.

Express the solution of the original problem in

terms of solutions for smaller problems. (Unlike

divideandconquer problems, it is not usually

sufficient to consider one decomposition, but

many different ones.) - Tablestructure Store the answers to the

subproblems in a table. This is done because

(typically) subproblem solutions are reused many

times, and we do not want to repeatedly solve the

same problem. - Bottomup computation Combine solutions on

smaller subproblems to solve larger subproblems,

and eventually to arrive at a solution to the

complete problem. (Our text also discusses a

topdown alternative, called memoization.)

- The most important question in designing a DP

solution to a problem is how to set up the

subproblem structure. This is called the

formulation of the problem. - Dynamic programming is not applicable to all

optimization problems.

There are two important elements that a problem

must have in order for DP to be applicable.

- Optimal substructure Optimal solution to the

problem contains within it optimal solutions to

subproblems. This is sometimes called the

principle of optimality. It states that for the

global problem to be solved optimally, each

subproblem should be solved optimally. - Polynomially many subproblems An important

aspect to the efficiency of DP is that the total

number of distinct subproblems to be solved

should be at most a polynomial number.

Strings

- One important area of algorithm design is the

study of algorithms for character strings. - There are a number of important problems here.

Among the most important has to do with

efficiently searching for a substring or

generally a pattern in large piece of text. (This

is what text editors do when you perform a

search.) - In many instances you do not want to find a

piece of text exactly, but rather something that

is similar. This arises for example in genetics

research. Genetic codes are stored as long DNA

molecules. The DNA strands consists of a string

of molecules of four basic types C, G, T, A.

Exact matches rarely occur in biology because of

small changes in DNA replication. For this

reason, it is of interest to compute similarities

between strings that do not match exactly. - One common method of measuring the degree of

similarity between two strings is to compute

their longest common subsequence.

Longest Common Subsequence

Dynamic Programming Approach

Dynamic Programming Approach (cont.)

Implementing the Rule

Extracting the Actual Sequence

Time and Space Bounds and an Example

- The running time of the algorithm LCS is clearly

O(mn) since there are two nested loops with m and

n iterations, respectively. The running time of

the algorithm getLCS is O(mn). Both algorithms

use O(mn) space.

Y 0 1 2 3 4 n

Y 0 1 2 3 4 n

B D C B

B D C B

XBACDB YBDCB LCSBCB

0 0 0 0 0

0 1 1 1 1

0 1 1 1 1

0 1 1 2 2

0 1 2 2 2

0 1 2 2 3

0 0 0 0 0

0 1 1 1 1

0 1 1 1 1

0 1 1 2 2

0 1 2 2 2

0 1 2 2 3

0 1 2 3 4 5

BACDB

0 1 2 3 4 5

BACDB

X

m

m

LCS Length Table

with back pointers included

Longest common subsequence example.

READ Ch. 15.4 in CLRS.

Chain Matrix Multiplication

- This problem involves the question of

determining the optimal sequence for performing a

series of operations. - This general class of problem is important in

compiler design for code optimization and in

databases for query optimization. - We will study the problem in a very restricted

instance, where the dynamic programming issues

are easiest to see. - Suppose that we wish to multiply a series of

matrices - Matrix multiplication is an associative but not

a commutative operation. This means that we are

free to parenthesize the above multiplication

however we like, but we are not free to rearrange

the order of the matrices. - Also recall that when two (non-square) matrices

are being multiplied, there are restrictions on

the dimensions. A matrix has p rows and

q columns. You can multiply a matrix

A times a matrix B, and the result will

be a matrix C. The number of

columns of A must equal the number of rows of B. - In particular, for and
- There are total entries in C and each

takes O(q) time to compute, thus the

total time

(e.g. number of multiplications) to

multiply these two matrices is

A B C

q

p

p

r

r

q

Problem Formulation

Naive Algorithm

Dynamic Programming Approach

Dynamic Programming Formulation

Implementing the Rule

Extracting Optimum Sequence

Chain Matrix Multiplication Example

Recursive Implementation

Recursive Implementation (cont.)

Memoization

Memoization (cont.)

Polygons and Triangulations

Polygon

Triangulations (cont.)

Minimum-Weight Convex Polygon Triangulation

Correspondence to Binary Trees

Correspondence to Binary Trees (cont.)

Dynamic Programming Solution

Dynamic Programming Solution (cont.)

- This subdivides the polygon into the

subpolygons and

whose minimum weight are already known to

us as ti,k and tk1,j. - In addition we should consider the weight of

newly added triangle - Thus, we have the following recursive rule

- Note that this has exactly the same structure

as the recursive definition used in the

chain-matrix multiplication algorithms. - The same algorithm can be applied

with only minor changes.

READ Ch. 15.3 in CLRS and again all slides about

triangulations (this part was taken from the

first edition of the book (Ch. 16.4), you will

not find this in the second edition).

Homework 1 See on the class web-page.