# Analysis of Algorithms - PowerPoint PPT Presentation

PPT – Analysis of Algorithms PowerPoint presentation | free to download - id: acaa6-YjFiZ The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Analysis of Algorithms

Description:

### Asymptotic analysis keeps the student's head in the clouds, while attention to ... Complexity analysis: to measure and predict the behavior (running time, storage ... – PowerPoint PPT presentation

Number of Views:250
Avg rating:3.0/5.0
Slides: 48
Provided by: aaro3
Category:
Tags:
Transcript and Presenter's Notes

Title: Analysis of Algorithms

1
Analysis of Algorithms
Aaron Tan
http//www.comp.nus.edu.sg/tantc/cs1101.html
2
Introduction to Analysis of Algorithms
• After you have read and studied this chapter, you
should be able to
• Know what is analysis of algorithms (complexity
analysis)
• Know the definition and uses of big-O notation
• How to analyse the running time of an algorithm

3
Introduction (1/2)
• Two aspects on writing efficient codes
• Programming techniques
• Implementation of algorithms
• Practitioners viewpoint
• Asymptotic analysis (big-O notation, etc.)
• Analysis and design of algorithms
• Theoreticians viewpoint

Asymptotic analysis keeps the students head in
the clouds, while attention to implementation
details keeps his feet on the grounds.

4
Introduction (2/2)
• Programming techniques versus Algorithm design

int f1 (int n) int a, sum0 for (a1
altn a) sum a return sum

5
Sum of Two Elements (1/5)
• Given this problem
• A sorted list of integers list and a value is
given. Write a program to find the indices of
(any) two distinct elements in the list whose sum
is equal to the given value.
• Example
• list 2, 3, 8, 12, 15, 19, 22, 24
• sum 23
• answer elements 8 (at subscript 2) and 15 (at
subscript 4)

6
Sum of Two Elements (2/5)
• Algorithm A
• list 2, 3, 8, 12, 15, 19, 22, 24
• sum 23
• answer elements 8 (at subscript 2) and 15 (at
subscript 4)

n size of list for x from 0 to n 2 for y
from x 1 to n 1 if ((listx
are x and y)
7
Sum of Two Elements (3/5)
• Code for algorithm A

import java.util. class SumOfTwoA
public static void main(String args)
Scanner scanner new Scanner(System.in)
int list 2, 3, 8, 12, 15, 19, 22, 24
int n list.length
System.out.print("Enter sum ") int sum
scanner.nextInt() boolean found
false for (int x 0 x lt n-1 !found
x) for (int y x1 y lt n
!found y) if (listx
listy sum)
System.out.println("Indices at " x " and "
y) found true

8
Sum of Two Elements (4/5)
• Algorithm A used nested loop and scans some
elements many times.
• Algorithm B can you use a single loop to examine
each element at most once?
• If this can be done, it will be more efficient
than Algorithm A.

9
Sum of Two Elements (5/5)
• Code for algorithm B

10
The Race
Who will reach the finishing line first?
11
Complexity Analysis (1/2)
• Complexity analysis to measure and predict the
behavior (running time, storage space) of an
algorithm.
• We will focus on running time here.
• Inexact, but provides a good basis for
comparisons.
• We want to have a good judgment on how an
algorithm will perform if the problem size gets
very big.

12
Complexity Analysis (2/2)
• Problem size is defined based on the problem on
hand.
• Examples of problem size
• Number of elements in an array (for sorting
problems).
• Length of the strings in an anagram problem.
• Number of discs in the Tower of Hanoi problem.
• Number of cities in the Traveling Salesman
Problem (TSP).

13
Definition (1/4)
• Assume problem size is n and T(n) is the running
time.
• Upper bound Big-O notation
• Definition
• T(n) O(f(n)) if there are constants c and n0
• such that T(n) ? cf(n) when n ? n0
• We read the equal sign as is a member of (?),
because O(f(n)) is a set of functions.
• We may also say that T(n) is bounded above by
f(n).

14
Definition (2/4)
15
Definition (3/4)
• The functions relative rates of growth are
compared.
• For instance, compare f(n) n2 with g(n)
1000n.
• Although at some points f(n) is smaller than
g(n), f(n) actually grows at a faster rate than
g(n). (Hence, an algorithm with running time
complexity of f(n) is slower than another with
running time complexity of g(n) in this example.)
• Hence, g(n) O(f(n)).
• The definition says that eventually there is some
point n0 past which cf(n) is always larger or
equal to g(n).
• Here, we can make c 1 and n0 1000.

16
Definition (4/4)
• Besides the big-O (upper bound) analysis, there
are the Omega ? (lower bound) analysis, the Theta
? (tight bounds) analysis, and others.

17
Exercises (1/4)
• f(n) 1 2 3  n
• Show that f(n) O(n2)
• Proof
• 1 2 3  n n(n1)/2
• n2/2 n/2
• ? n2/2 n2/2
• n2
• The above is the running time of basic sorting
algorithms such as bubblesort, insertion sort,
selection sort.

18
Exercises (2/4)
• f(n) 17 n n/3
• Show that f(n) O(n)
• Proof 17 n n/3 ? 3n O(n)
• f(n) n4 n2 20n 100
• Show that f(n) O(n4)
• Proof n4 n2 20n 100 ? 4n4 O(n4)
• From the two examples above, it can be seen that
an expression is dominated by the term of the
highest degree.

19
Exercises (3/4)
• Tower of Hanoi
• Algorithm
• Tower(n, source, temp, dest)
• if (n gt 0)
• tower(n-1, source, dest, temp)
• move disc from source to dest
• tower(n-1, temp, source, dest)
• Let T(n) number of move to solve a tower of
n discs.

20
Exercises (4/4)
• Tower of Hanoi (cont.)
• Let T(n) number of move to solve a tower of n
discs.
• Prove that T(n) 2n 1.
• T(0) 0
• T(n) T(n 1) 1 T(n 1)
• 2 ? T(n 1) 1
• 2 ? (2n1 1) 1
• 2 ? 2n1 2 1
• 2n 1
• Hence T(n) O(2n)

21
Some Common Series
22
The Conversation
• Boss Your program is too slow! Rewrite it!
• You But why? All we need to do is to buy faster
computer!

Is this really the solution?
23
Complexity Classes (1/4)
• There are some common complexity classes.
• In analysis of algorithm, log refers to log2, or
sometimes written simply as lg.

n is problem size.
24
Complexity Classes (2/4)
• Algorithms of polynomial running times are
desirable.

25
Complexity Classes (3/4)
26
Complexity Classes (4/4)
27
Analysing Simple Codes (1/4)
• Some rules.
• Basic operations are those that can be computed
in O(1) or constant time.
• Examples are assignment statements, comparison
statements, and simple arithmetic operations.

28
Analysing Simple Codes (2/4)
• Code fragment 1
• temp x
• x y
• y temp
• Running time 3 statements O(1)
• Code fragment 2
• p list.size()
• for (int i 0 i lt p i)
• listi 3
• Running time 1 p statements O(p)

29
Analysing Simple Codes (3/4)
• Code fragment 3
• if (x lt y)
• a 1 b 2 c 3
• else
• a 2 b 4 c 8 d 13 e 51
• Running time max3, 5 statements 5 O(1)

In code fragments 2 and 3, we consider only
assignment statements as our basic operations.
Even if we include the loop test operation (i lt
p) and update operation (i) in fragment 2, and
the if test operation (x lt y) in fragment 3, it
will not affect the final result in big-O
notation, since they are each of constant time.
30
Analysing Simple Codes (4/4)
• Code fragment 4
• sum 0.0
• for (int k 0 k lt n k)
• sum arrayk
• avg sum/n
• Running time 1 n 1 statements n 2 O(n)
• Code fragment 5
• for (int i 0 i lt n i)
• for (int j 0 j lt i j)
• sum matrixij
• Running time 0 1 2  (n-1) n(n-1)/2
O(n2)

31
General Rules (1/3)
• Rule 1 Loops
• The running time of a loop is at most the running
time of the statements inside the loop times the
number of iterations.

32
General Rules (2/3)
• Rule 2 Nested loops
• Analyse these inside out. The total running time
of a statement inside a group of nested loops is
the running time of the statement multiplied by
the product of the sizes of all the loops.

33
General Rules (3/3)
• Rule 3 Selection statements
• For the fragment
• if (condition) S1
• else S2
• the running time of an if-else statement is
never more than the running time of the condition
test plus the larger of the running times of S1
and S2.

34
Worst-case Analysis
• We may analyze an algorithm/code based on the
best-case, average-case and worst-case scenarios.
• Average-case and worst-case analysis are usually
better indicators of performance than best-case
analysis.
• Worst-case is usually easier to determine than
average-case.

35
Running Time of Some Known Algorithms
• The following are worst-case running time of some
known algorithms on arrays. The problem size, n,
is the number of elements in the array.
• Sequential search (linear search) in an array
O(n).
• Binary search in a sorted array O(lg n).
• Simple sorts (bubblesort, selection sort,
insertion sort) O(n2).
• Mergesort O(n lg n).

36
Sequential Search vs Binary Search (1/2)
• Sequential/linear search Start from first
element, visit each element to see if it matches
the search item.

public static int linearSearch(int list, int
searchValue) for (int i 0 i lt
list.length i) if (listi
searchValue) return i
return -1
37
Sequential Search vs Binary Search (2/2)
• Binary search
• Works for sorted array.
• Examine middle element, and eliminate half of the
array.

public static int binarySearch(int list, int
searchValue) int left 0 int
right list.length - 1 int mid
while (left lt right) mid (left
right)/2 if (listmid
searchValue) return mid
else if (listmid lt searchValue)
left mid 1 else
right mid - 1 return -1
38
Analysis of Sequential Search
• Assume
• An array with n elements.
• Basic operation is the comparison operation.
• Best-case
• When the key is found at the first element.
Running time O(1).
• Worst-case
• When the key is found at the last element, or
• Average case
• Assuming that the chance of every element that
matches the key is equal, then on average the key
is found after n/2 compare operations. Running
time O(n).

39
Analysis of Binary Search
• Assume
• An array with n elements.
• Basic operation is the comparison operation.
• Best-case
• When the key is found at the middle element.
Running time O(1).
• Worst-case
• Running time O(lg n). Why?
• If you start with the value n, how many times can
you half it until it becomes 1?
• Examples Starting with 8, it takes 3 halving to
get it to 1 starting with 32, it takes 5
halving starting with 1024, it takes 10 halving.

40
Analysis of Sort Algorithms
• For comparison-based sorting algorithms, the
basic operations used in analysis is
• The number of comparisons, or
• The number of swaps (exchanges).
• Worst-case analysis
• All the three basic sorts selection sort,
bubble sort, and insertion sort have worst-case
running time of O(n2), where n is the array size.
• What is the worst-case scenario for bubble sort?
For selection sort? For insertion sort?

41
Maximum Subsequence Sum (1/6)
• Given this problem
• Given (possibly negative) integers a0, a1, a2, ,
an-1, find the maximum value of
• (For convenience, the maximum subsequence sum is
0 if all the integers are negative.)
• Example
• list -2, 11, -4, 13, -5, -2
• answer 20 (a1 through a3).
• Many algorithms to solve this problem.

42
Maximum Subsequence Sum (2/6)
• Algorithm 1

public static int maxSubseqSum(int list)
int thisSum, maxSum maxSum 0
for (int i 0 i lt list.length i)
for (int j 0 j lt list.length j)
thisSum 0 for (int k
i k lt j k) thisSum
listk // count this line if
(thisSum gt maxSum) maxSum
thisSum return maxSum
43
Maximum Subsequence Sum (3/6)
• Algorithm 1 Analysis

How many times is line thisSum listk
executed?
44
Maximum Subsequence Sum (4/6)
• Algorithm 2

45
Maximum Subsequence Sum (5/6)
• Algorithm 2 Analysis

Algorithm 2 avoids the cubic running time O(n3)
by removing the inner-most for-k loop in
algorithm 1. New running-time complexity is O(n2).
46
Maximum Subsequence Sum (6/6)
• Algorithm 3

47
End of file