Analysis of Algorithms - PowerPoint PPT Presentation

Loading...

PPT – Analysis of Algorithms PowerPoint presentation | free to download - id: acaa6-YjFiZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Analysis of Algorithms

Description:

Asymptotic analysis keeps the student's head in the clouds, while attention to ... Complexity analysis: to measure and predict the behavior (running time, storage ... – PowerPoint PPT presentation

Number of Views:250
Avg rating:3.0/5.0
Slides: 48
Provided by: aaro3
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Analysis of Algorithms


1
Analysis of Algorithms
Aaron Tan
http//www.comp.nus.edu.sg/tantc/cs1101.html
2
Introduction to Analysis of Algorithms
  • After you have read and studied this chapter, you
    should be able to
  • Know what is analysis of algorithms (complexity
    analysis)
  • Know the definition and uses of big-O notation
  • How to analyse the running time of an algorithm

3
Introduction (1/2)
  • Two aspects on writing efficient codes
  • Programming techniques
  • Implementation of algorithms
  • Practitioners viewpoint
  • Asymptotic analysis (big-O notation, etc.)
  • Analysis and design of algorithms
  • Theoreticians viewpoint

Asymptotic analysis keeps the students head in
the clouds, while attention to implementation
details keeps his feet on the grounds.

4
Introduction (2/2)
  • Programming techniques versus Algorithm design

int f1 (int n) int a, sum0 for (a1
altn a) sum a return sum

5
Sum of Two Elements (1/5)
  • Given this problem
  • A sorted list of integers list and a value is
    given. Write a program to find the indices of
    (any) two distinct elements in the list whose sum
    is equal to the given value.
  • Example
  • list 2, 3, 8, 12, 15, 19, 22, 24
  • sum 23
  • answer elements 8 (at subscript 2) and 15 (at
    subscript 4)

6
Sum of Two Elements (2/5)
  • Algorithm A
  • list 2, 3, 8, 12, 15, 19, 22, 24
  • sum 23
  • answer elements 8 (at subscript 2) and 15 (at
    subscript 4)

n size of list for x from 0 to n 2 for y
from x 1 to n 1 if ((listx
listy) sum) then found! (answers
are x and y)
7
Sum of Two Elements (3/5)
  • Code for algorithm A

import java.util. class SumOfTwoA
public static void main(String args)
Scanner scanner new Scanner(System.in)
int list 2, 3, 8, 12, 15, 19, 22, 24
int n list.length
System.out.print("Enter sum ") int sum
scanner.nextInt() boolean found
false for (int x 0 x lt n-1 !found
x) for (int y x1 y lt n
!found y) if (listx
listy sum)
System.out.println("Indices at " x " and "
y) found true

8
Sum of Two Elements (4/5)
  • Algorithm A used nested loop and scans some
    elements many times.
  • Algorithm B can you use a single loop to examine
    each element at most once?
  • If this can be done, it will be more efficient
    than Algorithm A.

9
Sum of Two Elements (5/5)
  • Code for algorithm B


10
The Race
100 metres ahead
Who will reach the finishing line first?
11
Complexity Analysis (1/2)
  • Complexity analysis to measure and predict the
    behavior (running time, storage space) of an
    algorithm.
  • We will focus on running time here.
  • Inexact, but provides a good basis for
    comparisons.
  • We want to have a good judgment on how an
    algorithm will perform if the problem size gets
    very big.

12
Complexity Analysis (2/2)
  • Problem size is defined based on the problem on
    hand.
  • Examples of problem size
  • Number of elements in an array (for sorting
    problems).
  • Length of the strings in an anagram problem.
  • Number of discs in the Tower of Hanoi problem.
  • Number of cities in the Traveling Salesman
    Problem (TSP).

13
Definition (1/4)
  • Assume problem size is n and T(n) is the running
    time.
  • Upper bound Big-O notation
  • Definition
  • T(n) O(f(n)) if there are constants c and n0
  • such that T(n) ? cf(n) when n ? n0
  • We read the equal sign as is a member of (?),
    because O(f(n)) is a set of functions.
  • We may also say that T(n) is bounded above by
    f(n).

14
Definition (2/4)
15
Definition (3/4)
  • The functions relative rates of growth are
    compared.
  • For instance, compare f(n) n2 with g(n)
    1000n.
  • Although at some points f(n) is smaller than
    g(n), f(n) actually grows at a faster rate than
    g(n). (Hence, an algorithm with running time
    complexity of f(n) is slower than another with
    running time complexity of g(n) in this example.)
  • Hence, g(n) O(f(n)).
  • The definition says that eventually there is some
    point n0 past which cf(n) is always larger or
    equal to g(n).
  • Here, we can make c 1 and n0 1000.

16
Definition (4/4)
  • Besides the big-O (upper bound) analysis, there
    are the Omega ? (lower bound) analysis, the Theta
    ? (tight bounds) analysis, and others.

17
Exercises (1/4)
  • f(n) 1 2 3 … n
  • Show that f(n) O(n2)
  • Proof
  • 1 2 3 … n n(n1)/2
  • n2/2 n/2
  • ? n2/2 n2/2
  • n2
  • The above is the running time of basic sorting
    algorithms such as bubblesort, insertion sort,
    selection sort.

18
Exercises (2/4)
  • f(n) 17 n n/3
  • Show that f(n) O(n)
  • Proof 17 n n/3 ? 3n O(n)
  • f(n) n4 n2 20n 100
  • Show that f(n) O(n4)
  • Proof n4 n2 20n 100 ? 4n4 O(n4)
  • From the two examples above, it can be seen that
    an expression is dominated by the term of the
    highest degree.

19
Exercises (3/4)
  • Tower of Hanoi
  • Algorithm
  • Tower(n, source, temp, dest)
  • if (n gt 0)
  • tower(n-1, source, dest, temp)
  • move disc from source to dest
  • tower(n-1, temp, source, dest)
  • Let T(n) number of move to solve a tower of
    n discs.

20
Exercises (4/4)
  • Tower of Hanoi (cont.)
  • Let T(n) number of move to solve a tower of n
    discs.
  • Prove that T(n) 2n 1.
  • T(0) 0
  • T(n) T(n 1) 1 T(n 1)
  • 2 ? T(n 1) 1
  • 2 ? (2n1 1) 1
  • 2 ? 2n1 2 1
  • 2n 1
  • Hence T(n) O(2n)

21
Some Common Series
22
The Conversation
  • Boss Your program is too slow! Rewrite it!
  • You But why? All we need to do is to buy faster
    computer!

Is this really the solution?
23
Complexity Classes (1/4)
  • There are some common complexity classes.
  • In analysis of algorithm, log refers to log2, or
    sometimes written simply as lg.

n is problem size.
24
Complexity Classes (2/4)
  • Algorithms of polynomial running times are
    desirable.

25
Complexity Classes (3/4)
26
Complexity Classes (4/4)
27
Analysing Simple Codes (1/4)
  • Some rules.
  • Basic operations are those that can be computed
    in O(1) or constant time.
  • Examples are assignment statements, comparison
    statements, and simple arithmetic operations.

28
Analysing Simple Codes (2/4)
  • Code fragment 1
  • temp x
  • x y
  • y temp
  • Running time 3 statements O(1)
  • Code fragment 2
  • p list.size()
  • for (int i 0 i lt p i)
  • listi 3
  • Running time 1 p statements O(p)

29
Analysing Simple Codes (3/4)
  • Code fragment 3
  • if (x lt y)
  • a 1 b 2 c 3
  • else
  • a 2 b 4 c 8 d 13 e 51
  • Running time max3, 5 statements 5 O(1)

In code fragments 2 and 3, we consider only
assignment statements as our basic operations.
Even if we include the loop test operation (i lt
p) and update operation (i) in fragment 2, and
the if test operation (x lt y) in fragment 3, it
will not affect the final result in big-O
notation, since they are each of constant time.
30
Analysing Simple Codes (4/4)
  • Code fragment 4
  • sum 0.0
  • for (int k 0 k lt n k)
  • sum arrayk
  • avg sum/n
  • Running time 1 n 1 statements n 2 O(n)
  • Code fragment 5
  • for (int i 0 i lt n i)
  • for (int j 0 j lt i j)
  • sum matrixij
  • Running time 0 1 2 … (n-1) n(n-1)/2
    O(n2)

31
General Rules (1/3)
  • Rule 1 Loops
  • The running time of a loop is at most the running
    time of the statements inside the loop times the
    number of iterations.

32
General Rules (2/3)
  • Rule 2 Nested loops
  • Analyse these inside out. The total running time
    of a statement inside a group of nested loops is
    the running time of the statement multiplied by
    the product of the sizes of all the loops.

33
General Rules (3/3)
  • Rule 3 Selection statements
  • For the fragment
  • if (condition) S1
  • else S2
  • the running time of an if-else statement is
    never more than the running time of the condition
    test plus the larger of the running times of S1
    and S2.

34
Worst-case Analysis
  • We may analyze an algorithm/code based on the
    best-case, average-case and worst-case scenarios.
  • Average-case and worst-case analysis are usually
    better indicators of performance than best-case
    analysis.
  • Worst-case is usually easier to determine than
    average-case.

35
Running Time of Some Known Algorithms
  • The following are worst-case running time of some
    known algorithms on arrays. The problem size, n,
    is the number of elements in the array.
  • Sequential search (linear search) in an array
    O(n).
  • Binary search in a sorted array O(lg n).
  • Simple sorts (bubblesort, selection sort,
    insertion sort) O(n2).
  • Mergesort O(n lg n).

36
Sequential Search vs Binary Search (1/2)
  • Sequential/linear search Start from first
    element, visit each element to see if it matches
    the search item.

public static int linearSearch(int list, int
searchValue) for (int i 0 i lt
list.length i) if (listi
searchValue) return i
return -1
37
Sequential Search vs Binary Search (2/2)
  • Binary search
  • Works for sorted array.
  • Examine middle element, and eliminate half of the
    array.

public static int binarySearch(int list, int
searchValue) int left 0 int
right list.length - 1 int mid
while (left lt right) mid (left
right)/2 if (listmid
searchValue) return mid
else if (listmid lt searchValue)
left mid 1 else
right mid - 1 return -1
38
Analysis of Sequential Search
  • Assume
  • An array with n elements.
  • Basic operation is the comparison operation.
  • Best-case
  • When the key is found at the first element.
    Running time O(1).
  • Worst-case
  • When the key is found at the last element, or
    when the key is not found. Running time O(n).
  • Average case
  • Assuming that the chance of every element that
    matches the key is equal, then on average the key
    is found after n/2 compare operations. Running
    time O(n).

39
Analysis of Binary Search
  • Assume
  • An array with n elements.
  • Basic operation is the comparison operation.
  • Best-case
  • When the key is found at the middle element.
    Running time O(1).
  • Worst-case
  • Running time O(lg n). Why?
  • If you start with the value n, how many times can
    you half it until it becomes 1?
  • Examples Starting with 8, it takes 3 halving to
    get it to 1 starting with 32, it takes 5
    halving starting with 1024, it takes 10 halving.

40
Analysis of Sort Algorithms
  • For comparison-based sorting algorithms, the
    basic operations used in analysis is
  • The number of comparisons, or
  • The number of swaps (exchanges).
  • Worst-case analysis
  • All the three basic sorts selection sort,
    bubble sort, and insertion sort have worst-case
    running time of O(n2), where n is the array size.
  • What is the worst-case scenario for bubble sort?
    For selection sort? For insertion sort?

41
Maximum Subsequence Sum (1/6)
  • Given this problem
  • Given (possibly negative) integers a0, a1, a2, …,
    an-1, find the maximum value of
  • (For convenience, the maximum subsequence sum is
    0 if all the integers are negative.)
  • Example
  • list -2, 11, -4, 13, -5, -2
  • answer 20 (a1 through a3).
  • Many algorithms to solve this problem.

42
Maximum Subsequence Sum (2/6)
  • Algorithm 1

public static int maxSubseqSum(int list)
int thisSum, maxSum maxSum 0
for (int i 0 i lt list.length i)
for (int j 0 j lt list.length j)
thisSum 0 for (int k
i k lt j k) thisSum
listk // count this line if
(thisSum gt maxSum) maxSum
thisSum return maxSum
43
Maximum Subsequence Sum (3/6)
  • Algorithm 1 Analysis

How many times is line thisSum listk
executed?
44
Maximum Subsequence Sum (4/6)
  • Algorithm 2


45
Maximum Subsequence Sum (5/6)
  • Algorithm 2 Analysis

Algorithm 2 avoids the cubic running time O(n3)
by removing the inner-most for-k loop in
algorithm 1. New running-time complexity is O(n2).
46
Maximum Subsequence Sum (6/6)
  • Algorithm 3


47
End of file
About PowerShow.com