EE441 Data Structures Chapter V Complexity of Algorithms - PowerPoint PPT Presentation

1 / 22
About This Presentation
Title:

EE441 Data Structures Chapter V Complexity of Algorithms

Description:

... done by an algorithm for a given set of input, we need a way to: ... int mid; DataType midvalue; ... the work changes in a way that is proportional to n ... – PowerPoint PPT presentation

Number of Views:50
Avg rating:3.0/5.0
Slides: 23
Provided by: ozgurba
Category:

less

Transcript and Presenter's Notes

Title: EE441 Data Structures Chapter V Complexity of Algorithms


1
EE441 Data StructuresChapter VComplexity of
Algorithms
  • Özgür B. Akan
  • Department of Electrical Electronics
    Engineering
  • Middle East Technical University
  • akan_at_eee.metu.edu.tr
  • www.eee.metu.edu.tr/akan

2
Algorithms
  • An algorithm is a computable set of steps to
    achieve a desired result.
  • Not sufficient that algorithms perform the
    required tasks.
  • They should do so efficiently, making the best
    use of
  • Space
  • Time

Efficiency is important!!!
3
Efficiency in Time and Space
  • Time
  • Instructions take time
  • How fast does the algorithm perform?
  • What affects its run-time?
  • Space
  • Data structures take space
  • What kind of data structures can be used?
  • How does the choice of data structure affect the
    runtime?

4
Objectives of Complexity Analysis
  • Two algorithms that accomplish the same task
  • Which one is better???
  • Given an algorithm, is it possible to determine
    how long it will take to run?
  • Input is unknown
  • Do not want to trace all possible execution paths
  • For different input, is it possible to determine
    how an algorithms runtime changes?

5
Measuring Growth of Work
  • While it is possible to measure the work done by
    an algorithm for a given set of input, we need a
    way to
  • Measure the rate of growth of an algorithm based
    upon the size of the input
  • Compare algorithms to determine which is better
    for the situation

6
Sequential Search Algorithm
  • e.g. Search an n-element array for a match with
    a given key return the array index of the
    matching element or -1 if no match is found
  • int SeqSearch(DataType list , int n, DataType
    key)
  • // note DataType must be defined earlier
  • // e.g., typedef int DataType
  • // or typedef float DataType etc.
  • for (int i0 iltn i)
  • if (listikey)
  • return i
  • return 1
  • Here, at worst case n comparisons
    (operations) performed
  • expected (average) n/2 comparisons
  • expected computation time ? n

7
Sequential Search Algorithm
  • e.g., if the algorithm takes 1 ms with 100
    elements
  • it takes 5 ms with 500 elements 200ms
    with 20000 elements etc.
  • Another approach Binary Search Algorithm
  • (Using a sorted array)

8
Binary Search Algorithm
  • int BinarySearch(DataType list, int low, int
    high, DataType key)
  • int mid
  • DataType midvalue
  • while (lowlthigh)
  • mid(lowhigh)/2 // note integer division,
    middle of array
  • midvaluelistmid
  • if (keymidvalue) return mid
  • else if (keyltmidvalue) highmid-1
  • else lowmid1
  • return 1

9
Binary Search Algorithm
  • e.g. int list55,17,36,37,45
  • low0, high4 key44
  • 1) mid(04)/22 3) mid(44)/24
  • midvaluelist236 midvaluelist445
  • keygtmidvalue keyltmidvalue
  • lowmid13 highmid-13
  • 2) mid(34)/23 4) since high3ltlow4, exit the
    loop
  • midvaluelist337 return -1 (not found)
  • keygtmidvalue
  • lowmid14

10
Binary Search Algorithm
  • e.g.
  • int list55,17,36,37,45
  • low0, high4 key5
  • (the same example with different key)
  • 1) mid(04)/22 2) mid(01)/20
  • midvaluelist236 midvaluelist05
  • keyltmidvalue keymidvalue
  • highmid-11 return 0 (found)

11
Binary Search Algorithm
  • In the worst case, Binary Search makes ?log2n ?
    comparisons
  • e.g. n log2n
  • 8 3
  • 20 5
  • 32 5
  • 100 7
  • 128 7
  • 1000 10
  • 1024 10
  • 64000 16
  • 65536 16

(ceil) Smallest integer larger than or equal to
e.g. if Binary Search takes 1msec for 100
elements, it takes tk ?log2n ? 1mseck
?log2 100? k1/7 msec/comparison Hence,
t(1/7) ?log2n ? t500(1/7)
?log2500?9/7?1.29msec t20000(1/7)
?log220000?15/7?2.1msec
12
Binary Search Algorithm
  • Therefore,

13
Computational Complexity
  • Compares growth of two functions
  • Independent of constant multipliers and
    lower-order effects
  • Metrics
  • Big-O Notation O( )
  • Big-Omega Notation ?( )
  • Big-Theta Notation ?( )
  • Allow us to evaluate algorithms
  • Has precise mathematical definition
  • Used in a sense to put algorithms into families
  • May often be determined by inspection of an
    algorithm

14
Definition Big-O Notation
  • Function f(n) is O(g(n)) if there exists a
    constant K and some n0 such that
  • f(n)Kg(n) for all nn0
  • i.e., as n??, f(n) is upper-bounded by a
    constant times g(n).
  • Usually, g(n) is selected among
  • log n (note loganklogbn for any a,b??)
  • n, nk (polynomial)
  • kn (exponential)

15
Big-O Notation
Kg(n)
Upper Bound
f(n)
f(n) is O(g(n))
Work done
Our Algorithm
n0
Size of input
16
Comparing Algorithms
  • The O() of algorithms determined using the formal
    definition of O() notation
  • Establishes the worst they perform
  • Helps compare and see which has better
    performance

17
Big-O (Examples)
  • e.g. f(n)n2250n106 is O(n2)
  • because
  • f(n)n2n2n2 for n 103
  • 3n2

n0
K
e.g. f(n)2n1023n?n is O(2n) because
1023nlt2n for ngtn0 and ?n lt2n ?n f(n)32n
for ngtn0
K
18
Correct Interpretation of O()
  • O(1) or Order One
  • Does not mean that it takes only one operation
  • Does mean that the work does not change as n
    changes
  • Is notation for constant work
  • O(n) or Order n
  • Does not mean that it takes n operations
  • Does mean that the work changes in a way that is
    proportional to n
  • Is a notation for work grows at a linear rate

19
Big-Omega Notation
  • Function f(n) is ?(g(n)) if there exists a
    constant K and some n0 such that
  • Kg(n) f(n) for all nn0
  • i.e., as n??, f(n) is lower-bounded by a
    constant times g(n).

20
Big-Theta Notation
  • Function f(n) is ?(g(n)) if there exist
    constants K1 and K2 and some n0 such that
  • K1g(n) f(n) K2g(n) for all nn0
  • i.e., as n??, f(n) is upper and lower bounded by
    some constants times g(n).

K2g(n)
f(n)
f(n) is ?(g(n))
Work done
K1g(n)
n0
Size of input
21
Examples
  • 3n2 17
  • ?(1), ?(n), ?(n2)
  • O(n2), O(n3),
  • ?(n2)

22
Analogous to Real Numbers
  • f(n) O(g(n)) (a lt b)
  • f(n) ?(g(n)) (a gt b)
  • f(n) ?(g(n)) (a b)
Write a Comment
User Comments (0)
About PowerShow.com