O() Analysis of Methods and Data Structures Reasonable vs. Unreasonable Algorithms Using O() Analysis in Design - PowerPoint PPT Presentation

About This Presentation
Title:

O() Analysis of Methods and Data Structures Reasonable vs. Unreasonable Algorithms Using O() Analysis in Design

Description:

An element can be found by comparing and cutting the work in half. We cut work ... Comparing Data Structures and Methods. Data Structure Traverse Search Insert ... – PowerPoint PPT presentation

Number of Views:94
Avg rating:3.0/5.0
Slides: 97
Provided by: ccGa
Category:

less

Transcript and Presenter's Notes

Title: O() Analysis of Methods and Data Structures Reasonable vs. Unreasonable Algorithms Using O() Analysis in Design


1
O() Analysis of Methods and Data Structures
Reasonable vs. UnreasonableAlgorithms Using O()
Analysis in Design
Lecture 24
2
Now Available Online!
  • http//www.coursesurvey.gatech.edu

3
O() Analysis of Methods and Data Structures
4
The Scenario
  • Weve talked about data structures and methods to
    act on these structures
  • Linked lists, arrays, trees
  • Inserting, Deleting, Searching, Traversal,
    Sorting, etc.
  • Now that we know about O() notation, lets
    discuss how each of these methods perform on
    these data structures!

5
Recipe for Determining O()
  • Break algorithm down into known pieces
  • Well learn the Big-Os in this section
  • Identify relationships between pieces
  • Sequential is additive
  • Nested (loop / recursion) is multiplicative
  • Drop constants
  • Keep only dominant factor for each variable

6
Array Size and Complexity
  • How can an array change in size?
  • MAX is 30
  • ArrayType definesa array1..MAX of Num
  • We need to know what N is in advance to declare
    an array, but for analysis of complexity, we can
    still use N as a variable for input size.

7
Traversals
  • Traversals involve visiting every element in a
    collection.
  • Because we must visit every node, a traversal
    must be O(N) for any data structure.
  • If we visit less than N elements, then it is not
    a traversal.

8
Comparing Data Structures and Methods
LB
  • Data Structure Traverse
  • Unsorted L List N
  • Sorted L List N
  • Unsorted Array N
  • Sorted Array N
  • Binary Tree N
  • BST N
  • FB BST N

9
Searching for an Element
  • Searching involves determining if an element is a
    member of the collection.
  • Simple/Linear Search
  • If there is no ordering in the data structure
  • If the ordering is not applicable
  • Binary Search
  • If the data is ordered or sorted
  • Requires non-linear access to the elements

10
Simple Search
  • Worst case the element to be found is the Nth
    element examined.
  • Simple search must be used for
  • Sorted or unsorted linked lists
  • Unsorted array
  • Binary tree
  • Binary Search Tree if it is not full and balanced

11
Comparing Data Structures and Methods
LB
  • Data Structure Traverse Search
  • Unsorted L List N N
  • Sorted L List N N
  • Unsorted Array N N
  • Sorted Array N
  • Binary Tree N N
  • BST N N
  • FB BST N

?
12
Balanced Binary Search Trees
  • If a binary search tree is not full, then in the
    worst case it takes on the structure and
    characteristics of a sorted linked list with N/2
    elements.

13
Binary Search Trees
  • If a binary search tree is not full or balanced,
    then in the worst case it takes on the structure
    and characteristics of a sorted linked list.

7
11
14
42
58
14
Example Linked List
  • Lets determine if the value 83 is in the
    collection

Head
\\
42
5
19
35
83 Not Found!
15
Simple/Linear Search Algorithm
  • cur lt- head
  • loop
  • exitif(cur NIL) OR (cur.data target)
  • cur lt- cur.next
  • endloop
  • if(cur ltgt NIL) then
  • print( Yes, target is there )
  • else
  • print( No, target isnt there )
  • endif

16
Pre-Order Search Traversal Algorithm
  • As soon as we get to a node, check to see if we
    have a match
  • Otherwise, look for the element in the left
    sub-tree
  • Otherwise, look for the element in the right
    sub-tree

14
Left ???
Right ???
17
Find 9
If I have to watch one more of these I think Im
going to die.
LB
22
18
Big-O of Simple Search
  • The algorithm has to examine every element in the
    collection
  • To return a false
  • If the element to be found is the Nth element
  • Thus, simple search is O(N).

19
Binary Search
  • We may perform binary search on
  • Sorted arrays
  • Full and balanced binary search trees
  • Tosses out ½ the elements at each comparison.

20
Full and Balanced Binary Search Trees
  • Contains approximately the same number of
    elements in all left and right sub-trees
    (recursively) and is fully populated.

21
Binary Search Example
Looking for 89
22
Binary Search Example
Looking for 89
23
Binary Search Example
Looking for 89
24
Binary Search Example
89 not found 3 comparisons 3 Log(8)
25
Binary Search Big-O
  • An element can be found by comparing and cutting
    the work in half.
  • We cut work in ½ each time
  • How many times can we cut in half?
  • Log2N
  • Thus binary search is O(Log N).

26
What?
LB
  • N
  • 2
  • 4
  • 8
  • 16
  • 32
  • 64
  • 128
  • 256
  • 512
  • Searches
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9

27
What?
LB
  • log2(N)
  • log2(2)
  • log2(4)
  • log2(8)
  • log2(16)
  • log2(32)
  • log2(64)
  • log2(128)
  • log2(256)
  • log2(512)
  • Searches
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9


28
CS Notation
LB
  • lg(N)
  • lg(2)
  • lg(4)
  • lg(8)
  • lg(16)
  • lg(32)
  • lg(64)
  • lg(128)
  • lg(256)
  • lg(512)
  • Searches
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9


29
Recall
LB
  • log2 N k log10 N
  • k 0.30103...
  • So O(lg N) O(log N)

30
Comparing Data Structures and Methods
LB
  • Data Structure Traverse Search
  • Unsorted L List N N
  • Sorted L List N N
  • Unsorted Array N N
  • Sorted Array N Log N
  • Binary Tree N N
  • BST N N
  • FB BST N Log N

31
Insertion
  • Inserting an element requires two steps
  • Find the right location
  • Perform the instructions to insert
  • If the data structure in question is unsorted,
    then it is O(1)
  • Simply insert to the front
  • Simply insert to end in the case of an array
  • There is no work to find the right spot and only
    constant work to actually insert.

32
Comparing Data Structures and Methods
LB
  • Data Structure Traverse Search Insert
  • Unsorted L List N N 1
  • Sorted L List N N
  • Unsorted Array N N 1
  • Sorted Array N Log N
  • Binary Tree N N 1
  • BST N N
  • FB BST N Log N

33
Insert into a Sorted Linked List
  • Finding the right spot is O(N)
  • Recurse/iterate until found
  • Performing the insertion is O(1)
  • 4-5 instructions
  • Total work is O(N 1) O(N)

34
Inserting into a Sorted Array
  • Finding the right spot is O(Log N)
  • Binary search on the element to insert
  • Performing the insertion
  • Shuffle the existing elements to make room for
    the new item

35
Shuffling Elements
Note we must have at least one empty cell
Insert 29
36
Big-O of Shuffle
Worst case inserting the smallest number
101
35
77
Would require moving N elements Thus shuffle is
O(N)
37
Big-O of Inserting into Sorted Array
  • Finding the right spot is O(Log N)
  • Performing the insertion (shuffle) is O(N)
  • Sequential steps, so add
  • Total work is O(Log N N) O(N)

38
Comparing Data Structures and Methods
LB
  • Data Structure Traverse Search Insert
  • Unsorted L List N N 1
  • Sorted L List N N N
  • Unsorted Array N N 1
  • Sorted Array N Log N N
  • Binary Tree N N 1
  • BST N N
  • FB BST N Log N

39
Inserting into a Full and Balanced BST
  • Always insert when current NIL.
  • Find the right spot
  • Binary search on the element to insert
  • Perform the insertion
  • 4-5 instructions to create add node

40
Full and Balanced BST Insert
Add 4
12
41
3
98
7
35
2
41
Full and Balanced BST Insert
12
41
3
98
7
35
2
4
42
Big-O of Full Balanced BST Insert
  • Find the right spot is O(Log N)
  • Performing the insertion is O(1)
  • Sequential, so add
  • Total work is O(Log N 1) O(Log N)

43
Comparing Data Structures and Methods
LB
  • Data Structure Traverse Search Insert
  • Unsorted L List N N 1
  • Sorted L List N N N
  • Unsorted Array N N 1
  • Sorted Array N Log N N
  • Binary Tree N N 1
  • BST N N
  • FB BST N Log N Log N

44
What about a BST
LB
  • Not full balanced?

45
Comparing Data Structures and Methods
LB
  • Data Structure Traverse Search Insert
  • Unsorted L List N N 1
  • Sorted L List N N N
  • Unsorted Array N N 1
  • Sorted Array N Log N N
  • Binary Tree N N 1
  • BST N N N
  • FB BST N Log N Log N

46
Two Sorting Algorithms
  • Bubblesort
  • Brute-force method of sorting
  • Loop inside of a loop
  • Mergesort
  • Divide and conquer approach
  • Recursively call, splitting in half
  • Merge sorted halves together

47
Bubblesort Review
  • Bubblesort works by comparing and swapping values
    in a list

1 2 3 4 5
6
12
101
5
35
42
77
48
Bubblesort Review
  • Bubblesort works by comparing and swapping values
    in a list

1 2 3 4 5
6
101
77
5
12
35
42
Largest value correctly placed
49
  • procedure Bubblesort(A isoftype in/out Arr_Type)
  • to_do, index isoftype Num
  • to_do lt- N 1
  • loop
  • exitif(to_do 0)
  • index lt- 1
  • loop
  • exitif(index gt to_do)
  • if(Aindex gt Aindex 1) then
  • Swap(Aindex, Aindex 1)
  • endif
  • index lt- index 1
  • endloop
  • to_do lt- to_do - 1
  • endloop
  • endprocedure // Bubblesort

N-1
to_do
50
Analysis of Bubblesort
  • How many comparisons in the inner loop?
  • to_do goes from N-1 down to 1, thus
  • (N-1) (N-2) (N-3) ... 2 1
  • Average N/2 for each pass of the outer loop.
  • How many passes of the outer loop?
  • N 1

51
Bubblesort Complexity
LB
  • Look at the relationship between the two loops
  • Inner is nested inside outer
  • Inner will be executed for each iteration of
    outer
  • Therefore the complexity is
  • O((N-1)(N/2)) O(N2/2 N/2) O(N2)

52
Graphically
LB
2N-1
N-1
N-1
2N-1
53
O(N2) Runtime Example
  • Assume you are sorting 250,000,000 items
  • N 250,000,000
  • N2 6.25 x 1016
  • If you can do one operation per nanosecond (10-9
    sec) which is fast!
  • It will take 6.25 x 107 seconds
  • So 6.25 x 107
  • 60 x 60 x 24 x 365
  • 1.98 years

54
Mergesort
67
45
23
14
6
33
98
42
67
45
23
14
6
33
98
42
Log N
45
23
14
98
67
6
33
42
23
98
45
14
67
6
33
42
23
98
45
14
67
6
42
33
Log N
6
33
42
67
14
23
45
98
6
14
23
33
42
45
67
98
55
Analysis of Mergesort
  • Phase I
  • Divide the list of N numbers into two lists of
    N/2 numbers
  • Divide those lists in half until each list is
    size 1
  • Log N steps for this stage.
  • Phase II
  • Build sorted lists from the decomposed lists
  • Merge pairs of lists, doubling the size of the
    sorted lists each time
  • Log N steps for this stage.

56
Analysis of the Merging
23
98
45
14
67
6
33
42
Merge
Merge
Merge
Merge
23
98
45
14
67
6
42
33
Merge
Merge
6
33
42
67
14
23
45
98
Merge
6
14
23
33
42
45
67
98
57
Mergesort Complexity
  • Each of the N numerical values is compared or
    copied during each pass
  • The total work for each pass is O(N).
  • There are a total of Log N passes
  • Therefore the complexity is
  • O(Log N N Log N) O (N Log N)

Break apart
Merging
58
O(NLogN) Runtime Example
  • Assume same 250,000,000 items
  • NLog(N) 250,000,000 x 8.3
  • 2, 099, 485, 002
  • With the same processor as before
  • 2 seconds

59
Summary
  • You now have the O() for basic methods on varied
    data structures.
  • You can combine these in more complex situations.
  • Break algorithm down into known pieces
  • Identify relationships between pieces
  • Sequential is additive
  • Nested (loop/recursion) is multiplicative
  • Drop constants
  • Keep only dominant factor for each variable

60
Questions?
61
Reasonable vs. UnreasonableAlgorithms
62
Reasonable vs. Unreasonable
What you need to know for the homework!
  • Reasonable algorithms have polynomial factors
  • O (Log N)
  • O (N)
  • O (NK) where K is a constant
  • Unreasonable algorithms have exponential factors
  • O (2N)
  • O (N!)
  • O (NN)

63
Algorithmic Performance Thus Far
  • Some examples thus far
  • O(1) Insert to front of linked list
  • O(N) Simple/Linear Search
  • O(N Log N) MergeSort
  • O(N2) BubbleSort
  • But it could get worse
  • O(N5), O(N2000), etc.

64
An O(N5) Example
  • For N 256
  • N5 2565 1,100,000,000,000
  • If we had a computer that could execute a million
    instructions per second
  • 1,100,000 seconds 12.7 days to complete
  • But it could get worse

65
The Power of Exponents
  • A rich king and a wise peasant

66
The Wise Peasants Pay
  • Day(N) Pieces of Grain
  • 1 2
  • 2 4
  • 3 8
  • 4 16
  • ...

63 9,223,000,000,000,000,000 64
18,450,000,000,000,000,000
67
How Bad is 2N?
  • Imagine being able to grow a billion
    (1,000,000,000) pieces of grain a second
  • It would take
  • 585 years to grow enough grain just for the 64th
    day
  • Over a thousand years to fulfill the peasants
    request!

?
68
So the King cut off the peasants head.
LB
69
The Towers of Hanoi
A B
C
  • Goal Move stack of rings to another peg
  • Rule 1 May move only 1 ring at a time
  • Rule 2 May never have larger ring on top of
    smaller ring

70
Towers of Hanoi Solution
Original State
Move 1
Move 2
Move 3
Move 5
Move 4
Move 6
Move 7
71
Towers of Hanoi - Complexity
  • For 3 rings we have 7 operations.
  • In general, the cost is 2N 1 O(2N)
  • Each time we increment N, we double the amount of
    work.
  • This grows incredibly fast!

72
Towers of Hanoi (2N) Runtime
  • For N 64
  • 2N 264 18,450,000,000,000,000,000
  • If we had a computer that could execute a million
    instructions per second
  • It would take 584,000 years to complete
  • But it could get worse

73
The Bounded Tile Problem
Match up the patterns in thetiles. Can it be
done, yes or no?
74
The Bounded Tile Problem
Matching tiles
75
Tiling a 5x5 Area
25 available tiles remaining
76
Tiling a 5x5 Area
24 available tiles remaining
77
Tiling a 5x5 Area
23 available tiles remaining
78
Tiling a 5x5 Area
22 available tiles remaining
79
Tiling a 5x5 Area
2 available tiles remaining
80
Analysis of the Bounded Tiling Problem
  • Tile a 5 by 5 area (N 25 tiles)
  • 1st location 25 choices
  • 2nd location 24 choices
  • And so on
  • Total number of arrangements
  • 25 24 23 22 21 .... 3 2 1
  • 25! (Factorial) 15,500,000,000,000,000,000,000,
    000
  • Bounded Tiling Problem is O(N!)

81
Tiling (N!) Runtime
  • For N 25
  • 25! 15,500,000,000,000,000,000,000,000
  • If we could place a million tiles per second
  • It would take 470 billion years to complete
  • Why not a faster computer?

82
A Faster Computer
  • If we had a computer that could execute a
    trillion instructions per second (a million times
    faster than our MIPS computer)
  • 5x5 tiling problem would take 470,000 years
  • 64-ring Tower of Hanoi problem would take 213
    days
  • Why not an even faster computer!

83
The Fastest Computer Possible?
  • What if
  • Instructions took ZERO time to execute
  • CPU registers could be loaded at the speed of
    light
  • These algorithms are still unreasonable!
  • The speed of light is only so fast!

84
Where Does this Leave Us?
  • Clearly algorithms have varying runtimes.
  • Wed like a way to categorize them
  • Reasonable, so it may be useful
  • Unreasonable, so why bother running

85
Performance Categories of Algorithms
  • Sub-linear O(Log N)
  • Linear O(N)
  • Nearly linear O(N Log N)
  • Quadratic O(N2)
  • Exponential O(2N)
  • O(N!)
  • O(NN)

Polynomial
86
Reasonable vs. Unreasonable
  • Reasonable algorithms have polynomial factors
  • O (Log N)
  • O (N)
  • O (NK) where K is a constant
  • Unreasonable algorithms have exponential factors
  • O (2N)
  • O (N!)
  • O (NN)

87
Reasonable vs. Unreasonable
  • Reasonable algorithms
  • May be usable depending upon the input size
  • Unreasonable algorithms
  • Are impractical and useful to theorists
  • Demonstrate need for approximate solutions
  • Remember were dealing with large N (input size)

88
Two Categories of Algorithms
Unreasonable
1035 1030 1025 1020 1015 trillion billion million
1000 100 10
NN
2N
N5
Runtime
Reasonable
N
Dont Care!
2 4 8 16 32 64 128 256 512 1024
Size of Input (N)
89
Summary
  • Reasonable algorithms feature polynomial factors
    in their O() and may be usable depending upon
    input size.
  • Unreasonable algorithms feature exponential
    factors in their O() and have no practical
    utility.

90
Questions?
91
Using O() Analysis in Design
92
Air Traffic Control
Conflict Alert
93
Problem Statement
  • What data structure should be used to store the
    aircraft records for this system?
  • Normal operations conducted are
  • Data Entry adding new aircraft entering the area
  • Radar Update input from the antenna
  • Coast global traversal to verify that all
    aircraft have been updated coast for 5 cycles,
    then drop
  • Query controller requesting data about a
    specific aircraft by location
  • Conflict Analysis make sure no two aircraft are
    too close together

94
Air Traffic Control System
  • Program Algorithm Freq
  • 1. Data Entry / Exit Insert 15
  • 2. Radar Data Update NSearch 12
  • 3. Coast / Drop Traverse 60
  • 4. Query Search 1
  • 5. Conflict Analysis TraverseSearch 12

95
Questions?
96
(No Transcript)
Write a Comment
User Comments (0)
About PowerShow.com