Divide and Conquer Algorithms

1
Divide and Conquer Algorithms

• GENERAL METHOD
• Divide large problem into a similar, but smaller
  sub-problems of size n/b
• 2. Solve each sub-problem (recursively)
• 3. Combine results to solve original problem

2
Divide and Conquer Algorithms It is faster
3
Divide and Conquer Algorithms

• GENERAL METHOD
• Divide and conquer recurrence relations
• T(1) n1
• T(n)
• a T(n/b) f(n) ngt1

4
Divide and Conquer Algorithms

• Recursive application

...
...
...
5
Divide and Conquer Algorithms

• GENERAL METHOD
• Solving this recursive relations by
• induction method
• Example Let a2, b2, T(1)2 and f(n)n. We
  have
• T(n)2T(n/2)n
• 22T(n/4)n/2n
• 4T(n/4)2n
• 42T(n/8)n/42n
• 8T(n/8)3n

• T(n)2i T(n/2i) in
• for any Log2ngtigt1
• T(n)2log2nT(n/2Log2n)
• nLog2n , for i Log2n
• T(n)nT(1)n Log2n nLog2n2n
• T(n)O(n Log2n)

6
Divide and Conquer Algorithms

• Two important Divide and Conquer methods
• Quick Sort hard division, easy combination
• Merge Sort easy division, hard combination

7
Quick Sort

• Definition
• The quick sort function sorts elements of data
  structures
• using a divide and conquer approach. This means
  the
• function resolves part of the problem and then
  recursively
• solves increasingly smaller partitions of the
  problem until
• eventually the whole problem is solved. This is
  part of the
• reason the quick sort is also called the
  Partition- Exchange
• Sort.

8
Quick Sort

• Input unsorted sequence
• Output sorted sequence
• 1. If input size is 1, return
• 2. Choose pivot element (perhaps fisrt element)
• 3. Create sub-sequences L, E, and G
• less than, equal to, or greater than pivot
  element
• 3. Recursively call quick-sort on L and G
• 4. Merge 3 sorted sequences into one sorted
• sequence

9
Quick Sort

• Each time the function recurses, a "key" value of
  the structure is selected to be positioned.
• Values less than the key are passed to the left
  side of the structure and values that are
  greater than the key are passed to the right side
  of the structure.
• These "left to right" and "right to left" scans
  and swaps continue until a flag condition tells
  them to stop.
• The function then repeatedly scans through the
  structure from two directions.
• This occurs when the structure can be divided
  into two sorted partitions.
• The key is then "switched into its final
  position".
• Next the quick sort function is called to sort
  the partition that contains the values that are
  less than the key.
• Then the quick sort function is called to sort
  the partition that contains the values that are
  greater than the key.
• The above algorithm continues until a condition
  indicates that the structure is sorted.

10
Quick Sort

• A Step Through Example 
• This is the initial array that you are starting
  the sort with
• The array is pivoted about its first element p 3
• Find first element larger than pivot (underlined)
  and the last element not larger than pivot
  (italicised)

11
Quick Sort

• A Step Through Example 
• Swap those elements
• Scan again in both directions
• Swap

12
Quick Sort

• A Step Through Example 
• Scan
• The pointers have crossed swap pivot with
  italicised.
• Pivoting is now complete. Recursively sort
  subarrays on each side of pivot.
• The array is now sorted.

13
Quick Sort
14
Quick Sort

• A Quicksort Example in Pascal
• You need a procedure Quicksort that will have
  input variables of array of integers, left and
  right indexes of partition and the output
  parameter the sorted array.
• You then need to define a function within the
  procedure that will partition the list by placing
  all values that are less than the pivot value in
  the left partition and all values greater than
  the pivot in the right partition.
• Then the Quicksort algorithm is used.

15
Quick Sort

• until (keysoutput gt pivotkey)
• if m lt output then
• begin
• tempkeysm keysm keysoutput
  keysoutputtemp
• else donetrue
• end
• until (not done)
• listleft listoutput
• END
• END end of partition
• Now the Quicksort algorithm is used
• BEGIN Quicksort
• IF left lt right THEN BEGIN pivot partition
  (list, left, right)
• Quicksort (list, left, pivot - 1)
• Quicksort (list, pivot 1, right)
• the last two lines are recursive calls to the
  procedure that perform the sorting on the smaller
  partitions
• END
• END End of Quicksort  

• A Quicksort Example in Pascal
• PROCEDURE Quicksort (VAR list LIST-TYPE left,
  right, INTEGER)
• This code performs the sort in the descending
  order
• VAR pivot INTEGER
• PROCEDURE partition (VAR list LIST-TYPE left,
  right, output INTEGER)
• VAR m, temp, pivotkey, pivotloc INTEGER
• done BOOLEAN
• BEGIN
• m left output right 1
• pivotkey listleft donefalse
• Repeat
• begin
• m m1
• Repeat
• begin
• mm1
• end
• until (keysm lt pivotkey)
• outputoutput-1

16
Quick Sort

• Quick Sort Efficiency
• Best Case Situation
• Assuming that the list breaks into two equal
  halves, we have two lists of size N/2 to
• sort. In order for each half to be partitioned,
  (N/2)(N/2) N comparisons are
• made. Also assuming that each of these list
  breaks into two equal sized sublists, we
• can assume that there will be at the most log(N)
  splits. This will result in a best time
• estimate of O(Nlog(N)) for Quicksort.
• Worst Case Situation
• In the worst case the list does not divide
  equally and is larger on one side than the
• other. In this case the splitting may go on N-1
  times. This gives a worst-case time
• estimate of O(N2). And if the keys are already
  sorted in ascending order, the pivot
• will be the smallest key each time Partition
  procedure is called. In this case quick
• sort will be in the worst case situation
• Average Time
• The average time for Quicksort is estimated to be
  O(Nlog(N)) comparisons.

17
Quick Sort

• Ramdomized Sorting Algorithm
• Random ordered input
• Median of the tree elements ap, a(qp)/2,
  aq
• Time taken to ramdomize is ? (n2)
• Avarage case result occurs
• Las Vegas algorithm- always output the correct
  answer

18
Quick Sort

• Ramdomized Sorting Algorithm
• Algorithm RQuickSort(p,q)
• // sorts the elements ap,...aq which reside
  in the global array a1n into ascending order.
• // An1 is considered to be defined and must be
  gtall elements in a1n
• if (pltq) then
• if((q-5)gt5) then
• Interchange(a, Ramndom() mod(q-p1) p, p)
• jPartition(a, p. Q1) //j is the position
  of the partitioning element.
• RQuickSort(p,j-1)
• RQuickSort(j1,q)

19
Quick Sort

• Advantages And Disadvantages of the Quick Sort
• In theory the Quicksort is a very efficient
  algorithm needing n log n steps. However, in
• practice its performance depends on the pivot
  point. The farther we get from the
• median for the pivot value the more lopsided the
  partitions become and the greater the
• depth of the recursion needs to be.
• This algorithm also requires a higher level of
  knowledge on the part of the
• programmer. This is becuase this algorithm
  utilizes recursion which can be tricky to
• grasp for beginners.  

20
Quick Sort

• Qucik Sort Animations
• http//java.sun.com/applets/jdk/1.0/demo/SortDemo/
  example1.html
• http//www.cs.brockport.edu/cs/java/apps/sorters/q
  uicksort.html
• http//www.cs.ubc.ca/spider/harrison/Java/sorting-
  demo.html

21
Merge Sort

• Definition
• MergeSort is a recursive sorting procedure that
  uses O(n log n) comparisons in the
• worst case. To sort an array of n elements, we
  perform the following three steps in
• sequence

22
Merge Sort

• Input unsorted sequence
• Output sorted sequence
• If input size is 1, return
• Split sequence of size n into two sequences of
  size n/2 according to position
• Recursively call merge sort on sub-sequences
• Merge two sorted sequences into one sorted
  sequence
• While neither sequence is empty
• Compare first element in each sequence
• Remove smallest and insert into output
• Insert all remaining elements into output

23
Merge Sort

• Analyzing Merge Sort
• Number of levels in recursion tree is O(logn)
• Each element appears in one sequence per level
• Total work done is linear at each call (i.e. O(1)
  work per element)
• Therefore, total work is nO(logn) O(nlogn)

24
Merge Sort

• Recurrence Relations
• a nlt1
• T(n)
• 2T(n/2) cn ngt1
• T(n) 2(2T(n/4) c(n/2)) cn
• 4T(n/4) 2cn
• 2 i T(n/2 i ) icn

Recursion stops when n2i, (ilogn) T(n)
2 logn T(1) cnlogn an cnlogn
O(nlogn)
25
Merge Sort
26
Merge Sort

• Recurrence Relations
• a nlt1
• T(n)
• 2T(n/2) cn ngt1
• T(n) 2(2T(n/4) c(n/2)) cn
• 4T(n/4) 2cn
• 2 i T(n/2 i ) icn

Recursion stops when n2i, (ilogn) T(n)
2 logn T(1) cnlogn an cnlogn
O(nlogn)
27
Merge Sort

• A Merge Sort Example
• procedure Mergesort(first,last Index)
• begin
• if firstltlast then
• Mergesort(first, ë( first last)/2û)
• Mergesort( ë( first last)/2û1,last)
• Merge(first,ë( first last)/2û, ë( first
  last)/2û1,last)
• end if
• end Mergesort

28
Merge Sort
Assume that nlast-first1
29
Merge Sort
Analysis of Merge i0 j0 k0 q(firstlast)/2.
0 while((iltq)(jltq)) if (A1iltA2j)
AkA1i i ?(n) else AkA2j
j while(iltq) ?(n) AkA1i
i k ?(n) while(jltq) ?(n) AkA2j
j k
30
Merge Sort
Advantages and Disadvantages of Merge Sort In
the theory, no sorting algorithm based on
comparison of entire keys can do better
(nlogn). Merge sort uses 2n locations. Beside
that copy process is hard (alow high -gt blow
high).
31
Merge Sort
Improving The Performans of Merge Sort Copying
process is a disadvantage. An alternative to this
copying is to associate a new field of
information with each key. This field is used to
link the keys and and any associated information
together in a sorted list (keys and related
information are called records). Then the merging
of the sorted lists proceeds by changing the link
values and no records need to be moved at all. A
filed that contains only a link will generally be
smaller than an entire record, so less space will
be used.
32
Merge Sort
Merge Sort Animations http//www2.ics.hawaii.edu/
qizhang/ics665/mergesort.html http//www.cosc.ca
nterbury.ac.nz/people/mukundan/dsal/MSort.html
http//www.cs.brockport.edu/cs/java/apps/sorters/
mergesort_df.html http//www.cse.uta.edu/holder/
courses/cse2320/lectures/applets/mergesort/mergeso
rt.html