Medians and Matrix Multiplication

1
Medians and Matrix Multiplication

• Lecture 16 Medians and Matrix Multiplication
• Selection Problem
• Approximating the median
• Selection Algorithm
• Deriving a bound
• Matrix Multiplication
• Naïve Algorithm
• Strassens Method

• Lecture 15
• Quicksort
• Background
• Worst Case Analysis
• Best Case Analysis
• Average Case Analysis
• Empirical Comparison
• Randomized Quicksort

2
Selection and the Median

• Given an array, finding the sth smallest element
  is called the selection problem
• The median is the ceil(n/2)th smallest element
• Can find the median by sorting the array and
  choosing the ceil(n/2)th element, takes Q(nlogn)
  using heapsort or mergesort. Can we do better?
• If I have an algorithm for solving the selection
  problem, I can use it to find the median
• Likewise, well see that if I have an algorithm
  to find the median, I can solve the selection
  problem
• Our strategy will be to solve the selection
  problem with an approximate solution to the
  median problem, then use the solution to the
  selection problem to find the median exactly

3
Selection Algorithm
Find 4th smallest element
Pivot array around median p5 using pivotbis
function selection (T1..n,s) i ? 1 j ?
n repeat p ? median (Ti..j) pivotbis
(Ti..j,p,k,l) if s lt k then j ? k else
if s gt l then i ? l else return p
Only left side is relevant since 4 lt length of
left side
Pivot left side around its median p2
Only right side is relevant since 4 gt 4
Pivot right side around its median p3
Answer is 3 since 4th position is part of the
pivot
4
Selection Algorithm

• Dividing the array in half
• each iteration ensures
• worst case runtime O(logn)
• Would still work if we
• didnt use median
• Try p?Ti
• worst case O(n2)
• average time O(n)
• Better than O(nlogn) for
• average case can we
• improve worst case?

Find 4th smallest element
Pivot array around median p5 using pivotbis
Only left side is relevant since 4 lt length of
left side
Pivot left side around its median p2
Only right side is relevant since 4 gt 4
Pivot right side around its median p3
Answer is 3 since 4th position is part of the
pivot
5
Approximating the Median
function pseudomed (T1..n) if nlt 5 then
return adhocmed(T) z ? floor (n/5) array
Z1..z for i ? 1 to z do Zi ?
adhocmed(T5i-4..5i) return selection
(Z,ceil(z/2))
Find the median of each 5-element subarray, Find
the median of each of those medians. About 3n/10
elements below and 7n/10 elements above
6
Selection Algorithm
function selection (T1..n,s) i ? 1 j ?
n repeat p ? pseudomedian (Ti..j)
pivotbis (Ti..j,p,k,l) if s lt k then j ? k
else if s gt l then i ? l else return p
at most 7n/10 elements left
7 compares to find median of 5, done n/5 times
the call to selection on list of medians
t(n) lt 7n/5 (n-1)t(n/5)t(7n/10)
7
Deriving a Bound
t(n) lt 7n/5 (n-1)t(n/5)t(7n/10)
collect terms
t(n) lt 12n/5t(n/5)t(7n/10)
8
Deriving a Bound
t(n) lt 7n/5 (n-1)t(n/5)t(7n/10)
t(n) lt 12n/5t(n/5)t(7n/10)
t(n) lt 12/5(n9n/1081n/100)
n
next time through while loop
pseudmeds call to select on list of medians
n/5
7n/10
n/25
7n/50
7n/50
49n/100
9
Deriving a Bound
t(n) lt 7n/5 (n-1)t(n/5)t(7n/10)
t(n) lt 12n/5t(n/5)t(7n/10)
t(n) lt 12/5(n9n/1081n/100)
t(n) lt (12/5)n (19/1081/100)
n
n/5
7n/10
n/25
7n/50
7n/50
49n/100
10
Matrix Multiplication

• Who cares?
• Matrix operations are at the heart of scientific
  computation!
• Naïve algorithm in O(n3)
• Can we do better?
• If and are free and x is not

11
It Can Be Faster!
12
Naïve Algorithm
CAB is defined as
5 6 7 8
1 2 3 4
(1x5 2x7) (1x6 2x8) (3x5 4x7) (3x6 4x8)

x
19 22 43 50

O(n3)why?
13
Strassens Algorithm
5 6 7 8
m2m3 m1m2m5m6 m1m2m4-m7
m1m2m4m5
1 2 3 4

x
5 14 4257(-32) 425(-4)-0
425(-4)7
19 22 43 50


m1 (34-1) x (8-65) 6 x 7 42 m2 (1 x
5) 5 m3 (2 x 7) 14 m4 (1-3) x (8-6)
-2 x 2 -4 m5 (34) x (6-5) 7 x 1
7 m6 (2-3 1-4) x 8 -4 x 8
-32 m7 4 x (58-6-7) 4x0 0
14
Strassens Algorithm
5 6 7 8
m2m3 m1m2m5m6 m1m2m4-m7
m1m2m4m5
1 2 3 4

x
3x5 4x7 (34-1) x (8-65) 1x5 (1-3)x(8-6)
- 4x(58-6-7)
m1 (34-1) x (8-65) 6 x 7 42 m2 (1 x
5) 5 m3 (2 x 7) 14 m4 (1-3) x (8-6)
-2 x 2 -4 m5 (34) x (6-5) 7 x 1
7 m6 (2-3 1-4) x 8 -4 x 8
-32 m7 4 x (58-6-7) 4x0 0
15
Recursive Application
16
Recursive Application
17
Recursive Application
Why does commutativity matter here?
18
Complexity
Divide each matrix into fourths, use each fourth
in Strassens algorithm. Whats the complexity?

t(n)7t(n/2)Q(n2)
Solve the recurrence and find its order of
growth!
19
Decimal Wars

• Cant get fewer than 7 multiplications on a 2x2
  matrix or fewer than 21 on 3x3
• Pan 70x70 in 143,640 multiplications
• Strassen O(n2.81)
• 1979 O(n2.521813)
• 1980 O(n2.521801)

Imagine the excitement in January 1980 when this
improvement was made !!!