UMass Lowell Computer Science 91'503 Graduate Analysis of Algorithms Prof' Karen Daniels Fall, 2002

1
UMass Lowell Computer Science 91.503 Graduate
Analysis of Algorithms Prof. Karen Daniels
Fall, 2002

• Lecture 3
• Tuesday, 9/17/02
• Amortized Analysis

2
Overview

• Amortize To pay off a debt, usually by periodic
  payments Websters
• Amortized Analysis
• creative accounting for operations
• can show average cost of an operation is small
  (when averaged over sequence of operations) even
  though a single operation in the sequence is
  expensive
• result must hold for any sequence of these
  operations
• no probability is involved (unlike average-case
  analysis)
• guarantee holds in worst-case
• analysis method only no effect on code operation

3
Overview (continued)

• 3 ways to determine amortized cost of an
  operation that is part of a sequence of
  operations
• Aggregate Method
• find upper bound T(n) on total cost of sequence
  of n operations
• amortized cost average cost per operation
  T(n)/n
• same for all the operations in the sequence
• Accounting Method
• amortized cost can differ across operations
• overcharge some operations early in sequence
• store overcharge as prepaid credit on specific
  data structure objects
• Potential Method
• amortized cost can differ across operations (as
  in accounting method)
• overcharge some operations early in sequence (as
  in accounting method)
• store overcharge as potential energy of data
  structure as a whole
• (unlike accounting method)

4
Aggregate Method Stack Operations

• Aggregate Method
• find upper bound T(n) on total cost of sequence
  of n operations
• amortized cost average cost per operation
  T(n)/n
• same for all the operations in the sequence
• Traditional Stack Operations
• PUSH(S,x) pushes object x onto stack S
• POP(S) pops top of stack S, returns popped object
  
• O(1) time consider cost as 1 for our discussion
• Total actual cost of sequence of n PUSH/POP
  operations n

5
Aggregate Method Stack Operations (continued)

• New Stack Operation
• MULTIPOP(S,k) pops top k elements off stack S
• pops entire stack if it has lt k items
• MULTIPOP actual cost for stack containing s
  items
• Use cost 1 for each POP
• Cost min(s,k)
• Worst-case cost O(n)

6
Aggregate Method Stack Operations (continued)

• Sequence of n PUSH, POP, MULTIPOP ops
• initially empty stack
• MULTIPOP worst-case O(n) O(n2) for sequence
• Aggregate method yields tighter upper bound
• Sequence of n operations has O(n) worst-case cost
• Each item can be popped at most once for each
  push
• POP calls (including ones in MULTIPOP) lt push
  calls
• lt n
• Average cost of an operation O(n)/n O(1)
• amortized cost of each operation
• holds for PUSH, POP and MULTIPOP

7
Accounting Method

• Accounting Method
• amortized cost can differ across operations
• overcharge some operations early in sequence
• store overcharge as prepaid credit on specific
  data structure objects
• Let be actual cost of ith operation
• Let be amortized cost of ith operation
• Total amortized cost of sequence of operations
  must be upper bound on total actual cost of
  sequence
• Total credit in data structure
• must be nonnegative for all n

8
Accounting Method Stack Operations

• Paying for a sequence using amortized cost
• start with empty stack
• PUSH of an item always precedes POP, MULTIPOP
• pay for PUSH store 1 unit of credit
• credit for each item pays for actual POP,
  MULTIPOP cost of that item
• credit never goes negative
• total amortized cost of sequence of n ops is O(n)

9
Potential Method

• Potential Method
• amortized cost can differ across operations (as
  in accounting method)
• overcharge some operations early in sequence (as
  in accounting method)
• store overcharge as potential energy of data
  structure as a whole
• (unlike accounting method)
• Let ci be actual cost of ith operation
• Let Di be data structure after applying ith
  operation
• Let F(Di ) be potential associated with Di
• Amortized cost of ith operation
• Total amortized cost of n operations
• Must have to pay in advance

terms telescope
10
Potential Method Stack Operations

• Potential function value number of items in
  stack
• guarantees nonnegative potential after ith
  operation
• Amortized operation costs (assuming stack has s
  items)
• PUSH
• potential difference
• amortized cost
• MULTIPOP(S,k) pops kmin(k,s) items off stack
• potential difference
• amortized cost
• POP amortized cost also 0
• Amortized cost O(1) total amortized cost of
  sequence of n operations O(n)

11
Dynamic Tables Overview

• Dynamic Table T
• array of slots
• Ignore implementation choices stack, heap, hash
  table...
• if too full, increase size copy entries to T
• if too empty, decrease size copy entries to T
• Analyze dynamic table insert/delete
• Actual expansion/contraction cost is large
• Show amortized cost of insert/delete O(1)
• Load factor a(T) numT/sizeT
• empty table a(T) 1
• full table a(T) 1

12
Dynamic Tables Table (Expansion Only)

• Load factor bounds (double size when T is full)
• Sequence of n inserts on initially empty table
• Worst-case cost of insert is in O(n)
• Worst-case cost of sequence of n inserts is in
  O(n2)

LOOSE
elementary insertion
13
Dynamic Tables Table Expansion (cont)
Amortized Analysis

• Aggregate Method
• ci i if i-1 is exact power of 2
• 1 otherwise
• total cost of n inserts

• Accounting Method
• intuition for 3
• each item pays for 3 elementary insertions
• inserting itself into current table
• expansion moving itself
• expansion moving another item that has already
  been moved

14
Dynamic Tables Table Expansion (cont)
Amortized Analysis

• Potential Method
• Value of potential function F(T)
• 0 right after expansion
• builds to table size by time table is full
• always nonnegative, so sum of amortized costs of
  n inserts is upper bound on sum of actual costs

• Amortized cost of ith insert
• Fi potential after ith operation
• Case 1 insert does not cause expansion
• Case 2 insert causes expansion

use these
15
Dynamic TablesTable Expansion Contraction

• Load factor bounds
• (double size when T is full) (halve size when T
  is ¼ full)
• Delete pseudocode analogous to insert

Amortized Analysis

• Potential Method
• Value of potential function F(T)
• 0 for empty table
• 0 right after expansion or contraction
• builds as a(T) increases to 1 or decreases to ¼
• always nonnegative, so sum of amortized costs of
  n inserts is upper bound on sum of actual costs
• 0 when a(T)1/2
• numT when a(T)1
• numT when a(T)1/4

16
Dynamic TablesTable Expansion Contraction
Amortized Analysis

• Potential Method

17
Dynamic TablesTable Expansion Contraction
Amortized Analysis

• Potential Method

• Analyze cost of sequence of n inserts and/or
  deletes
• Amortized cost of ith operation
• Case 1 insert
• Case 1a ai-1 gt ½. By previous insert analysis
• Case 1b ai-1 lt ½ and ai lt ½
• Case 1c ai-1 lt ½ and ai gt ½

holds whether or not table expands
no expansion
no expansion
18
Dynamic TablesTable Expansion Contraction
Amortized Analysis

• Potential Method

• Amortized cost of ith operation (continued)

• Case 2 delete
• Case 2a ai-1 gt ½.
• Case 2b ai-1 lt ½ and ai lt ½
• Case 2c ai-1 lt ½ and ai lt ½

no contraction
contraction
Conclusion amortized cost of each operation is
bounded above by a constant, so time for sequence
of n operations is O(n).
19
Example Dynamic Closest Pair
S
S
S
source Fast hierarchical clustering and other
applications of dynamic closest pairs, David
Eppstein, Journal of Experimental Algorithmics,
Vol. 5, August 2000.
20
Example Dynamic Closest Pair (continued)
S
S
S
source Fast hierarchical clustering and other
applications of dynamic closest pairs, David
Eppstein, Journal of Experimental Algorithmics,
Vol. 5, August 2000.