Self-Adjusting Computation Umut Acar [e-mail] Carnegie Mellon University

1
Self-Adjusting ComputationUmut
Acarumut_at_cs.cmu.eduCarnegie Mellon University
2
Example Static Convex Hulls

• Convex hull smallest enclosing polygon

3
Example Dynamic Convex Hulls

• Insert/delete points

4
Example Dynamic Convex Hulls

• Insert/delete points

5
Kinetic Convex Hulls
6
Algorithms Community

• Problem specific solutions
• Many papers
• Agarwal, Atallah, Basch, Bentley, Chan, Cohen,
  Demaine, Eppstein, Even, Frederickson, Galil,
  Guibas, Henzinger, Hershberger, King, Italiano,
  Janardan, Mehlhorn, Micciancio, Mulmuley,
  Overmars, Sleator, Tamassia, Tarjan, Thorup...
• Efficient algorithms but complex

7
Programming-LanguagesCommunity

• Incremental computation
• Static-to-dynamic transformation
• Many papers
• Alpern, Demers, Field, Hoover, Horwitz, Hudak,
  Liu, de Moor, Paige, Pugh, Reps, Ryder, Strom,
  Teitelbaum, Weiser, Yellin...
• Most effective techniques
• Static dependence graphs
• Memoization
• Work well for a limited class of problems

8
Self-Adjusting Computationthe Best of Both
Worlds

• Language techniques
• Algorithmic techniques
• Dynamize any persistent program
• Persistent data structures
• Any single assignment or functional program
• Closely match problem-specific performance
• Sorting
• Kinetic Data Structures
• Dynamic Trees

9
Representing Executions withFunction Call Trees

• Control Dependences

main
g
f
h
k
m
10
Dynamic Dependence Graph Call Tree Data
Dependences

• Control Dependences
• Data Dependences

main
g
f
h
k
m
1
2
3
c
b
a
11
Change Propagation
main
g
f
h
k
m
1
2
3
c
b
a
12
Change Propagation
main
g
f
h
k
m
1
2
3
0
c
b
a
13
Change Propagation
main
g
f
h
k
m
1
2
3
0
c
b
a
14
Change Propagation
main
g
f
h
k
0
2
3
c
b
a
15
Change Propagation
main
g
f
h
k
5
0
2
3
c
b
a
16
Change Propagation
main
g
f
h
k
5
0
2
3
c
b
a
17
Change Propagation
main
g
f
h
k
0
2
5
c
b
a
18
Change Propagation
main
g
f
h
k
0
2
5
c
b
a
19
SummaryChange Propagation
main
g
f
h
k
m
1
2
3
0
c
b
a
20
SummaryChange Propagation
main
main
g
f
g
f
h
h
k
m
k
0
2
5
1
2
3
0
c
b
a
c
b
a
Acar,Blelloch,Harper POPL 2002
21
Trace Call Trees Values Read
3
5
5
4
8
3
2
1
5
8
22
Trace Call Trees Values Read
3
3
5
5
5
7
4
8
3
2
4
4
1
1
5
8
1
5
5
23
Edit Distance Time for Red Calls
3
3
5
5
5
7
4
8
3
2
4
4
1
1
5
8
1
5
5
24
Trace Stability A Measure of Input
Sensitivity
3
3
5
5
5
7
4
8
3
2
4
4
1
1
5
8
1
5
5
Acar,Blelloch,Harper,Vittes,Woo SODA 2004
25
Dynamic Dependence GraphsRe-execute Functions
from Scratch
26
Combine DDGs MemoizationRe-use Subtrees
27
Adaptive Memoization Re-use Approximate Subtrees
Acar,Blelloch,Harper 2004
28
Adaptive Functional Programming
Acar,Blelloch,Harper POPL 02

• Algorithmic and language techniques for
  self-adjusting computation
• Efficient representation for dynamic DGs
• Selective dependence tracking
• Changeable values
• Values sensitive to input change
• Stable values
• Value insensitive to input change

29
Modifiable References

• Holds a changeable value
• Change and propagate
• Primitives
• mod create a modifiable
• read read a modifiable
• write write to a modifiable

30
Example with Modifiables

• reads must write

a mod (write 2) b mod (read xa in
write(f(x)))
2
a
f(2)
b
31
Modifiables andDynamic Dependence Graphs
read xa in
a mod (write 2) b mod (read xa in
write(f(x)))
f
2
a
f(2)
b
32
Control Dependences withTime Stamps
1

• Edges Reads Time Interval
• Time-stamps created by read/write
• Dietz-Sleator data structure
• Overhead
• Time O(1)
• Space O(reads mods)

2
a
4
4-15
read xa...
b
f(2)
read xa...
15
Execution
33
Standard Quicksort
datatype int list NIL CONS
(int int list) fun qsort (l int list)
fun qs (l,rest) case l of NIL gt
rest CONS(h,t) gt (smaller,
bigger) partition(h,t) sbigger qs
(bigger,rest) qs (smaller,
CONS(h,sbigger)) begin qs(l,NIL) end
34
Self-Adjusting Quicksort
datatype int mlist NIL
CONS (int (int mlist) mod) fun qsort (l int
mlist) fun qs (l,rest) read ll in
case l of NIL gt write (rest)
CONS(h,t) gt (less,bigger)
partition(h,t) sbigger mod
(qs(bigger,rest))) qs(less,CONS(h,sbigger
)) begin mod (qs(l,NIL)) end
35
Execute, Change, and Propagate
( Step 1 Execute ) in 4,5,1,0,2 mod_in
modList.fromList(in) mod_out qsort
(mod_in) print(mod_out) ( 0,1,2,4,5
) ( Step 2 Change Input ) insert
(mod_in,3,5) ( Step 3 Update
) propagate() print(mod_out) (
0,1,2,3,4,5 )
36
Performance of Quicksort

• Self-adjusting Quicksort updates its output
• in expected O(logn) time for changes at the end
  of the input
• in expected O(n) time for changes at the
  beginning of the input
• Merge Sort Expected O(logn) any change
• Insertion Sort Expected O(n) any change
• Sum/Min/Max on lists Expected O(logn)

37
Trace Stability and PerformanceSODA 04

• Theorem If program P is O(f(n))-stable for some
  class of input changes D then using Dynamic DGs
  output-update takes
• O(q(n) f(n)) time.
• q(n) time for the priority queue
• Worst case q(n) O(log(f(n))
• Typical case q(n) O(1)
• Theorem reduces dynamic problems to static

38
Kinetic Data Structures

• Introduced by Basch,Guibas,Hershberger 97
• Compute a property of moving objects
• Example Kinetic convex hulls
• Previous work Hand-designed, hand-coded

39
Our Work Automatic KinetizationAcar,Blelloch,V
ittes, 04

• Comparisons modifiables
• Compute the failure time of comparisons
• When a comparison fails
• Flip its value (true ! false, false ! true)
• Change propagate
• Applications
• Mergesort, Quicksort expected O(1) stable
• Mergehull expected O(logn) stable
• Quickhull, Chans No meaningful bounds

40
Static Quickhull
fun findHull(line as (p1,p2),l,hull) ll
filter l (Geo.lineside line) case ll of
NIL gt CONS(p1, hull) _ gt pm
max (Geo.dist line) ll right
findHull((pm,p2),ll,hull,dest)
findHull((p1,pm),ll,right) fun quickHull l
(mx,xx) minmax (Geo.minX,
Geo.maxX) l findHull((mx,xx),l,CONS(xx,NIL)
41
Self-Adjusting Quickhull
fun findHull(line as (p1,p2),l,hull) ll
filter l (Geo.lineside line) mod(read llll in
case ll of NIL gt write (CONS(p1,
hull)) _ gt pm max (Geo.dist line)
ll right mod (findHull((pm,p2),ll,hull))
findHull((p1,pm),ll,right)) fun quickHull
l (mx,xx) minmax (Geo.minX,
Geo.maxX) l mod (findHull ((mx,xx),l,CONS(xx,NIL
)))
42
Kinetic Quickhull
fun findHull(line as (p1,p2),l,hull) ll
filter l (Kin.lineside line) mod(read llll in
case ll of NIL gt write (CONS(p1,
hull)) _ gt pm max (Kin.dist line)
ll right mod (findHull((pm,p2),ll,hull))
findHull((p1,pm),ll,right)) fun quickHull
l (mx,xx) minmax (Kin.minX,
Kin.maxX) l mod (findHull ((mx,xx),l,CONS(xx,NIL
)))
43
Kinetic Quickhull Demo
44
Dynamic Trees ProblemSleator, Tarjan, 83

• Fundamental problem
• Flow algorithms, dynamic graph algorithms
• Given a forest of weighted trees support
• Tree Changes e.g., insert/delete edges
• Queries e.g., heaviest edge on a path?
• Problem-specific solutions
• Link-Cut Trees Sleator,Tarjan, 83, 85
• Topology Trees Frederickson, 93
• Top TreesAlstrup,Holm,Licht.,Thorup, 97

45
Self-Adjusting Tree ContractionSODA 04

• Tree Contraction Miller, Reif, 85
• Randomization for symmetry breaking
• Expected O(logn)-stable
• An expected O(logn)-time solution to dynamic
  trees problem
• Simple, general, and efficient

46
Experimental ResultsLC-Trees SA-Tree
Contraction
microseconds
PathQuery
WeightChange
Insert Delete Edges
47
Other Applications and Queries

• Link-Cut Trees
• Path queries
• Subtree queries
• Non-local queries
• Centers
• Medians
• Diameters

• SA-Tree Contraction
• Path queries
• Subtree queries
• Non-local queries
• Centers
• Medians
• Diameters

48
Conclusion

• Foundations
• Dynamic Dependence Graphs
• AFL language
• Trace Stability
• Applications
• Sorting, min/max,...
• Kinetic Data Structures
• Dynamic trees
• Future Work Multiple write, more applications....

49
Credits

• Guy Blelloch
• Robert Harper
• Srinath Sridhar
• Robert Tarjan
• Jorge Vittes
• Virgina Vassilevska
• Renato Werneck
• Maverick Woo