Data Flow Analysis for Software Prefetching Linked Data Structures in Java

1
Data Flow Analysis for Software Prefetching
Linked Data Structures in Java

• Brendon Cahoon
• Dept. of Computer Science
• University of Massachusetts
• Amherst, MA

Kathryn S. McKinley Dept. of Computer
Sciences University of Texas at Austin Austin, TX
2
Motivation

• Object-oriented languages are mainstream
• Key performance issues
• Same old processor-memory gap, parallelism
• Combination of modern processors and languages
  results in poor memory performance

3
RSIM PerformanceCompiled Java (Vortex) no GC
4
Prefetching Arrays vs. Objects

• Most prior work concentrates on arrays
• Compilers directly prefetch any element
• Loop transformations enable effective scheduling
• Successful results using both hardware and
  software
• Cannot use same techniques on linked data
  structures
• Objects are small and disjoint
• Access patterns are less regular and predictable
• Only know the address of directly connected
  objects

5
Software Data Prefetching for Java

• Hide memory latency from linked structure
  traversals
• Introduced by Luk and Mowry for C programs
• We add data flow and interprocedural analysis
• Identify pointer structures from declaration
• Find pointer chasing in loops and self recursive
  calls
• Challenges introduced by Java
• Dynamically allocated objects make analysis
  difficult
• Small methods obscure context

6
Outline

• Data flow analysis for identifying linked
  structures
• New intra and interprocedural analysis
• Greedy prefetching
• Jump-pointer prefetching
• Experimental results

7
Identifying Linked Structure Traversals

• We define a data flow solution
• Intraprocedural for loops
• Interprocedural for recursion
• Benefits
• Independent of program representation
• Many compilers use data flow frameworks
• May be composed with other analyses

• Loop
• while (o ! null)
• t o
• o t.next
• Recursion
• method visit()
• .
• if (this.next ! null)
• visit(this.next)

8
Data Flow Analysis

• Data flow information
• Sets of tuples ltvariable, field name,
  statement, statusgt
• Status values not recurrent, possibly, recurrent
• Not recurrent initial value
• Possibly first use of a field reference
• Recurrent an object accessed in linked
  structure traversal
• Intraproceedural forward, flow-sensitive, may
  analysis
• Interprocedural bidirectional, context-sensitive

9
Analysis Examples

• while (o ! null)
• s1 t o.next
• s2 o t
• while (o ! null)
• s1 o o.next
• s2 o bar()
• s1 o o.next
• s2 o o.next

• 1st Iteration
• s1 o is not recurrent, set t to possibly
• s2 t is possibly, set o to possibly

2nd Iteration s1 o is possibly, set t
to recurrent s2 t is recurrent, set o
to recurrent
1st Iteration s1 set o to possibly s2 set o
to not recurrent
s1 set o to possibly s2 set o to possibly
10
Analysis Extensions for Common Idioms

• Track objects in fields or arrays
• Class based field assignments
• Arrays are monolithic
• Indirect recurrent objects
• Unique objects referenced by linked structures

while (e.f ! null) o e.f e.f o.next
o.compute()
while (e.hasMoreElements()) o
(ObjType)e.nextElement() o.compute()

11
Greedy Prefetching

• Prefetch directly connected objects
• Algorithm consists of two steps
• Detect accesses to linked structures
• Schedule prefetches
• When object is not null
• Completely hiding latency is difficult

12
Greedy Prefetching Example
int sum (Dlist l) int s 0 while (l
! null) s l.data l
l.next return s
Doubly linked list
13
Greedy Prefetching Example
int sum (Dlist l) int s 0 while (l
! null) prefetch(l.next) s
l.data l l.next return s
Doubly linked list Greedy prefetching
14
Jump-Pointer Prefetching

• Prefetch indirectly connected objects
• Tolerates more latency than greedy prefetching
• Algorithm contains three steps
• Find linked data structure traversal and creation
  sites
• Create jump-pointers
• When creating or traversing the linked structure
• Schedule prefetches
• Prefetch special jump-pointer field

15
Inserting Jump-Pointers at Creation Time

• Void add(ObjType o)
• ListNode n new ListNode(o)
• jumpObj jumpQueuei
• jumpObj.jmp n
• jumpQueueisize n
• if (head null)
• head n
• else
• tail.next n
• tail n

1
2
3
4
5
jumpObj n
16
Jump-Pointer Prefetching Example
int sum (Dlist l) int s 0 while (l
! null) prefetch(l.jmp) s
l.data l l.next return s
Doubly linked list Jump-pointer prefetching
17
Experimental Results

• Object-oriented Olden benchmarks in Java
• Simulation using RSIM
• Out-of-order, superscalar processor
• Compile programs using Vortex
• Translate Java programs to Sparc assembly
• Contains object-oriented, traditional
  optimizations
• Linked structure analysis, greedy and
  jump-pointer prefetching

18
Prefetching Performance
health mst perimtr treeadd bh
bisort tsp voronoi em3d power
19
Prefetch Effectiveness
health mst perimtr treeadd bh
bisort tsp voronoi em3d power
20
Static Prefetch Statistics
Program Interprocedural Interprocedural Intra- Procedural Fields
Program Mono Poly Intra- Procedural Fields
Health 8 1 5
Mst 3
Perimeter 9 8
Treeadd 2
BH 16 8 10
Bisort 4 4
Tsp 6 14
Voronoi 14 1
Em3d 20
Power 4
21
Contributions and Future Work

• New interprocedural data flow analysis for Java
• Evaluation of prefetching on Java programs
• Prefetching hides latency, but
• Room for improvement
• Other uses for analysis (work in progress)
• Garbage collection prefetching, object traversal
• Prefetching arrays of objects