An Overview of the SUIF2 Base Infrastructure

1
An Overview of the SUIF2 Base Infrastructure

• David Heine
• Stanford University
• http//suif.stanford.edu/

2
SUIF Development Overview

• I. Demo Interactive Compiler
• II. SUIF IR and Extensions
• III. Building a Pass
• IV. Annotations and IR Extensions
• V. Getting Started

3
I. Demo Interactive Compiler

• gt cat b.c
• int main() if (5 1 7)
  return(0) return(1)
• gt ls NCIHOME/bin executables
• gt c2suif b.c generate a
  .suif file
• gt sbrowser b.suif browse the
  .suif file
• gt ls NCIHOME/solib/lib.so DLLs

4
Running the Suifdriver

• SuifDriver - a simple standalone program that
  loads DLLs and executes modules
• gt suifdriver -i
• suifgt list_modules
• suifgt require usefulpasses
• suifgt list_modules
• suifgt list_interfaces
• suifgt load b.suif
• suifgt check_types
• suifgt require transforms
• suifgt fold_statements
• suifgt require s2c
• suifgt s2c -D ARCHLinux b.out.c
• gt cat b.out.c
• Shell/perl/tcl scripts that compose passes
• c2suif, suif2c, f2suif, pass_driver

5
TclSuif examples

• TclSUIF - using a TCL shell to load DLLs and
  execute modules
• gt cat /.wishrc
• source env(NCIHOME)/suif/suif2b/tclsuif/tclsuif/u
  tils.tk
• source env(NCIHOME)/suif/suif2b/tclsuif/tclsuif/t
  clsuif.tk
• suif_init
• gt wish8.0
• suif require transforms
• suif load b.suif
• show_object suif_fsb

6
II. The SUIF IR

• SUIF IR - An ownership tree embedded in a
  labelled directed graph
• Nodes are sufficient to handle semantics for C
  and Fortran
• Nodes are generic enough to be extended to handle
  Object semantics
• Object Hierarchy
• e.g. BranchStatement inherits from Statement
• Metaclass system provides for multiple
  independent extensions
• Ownership Tree
• e.g. An IfStatement owns a condition, then part,
  and else part.
• e.g. A LoadVariableExpression references its
  source variable.

7
SUIF IR Hierarchy
Object SuifObject SuifBrick IntegerBrick
StringBrick SuifObjectBrick OwnedSuifObjectB
rick AnnotableObject Annote
BrickAnnote GeneralAnnote SymbolTabl
eObject FileSetBlock GlobalInformationBlock
CInformationBlock TargetInformationBlock
SymbolTable BasicSymbolTable GroupSymbo
lTable ScopedObject FileBlock Definitio
nBlock ProcedureDefinition VariableDefinit
ion ValueBlock ExpressionValueBlock
MultiValueBlock RepeatValueBlock Undefin
edValueBlock ExecutionObject Statement
Expression
SymbolTableObject Type DataType ArrayType
MultiDimArrayType GroupType StructType
UnionType NumericType BooleanType Floa
tingPointType IntegerType EnumeratedType
PointerType VoidType LabelType Proce
dureType CProcedureType QualifiedType Symbol
VariableSymbol FieldSymbol NestingVariabl
eSymbol ParameterSymbol CodeLabelSymbol Pr
ocedureSymbol
Statement StatementList ScopeStatement WhileSta
tement DoWhileStatement ForStatement IfStatemen
t BranchStatement MultiWayBranchStatement JumpI
ndirectStatement JumpStatement ReturnStatement
StoreStatement StoreVariableStatement CallStatem
ent EvalStatement VaEndStatement VaStartOldStat
ement VaStartStatement LabelLocationStatement M
arkStatement
Expression Constant IntConstant FloatCons
tant BinaryExpression UnaryExpression SelectExp
ression SymbolAddressExpression LoadExpression
LoadValueBlockExpression LoadVariableExpression
ArrayReferenceExpression MultiDimArrayExpressio
n FieldAccessExpression VaArgExpression
8
Program Representation
owns
ProcedureDefinition
references
a b
SymbolTable
destination
StoreVariableStatement
a
source
b
LoadVariableExpression
9
The Top of the SUIF Ownership Tree

• SuifEnv contains a FileSetBlock
• FileSetBlock contains FileBlocks
• FileBlocks contain ProcedureDefinitions and
  VariableDefinitions
• ProcedureDefinitions contain ExecutionObjects
• ExecutionObjects
• Statements
• Expressions
• Machsuif instructions
• Machsuif instruction lists
• SymbolTables (Scopes) at all levels implicitly
  linked for lookup

10
Ownership Relationships
FileSetBlock external_symbol_table symbol_table
_objects file_set_symbol_table file_blocks
symbol_table definition_block variable_def
initions initialization procedure_definitio
ns body source_ops symbol_table
definition_block information_blocks
BasicSymbolTable, GroupSymbolTable symbol_table_
objects ExecutionObject source_ops
Statement child_statements StatementList stateme
nt (child_statements) ScopeStatement body
(child_statements) symbol_table definition_block
WhileStatement, DoWhileStatement condition
(source_ops) body (child_statements) ForStatement
lower_bound (source_ops) upper_bound
(source_ops) step (source_ops) body
(child_statements) pre_pad (child_statements) IfS
tatement condition (source_ops) then_part
(child_statements) else_part (child_statements) B
ranchStatement, MultiBranchStatement decision_ope
rand (source_ops) JumpIndirectStatement target
(source_ops) ReturnStatement return_value
(source_ops) StoreStatement value
(source_ops) destination_address
(source_ops) StoreVariableStatement value
(source_ops) CallStatement callee_address (
source_ops) EvalStatement expressions
(source_ops) VaStartStatement, VaStartOldStatement
, VaEndStatement ap_address (source_ops)
Expression BinaryExpression source1
(source_ops) source2 (source_ops) UnaryExpression
source (source_ops) SelectExpression selector
(source_ops) selection1 (source_ops) selection2
(source_ops) LoadExpression source_address
(source_ops) LoadValueBlockExpression value_block
(source_ops) ArrayReferenceExpression base_array
_address (source_ops) index (source_ops) MultiDim
ArrayExpression base_array_address
(source_ops) offset (source_ops) index
(source_ops) elements (source_ops) FieldAccessExp
ression base_group_address (source_ops) VaArgExpr
ession ap_address (source_ops)
SuifBrick OwnedSuifObjectBrick object AnnotableO
bject annotes BrickAnnote bricks Type ArrayType
lower_bound upper_bound MultiDimArrayType lower
_bounds upper_bounds GroupType group_symbol_tabl
e Symbol FieldSymbol bit_offset NestingVariableSy
mbol bit_offset ValueBlock ExpressionValueBlock
expression (source_ops) MultiValueBlock sub_block
RepeatValueBlock sub_block
11
III. Writing SUIF passes

• About Passes
• Building the EmptyPass
• Tools for writing non-empty passes
• Traversing the IR
• Changing code
• A selection of useful primative functions
• Cloning code
• Available data structures
• Building interesting passes
• Debugging Hints

12
What Passes Do

• Compose other passes
• Analyze the IR
• Annotate the IR
• Transform the IR
• Enhance the IR semantics

13
Before You Write a Pass

• Check if it already exists!
• Leverage existing tools to reduce the complexity
  of your pass
• Use available passes to simplify the SUIF input
• Use clean up passes after your pass
• Making it extensible
• Publish interfaces in the SuifEnv for extensions
• Use CascadingMapltgt for SuifObject specific actions

14
SUIF Pass Classes

• FrontendPass
• Produces a FileSetBlock
• implement FileSetBlock build_file_set_block(void)
  
• Pass
• Uses a FileSetBlock
• implement void do_file_set_block(FileSetBlock )
• PipelinablePass
• Subclass of Pass that uses procedures of a
  FileSetBlock
• implement void do_procedure_definition(ProcedureDe
  finition )
• BackendPass
• consumes a FileSetBlock
• implement void do_file_set_block(FileSetBlock )

15
Writing the EmptyPass

• Building the EmptyPass with command line options
• include ltsuifpasses/passes.hgtinclude
  ltsuifkernel/command_line_parsing.hinclude
  ltsuifkernel/suif_env.hgtinclude ltstdio.hgtclass
  EmptyPass public Pass public
  EmptyPass(SuifEnv env, LString name
  empty_pass) Pass(env, name),
  _verbose(false) virtual void initialize()
  Passinitialize() _command_line-gtset_descripti
  on(Empty pass that does nothing)
  _flags-gtadd(new OptionLiteral(-v, Do nothing
  verbosely, _verbose, true))virtual
  Module clone() const return(Module) this
  virtual void do_file_set_block(FileSetBlock )
  if(_verbose) printf(Verbose\n)protected
  bool _verboseextern C void
  init_empty(SuifEnv ) ...

16
Building the EmptyPass

• Standard Makefile for a library
• SRCSempty.cpp
• OBJ_FILESempty.o
• TARGET_LIB empty
• LIBS -lcommon -liokernel -lsuifkernel
  -lsuifpasses
• include (NCIHOME)/Makefile.std
• gt source NCIHOME/nci_setup.csh
• gt gmake
• gt ls (NCIHOME)/solib/libempty.so
• gt suifdriver -e require empty empty --help
• gt suifdriver -e require empty load b.suif
  empty_pass save b.copy.suif

17
Simplifying Your Pass

• Use available passes to put the code into the
  form you want
• Dismantle complex IR nodes
• IfStatements, WhileStatements, DoWhileStatements,
  ForStatements, ScopeStatements
• Add for guards, folding
• MultiDimArray transforms
• Clean up passes
• fold_statements copyprop, deadcode_pass
• rename_colliding_symbols - Use after creating new
  variables
• recycle_trash - Use after removing or replacing
  objects
• gc_symbol_table - Use after replacing variables

18
Tools for Traversing the IR

• Iterators
• imperative iteration over the ownership tree
• local iteration over an objects referenced
  objects
• local iteration over an objects named fields
• Walkers
• closure applied to IR objects
• Object Collections
• build a list of the IR objects

19
Tools for changing code (the utils directories)

• The Utils and cfeutils libraries have lots of
  useful interfaces for common functionality
• IR builders, folding
• Creating nodes
• Typebuilder - maintain some invariants
  (typebuilder)
• Nodebuilder - simplified object creation
• Substitution
• replace_expression, replace_statement
• Evaluation - folding
• Duplication - inlining and cloning

20
Useful Primitives

• Creating a type Object
• IntegerType create_integer_type(suif_env,
  bit_size, bit_alignment, is_signed)
• Using the typebuilder
• IntegerType TypeBuilderget_integer_type(bit_siz
  e, bi_alignment, is_signed)
• Replacing Expressions/Statements
• Expression replace_expression(Expression ,
  Expression )
• Statement replace_statement(Statement ,
  Statement )
• Removing Suif Objects
• SuifObject remove_suif_object(SuifObject obj)
• Inserting Statements
• Statement insert_statement_after (Statement
  loc, Statement )
• Statement insert_statement_before (Statement
  loc, Statement )

21
Cloning the Representation
owns
ProcedureDefinition
references
original
cloned
y clone_statement(x)
SymbolTable
y
x
StoreVariableStatement
a
StoreVariableStatement
LoadVariableExpression
b
LoadVariableExpression
22
Available Data Structures

• Templatized structures
• list, vector
• indexed_list
• maps, hash tables
• splay tree
• Segmented strings, string dictionary
• Lattice value classes
• Infinite precision integers, rationals
• Compiler structures
• bitvector, sparse bitvector
• directed graphs - adjacency list and dense matrix
  
• Directed graph algorithms
• iterated dominance frontiers
• SCCs

23
A Useful? Pass Example

• Count all direct calls
• class CountDirectCallPass public
  PipelinablePass void do_procedure_definition(Pr
  ocedureDefinition
  proc_def) int count for (IterltCallStatement
  gt iter object_iteratorltCallStatementgt(pr
  oc_def) iter.is_valid() iter.next())
  CallStatement call iter.current() if
  (get_procedure_target_from_call(call) ! NULL)
  count printf(Procedure s d
  direct calls\n, proc_def-gtget_procedure
  _symbol()-gt get_name().c_str(),
  count)

24
An Instrumentation Pass

• instrument all normal procedure entries and exits
• class InstrumentProcedureStack public
  PipelinablePass void do_procedure_definition(P
  rocedureDefinition proc_def) Statement
  pre_ret build_proc_push_statement(proc_def)
  Statement enter_proc build_proc_pop_stateme
  nt(proc_def) suif_assert_message
  (is_kind_ofltStatementgt
• (proc_def-gtget_body()),
  (Expected Statement)) / insert
  instrumentation code on procedure entry /
  insert_statement_before(toltStatementgt(
  proc_def-gtget_body()), enter_proc)
  listltReturnStatementgt l
  collect_objectsltReturnStatementgt(proc_def)
  for (listltReturnStatementgtiterator iter
  l-gtbegin() iter ! l-gtend() iter)
  ReturnStatement ret iter
  insert_statement_before(ret,
• deep_suif_cloneltStatementgt(pre_re
  t)) delete pre_ret delete l

25
IV. Annotations and IR Extensions

• Annotating the IR - LString-indexed information
• Flat Annotations - list of Bricks
• Structured annotations
• Extending the IR- adding semantic information
• Adding new Expression or Statement Types
• What is the difference?
• Annotations may be safely removed

26
When to Create New IR nodes

• Building a Frontend Pass
• To capture high-level semantics for analysis
  (OSUIF)
• To capture AST (abstract syntax tree) for
  software engineering reasons
• Build new IR nodes to match the parse tree (e.g.
  cfor)
• Build dismantlers to standard representation
• High level semantic operations
• Succinct representation - easy to read, easy to
  transform further
• e.g. SPMD nodes
• Persistent interfaces between passes
• e.g. region graph, ecr nodes, interprocedural SSA

27
Extending the RepresentationAdding Annotations

• Unstructured Annotation
• void mark_direct_target(SuifEnv s, CallStatement
  call,
• ProcedureSymbol ps)
• SuifObjectBrick br create_suif_object_brick(s
  , ps)
• BrickAnnote an create_brick_annote(s,
  call_targets)
• an-gtappend_brick(br)
• call-gtappend_annote(an)
• Structured Annotation
• concrete CallTargetAnnote GeneralAnnote
• listltProcedureSymbol referencegt targets
• void mark_direct_target(SuifEnv s, CallStatement
  call,
• ProcedureSymbol ps)
• CallTargetAnnote an
• create_call_target_annote(s,
  call_targets)
• an-gtappend_target(ps)
• call-gtappend_annote(an)

28
Extending the RepresentationAdding IR Nodes -
SPMDStatement

• An Interface for SPMD code transformation
• concrete SPMDStatement Statement
• interface to spmd dismantler, a lower level
  
• code generator
• SymbolTable owner spmd_symbol_table
• build create_basic_symbol_table(
  _suif_env )
• ParameterSymbol reference spmd_pid
• VariableSymbol reference spmd_nproc
• Statement owner spmd_body in
  child_statements
• Statement owner serial_body in
  child_statements
• listltSPMDAccess ownergt spmd_externals
• listltSPMDTransform ownergt spmd_transforms
• for spmd dismantler implementation
• Statement owner serial_init default 0
• Expression owner serial_par_test default
  0

29
Debugging Hints

• Use existing passes to check the consistency of
  the IR
• check_types - type checker
• validate_suif - object ownership validation
• Generate the problem with a smaller input program
• Debugging with dynamically loaded libraries
• Build a standalone pass
• Dealing with GDB and DLLs
• break main run break dll_init_hook continue
  break where_you_need_to_break
• set overload-resolution off

30
V. Getting Started

• Visit Web page The SUIF2 Infrastructure Guide
  (http//suif.stanford.edu/suif/suif2/doc/)
• Overview of the SUIF2 system (postscript)
• Download and run the compiler
• Available programs and compiler modules
• Writing your own SUIF pass
• SUIF program representation guide (postscript)
• Organization of the SUIF source directories
• SUIF programmer guide (postscript)
• SUIF interfaces guide
• Examples of a pass and an IR module using hoof
• SUIF System Programmers Reference
• All the high-level interfaces documented
  (example)
• Generated from the .h files by Doxygen
• All the cross references, e.g. class hierarchy
  information