Functional Coverage Driven Test Generation for Validation of Pipelined Processors

1
Functional Coverage Driven Test Generation for
Validation of Pipelined Processors

• Prabhat Mishra
• Dept. of Computer and Information Science and
  Engineering
• University of Florida, USA
• Nikil Dutt
• Center for Embedded Computer Systems
• Donald Bren School of Information and Computer
  Sciences
• University of California, Irvine, USA

2
Outline

• Functional Verification of Microprocessors
• A Challenge
• Related Work
• Architecture Model
• Functional Fault Models
• Test Generation Techniques
• A Case Study
• Conclusion

3
Functional Verification of SOC Designs
2000
1000B
2007
200
10B
2001
Engineer Years
Simulation Vectors
100M
20
1995
100M
1M
10M
Logic Gates
Source Synopsys
71 of SOC re-spins are due to logic bugs
Source G. Spirakis, keynote address at DATE 2004
4
Functional Validation of Microprocessors

• Functional validation is a major bottleneck
• Deeply pipelined complex microarchitectures
• Logic bugs increase at 3-4 times/generation
• Bugs increase (exponential) is linear with design
  complexity growth.

5
Pentium 4 Bugs Breakdown
Source Bob Bentley, HLDVT 2002
Micro-architectural complexity is a major
contributor
6
Bottlenecks of Functional Verification

• Lack of high-level models to
• Capture micro-architectural details
• Enable early functional validation
• Lack of a suitable functional coverage metric
• Code coverage, toggle coverage not sufficient
• Cannot determine if all pipeline interactions
  (with hazards/exceptions) are considered.
• Our contribution
• High-level modeling of pipelined architectures
• Propose functional fault models
• Define functional coverage
• Coverage-driven test generation techniques

7
Related Work

• Directed test program generation
• Aharon et al., DAC 1995, Shen et al., DAC 1999,
  Fine et al., DAC 2003.
• Test generation for pipelined processors
• Ur and Yadin, DAC 1999
• Iwashita et al., ICCAD 1994
• Campenhout et al., DAC 1999
• No coverage metric for pipeline interactions
• Functional test program generation
• Chen et al., DAC03, Lai and Cheng, DAC01
• Thatte et al., IEEE Computers, 1980
• Applied in the context of manufacturing testing

8
Functional Validation of Pipelined Processors
Test Generator
Pipelined Processor
TestGen
MOV R1, 011 MOV R2, 010 ADD R3, R1, R2 R3 101
Test Program
R3 101 ?
Check Result
Verifies the functionality of the processor using
assembly programs
9
Functional Validation of Pipelined Processors
Test generation is considered in this work
Test Generation
Pipelined Processor
TestGen
MOV R1, 011 MOV R2, 010 ADD R3, R1, R2 R3 101
Test Program
R3 101 ?
Check Result
10
Test Generation for Microprocessors

• Three components
• Architecture model
• Pipelined processor
• Functional fault models
• Functional coverage
• Test generation techniques
• Coverage-driven test generation

11
Outline

• Introduction
• Related Work
• Architecture Model
• Functional Fault Models
• Test Generation Techniques
• A Case Study
• Conclusion

12
Architecture Model of a Pipelined Processor

• Modeling plays a central role in test generation
• Requirements for a good architecture model
• Based on functional description from a user
  manual.
• Treats the processor organization and
  instruction-set as parameters of the test
  generation procedures.
• Supports a functional fault model describing
  faults in various computations.
• Allows fault model developers to describe faulty
  behavior without knowing the details of the
  implementation.
• We use a graph-based model to capture the
  structure and behavior of the processor

13
Architecture Model - Structure
Graph Model
Graph (Nodes, Edges) Nodes units U
storages Edges data-transfer edges U
pipeline edges
14
Architecture Model - Behavior
OFFSET
SRC1
SRC
STORE
(OPCODE ADD (OPERANDS (SRC1 reg) (SRC2 imm)
(DEST reg)) (BEHAVIOR DEST SRC1 SRC2)
)
(OPCODE STORE (OPERANDS (SRC reg) (SRC1 reg)
(OFFSET imm)) (BEHAVIOR MSRC1OFFSET
SRC) )
15
Architecture Model - Mapping

• Mapping between the graph nodes in the structure
  and behavior

Fetch
L1I
Decode
ALU
AddrCalc
opcode-to-unit
RF
argument-to-storage
LdSt
L1D
L2
MM
WB
Pipeline edge
Unit
Data-transfer edge
Storage
16
Outline

• Introduction
• Related Work
• Architecture Model
• Functional Fault Models
• Test Generation Techniques
• A Case Study
• Conclusion

17
Functional Fault Models

• Register Read/Write
• All registers are written and read.
• Operation Execution
• All operations are executable.
• Execution Path
• Each execution path (taken by an operation) works
  correctly
• Consists of one pipeline path and multiple
  data-transfer paths
• Pipeline Execution
• All pipeline interactions are activated.

18
Register Read/Write Faults

• The fault can be due to an error in
• reading
• register decoding
• register storage
• prior writing
• Whatever may be the reason, the outcome is an
  unexpected value.

19
Operation Execution Faults

• The fault can be due to an error in
• Operation decoding
• erroneous decoding returns incorrect opcode
• Control generation
• incorrect execution unit gets selected
• Final implementation
• execution unit can be faulty
• The outcome is an unexpected result.

20
Faults in Execution Path

• Execution path
• During execution of an operation, one pipeline
  path and one/more data-transfer paths get
  selected
• these activated paths are defined as execution
  path
• The fault can be due to an error in any of the
  paths
• A path is faulty if any of its nodes or edges are
  faulty
• A node is faulty if does not execute correctly
• An edge is faulty if it does not transfer
  data/inst. correctly
• The outcome is an unexpected result.

21
Pipeline Execution Faults

• The fault can be due to an incorrect
  implementation of the pipeline controller
• Erroneous hazard detection
• Incorrect stalling
• Erroneous flushing
• Wrong exception handling
• The outcome is an unexpected result.

22
Functional Fault Models

• Register Read/Write
• Independent of implementation (pipeline).
• Operation Execution
• Independent of implementation (pipeline).
• Execution Path
• Considers one operation in the pipeline.
• Pipeline Execution
• Presence of multiple operations in the pipeline.

23
Functional Coverage

• Faults detected (numerator)
• A fault is covered if it is activated by an
  instruction sequence and the result is read.
• Detectable faults (denominator)
• A processor with 32 registers, 101 instructions
• 32 reg. read/write, 101 operation execution
  faults

24
Outline

• Introduction
• Related Work
• Architecture Model
• Functional Fault Models
• Test Generation Techniques
• A Case Study
• Conclusion

25
Coverage-driven Test Generation

• Developed test generation algorithms for
• Register read/write
• Operation execution
• Execution path
• Pipeline execution

26
Test Generation for Register Read/Write

• Algorithm 1
• Input Graph model of the architecture G.
• Output Test programs for detecting faults in
  reg. read/write.
• begin
• TestProgramList
• for each register reg in architecture G
• valuereg GenerateUniqueValue(reg)
• writeInst an instruction that writes valuereg
  in reg.
• testprogreg createTestProgram(writeInst)
• TestProgramList TestProgramList ?
  testprogreg
• endfor
• return TestProgramList
• end

CreateTestProgram 1. Assigns values to
unspecified locations 2. Creates initialization
instructions for sources 3. Creates instructions
for reading destinations
27
Test Generation for Operation Execution

• Algorithm 2
• Input Graph model of the architecture G.
• Output Tests for detecting faults in operation
  execution
• begin
• TestProgramList
• for each operation oper in architecture G
• testprogoper createTestProgram(oper)
• TestProgramList TestProgramList ? testprogoper
  
• endfor
• return TestProgramList.
• end

28
Test Generation for Execution Path

• Input Graph model of the architecture G.
• Output Test programs for detecting faults in
  execution path.
• begin
• TestProgramList
• for each pipeline path path in architecture G
• opgrouppath operations supported in path.
• execpath path and all data-transfer paths
  connected to it
• operpath randomly select an operation from
  opgrouppath
• if (operpath activates all edges in execpath)
  opspath operpath
• else opspath opgrouppath endif
• for all operations oper in opspath
• for all source/destination operands opnd of
  oper
• for all possible register values val of opnd
• newOper assign val to opnd of oper.
• test progoper createTestProgram(newOper).
• TestProgramList TestProgramList ?
  testprogoper
• endfor
• endfor
• endfor

29
Test Generation for Pipeline Execution

• Begin
• L1 for each unit node unit in architecture G
• L2 for each exception exon possible in unit
• templateexon template for exception exon
• test progunit createTestProgram(templateexon
  )
• TestProgramList TestProgramList ?
  testprogunit
• endfor
• L3 for each hazard haz in RAW, WAW, WAR,
  control
• templatehaz template for hazard haz
• if haz is possible in unit
• testprogunit createTestProgram(templatehaz
  )
• TestProgramList TestProgramList ?
  testprogunit
• endif
• endfor
• L4 for each parent unit parent of unit
• operparent an operation supported by parent
• resultOps createTestProgram(operparent )
• test progunit a test program to stall unit
  (if exists)
• test progparent resultOps ? testprogunit

30
Outline

• Introduction
• Related Work
• Architecture Model
• Functional Fault Models
• Test Generation Techniques
• A Case Study
• Conclusion

31
A Case Study

• Applied on two pipelined architectures
• VLIW implementation of DLX
• RISC implementation of Sparc V8 (LEON)
• Architecture Specification
• Using Architecture Description Language (ADL)
• EXPRESSION ADL
• Test generation and coverage estimation
• Random/Constrined-random test generation
• Using Specman Elite framework
• Coverage-driven test generation
• Using our test generation algorithms

32
Test Generation and Coverage Estimation
Automatic
Architecture Specification (ADL Description)
Manual
ISA Specification (e Description)
Coverage Specification
Pipelined Implementation (e Description)
Coverage Estimation
Simulator
Random
Test Generation
Specman Elite
Directed
External Test Programs (generated by our
algorithms)
33
Specification of the DLX Processor
PC
Memory
Fetch
Structure
( ARCHITECTURE_SECTION ..........
(FetchUnit Fetch (CAPACITY 4) (TIMING (all
1)) (OPCODES all) (LATCHES (OTHER
PCLatch)(OUT DLatch)) ) ( PIPELINE_SECTION
(PIPELINE Fetch Decode Execute MEM WB) (Execute
(ALTERNATE ALU MUL FADD DIV)) (FADD (PIPELINE
FADD1 .. FADD3 FADD4)) (DTPATHS (TYPE
UNI (RF Decode P7 C4 P8) (WB RF P5 C3
P6) ) (TYPE BI (MEM MEMORY P4 C2 P3) ) )
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
FADD3
FADD4
MUL7
MEM
WriteBack
34
Specification of the DLX Processor
PC
Memory
Fetch
Structure
( ARCHITECTURE_SECTION ..........
(FetchUnit Fetch (CAPACITY 4) (TIMING (all
1)) (OPCODES all) (LATCHES (OTHER
PCLatch)(OUT DLatch)) ) ( PIPELINE_SECTION
(PIPELINE Fetch Decode Execute MEM WB) (Execute
(ALTERNATE ALU MUL FADD DIV)) (FADD (PIPELINE
FADD1 .. FADD3 FADD4)) (DTPATHS (TYPE
UNI (RF Decode P7 C4 P8) (WB RF P5 C3
P6) ) (TYPE BI (MEM MEMORY P4 C2 P3) ) )
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
FADD3
FADD4
MUL7
MEM
WriteBack
35
Specification of the DLX Processor
PC
Memory
Fetch
Structure
( ARCHITECTURE_SECTION ..........
(FetchUnit Fetch (CAPACITY 4) (TIMING (all
1)) (OPCODES all) (LATCHES (OTHER
PCLatch)(OUT DLatch)) ) ( PIPELINE_SECTION
(PIPELINE Fetch Decode Execute MEM WB) (Execute
(ALTERNATE ALU MUL FADD DIV)) (FADD (PIPELINE
FADD1 .. FADD3 FADD4)) (DTPATHS (TYPE
UNI (RF Decode P7 C4 P8) (WB RF P5 C3
P6) ) (TYPE BI (MEM MEMORY P4 C2 P3) ) )
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
FADD3
FADD4
MUL7
MEM
WriteBack
36
Specification of the DLX Processor
PC
Memory
Fetch
Structure
( ARCHITECTURE_SECTION ..........
(FetchUnit Fetch (CAPACITY 4) (TIMING (all
1)) (OPCODES all) (LATCHES (OTHER
PCLatch)(OUT DLatch)) ) ( PIPELINE_SECTION
(PIPELINE Fetch Decode Execute MEM WB) (Execute
(ALTERNATE ALU MUL FADD DIV)) (FADD (PIPELINE
FADD1 .. FADD3 FADD4)) (DTPATHS (TYPE
UNI (RF Decode P7 C4 P8) (WB RF P5 C3
P6) ) (TYPE BI (MEM MEMORY P4 C2 P3) ) )
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
FADD3
FADD4
MUL7
MEM
WriteBack
37
Specification of the DLX Processor
PC
Memory
Fetch
Structure
( ARCHITECTURE_SECTION ..........
(FetchUnit Fetch (CAPACITY 4) (TIMING (all
1)) (OPCODES all) (LATCHES (OTHER
PCLatch)(OUT DLatch)) ) ( PIPELINE_SECTION
(PIPELINE Fetch Decode Execute MEM WB) (Execute
(ALTERNATE ALU MUL FADD DIV)) (FADD (PIPELINE
FADD1 .. FADD3 FADD4)) (DTPATHS (TYPE
UNI (RF Decode P7 C4 P8) (WB RF P5 C3
P6) ) (TYPE BI (MEM MEMORY P4 C2 P3) ) )
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
FADD3
FADD4
MUL7
MEM
WriteBack
38
Specification of the DLX Processor
PC
Memory
Fetch
Structure
( ARCHITECTURE_SECTION ..........
(FetchUnit Fetch (CAPACITY 4) (TIMING (all
1)) (OPCODES all) (LATCHES (OTHER
PCLatch)(OUT DLatch)) ) ( PIPELINE_SECTION
(PIPELINE Fetch Decode Execute MEM WB) (Execute
(ALTERNATE ALU MUL FADD DIV)) (FADD (PIPELINE
FADD1 .. FADD3 FADD4)) (DTPATHS (TYPE
UNI (RF Decode) (WB RF) ) (TYPE
BI (MEM MEMORY) ) )
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
FADD3
FADD4
MUL7
MEM
WriteBack
39
Specification of the DLX Processor
Structure
PC
Memory
Fetch
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
Behavior
(OPCODE ADD (OPERANDS (SRC1 rf) (SRC2 imm)
(DEST rf)) (BEHAVIOR DEST SRC1 SRC2)
(FORMAT ) )
FADD3
FADD4
MUL7
MEM
WriteBack
40
Specification of the DLX Processor
Structure
PC
Memory
Fetch
Decode
Register File
DIV
FADD1
IALU
MUL1
FADD2
MUL2
Behavior
Mapping
(OPCODE ADD (OPERANDS (SRC1 rf) (SRC2 imm)
(DEST rf)) (BEHAVIOR DEST SRC1 SRC2)
(FORMAT ) )
FADD3
FADD4
MUL7
MEM
WriteBack
41
Coverage Estimation

• Instruction definition is used
• opcode, dest, src1, src2
• Register read/write
• coverage of src1 and src2 indicates reads.
• coverage of dest indicates writes.
• Operation execution
• coverage of opcode field
• Pipeline execution
• use variable for each stall/exception
• cross-coverage is used to estimate coverage of
  multiple exception scenarios.

42
Validation Flow
43
Test Generation for VLIW DLX
An entry indicates number of test programs
indicates the fault coverage using the given test
programs for that fault model
Random or constrained-random techniques could not
activate any multiple exception scenarios -
Low coverage in pipeline execution
44
Quality of the Proposed Fault Model
VHDL implementation of DLX is used

• As expected, our fault model performed well
• A small set of test programs generated a high
  code coverage
• The test programs are the same ones listed in the
  previous slide (last column of the VLIW DLX table)

45
Test Generation for LEON2 Processor
An entry indicates number of test programs
indicates the fault coverage using the given test
programs for that fault model

• The trend is similar in both architectures.
• Due to bigger pipeline structure (more
  interactions) VLIW
• DLX has lower fault coverage than LEON2 model.

46
The Framework is Available

• https//www.verificationvault.com
• It includes
• VLIW DLX models
• e specification for reference (ISA model)
• Pipelined implementation in e.
• Components for random/directed test generation
  and incorporation of external tests
• Components for data/temporal checking and
  coverage estimation.

47
Summary

• Functional verification is a major challenge
• Due to lack of a comprehensive coverage metric
• Presented a coverage driven test generation
  technique for pipelined processors
• General architecture model (derived from ADL
  spec.)
• Proposed functional fault models and coverage
  metric
• Developed coverage-driven test generation
  algorithms
• Demonstrated using Specman Elite framework
• On VLIW DLX and LEON processors
• Future work
• Compare against other coverage measures.
• Apply for validation of todays microprocessors

48

• Thanks!

49
Instruction-Set Models

• type opcodes
• LB, SB, , ADDI, SUBI, // I-type opcodes
• ADD, SUB, MULT, // R-type opcodes
• J, JAL, TRAP, RFE, // J-type opcodes
• type registers R0, R1, R2, R31
• type immediate int (bits 16)
• struct instructions
• opcode opcodes
• when I_type instructions
• src1 cpu_reg_t dest
  registers src2 immediate
• when R_type instructions
• src1 cpu_reg_t src2
  registers dest registers
• when J_type instructions src1
  offset

50
Functional Coverage Specification

• Register Read/Write
• All registers are written and read.
• Operation Execution
• All operations are executable.
• Pipeline Execution
• All pipeline interactions are activated.

51
Test Generation

• Random
• Modification of a variable decides the number of
  tests to generate.
• Constrained-random
• Several constraints are used to generate
  directed tests
• For example, to generate tests for register
  read/write, R-type operations are used.
• R-type 3 register operands
• I-type 2 register operands
• J-type 0 register operands