Stop Writing Assertions! Efficient Verification Methodology

1
Stop Writing Assertions!Efficient Verification
Methodology

• Dave Whipp DVClub SV, August 2008

2
Todays Talk

• An Overview of Methodology Creation
• Methodology as User Interface
• Making ESL work as a Hardware Design Flow
• Evolution of a Interface Definition Language
• Transaction Level Assertions
• Transaction Level Debug

3
A HW Development Flow
Big Paper Spec
ISS Model
Verification
Coverage
Debug
Design
Checkers
Testbench
Tests
RTL
Synthesis
Directed
Clocks, Resets
C Model
Assertions
BFMs,
Assertions
Random
Scoreboard
TLMs
Formal
4
User Model for Running Tests
Run Predictor
Build Testbench

Generate Test
Compare Behaviors
Run DUT
5
Part 1

• Methodology Design

6
Purpose of a Flow

• Conceptual Framework
• Where do things live?
• How do things work?
• Keep out of the way
• Dont make life difficult for people
• Define Metaphors
• Subtly influence future directions
• Anchor for Variation
• Give people something to complain about

7
Two Philosophies

• Conformity
• Emphasize Standardization
• Diversity
• Emphasize Innovation
• Synergy or Conflict?

8
Understanding Variation

• To understand what to standardize
• you need to understand what not to standardize
• Personal Preferences
• Technical Aspects of the Designs
• Supporting Legacy
• Seeking the Next Big Thing

9
Personal Preferences
Emacs!
vi
10
Personal Preferences

• Choice of editor doesnt affect others
• At least, not much
• Choice of scripting language has greater impact
• But is encapsulated
• A scripts users dont see the implementation
  language
• Choice of HVL affects whole team
• Cant write E tests for a Vera testbench!
• But a unit testbench isnt seen by other units
• A good flow will allow encapsulation of
  preferences
• I can go to any unit and build run its tests
• Enables rapid localization of infrastructure
  issues

11
Technological Differences
12
Technical Characteristics
PCIE
Graphics Pipe
Frame Buffer
Video
Off-chip Memory
13
Reuse Vs Stagnation

• Reuse considered Good
• Avoid reinventing the wheel
• Build on the shoulders of giants
• Reuse invites Inertia
• Reuse can propagate dependencies
• Dependencies make things harder to change
• Resistance to change is known as inertia
• Inertia can lead to Stagnation
• Improper reuse accumulates dependencies
• Reused code that is not understood will bit-rot
• To avoid stagnation, inject agitation

14
Are Single Paradigm Projects Possible?
Paradigm 1
Unit A
Unit B
Unit B
Unit B
Paradigm 2
Unit C
Unit C
Paradigm 3
Unit D
time
Project 1
Project 2
Project 3
15
Watch some Real Users
Image courtesy of DAZ Productions
16
How to Watch Users

• Meetings and Discussions
• Coffee-Break Grousing
• Bug Reports
• Keep Track of Support Requests
• create FAQs
• VNC (Remote Desktop)
• Instrumentation

17
Build Time Distribution 10,000 per sample
18
Build Time Distribution 1000 per sample
19
Part 2

• ESL for Transaction Design

20
A HW Development Flow
(BAD)
Big Paper Spec
ISS Model
Verification
Coverage
Debug
Design
Checkers
Testbench
Tests
RTL
Synthesis
Clocks, Resets
C Model
Directed
Assertions
BFMs,
Assertions
Randoms
Scoreboard
TLMs
21
A HW Development Flow
(BAD)
Big Paper Spec
ISS Model
Verification
Coverage
Debug
Design
Checkers
Testbench
Tests
RTL
Synthesis
Clocks, Resets
C Model
Directed
Assertions
BFMs,
Assertions
Randoms
Scoreboard
TLMs
22
A HW Development Flow (Better)
Small Paper Spec
Coverage
Triage
ISS Model
ESL
Verification
C Model
Debug
Randoms
Testbench
Interfaces
Validation
Design
Assertions
Scoreboards
TLMs
RTL
BFMs
Assertions
Directed Tests
Clocks, Resets
Synthesis
23
Who Writes Assertions?

• Designers
• Bottom Up Assumptions
• Verification Engineers
• Top-down Intent

24
Who Writes Assertions?

• Designers
• Bottom Up Assumptions
• Verification Engineers
• Top-down Intent
• Architects
• The Specification
• Top Down Assumptions
• Bottom Up Intent

25
Where to Write Assertions

• The RTL
• Inline
• Bound
• The Testbench
• Scoreboard
• Environment
• E.g. Post Process Log file

26
Where to Write Assertions

• The RTL
• Inline
• Bound
• The Testbench
• Scoreboard
• Environment
• Post Process Log files
• The Specification
• C Models (?)

27
Where To Write Specification Assertions
Functionality Model
RTLDesign
Performance Model
28
Where To Write Specification Assertions
ISS Model
Transaction Model
Correlation Model
RTLDesign
Performance Model
Debug/Triage Model
29
Where To Write Specification Assertions
Transaction Model
Structural Model
RTLDesign
Performance Model
30
Interfaces Vs State

• Two approaches to comparing models
• Compare Architectural State
• Registers/flops within the design whose existence
  is required by the specification
• Compare externally visible behavior
• Compare interface traffic
• B. F. Skinner?

31
Birth of an IDL

• Interface Description Language
• Initially, a language just to define signals
• Interface a2b
• clock clk
• down U valid 1
• up U busy 1
• down U cmd 24
• down U data 32

32
Evolution of an IDL

• Quickly added flow-control protocol abstraction
• Interface a2b
• clock clk
• flow valid_busy
• down U cmd 24
• down U data 32
• From this we can generate
• Testbench components (BFMs producers, consumers)
• Protocol Assertions

33
Continued Evolution of an IDL

• Separation of packet structure from interface
• group SOP
• down U cmd 24
• group MOP
• down U data 32
• group EOP
• down U checksum 32
• Interface a2b
• clock clk
• flow valid_busy
• packet SOP, MOP, EOP

34
Transaction Assertions in YACC

• valid_interface_traffic
• valid_interface_traffic transaction
• transaction begin middle end
• begin SOP
• middle
• middle MOP
• end EOP

35
Cycle Level Assertions in SVA
clk cmd
SOP
EOP
MOP
MOP

• sequence valid_trans
• (cmdSOP)
• (1 cmd ! SOP cmd ! EOP) 0
• 1 cmd EOP
• endsequence
• a_well_formed_transaction assert _at_(posedge clk)
• cmd SOP -gt sequence (valid_trans)

36
Transaction Level Assertions in SVA
clk valid cmd
MOP
MOP
SOP
EOP

• sequence valid_transaction
• (cmdSOP)
• (1 cmd ! SOP cmd ! EOP) 0
• 1 cmd EOP
• endsequence

37
Transaction Level Assertions in SVA
clk valid busy cmd
MOP
MOP
SOP
EOP

• event sample
• always _at_(posedge clk)
• if (valid ! busy) -gt sample
• assert _at_(sample)
• cmd SOP -gt sequence (valid_transaction)

38
Transaction Level Assertions in SVA
clk valid busy cmd
MOP
MOP
SOP
EOP

• event sample
• always _at_(posedge clk)
• if (valid ! busy) -gt sample
• assert _at_(sample)
• cmd BEGIN -gt sequence (valid_transaction)

39
Grammer using IDL

• assert temporal representation
• data SOP gt past( EOP )
• data MOP gt past( SOP MOP )
• data EOP gt past( SOP MOP )
• assert bnf representation
• bnf SOP
• -gt MOP past( SOP.length-1 sample( SOP ) )
• -gt EOP

40
Multi-Unit Assemblies
A
E
B
C
D
F
G
a2b
b2c
c2d
d2e
e2f
f2g
A simple pipeline
41
Multi-Unit Assemblies
a2b
b2c
c2d
A
B
C
D
E
F
G
d2e
e2f
f2g
Simple rearrangement
42
Multi-Unit Assemblies
a2b
b2c
c2d
A
B
C
D
E
F
G
d2e
e2f
f2g
Identify units with similar behaviors
43
Multi-Unit Assemblies
cf2d
d2be
BE
CF
D
be2cf
a2be
A
cf2g
G
Extract common behavior into unified
components be2cf b2c e2f
44
Reusing Interface Definitions
A
E
B
C
D
F
G
BE
CF
D
A
G
How to maximize reuse between these two
architectures?
45
Packets as Traffic Streams

• group b2c
• down U data 32
• group e2f
• down U data 32
• Interface be2cf
• clock clk
• flow valid_credit
• packet b2c, e2f

46
Time Units of Temporal Expressions

• Group b2c
• down U value 4
• assert value ! past( value )
• Group e2f
• down U value 4
• assert ( value 0 ) gt ( past( value ) ! 0
  )
• Interface be2cf
• packet b2c, e2f
• assert b2c gt ( b2c.value ! past( b2c.value
  sample(b2c) ) )
• assert past( e2f e2f.value 0 ) gt (b2c
  b2c.value ! 0)

47
Example
Renderer
Memory
48
The Traffic

• group mem_write
• down U address 16
• down U data 1
• group sync
• down U shape 2
• enum SQUARE, CIRCLE, TRIANGE, BLANK
• down U radius 3

49
Accumulate Memory State

• group mem_write
• down U address 16
• down U data 1
• assign mem x 0 .. 15 y 0 .. 15
• past( data sample( address x,y ))

50
Add Predicates

• group_more mem_write
• assign is_circle r 0..7 (
• x -8 .. 7
• y -8 .. 7
• mem x8 y8 ( x2 y2 lt r2
  )
• )

51
Define the interface

• interface render2memory
• clock mem_clk
• flow valid_busy
• packet sync, mem_write
• assert correct sync shape
• sync sync.shape CIRCLE
• gt past( mem_write.is_circle sync.radius )

52
Example
Renderer
Memory
assert correct sync shape sync sync.shape
CIRCLE gt past( mem_write.is_circle
sync.radius )
53
Summary

• Efficient Verification
• Understand Variation
• Encourage Diversity
• avoid anarchy
• Encourage Standardization
• avoid stagnation
• Frontload The Schedule
• Provide Verification with Executable Spec (ESL)
• C Model, Correlation Model (pre validated)
• Interfaces, Transactions (definitions for code
  generators)
• Assertions, Testpoints (neutral language, pre
  validated)

54
Questions

• http//dave.whipp.name/dv

55
Summary

• Architects should write assertions
• Validated assertions are input to Verification
• Assertions must be directly reusable across
  models
• Manual recoding invites errors
• Explicitly model the structure that is common to
  architectural models and to design
• Tie assertions to these common points

56
Threading Models

• Thread Follows Resource
• Traditional SystemC approach
• Thread Follows Transaction
• Think Multithreaded ISS
• Cycle-based Model
• The Old way of doing things
• A non-threaded version of thread follows
  resource
• Different models are convenient at various times
• But SystemC forces you to commit early to one
  approach
• Therefore need extra code for transaction level
  debug

57
70

• 70 of schedule is Verification
• What is correct figure
• 0
• synthesize architectural models, constraints
• Correct by construction!
• 100
• synthesize verification models, constraints
• no design needed!

58
Transactions Vs Cycles
Data min_val (Addr a1, Addr a2) Data d1
mem_read(a1) Data d2 mem_read(a2)
if (d1 lt d2) return d1 else return d2
Pipelined Bus
t1 t2 t3 t4 t5 t6
Address
a1
Data
d1
59
Unit Level System Testing
Graphics Pipeline (Transaction Model)
Stage N-1
Stage N1
Stage N
DIFF
RTL Unit
60
Avoiding Stagnation

• New Challenges
• New Tools
• New Platforms
• New People
• New Ideas
• Refactoring
• Testability
• D.R.Y.