Taking On The Verification Challenge With Better Performance, Capacity, and Productivity

1
Taking On The Verification Challenge With Better
Performance, Capacity, and Productivity

2 Atwood LaneAndover, Ma 01810978 689
7286 www.avery-design.com
2
Verification BottlenecksLimits Effectiveness and
Efficiency of Teams

• System designs using multiple industry standard
  interface protocols
• Engineers must develop custom verification models
  for each protocol employed
• limited functionality feature creep
• limited HVL support
• limited reusability
• targets protocol subsets
• limited functional coverage analysis
• Engineers must develop comprehensive test suites
  for each protocol
• Engineers must develop robust debug and analysis
  methods (whos wrong, design or model)
• System verification lacks sufficient simulation
  performance and capacity solutions to meet
  throughput needs

Design iteration time is comprised of
lengthy intermediate steps
3
Verification BreakthroughsDelivers Dramatic
Process Improvements

• Verification IP for industry standard interface
  protocols mitigates SOCs design time by using
• Robust BFMs and multi-protocol transaction-level
  test development methods (TBV)
• Comprehensive assertion-based verification (ABV)
• Rigorous functional coverage protocol exercise
  (CBV)
• Reuse methodology (RM)
• Distributed Parallel simulation scales
  performance and capacity solutions meet
  throughput needs

SPLIT TIME 1 SPLIT TIME 2 SPLIT
TIME 3 SPLIT TIME 4
Do more in less time!
4
The Avery Vision

• 103 system verification challenge
• 10X Functional Test Development Effectiveness
• 10X Simulation Performance
• 10X Simulator Capacity
• Innovate new verification simulation-based
  technologies addressing functional verification
  and IP
• Build on customers investments in tools, language
  standards, and engineering know how

• Shrink Schedules
• Lower Dev Costs

5
Avery Solutions

• Verification IP Solutions
• Includes models, protocol checkers, compliance
  test suites and reference verification frameworks
• PCI-Xactor
• Verification IP for all PCI protocols
• Proven with over 5 IP companies and dozens of end
  users
• SATA-Xactor
• ATA/ATAPI-6, SATA I II, CE-ATA
• Advanced testbench automation
• TestWizard, transaction-based testbench
  automation
• Insight, Coverage-driven Automatic Functional
  Test Generation
• Targets deep corner cases using Fusion
  Simulation
• SimCluster
• Verilog/VHDL distributed parallel simulation
• Scale performance and capacity by 5-10X

6
Select Avery Customers
7
SimClusterScalable Performance and Capacity
8
SimClusterDistributed Parallel Verilog Simulation

• Eliminate performance and capacity limitations of
  stand-alone simulators (physical memory)
• General purpose architecture supports most
  popular computing solutions from 2 to 10s of
  computers
• Simulate 100M gates on 10K PC cluster

Sun ES450, ES4500, and Quad PCs
Linux clusters
9
SimCluster Offers New Options
Works in conjunction with emulation for faster
interactive debug
Accelerates full timing gate-level simulation
10
Typical Applications of SimCluster

• RTL level
• Chip and system level simulation
• Debug simulation or emulation errors faster
• Gate level
• Functional tests (parallel vectors)
• Manufacturing tests
• Scan test/ATPG (serial/parallel modes)
• Delay test
• Embedded memory BIST
• I/O parametric
• Hierarchy - Logical, Layout, Flat
• Accelerates Timing-based simulations - Pre/Post
  Layout Full timing (SDF)

11
SimCluster Supports Automated Process

• Simulation Analyzer
• Runs on existing single process simulation
• Estimates design activity levels based on toggle
  coverage samples for compute load analysis
• Finds clocks in RTL and gate-level designs and
  provides timestep information for synchronization
  analysis
• Performs interconnect analysis
• Generates design hierarchy report
• Auto-Partitioner
• User specifies of partitions
• RTL-level follows intrinsic design hierarchy
• Partitions at branches and sub-branches in
  hierarchy
• Gate-level flattens and/or re-groups logic
• Scan chains
• Clock domains / Register boundaries
• Generates partitioned Verilog design files
• Adds clock-based synchronization runtime controls
• Verilog-based only

12
SimCluster Performance
13
Verification IP SolutionsFor Advanced IO
Standards
14
Averys Verification IP Solutions

• Flexible and reusable verification environment
  supports
• All topologies
• All protocols
• Verilog source for models and tests provides
  open and flexible environment
• Supports all verification environments
  SystemVerilog, Vera, Specman, SystemC, VHDL,
  C/C
• Provides complete compliance verification of
  cores, IC components, and complete systems
• Supports comprehensive assertion-based
  verification of all IP protocols
• Innovative solutions
• Reusable, portable tests can be used for many
  configurations without modifications
• Innovative DUT integration enables
  initiator/completer control of core back-end
  interfaces for maximal compliance testing
• Leverages TestWizard for advanced testbench
  automation
• SuperMonitor tranactional recording performs
  end-2-end verification of endpoints, bridges, and
  switches
• Increases productivity by up to 5 times
• Improves design integrity
• Design IP partnerships for proven silicon
  solutions

15
Verification IP Architecture
Assertions
Assertions
Fabric Discovery
Enumeration Bus/Device/Function Map
TXN.2.2111 CHECKED
Enumeration Bus/Device/Function Map
TXN.2.2111 CHECKED
B0 D0 F0 ROOT_COMPLEX global index 1
prefetch
memory
TXN.3.235 CHECKED
B0 D0 F0 ROOT_COMPLEX global index 1
prefetch
memory
TXN.3.235 CHECKED
Verification
Verification
TXN.2.24 CHECKED
TXN.2.24 CHECKED
the other side B1 D0 F0 SWITCH_UPSTREAM global
index 2
the other side B1 D0 F0 SWITCH_UPSTREAM global
index 2
BIOS
BIOS
B2 D1 F0 SWITCH_DOWNSTREAM global index 3
prefetch
memory
TXN.3.214 NA
B2 D1 F0 SWITCH_DOWNSTREAM global index 3
prefetch
memory
TXN.3.214 NA
CFG
CFG
Frameworks
Frameworks
...
...
the other side B3 D0 F0 ENDPOINT global index 4
prefetch
memory
the other side B3 D0 F0 ENDPOINT global index 4
prefetch
memory
TXN
TXN
DLL.3.15 CHECKED
DLL.3.15 CHECKED
LINK
LINK
DLL.3.16 NA
DLL.3.16 NA
Symbol Tracker
Symbol Tracker
PHY
PHY
DLL.4.12 CHECKED
DLL.4.12 CHECKED
HOT PLUG
HOT PLUG
DLL.4.17 ASSERTED
DLL.4.17 ASSERTED
INTX
INTX
Compliance
Compliance
PHY.2.12 CHECKED
PHY.2.12 CHECKED
PME_ACK
TIME RX1 TX1 RX2 TX2
PME_ACK
TIME RX1 TX1 RX2 TX2
CFG.10.01 NA
CFG.10.01 NA
Tests Segments
Tests Segments
TRANS_ORDER
3800
---
00
---
00
TRANS_ORDER
3800
---
00
---
00
CFG.8.53 NA
CFG.8.53 NA
RANDOM
7800
---
COM
---
COM
RANDOM
7800
---
COM
---
COM
DLL.5.212 CHECKED
DLL.5.212 CHECKED
11800
---
PAD
---
PAD
11800
---
PAD
---
PAD
------------------------------
------------------------------
15800
---
PAD
---
PAD
15800
---
PAD
---
PAD
Checked items 109 (25.41)
Checked items 109 (25.41)
16001 COM
---
COM
---
16001 COM
---
COM
---
Asserted items 1 (0.23)
Asserted items 1 (0.23)
19800
---
0a
---
0a
19800
---
0a
---
0a
20001 PAD
---
PAD
---
20001 PAD
---
PAD
---
23800
---
02
---
02
23800
---
02
---
02
Compliance
Compliance
24001 PAD
---
PAD
---
24001 PAD
---
PAD
---
BFM
BFM
Scenarios
27800
---
00
---
00
27800
---
00
---
00
0
API
0
API
Scenarios
Test Coverage
Test Coverage
Packet Tracker
Packet Tracker
BIOS
BIOS
----------------------------------------

D CREDITS S
START FINISH I E
TIME TIME R COMMAND HDR DATA Q
----------------------------------------
1734021 1738021 D IFC2_CPL 0 0 ---
1754001 1758001 U IFC2_CPL 5 88 ---
1742021 1766021 D MSGD --- ---- 000
--------- --------- - ---------- --- ---- --
1770021 1786021 D CFG_RD_0 --- ---- 001
1930001 1934001 U ACK --- ---- 001
2026001 2030001 U FC_P 6 89 ---
2090001 2094001 U FC_NP 10 9 ---
2442001 2462001 U CMPLT_D --- ---- 000
link_test111_2port
link_test111_2port
CFG
CFG
Index Test Name Result
Index Test Name Result
DUT
DUT
TXN
TXN
-----
-------------
------
-----
-------------
------
LINK
LINK
1 test111rc FAILED
1 test111rc FAILED
PHY
PHY
2 test111ep PASSED
2 test111ep PASSED
INTERMIX
INTERMIX
RANDOM
RANDOM
link_test412_2port
link_test412_2port
Index Test Name Result
Index Test Name Result
-----
-------------
------
-----
-------------
------
BFM
BFM
1 test412rc PASSED
1 test412rc PASSED
API
API
N
N
2 test412ep PASSED
2 test412ep PASSED
link_test5210_2port
link_test5210_2port
Index Test Name Result
Index Test Name Result
-----
-------------
------
-----
-------------
------
1 test5210rc FAILED
1 test5210rc FAILED
2 test5210ep FAILED
2 test5210ep FAILED
16
IP PartnersEnsures the Highest Quality Models
IP Partner Relationships Under works
17
PCI Protocol Suite
18
PCI-Xactor Delivers a Compliance and
Interoperability Solution and Process
Complete PCI Topology

• Flexible and reusable environment
• Supports Endpoint, Root Complex, Switch, and
  Bridge verification frameworks
• Robust test development API
• Supports Verilog, SystemVerilog, Vera, Specman,
  SystemC, VHDL, C/C
• Comprehensive protocol checklist verification
• Transaction profiling
• Performance trace analysis
• Comprehensive compliance test suites for each
  framework supported
• All protocols
• PCI 2.2
• PCI-X 1.0/2.0 (DDR, QDR)
• PCI Express 1.0a 1.1 (x1 to x32 support)

Increase verification productivity by 5
times! Improve design integrity!
19
And Advanced Switching

• PCI Express BFMs
• N-Port Root Complex (A)
• Endpoint (B)
• N-Port Switch (C)
• Bridge (PCI Express to PCI-X) (D)
• MAC/PHY
• AS BFMs (1.1)
• Core BFM
• TL
• PI-0 to PI-5
• PI-8 (Base)
• Other PIs (Adaptations)
• SLS, SQ, SDT
• DLL
• MAC/PHY
• PIPE, serial, 8b/10b symbol
• Complete Native AS Devices (BFMs)
• Host and IO Switch (1), (2)
• FM nodes (3)

20
PCIe and AS Base Infrastructure
21
ATA Protocol Suite
22
Parallel ATA and Serial ATA Verification
ATA/ATAPI-7 CE-ATA SATA I/II Host Controller
ATA/ATAPI-7 CE-ATA SATA I/II Device Controller
HDD CDROM DVD
Serial ATA 1.5/3.0 Gbps
Driver/Firmware

• Supports Host and Device models
• Supports PIO and PACKET interfaces
• Advanced SATA II features - NCQ
• Over 200 protocol checkers
• Compliance test suites verify all ATA commands
  and protocol-specific layers
• DUT integration for different driver/firmware
  interfaces

23
PCI Express to Serial ATA Verification
Controller
South Bridge
HDD
SATA I/II Host Controller
PCIE RC
PCIE EP
PCI Express xN
Serial ATA 1.5/3.0 Gbps

• PCI Express root complex and endpoint BFMs
• SATA host and device BFMs including complete
  Serial ATA HDD with storage
• Develop tests that models PCI Express software
  driver
• SATA over PCI Express test driver
• Serial ATA HDD BFM completes ATA commands
• Checklist coverage monitor measures feature set
  verified

24
Advanced Testbench Automation
25
The Avery SolutionsRequired Elements for
Effective Functional Verification

• TestWizard
• Transaction-based testbench automation
• Up to 3 X better efficiency using native Verilog,
  VHDL, and C/C
• Insight
• Coverage-guided automatic functional test
  generator
• Overcomes limitations of random testing by
  finding deep corner cases that satisfy
  user-directed coverage requirements

26
TestWizard Is
27
TestWizard Verification is

• Transaction modeling using complex data
  structures
• Test generators generate random transaction
  sequences based on test parameters
• Transactors stream transactions in/out of DUT
• Result checking is dynamic self-checking using
  transaction database queries
• Assertion-based verification checks protocol
  compliance on the fly
• Closed-loop functional assertions coverage
  analysis adapts traffic shape for more effective
  efficient testing

28
State of Art in Functional Testing

• Simulation-based
• Directed testing
• Generate testcases for each test assertion and
  related test parameters
• Interfaces exercised independently
• Block, module, and system-level
• Pseudo-random testing
• Generate testcases for one or more test
  assertions concurrently
• Test parameters selected randomly from enumerated
  list
• Multiple interfaces exercised concurrently
• Block, module, system-level
• Auto-directed testing
• Automatic functional test generation
• Deterministic process using SAT solver engine(s)
• Guided by functional coverage targets
• Formal methods
• Formal model checkers
• Prove design complies with design properties
• Or not, generate counter examples
• Hybrid, Semi-formal checkers

Avery Solutions
29
Functional Testing Methods
Insight
TestWizard
30
Insight - Next Generation of Functional
Verification

• When directed and random testing runs out of gas
• Random test runaway
• Overlooked/missed tests and test parameters

Testcases become exponentially more complex to
develop and debug
Effort
Bugs Found
31
Insight Supports Property- and Coverage-Driven
Verification

• Builds on current testbench methodolgies
• Supports complete Verilog HDL
• Supports TestWizard features
• Coverage guides test generation directly
• random probabilities no longer dominate
• Coverage points analyze various user-specified
  parameters
• All assertions triggered
• All assertions pass
• All functional coverage monitor points exercised
• All reactive result checkers have been triggered

32
Random Case Runaway
2
20
8
20
5
32
64
Coverage Point
64
Input combinations 2(2 8 32 64)
Even highly constrained random test run lengths
are prohibitivewhen multiple, wide word
variables are prevalent Random scenarios have
no ties to satisfiability on coverage points
33
Insight Highlights

• Increases design quality by finding corner case
  testcases that verify design compliance to formal
  properties and coverage requirements
• Reduces verification cost by finding bugs early
  in the verification cycle
• Ensures reuse of properties between block and
  system-level verification
• Augments TestWizard-based testbench environment
• Testcases can be added to any regression
  testsuite thus enhancing property compliance
  verification for all regression runs and design
  iterations

34
Fusion Simulation Technology

• Unique combination of logic and symbolic
  simulation
• Fewer limitations than Binary Decision Diagram
  (BDD-based approaches for formulating
  satisfiability (SAT) targets.
• avoids many memory explosion issues
• Logic simulation seeds the state space thus
  dramatically reducing the complexity of each SAT
  target.

35
Insight Example
Symbolic Simulation
Random variable
Assertion/Property
Coverage Monitor
SAT Target
36
Insight Example

• Insight solves for missed coverage points
• Egress result checker (one per port)
• Ingress packet traffic monitor (one per port)
• For egress monitor (port 0), generates 3 directed
  tests using existing testbench and assertions
  (design properties)
• Outputs directed test sequencer for regression
  testing

top.mon0.verify_packet, top.mon0.verify_packet_PH
("verify_ip_icmp_timeout_msg_latency"
32'sh00000001) 1 10.0 ("verify_ip6_icmp_timeo
ut_msg_latency" 32'sh00000002) 1
10.0 ("verify_ip6_flow_latency" 32'sh00000003)
4 40.0 ("verify_mpls_stack_forwarding"
32'sh00000004) 3 40.0 ("verify_ldp_map_reque
st" 32'sh00000005) 1 10.0 (default) 0
0.0
37
Insight for PCI Express
Assertions
Assertions

• Insight targets coverage points that are not
  tested
• Generates compliance test by formally solving
  temporal representation
• Builds directed scenario from one or more
  parameterized test segments building blocks
• Forces other IOs that are constrained by
  properties assumptions on IOs
• Segments and properties constrain test scenarios
  within valid protocol set

TXN.2.2111 CHECKED
TXN.2.2111 CHECKED
TXN.3.235 CHECKED
TXN.3.235 CHECKED
Verification
Verification
TXN.2.24 CHECKED
TXN.2.24 CHECKED
BIOS
BIOS
TXN.3.214 NA
TXN.3.214 NA
CFG
CFG
Frameworks
Frameworks
...
...
TXN
TXN
DLL.3.15 CHECKED
DLL.3.15 CHECKED
LINK
LINK
DLL.3.16 NA
DLL.3.16 NA
PHY
PHY
DLL.4.12 CHECKED
DLL.4.12 CHECKED
HOT PLUG
HOT PLUG
DLL.4.17 ASSERTED
DLL.4.17 ASSERTED
INTX
INTX
Compliance
Compliance
PHY.2.12 CHECKED
PHY.2.12 CHECKED
PME_ACK
PME_ACK
CFG.10.01 NA
CFG.10.01 NA
Tests Segments
Tests Segments
TRANS_ORDER
TRANS_ORDER
CFG.8.53 NA
CFG.8.53 NA
RANDOM
RANDOM
DLL.5.212 CHECKED
DLL.5.212 CHECKED
------------------------------
------------------------------
Checked items 109 (25.41)
Checked items 109 (25.41)
Asserted items 1 (0.23)
Asserted items 1 (0.23)
Compliance
Compliance
BFM
BFM
Scenarios
0
API
0
API
Scenarios
Test Coverage
Test Coverage
BIOS
BIOS
link_test111_2port
link_test111_2port
CFG
CFG
Index Test Name Result
Index Test Name Result
DUT
DUT
TXN
TXN
-----
-------------
------
-----
-------------
------
LINK
LINK
1 test111rc FAILED
1 test111rc FAILED
PHY
PHY
2 test111ep PASSED
2 test111ep PASSED
INTERMIX
INTERMIX
RANDOM
RANDOM
link_test412_2port
link_test412_2port
Index Test Name Result
Index Test Name Result
-----
-------------
------
-----
-------------
------
BFM
BFM
1 test412rc PASSED
1 test412rc PASSED
API
API
N
N
2 test412ep PASSED
2 test412ep PASSED
link_test5210_2port
link_test5210_2port
Index Test Name Result
Index Test Name Result
-----
-------------
------
-----
-------------
------
1 test5210rc FAILED
1 test5210rc FAILED
2 test5210ep FAILED
2 test5210ep FAILED
38
Assertion Coverage Constraints

• Checklist assertions comprise formal properties
• Adding coverage monitor on checklist assertions
  provides Insight coverage goals
• Generate new tests for increased trigger coverage
• Find out whether design potentially violates
  protocol

("TXN.3.235" 32'sh000000a9) 1298
0.00 ("TXN.3.236" 32'sh0000017c) 0
0.00 ("TXN.3.24" 32'sh000001be) 1
0.00 ("TXN.3.25" 32'sh00000149) 1
0.00 ("TXN.3.26" 32'sh0000014a) 0
0.00 ("TXN.3.27" 32'sh00000041) 22154
0.00 ("TXN.3.31" 32'sh00000040) 22154
0.00 ("TXN.3.32" 32'sh00000016) 17865
0.00 ("TXN.3.33" 32'sh00000233) 1298
0.00 ("TXN.3.35" 32'sh00000019) 0
0.00 ("TXN.3.36" 32'sh0000006e) 1298
0.00 ("TXN.3.37" 32'sh00000005) 12286
0.00 ("TXN.4.010" 32'sh000000c2) 0
0.00 ("TXN.4.011" 32'sh000000c3) 0
0.00 ("TXN.4.012" 32'sh000000c4) 0 0.00
Assertion checkerstrack and profileeach of the
PCI Checklist Items
39
Conclusion

• TestWizard is a robust transaction-level
  verification solution
• Models complex transactions
• Generates directed and random transaction-level
  tests
• Supports powerful transaction database for result
  validation of complex transaction processing
• Assertions provide powerful and re-usable
  protocol verification
• Validates feature set coverage of system
  operation and port traffic
• Insight using SAT solver is 1st coverage-driven,
  auto-directed functional test generator
• Delivers 3X better productivity with up to 10X
  less verification code
• Improves design integrity with advanced and
  measurable methods
• Builds on existing simulators, HDLs, and methods
  you already use today!

40
Now its your turn to decide how fast your really
are?
SPLIT TIME
WRITE BFMs
SIMULATE DEBUG
WRITE SCENARIOS
Get Avery and elevate your performance!