2018 Wilson Research Group

Functional Verification Study

Harry D. Foster
Chief Scientist Verification
IC Verification Solutions (ICVS)

Aug 2018
A Tale of Two Technologies
ASIC & FPGA Functional Verification Trends

It was the best of times, it was the worst of times...

A Tale of Two Cities — Charles Dickens

© Mentor Graphics Corporation

 Worldwide IC Market Growth
50%

40%
35% 33%
30% 28%
25%

20%
16%
IC Market Change

19% 16%
10% 8% 9%
4% 7% 4%
5%
2% 1% 3%
0%

-2%
-10% -9% -6% -4%
-10%

-20%

-30%
-33%
-40%
98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18F 19F
Year
Source: IC Insight, McClean Report 2018 (June Update), A Complete Analysis and Forecast of the Integrated Circuit Industry
© Mentor Graphics Corporation
 Overall World Wide Semiconductor Market
$600

$500

$509.10
Semiconductor Market ($B)

$400
$444.70
$300
$364.00
$200

$100

$0
2016 2017 2018F
Source: IC Insight, McClean Report 2018 (June Update), A Complete Analysis and Forecast of the Integrated Circuit Industry
© Mentor Graphics Corporation
 Global FPGA Market
$10.00

$9.00

$8.00
$8.80
FPGA Market ($B)

$7.00

$6.00

$5.00

$4.00
$4.70
$3.00

$2.00

$1.00

$0.00
2017 2027F

Source: Statistics MRC 2018

© Mentor Graphics Corporation

5 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA Applications

Data Processing Automotive Mil/Aero

Consumer Telecommunications Industrial

© Mentor Graphics Corporation

6 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Number of Required Spins Before Production

50% 2012
2014
2016
40%
2018
Design Projects

30%

20%

10%

0%
1 (FIRST SILICON 2 3 4 5 6 7 SPINS or MORE
SUCCESS)
Number of Required ASIC Spins Before Production

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

7 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Non-trivial Bug Escapes into Production

35% 84% of FPGA design projects 2016

have non-trivial bugs escape 2018

30% into production
25%
Design Projects

20%

15%

10%

5%

0%
0 1 2 3 4 5 6 or More
Non-trivial FPGA Bug Escapes into Production

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

8 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Study Background
 Worldwide study
— North and South America, Europe, Mideast, Africa, China, India, Pac Rim, Australia, Japan

 Sample frame consisted of 1205 participants

— 31% smaller than our 2016 study (1738)
— 36% smaller than our 2014 study (1886)
— 89% larger than our 2012 study
— 5.4x larger than the 2004 Ron Collett International study

 Confidence interval 95%

— ±4% Margin of Error
© Mentor Graphics Corporation
9 H Foster, WRG Functional Verification Study, July 2016
9 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section A.

ABOUT PARTICIPANT’S COMPANY

© Mentor Graphics Corporation

 2018 Study Participation by Market Segment
PC/Workstation/Server/Mainframe
Peripherals (printers, monitors, storage devices, UPSs, etc.)
Office Equipment (copiers, etc.)
Wireless: cellular handsets, cordless phones, wireless LANs, radios, pagers,…
Wireless: base stations, satellite, TV transmission, radar
Networking: backbone routers, switches, other WAN products
FPGA
Networking: Optical products
ASIC/IC
Networking: hubs, NIC, LAN routers, bridges, other LAN products
Networking: fax, modems, video-conferencing, set-top boxes, internet phones
Consumer Audio, Video (TVs, DVDs, cameras, etc.), Games
Automotive
Aerospace/Military
Medical equipment
Household Appliances
Industrial Controls
Other

0% 5% 10% 15% 20%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

11 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2016 vs 2018 ASIC Study Participation by Market Segment
PC/Workstation/Server/Mainframe
Peripherals (printers, monitors, storage devices, UPSs, etc.)
Office Equipment (copiers, etc.)
Wireless: cellular handsets, cordless phones, wireless LANs, radios, pagers,…
Wireless: base stations, satellite, TV transmission, radar
Networking: backbone routers, switches, other WAN products
2016
Networking: Optical products
2018
Networking: hubs, NIC, LAN routers, bridges, other LAN products
Networking: fax, modems, video-conferencing, set-top boxes, internet phones
Consumer Audio, Video (TVs, DVDs, cameras, etc.), Games
Automotive
Aerospace/Military
Medical equipment
Household Appliances
Industrial Controls
Other

0% 5% 10% 15% 20%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

12 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2016 vs 2018 FPGA Study Participation by Market Segment
PC/Workstation/Server/Mainframe
Peripherals (printers, monitors, storage devices, UPSs, etc.)
Office Equipment (copiers, etc.)
Wireless: cellular handsets, cordless phones, wireless LANs, radios, pagers,…
Wireless: base stations, satellite, TV transmission, radar
Networking: backbone routers, switches, other WAN products
2016
Networking: Optical products
2018
Networking: hubs, NIC, LAN routers, bridges, other LAN products
Networking: fax, modems, video-conferencing, set-top boxes, internet phones
Consumer Audio, Video (TVs, DVDs, cameras, etc.), Games
Automotive
Aerospace/Military
Medical equipment
Household Appliances
Industrial Controls
Other

0% 5% 10% 15% 20% 25%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

13 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2018 Study Participation by Job Title
Hardware designer

Verification engineer

Emulation engineer

FPGA Prototyping Engineer

Post-Silicon Validation Engineer

Product engineer

Test engineer

Safety Engineer

System architect FPGA

ASIC/IC
Team leader

Software engineer

Manager

CAD support engineer

Other

0% 5% 10% 15% 20% 25% 30% 35%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

14 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2016 vs. 2018 ASIC Study Participation by Job Title
Hardware designer

Verification engineer

Emulation engineer

FPGA Prototyping Engineer

Post-Silicon Validation Engineer

Product engineer

Test engineer

Safety Engineer

System architect

Team leader

Software engineer
2016
Manager
2018
CAD support engineer

Other

0% 5% 10% 15% 20% 25% 30% 35% 40%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

15 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2016 vs. 2018 FPGA Study Participation by Job Title
Hardware designer

Verification engineer

Emulation engineer

FPGA Prototyping Engineer

Post-Silicon Validation Engineer

Product engineer

Test engineer

Safety Engineer

System architect

Team leader

Software engineer
2016
Manager
2018
CAD support engineer

Other

0% 5% 10% 15% 20% 25% 30% 35% 40%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

16 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2018 Functional Verification Study Participation by Region
North America

Sourth America

Europe

Mideast / Africa

China

India

Japan

PacRim / Austraila

0% 5% 10% 15% 20% 25% 30% 35% 40%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

17 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2018 Ratio of ASIC vs FPGA Projects by Region
North America

Sourth America

Europe

Mideast / Africa
ASIC
China FPGA

India

Japan

PacRim / Austraila

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

18 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section B.

PARTICIPANT’S CURRENT DESIGN

© Mentor Graphics Corporation

 Study Participation by Targeted Implementation
FPGA

High-Performance SoC FPGA

Gate array

Embedded array

FPGA 2016
Standard cell
FPGA 2018
ASIC 2016
Structured ASIC
ASIC 2018

Structured custom

Full custom

Other

0% 5% 10% 15% 20% 25% 30%

Design Projects

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

20 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 High-Performance SoC FPGA Adoption Percentage by Type

Xilinx Zynq

Altera Arria

Altera Cyclone
41% of FPGA projects are
Altera Stratix implementing with
programmable SoC FPGAs
Microsemi's
SmartFusion

Other Programmable
SoC FPGA

0% 10% 20% 30% 40% 50% 60%

Current High-Performance SoC FPGA Type Being Implemented by Project

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

21 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC Study Participation by Gate Count (Design Size)
2014: 18% < 500K Gates 2014
20% 2016: 21% < 500K Gates 2016
2018: 29% < 500K Gates 2018

15%
Study Participants

10%

5%

0%

Gates of Logic and Datapath Excluding Memories

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

22 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC Study Participation by Gate Count (Design Size)
2014
20% 2016
2018

15%
Study Participants

10%

5%

0%

Gates of Logic and Datapath Excluding Memories

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

23 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Study Participants by Design Size
40%
36%
< 1M
35%
33% 1M-40M
31%
30% >40M
Design Projects

25%

20%

15%

10%

5%

0%
Design Size

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

24 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Study Participation by Geometry Feature Size
25%
2016
2018
20%
Study Participants

15%

10%

5%

0%
0.15µ or 0.13µ 0.09µ (90nm) 0.065µ 0.045µ 0.032µ 0.028µ 0.022µ 0.014µ 0.010µ 0.007µ (7nm)
larger (65nm) (45nm) (32nm) (28nm) (22nm) (14nm) (10nm) or smaller

Geometry Feature Size

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

25 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Number of Embedded Microprocessors
40%
66% of designs contain embedded processors 2016
35% 2018
48% of designs contain 2 or more processors
30%
13% of designs contain 8 or more processors
ASIC Design Projects

25%

20%

15%

10%

5%

0%
0 1 2 3 4 5 6 7 8 or more
Number of Embedded Microprocessors

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

26 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Mean Number of Embedded Microprocessors by Design Size
5
< 1M
4 1M-40M
>40M
Mean Number of Embedded Processors

2
2

1 1

ASIC: Mean Number of Embedded Microprocessors by Design Size

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

27 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Number of Embedded Microprocessors
40%

64% of designs contain embedded processors 2016

35%
2018
43% of designs contain 2 or more processors
30%
14% of designs contain 4 or more processors
FPGA Design Projects

25%

20%

15%

10%

5%

0%
0 1 2 3 4 5 6 7 8 or more
Number of Embedded Microprocessors

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

28 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC vs. FPGA: Number of Embedded Microprocessors
40% • 66% of ASIC designs contain 1 or more embedded processors
• 48% of ASIC designs contain 2 or more embedded processors
35%

• 64% of FPGA designs contain 1 or embedded processors

30%
• 43% of FPGA designs contain 2 or more embedded processors
Design Projects

25%

20%
FPGA
ASIC
15%

10%

5%

0%
0 1 2 3 4 5 6 7 8 or more
Number of Embedded Microprocessors

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

29 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: ON-CHIP "internal" Bus Adoption
None used in this design
Proprietary
Arm AMBA APB
Arm AMBA AHB
Arm AMBA AXI
Arm AMBA ACE
Arm AMBA CHI
IBM CoreConnect
Sonics
TileLink (RISC-V)
Arteris FlexNoC
Arteris (Ncore)
Other
0% 5% 10% 15% 20% 25% 30% 35%

ASIC Design Projects

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

30 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: ON-CHIP "internal" Bus Adoption
None used in this design
Proprietary
Arm AMBA APB
Arm AMBA AHB
Arm AMBA AXI
Arm AMBA ACE
Arm AMBA CHI
IBM CoreConnect
Sonics
TileLink (RISC-V)
Arteris FlexNoC
Arteris (Ncore)
Other
0% 5% 10% 15% 20% 25% 30% 35%

FPGA Design Projects

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

31 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Number of Embedded DSP Cores
70%
2016
2018
60%

50%
ASIC Design Projects

40%

30%

20%

10%

0%
0 1 2 3 4 5 or more
Number of Embedded DSP Cores

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

32 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Mean Number of Number of Embedded DSP Cores by Design Size

1.5 < 1M
1.4 1M-40M
>40M
Mean Number of Embedded DSP Cores

1.0

0.7

0.5 0.4

0.0

ASIC: Mean Number of Embedded DSP Cores by Design Size

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

33 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Number of Embedded DSP Cores
70%
2016
2018
60%

50%
FPGA Design Projects

40%

30%

20%

10%

0%
0 1 2 3 4 5 or more
Number of Embedded DSP Cores

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

34 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Projects Working on Safety Critical Design
70%
2016
2018
59%
60%
54%

50%
46%
ASIC Design Projects

41%
40%

30%

20%

10%

0%
Safety Critical Design Not Safety Critical

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

35 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Adoption for Specific Functional Safety Standards
DO-254 - Avionics

ISO26262 - Automotive

IEC61508 - Industrial

IEC61511 - Process Industry

IEC61513 - Nuclear

IEC60601 - Medical

EN50129 - Railway

ISO25119 - Agriculture &…

MIL-STD-882 - Military

Other

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Safety Critical ASIC Design Projects

** Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

36 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Projects Working on Safety Critical Design
70%
2016
2018
60%
56%
53%
50% 47%
44%
FPGA Design Projects

40%

30%

20%

10%

0%
Safety Critical Design Not Safety Critical

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

37 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Adoption for Specific Functional Safety Standards
DO-254 - Avionics

ISO26262 - Automotive

IEC61508 - Industrial

IEC61511 - Process Industry

IEC61513 - Nuclear

IEC60601 - Medical

EN50129 - Railway

ISO25119 - Agriculture &…

MIL-STD-882 - Military

Other

0% 5% 10% 15% 20% 25% 30% 35%

Safety Critical FPGA Design Projects

** Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

38 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Adoption for Specific Functional Safety Standards
DO-254 - Avionics

ISO26262 - Automotive

IEC61508 - Industrial

IEC61511 - Process Industry

IEC61513 - Nuclear

IEC60601 - Medical FPGA

ASIC
EN50129 - Railway

ISO25119 - Agriculture &…

MIL-STD-882 - Military

Other

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Safety Critical Design Projects

** Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

39 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Design Projects Implementing Security Features
70%
2016
2018
60%
60% 58%

50%
ASIC Design Projects

42%
40%
40%

30%

20%

10%

0%
Security Features No Security Features
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

40 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Design Projects Implementing Security Features
70%
2016
2018
60%
54%
52%
50% 48%
46%
FPGA Design Projects

40%

30%

20%

10%

0%
Security Features No Security Features
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

41 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section C.

PROJECT RESOURCES

© Mentor Graphics Corporation

 ASIC: Mean Peak Number of Engineers on a Project
12.0 11.6
Design Engineers 11.0 11.0
Verification Engineers 10.5 10.5
10.1
10.0

8.5 8.4
8.1
8.0 7.8 7.6
Design Projects

6.0
4.8

4.0

2.0

0.0
2007 2010 2012 2014 2016 2018
Mean Peak Number of Engineers on ASIC Projects
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

43 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Mean Peak Number of Engineers By Design Size
18
< 1M
16 1M-40M 15.25
Mean Peak Number of Engineers on Project

14.83
>40M
14

12
10.36
10 9.07

5.78
6 5.31

0
Design Engineers Verification Engineers
2018 Mean Peak Number of ASIC Engineers by Design Size
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

44 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Mean Peak Number of Engineers on a Project

6.0 • Design Engineers growing at a 3.79% CAGR

• Verification Engineers growing at a 9.98% CAGR
Mean Peak Number of Engineers on Project

5.0
5.0
4.6
4.3 4.3
4.0
4.0
3.5 3.6

3.0 2.6

2.0

1.0

0.0
2012 2014 2016 2018
Design Engineers
Mean Peak Number of Engineers on FPGA Projects
Verification Engineers
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

45 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Mean % Time Design Engineer is Doing Design vs Verification

100%

80%
47% 46%
53%
Design Projects

60%
Doing Design
Doing Verification
40%

53% 54%
20%
47%

0%
2014 2016 2018
Mean Percentage Time ASIC Design Engineer is Doing Design vs Verification
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

46 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Mean % Time Design Engineer is Doing Design vs Verification

100%

80%
44%
53% 51%
Design Projects

60%
Doing Design
Doing Verification
40%

56%
47% 49%
20%

0%
2014 2016 2018
Mean Percentage of Time FPGA Design Engineer is Doing Design vs Verification

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

47 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Where Verification Engineers Spend Their Time

44%

Test Planning
3%
Testbench Development

13% Creating Test and Running Simulation

Debug
21%
Other
19%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

48 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Where Verification Engineers Spend Their Time

42%

Test Planning
4%
Testbench Development

14% Creating Test and Running Simulation

Debug
20%
Other
19%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

49 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Percentage of Project Time Spent in Verification
2012
2012: Average 55%
2014: Average 57% 2014
25%
2016: Average 54% 2016
2018: Average 53% 2018
20%
Design Projects

15%

10%

5%

0%
1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% >80%
Percentage of ASIC Project Time Spent in Verification
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

50 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Percentage of Project Time Spent in Verification
2012: Average 44% 2012
2014: Average 46%
25% 2014
2016: Average 49%
2018: Average 50%
2016
20%
2018
Design Projects

15%

10%

5%

0%
1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% >80%
Percentage of FPGA Project Time Spent in Verification
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

51 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: How Long is Your Current Project Schedule
Less than a month Median: 7-9 Months
1 - 3 months

4 - 6 months

7 - 9 months

10 - 12 months

13 - 18 months

19 - 24 months

Greater than 24 months

0% 5% 10% 15% 20% 25% 30%

2018 ASIC Design Projects

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

52 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: How Long is Your Current Project Schedule
Less than a month Median: 7-9 Months
1 - 3 months

4 - 6 months

7 - 9 months

10 - 12 months

13 - 18 months

19 - 24 months

Greater than 24 months

0% 5% 10% 15% 20% 25% 30%

2018 FPGA Design Projects

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

53 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC vs FPGA: How Long is Your Current Project Schedule
Less than a month ASIC
FPGA

1 - 3 months

4 - 6 months

7 - 9 months

10 - 12 months

13 - 18 months

19 - 24 months

Greater than 24 months

0% 5% 10% 15% 20% 25% 30%

Design Projects
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

54 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Number of FPGAs Completed by Project Team per Year
40%

2014
35%
2016
2018
30%

25%
Design Projects

20%

15%

10%

5%

0%
1 2 3 4 5 6 or more
Number of FPGAs Completed by Project Team per Year
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

55 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section D.

SIMULATION CHARACTERISTICS

© Mentor Graphics Corporation

 ASIC: Typical Regression Time

2014
25% 2016
2018

20%
Design Projects

15%

10%

5%

0%
<5 hrs 5-<9 hrs 9-<17 hrs 17-<24 hrs 1-<1.5 days 1.5-<2 days 2-<3 days 3-<4 days 4-7 days > 7 days

Typical ASIC Regression Time

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

57 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Typical Regression Time
50%
2014
45%
2016
40% 2018

35%

30%
Design Projects

25%

20%

15%

10%

5%

0%
<9 hrs 9-48 hrs > 48hrs
Typical ASIC Regression Time
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

58 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Typical Regression Time
45%
2014
40%
2016
35% 2018

30%
Design Projects

25%

20%

15%

10%

5%

0%
<5 hrs 5-<9 hrs 9-<17 hrs 17-<24 hrs 1-<1.5 days 1.5-<2 days 2-<3 days 3-<4 days 4-7 days > 7 days

Typical FPGA Regression Time

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

59 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Typical Regression Time
60%
2014
2016
50%
2018

40%
Design Projects

30%

20%

10%

0%
<9 hrs 9-48 hrs > 48hrs
Typical FPGA Regression Time
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

60 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Longest Regression Time
20%

2016
2018

15%
Design Projects

10%

5%

0%
<5 hrs 5-<9 hrs 9-<17 hrs 17-<24 hrs 1-<1.5 days 1.5-<2 days 2-<3 days 3-<4 days 4-7 days > 7 days

Longest ASIC Regression Time

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

61 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Longest Regression Time
2016
40%
2018

30%
Design Projects

20%

10%

0%
<9 hrs 9-48 hrs > 48hrs
Longest ASIC Regression Time
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

62 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Longest Regression Time
35%

2016
30%
2018

25%
Design Projects

20%

15%

10%

5%

0%
<5 hrs 5-<9 hrs 9-<17 hrs 17-<24 hrs 1-<1.5 days 1.5-<2 days 2-<3 days 3-<4 days 4-7 days > 7 days

Longest FPGA Regression Time

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

63 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Longest Regression Time
60%

2016

50% 2018

40%
Design Projects

30%

20%

10%

0%
<9 hrs 9-48 hrs > 48hrs
Longest FPGA Regression Time
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

64 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section E.

LANGUAGES AND LIBRARIES

© Mentor Graphics Corporation

 ASIC: Design Language Adoption Next Twelve Months
90%
2012
80% 2014
2016
70% 2018
Next Year
60%
Design Projects

50%

40%

30%

20%

10%

0%
VHDL Verilog SystemC SystemVerilog C/C++ OTHER Design
ASIC Design Language Adoption ** Multiple answers possible

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

66 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Design Language Adoption Next Twelve Months
80%
2012
70%
2014
2016
60% 2018
Next Year
50%
Design Projects

40%

30%

20%

10%

0%
VHDL Verilog SystemC SystemVerilog C/C++ OTHER Design
FPGA Design Language Adoption
* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

67 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Verification Language Adoption Next Twelve Months
90%
2012
80% 2014
2016
70% 2018
Next Year
60%
Design Projects

50%

40%

30%

20%

10%

0%
VHDL Verilog Vera System C SystemVerilog Specman e C/C++ OTHER Testbench
ASIC Verification Language Adoption ** Multiple answers possible

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

68 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: SystemVerilog Adoption wrt Design Size
90%

79%
80% < 1M
73% 1M-40M
70% >40M

60%
Design Projects

50%
45%

40%

30%

20%

10%

0%
SystemVerilog

2018 SystemVerilog Adoption wrt Design Size

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

69 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Verification Language Adoption
80%
2012
70%
2014
2016
60% 2018
Next Year
50%
Design Projects

40%

30%

20%

10%

0%
VHDL Verilog SystemC SystemVerilog C/C++ OTHER Design
FPGA Design Language Adoption
* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

70 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Assertion Language Adoption Next Twelve Months
90%
2014
80%
2016
70%
2018
60%
Next Year
Design Projects

50%

40%

30%

20%

10%

0%
Accellera Open Verification Library SystemVerilog Assertions (SVA) PSL Other
(OVL)
ASIC Assertion Language Adoption * Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

71 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Assertion Language Adoption Next Twelve Months
60%
2014

50% 2016

2016
40%
Next Year

30%

20%

10%

0%
Accellera Open Verification Library SystemVerilog Assertions (SVA) PSL Other
(OVL)
FPGA Assertion Language Adoption * Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

72 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section F.

CLOCKING AND POWER

© Mentor Graphics Corporation

 ASIC: Number of Asynchronous Clock Domain
35%
2012
2014
30% 2016
2018
25%
Design Projects

20%

15%

10%

5%

0%
1 2 3--4 5--10 11--20 21--30 31--50 >50
ASIC: Number of Asynchronous Clock Domain
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

74 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Number of Asynchronous Clock Domain
50%
2012
45% 2014
2016
40% 2018

35%
Design Projects

30%

25%

20%

15%

10%

5%

0%
1 2 3--4 5--10 11--20 21--30 31--50 >50
FPGA: Number of Asynchronous Clock Domain
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

75 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Actively Manage Power

100%

28% 28% 29%

80% 41% 38% 38%

Actively Manage Power

Design Projects

60%
No Power Management

40%
72% 72% 71%
59% 62% 62%
20%

0%
2007 2010 2012 2014 2016 2018
Designs that Actively Manage Power
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

76 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Power Management Features Verified
Hypervisor/OS control of power
management

Application-level power
management

Operation in each system power

state

Interactions between power

domains 2012
Hardware power control 2018
sequence generation

Transitions between system

power states

Power domain state

reset/restoration

Power domain power

down/power up

0% 10% 20% 30% 40% 50% 60% 70% 80%

Design Projects
* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

77 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Notation to Specify Power Intent
40%
2012
35% 2014
2016
30%
2018
25%
Design Projects

20%

15%

10%

5%

0%
CPF 1.0 CPF 1.1 CPF 2.0 UPF 1.0 (Accellera) UPF 2.x (IEEE 1801) UPF 3.0 Internal / Proprietary
ASIC: Notation to Specify Power Intent
* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

78 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section G.

SIMULATION METHODOLOGIES

© Mentor Graphics Corporation

 ASIC: Methodologies and Testbench Base-Class Libraries
80%
2014
2016
70%
2018
60% Next Year

50%
Design Projects

40%

30%

20%

10%

0%
Accellera UVM Accellera OVM Mentor AVM Synopsys VMM Synopsys RVM Cadence eRM Cadence URM OSVVM UVVM None/Other
Portable
Stimulus ASIC: Methodologies and Testbench Base-Class Libraries
* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

80 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Methodologies and Testbench Base-Class Libraries
60%
2014
2016
50% 2018
Next Year
40%
Design Projects

30%

20%

10%

0%
Accellera UVM Accellera Portable OVM Mentor AVM OSVVM UVVM None/Other
Stimulus
FPGA: Methodologies and Testbench Base-Class Libraries
* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

81 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section H.

VERIFICATION TECHNIQUES

© Mentor Graphics Corporation

 ASIC: Formal Technology Adoption
40%
2012

35% 2018

30%

25%
Design Projects

20%

15%

10%

5%

0%
Formal property checking Automatic formal verification
ASIC: Adoption of Static Techniques

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

83 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Formal Technology Adoption
25% 2012
2014
2016
20% 2018

15%
Design Projects

10%

5%

0%
Formal property checking Automatic formal verification
FPGA: Adoption of Static Techniques

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

84 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Adoption of Dynamic Techniques

Code coverage

Functional coverage

Assertions
2012
2014
2016

Constrained-Random 2018

0% 10% 20% 30% 40% 50% 60% 70% 80%

Design Projects
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

85 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Adoption of Dynamic Techniques

Code coverage

Functional coverage

Assertions
2014
2016
2018

Constrained-Random

0% 10% 20% 30% 40% 50% 60% 70% 80%

Design Projects
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

86 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Signoff Criteria
When all tests documented in the verification plan are complete and pass

When the project plan schedule says sign-off, assuming verification ok

When the emulated or prototyped design is working in-lab

When code or functional coverage says we have achieved our target

When we can no-longer think of any more tests to write

When the rate of bugs found per week drops below a specified goal

2014
When the project plan says sign-off, regardless of status
2018

Other

0% 10% 20% 30% 40%

ASIC Design Projects * Multiple answers possible

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

87 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Signoff Criteria
When all tests documented in the verification plan are complete and pass

When the project plan schedule says sign-off, assuming verification ok

When the emulated or prototyped design is working in-lab

When code or functional coverage says we have achieved our target

When we can no-longer think of any more tests to write

When the rate of bugs found per week drops below a specified goal

When the project plan says sign-off, regardless of status

2014

Other 2018

0% 10% 20% 30% 40% 50% 60%

FPGA Design Projects * Multiple answers possible

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

88 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section I.

VERIFICATION RESULTS

© Mentor Graphics Corporation

 ASIC: Completion to Project's Original Schedule
40%

2012: 67% Behind Schedule 2012

2014: 61% Behind Schedule 2014
30%
2016: 69% Behind Schedule
2016
2018: 69% Behind Schedule
2018
Design Projects

20%

10%

0%
More than 10% 10% EARLY ON-SCHEDULE 10% BEHIND 20% 30% 40% 50%
EARLY SCHEDULE
Actual ASIC design completion compared to project's original schedule
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

90 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Completion to Project's Original Schedule
40%

2012: 67% Behind Schedule 2012

2014: 59% Behind Schedule
2014
30% 2016: 65% Behind Schedule
2018: 64% Behind Schedule 2016
2018
Design Projects

20%

10%

0%
More than 10% 10% EARLY ON-SCHEDULE 10% BEHIND 20% 30% 40% 50% >50% BEHIND
EARLY SCHEDULE SCHEDULE
Actual FPGA design completion compared to project's original schedule
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

91 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Number of Required Spins Before Production

50% 2012
2014
2016
40%
2018
Design Projects

30%

20%

10%

0%
1 (FIRST SILICON 2 3 4 5 6 7 SPINS or MORE
SUCCESS)
Number of Required ASIC Spins Before Production

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

92 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Non-trivial Bug Escapes into Production

35% 84% of FPGA design projects 2016

have non-trivial bugs escape 2018

30% into production
25%
Design Projects

20%

15%

10%

5%

0%
0 1 2 3 4 5 6 or More
Non-trivial FPGA Bug Escapes into Production

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

93 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 2018 FPGA Verification Technique Adoption and Bug Escapes

61%
Code Coverage
77%

57%
Functional Coverage
65%

46%
Assertions
59%

Bug Escapes
30% No Bug Escapes
Constrained-Random
37%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Percentage Adoption
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

94 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: FPGA Safety Critical Designs with Production Bug Escapes

Percentage of FPGA Projects Working on Safety Critical Designs with Bug Escapes to Production

15%

No Bug Escapes into Production

Bug Escapes into Production

85%

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

95 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Type of Flaws Contributing to Respin
60%
2012
2014
50%
2016
2018
40%
Design Projects

30%

20%

10%

0%
LOGIC OR CLOCKING TUNING ANALOG CROSSTALK POWER MIXED-SIGNAL YIELD OR TIMING – PATH FIRMWARE TIMING – PATH IR DROPS OTHER
FUNCTIONAL CIRCUIT CONSUMPTION INTERFACE RELIABILITY TOO SLOW TOO FAST

Type of ASIC Flaws Contributing to Respin * Multiple answers possible

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

96 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Type of Flaws Contributing to a Production Issue
60%
2012
2014
50%
2016
2018
40%
Design Projects

30%

20%

10%

0%
LOGIC OR CLOCKING CROSSTALK POWER MIXED-SIGNAL TIMING – PATH FIRMWARE TIMING – PATH OTHER
FUNCTIONAL CONSUMPTION INTERFACE TOO SLOW TOO FAST

Type of FPGA Flaws Contributing to a Production Issue

* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

97 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC vs FPGA Flaws
50% FPGA

ASIC

40%
Design Projects

30%

20%

10%

0%
LOGIC OR FUNCTIONAL CLOCKING
ASIC vs FPGA Flaws
* Multiple answers possible
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

98 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Root Cause of Functional Flaws
80%
2012
70% 2014
2016
60%
2018
50%
Design Projects

40%

30%

20%

10%

0%
DESIGN ERROR CHANGES IN SPECIFICATION INCORRECT or INCOMPLETE FLAW IN INTERNAL REUSED FLAW IN EXTERNAL IP BLOCK or
SPECIFICATION BLOCK, CELL, MEGACELL or IP TESTBENCH

Root Cause of ASIC Functional Flaws * Multiple answers possible

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

99 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Root Cause of Functional Flaws
90%
2012
80% 2014

70% 2016
2018
60%
Design Projects

50%

40%

30%

20%

10%

0%
DESIGN ERROR CHANGES IN SPECIFICATION INCORRECT or INCOMPLETE FLAW IN INTERNAL REUSED FLAW IN EXTERNAL IP BLOCK or
SPECIFICATION BLOCK, CELL, MEGACELL or IP TESTBENCH

Root Cause of FPGA Functional Flaws * Multiple answers possible

Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

100 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 ASIC: Biggest Challenge
CREATING SUFFICIENT TESTS TO VERIFY THE DESIGN (Coverage
Closure)

KNOWING MY VERIFICATION COVERAGE

MANAGING THE VERIFICATION PROCESS

TIME TO ISOLATE AND RESOLVE A BUG

TIME TO DISCOVER THE NEXT BUG

2012
2014
DEFINING APPROPRIATE COVERAGE METRICS 2016
2018
Other

0% 5% 10% 15% 20% 25% 30% 35% 40%

Design Projects
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

101 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 FPGA: Biggest Challenge
CREATING SUFFICIENT TESTS TO VERIFY THE DESIGN (Coverage
Closure)

KNOWING MY VERIFICATION COVERAGE

MANAGING THE VERIFICATION PROCESS

TIME TO ISOLATE AND RESOLVE A BUG

TIME TO DISCOVER THE NEXT BUG

2016
2018
DEFINING APPROPRIATE COVERAGE METRICS

Other

0% 10% 20% 30% 40% 50%

Design Projects
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
© Mentor Graphics Corporation

102 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 Section J.

CONCLUSIONS & MORE INFORMATION

© Mentor Graphics Corporation

 Conclusions

 IC/ASIC projects are mature in their processes

 FPGA projects are being forced to mature their processes

 FPGA projects have a long way to go to prevent bug

escapes into production

© Mentor Graphics Corporation

104 H Foster, WRG Functional Verification Study, July 2016
104 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
 More Information
 A video of this web seminar will be available out on the Verification
Academy in a few weeks

 My Verification Horizon Blog has all the data from previous studies,
and we are in the process of releasing this years study

 We are planning papers and presentations based on this study at

various conferences and seminars this next year
© Mentor Graphics Corporation
105 H Foster, WRG Functional Verification Study, July 2016
105 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
© Mentor Graphics Corporation
www.mentor.com