Gupta Thesis 2017
Gupta Thesis 2017
by
Kshitiz Gupta
2017
The Thesis Committee for Kshitiz Gupta
certifies that this is the approved version of the following thesis:
APPROVED BY
SUPERVISING COMMITTEE:
Mark McDermott
Automatic generation of coverage directives targeting
signal relationships by statically analyzing RTL
by
THESIS
Presented to the Faculty of the Graduate School of
The University of Texas at Austin
in Partial Fulfillment
of the Requirements
for the Degree of
v
Automatic generation of coverage directives targeting
signal relationships by statically analyzing RTL
vi
Table of Contents
Acknowledgments v
Abstract vi
List of Tables ix
List of Figures x
Chapter 1. Introduction 1
vii
3.7.2 Finding state registers . . . . . . . . . . . . . . . . . . . 27
3.7.3 Writing cover properties . . . . . . . . . . . . . . . . . . 28
3.7.4 Other salient features . . . . . . . . . . . . . . . . . . . 29
Chapter 4. Results 31
4.1 Tool Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 Case study : V-scale . . . . . . . . . . . . . . . . . . . . . . . 33
4.2.1 Multiply and divide unit . . . . . . . . . . . . . . . . . . 34
4.2.2 CSR file . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.3 Case study: Amber ARM . . . . . . . . . . . . . . . . . . . . . 37
4.3.1 Multiply unit . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3.2 Decode unit . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3.3 Wishbone unit . . . . . . . . . . . . . . . . . . . . . . . 38
4.3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Appendices 45
Bibliography 74
viii
List of Tables
ix
List of Figures
x
Chapter 1
Introduction
1
to understand, scalable and focuses on exercising the ambiguity in the control
flow of the design. We take the approach of statically analyzing the given RTL
and automatically generating coverage directives which target the ambiguity
in signal relationships. The goal of these coverage metrics is to ensure complete
coverage of RTL. We target cases which have some form of ambiguity, which
makes line coverage ineffective in covering them. We propose the use of these
directives alongside line coverage.
2
Chapter 2
3
specified by a user. It is one of the most widely used metric to measure
coverage.
6. State coverage – State coverage targets the extracted FSMs from RTL.
State coverage of an FSM ensures that all the states in the FSM have
been visited.
7. Transition coverage – This also targets the extracted FSMs and en-
sures that all transitions (arcs) in the extracted state machine are exer-
cised.
4
The above metrics come with their share of drawbacks. Hardware designs have
multiple communicating processes as interacting FSMs, and in such cases line
coverage, branch coverage and condition coverage metrics are not complete
for making sure all the possible signal interactions have been exercised. Path
coverage on the other hand is a very stringent requirement as the number of
paths can be exponential. Coverage metrics like toggle coverage are indirect
metrics at best and do not help in identifying the coverage of the control
behavior. Full state and transition coverage are not scalable metrics as the
number of states and arcs in a design can be exponential.
The authors in [33, 34] build a global state graph and then identify
important control events for coverage. They attempt to control the size of the
graph using heuristics. In [35, 41], the authors use input from the designer
about the important state registers to construct an extracted control flow
machine (ECFM). The coverage of the test vectors is then measured as the
state and transition coverage of the ECFM. The work done in [45, 46] uses
heuristics to identify important control states and generate path tours to be
used for coverage measurement. The major drawback of these approaches is
use of some form of explicit FSM extraction, which challenges practical use
of such approaches. The authors in [26, 29] propose tag coverage as a metric.
The calculus presented allows for tag propagation with every test vector and
coverage is computed as a percentage of the total number of tags. The authors
5
of [37] propose the TRIO (Input/Output TRansition) fault model to be used
for coverage measurements. They propose this model for use in functional test
selection.
6
2.4 Tool Support
7
The work done in [37] talks about using coverage metrics for functional test
selection for testing.
8
Chapter 3
3.1 Motivation
1. We want to look at metrics which are more expressive than line coverage,
while at the same time avoid the exponential nature of techniques such
as state enumeration, symbolic execution and FSM extraction.
2. We want to use static RTL analysis for our metrics. The RTL model
contains the detailed description of the design’s functionality. Also, dur-
ing implementation a designer may take decisions which may not be
reflected well in coverage metrics built purely from specifications. We
argue against analyzing simulation traces as there is uncertainty of ever
hitting a corner case.
9
4. Coverage metrics should ensure that the RTL description has been thor-
oughly exercised.
Overall, we are looking for metrics which are automatic, can be generated by
static analysis of the RTL, avoid state-explosion and focus on the control flow
of the design. Keeping the above pointers in mind, we look at property syn-
thesis by static analysis of the RTL. We will focus on writing SystemVerilog
cover properties by analyzing RTL written in Verilog HDL. The SystemVer-
ilog properties can be easily integrated with any modern simulator to provide
coverage goals.
10
also be thought of as state variables of that particular always block. The
variables on the right-hand side of the assignments are called Data inputs
and the variables in the predicates of the control structures are called the
Control inputs.
We look at the intersection between two always blocks and how coverage
fits in the next section.
11
3.3 Coverage goals
On the other end of the spectrum, let us define control path coverage.
Control path coverage attempts to look at all possible paths that can be taken
by the code when executing. 100% control path coverage will be a cross-
product of all the paths that all the always blocks individually can take. Full
control path coverage includes all possible ways the RTL can be executed
and therefore, we argue that it covers the control flow of the design. In this
section, when we say path coverage, we mean control path coverage. It is also
a superset of metrics like line coverage and branch coverage. However, the
possible number of such paths can be exponential. Such a metric would not
scale well with designs. In this thesis, instead of taking combinations over
all the always blocks, we will look at the subset of all pairs of always blocks.
Therefore, we will look at pairs of always blocks with the motivation of writing
cause-effect properties to get better coverage.
Consider a case depicted in Fig. 3.1. These are code snippets from two
different always blocks in the same module. To get complete line coverage,
2 tests would be enough : (x = 0, y = 0) and (x = 1, y = 1). However, to
get complete path coverage, we need 4 tests : (x = 0, y = 0), (x = 0, y = 1),
12
// Always #1 // Always #2
// always@ (*) // always@ (*)
if ( x ==0) begin if ( y ==0) begin
... ...
end end
else begin else begin
... ...
end end
13
2. Consider another case where the two blocks are not independent. That is,
x and y have some correlation. We can divide this case into 2 sub-cases.
We call the first case as deterministic dependence. Consider Fig. 3.2
where x and y are completely linked. In this case, complete path coverage
is not allowed by the code due to the dependence between the blocks. In
this situation, complete line coverage is enough to ensure complete path
coverage(considering only allowable cases). This is because Block 2 can
only execute when Block 1 executes.
We conclude with this discussion that control path coverage can include a lot
of redundancy. A cross product between a set of paths should only be taken
14
// Always #1 // Always #2
// always@ ( posedge clk ) // always@ (*)
if ( x ==0) begin if ( z ==0) begin
if ( a ==0) begin if ( y ==0) begin
y <=0; ...
end end
end end
else begin else begin
y <=0; ...
end end
if they have some dependence on each other. At the same time, it must be
ensured that the dependency is non-deterministic, or else line coverage can
cover those cases.
15
o wb dat is assigned some value if wishbone st is WB WAIT ACK along with
some other conditions. It is possible to get 100% line coverage in this situation
without necessarily exercising both the situations where o wb dat is assigned
extra write r due to each of the wishbone st assignments here. Therefore,
additional coverage points are required, which we will present in the form of
SystemVerilog cover properties.
16
of another register. To ensure that the effect is covered, consequents are made
from the conditions surrounding the effect assignment. The tool will analyze
the RTL statically to create these relationships, generate the conditions and
match them to write SystemVerilog properties.
17
always @ ( posedge i_clk or negedge quick_n_reset ) begin
// Always #1
if ( ! quick_n_reset )
o_wb_dat <= ’ d0 ;
else if ( wishbone_st == WB_WAIT_ACK && i_wb_ack &&
extra_write_r )
o_wb_dat <= e xt ra _wr it e_ da ta _r ;
else if ( start_access )
o_wb_dat <= i _ d ca c h e_ w r it e _ da t a ;
end
18
Figure 3.5: State and data registers
also depend on the history of the module. However, this dependency would
be indirect as it would go through the aforementioned state registers. This
heuristic is similar to the method use in Mentor Questa SIM to find FSMs in
a design [12]. It is also similar to the work done in [39]. The wishbone module
from Amber ARM [4] has been considered again as an example in the next
section.
Analyzing the wishbone module in ARM Amber [4] by the above heuris-
tic identifies 4 registers - wishbone st, extra write r, dcache cached rreq r
and dcache uncached rreq r. These registers are responsible for maintaining
a state of the module, as they store in them some form of the history of the
module’s execution.
In a module written purely with clocked always blocks, the state reg-
isters can be identified by the intersection of the output and control signals.
19
However, when writing Verilog HDL with combinational always blocks, such
an approach will not work. We look at that in the next section.
Following the above idea, we start with the combinational always block
in the design. We see state in control and next state as an output signal.
We follow the signal next state and reach the clocked always block, where
next state is a control signal and state is an output signal. We have terminated
the process at a clocking event and conclude that state is an important register.
The basic approach has been presented. We will present a short discus-
sion about the presented methodology before talking about a Python based
implementation in Section 3.7.
20
Figure 3.6: Combinational always blocks
21
reduce the number of paths to be covered by identifying redundant and
infeasible paths.
3. Toggle Coverage – Metrics like toggle coverage and bit coverage are
indirect metrics at best. They do not actually help in identifying how
much of the control behavior has been covered. Our proposed metric
is directly associated with the control flow of the design and signal re-
lationships, and thus would better in identifying holes in control flow
coverage.
The proposed cover properties are attempting to highlight states in the design
by static analysis of the RTL. They are trying to identify corner-case execution
paths which will not be covered by traditional code coverage metric. Every
property attempts to target a particular state that the module can be in. A
benefit of our approach is the scalability of our heuristics. All the methods
proposed in the previous sections will by polynomial in both time and memory
for the number of lines of RTL.
22
of the interactions we discover are not meant to be executed and the conditions
around the design are such that they can never be executed. The states that
some properties identify can be unreachable states. We used the formal tool
Cadence JasperGold on the properties generated by our Python-based tool
and found some unreachable properties. We address this issue in Chapter 4
with the help of case studies for more details and experimental evidence.
3.7 Implementation
Pyverilog [52] is an open-source toolkit for RTL analysis and code gen-
eration of Verilog HDL. It consists of 4 key engines: parser, dataflow analyzer,
control-flow analyzer and a Verilog code generator.
23
Start
Parser
Create AST from Verilog (Section 3.7.1)
Module Analyzer
Find module definitions (Section 3.7.1)
Make lists for state reg analysis (Section 3.7.2)
Signal Analyzer
Resolve constants and manage scope (Section 3.7.1)
Bind Analyzer
Create dataflow graphs for all regs (Section 3.7.1)
Property Writer
Analyze dataflows and write SystemVer-
ilog cover properties (Section 3.7.3)
Stop
24
• Dataflow Analyzer – The dataflow analyzer constructs a dataflow
graph representing the relationship among signals by analyzing the AST
generated by the parser. A Verilog code snippet and the extracted
dataflow graph has been presented in Fig. 3.8. Both the graphical and
textual representations of the graph have been shown in Fig. 3.8. The
analyzer itself has three subsystems - module analyzer, signal analyzer,
and bind analyzer. These subsystems have been implemented using a
standard visitor pattern where a specific function for each visited AST
node is called by its class name recursively. The module analyzer tra-
verses the input AST to build a list of modules and their inputs, outputs
and parameters. The signal analyzer traverses the AST tree again to
analyze all the signal declarations and associate them with a scope. The
constants are also resolved in this step. Finally, the bind analyzer gener-
ates a dataflow graph for each signal, i.e., a tree consisting of arithmetic
and logical operators, assignment conditions (branches) and assigned val-
ues. The dataflow analyzer has been heavily modified and new engines
have been added around it to support our approach.
25
if ( rst ) begin
state <=0;
end
else begin
if ( state ==1 ’ b0 ) begin
if ( x ) begin
state <=1 ’ b0 ;
end
else begin
state <=1 ’ b1 ;
end
end
else begin
state <=1 ’ b0 ;
end
end
26
3.7.2 Finding state registers
• When parsing the AST, the three lists are populated by analyzing the
assignment statements and conditional operators.
• After the module has been analyzed, we search for the state registers.
For a clocked always block, we look for an intersection within the control
list and the state list. If this intersection is not empty, we append that
to the list of module state registers.
• For combinational always blocks (no clock), we employ the method pro-
posed in Section 3.5.2. We look for always blocks whose data or control
set intersects with the output set of combinational block being analyzed.
If any of the shortlisted always blocks themselves are combinational, the
same search kicks off again till we terminate at a clocked always block.
When a clocked block is reached, we look for an intersection of the state
set of the clocked always block and the control set of the combinational
always block. If this intersection is not empty, we add the register to the
list of module state registers.
27
• Other heuristics (For eg. considering the signals with the most appear-
ance in predicates) can also be integrated easily at this step if future
works demands it.
After finding this list of state registers, we proceed towards looking at dataflow
trees and writing SystemVerilog cover properties.
Another subsystem is added after the bind analyzer called the property
writer. The property writer uses the list of module state registers(from Section
3.7.2) and dataflow trees(generated by the bind analyzer) to write properties.
• The system starts by analyzing the dataflow trees of the state registers to
create a valuetable. It traverses the tree to generate a valuetable with the
assignment value and the corresponding predicates which would result in
that value. Since we are focusing on tackling ambiguity, the assignment
values with only one condition are rejected, as those cases can be covered
by line coverage. The remaining cases are stored in the table. Thus, the
antecedents for the properties are ready.
• The dataflow graphs of all the registers are traversed and the conditions
are analyzed and compared with the valuetable created in the previous
step. When a match is found between a condition and an entry on
the valuetable, it is noted by the tool and valid properties are written.
28
Another engine called the SVWriter is responsible for writing all the
SystemVerilog properties with the correct syntax and declarations.
• This analysis follows the same visitor pattern as done by other engines
in the toolkit. It starts with the top module, writes the necessary prop-
erties, and then continues along the hierarchy to write the properties for
each design unit.
Some important features of the tool that were added have been men-
tioned in this section.
• The tool offers the capability to black box certain modules. This has been
done to ensure the robustness of the tool. Black boxing can be happen
in two cases. The first when the user provides a list of modules which
should not be analyzed. This can be to ensure faster processing times for
keeping system memory considerations. The second case is when the tool
itself cannot find a particular module (when it is elaborating instance
declarations), it automatically black boxes the module and continues
with the processing instead of aborting.
• It was observed that the properties were not unique. This is possible
when a control block has multiple assignment statements, which results
in different registers having the same resulting property. To avoid ver-
bosity in the output, a uniqueness check can be done to filter out multiple
29
instances of the same property.
30
Chapter 4
Results
The Python tool was run over multiple open-source designs, and the
results obtained have been tabulated in Table 4.1. The presented designs
include cores such as Amber ARM [4], V-scale [6], OR1200 [5] and a 32-
bit MIPS [3]. Controller modules such as SDRAM controllers (16 and 32-
bit), PID controller and an I2C controller have also been analyzed. These
designs are available at OpenCores [3]. Other miscellaneous designs available
at OpenCores such as an AES core, Reed Solomon decoder, Wishbone FLASH
interface and a Wishbone controlled UART have also been analyzed. Table
4.1 presents for each design the total number of cover properties generated by
running the tool in both normal and exhaustive mode (Section 3.7.4).
We would like to mention that there are designs such as the OpenSPARC
[9] where the tool was unable to generate properties due to it not being able to
identify state registers. The RTL has to be behavioral description of the de-
sign for the tool to identify ambiguity. Structural RTL with the state machine
spread across multiple modules cannot be handled by the tool in its current
form.
31
State Cover Cover
Lines of
Design registers properties properties
RTL
identified (normal) (exhaustive)
Amber ARM 6317 7 101 234
AES128 2895 1 182 266
I2C controller 1968 3 229 250
MIPS32 2185 1 70 81
OR1200 31640 8 200 237
PID controller 2151 3 18 37
Reed Solomon
4321 30 1512 1684
decoder
16-bit SDRAM
741 3 75 186
controller
32-bit SDRAM
3961 5 181 212
controller
V-scale 1727 3 26 29
Wishbone
112 1 6 8
FLASH
Wishbone
1934 2 139 202
UART
32
A concern raised with these properties is their possible unreachability.
A formal analysis using Cadence JasperGold of the designs presented in Table
4.1 found about 30% of the properties to be unreachable.
We present case studies with two different cores in the following sec-
tions. The arguments for the usefulness of these properties have been dis-
cussed. Formal analysis has also been used to analyze the properties and
study the issue of unreachability in more detail.
We discuss specific results with respect to the V-scale [6] core developed
as an implementation of RV32IM [56]. The results presented are when our
Python tool was run in exhaustive mode. As mentioned, a total of 29 properties
were generated for the design. These were distributed as 24 properties for the
md unit (multiply and divide) and 5 properties for the csr file unit (control and
status registers). All 29 properties are attached in Appendix B. To test the
validity of the properties, they were instantiated with the testbench provided
by the designers to be run for all the 47 tests provided in the package. The
results are presented in Table 4.2
33
// V - scale md unit code snippet
always @ (*) begin
case ( state )
s_idle : next_state = ( req_valid ) ? s_compute :
s_idle ;
s_compute : next_state = ( counter == 0) ?
s_setup_output : s_compute ;
s_setup_output : next_state = s_done ;
s_done : next_state = s_idle ;
default : next_state = s_idle ;
endcase // case ( state )
end
For the md unit, 10 properties were not covered (2, 3, 5, 7, 10, 17,
18, 19, 20, 22). The thing to note is that 100% line coverage was reported
for the md unit. The full design was then formally checked for reachability
of the properties using Cadence JasperGold. Of the 24 md unit properties, 4
properties were proven unreachable by the formal tool (5, 17, 18, 20). This
implies that the other 6 uncovered properties are valid scenarios for the design
to be exercised in, which the provided testbench does not cover.
It was observed that all the 4 unreachable properties have the same
antecedent. Formal analysis found this antecedent unreachable, thus making
the 4 implication properties unreachable. Consider Fig. 4.1 which shows the
code snippet where the antecedent comes from. The default option can never
be exercised, thus leading to unreachable code.
34
// V - scale csr file code snippet
next_htif_state = htif_state ; // Highlight
case ( htif_state )
HTIF_STATE_IDLE : begin
if ( hti f_ pc r_ re q_ va li d ) begin
htif_fire = 1 ’ b1 ;
next_htif_state = HTIF_STATE_WAIT ;
end
end
HTIF_STATE_WAIT : begin
if ( h ti f _ pc r _ re s p _r e a dy ) begin
next_htif_state = HTIF_STATE_IDLE ;
end
end
endcase // case ( htif_state )
An analysis similar to Section 4.2.1 was done for the csr file. The 2
uncovered properties were proven unreachable (26, 27). It was observed that
both the properties have the same antecedent, which was proven unreachable.
Consider the code snippet in Fig. 4.2. The tool generates a property at-
tempting to make the highlighted statement as the cause (antecedent) for the
property. However, the case statement following the statement will always
take precedence, no matter what the input.
4.2.3 Discussion
There were properties written by the tool which were uncovered in all
the tests but proven reachable through formal analysis (Section 4.2.1). We
35
present an analysis of these properties here. All the properties can be found
in Appendix B. A total of 6 properties fall in this case (2, 3, 7, 10, 19, 22). 3
(7, 19, 22) out of these 6 properties have antecedent as (reset). That means,
these 3 properties target different cases of system startup. Another property
(16) which has an antecedent as (reset) was covered by the tests. The other
3 properties (2, 3, 10) also share the same antecedent. The antecedent corre-
sponds to the end of an operation (multiply/divide). All the three properties
cover cases where a new request is given to the md unit right after the previ-
ous request ends. The properties differ in what kind of new request has been
given. Property 12 (prop 12) shares the same antecedent and was covered
by the tests. There are covered properties which share consequents with the
three properties as well. In summary, the antecedents and consequents of the
properties have been covered independently, however some special interactions
(which are reachable) have not been exercised. It can be observed that the
targeted ambiguity is not just an artifact of the RTL written, but rather cov-
ers interesting behavioral/functional cases. The cases uncovered by the tests
(different start-up from reset, back to back requests) do qualify for corner-case
behavior of the multiply and divide unit under consideration.
36
can never be exercised.
We present an analysis similar to Section 4.2 for the Amber ARM [4].
Due to the large number of properties in the exhaustive mode (Table 4.1),
we limit our discussion to results of normal mode execution of the tool. The
tool when run with Amber ARM generates 101 properties distributed between
multiply unit (3), decode unit (22) and the wishbone module (76). It should
be noted that while running the tool, the icache and dcache in the design
have been black-boxed. These properties have been enumerated in Appendix
C. With all the 59 test cases available for the Amber unit, 47 out of 101
properties were covered. The results of the tests are given in Table 4.3.
37
4.3.2 Decode unit
Another formal test was done with the same properties, with the differ-
ence being the DUV. Instead of formally analyzing the complete Amber core,
we instead focused on analyzing the decode module as a standalone unit. This
analysis found a trace for 4 out of the 6 previously unreachable properties.
The remaining 2 properties were found to be unreachable due to unreachable
consequents (unreachable dead code).
38
the individual wishbone unit.
All the 8 properties (48. 52. 53. 58, 63, 85, 90, 91) have reachable
consequents and antecedents, which means that they cannot be justified as
dead code. Let us consider prop 53 in Appendix C as an example. This prop-
erty is about starting from a reset state and going to a state where the design
waits for an acknowledgement. This situation can never happen. prop 58 is
about waiting for an acknowledgement after a previous acknowledgement was
received and there is no other request. These properties targeted ambiguity
in the design, however the states they target are unreachable. This presents
evidence of the unreachability issue we expressed in Section 3.6.
4.3.4 Discussion
The observations made in Section 4.2.3 can also be made for this unit.
We will discuss the properties uncovered by all the tests but proven reach-
able by formal analysis. For the multiply unit, the uncovered reachable prop-
erty (24) deals with the reset state. 11 (31, 32, 33, 37, 38, 39, 40, 69, 81, 82,
95) out of the 26 uncovered reachable properties for the wishbone unit also
deal with the reset state. The remaining 15 properties in the wishbone unit
(29, 26, 24, 35, 36, 47, 50, 64, 72, 78, 79, 87, 89, 93, 100) correspond to behav-
ioral/functional cases. For example, properties 26 and 29 cover cases where
the wishbone st register (the main control register in the design) goes from
BURST3 to WB WAIT ACK and receives an acknowledgement in the very next cy-
cle. Property 87 also targets a similar case but where the state goes from IDLE
39
to WB WAIT ACK and receives a similar fast acknowledgement. Therefore, these
uncovered properties are targeting behavioral corner-cases where the design
should be tested (different system start-ups, fast acknowledgements).
Sections 4.3.2 and 4.3.3 also present arguments for the usefulness of
some of the unreachable properties. It has been formally shown that of 19
out of the 27 unreachable properties point at either actual dead code in the
individual units or pseudo-dead code, i.e., logic available in the design which
cannot be exercised by the complete module. A total of 8 properties out of
101 (7.9%) are actually unreachable due to constraints in the specification.
4.4 Summary
1. Table 4.1 demonstrates the basic capabilities of the tool and showcases
its application on a range of designs.
2. Two case studies (Sections 4.2 and 4.3) to discuss the properties, present
experimental results and tackle the issue of unreachability. For both
cases, the properties generated by the tool were added to the testbenches
provided with the designs. The coverage results obtained after executing
all the results were analyzed. A formal analysis using Cadence Jasper-
Gold was also done with all the properties to test for unreachability.
40
3. Properties were found (Sections 4.2.1, 4.3.1, 4.3.3) which were uncov-
ered but proven reachable. Discussions were presented (Sections 4.2.3
and 4.3.4) to show that these properties target interesting cases for the
execution of the design.
4. The unreachable properties were analyzed for the reasons of their un-
reachability. It was presented in Sections 4.2.1 and 4.2.2 that some
of the unreachable properties were due to verbose code in the design.
Sections 4.3.2 and 4.3.3 also presented cases where the properties were
reachable in the individual design units, but unreachable in the big-
ger system. Some properties (Section 4.3.3) still remained which tar-
geted unreachable states in the design, highlighting a drawback of the
approach. However, such properties only made up 29.6% of the total
unreachable properties (8 out of 27 for the Amber).
41
Chapter 5
42
The drawback of unreachability has been discussed. Properties were
found to be unreachable as they targeted unreachable states. The current
approach also does not take into account multiple states together. For designs
with multiple state registers with ambiguity, directives can be built by taking
different combinations.
For future work, this can be extended for both correctness and com-
pleteness. For correctness, the tool can be made smarter to avoid unreachable
properties. Using a constraint solver in the tool can help alleviate some prob-
lems. Heuristics (or even input from the user) can be used to avoid unreach-
able properties stemming from known unreachable code in the design (such as
default in case statements).
43
units it analyzed. Therefore, a better approach for finding these important
control registers can assist the tool’s completeness.
44
Appendices
45
Appendix A
46
next_state = 3 ’ b000 ;
case ( state )
IDLE : if ( req_0 == 1 ’ b1 ) begin
next_state = GNT0 ;
end else if ( req_1 == 1 ’ b1 ) begin
next_state = GNT1 ;
end else begin
next_state = IDLE ;
end
GNT0 : if ( req_0 == 1 ’ b1 ) begin
next_state = GNT0 ;
end else begin
next_state = IDLE ;
end
GNT1 : if ( req_1 == 1 ’ b1 ) begin
next_state = GNT1 ;
end else begin
next_state = IDLE ;
end
default : next_state = IDLE ;
endcase
end
// - - - - - - - - - - Seq Logic - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
always @ ( posedge clock )
begin : FSM_SEQ
if ( reset == 1 ’ b1 ) begin
state <= IDLE ;
end else begin
state <= next_state ;
end
end
// - - - - - - - - - - Output Logic - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
always @ ( posedge clock )
begin : OUTPUT_LOGIC
if ( reset == 1 ’ b1 ) begin
gnt_0 <= 1 ’ b0 ;
gnt_1 <= 1 ’ b0 ;
end
else begin
47
case ( state )
IDLE : begin
gnt_0 <= 1 ’ b0 ;
gnt_1 <= 1 ’ b0 ;
end
GNT0 : begin
gnt_0 <= 1 ’ b1 ;
gnt_1 <= 1 ’ b0 ;
end
GNT1 : begin
gnt_0 <= 1 ’ b0 ;
gnt_1 <= 1 ’ b1 ;
end
default : begin
gnt_0 <= 1 ’ b0 ;
gnt_1 <= 1 ’ b0 ;
end
endcase
end
end // End Of Block OUTPUT_LOGIC
endmodule // End of Module arbitergg
48
Appendix B
// MD properties
prop_1 : cover property ( @ ( posedge ( clk ) ) ((!( reset ) ) &&(!((
state ==0) ) ) &&(( state ==1) ) &&(!(( counter ==0) ) ) ) | - >##1
((!(( state ==0) ) ) &&(( state ==1) ) &&(( op ==0) ) &&( a [ counter
]) ) ) ;
49
state ==0) ) &&( req_valid ) ) | - >##1 ((!(( state ==0) ) ) &&((
state ==1) ) &&(( op ==0) ) &&( a [ counter ]) ) ) ;
50
&&(!(( state ==0) ) ) &&(!(( state ==1) ) ) &&(!(( state ==2) ) )
&&(!(( state ==3) ) ) ) | - >##1 ((( state ==0) ) &&( req_valid )
&&(!(( op ==2) ) ) ) ) ;
// csr properties
prop_25 : cover property ( @ ( posedge ( clk ) ) ((!( htif_reset ) )
&&(( htif_state ==0) ) &&( h ti f_ pc r_ re q_ va li d ) ) | - >##1 ((!((
htif_state ==0) ) ) &&(( htif_state ==1) ) &&(
h t if _ p cr _ r es p _ re a d y ) ) ) ;
51
prop_27 : cover property ( @ ( posedge ( clk ) ) ((!( htif_reset ) )
&&(!(( htif_state ==0) ) ) &&(!(( htif_state ==1) ) ) ) | - >##1
((( htif_state ==0) ) &&( ht if _p cr _r eq_ va li d ) ) ) ;
52
Appendix C
// Decode Properties
prop_1 : cover property ( @ ( posedge ( i_clk ) ) ((!((!(
quick_n_reset ) ) ) ) &&((!( i_access_stall ) ) ) &&(!(
ins tructi on_val id ) ) &&(( control_state ==16) ) ) | - >##1 ((!(
ins tructi on_val id ) ) &&(!(( control_state ==16) ) ) &&(!((
control_state ==14) ) ) &&(!(( control_state ==13) ) ) &&(!(((
m t r a n s _ n u m _ r e g i s t e r s ==1) &&( control_state ==11) ) ) ) &&(!((
control_state ==10) ) ) &&(!((( control_state ==7) ||(
control_state ==19) ) ) ) &&(!(( control_state ==6) ) ) &&(!((
control_state ==12) ) ) &&(( control_state ==15) ) ) ) ;
53
control_state ==14) ) ) &&(!(( control_state ==13) ) ) &&(((
m t r a n s _ n u m _ r e g i s t e r s ==1) &&( control_state ==11) ) ) &&(!(
dabt ) ) &&( write_pc ) ) | - >##1 ((!( in struct ion_va lid ) )
&&(!(( control_state ==16) ) ) &&(!(( control_state ==14) ) )
&&(!(( control_state ==13) ) ) &&(!((( m t r a n s _ n u m _ r e g i s t e r s
==1) &&( control_state ==11) ) ) ) &&(!(( control_state ==10) ) )
&&(!((( control_state ==7) ||( control_state ==19) ) ) ) &&((
control_state ==6) ) ) ) ;
54
control_state ==14) ) &&( i_multiply_done ) &&(!(
o _m u l ti p l y_ f u nc t i on [1]) ) ) | - >##1 ((!(((
i ns t r uc t i on _ e xe c u te &&( control_state ==20) ) &&(!( conflict
) ) ) ) ) &&(( control_state ==15) ) ) ) ;
55
ins tructi on_val id ) ) &&(!(( control_state ==16) ) ) &&(!((
control_state ==14) ) ) &&(!(( control_state ==13) ) ) &&(((
m t r a n s _ n u m _ r e g i s t e r s ==1) &&( control_state ==11) ) ) &&( dabt
) ) | - >##1 ((!((( i n s tr u c ti o n _e x e cu t e &&( control_state
==20) ) &&(!( conflict ) ) ) ) ) &&(!(( control_state ==15) ) ) &&((
control_state ==12) ) &&( r e s t o r e _ b a s e _ a d d r e s s ) ) ) ;
56
quick_n_reset ) ) ) ) &&((!( i_access_stall ) ) ) &&(!(
ins tructi on_val id ) ) &&(!(( control_state ==16) ) ) &&(!((
control_state ==14) ) ) &&(!(( control_state ==13) ) ) &&(!(((
m t r a n s _ n u m _ r e g i s t e r s ==1) &&( control_state ==11) ) ) ) &&((
control_state ==10) ) &&(!(( m t r a n s _ i n s t r u c t i o n _ n x t
[15:0]!=0) ) ) &&( dabt ) ) | - >##1 ((!( instr uction _valid ) )
&&(!(( control_state ==16) ) ) &&(!(( control_state ==14) ) )
&&(!(( control_state ==13) ) ) &&(!((( m t r a n s _ n u m _ r e g i s t e r s
==1) &&( control_state ==11) ) ) ) &&(!(( control_state ==10) ) )
&&(!((( control_state ==7) ||( control_state ==19) ) ) ) &&(!((
control_state ==6) ) ) &&(( control_state ==12) ) ) ) ;
57
prop_19 : cover property ( @ ( posedge ( i_clk ) ) ((!((!(
quick_n_reset ) ) ) ) &&((!( i_access_stall ) ) ) &&(!(
ins tructi on_val id ) ) &&(!(( control_state ==16) ) ) &&(!((
control_state ==14) ) ) &&(!(( control_state ==13) ) ) &&(!(((
m t r a n s _ n u m _ r e g i s t e r s ==1) &&( control_state ==11) ) ) ) &&((
control_state ==10) ) &&(!(( m t r a n s _ i n s t r u c t i o n _ n x t
[15:0]!=0) ) ) &&( dabt ) ) | - >##1 ((!((( i n s tr u c ti o n _e x e cu t e
&&( control_state ==20) ) &&(!( conflict ) ) ) ) ) &&(!((
control_state ==15) ) ) &&(( control_state ==12) ) &&(
restore_base_address )));
58
// Multiply Properties
prop_23 : cover property ( @ ( posedge ( i_clk ) ) ((!((!(
quick_n_reset ) ) ) ) &&((!( i_access_stall ) ) ) &&( i_execute )
&&(( count ==0) ) &&(!( enable ) ) ) | - >##1 ((( count ==0) ) ) ) ;
// Wishbone Properties
prop_26 : cover property ( @ ( posedge ( i_clk ) ) ((!((!(
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&((
wishbone_st ==3) ) &&( i_wb_ack ) ) | - >##1 ((!((!(
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) &&(
extra_write_r ) ) ) ;
59
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&((
wishbone_st ==3) ) &&( i_wb_ack ) ) | - >##1 ((!((!(
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) &&(!(
extra_write_r ) ) ) ) ;
60
prop_35 : cover property ( @ ( posedge ( i_clk ) ) ((!((!(
quick_n_reset ) ) ) ) &&(( wishbone_st ==0) ) &&(!(
wait_write_ack ) ) &&(( d c a c h e _ u n c a c h e d _ r r e q _ c ||
d c a c h e _ c a c h e d _ r r e q _ c ) ) &&(!( d ca c h e_ r e ad _ q wo r d _c ) ) )
| - >##1 ((!((!( quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) )
&&(!(( wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) ) ) ;
61
wishbone_st ==0) ) &&(!( wait_write_ack ) ) &&(!((
d c a c h e _ u n c a c h e d _ r r e q _ c || d c a c h e _ c a c h e d _ r r e q _ c ) ) ) &&(!(
write_req_c ) ) &&(( icac he_rea d_req_ c &&
i c ac h e _r e a d_ q w or d _ c ) ) ) ) ;
62
prop_45 : cover property ( @ ( posedge ( i_clk ) ) ((!((!(
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) &&(!(
extra_write_r ) ) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&((
wishbone_st ==0) ) &&(!( wait_write_ack ) ) &&((
d c a c h e _ u n c a c h e d _ r r e q _ c || d c a c h e _ c a c h e d _ r r e q _ c ) ) &&(
d c ac h e _r e a d_ q w or d _ c ) ) ) ;
63
extra_write_r ) ) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&((
wishbone_st ==0) ) &&( start_access ) &&( dcache_req_c ) &&(!((
byte_enable ==1) ) ) &&(!(( byte_enable ==2) ) ) &&(!((
byte_enable ==4) ) ) &&(!(( byte_enable ==8) ) ) &&((
byte_enable ==3) ) ) ) ;
64
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) &&(!(
extra_write_r ) ) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&((
wishbone_st ==0) ) &&(!( start_access ) ) &&((!(
wait_write_ack ) ) ) ) ) ;
65
prop_60 : cover property ( @ ( posedge ( i_clk ) ) (((!(
quick_n_reset ) ) ) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&((
wishbone_st ==0) ) &&( start_access ) &&( dcache_req_c ) &&((
byte_enable ==1) ) ) ) ;
66
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) &&(!(
extra_write_r ) ) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&((
wishbone_st ==0) ) &&( start_access ) &&( dcache_req_c ) &&(!((
byte_enable ==1) ) ) &&(!(( byte_enable ==2) ) ) &&(!((
byte_enable ==4) ) ) &&(!(( byte_enable ==8) ) ) &&(!((
byte_enable ==3) ) ) &&(!(( byte_enable ==12) ) ) ) ) ;
67
wishbone_st ==0) ) &&(!( wait_write_ack ) ) &&((
d c a c h e _ u n c a c h e d _ r r e q _ c || d c a c h e _ c a c h e d _ r r e q _ c ) ) ) ) ;
68
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) &&(!(
extra_write_r ) ) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&(
d c a c h e _ u n c a c h e d _ r r e q _ r ) &&((( wishbone_st ==4) ||(
wishbone_st ==0) ) ) ) ) ;
69
wishbone_st ==0) ) &&( start_access ) &&( dcache_req_c ) &&(!((
byte_enable ==1) ) ) &&(!(( byte_enable ==2) ) ) &&(!((
byte_enable ==4) ) ) &&(( byte_enable ==8) ) ) ) ;
70
) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&(
d c a c h e _ u n c a c h e d _ r r e q _ r ) &&((( wishbone_st ==4) ||(
wishbone_st ==0) ) ) ) ) ;
71
prop_94 : cover property ( @ ( posedge ( i_clk ) ) ((!((!(
quick_n_reset ) ) ) ) &&(!(( wishbone_st ==0) ) ) &&(!((
wishbone_st ==1) ) ) &&(!(( wishbone_st ==2) ) ) &&(!((
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) &&(!(
extra_write_r ) ) ) | - >##1 ((!((!( quick_n_reset ) ) ) ) &&((
wishbone_st ==0) ) &&(!( wait_write_ack ) ) &&((
d c a c h e _ u n c a c h e d _ r r e q _ c || d c a c h e _ c a c h e d _ r r e q _ c ) ) &&(!(
d c a c h e _ c a c h e d _ r r e q _ c ) ) &&( d c a c h e _ u n c a c h e d _ r r e q _ c ) ) ) ;
72
wishbone_st ==3) ) ) &&(( wishbone_st ==4) ) &&( i_wb_ack ) ) ) ;
73
Bibliography
[2] http://accellera.org/downloads/standards/uvm.
[3] http://opencores.org.
[4] http://opencores.org/project,amber.
[5] http://openrisc.io.
[6] http://riscv.org/2015/09/risc-v-in-verilog/.
[7] http://standards.ieee.org/findstds/standard/1800-2012.html.
[9] http://www.oracle.com/technetwork/systems/opensparc/.
[11] Jaspergold property synthesis apps property synthesis throughout the de-
sign flow for application in formal verification, simulation and emulation.
74
[14] Questa cdc and formal technologies datasheet.
[16] Prashant Aggarwal, Darrow Chu, Vijay Kadamby, and Vigyan Singhal.
Planning for end-to-end formal using simulation-based coverage: invited
tutorial. In Proceedings of the International Conference on Formal Meth-
ods in Computer-Aided Design, pages 9–16. FMCAD Inc, 2011.
[17] IEEE Standards Association et al. Ieee standard for verilog hardware
description language. Design Automation Standards Committee, IEEE
Std 1364TM-2005, 2, 2005.
[18] Viraj Athavale, Sam Hertz, Darshan Jetly, Vijay Ganesan, Jim Krysl, and
Shobha Vasudevan. Using static analysis for coverage extraction fromem-
ulation/prototyping platforms. In Proceedings of the eighth IEEE/ACM/I-
FIP international conference on Hardware/software codesign and system
synthesis, pages 207–214. ACM, 2012.
[19] Viraj Athavale, Sai Ma, Samuel Hertz, and Shobha Vasudevan. Code
coverage of assertions using rtl source code analysis. In Proceedings of
the 51st Annual Design Automation Conference, pages 1–6. ACM, 2014.
[21] Mike Benjamin, Daniel Geist, Alan Hartman, Yaron Wolfsthal, Gerard
Mas, and Ralph Smeets. A study in coverage-driven test generation. In
75
Design Automation Conference, 1999. Proceedings. 36th, pages 970–975.
IEEE, 1999.
[22] Jayanta Bhadra, Magdy S Abadir, Li-C Wang, and Sandip Ray. A survey
of hybrid techniques for functional verification. IEEE Design & Test of
Computers, 24(2):0112–122, 2007.
[23] Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. Klee: Unassisted
and automatic generation of high-coverage tests for complex systems pro-
grams. In OSDI, volume 8, pages 209–224, 2008.
[25] Edmund M Clarke, William Klieber, Miloš Nováček, and Paolo Zuliani.
Model checking and the state explosion problem. In LASER Summer
School on Software Engineering, pages 1–30. Springer, 2011.
76
[28] Eman El Mandouh and Amr G Wassal. Covgen: A framework for au-
tomatic extraction of functional coverage models. In Quality Electronic
Design (ISQED), 2016 17th International Symposium on, pages 146–151.
IEEE, 2016.
[30] Monica Farkash-presenter. Mining coverage data for test set coverage
efficiency.
[31] Onur Guzey and Li-C Wang. Coverage-directed test generation through
automatic constraint extraction. In High Level Design Validation and
Test Workshop, 2007. HLVDT 2007. IEEE International, pages 151–
158. IEEE, 2007.
[32] William C Hetzel and Bill Hetzel. The complete guide to software testing.
John Wiley & Sons, Inc., 1991.
77
[34] Richard C Ho, C Han Yang, Mark A Horowitz, and David L Dill. Archi-
tecture validation for processors. ACM SIGARCH Computer Architec-
ture News, 23(2):404–413, 1995.
[35] Yatin Vasant Hoskote, Dinos Moundanos, and Jacob A Abraham. Auto-
matic extraction of the control flow machine and application to evaluating
coverage of verification vectors. In Computer Design: VLSI in Computers
and Processors, 1995. ICCD’95. Proceedings., 1995 IEEE International
Conference on, pages 532–537. IEEE, 1995.
[36] Jing-Yang Jou and C Liu. Coverage analysis techniques for hdl design
validation. Proc. Asia Pacific CHip Design Languages, pages 48–55,
1999.
[37] Jian Kang, Sharad C Seth, and Vijay Gangaram. Efficient rtl coverage
metric for functional test selection. In VLSI Test Symposium, 2007. 25th
IEEE, pages 318–324. IEEE, 2007.
[38] Chien-Nan Jimmy Liu and Jing-Yang Jou. An efficient functional cover-
age test for hdl descriptions at rtl. In Computer Design, 1999.(ICCD’99)
International Conference on, pages 325–327. IEEE, 1999.
[39] Chien-Nan Jimmy Liu and Jing-Yang Jou. An automatic controller ex-
tractor for hdl descriptions at the rtl. IEEE Design & Test of Computers,
17(3):72–77, 2000.
78
[40] Joan C Miller and Clifford J Maloney. Systematic mistake analysis of
digital computer programs. Communications of the ACM, 6(2):58–63,
1963.
[41] Dinos Moundanos, Jacob A Abraham, and Yatin Vasant Hoskote. Ab-
straction techniques for validation coverage analysis and test generation.
IEEE Transactions on Computers, 47(1):2–14, 1998.
[43] Jaan Raik, Uljana Reinsalu, Raimund Ubar, Maksim Jenihhin, and Peeter
Ellervee. Code coverage analysis using high-level decision diagrams. In
Design and Diagnostics of Electronic Circuits and Systems, 2008. DDECS
2008. 11th IEEE Workshop on, pages 1–6. IEEE, 2008.
[44] Sébastien Regimbal, J-F Lemire, Yvon Savaria, Guy Bois, El Mostapha
Aboulhamid, and A Baron. Automating functional coverage analysis
based on an executable specification. In System-on-Chip for Real-Time
Applications, 2003. Proceedings. The 3rd IEEE International Workshop
on, pages 228–234. IEEE, 2003.
79
[46] Jian Shen and Jacob A Abraham. An rtl abstraction technique for pro-
cessor microarchitecture validation and test generation. Journal of Elec-
tronic Testing, 16(1):67–81, 2000.
[47] David Sheridan, Lingyi Liu, Hyungsul Kim, and Shobha Vasudevan. A
coverage guided mining approach for automatic generation of succinct
assertions. In Proceedings of the 2014 27th International Conference
on VLSI Design and 2014 13th International Conference on Embedded
Systems, pages 68–73. IEEE Computer Society, 2014.
[48] Kanna Shimizu and David L Dill. Deriving a simulation input generator
and a coverage metric from a formal specification. In Design Automation
Conference, 2002. Proceedings. 39th, pages 801–806. IEEE, 2002.
[49] Vigyan Singhal and Prashant Aggarwal. Using coverage to deploy for-
mal verification in a simulation world. In International Conference on
Computer Aided Verification, pages 44–49. Springer, 2011.
[51] Rob Sumners, Jayanta Bhadra, and Jacob Abraham. Automatic vali-
dation test generation using extracted control models. In VLSI Design,
2000. Thirteenth International Conference on, pages 312–317. IEEE,
2000.
80
[52] Shinya Takamaeda-Yamazaki. Pyverilog: A python-based hardware de-
sign processing toolkit for verilog hdl. In International Symposium on
Applied Reconfigurable Computing, pages 451–460. Springer, 2015.
[53] Serdar Tasiran and Kurt Keutzer. Coverage metrics for functional valida-
tion of hardware designs. IEEE Design & Test of Computers, 18(4):36–
45, 2001.
[54] Shireesh Verma, Ian G Harris, and Kiran Ramineni. Automatic genera-
tion of functional coverage models from ctl. In High Level Design Valida-
tion and Test Workshop, 2007. HLVDT 2007. IEEE International, pages
159–164. IEEE, 2007.
[56] Andrew Waterman, Yunsup Lee, David A Patterson, and Krste Asanovi.
The risc-v instruction set manual. volume 1: User-level isa, version 2.0.
Technical report, DTIC Document, 2014.
81