VCS Training

8 Coverage Metrics

Learning Objectives

Upon completion of this exercise you will be able to:

• Include coverage metrics in your VCS simulations
• Interpret the coverage metrics results and merge results from
multiple simulations
• Use the auto-grading feature of VCS with Coverage Metrics to
find a subset of test vectors that meet a user-defined coverage goal.

30 minutes

UNIT Unit 8

Coverage Metrics 8-1

 Part I- FSM Coverage
1. Go to the subdirectory lab8

What other types of coverage are available with VCS Coverage Metrics? __________
_____________________________________________________________________

What are the vcs & simv options to invoke each type of coverage? _______________
_____________________________________________________________________

What option would you use to specify line and condition coverage only?__________

Let’s take a closer look at FSM (Finite State Machine) coverage.

1. Compile so that FSM coverage is enabled.

VCS COMMAND LINE:_____________________________________________

2. Rerun the simulation with FSM coverage.

SIMV COMMAND LINE:____________________________________________

3. Invoke cmView with FSM coverage.

CMVIEW COMMAND LINE:_______________________________________

4. In the main cmView window, left click on the FSM Coverage icon.
5. Left click on Module List to display all modules where a finite state machine was
found. You should see that the CPU module has one state machine.
6. Double click left on the CPU module (under the List of all modules). This will bring
up a window with coverage details for the FSM in the CPU module. Double click the
Transitions tab you will see a matrix of transitions. A green check indicates the
transition was covered during your simulation. A red X indicates that the transition

Coverage Metrics 8-2

VCS Training

was not covered. An empty (gray) box indicates that VCS found that transition was
not possible.
Which state transitions were not covered?________________________________
__________________________________________________________________

7. Click on the States tab. This displays each possible state in the FSM and whether it
was covered. There are five states in this FSM and each one was covered at least once
during the simulation run.
8. Click on the Reachability tab. This shows information about transitions between two
states through intervening states. Lets look at an example…
9. Click on the yellow box that represents the transition from MEMORY to FETCH.
The 1 / 2 displayed in this box indicates that there are two possible transitions from
MEMORY to FETCH and that only one was covered. After clicking this transition
you will see that VCS has determined two possible transitions:
MEMORY -> WRITE_BACK -> FETCH (covered)
MEMORY -> FETCH (not covered)
10. Exit the cmView GUI

 Part II - Merging Coverage Results and Auto-Grading

So far we have been writing coverage results into default filenames under the
simv.cm directory. For example…

simv.cm/coverage/verilog/test.line
simv.cm/coverage/verilog/test.fsm
This is fine as long as there is only one test vector to run. But what if there are
multiple test vectors? You don’t want to overwrite the default filenames with each
new test vector that is simulated. It is more useful save the coverage results for each
simulation with a new vector and then merge the results . In this section we will
examine how to do this.

1. First clean up the directory you are working in. Make sure you have exited the
cmView GUI and then run the clean script.
2. The directory CODE contains five different tests. Copy the first test into the run
directory
cp CODE/text1 text_segment

Coverage Metrics 8- 3
 cp CODE/data1 data_segment
3. Run a vcs compile and enable all types of coverage (see “run_all” script)
VCS COMMAND LINE_____________________________________________

4. Examine the contents of simv.cm/coverage. It should be empty at this point.

5. Run simv with all coverage types enabled. Be sure to include the option to rename the
coverage metrics report files. Use test1 as the filename root for the report files.
SIMV COMMAND LINE____________________________________________

6. One by one, copy the other vectors into the run directory and rerun the simulation
with all coverage types enabled. For each simulation run, be sure to rename the
coverage report files. (A script called run_all is provided.)
What file contains condition coverage results for test5?_____________________
What file contains line coverage results for test2?__________________________

7. We will now merge the data together. At the command line type:
vcs –cm_pp –cm_dir test1 –cm_dir test2 –cm_dir test3 –cm_dir test4 –cm_dir test5
–cm_name merged

This will create a merging of all the 5 tests into a merged database and the merged
coverage report will be stored in "simv.cm/reports" with the filename prefix "merged"
(i.e. simv.cm/reports/merged.long_l will contain the merged coverage results from the 5
tests).

8. You can also observe these data and do test grading through GUI. Invoke the cmView
GUI. Type:
vcs –cm_pp gui –cm line+tgl+cond+fsm –cm_dir test1 –cm_name test1

Go to the “File” on the menu bar and drop down to “Open Coverage Statement”. This
will open the hierarchy and coverage window. On the coverage menu bar select “add”,
this opens the window that will allow you to add the line coverage data for grading. In
the filter window or thru selecting, set the path for ./test2/*.line select test2.line and add
it. You can do the same steps to add other tests ( test3 to test5). Close the window after
adding the line coverage. You can now compare tests using the compare menu.

9. We will use cmView’s auto-grading feature to determine a subset of test vectors that
meet a user-defined coverage goal. From the main cmView window select

Coverage Metrics 8-4

VCS Training

Tools -> AutoGrading

This brings up the cmAutoGrade window.
10. Click any one of the three tabs for Statement, Condition, or Toggle coverage to view
the data for each test listed in the Test Name column.

The four kinds of coverage data are described as follows:

Incremental data is the amount of additional coverage provided by a given test

over the previous test.
Difference data is the difference in coverage between a given test and the
accumulated total up to the previous total.
Test coverage data is the coverage for a given test. That is, the data reflects the
total coverage for that test only.
Accumulated data is the cumulative result of all tests added in the current session
up to that point in the order of tests.
11. Enter 65 desired test coverage goal and select Line Coverage. Then click Run
AutoGrade The Test Grading window appears.
What subset of test vectors are required to obtain 65% line coverage? _____________
_____________________________________________________________________

12. Exit the cmView GUI.

Congratulations. You’re done with the Coverage Metrics lab!

Coverage Metrics 8- 5