A SYSTEMATIC APPROACH TO SYNTHESIS OF VERIFICATION TEST-

SUITES FOR MODULAR SOC DESIGNS

Sudhakar Surendran Rubin Parekhji R. Govindarajan
[email protected] [email protected] [email protected]
Camera Phone Business Unit SoC Design Technology Super Computer Edn. And
Texas Instruments India Ltd Center Res. Center
Bangalore 560 093, India Texas Instruments India Ltd Indian Institute of Science
Bangalore 560 093, India Bangalore 560 012, India

ABSTRACT modules, etc., which are time-consuming and error

Verification is one of the important stages in prone.
designing an SoC (System on Chips) that consumes In our work, we present a methodology that
upto 70% of the design time. In this work, we enables automatic generation of test-cases from re-
present a methodology to automatically generate usable components and with minimum inputs. The
verification test-cases to verify a class of SoCs and main contributions are: (i) identification of
also enable re-use of verification resources created abstractions and structures in creation of test-cases,
from one SoC to another. A prototype (ii) identification of components of a test-case that
implementation for generating the test-cases is also are re-usable, (iii) development of a framework that
presented. enables capturing these components at module
level, independent of the SoC and its variations and
I. INTRODUCTION (iv) development of another framework to generate
System on Chips (SoCs) are complex designs non-re-usable components of a test-case using
with heterogeneous modules (CPU, memory, etc.) properties captured at module and SoC levels. As
integrated in them. Verification is one of the an illustration of the methodology, a prototype
important stages in designing an SoC. It is the implementation for generating two specific classes
process of checking whether the transformation from of test-cases, one for verifying memory modules and
architectural specification to design implementation another for verifying data transfer modules is also
is correct. Verification involves creating the following presented.
resources: (i) a test-plan that identifies the The paper is organized into the following
conditions to be verified, (ii) a test-case that sections: Section II discusses the background
generates the stimuli to verify the conditions required to understand the verification process and
identified, and (iii) a test-bench that applies the reviews the related work from the literature. Section
stimuli and monitors the output from the design. III proposes the methodology to synthesize the test-
Test-cases can be created in a programming cases. Sections IV and V discuss the application of
language like `C’ [1] or in a hardware verification the methodology to generate test-cases for
language like Specman’s `e’ [2]. In our work, we verification of memory and data transfer modules.
focus on programming language based test-cases. Section VI discusses the prototype implementation
Also, we focus on the class of SoCs that are used of our methodology. Section VII presents a
for digital signal processing (DSP). quantitative analysis of generating test-cases using
Verification is a time-consuming effort taking our methodology. Section VIII concludes the paper.
about 28% [3] to 70% [4] of the total time spent in
designing an SoC. The complex and largely manual II. BACKGROUND AND RELATED WORK
nature of verification makes it a time-consuming An SoC is designed in the following stages:
task. As the number and the complexity of modules architecture definition, design implementation,
used in the SoC increases, the time spent in functional verification, design for test and physical
creating the test-cases also increases. Hence, the design. Of these stages, functional verification is
test-cases are often built from a library of re-usable one of the important and time-consuming stages and
components [5]. Yet a significant amount of effort is is the main focus of this work. The term verification
spent in creating the test-cases manually using refers to functional verification in this paper.
these components. Currently the automatic test- SoCs that are targeted for DSP are often
case generators are either restricted to generation of implemented with an embedded DSP processor.
simple test-cases or complex test-case generators These SoCs are capable of reading data from input
[6] [7] which require detailed inputs on the sources, processing them using DSP algorithms and
configuration of the SoC, the interaction between writing the processed data to output sinks. This

978-1-4244-2596-9/08/$25.00 ©2008 IEEE 91

requirement is met by using a central processing by capturing the types of modules, their properties
unit (CPU) usually a DSP, a set of (data transfer) and configuration in addition to other information.
modules capable of transferring data in and out of The specific scenarios for which the test-case must
the SoC, memories, control modules to control be created are also captured. X-Gen then
system resources (like clock, power, etc.), and generates the test-case using the model of the SoC
modules like direct memory access (DMA) controller and the test scenario. X-Gen has limited capability
to transfer data between the memory and the data to generate expected results for the test-case
transfer modules. created.
Figure 1 shows a generic representation of the Cheng et al. [6] have proposed a methodology
SoC. Here SYS_CTRL_A, SYS_CTRL_B and SALVEM (Software Application Level Verification
EXT_MEM_CTRL are system control modules, Methodology) that generates test-cases composed
INT_CTRL is an interrupt control module, Memory_A of code snippets. The code snippets initiate and
and Ext_Mem are memories, PERI_A and PERI_B exercise specific operations on an SoC and are
are data transfer modules and TB_A and TB_B are associated with a specific module. In addition to the
the test-benches for PERI_A and PERI_B code snippets, the dependency between them and
respectively. The test-benches shown are not a part the constraints to generate the parameters for them
of the SoC but are present in the verification are also captured. Using these as inputs, SALVEM
environment of the SOC. The control path indicates generates the test-case.
the modules controlled by the system control module In addition to the above, several forms of test-
e.g., SYS_CTRL_A controls PERI_A, PERI_B, case automation have been known to exist in
SYS_CTRL_B and Memory_A. individual design teams. An illustration is a basic
SoC verification focuses on checking the stage test-case generator that uses SoC and
integration of the modules used. The verification of module configuration captured in a machine-
individual modules is done independently [8, 9] readable format.
before they are integrated. Though the modules are To summarize, the existing approaches for
used in the SoC, the test-cases and their automating test-case generation for SoCs are SoC-
components are not re-used. Also, there is no re- centric and not module-centric, thus, non-re-useable
use from one SoC to another though they use the from module level to SoC level. The work presented
same modules. The non-re-usability is due to the in this paper towards the automatic generation of
differences in SoC configuration like memory map, verification test-cases involves identifying the
DMA event to module event mapping, etc. components of the test-case, the functionality and
properties of the modules that are independent of
the SoC and can be captured at the module level,
thereby allowing re-use from module level to SoC
level verification and also from one SoC to another.
The ability to re-use module resources at SoC level
is a significant difference compared to previous
approaches. The methodology proposed is generic
and can be applied for a wider set of test-cases and
Figure 1: Generic SoC configuration SoCs by identifying the appropriate components and
properties that must be captured.
Integration verification checks whether the SoC
formed by integrating these modules operates as
III. METHODOLOGY TO GENERATE TEST-
per specification. Integration verification happens in
CASES
the following stages: basic (checks read and write
Figure 2 highlights the focus of our work on SoC
access to all the modules in the SoC), integration
test-case generation. The contribution of this work
(checks if the module’s signals are connected as per
is to automate the test-case generation for the
specification), functionality (checks if the modules
highlighted modules. Further, the focus of our
integrated into the SoC perform the functions they
verification is on integration, functionality and
are expected to perform), performance (checks if the
application checks.
performance or bandwidth requirements of the SoC
Figure 3 lists the steps followed in our
are met) and application (checks if the SoC is
methodology. First, we analyze the test-cases and
capable of running the end application).
identify A, the steps involved in the execution of the
Emek et al. [6] have proposed a model based
test-case. We then identify B, the components of a
test-case generator X-Gen designed for SoC
test-case. The components are code segments of
verification. X-Gen captures the model of the SoC
the test-case that perform an atomic (self-contained)

92
operation, e.g., code segment to configure the DMA identified the following parameters for the interface:
module. DMA event number, base address of the source or
destination module, offset of the memory element in
the module, width of the memory element, depth of
the memory element, addressing mode for the
memory element (FIFO or normal RAM) and
endianness. (This is an illustration for DMA module.
Similar set of parameters have been identified for
other modules as well).
Figure 2: Focus on verification in our work Identifying B and its parameters is a medium
complex task (based on author’s experience) and
B consists of two parts: B1: Collection of re- requires a good understanding of the class of test-
usable components of the test-case. These case that is intended to be automatically generated.
components can be captured at module level, (as We then define C, the abstract test-case to
functions or macros), and can be coded once for the capture the core functionality performed in the test-
modules during module level verification. They can case. It is used to generate the actual test-case.
then be re-used at SoC level and also from one SoC Next, we identify D, the properties that are captured
to another. as meta-files, and are required to synthesize: (i) the
B2: SoC or test-case specific non-re-usable parameters for component B1 and (ii) the non-re-
components, i.e., components whose contents and usable test-case component B2. The properties
existence depend on the SoC configuration and the capture information like the relationship between
test-case. modules, parameters used by the functions, etc. D
consists of two parts:
D1: Collection of module level properties.
These properties capture the specific characteristics
of the module that are required to synthesize the
parameters of B1 and contents of B2. Similar to B1,
D1 can be created during module level verification
and re-used thereafter.
D2: Collection of SoC level properties. These
properties capture the specific characteristics of the
SoC that differ from one SoC to another and are
required to synthesize the parameters of B1 and
contents of B2. Though D2 changes with the SoC
configuration D2 is structured such that sections of
D2 can either be re-used from other SoC or
Figure 3: Steps followed in our methodology generated from the architecture specification. The
extent of re-use depends on factors like software
In our work, we have structured B2 such that the compatibility, module or sub-system re-use and
components therein can be generated automatically. others. Examples for D2 are base address of
B2 is made up of B1 components and other control modules and DMA event mapping.
logic code segments that are specific to the SoC or The generation of D is straight-forward and
the test-case. For example, in the code segment for some sections of D can even be made as a part of
an interrupt service routine, the code segment that the SoC and module architecture specification.
checks if an interrupt has been triggered can be The last step in our methodology is defining the
captured at the module level. But the interrupt synthesizer (E) that generates the actual test-case
service routine itself is non-re-usable since the (F) using inputs C, D1 and D2 to drive the test-case
interrupt connection can change from one SoC to generation process, using components B1 and B2 in
another. the test-case themselves. Figure 4 presents a high-
The next step is to identify the parameters for level overview of the synthesizer and also shows the
B1. Care is taken to identify the parameters such inputs used and the output generated. E internally
that they are re-usable and can be generated easily generates B2 and then F (that uses B1 and B2)
with minimum information. For example, to make using the algorithm which has been identified (which
the component that configures the DMA module re- is based on A). The generated test-case F will
usable, it should be made independent of any contain calls to the B1 components captured at the
specific DMA architecture. To do this, we have module level.

93
 system control modules in the correct order. The
steps used in the synthesizer to generate the test-
case are shown in Figure 5.
V. DATA TRANSFER MODULE TEST-CASE
GENERATION
We analyzed the data transfer module test-
cases and identified that the following modules are
active: CPU, DMA control module, interrupt control
module, system control modules and data transfer
modules. We consider the test-bench in the
verification environment as a data transfer module
Figure 4: High level overview of the synthesizer for identifying the components and properties that
must be captured.
IV. MEMORY MODULE TEST-CASE
GENERATION
We analyzed the memory module test-cases
and identified the following modules to be active in
the test-cases: CPU, system control and memory
modules. The component of the test-case that
performs the configuration operation can be made
re-usable and must be captured at the module level.
In addition to this, the component that executes the
algorithm to verify the memory module can be made
re-usable and hence must be captured as a module Figure 5: Steps to generate memory module test-case
level component.
Figure 6 shows the components of the data
The system control configuration component
transfer test-case that we identified to be captured at
takes the base address of the module and the
module level. The parameter interface for these
properties of the module being controlled as
components abstract out the SoC specific and test-
parameters so that they are re-usable. Similarly, the
case specific details so that they are re-usable.
memory verification algorithm components take the
The components of system control module take
properties of the memory module and the module on
the base address of the module and the properties
which the algorithm must be executed as inputs.
of the module being controlled as parameters. The
The abstract test-case captures the memory
components that operate on interrupt events take
modules that must be verified, the memory
the properties of the event as inputs like the event
verification algorithms to be used to verify them and
number, source event or destination event, etc. The
the module on which the algorithms must be run.
component that configures the DMA module takes
The module level properties are captured for
as input the parameters discussed in Section III.
memory modules, system control modules, and the
memory verification algorithm only. The properties of
other modules are not captured since they are not
needed to generate parameters or code segments in
our methodology.
We propose to capture the following properties
at the module level: (i) properties of the memory
modules e.g. number of rows and (ii) constraints to
generate parameters.
We propose to capture the following properties
at the SoC level: (i) base address of the modules, (ii)
memory banking details and (iii) the relationship
between the system control module and other
modules. Figure 6: Components captured at module level
The following code segments of the test-case
are SoC specific or test-case specific and hence are The components of the data transfer module
generated internal to the synthesizer: (i) the take the following parameters as inputs: (i) the base
parameter of the components captured at module address of the module, (ii) the mode of operation of
level and (ii) the code sequence to set up all the the module and (iii) the values required for data

94
transfer e.g., the address to which the transfer must properties of the modules and the SoC are captured
be made. as ASCII files. The abstract test-case is also
The abstract test-case captures (i) the data flow captured as an ASCII file. Perl [10] language is
in the test-case and (ii) the memory modules to be used to implement the test-case synthesizer.
used as source or destination buffers for the data Figure 8 shows the major blocks in our
transfers in the test-case. implementation of the test-case synthesizer (TS).
The module level properties are captured for The TS consists of two major blocks namely, the
data transfer modules and interrupt control modules subroutine extractor and the synthesizer. The
only. We propose to capture the following properties subroutine extractor processes the module level
of the data transfer module at the module level: (i) meta-files and extracts the parameter generating
event properties, (ii) modes of operation supported subroutines, (that capture the constraints to
and (iii) constraints to generate the parameters. For generate the parameters). These subroutines are
the interrupt control module we propose to capture then added to the synthesizer’s subroutine library.
the mapping of the input interrupt to output interrupt The synthesizer consists of four major blocks:
as a property. the meta-file processor, the abstract test-case
We propose to capture the following as processor, the data structure generator and the code
properties at SoC level: (i) base address of all the generator. Each block generates one or more data
peripherals in the SoC, (ii) interrupt mapping for the structures that are used by the subsequent blocks.
interrupt controllers (iii) DMA event mapping for the
DMA controllers, (iv) the relation between system
control module and other modules and (v) the
modes supported by the modules at the SoC level.
The following components of the SoC are test-
case specific or SoC specific and hence are
automatically generated: the parameters of the
components captured, the code segment to set up
the system control modules in the required order,
the code segment to configure the peripherals that
handle interrupts, the interrupt service routine code
segment and the code segment that handles data
transfer. Figure 8: Blocks in test-case synthesizer implementation
The data transfer module test-case synthesizer
generates the test-case using the steps shown in VII. QUANTITATIVE ANALYSIS
Figure 7. Figure 9 shows the process involved in creating
and executing a test-case manually. Of these steps,
significant amount of time is spent on Steps 2, 3, 4
and 8. Step 2 consumes six to twelve person-
months (for a representative class of designs based
on the SoC verification projects that the author has
worked on). But this effort has to be spent once and
can be re-used across many SoC projects with
minor SoC-specific modifications, the effort for which
can be within one to two person-weeks.
The effort spent in Steps 3, 4 and 8 depends on
the module functionality and the complexity of the
SoC in which the modules are integrated. Also,
these steps are specific to the SoC and the module
they are targeted. On an average, it takes one to
four person-months for Steps 3, 4, and 8 together for
Figure 7: Steps to generate data transfer module test-case generating the test-cases manually per module.
This effort varies based on (i) the complexity of the
VI. IMPLEMENTATION DETAILS SoC, (ii) the number of modules, (iii) the complexity
We have implemented a prototype of the of the modules, and other such factors.
methodology proposed. In our implementation we With the above steps as reference, our
generate ‘C’ language test-cases. The components methodology for automatic test-case generation
of the modules are captured in ‘C’ libraries and, the impacts Steps 2, 3, 4, and 8. The author spent

95
approximately five weeks in developing the REFERENCES
prototype (discussed in Section VII) to illustrate the 1. Brian W. Kernighan and Dennis Ritchie, “C Programming
benefits of the proposed methodology. The Language”, Pearson Education, second edition, March 1989.
production system that can be used in SoC 2. Cadence, “e Language Reference”, 2005.
3. United Business Media EETimes online. 2005 Electronic
verification projects will be much more complex than
Design Automation Branding Study - Chip Design.
the prototype and is estimated to take about six http://i.cmpnet.com/eetimes/eda/edasurvey chip.pdf , 2005.
person-months. But this is a one-time effort and can 4. 2002 IC/ASIC Functional Verification Study. Technical report,
be re-used across many SoC verification projects Collett International Research, 2002.
(similar to Step 2). 5. F. Hunsinger, S. Francois, and A.A. Jerraya, “Definition of a
The effort spent in Steps 3 and 4 for automatic systematic method for the generation of software test
generation is five person-days. It is estimated that a programs allowing the functional verification of system on chip
(soc)”, 4th International Workshop on Microprocessor Test
similar effort is required for Step 8. Thus the total
and Verification: Common Challenges and Solutions, pp 11–
effort required for automatic generation is ten 16, May 2003.
person-days per module. This is significantly lower 6. R. Emek, I. Jaeger, Y. Naveh, G. Bergman, G. Aloni, Y. Katz,
when compared to one to four person-months per M. Farkash, I. Dozoretz, and A. Goldin, “X-gen: a random
module spent on manual generation. The benefits test-case generator for systems and socs”, Seventh IEEE
accrue further if the components and the properties International High-Level Design Validation and Test
of the modules are re-used (across multiple SoCs) Workshop, pp. 145–150, October 2002.
and the properties of the SoC are captured as a part 7. A. Cheng, C. Lim, and A. Parashkevov, “A software test
program generator for verifying system-on-chips”, Tenth IEEE
of the SoC specification. International High-Level Design Validation and Test
Workshop, pp. 79–86, December 2005.
8. K. Albin, “Nuts and bolts of core and soc verification”, 38th
Design Automation Conference, pp. 249–252, 2001.
9. W. Yang, M. Chung, and C. Kyung, “Current status and
challenges of soc verification for embedded systems market”,
IEEE International [Systems-on-Chip] SOC Conference, pp.
213–216, September 2003.
10. Larry Wall, Jon Orwant, and Tom Christiansen, “Programming
Perl”, O’Reilly Media Incorporated, third edition, July 2000.

Figure 9: Process involved in creating and executing a test-case

VIII. CONCLUSIONS
In this work, we have presented a systematic
approach to synthesize verification test-cases. The
approach is based on identification of re-usable
components and properties that can be captured at
the module-level and the SoC-level. As an
illustration the methodology has been implemented
to generate memory and data transfer test-cases.
The methodology can be extended and applied to
other categories of test-cases too. The framework
presented in this work for the automatic generation
of test-cases further helps to target more efficient
and more comprehensive verification of SoC
designs.

96