DO Qualification Kit

Simulink® Design Verifier™

Tool Qualification Plan

R2020b

June 16, 2020 qualkitdo_sldv_tqp

How to Contact MathWorks
Latest news: www.mathworks.com
Sales and services: www.mathworks.com/sales_and_services
User community: www.mathworks.com/matlabcentral
Technical support: www.mathworks.com/support/contact_us
Phone: 508-647-7000

The MathWorks, Inc.

1 Apple Hill Drive
Natick, MA 01760-2098
DO Qualification Kit Simulink® Design Verifier™ Tool Qualification Plan
© COPYRIGHT 2017-2020 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used or
copied only under the terms of the license agreement. No part of this manual may be photocopied or reproduced
in any form without prior written consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or
through the federal government of the United States. By accepting delivery of the Program or Documentation, the
government hereby agrees that this software or documentation qualifies as commercial computer software or
commercial computer software documentation as such terms are used or defined in FAR 12.212, DFARS Part
227.72, and DFARS 252.227-7014. Accordingly, the terms and conditions of this Agreement and only those rights
specified in this Agreement, shall pertain to and govern the use, modification, reproduction, release, performance,
display, and disclosure of the Program and Documentation by the federal government (or other entity acquiring for
or through the federal government) and shall supersede any conflicting contractual terms or conditions. If this
License fails to meet the government’s needs or is inconsistent in any respect with federal procurement law, the
government agrees to return the Program and Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See
www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be
trademarks or registered trademarks of their respective holders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents
for more information.

June 16, 2020 qualkitdo_sldv_tqp

Revision History
March 2017 New for DO Qualification Kit Version 3.3 (Applies to Release 2017a)
September 2017 Revised for DO Qualification Kit Version 3.4 (Applies to Release 2017b)
March 2018 Revised for DO Qualification Kit Version 3.5 (Applies to Release 2018a)
September 2018 Revised for DO Qualification Kit Version 3.6 (Applies to Release 2018b)
March 2019 Revised for DO Qualification Kit Version 3.7 (Applies to Release 2019a)
September 2019 Revised for DO Qualification Kit Version 3.8 (Applies to Release 2019b)
March 2020 Revised for DO Qualification Kit Version 3.9 (Applies to Release 2020a)
September 2020 Revised for DO Qualification Kit Version 3.10 (Applies to Release 2020b)

June 16, 2020 qualkitdo_sldv_tqp

Contents
1 Introduction ................................................................................................................................................ 1-1
2 Tool Overview and Identification ............................................................................................................... 2-1
2.1 Simulink Design Verifier Product Description ..................................................................................... 2-1
2.2 Simulink Design Verifier Product Identification .................................................................................. 2-1
3 Certification Considerations ....................................................................................................................... 3-1
3.1 Requirement for Qualification ............................................................................................................ 3-1
3.2 Certification Credit ............................................................................................................................. 3-2
3.3 Formal Methods Consideration .......................................................................................................... 3-4
4 Tool Development Life Cycle – Tool User ................................................................................................... 4-1
4.1 Tool Artifacts ...................................................................................................................................... 4-1
4.2 Planning .............................................................................................................................................. 4-1
4.3 Tool Operational Requirements ......................................................................................................... 4-1
4.4 Integration .......................................................................................................................................... 4-2
4.5 Verification ......................................................................................................................................... 4-2
4.6 Configuration Management ............................................................................................................... 4-2
4.7 Quality Assurance ............................................................................................................................... 4-2
4.8 Qualification Liaisons .......................................................................................................................... 4-2
5 Additional Considerations .......................................................................................................................... 5-1
5.1 Customer Bug Reporting Considerations ........................................................................................... 5-1
5.2 Tool Validation After Installing Updates ............................................................................................. 5-1
6 Tool Life Cycle Data .................................................................................................................................... 6-1
7 Schedule ..................................................................................................................................................... 7-1

June 16, 2020 qualkitdo_sldv_tqp i

1 Introduction

This document comprises the Tool Qualification Plan (reference DO-330 Section 10.1.2) for the design error
detection capability of Simulink Design Verifier. It is intended for use in the DO-178C and DO-330 tool
qualification process for Criteria 3 TQL-5 tools.

As you review this document, notice the use of <Insert Information> to indicate where you should
customize the document.

June 16, 2020 qualkitdo_sldv_tqp 1-1

2 Tool Overview and Identification

2.1 Simulink Design Verifier Product Description

The Simulink Design Verifier uses formal methods to identify hidden design errors in models. It detects
blocks in the model that result in integer overflow, dead logic, array access violations, and division by zero.
It can formally verify that the design meets functional requirements. For each design error or requirements
violation, it generates a simulation test case for debugging.

Simulink Design Verifier generates test cases for model coverage and custom objectives to extend existing
requirements-based test cases. These test cases drive your model to satisfy condition, decision, modified
condition/decision (MCDC), and custom coverage objectives. In addition to coverage objectives, you can
specify custom test objectives to automatically generate requirements-based test cases.

2.2 Simulink Design Verifier Product Identification

Software Tool Version (Release) Tool Vendor
Simulink Design Verifier Version 4.4 (R2020b) The MathWorks, Inc.
1 Apple Hill Drive
DO Qualification Kit Version 3.10 (R2020b) Natick, MA, 01760-2098 USA

June 16, 2020 qualkitdo_sldv_tqp 2-1

3 Certification Considerations

This section provides certification considerations for the Design Error Detection capabilities of the Simulink
Design Verifier tool.

3.1 Requirement for Qualification

To determine whether a tool must be qualified, you must answer the following questions. If you answer yes
to all three questions, you must qualify the tool.

Question Answer
Can the tool insert an error into the airborne software or fail to detect an existing error in the Yes1
software within the scope of its intended usage?

Will the output of the tool not be verified as specified in Section 6 of DO-178C, DO-278A, DO- Yes
331, DO-332 or DO-333?

Are processes of DO-178C, DO-278A, DO-331, DO-332 or DO-333 eliminated, reduced, or Yes
automated by the use of the tool? Will you use output from the tool to meet an objective or
replace an objective of DO-178C, DO-278A, DO-331, DO-332 or DO-333, Annex A or Annex C?

Given that the answer to all the preceding questions is yes, the Simulink Design Verifier design error
detection must be qualified.

To determine the qualification type (Criteria 1, Criteria 2, or Criteria 3), you must answer the following
questions about the tool.

Question Answer
Is the tool output part of the airborne software, such that the output can insert an error into No
the software?

Could the tool fail to detect an error in the airborne software and is the tool also used to No
justify the elimination or reduction of either of the following:
• Verification processes other than that automated by the tool.
• Development processes that could have an impact on the airborne software.
Could the tool fail to detect an error in the airborne software? Yes

Because the answer to the preceding first and second questions are no, Simulink Design Verifier can be
qualified as a Criteria 3 tool. The tool qualification level will therefore be TQL-5.

1
Simulink Design Verifier might fail to detect an error.

June 16, 2020 qualkitdo_sldv_tqp 3-1

3.2 Certification Credit
The following table shows the certification credit (see DO-331 and DO-333 Annex A or Annex C Objectives),
being sought for the Simulink Design Verifier design error detection capabilities. DO-331 references are
prefaced with MB for the table and section numbers. DO-333 references are prefaced with FM.

Table 1 Simulink Design Verifier Design Error Detection with Respect to DO-331
and DO-333 Objectives

DO-331 DO-333
Objective Table Reference Table Reference Credit Taken

(2) High-level MB.A-3 MB.6.3.1.b FM.A-3 FM.6.3.b Partial - Simulink Design

requirements are MB.C-3 FM.C-3 FM.6.3.c Verifier can be used to detect
accurate and FM.6.3.1.b certain design errors in the
consistent high-level requirements
expressed by Simulink models

(4) High-level MB.A-3 MB.6.3.1.d FM.A-3 FM.6.3.e Partial - Simulink Design

requirements are MB.C-3 FM.C-3 FM.6.3.1.d Verifier can be used to detect
verifiable. unreachable conditions and
decisions in the high-level
requirements expressed by
Simulink models

(7) Algorithms are MB.A-3 MB.6.3.1.g FM.A-3 FM.6.3.h Partial - Simulink Design
accurate. MB.C-3 FM.C-3 FM.6.3.1.g Verifier can be used to detect
certain design errors in the
algorithms expressed by
Simulink models

(2) Low-level MB.A-4 MB.6.3.2.b FM.A-4 FM.6.3.b Partial - Simulink Design

requirements are MB.C-4 FM.C-4 FM.6.3.c Verifier can be used to detect
accurate and FM.6.3.2.b certain design errors in the
consistent. low-level requirements
expressed by Simulink models

(4) Low-level MB.A-4 MB.6.3.2.d FM.A-4 FM.6.3.e Partial - Simulink Design

requirements are MB.C-4 FM.C-4 FM.6.3.2.d Verifier can be used to detect
verifiable. unreachable conditions and
decisions in the low-level
requirements expressed by
Simulink models

(7) Algorithms are MB.A-4 MB.6.3.2.g FM.A-4 FM.6.3.h Partial - Simulink Design
accurate. MB.C-4 FM.C-4 FM.6.3.2.g Verifier can be used to detect
certain design errors in the
algorithms expressed by
Simulink models

June 16, 2020 qualkitdo_sldv_tqp 3-2

 DO-331 DO-333
Objective Table Reference Table Reference Credit Taken

(9) Software MB.A-4 MB.6.3.3.b FM.A-4 FM.6.3.c Partial - Simulink Design

architecture is MB.C-4 FM.C-4 FM.6.3.3.b Verifier can be used to detect
consistent. certain design errors in the
software architecture
expressed by Simulink models

(11) Software MB.A-4 MB.6.3.3.d FM.A-4 FM.6.3.e Partial - Simulink Design

architecture is MB.C-4 FM.C-4 FM.6.3.3.d Verifier can be used to detect
verifiable unreachable conditions and
decisions in the software
architecture expressed by
Simulink models

(A/C-3, FM8) N/A N/A FM.A-3 FM.6.3.6.a Full - accomplished as part of

(A/C-4, FM14) FM.C-3 FM.6.3.6.b the Simulink Design Verifier
Formal analysis cases FM.A-4 qualification
and procedures are FM.C-4
correct.

(A/C-3, FM9) N/A N/A FM.A-3 FM.6.3.6.c Partial - Simulink Design

(A/C-4, FM15) FM.C-3 Verifier performs the analysis
Formal analysis FM.A-4 but the user must explain
results are correct FM.C-4 discrepancies found by the
and discrepancies analysis
explained.

(A/C-3, FM10) N/A N/A FM.A-3 FM.6.3.i Full - accomplished as part of

(A/C-4, FM16) FM.C-3 the Simulink Design Verifier
Requirements FM.A-4 qualification
formalization is FM.C-4
correct.

(A/C-3, FM11) N/A N/A FM.A-3 FM.6.2.1 Full - accomplished as part of

(A/C-4, FM17) FM.C-3 the Simulink Design Verifier
Formal method is FM.A-4 qualification
correctly justified and FM.C-4
appropriate.

June 16, 2020 qualkitdo_sldv_tqp 3-3

3.3 Formal Methods Consideration
Simulink Design Verifier uses Prover Plug-In by Prover® Technology and Polyspace® Code Prover™ engine to
identify design errors in Simulink models. Prover Plug-In uses sound model checking method with
theoretical basis provided in the following studies:

Sheeran, Mary and Gunner Stålmarck. “A Tutorial on Stålmarck's Proof

Procedure for Propositional Logic.” Formal Methods in System Design. 16
doi:10.1023/A:1008725524946. January 2000 : 23–58.

Sheeran, Mary, Satnam Singh, and Gunner Stålmarck. “Checking Safety Properties Using Induction and a
SAT-Solver.” Proceedings of Formal Methods in Computer-Aided Design, Springer-Verlag,
November 2000.

Moskewicz, Matthew W., Conor F. Madigan, Ying Zhao, Lintao Zhang, and Sharad Malik. "Chaff:
Engineering an Efficient SAT Solver." Proceedings of the 38th Annual Design Automation
Conference, ACM, 2001 : 530–535.

McMillan, Kenneth L. "Interpolation and SAT-based Model Checking." In International Conference on

Computer Aided Verification, Springer Berlin Heidelberg, 2003 : 1–13.

Bjesse, Per and Koen Claessen. "SAT-based Veification without State Space Traversal." International
Conference on Formal Methods in Computer-Aided Design, Springer Berlin Heidelberg, 2000 : 409–
426.

Chvátal, Václav. Linear Programming. New York: W. H. Freeman and Company, 1983.

Williams, H. P. Model Building in Mathematical Programming. 3rd edition. Wiley, 1993.

The Polyspace Code Prover engine uses sound abstract interpretation method with theoretical basis
provided in DO Qualification Kit Polyspace Code Prover Theoretical Foundation.

June 16, 2020 qualkitdo_sldv_tqp 3-4

4 Tool Development Life Cycle – Tool
User

4.1 Tool Artifacts

The following table identifies the Simulink Design Verifier artifacts that are available in the Artifacts
Explorer, the Simulink Design Verifier folder. The User’s Guide artifact is in the \doc subfolder.

Artifact File
Simulink Design Verifier Tool Qualification Plan, R2020b (this qualkitdo_sldv_tqp.docx/pdf
document)

Simulink Design Verifier Tool Operational Requirements, R2020b qualkitdo_sldv_tor.docx/pdf

Simulink Design Verifier Test Cases and Procedures, R2020b qualkitdo_sldv_tcp.docx/pdf

Execute Qualification Tests and Review Test Results for Simulink qualkitdo_sldv_run.mlx
Design Verifier, R2020b

Simulink Design Verifier User’s Guide, R2020b sldv.pdf

Simulink Design Verifier Reference, R2020b sldv_ref.pdf

4.2 Planning
The Plan for Software Aspects of Certification (PSAC) or Plan for Software Aspects of Approval (PSAA)
designates that the Simulink Design Verifier design error detection capabilities will be qualified as a Criteria
3, TQL-5 tool.

This document provides the Tool Qualification Plan for the Simulink Design Verifier.

4.3 Tool Operational Requirements

Tool Operational Requirements for the Simulink Design Verifier are in the Simulink Design Verifier Tool
Operational Requirements

User information for the Simulink Design Verifier design error detection capabilities are in:

• Simulink Design Verifier User’s Guide, R2020b

• Simulink Design Verifier Reference, R2020b

June 16, 2020 qualkitdo_sldv_tqp 4-1

 The applicant will:

• Review the Tool Operational Requirements for applicability to the project under consideration.
• Configure the Tool Operational Requirements in a configuration management system.

4.4 Integration
Instructions for installing Simulink Design Verifier are available at the MathWorks Documentation Center,
R2020b > Installation.

The applicant will:

• Install Simulink Design Verifier using the installation instructions

• Create a Tool Installation Report (DO-330, Table T-0 objective 3)

4.5 Verification
Requirements-based test cases and procedures ar developed from the tool operational requirements by
MathWorks and provided to the tool user to execute in the installed environment. The test cases and
procedures will be developed in the form of the Simulink models, MATLAB test scripts and Simulink Report
Generator setup file that exercise the capabilities being qualified in Simulink Design Verifier.

For general information about the testing procedure, see Simulink Design Verifier Test Cases and
Procedures. Executing the test procedure generates results that are published in tool qualification reports;
see artifact Execute Qualification Tests and Review Test Results for Simulink Design Verifier for more
information..

The applicant will:

• Review the test cases and procedures for applicability to the project under consideration.
• Configure the test cases and procedures in a configuration management system.
• Execute the test cases and procedures in the installed environment.
• Review the test results
• Configure the test results in a configuration management system

4.6 Configuration Management

The applicant will implement the project-specific configuration management activities to handle the
shipped and user-generated tool qualification data (DO-330, Table T-8).

4.7 Quality Assurance

The applicant will implement the project-specific quality assurance for the tool qualification activities
performed by the tool users (DO-330, Table T-9).

4.8 Qualification Liaisons

The applicant will implement the project-specific qualification liaisons activities (DO-330, Table T-10).

June 16, 2020 qualkitdo_sldv_tqp 4-2

5 Additional Considerations

5.1 Customer Bug Reporting Considerations

MathWorks reports known critical bugs brought to its attention on its bug report system at
www.mathworks.com/support/bugreports. The bug reports are an integral part of the documentation for
each release.

The bug report system provides an interface for customers to view and submit bug reports. Users can track
the status of open bugs. Users can choose to receive notifications for new or updated bug reports. The bug
reports on this web site include internally and externally nominated bugs. If applicable, bug reports include
provisions for known workarounds or file replacements. Customers can use the bug report mechanism to
nominate bugs. These nominations are processed and evaluated by The MathWorks, Inc. development
organization.

To open the DO Qualification Kit bug report checks, from the Simulink toolstrip:

• Open the DO Qualification Kit app and select Bug reports.

• Open the Modeling tab and select Model Advisor.

The Model Advisor opens. Select and run the check for the bug report you would like to review.

5.2 Tool Validation After Installing Updates

Changing the tool installation can affect tool behavior, requiring the user to revalidate the tool. When the
user installs a release update or a patch to address issues in a bug report, the user must repeat the tool
qualification steps (rerun tests and review results) to verify that the tool satisfies the tool operational
requirements.

June 16, 2020 qualkitdo_sldv_tqp 5-1

6 Tool Life Cycle Data

The following table shows the life cycle data for Simulink Design Verifier. The table maps the documents
and artifacts to DO-330 life cycle data items. For additional information about the documents and artifacts,
contact MathWorks.

In the following table, the Tool User Action column defines the responsibility of the tool user with regard to
making the data available to certification authorities. For data marked “Submit”, the tool-user must deliver
the data to the certification authorities. When marked “Available”, the data must be available at the tool-
user’s or tool vendor’s site for inspection by the certification authorities.

Table 2 Life Cycle Data

Reference and Data Support Documents or Artifacts Tool Note

User
Action
DO-330, section 10.1.1 <Insert reference to the PSAC or PSAA Submit MathWorks provides a template for
Tool-Specific Information in artifact> the PSAC.
Plan for Software Aspects of The tool user can customize the
Certification (PSAC) template as appropriate for their
project.
The information in these sections
can be used to create the artifact:
• Tool Overview and
Identification on page 2-1
• Certification Credit on page 3-2
DO-330, section 10.1.2 <Insert reference to the Simulink Design Submit MathWorks provides the tool
Tool Qualification Plan Verify Tool Qualification Plan (this specific TQP artifact.
document) artifact> The tool user can customize the
artifact as appropriate for their
project.

DO-330, section 10.3.1 <Insert reference to the Simulink Design Available Authored by MathWorks.
Tool Operational Verifier Tool Operational Requirement
Requirements artifact and tool user documentation>

DO-330, section 10.3.3 and Execute Qualification Tests and Review Available MathWorks provides test cases.
10.2.5 Test Results for Simulink Design Verifier The tool user can add or remove
Test Cases and Procedures Simulink Design Verifier Test Cases and test cases as appropriate for the
Procedures tests used in their project.
Test Cases
qualkitdo_sldv_rs.rpt

June 16, 2020 qualkitdo_sldv_tqp 6-1

Reference and Data Support Documents or Artifacts Tool Note
User
Action
DO-330 section 10.3.4 and Summary qualification reports. Available MathWorks provides the tool
10.2.6 qualification reports.
Test Results The tool user provides test results
as appropriate for the tests
executed in their project.

DO-330 section 10.1.16 <Insert reference to Software Submit Provided by tool user.
Tool-specific information in Accomplishment Summary artifact>
Software Accomplishment
Summary (SAS)

DO-330 section 10.1.15 <Insert reference to the Tool Submit Provided by tool user.
Tool Accomplishment Qualification Accomplishment Summary
Summary artifact>

June 16, 2020 qualkitdo_sldv_tqp 6-2

7 Schedule

<Insert tool schedule>

June 16, 2020 qualkitdo_sldv_tqp 7-1