Testing Simulink Models

Fraser Macmillen

© 2015 The MathWorks, Inc.

1
Test Infrastructure
▪ Model set-up desired parameters, variants, operating point, etc.
e.g. test start up script

▪ Model stimulus desired inputs driving the model

e.g. signal builder block, .xlsx, test sequence

▪ Views of behaviour signal traces, read-outs, animations, etc.

e.g. scopes, simulation data inspector

▪ Verification of behaviour desired behaviour is checked

e.g. verification blocks, post-simulation scripts

2
3
 Common challenges:
• Problem: cannot do anything unless a particular script is run first
Solution: use project startup, data dictionaries, models always “ready to go”

• Problem: model is tied to particular means of stimulus (from file, signal builder, etc)
• Solution: use test harnesses + variants

• Problem: changes to the design and test mixed together

• Solution: save test infrastructure externally to your design; separate source control

• Problem: one person’s system is another person’s component

• Solution: model referencing, suitable interfaces

• Problem: performance degraded by infrastructure not needed for “my test”

• Solution: multiple harnesses / variants
4
Simulink Test

5
Why Simulink Test?
Saves you time:
▪ Creating / managing test infrastructure
▪ generating & (re)-running multiple tests
▪ reporting results
▪ a common test environment –
everyone doing things in a consistent manner

Gives you capability:

▪ new ways of authoring test scenarios
▪ easy integration with other tools
(Requirements, Coverage, Test Generation, MATLAB Unit Test, Continuous Integration)

6
7
Simulink Test Overview
1. Test Harnesses 2. Test Sequence Block
Stimulus Integration 3. Test Manager

• Synchronized, simulatable test • Inputs and assessments based on logical, • Author, execute, manage test cases
environment temporal conditions • Review, export, report

Main Model

Component
under test

Test Harness

8
 Agenda
▪ Creating Test Harnesses

▪ Creating Test Cases

▪ Testing against Requirements

▪ Reporting

▪ Coverage analysis

▪ Multi-release regression testing

▪ Continuous integration

9
Test Harnesses

10
11
What if you already have a harness model....

12
13
Common questions...

Do I need a separate harness for each test?

14
Test Harness Release Highlights

R2017a:
▪ Test harness import
▪ Create harnesses for components with physical (Simscape) connections
▪ More control over synchronisation

R2017b:
▪ Harness create/re-build callbacks
▪ Model comparison prior to synchronisation

15
Test Cases

&

Test Stimuli

16
 Create a test case
using the original signal builder

17
18
What have we done so far....

▪ Created and imported test harnesses

▪ Created a test case for running multiple simulations (iterations) with

different scenarios

19
Common questions...

When should I use iterations vs multiple test cases?

20
Use iterations if:
▪ Same model/harness & test type
▪ Same set-up (callbacks)
▪ Usually run together
▪ Relate to same requirements(s)
▪ Can use fast-restart

Use separate test cases if:

▪ Need independent configuration control
▪ Different model/harness/test type or callbacks
▪ Relate to distinct requirements
▪ Distinct control of coverage

21
 Create a test case
using real-world recorded data

22
My data

23
Importing time-stamped data from Excel or text files

% pre-process .xlsx file

% get import options
importOptions = detectImportOptions('SiteWindDataRecorded.xlsx');
% set sheet
importOptions.Sheet = '2011_05_23';
% tell it that Time is in a date-time format
importOptions = setvartype(importOptions,'Time','datetime');
importOptions = setvaropts(importOptions,'Time', 'DatetimeFormat', 'HH:mm:ss.SSS');
% read data in
T = readtable('SiteWindDataRecorded.xlsx',importOptions);
% convert to timetable
TT = table2timetable(T);
% re-sample to 1sec intervals
TTT = retime(TT,'secondly','nearest'); Time WindSpeed WindDirection
0 14.59 214.9
1 15.25 218.2
2 16.46 212.2
3 16.08 207.3 24
25
What have we done so far....

▪ Created and imported test harnesses

▪ Created a test case for multiple simulations (iterations)

▪ Created a test case importing real-world data from Excel using root
import mapping

26
Testing Against Requirements
(Verification)

27
Good quality textural requirements....
# Property Description
1 Correct Requirement has no errors and is not an error

2 Compliant with one or more documented upper level requirements (operational, customer needs, etc)

3 Complete Each requirement covers all aspects of the requirement’s intent.

4 Consistent Is not in conflict with any other requirement. Is consistent with the environment

5 Validated Ensures the requirement will lead to the right design, i.e. reflects fully, correctly and objectively system objectives,
scope, operational use, etc
6 Achievable Can be implemented in a cost-effective manner that considers cost and
schedule constraints
7 Unambiguous The requirement has only one possible interpretation.
Questions are:
Could the requirement be read different ways by different people?
What are the different interpretations of the requirement?
8 Verifiable Expected performance or functionality expressed in a manner that allows
verification to be objective, preferably as a result of an observable, ideally measurable, effect
9 Singular Use a unique “shall” in each textual requirement to express a single design
Demand (unique intent).
10 Positive Negative requirements are very difficult, if not impossible, to verify.
Negative requirement may be used only for safety requirements
11 Adequate Each requirement is expressed as a problem statement
i.e. it defines what is needed, not a solution, except if a particular
implementation is a constraint to be resolved by design and test 28
This model had requirements such as...

These are ambiguous, incomplete, and not clearly verifiable

29
Hopefully a bit better is...

30
 Example 1:
Using verify() to test against a requirement

31
32
Further considerations...

▪ Testing at an appropriate level

i.e. system – sub system – component

▪ Verification of more complex requirements

33
 Example 2:
Using custom criteria to test against a
requirement

34
35
36
Test across multiple operating points?

(trim conditions)

37
38
Incorporating coverage analysis

39
40
Regression & cross-release testing

41
42
Continuous Integration

43
44
Conclusions

45
 Benefits of Simulink Test
▪ Ease of creation, organisation & control of test harnesses

▪ Ease of driving your models with data from various sources

▪ Ease of verify() for in-harness/model verification of requirements

▪ Ease of test case set-up for multiple inputs, parameters, operating points, etc.

▪ Ease of reporting

▪ Ease of integration: requirements, coverage, MATLAB Unit Test,

continuous integration, …

46
47