An Approach: SysML-based Automated

Requirements Verification

Aurelijus Morkevicius Nerijus Jankevicius

Department of Information Systems No Magic Europe
Kaunas University of Technology Kaunas, Lithuania
Kaunas, Lithuania [email protected]
[email protected]

Abstract—Systems Modeling Language (SysML) is used to both verification by testing and verification by analysis, in this
capture systems design as descriptive and analytical system paper we focus on the first task only. The question is how to
models, which relate text-based requirements to the system utilize SysML provided infrastructure to successfully achieve
design model and provide an infrastructure to support analysis verification objectives on the system model: what method and
and verification. However, SysML is not a methodology, nor a what tool or toolset to use in combination with SysML? In this
method. This opens-up discussions of how to utilize SysML paper we are proposing a new approach of how model of the
provided infrastructure to successfully achieve analysis and system, expressed with sufficient precision in SysML, can be
verification objectives in the context of a particular engineering used to support automated requirements verification.
problem. In this paper a new approach of how model of the
system, expressed with sufficient precision in SysML, can be used In particular, the approach combines several different
to support early requirements validation and design verification, techniques: (i) formalization of text-based requirements, (ii)
particularly when coupled with standard-based execution and requirements traceability, (iii) gap and coverage analysis, (iv)
simulation environment, is introduced. definition of test cases and analysis models, (v) automated
requirements verification, and (vi) recording and publishing
Keywords—SysML; Verification and Validation, Requirements verification results. The approach is proven by a case-study
Verification; Automation, System Requirements, MBSE from an automotive industry and real-life findings in different
I. INTRODUCTION systems engineering domains.
Model Based System Engineering (MBSE) promises It is important to understand that there is no way to perform
increase in productivity by shifting from documents-centric automated requirements verification without having a specific
repository to models. To reach this promise organization needs software. The strength of MBSE lays in the tools. There are a
to implement proper practices to enable productive modelling. lot of modelling tools in the market these days, everyone with
Nowadays, MBSE is enabled by Systems Modeling Language. its strengths and weaknesses. This research is carried out using
It is used to model complex systems, such as aircrafts, space Cameo Simulation Toolkit™, a plugin for MagicDraw® CASE
devices, submarines, trains, cars, etc. In this context, the tool. It was chosen because of several published studies, e.g.
process of design and development requires a strong and sound [5], and multiple papers, e.g. [6], [7], [8].
verification and validation (V&V) task, which can be a major The rest of this paper is structured as follows: in section 2,
bottleneck in the development of any complex system, since it the related works are analyzed; in section 3, the proposed
may range from 50 to 80% of the total design effort [1], [2], approach for automatic requirements verification is presented;
especially in safety-critical areas. in section 4, evaluation of the proposed approach is described;
In this paper we focus into a subset of V&V task - in section 5, the achieved results, conclusions, and future work
verification only. Verification is defined as the confirmation by directions are indicated.
examination and provision of objective evidence that the
specified requirements have been fulfilled [3]. For instance, the II. RELATED WORK
method of verification for a system requirement that “The A number of requirements verification methods and
system shall weigh between 98 and 100 pounds” may be techniques for MBSE are currently used by the systems
performed by testing or analysis [4]. To verify the requirement engineering community [20]. Most of them are proprietary
by testing, a test case is defined to weigh the system on a scale methods for internal use only. Other methods are analyzed and
and compare the measured weight against the required weight. described in this section.
To verify this requirement by analysis, the estimated weight of
In [9], an approach to verify complex systems using SysML
each component is summed to estimate the overall system
as a language which describes the system structure and
weight. To perform both without manipulating the real system,
requirements is proposed. Petri nets and temporal logic are
either because it is not yet defined or available, or because it
used respectively to formalize the system behavior and
cannot be exercised directly due to cost, time, resources or risk
constraints, model of a system is needed. Though our interest is

978-1-4799-1920-8/15/$31.00 ©2015
Authorized licensed use limited IEEE of Zagreb. Downloaded on November 22,2023 at 08:43:53 UTC from IEEE Xplore. Restrictions apply.
to: University
requirements. The benefit of such formalization is to allow an D. Evaluation of default or alternative system
automatic formal verification of requirements. configurations:
There is a large number of research papers about semantics a. by analysis
for models that are created using Unified Modeling Language
(UML) [19], e.g. [10], [11], [18]. Most of the research work b. by testing
done on UML is applicable to SysML, as SysML is the E. Verification of requirements
extension of UML. [11] depicts a simple example how the base
semantics can support formal verification through the theorem F. Capture of verification results
proving. The initial results show that the base semantics, when A process of verification by analysis, by testing, and by the
mature, can play an important role in the formal verification of combination of both is depicted in Fig. 1.
UML models.
TURTLE is a real-time UML profile supported by a toolkit
which enables application of formal verification techniques to
the analysis, design and deployment phases of systems design
trajectory [12]. [13] provides an extension to the TURTLE
methodology with a requirement capture phase. SysML
requirement diagrams are introduced. Temporal requirements
(TR) are formally expressed using a dedicated language based
on Allen's interval algebra. TRs serve as starting point to
automatically synthesize observers and to guide the verification
process applied to the TURTLE model of the system.
Verification results are automatically collected in traceability
matrices.
To integrate simulation capabilities into SysML, [14]
proposes the concept of the Evaluation View, a discrete
diagram to specify enterprise information system architecture
under evaluation and the conditions under which performance
requirements should be verified. A corresponding SysML
profile, called Enterprise Information System (EIS) profile, has
been defined.
[2] proposes a unified method for verification and
validation in software and systems engineering with three Fig. 1. Requirements verification process
distinctive features: model-checking, program analysis, and
software engineering techniques. With respect to the latter, a A proposed approach is related to verification by analysis
real life example is shown. only. An approach is implemented in the MagicDraw modeling
Use of SysML parametric for capturing and analyzing tool and successfully applied in several real-life projects.
formal requirements has been defined in [22], [15]. In relation In the subsections below, every step of the process is
to SysML parametric [21] shows how text only requirements described in detail.
can be parameterized and their properties formally typed.
A. Formalization of text-based requirements
Overall, work done in this area has very little prove of its
Standard SysML requirements model provides text-based
successful application on real-industry cases or is a very
requirements specification without any notion of formalization
specific to a small area of applications and specific tools
[9]; however, SysML provides the capability to refine textual
dependent. Moreover, UML-based methods requires a specific
requirement by any model element using the refine
non standardized UML profiles. We are proposing a more
relationship. We found the refine relationship perfect to be
generic, easy to use approach, applicable to majority of SysML
used between requirements and constraint blocks that are
modeling tools for different system engineering domains. capable to contain formal mathematical expressions, e.g.,
III. AUTOMATED REQUIREMENTS VERIFICATION boolean equation. It is important to understand that refine in
this case does not mean verify. It only indicates that the text-
This section describes the proposed approach in detail. based requirement is defined in a formal – machine readable
The approach consists of the following steps: format.
A. Formalization of text-based requirements B. Definition of analysis context
To run experiments on the system model modeled in
B. Definition of analysis context
SysML, the analysis context must be defined. The purpose of
C. Binding system properties to constraint parameters this is to (i) provide a structure composed of system under test,
(ii) use of one or more constraint blocks to evaluate one or
another numerical characteristic of the system under test, (iii)

Authorized licensed use limited to: University of Zagreb. Downloaded on November 22,2023 at 08:43:53 UTC from IEEE Xplore. Restrictions apply.
store test verdict and other result data, and (iv) avoid any direct F. Capture verification results.
impact to the system under test model. Verification results can be saved to the model in the form
Analysis context is defined in SysML block definition of the instance specification element(s). In most cases that is a
diagram and used in SysML parametric diagram, where set of instances for every subcomponent of the system. Instance
numerical characteristics (value properties in terms of SysML) specifications can then be represented in the modeling tool, e.g.
of a system under test or any deep nested part of it are bind to tabular form, or exported to the excel format respectively.
constraint block parameters for mathematical evaluation. In the
block definition diagram it is common to depict text-based IV. CASE STUDY
requirements and show traceability between them and value In this section, we describe a case study for the proposed
properties they are satisfied by. approach. It is a car breaking system case study to verify
requirements by analysis.
C. Binding system properties to constraint paremeters using
SysML Parametrics The car breaking system is defined as a part of a vehicle
In SysML, parametric diagrams are used to create systems model in SysML. Requirements for the system are depicted in
of equations that can constrain properties of blocks [22]. Fig. 2.
Blocks are basic structural elements representing, e.g. structure Stopping distance depends on the pavement and the initial
of a system [16]. Each block may consist of value properties. speed of the vehicle. Our task is to verify if stopping distance is
Parametric diagram connects these value properties to the in range of requirements for a particular vehicle configuration.
parameters of constraint block using binding connectors.
Parameter values are referenced in the expression of the
constraint block and the result is provided. The result may be
used for further calculations within the same context. Boolean
result can be treated as a verdict indicating whether
requirement test has passed or failed [17].
We bind our requirement, formalized in the previous step,
to value properties of the system under test. Parameter for
capturing the result is not needed as we treat constraint
property, representing the formal requirement in the context of
analysis, as a verdict. In case the expected result is not a
boolean, value property needs to be added to the context of
analysis and bind to the result parameter of the constraint
block.
D. Evaluting system configurations by testing
SysML parametric model is a standard-based tool to enable
automatic requirements verification in the defined context of
Fig. 2. Stopping distance requirements
analysis [16]. Values needed for the evaluation of constraint
can be passed in a form of the instance specification of the First, we formalize stopping distance requirement by
system under test. Every single instance specification may have adding a new constraint block named stopping distance
different set of input parameters. In other words, each of them verification. It contains two expressions. The first expression
represents different variant of the system under test. calculates the required distance; the function for it is generated
Alternatively, input values may be changed at run-time. The out of the table depicted in Fig. 2. The second expression
run-time approach is more flexible to observe how the change calculates the Boolean result, whether stopping distance
in one of the inputs influences the result. It is a perfect way to requirement is true or false (Fig. 3). Graphic for the function is
find optimal system under test configuration, export it to the depicted in Fig. 4. Stopping distance verification constraint
instance specification in the model or save results as default block consists of five parameters: (i) alpha that is based on
values to the system design model. pavement condition: 0.6 for “wet” or 0.8 for “dry”, (ii) required
E. Verify requirements distance, (iii) initial speed, (iv) pavement that can be “wet” or
“dry”, and (v) stopping distance that is the actual stopping
Run-time verification of requirements is implemented in
distance for the particular type of vehicle. We draw refine
the MagicDraw modeling tool as a part of the proposed
relationship between our requirement and a newly created
approach. Provided environment allows to observe how the
constraint block to indicate formalization of our text-based
change in one input influences the result of verification by
requirement.
automatically highlighting requirements that fail and, based on
the set-up traceability, allow to see a text of the failing
requirement and provides capability to navigate to it in the
model if needed.

Authorized licensed use limited to: University of Zagreb. Downloaded on November 22,2023 at 08:43:53 UTC from IEEE Xplore. Restrictions apply.
 Fig. 5. Analysis context

Third, we create SysML L parametric diagram, where

stopping distance evaluation and
a verification constraints are
bind to each other and to value properties of a vehicle (Fig. 6).

Fig. 3. Requirement formalization

Actual stopping distance is calculateed using another

expression on a separate constraint block. It consists
c of a single
expression for evaluating vehicle stoppping distance in
accordance to its gross weight, number off breaking wheels
(numBrakingWheels), speed, tire friction coefficient
c on dry
surface (tireMu), tire diameter (rollingDiaam), and braking
torque.

Fig. 6. SysML parametric diagram

In the parametric diagram, constraint

c blocks are represented
as constraint properties: a usagge of constraint in the particular
context. To maintain traceabillity to the requirement we are
verifying constraint block (the requirement we have
formalized). We create verrify relationship between the
constraint property and the requuirement.
Fig. 4. Requirement function
Fourth, to run the modeled scenario we need to build
Second, we select a vehicle as the contexxt for analysis. We several different vehicle conffigurations with different input
create value property stopping for the vehhicle distance and values for weight, speed, and other parameters. For this, we
draw satisfy relationship between the stopping
s distance create four instance specificattions for the vehicle. We also
property and the requirement we are about to verify (Fig. 5). create instance specifications for
f every part of the vehicle and
fill all created instance specificaations with values.
Gross er Of
Numbe Required Stopping
# Name Weight Speed Wheeels distance Distance Verdict
1 Vehicle 1 3200 65 4 176.5487 204.47077 fail
2 Vehicle 2 2900 80 4 267.4347 280.6936 fail
3 Vehicle 3 3100 95 4 377.1247 423.1199 fail
4 Vehicle 4 2500 75 4 235.05 212.67531 pass
Fig. 7. Verification results

Authorized licensed use limited to: University of Zagreb. Downloaded on November 22,2023 at 08:43:53 UTC from IEEE Xplore. Restrictions apply.
 Fifth, we execute SysML parametric model for all four extend the approach in the near future to also support
created variants of a vehicle and get the results (Fig. 7). verification by testing and, finally, we seek to combine both to
solve even more complex verification and validation tasks in
model-based systems engineering.
ACKNOWLEDGMENT
The authors would like to thank:
• No Magic Europe, especially the MagicDraw product
team for comprehensive support.
• Phoenix Integration, Inc. for the SysML case study
model.

Fig. 8. Requirements verification at runtime

REFERENCES

In the Fig. 7 we can see four different configurations of the [1] I. Averant. Static Functional Verification with Solidify, a New Low-Risk
vehicle. The last column is a verdict, indicating that only Methodology for Faster Debug of ASICs and Programmable Parts.
configuration 4 has passed the verification criteria. However, Technical report, Averant, Inc.s, 2001.
we do have another requirement for a vehicle weight, which is [2] L. Alawneh, M. Debbabi, F. Hassaine, Y. Jarraya, A. Soeanu, "A unified
approach for verification and validation of systems and software
in range of 2700 and 3200 pounds. We can see that vehicle 4 engineering models," Engineering of Computer Based Systems, 2006.
does not satisfy it. At the run-time we change the mass to the ECBS 2006. 13th Annual IEEE International Symposium and
allowed minimum and immediately see, if the vehicle 4 passes Workshop. pp.10, 418, 27-30 March 2006.
the verification or not. As we can see in Fig. 8, after change of [3] D. E. Stevenson. Verification and Validation of Complex Systems. In
mass, vehicle 4 still passes the requirement. As a result, we Proceedings of ANNIE 2002, Smart Engineering System Design.
mark latter configuration as the best. ASME, November 2002.
[4] S. Friedenthal, A. Moore, R. Steiner, A Practical Guide to SysML.
Sixth, we export results of the best found configuration to Burlington, MA, USA: Elsevier. 2008.
default values of the system design. [5] R. Cloutier, M. Bone, Compilation of SysML RFI- Final Report,
Systems Modeling Language (SysML) [Online]. Available:
V. CONCLUSION AND FUTURE WORKS http://www.omgwiki.org/ MBSE/
lib/exe/fetch.php?media=mbse:omg_rfi_final_report_02_20_2010-1.pdf
The analysis of existing verification and validation methods
[6] D. Kaslow, G. Soremekun, Hongman Kim, S. Spangelo, "Integrated
for the model-based systems engineering revealed that there are model-based systems engineering (MBSE) applied to the Simulation of a
multiple different methods available. We have also identified CubeSat mission," Aerospace Conference, 2014 IEEE , vol., no.,
that existing methods has very little prove of their successful pp.1,14, 1-8 March 2014.
application on real-industry cases or is a very specific to a [7] Spangelo, S. C., et al.: Applying model based systems engineering
small area of applications and specific tools dependent. (MBSE) to a standard CubeSat (2012) Aerospace Conference IEEE.
Moreover, UML-based methods require a specific non [8] C. Delp, D. Lam, E. Fosse, Cin-Young Lee. Model based document and
standardized UML profiles to be used. We found a need to report generation for systems engineering (2013) Aerospace Conference,
2013 IEEE.
propose a more generic, easy to use approach, applicable on
[9] M.V. Linhares, R.S. de Oliveira, J. Farines, F. Vernadat, "Introducing
majority of SysML modeling tools for different system the modeling and verification process in SysML," Emerging
engineering domains. Technologies and Factory Automation, 2007. ETFA. IEEE Conference
pp. 344,351, 25-28 Sept. 2007.
In this paper, we proposed a new approach of how model of
[10] Y. Jarraya, M. Debbabi, J. Bentahar. On the meaning of SysML activity
the system, expressed with sufficient precision in SysML, can diagrams. In Engineering of Computer Based Systems (ECBS), pp. 95–
be used to support automated requirements verification by 105, San Francisco, CA, USA. IEEE Computer Society. 2009.
analysis. We have implemented the proposed approach in the [11] A. Kraemer, P. Herrmann, Reactive semantics for distributed UML
MagicDraw CASE tool and demonstrated an example case activities. In Hatcliff, J. and Zucca, E., editors, Formal Techniques for
study from automotive domain. We have also showed its Distributed Systems (FORTE), vol. 6117 of LNCS, pp. 17– 31,
integrity with the existing and widely used systems modeling Amsterdam, The Netherlands. Springer. 2010.
language (SysML). [12] B. Fontan, L. Apvrille, P. de Saqui-Sannes, J.P. Courtiat, "Real-Time
and Embedded System Verification Based on Formal Requirements,"
The disadvantage of the proposed approach is the Industrial Embedded Systems, 2006. IES '06. International Symposium
duplication of information having both informal and formal Vol. 1, No. 10, p.p. 18-20 Oct. 2006.
requirements as separate entities. If the change appear in the [13] L. Apvrille, J.-P. Courtiat, C. Lohr, P. de Saqui-Sannes, "TURTLE: A
Real-Time UML Profile Supported by a Formal Validation Toolkit",
text of informal requirement, there is no way to automatically IEEE Trans. on Software Engineering, Vol. 30, No. 7, July 2004,
update the formal one. From the other point of view, the pp.473-487.
approach separates two different levels of abstraction and [14] A. Tsadimas, G.D. Kapos, V. Dalakas, M. Nikolaidou, D.
automatic synchronization option between the two can be Anagnostopoulos, "Integrating simulation capabilities into SysML for
consider in the future enterprise information system design," System of Systems Engineering
(SOSE), 2014 9th International Conference on , pp.272,277, 9-13 June
Currently, the approach is oriented to automated 2014.
requirements verification by analysis only; however, we plan to

Authorized licensed use limited to: University of Zagreb. Downloaded on November 22,2023 at 08:43:53 UTC from IEEE Xplore. Restrictions apply.
[15] A. Morkevicius, S. Gudas, D. Silingas. Model-Driven Quantitative [19] OMG. Unified Modeling Language (OMG UML) Infrastructure, V2.1.2.
Performance Analysis of UPDM-Based Enterprise Architecture. Object Management Group, Needham 2007.
Proceedings of the 16th International Conference on Information and [20] J. Estefan, Survey of Candidate Model-Based Systems Engineering
Software Technologies. Kaunas, 218-223, 2010. (MBSE) Methodologies, rev. B. Seattle, WA, USA, 2008: International
[16] OMG. Systems Modeling Language (OMG SysML) Version 1.3. Council on Systems Engineering (INCOSE). INCOSE-TD-2007-003-02.
Needham. 2012. Accessed April 20, 2015.
[17] OMG. UML Testing Profile (UTP) Version 1.2. Needham. 2013. [21] R. Karban, L. Andolfato, M. Zamparelli. "Toward Model Re-usability
[18] A.G. Romero, K. Schneider, M.G. Ferreira, V. Goncalves, "Using the for the Development of Telescope Control Systems". ICALEPCS 2009,
base semantics given by fUML for verification," Model-Driven October 12, 2009.
Engineering and Software Development (MODELSWARD), 2014 2nd [22] R. S. Peak, R. M. Burkhart, S. A. Friedenthal, M.W. Wilson, M, Bajaj. I.
International Conference pp.5,16, 7-9 Jan. 2014. Kim. Simulation-Based Design Using SysML—Part 1: A Parametrics
Primer. INCOSE Intl. Symposium 2007.

Authorized licensed use limited to: University of Zagreb. Downloaded on November 22,2023 at 08:43:53 UTC from IEEE Xplore. Restrictions apply.