Model Based Esting - 2023
Model Based Esting - 2023
Abstract— Model-based testing (MBT) is a promising Unified Modeling Language (UML) is a popular option
approach for generating test cases from system models, for software modeling as it offers a high level of
providing high levels of automation and effectiveness. The expressiveness [8]. Activity Diagram is the most used UML
potential for automation in MBT is possible if the model is diagram in model-based test case generation [6]. UML
formal and machine-readable. A commonly employed formal
modeling technique is the representation of a system as an
Activity Diagram is a semi-formal specification that can be
extended finite state machine (EFSM). However, in practice, used to describe the workflow of the system and captures
formal models are not common in the industry. Activity critical system behaviors [9]. Although this Diagram is widely
diagrams, on the other hand, are well-suited for generating test used in this field, there is a lack of automated techniques for
cases, but their lack of formal semantics can lead to ambiguous test generation from UML activity diagrams [10]. In addition
interpretations and make them unsuitable for automation. In to the mentioned problem, due to the lack of formal semantics,
this paper, we propose an efficient approach that maps UML the use of UML diagrams can lead to inconsistency,
Activity Diagrams into EFSMs, providing a formal modeling of transformation problems, and different interpretations [5].
the system under test (SUT) and utilizes JUnit and ModelJUnit Another issue of note is that UML specification presents a
Java libraries to automatically generate test cases, using
coverage measures to evaluate them. Our approach aims to
challenge in terms of navigation and comprehension [11].
overcome the limitations of automation in MBT, while achieving Using formal models is one approach to mitigate these
efficient coverage and execution time metrics. issues, as they offer precise semantics for representing system
behavior [8]. Finite State Machines (FSMs) are a favorite
Keywords—Model-Based Testing; Automatic Test Case
generation; Activity Diagram; Extended Finite State Machine; notation in formal system modeling and testing software [5].
ModelJUnit; JUnit Although, formal models are rarely used in practice, probably
because developers lack the necessary training and familiarity
I. INTRODUCTION with the mathematical notation [11].
Software engineering is a field that employs a methodical Formal models are highly recommended in Model-Based
and structured approach to the development, operation, and Testing (MBT) due to their ability to automate the testing
maintenance of software systems [1]. The goal of software process, resulting in improved efficiency and effectiveness
organizations is to develop and deliver software within [5]. Extended finite state machine (EFSM) is one of the formal
specified timeframes and budgets [2]. However, software models that have received significant attention and extensive
development, particularly for complex systems, is prone to study over the past few decades [12]. An EFSM model is an
human errors, necessitating the assessment of software for enhanced model based on FSM. This model can represent
errors during source code generation [3]. Software testing many complex systems containing both control and data parts
plays a crucial role in identifying faults and is a vital but costly [13].
phase in software development and maintenance. One of the
most challenging aspects of testing is generating test cases, In the context of this paper, we propose an efficient and
which is essential to ensure the success of the testing phase systematic approach that maps UML Activity Diagrams into
[4]. EFSM following some transformation rules. By using JUnit
and ModelJUnit Java libraries, test cases are automatically
Modeling techniques can enhance software testing generated from EFSM. The main contribution of this paper is
productivity when used in this context [5]. Model-based test the definition of transformation rules to map the various
case generation (MB-TCG) is a methodology that involves the elements of the UML Activity Diagram into constructions of
generation of tests using system models, which are based on the Extended Finite State Machine (EFSM). Furthermore, to
model-based testing (MBT) [6]. Using MBT allows testing to tackle the issue of complexity and understandability of the
begin early in the software development process, running tests UML specification problem, we utilize an updated version of
without requiring access to the source code of the system the metamodel for UML Activity Diagrams, specifically
under test, owing to its black-box nature. This technique uses tailored for simplicity and applicability while ensuring a close
system models to generate and execute test cases alignment with the EFSM metamodel used in our research.
automatically, reducing testing time and effort [7].
https://doi.org/10.51408/csit2023_05 26
The rest of this paper is organized as follows: Section II = ci . aI ∈ A is the initial state, and aF ∈ A is the final state.
describes some related research. Afterwards, Section III There exists only one transition t ∈ T such that (aI , t) ∈ F, and
presents UML Activity Diagram and Extended Finite State for any t' ∈ T, (t', aI) ∉ F and (aF, t') ∉ F [9].
Machine. Section IV describes our approach and its
implementation. In Section V, the results obtained by In this paper, we use some of the most common elements
implementing the proposed approach on a sample scenario of an Activity Diagram, which are as follows:
have been discussed. Finally, the conclusion and future work • Initial Node: An InitialNode is a control node that
are placed in Section VI. marks the beginning of a process or activity flow.
II. RELATED WORK • Final Node: A FinalNode is a control node that marks
Numerous research studies have been conducted on the the completion of an activity or process flow.
subject of generating test cases from UML Activity Diagrams. • Executable Activity Node: ExecutableNode is a type
One of the most relevant related works in the area of of action node that can contain executable behavior.
automated test case generation from UML models is the article
by Smith et al. [5]. They proposed an approach for generating • Merge Node: A MergeNode is a control node that
test cases from UML sequence diagrams using Extended merges multiple incoming control flows, allowing
Finite State Machines (EFSMs). Their approach showed for their convergence without any synchronization.
promising results, but one of its weaknesses was that it only
• Decision Node: A DecisionNode is a control node
worked for sequence diagrams. We built our work upon the
that represents a decision point, where one of several
approach proposed by this article, but we extended it to work
outgoing control flows is chosen based on a
with UML activity diagrams (AD) since ADs are more
condition or criteria evaluation.
commonly used in software development, as shown in a recent
survey [6]. Our approach can generate test cases that cover a • Input and Output Pins: Input and Output Pins
wider range of scenarios since Activity diagrams can model represent the transfer of data or objects between
more complex behavior and allow for more varied interactions activity nodes, where an Input Pin accepts values or
between objects than sequence diagrams [14]. objects as input to an activity node, and an OutputPin
produces values or objects as output from an activity
In [15], the authors propose an approach for generating
node.
scenario-based test cases from UML Activity Diagrams. The
proposed approach utilizes an intermediate model named • Send and Receive Signal Actions: A Signal
Extended Activity Dependency Graph (EADG), which represents inter-object communication without the
extends activity graphs to generate test scenarios. However, need for a reply, initiating an asynchronous reaction
this approach differs from ours in that it does not utilize in the receiver [8].
model-driven engineering (MDE) concepts or formal models
for test generation. Other elements of Activity Diagrams defined by UML 2.5
are not in the scope of this article.
In [16], the authors propose a novel method for generating
test cases using UML Activity and Sequence Diagrams. Their B. Extended Finite State Machine
approach involves the conversion of Sequence Diagram into a The Extended Finite State Machine (EFSM) is a well-
graph, which refers to Sequence Graph, and transforming the known formal specification technique that is commonly
Activity Diagram into the Activity Graph. Test suite employed to define the various states and actions of a software
generation is achieved by merging the graphs into a single system. This method is widely used to describe the behavior
software graph. While this methodology shares similarities of software systems in a precise and unambiguous manner
with ours, it diverges by not employing MDE concepts or [18].
formal models for test generation. Moreover, the approach An EFSM can be formally represented by a 6-tuple (𝑠𝑠0 , S,
necessitates the manual creation of both Activity and
V, I, O, T) where S is a finite set of states with initial state 𝑠𝑠0 ;
Sequence Diagrams, which can make it difficult to use the tool
V is a finite set of context variables; I is a set of transition
for more complex software systems.
inputs; O is a set of transition outputs; and T is a finite set of
III. BACKGROUND transitions.
27
IV. PROPOSED APPROACH For switch decision nodes, this rule creates a new
This section presents an automated approach for state and transition for each guard. Regardless of the
generating test cases from UML activity diagrams using type of decision node, each transition has output,
EFSM. The methodology involves several steps that begin guard, and action labeled with the guard of the
with translating the UML activity diagram into a formal specific control flow, and any output pin from the
EFSM model using Atlas Transformation Language. The node before the decision node is treated as the
generated EFSM model is used to automate the test generation transition's input. After creating each new state, this
process, where test cases are generated based on EFSM-based rule updates both the previous and current states.
methods. These methods provide coverage for different paths
and states within the system. ModelJUnit and JUnit libraries
are used to facilitate the generation of executable test cases.
Acceleo is then used to perform a Model-To-Text (M2T)
transformation, converting the generated test cases from a
model representation into a textual representation that can be
executed.
A. Metamodeling Constructs
In order to establish a clear and structured foundation for
our research, we defined two metamodels: the UML Activity
Diagram metamodel, which serves as a source for our model
transformation process, and the Extended Finite State
Machine metamodel, which represents the target model. To
implement these metamodels, we utilized the Eclipse
Modeling Framework (EMF) and represented them in Ecore.
The official UML specification [8] can be complex to
navigate and hard to understand. Therefore, this metamodel
has been heavily criticized, and the use of simplified
metamodels is prevalent in most of the literature on this
subject [11]. To address this issue, a simplified metamodel for
the Activity Diagram is proposed in this study, depicted in Fig.
1, in comparison to the metamodel specified by the OMG.
The proposed metamodel contains 19 metaclasses and
eliminates constructs that are not frequently used in practice.
By streamlining the metamodel, it is easier to understand and
apply in practice, providing a more practical and Fig. 1. Activty Diagram Metamodel
straightforward approach to Activity Diagrams.
As shown in Fig. 2, the metamodel used for EFSM is
comprised of six metaclasses, with EFSM serving as an
abstraction of an Extended Finite State Machine. Within the
EFSM entity, there are states, transitions, and context
variables [5].
B. Metamodels Transformation Principles
A detailed description of the rules for transforming an
Activity Diagram into an Extended Finite State Machine is
provided in this section. The defined transformation rules are
listed below:
• RInitialNode: For the node of type InitilalNode, first
an EFSM is created with the same name of the AD,
then initial state S0 is added to it. The initial state is Fig. 2. Extended Finite State Machine Metamodel [5]
used to update both the previous state and the current
state. • RMergeNode: This rule handles merge nodes in the
• RDecisionNode: RDecisionNode rule operates in Activity Diagram (AD) by creating a new transition
two stages, depending on the type of decision node for each node that has a control flow leading to the
being processed. For each conditional decision node, merge node. These transitions do not have any
this rule creates one new state and two new output, guard, or action, but any output pin of the
transitions. The transition with a guard of "true" nodes connected to the merge node is considered to
leads to the newly created state, while the transition be a corresponding transition's input. The transitions
with a guard of "false" returns to the previous state. all lead to the state that was already created for the
28
node after the merge node using the model, we utilized the transformation rules implemented in
"RActivityNode" rule. After creating these ATL.
transitions, both the previous and current states are
After the execution of the transformation rules, we
updated.
obtained an EFSM model that reflected the behavior of the
• RSignalActivityNode: For nodes of type system. Fig. 4 illustrates the resulting EFSM model that was
ExecutableActivityNode, SendSignalActionNode, generated. We created this model using the Visual Paradigm
and ReceiveSignalActionNode, this rule creates a Drawing Tool by utilizing the XML output file generated from
new state and transition. The new state is connected executing the transformation rules on the ATM Activity
to the previous state using the transition, and all pins Diagram.
of the corresponding node in the AD are considered
as inputs for the transition. Additionally, this rule
checks all control flows of a node to see if any flow
goes backward without any conditions. If such flows
exist, a new transition is created to connect the
corresponding states to each other, and the inputs of
these transitions are labeled as previously explained.
After creating these new states and transitions, the
previous and current states are updated as usual.
Our implementation of the transformation rules relied on
Atlas Transformation Language (ATL).
In this study, we are utilizing a slightly modified version
of some of the lazy rules proposed in [5], in combination with
our matched rules. Specifically, the following lazy rules are
employed:
• LrInitialState: The initial state S0 is created,
followed by incrementing the state order. Both the
previous state and the current state are updated to the
newly created initial state. Additionally, the name of
the Activity Diagram is stored in a variable.
• LrState: A new state is generated, the state order is
incremented, the previous state is updated to the
current state, and the current state is changed to the Fig. 3. ATM Activity Diagram
newly created state.
• LrTransition: A transition is established, connecting
the previous state to the current state. The transition's
input is labeled with the input/output pin of the
Activity Diagram. The output, guard, and action can
be null and depend on the type of the activity node.
In our study, we used an automatic four-step approach for
generating test cases. This approach involves implementing
the EFSM model interface, implementing the adapter,
generating the test cases, and then concretizing them. These
four steps were automatically generated using Acceleo.
V. EXPERIMENT
This section presents an overview of our proposed Fig. 4. ATM EFSM Model
approach, as well as the results and a discussion of the
experiment. Afterwards, we followed the 4-step automatic approach,
as previously described, to generate test cases using classes
A. Case Study
AtmModel, AtmAdapter, AtmTest, and AtmJUnit. These
This section describes a case study demonstrating the classes were obtained through the implementation of the four
practical application of our approach. The UML Activity steps in the approach. Finally, our approach was put to the test
Diagram in Fig. 3, depicts an ATM (Automatic Teller through a series of test cases designed to assess its
Machine) withdrawal process. Initially, we created an Activity effectiveness. To execute the tests, we utilized the Eclipse
Diagram model that described the behavior of the system, Modeling Framework (EMF) in conjunction with the
using the Activity Diagram editor implemented in the EMF. ATMTest and ATMJUnit classes.
To transform this Activity Diagram model into an executable
29
Reflecting the importance of coverage criteria to test Similar considerations apply to the transition coverage.
engineers when creating the test cases [19], we used three For action coverage, all possible actions are performed by
distinct coverage types to evaluate the application of our executing each test case. Note that the same explanation
approach. A test model can be represented by an abstract applies to all the other generated test cases, where the
regular set P ⊆ Σa , where Σ is an alphabet denoting possible coverage metrics should be interpreted within the context of
actions, and a ∈ N ∪ {∗} indicating that each test can have each individual test case.
any length. Considering the fact that P may be huge or even
C. Discussion
infinite, including 'a' in N ∪ {∗} means that tests can vary in
length within specific boundaries [20]. As a result, coverage Our approach's novelty lies in its ability to automate the
criteria can be outlined as in Definition 1. entire testing process, with the exception of manually defined
stubs. To the best of our knowledge, the level of automation
Definition 1 (coverage). A set of tests S ⊆ P is said to cover we have achieved with ADs has not been reported before,
a test-model P ⊆ Σa under the coverage criteria C = {C(i)}i∈I , making our approach a valuable contribution to the field. To
C(i) ⊆ Σa if ∀i ∈ I : (C(i) ∩ P ≠ ∅ ⇒ C(i) ∩ S ≠ ∅) and further illustrate the superior automation achieved by our
∪i∈I C(i) = Σa , where the index set I is used to name the approach, we conducted a comprehensive comparison with
coverage sets and the coverage sets C(i) are used to identify three of the most automated and recent works in the field, as
the aspects that are covered by the tests [20]. shown in Table II.
During testing, we tracked state, action, and transition
TABLE I. GENERATED TEST CASES
coverage metrics for each individual test case, as presented in
Table I. The state coverage metric indicates the number of Test Time
states visited, and its formula can be expressed as (1), where card psw value State Action Trans.
case (ms)
|SV| is the number of states visited during testing and |TS| is
T1 111 123 5/14 12/12 60/144 6.0
the number of total states.
T2 111 246 5/14 12/12 60/144 13.6
state coverage = (|SV|/|TS|) ×100 (1)
T3 222 246 2/14 12/12 24/144 7.2
The action coverage metric reflects the number of actions
T4 111 123 50 12/14 12/12 144/144 16.7
performed during testing. The action coverage formula is
given by (2), where |AP| is the number of actions performed T5 111 123 200 12/14 12/12 144/144 23.5
during testing and |TA| is the number of total actions.
Total _ _ _ 14/14 12/12 144/144 _
action coverage =(|AP|/|TA|) ×100 (2)
The transition coverage metric measures the number of
transitions that were visited. The transition coverage formula
is shown in (3), where |TV| is the number of states triggered
during testing and |TT| is the number of total transitions.
transition coverage = (|TV|/|TT|) ×100 (3)
Prior to initiating the tests, we established initial values for
the card attribute (111), psw attribute (123), and balance
attribute (100.00) belonging to the Bank class of the SUT.
B. Obtained Results
The generated test cases and the results obtained from their Fig. 5. The state, action, and transition coverage, along with the execution
execution are all reported in Table I. We have also included a time (in milliseconds), for each generated test case
visually informative bar chart, shown in Fig. 5 to illustrate the
metrics associated with each test case. TABLE II. COMPARISON OF THE LEVEL OF AUTOMATION
Level of
Fig. 5 displays the coverage metrics for each test case Approach
Automation
Description
generated using our approach. It is important to note that the
possible number of states, actions, and transitions that can be M. Rocha et al. [11] Very High Achieves superior automation, but
not applicable to ADs.
visited may vary depending on the specific test case and its
associated parameters. A. Hettab et al. [15] Moderate Test data can only be generated
In particular, for the initial set of generated test cases, the manually from the automatically
generated test scenarios.
total number of states in the ATM system is determined to be
14 based on the EFSM representation in Fig. 2. However, Meiliana, I. et al [16] Fairly High Testing itself is automated, but the
since the value is not generated within the scope of the test approach necessitates the manual
creation of 2 UML diagrams.
cases T1, T2, and T3, the actual number of states that can be
visited is reduced. According to our EFSM analysis, only 5 Our approach High Automates the entire testing
process except for stubs.
states are possible to be visited, given the absence of the
value.
30
Direct comparison between works with different levels of [11] M. Rocha, A. Simão, and T. Sousa, “Model-based test case generation
from UML sequence diagrams using extended finite state machines,”
automation would not be valid [21]. Given that no other work Software Quality Journal, vol. 29, no. 3, 2021.
has reported achieving the same level of automation with [12] R. Yang, Z. Chen, Z. Zhang, and B. Xu, “EFSM-Based Test Case
ADs, our proposed approach and the results obtained can only Generation: Sequence, Data, and Oracle,” International Journal of
be discussed in a more general sense. As outlined in Table I, Software Engineering and Knowledge Engineering, vol. 25, no. 4,
2015.
by executing all the generated test cases, we achieve complete [13] J. Zhang, R. Yang, Z. Chen, Z. Zhao, and B. Xu, “Automated EFSM-
coverage of all EFSM actions, states, and transitions. based test case generation with scatter search,” in 2012 7th
Furthermore, the efficiency of our approach is demonstrated International Workshop on Automation of Software Test, AST 2012 -
by the fact that it generates test cases in less than 1 second. Proceedings, 2012.
[14] M. L. Shoemaker, UML Applied: A .NET Perspective. Apress, 2004.
VI. CONCLUSION AND FUTURE WORK [15] A. Hettab, A. Chaoui, M. Boubakir, and E. Kerkouche, “Automatic
scenario-oriented test case generation from UML activity diagrams: a
In this paper, we proposed an automated model-based test graph transformation and simulation approach,” International Journal
case generation approach using Activity Diagrams. To of Computer Aided Engineering and Technology, vol. 16, no. 3, pp.
379–415, 2022.
formalize the activity diagrams, we proposed a simplified [16] Meiliana, I. Septian, R. S. Alianto, Daniel, and F. L. Gaol,
metamodel that was used to map ADs to an EFSM “Automated Test Case Generation from UML Activity Diagram and
representation using some transformation rules. We then Sequence Diagram using Depth First Search Algorithm,” in Procedia
applied a four-step approach to generate test cases, using JUnit Computer Science, 2017.
[17] T. Ahmad, J. Iqbal, A. Ashraf, D. Truscan, and I. Porres, “Model-
and ModelJUnit to execute the tests. Our approach was able based testing using UML activity diagrams: A systematic mapping
to achieve good coverage and time results, demonstrating the study,” Computer Science Review, vol. 33. 2019.
effectiveness of our methodology. [18] X. Zhou, R. Zhao, and F. You, “EFSM-based test data generation with
Multi-Population Genetic Algorithm,” in Proceedings of the IEEE
As future work, other UML diagrams, such as state and International Conference on Software Engineering and Service
class diagrams, can be incorporated into the proposed test Sciences, ICSESS, 2014.
generation process. Moreover, advanced testing techniques, [19] V. Rechtberger, M. Bures, and B. S. Ahmed, “Overview of Test
Coverage Criteria for Test Case Generation from Finite State
such as mutation testing, can be used to enhance the generated Machines Modelled as Directed Graphs,” in Proceedings - 2022 IEEE
test suite. Finally, our approach can be applied to a real-world 14th International Conference on Software Testing, Verification and
software system to further evaluate its effectiveness and Validation Workshops, ICSTW 2022, 2022.
scalability. [20] A. Elyasaf, E. Farchi, O. Margalit, G. Weiss, and Y. Weiss,
“Generalized Coverage Criteria for Combinatorial Sequence Testing,”
IEEE Transactions on Software Engineering, May. 2023.
REFERENCES [21] M. Blackburn, R. Busser, and A. Nauman, “Why Model-Based Test
[1] F. Morsali and M. R. Keyvanpour, “Search-based software module Automation is Different and What You Should Know to Get Started,”
clustering techniques: A review article,” in 2017 IEEE 4th in Proceedings of the International Conference on Practical Software
International Conference on Knowledge-Based Engineering and Quality and Testing, Washington, USA, 2004.
Innovation, KBEI 2017, 2018.
[2] N. Mottaghi and M. R. Keyvanpour, “Test suite reduction using data
mining techniques: A review article,” in 18th CSI International
Symposium on Computer Science and Software Engineering, CSSE
2017, 2018.
[3] S. Kashefi Gargari and M. R. Keyvanpour, “Comparative Analytical
Survey on SBST Challenges from the Perspective of the Test
Techniques,” International Journal of Information and
Communication Technology Research, vol. 14, no. 2, pp. 32–40, Jun.
2022.
[4] M. R. Keyvanpour, H. Homayouni, H. Shirazi, and H. Shirazee,
“Automatic software test case generation: An analytical classification
framework,” International Journal of Software Engineering and Its
Applications, vol. 6, no. 4, pp. 1-6, Oct. 2012.
[5] M. Rocha, A. Simão, T. Sousa, and M. Batista, “Test case generation
by EFSM extracted from UML sequence diagrams,” in Proceedings
of the International Conference on Software Engineering and
Knowledge Engineering, SEKE, 2019.
[6] M. L. Mohd-Shafie, W. M. N. W. Kadir, H. Lichter, M.
Khatibsyarbini, and M. A. Isa, “Model-based test case generation and
prioritization: a systematic literature review,” Softw Syst Model, vol.
21, no. 2, 2022.
[7] M. L. Mohd-Shafie, W. M. N. W. Kadir, M. Khatibsyarbini, M. A.
Isa, I. Ghani, and H. Ruslai, “An EFSM-Based Test Data Generation
Approach in Model-Based Testing,” Computers, Materials and
Continua, vol. 71, no. 2, 2022.
[8] “OMG Unified Modeling Language TM (OMG UML),” 2015.
[Online]. Available: http://www.omg.org/spec/UML/2.5
[9] M. Chen, P. Mishra, and D. Kalita, “Coverage-driven automatic test
generation for UML activity diagrams,” in Proceedings of the ACM
Great Lakes Symposium on VLSI, GLSVLSI, 2008.
[10] F. A. D. Teixeira and G. Braga E Silva, “Easytest: An approach for
automatic test cases generation from UML activity diagrams,” in
Advances in Intelligent Systems and Computing, Springer Verlag,
2018, pp. 411–417.
31