A weight optimized artificial neural

network for automated software test oracle

K. Kamaraj, C. Arvind & K. Srihari

Soft Computing
A Fusion of Foundations,
Methodologies and Applications

ISSN 1432-7643

Soft Comput
DOI 10.1007/s00500-020-05197-9

1 23
Your article is protected by copyright and
all rights are held exclusively by Springer-
Verlag GmbH Germany, part of Springer
Nature. This e-offprint is for personal use only
and shall not be self-archived in electronic
repositories. If you wish to self-archive your
article, please use the accepted manuscript
version for posting on your own website. You
may further deposit the accepted manuscript
version in any repository, provided it is only
made publicly available 12 months after
official publication or later and provided
acknowledgement is given to the original
source of publication and a link is inserted
to the published article on Springer's
website. The link must be accompanied by
the following text: "The final publication is
available at link.springer.com”.

1 23
 Author's personal copy
Soft Computing
https://doi.org/10.1007/s00500-020-05197-9 (0123456789().,-volV)(0123456789().
,- volV)

METHODOLOGIES AND APPLICATION

A weight optimized artificial neural network for automated software

test oracle
K. Kamaraj1 • C. Arvind2 • K. Srihari3

 Springer-Verlag GmbH Germany, part of Springer Nature 2020

Abstract
Software testing has its main goal as designing new test case sets in a manner in which it is able to depict its maximum
faults. As soon as these test cases have been designed, Oracle software provides a method in which the software has to
behave for a particular test case given. Prioritization of such test cases with the execution of their components specifying
inputs, their operation and their outcome will determine as to whether the application and their properties are working in
the right manner. The prioritization methods are as follows: initial ordering, random ordering and finally reverse ordering
that were based on fault detection abilities. For developing software applications, a test suite that was less commonly
known as the suite for checking the validity of software was employed. The test suite contained a detailed set of
instructions and goals for each test case collection based on the system and its configuration used during testing.
Automating the generation of a test case and test oracle was researched in an extensive manner. From among the automated
test oracle, the artificial neural network (ANN) was used extensively but with a high cost of computation. This work
proposed a weight optimized ANN using stochastic diffusion search to find the optimal weights with a unique fitness
function such that computational time is reduced and misclassification rate reduced.

Keywords Evolutionary algorithms  Artificial neural network  Neural science  Soft computing  Software testing 
Test cases  Test oracle  Stochastic diffusion search

1 Introduction software. The importance of any failure was determined by

means of the likelihood of encountering such failure. Thus,
The community of software engineering has now turned its the software was tested in accordance with the model that
attention to the testing of statistical software (Mao et al. had highlighted critical usage and several test cases that
2006). The primary idea here was the reliability of the were to be executed for statistically simulating software
software that was dependent on the manner of using the usage. But it does need time and is generally prone to
errors when manually generating all outputs for the test
cases and compare them to actual outputs of that of the
Communicated by V. Loia. application under test (AUT). The result of this automated
test oracle was required in such statistical software testing
& K. Kamaraj
[email protected]; [email protected] for the automatic generation of the output expected for
purposes of comparison.
C. Arvind
[email protected] Software testing, at one time, was identified as a distinct
phase in the life cycle of software development. There are
K. Srihari
[email protected] several researchers who have in the past attributing soft-
ware failure to a lack of testing. Software testing has been
1
Department of Computer Science and Engineering, KPR done at several levels such as unit testing, acceptance
Institute of Engineering and Technology, Coimbatore, India testing, system testing, integration testing, module testing
2
Department of Electronics and Communication Engineering, and regression testing. These levels were broadly grouped
Karpagam College of Engineering, Coimbatore, India into two approaches: White-box testing and Black-box
3
Department of Computer Science and Engineering, SNS testing. The former was done by code in hand by the
College of Engineering, Coimbatore, India

123
 Author's personal copy
K. Kamaraj et al.

programmer. It is also known as structural testing. The Neural computation is the hypothetical information
latter which is also called functional testing takes into processing performed by networks of neurons. A system is
consideration the software to be a black-box that is hidden said to be adaptive if it transforms itself with the changes in
with an implementation. Inputs for this are given under the its environment. The definition of a generalization is a
test to the software and the outputs are compared to the broad statement or idea that appeals to a lot of people or
expected ones (Catherine and Djodilatchoumy 2019). Test situations. Fault injection testing is a software testing
oracles were found to be dependable sources that help in technique that deliberately includes errors to a system to
indicating the manner in which the test software has to guarantee it can withstand and recover from error condi-
operate. Particularly, they are employed for the evaluation tions. To ‘‘converge’’ in machine learning is to have a
of results that are produced by such software. But, for the mistake so close to local/global minimum.
purpose of generating automated test oracles, it may A recurrent neural network displays over time. It is used
become necessary to map its input domain automatically to for sequential inputs where the time factor is the main
the output domain. The automation of test oracle will need differentiating factor between the elements of the
a mapping of the input domain to that of the output domain sequence. The topology of recurrent neural network and
and make a comparison of the actual results with the non-recurrent neural network is represented in Fig. 1.
expected results in order to ensure verification of the After this, the machine learning algorithm will be
behaviour of SUT is complete. trained on that of the training dataset. Once this is done, the
The selection of features is crucial for simplifying this new test cases are classified in accordance with their dif-
model and bring down the time taken for training thus ferent classes of priority. Lastly, the predicted cases having
enhancing its generalization. The available time, resources the highest priority will get executed first thus revealing all
and experience level will determine the method to be used. important faults in the early stages itself. The ANN is a set
The stochastic diffusion search (SDS) was also placed in of input output network that is associated with every single
the same context with the other traditional algorithms as connection. It contains an input layer and either one or
well. Recently, the approaches based on machine learning more intermediate layer and one output layer. The layer of
were used for the prioritization of test cases. The concept the neural network was performed by means of adjusting
of applying such approaches includes the creation of a the connection and its weight. By means of updating its
dataset for training that was based on different features of weight network performance is greatly improved. Based on
such test cases. These were later labelled using priority the connection of the ANN, this may be grouped into two:
classes and certain other criteria. Machine learning algo- the feed-forward network and the recurrent network. The
rithms were later trained based on training datasets after former denotes the network that is in connection existing
which they were classified in accordance with their classes between the units that do not form any cycle and in the case
of priority. Finally, these test cases predicted to have the of the latter, the connections form a cycle (Bala and Kumar
highest level of priority is executed first thus revealing all 2017). The neural network and its behaviour have been
faults in the early stages. affected by the transfer function, architecture and the
An artificial neuron network (ANN) is a computational learning rule. The neurons of this neural network get
design based on the structure and functions of vital neural activated by means of the input and its weighted sum. The
networks. ANNs are rated logically simple mathematical signal for activation will be passed by means of a transfer
examples to improve subsisting data analysis technologies. function for producing one single output of a neuron. The
Remarkable features of ANN are: an artificial neural net- nonlinearity of this network was produced by using this
work consists of a large number of ‘‘neuron’’ like pro- transfer function. At the time of training, inter-connection
cessing elements; all these have a large number of weight gets optimized until such time the network reaches
weighted connections between them, and the similarities its specified accuracy level. This has many advantages such
between the elements present a distributed description of as the good ability of learning, not being affected by noise
data. An ANN applications are medical diagnosis, indus- and parallelism.
trial, operational analysis, marketing, etc., the neural net- The hypothesis used in this work concerns the optimal
work learns through various learning schemes that are weights in neural network. The hypothesis is based that
characterized as supervised or unsupervised learning. weights based on gradient descent search finds sub-optimal
Supervised learning deals with identified data, high com- solutions. Obtaining optimal weights for neural networks is
plexity and obtain accurate outcomes. Some types are: a non-deterministic polynomial problem and can be best
gradient descent and stochastic. An unsupervised learning solved using heuristic algorithms. Among heuristic algo-
manipulates unlabelled data, low complexity and creates rithms, genetic algorithm (GA) has been extensively used
moderate results. Some types are: Hebbian ad competitive. but it struggles from slow convergence. Other techniques
including particle swarm optimization (PSO) have been

123
 Author's personal copy
A weight optimized artificial neural network for automated software test oracle

Fig. 1 a Recurrent neural Hidden

network and b non-recurrent Input Layers Output
neural network Layer Layer

(a) Recurrent neural network (b) Non recurrent neural network

used but these struggle either from local optima or global defining tolerance and its comparison. This approach was
optima problem. The stochastic diffusion search (SDS) further evaluated by using the strength of the industry and
algorithm has been found to be ideal due to its balanced its case study that had been injected using certain faults.
local and global search. The proposed weight-optimized The proposed oracle’s quality was further measured by
ANN using SDS technique is compared with PSO. The means of assessing its practicality, misclassification error,
remaining part of the investigation is organized into the precision and finally accuracy. The process of mutation
following sections. Section 2 discusses related works in the testing has been considered to give an evaluation frame-
literature. Section 3 explains various methods used in the work by means of implementing two types of a version of
work. Section 4 discusses experimental results, and Sect. 5 its case study which were the Golden version and the
concludes the work. Mutated version. Also, there was a comparative study
made on the existing oracles and the proposed one. Results
proved that the proposed approach was able to automate
2 Literature survey the process of oracle generation by about 97% based on
this experiment. The accuracy of the oracle proposed was
Shahamiri et al. (2010) had made a proposal of another about 98.26%, and it was able to detect about 97.7% of
attempt at studying applications of the ANN to be a single- injected faults.
network oracle for verifying their logical modules. Ini- The multi-networks oracles that were based on the ANN
tially, the logical modules that are under the test were were introduced for handling in an automatic manner. They
modelled using the neural network with a training dataset were an enhanced version of the earlier oracles based on
that was generated on the basis of their software applica- the ANN. The model proposed had been evaluated by
tions. Subsequently, the approach was applied for testing means of a framework that was provided by that of
the subject and its registration. The quality of the oracle mutation testing which was applied to for testing the two
was measured by means of assessing practicality, mis- different industry-sized case studies as in Shahamiri et al.
classification errors, precision and accuracy. This was done (2012). Particularly, there was a mutated version of every
by using the implementation of two versions of mutation case study that was provided and further injected with
testing: the Golden version and the Mutated version. certain faults. After which there was a fault-free version of
Results proved that neural networks were reliable and also the same that was developed in the form of a Golden
applicative in the form of oracles for verifying modules. version for evaluation of the capacity of the oracle pro-
Shahamiri et al. (2011) had made a new proposal of posed that finds injected faults. At the same time, the actual
another automated framework of test oracle for addressing quality of this oracle gets measured by means of an
these challenges. The I/O relationship analysis was assessment of its precision, misclassification error, accu-
employed for generating output domain in an automatic racy and recall. Also, the results of the oracle proposed
manner and there were several multi-networks oracles that were compared to the former ANN-based oracles. The
were based on the ANNs that were introduced for handling accuracy was about 98.93%, and it was able to detect about
the subsequent challenges. The final challenge was 98% of the faults injected. The study and its results proved
addressed by making use of an automated comparator that that the proposed oracle had a quality which was better and
could adjust the precision of the oracle by means of also its applicability levels were good.

123
 Author's personal copy
K. Kamaraj et al.

Hocrodeyro (2018) had proposed another new concept that described the algorithm and its behaviour based on
that made use of the artificial neural network to be the convergence to the global optimum, minimal convergence
automated one for the software system which was tested. criteria and linear complexity. The SDS can be made
The neural network was trained means of a backpropaga- applicable to the objective functions that are ever-changing
tion algorithm which was kept on a set of certain test cases and this makes it useful in solving problems of various
that were applied to the system’s original version. The fields like eye tracking in the facial images. This was done
training of the network was made on a ‘‘black-box’’ by means of combining the n-tuple network, the decision of
approach as only the inputs and the outputs of a system site for the transmission equipment, mouth locating in the
were presented here. The network that is trained can be facial images of humans and the stochastic search network.
used in place of an artificial oracle to evaluate the output The SDS algorithm:
and its correctness by means of a faulty version of the same Initialising agents ðÞ
software. There were experimental results that were pre-
While ðstopping condition is not met)
sented with a new two-layer neural network for detection
of faults inside the mutated code belonging to a credit Testing hypotheses ðÞ
approval application that was small. Results proved to be Diffusion hypotheses ðÞ
promising for a range of infected faults. Hooda and End
Chhillar (2018) had further made a new proposal of another
concept with the UML state-chart diagram along with
tables for the generation of a test case wherein the artificial • Initialization phase: In this phase, every agent makes a
neural network is the tool used for optimization and also random choice of the hypothesis and these subsets are
for the reduction in redundancy in test cases generated with used to lead the process of search.
the GA. The neural network was trained by means of a • Test phase: Once initialization is done, every agent is
backpropagation algorithm for a set of test cases that were assigned to a hypothesis in a random manner on the
applied to the system’s original version. basis of an objective function; in case the evaluation of
Most works in the literature have used neural networks hypothesis is successful, the agent will be set to active
and its variants including Radial basis functions. Marquardt and if not inactive. Thus, at the end of a test phase,
algorithm, conjugate gradient algorithm and variable every agent will adopt any one of the two Boolean
learning rate algorithm are some popular techniques that items.
have been effectively used for learning in neural networks. • Diffusion phase: At the time of diffusion, information
Ensemble classifiers including Boosting and Bagging on the hypotheses get exchanged among agents. The
techniques have been used with higher error rates (Duan purpose behind this phase was for the active agents to
et al. 2015). be able to communicate hypothesis to the ones that are
inactive. There were different communication methods
called recruitment strategies. They are active recruit-
3 Methodology ment, passive recruitment, context-sensitive mecha-
nism, context-free mechanism and dual recruitment
This section details on stochastic diffusion search (SDS), (Majid al-Rifaie and Bishop 2013). A standard SDs will
artificial neural network (ANN). apply a passive strategy of recruitment wherein the
inactive agent chooses the other in a random manner.
3.1 Stochastic diffusion search (SDS) The active agent’s hypothesis was diffused to that of the
optimization inactive one and if not the inactive agent will choose a
hypothesis randomly. For the purpose of this work,
The stochastic diffusion search (SDS) had been described there was a context-sensitive strategy that was applied
by Bishop to be a matching algorithm that was based on the in which the active agents were given a new hypothesis
population using direct patterns of communication like the for setting up the agents free and for improving
cooperative transport that was found in the social insects diversity.
for performing search evaluation along with optimization
hypothesis. The SDS was applicable to the search problems
and their optimization wherein the component functions 3.2 Artificial neural network (ANN) classifier
were independently evaluated with swarm agents main-
taining a hypothesis on the optima (Pandian 2019). On ANN refers to an adaptive system that is complex and can
being compared to the other algorithms that were inspired convert its internal structure on the basis of the information
by nature, the SDS had a strong framework in mathematics that is passed through it. This can be achieved by means of

123
 Author's personal copy
A weight optimized artificial neural network for automated software test oracle

adjusting the connection and its weight. The weight is the to be accurate as the ANN depends on such types of data
number that controls the signal that exists between two for simulating the behaviour of the SUT.
different neurons. These weights will be adjusted for For the training phase, every record of training samples
improving results. will have to be applied for making the test oracle accurate.
The artificial neural network can be a good example of For increasing network quality, the ANN results had been
supervised learning. It acquires knowledge as a connected compared to the correct results that were given by the
unit of the network. It may be challenging for humans to training samples. Network error (MSE) was measured by
extract such knowledge and the factor was motivated to means of computing the actual distance between their
extract classification of data mining rules. This process expected results and the results generated by the network.
begins with a dataset grouped into two: the training sample For achieving an adequate rate of error, the parameters of
and the test sample. The training sample is to learn the the network (the neurons and biases weights) were adjusted
network and the test sample to measure the classifier by means of back-propagating error data. The entire pro-
accuracy. Dataset division will be done by different cess was repeated and duly continued until such time the
methods such as the hold-out method, random sampling adequate rate of error was obtained. Now, the network
and cross-validation. The learning steps of the neural net- adjustment cycles were completed and the resultant net-
work are as shown below: work was ready to be employed. Once training was com-
pleted, the ANN had to generate expected results for a
• The structure is defined with certain nodes in hidden,
combination of an input by a training dataset. For incorrect
input and output layers.
ANN results, the structure of the network had to be mod-
• There is an algorithm which is employed for the process
ified using a trial and error method.
of learning.
If the ANN was correctly trained, a single-network
The neural network’s ability to make adjustments to the oracle can be used. The process is: with the test cases a
network structure and also its learning ability done by logical module under the test will be executed with the
changing the weight to make it very useful in artificial input vector that was provided by a test case. At the same
intelligence. time, the oracle was given using similar inputs. The actual
outputs that had to be evaluated were generated with the
SUT and the outputs expected by the ANN were used for
ALGORITHM
comparison. Differences were reported as faults taking into
Input: dataset D, the learning rate, the network. consideration the thresholds defining tolerance of
Output: A trained neural network. comparison.
• Step 1: Receive input. Backpropagation (BP) was found to be a very popular
• Step 2: Weight input. Every input that is sent to the network has technique used for training the ANNs and also for methods
to be weighted, i.e., multiplied using a random value falling of optimization such as gradient descent (Williams and
between - 1 and ? 1. Bishop 2014). This method computes the gradient of all
• Step 3: Sum the weighted input. loss functions in connection with the weights found in the
• Step 4: Generate the output: the actual network output will be network. With the right quantity of input nodes, the hidden
produced by passing the sum through an activation function. layers, and the output layers n, k and m are the overall
quantity of their input instances which is xpi which implies
that the instance of p’s ith input value, and vki indicates the
ith node of that of the hidden input layer of that of the kth
ANNs were trained by making use of a dataset that was
node weight. xjk denotes the actual node weight which is
composed of various inputs along with some corresponding
from the hidden of that of k to j and its output layer. For
results that were generated on the basis of the logical rules.
making this easy, the threshold will be the connection
After this, it was duly applied for verifying case studies.
weights and the hidden outputs of hidden layer node k in
Before the ANN was trained all non-numeric inputs, as
(1):
well as outputs, were normalized into certain numeric !
values owing to the fact that the ANNs were able to learn Xn
zpk ¼ f ðnetpk Þ ¼ f vki xpi ð1Þ
from their numeric values. The I/O datasets or training
i¼0
samples were applied for training the ANN. It had to be
noted that even before moving to the training phase, the Output layer nodes for the node j in (2):
necessary training samples have to be given by the expert
knowledge of the domain and the verified needs of the
software or other reliable approaches used. The dataset has

123
 Author's personal copy
K. Kamaraj et al.

!
X
n hidden layer and actual output value for each unit of ypj
ypj ¼ f ðnetpj Þ ¼ f wjk zpk ð2Þ which was computed. The next phase was in a reverse
i¼0 process in which if the anticipated output value is not
wherein, standard sigmoid function is chosen as incentive obtained by the output layer, the recursive calculation is
function in (3): done layer by layer for the difference of error which is
between the actual and the anticipated output will be done.
1
f ðxÞ ¼ ð3Þ The gradient descent technique now alters all the weights
1 þ ex of Dvki ; Dxjk , thus ensuring the overall error function is
The global error function may be given as in Eq. (4): kept minimal.
X
P
1XX
p m
E¼ EP ¼ ðtpj  ypj Þ2 ð4Þ 3.3 Proposed artificial neural network
p¼1
2 p¼1 j¼1
The ANN learning algorithms used in the literature include
where EP is error of the sample p, tpj is the ideal result. The
gradient descent search, Levenberg–Marquardt algorithm.
adjustment formulae of weights are thus.
The weights chosen by these techniques are sub-optimal,
The weight adjustment equation of output layer neurons
and to overcome these problem, we proposed a stochastic
in (5):
! diffusion approach.
Xp Xm ANN architecture was determined by means of the
Dxjk ¼ g dpi xjk zpk ð1  zpk Þxpi ð5Þ actual number of inputs, outputs, neurons and also the
p¼1 j¼1
interlinks between them. According to past experience, it is
wherein g represents the learning rate, in which general evident that the solutions for complex issues are needed for
range is 0.1–0.3. creating an ANN architecture that was complex. On
The weight adjustment equation of hidden layer neurons working with the SDS, different experiences show that the
in (6): methods are very efficient in comparison with the other
! problems in optimization at the time complexity has been
X p Xm
Dvki ¼ g dpi xjk zpk ð1  zpk Þxpi ð6Þ measured by the dimensions of the system. Keeping the
p¼1 j¼1 multilayer significance in mind, the ANN structures were
taken into consideration.
The actual notion of BP was to ensure the learning
A generalization property of the ANN was found to be
process is split into two different phases; the first was the
the very basic paradigm of performance. It was, in fact, the
process of forward propagation and for this input data were
very essence of the property that approved the fact that the
given through a processing done layer by layer for each
‘‘network has learnt’’ to provide answers to input that were

Table 1 Sample 4-3-2 ANN

w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 w12 w13 w14

Agent 1 0.237 0.5725 0.25 0.6544 0.8832 0.1108 0.2298 0.3818 0.6649 0.3599 0.6531 0.8536 0.2069 0.6292
Agent 2 0.0267 0.074 0.046 0.8919 0.9116 0.4655 0.8682 0.8379 0.8817 0.272 0.9053 0.3654 0.4511 0.522
Agent 3 0.5568 0.2514 0.9089 0.303 0.4037 0.5057 0.634 0.4814 0.824 0.452 0.7472 0.1321 0.2893 0.4096
Agent 4 0.8231 0.3594 0.9051 0.3727 0.9275 0.2581 0.9594 0.3086 0.5026 0.321 0.8861 0.7562 0.5486 0.7083
Agent 5 0.7351 0.4426 0.4009 0.6627 0.6503 0.4862 0.4173 0.7973 0.0676 0.8384 0.2902 0.4974 0.4307 0.0908
Agent 6 0.5477 0.6935 0.6435 0.4136 0.7795 0.3227 0.9103 0.2199 0.1622 0.9778 0.9171 0.0353 0.2687 0.7881
Agent 7 0.4743 0.9293 0.8522 0.9685 0.4228 0.1217 0.7859 0.442 0.7075 0.6262 0.0036 0.6824 0.1576 0.2868
Agent 8 0.3573 0.0759 0.4673 0.1387 0.6814 0.7638 0.9711 0.2087 0.8588 0.8322 0.9356 0.0424 0.0683 0.6864
Agent 9 0.7827 0.7912 0.063 0.7601 0.755 0.9875 0.4851 0.1174 0.5364 0.7603 0.4253 0.0536 0.7093 0.1494
Agent10 0.7644 0.529 0.9477 0.3791 0.8858 0.8968 0.2985 0.4134 0.6723 0.7703 0.7786 0.342 0.0479 0.2519
Agent 11 0.5939 0.0901 0.3618 0.0066 0.4239 0.9899 0.9215 0.7198 0.6409 0.976 0.1704 0.982 0.5057 0.9372
Agent 12 0.9871 0.3112 0.9245 0.3509 0.6172 0.6903 0.2013 0.3435 0.7776 0.4969 0.9538 0.2825 0.5884 0.1988
Agent 13 0.1559 0.6329 0.9015 0.7604 0.9666 0.1044 0.4678 0.1751 0.5841 0.3946 0.9063 0.6435 0.9699 0.2183
Agent 14 0.6928 0.5842 0.0086 0.4265 0.4986 0.8908 0.9317 0.4651 0.7957 0.6193 0.3516 0.4709 0.6735 0.141
Agent 15 0.945 0.18 0.2838 0.9154 0.9423 0.8992 0.7358 0.0307 0.2963 0.65 0.4726 0.8025 0.8757 0.243

123
 Author's personal copy
A weight optimized artificial neural network for automated software test oracle

Table 2 Fitness value 4 Sample working

Agent 1 0.43
Agent 2 0.8215
Agent 3 0.154 Tables 1, 2, 3, 4 and 5 show the sample 4-3-2 ANN, fitness
Agent 4 0.1774 value, hypothesis evaluation, new weights iteration 1 and
Agent 5 0.0184 fitness value—iteration 1.
Agent 6 0.5923
Agent 7 0.6024
Agent 8 0.9534 5 Results and discussion
Agent 9 0.7551
Agent 10 0.4108
Software oracles have been extensively used as a sup-
Agent 11 0.8069
porting tool for test automation. Among open source tools,
Selenium and Katalon Studio are extensively used and both
Agent 12 0.4302
do not support test oracles. This work proposed techniques
Agent 13 0.463
to implement test oracles. Simulations were carried out
Agent 14 0.0194
using MATLAB and customized WEKA tool. WEKA is an
Agent 15 0.7367
open-source tool available free of cost. In this work, we
dynamically injected faults and trained the neural network.
When new faults were injected (single and multiple), the
Table 3 Hypothesis evaluation testing data were compared with the injected fault and the
Agent 3
misclassification rate computed. Table 6 lists the injected
Agent
faults.
14
The process followed by the experiment starts with the
Agent 5
test case generation. The attributes of input were created
Agent 4
with the program’s specification which was tested and the
outputs duly generated by means of execution of the tested
program. Data now undergoes a process of pre-processing
wherein all the input attributes were normalized for which
valid. This means the testing data must have a similar
the range was determined using the maximum and the
distribution as a new set of training pairs. It had a simple
minimum values for attributes. The binary inputs and their
numerical procedure with sufficient information and the
outputs were assigned a value which was either 0 or 1.
SDS showed this in a dynamic training approach aside
Continuous output was treated differently and outputs of
from optimization and the synthesis of the ANN.

Table 4 New weights Iteration 1

w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 w12 w13 w14

Agent 1 0.3696 0.3268 0.7159 0.2133 0.393 0.777 0.2156 0.6097 0.4939 0.1794 0.6864 0.0204 0.3857 0.0013
Agent 2 0.4591 0.4029 0.5957 0.0757 0.909 0.1313 0.2831 0.4722 0.7378 0.46 0.7763 0.0091 0.9921 0.7416
Agent 3 0.5568 0.2514 0.9089 0.303 0.4037 0.5057 0.634 0.4814 0.824 0.452 0.7472 0.1321 0.2893 0.4096
Agent 4 0.8231 0.3594 0.9051 0.3727 0.9275 0.2581 0.9594 0.3086 0.5026 0.321 0.8861 0.7562 0.5486 0.7083
Agent 5 0.7351 0.4426 0.4009 0.6627 0.6503 0.4862 0.4173 0.7973 0.0676 0.8384 0.2902 0.4974 0.4307 0.0908
Agent 6 0.9028 0.4144 0.4656 0.6994 0.9641 0.8743 0.0121 0.9637 0.5065 0.4841 0.3377 0.9044 0.6421 0.667
Agent 7 0.2876 0.5461 0.1119 0.5123 0.1945 0.8309 0.5209 0.5145 0.4271 0.1946 0.1624 0.2154 0.282 0.1191
Agent 8 0.1062 0.7091 0.501 0.6142 0.1302 0.592 0.0755 0.7632 0.8943 0.1578 0.3257 0.2909 0.3262 0.9272
Agent 9 0.9308 0.437 0.9125 0.8577 0.3587 0.6401 0.9873 0.8291 0.8022 0.5111 0.8139 0.2592 0.0743 0.6207
Agent 10 0.8913 0.4401 0.4403 0.628 0.9829 0.7583 0.1058 0.2436 0.8768 0.8245 0.5114 0.1021 0.0558 0.3355
Agent 11 0.3339 0.4164 0.8864 0.5148 0.5455 0.6148 0.3903 0.2481 0.1036 0.171 0.2873 0.3764 0.8091 0.1939
Agent 12 0.9175 0.8878 0.9353 0.0507 0.3051 0.0819 0.5364 0.8415 0.3602 0.9976 0.1609 0.4501 0.0312 0.783
Agent 13 0.1445 0.7322 0.9281 0.7404 0.5285 0.9505 0.9438 0.275 0.4715 0.5866 0.4304 0.2042 0.0792 0.5197
Agent 14 0.6928 0.5842 0.0086 0.4265 0.4986 0.8908 0.9317 0.4651 0.7957 0.6193 0.3516 0.4709 0.6735 0.141
Agent 15 0.4317 0.2947 0.7789 0.3039 0.2269 0.241 0.3978 0.0818 0.6316 0.6986 0.1014 0.1733 0.2706 0.2261

123
 Author's personal copy
K. Kamaraj et al.

Table 5 Fitness value— 11

Agent 1 0.2407

Misclassificaon of correct output

Iteration 1
Agent 2 0.9955
9
Agent 3 0.8617
Agent 4 0.2254
7
Agent 5 0.2348
Agent 6 0.6448
5
Agent 7 0.0511
Agent 8 0.8563
3
Agent 9 0.136 1 2 3 4 5
Agent 10 0.3065 Number of injected faults
Agent 11 0.106 Arficial Neural Network PSO Neural Network
Agent 12 0.5357 Proposed Neural Network

Agent 13 0.6353
Fig. 2 Misclassification of correct output
Agent 14 0.7604
Agent 15 0.5178
88.81%, by 43.39% and by 42.83% than ANN at iteration
number 2, 3 and 5, respectively. The misclassification of
every example were placed within the right interval that correct output of proposed neural network performs better
was specified by the possible values and the actual number by 7.38%, by 7.36% and by 7.28% than PSO-ANN at
of intervals. The data which was processed was used in iteration number 2, 3 and 5, respectively.
place of the training dataset for the neural network and Figure 3 shows that the misclassification of wrong out-
their parameters were determined even before the training. put of proposed neural network performs better by 9.01%,
Network training included the presenting of the dataset by 37.56% and by 41.81% than ANN at iteration number 1,
completely for the one epoch and the actual number of 3 and 5, respectively. The misclassification of correct
epochs used for training had also been specified. Back- output of proposed neural network performs better by
propagation training algorithm further concluded at the 4.44%, by 4.29% and by 5.03% than PSO-ANN at iteration
time the maximum epochs were met or when the minimum number 1, 3 and 5, respectively.
error rate was achieved. The network was later used in the Figure 4 shows that the convergence of ANN occurred
form of an oracle for predicting the right outputs for the at iteration number 550 with value of 0.36.
regression tests conducted subsequently. Figure 5 shows that the convergence of proposed ANN
Figures 2, 3, 4 and 5 show the results for Misclassifi- occurred at iteration number 500 with value of 0.23.
cation of correct output, Misclassification of wrong output
and convergence, respectively. The misclassification rate 5.1 Results for synthetic dataset
states the number of an individual that are classified by the
process in a distinct category. Convergence is presented by Synthetic data is data that’s artificially manufactured rather
specific infinite series and functions of addressing a limit than produced by real-world events. Synthetic data is cre-
more and more closely as an argument (variable) of the ated algorithmically, and it is used as a stand-in for test
function increases or decreases as the number of terms of datasets of production or operational data, to authenticate
the series increases (Tables 7, 8). geometrical models and, frequently, to train machine
Figure 2 shows that the misclassification of correct learning models. Synthetic data often produce authentic
output of proposed neural network performs better by data and allows a baseline to be set. Another use of

Table 6 Injected faults

Original statement Injected faults

Use of OR operator to check whether one Removed to show only one operator
of the two argument is available
OR operator used Replaced OR operator with AND operator
Greater than or equal to operator used Replaced with greater than operator
Greater than or equal to operator used Replaced with equal operator
Greater than or equal to operator used Replaced with less than operator
AND operator used to check two criteria Replaced AND with OR operator
Argument change Changed numerical value

123
 Author's personal copy
A weight optimized artificial neural network for automated software test oracle

14 Table 8 Misclassification of wrong output

Misclassificaon of wrong output

13
12
Number of Artificial neural PSO neural Proposed neural
injected faults network network network
11
10 1 5.83 6.67 6.38
9
2 8.33 7.5 7.12
8
3 11.67 8.33 7.98
7
6 4 12.5 9.17 8.68
5 5 13.33 9.17 8.72
1 2 3 4 5
Number of injected faults

Arficial Neural Network PSO Neural Network

Proposed Neural Network Table 9 Misclassification of correct output
Number of injected faults—eligibility ANN Proposed
Fig. 3 Misclassification of wrong output output ANN

0.7 1 7.92 3.32

0.6 2 8.43 3.3
3 9.24 5.83
0.5
Convergence

4 9.87 6.61
0.4
5 11.1 7.2
0.3

0.2

0.1
50

1000
1050
0

100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950

Table 10 Misclassification of wrong output

Iteration Number Number of injected faults—eligibility ANN Proposed
ANN output ANN

Fig. 4 Convergence for ANN 1 6.15 6.92

2 8.73 7.85
0.45 3 12.34 8.65
0.4 4 12.92 9.62
0.35 5 13.81 9.53
Convergence

0.3

0.25

0.2 synthetic information is to protect the privacy and confi-

0.15 dentiality of authentic data.
0.1 Tables 9 and 10 show the results for misclassification of
50

1000
1050
0

100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950

correct output, misclassification of wrong output. Fig-

Iteration Number ures 6, 7, 8 and 9 show the results for misclassification of
Proposed ANN correct output, misclassification of wrong output and con-
vergence, respectively.
Fig. 5 Convergence for proposed SDS-ANN
Figure 6 shows that the misclassification of correct
output of proposed NN performs better by 81.85%, by
87.46%, by 45.25%, by 39.56% and by 42.62% than ANN
Table 7 Misclassification of correct output at iteration number 1, 2, 3, 4 and 5, respectively.
Number of Artificial neural PSO neural Proposed Figure 7 shows that the misclassification of wrong out-
injected faults network network neural network put of proposed NN performs better by 11.78%, by
1 7.5 3.34 3.13
10.61%, by 35.15%, by 29.28% and by 36.67% than ANN
2 8.13 3.37 3.13
at iteration number 1, 2, 3, 4 and 5, respectively.
Figure 8 shows that the convergence of ANN occurred
3 8.75 6.06 5.63
at iteration number 650 with value of 0.37.
4 9.38 6.71 6.25
Figure 9 shows that the convergence of proposed ANN
5 10.63 7.4 6.88
occurred at iteration number 500 with value of 0.24.

123
 Author's personal copy
K. Kamaraj et al.

12 0.45
Misclassification of Correct Output

10 0.4

Convergence
8
0.35
6
0.3
4
0.25
2
0.2
0

50
0

1000
1050
100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950
1 2 3 4 5
Number of Injected Faults-Eligibility Output Iteration Number
ANN Proposed ANN Proposed ANN

Fig. 6 Misclassification of correct output Fig. 9 Convergence for proposed ANN

within the same group. Results show that the misclassifi-

16
cation of correct output of proposed neural network per-
Misclassification of Wrong Output

14 forms better by 82.22%, by 88.81%, by 43.4%, by 40.1%

12 and by 42.83% than ANN at iteration number 1, 2, 3, 4 and
10
5, respectively. The misclassification of wrong output of
proposed neural network performs better by 13.44%, by
8
10.5%, by 33.4%, by 30.73% and by 36.97% than ANN at
6 iteration number 1, 2, 3, 4 and 5, respectively.
4
1 2 3 4 5
Number of Injected Faults-Eligibility Output
ANN Proposed ANN
Compliance with ethical standards

Fig. 7 Misclassification of wrong output Conflict of interest The authors declare that they have no conflict of
interest.

Human and animal rights statement No animals/humans are involved

0.7 in this research work.
0.65
0.6
References
Convergence

0.55
0.5
0.45 Bala R, Kumar DD (2017) Classification using ANN: a review. Int J
0.4
Comput Intell Res 13(7):1811–1820
Catherine JM, Djodilatchoumy S (2019) Single and multi-network
0.35
ANNs as test oracles—a comparison. Int J Comput Sci Eng
0.3 7(1):311–315
50
0

1000
1050
100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950

Duan Z, Wang Y, Xing Y (2015) Sound quality prediction of vehicle

Iteration Number interior noise under multiple working conditions using back-
ANN propagation neural network model. J Transp Technol 5(02):134
Hocrodeyro >G (2018) Using a neural network in the software
Fig. 8 Convergence for ANN testing process. Doctoral dissertation, BHTE
Hooda I, Chhillar RS (2018) Test case optimization and redundancy
reduction using GA and neural networks. Int J Electr Comput
6 Conclusion Eng 8(6):5449
Majid al-Rifaie M, Bishop JM (2013) Stochastic diffusion search
Software testing has been divided into three different review. J Behav Robot 3(2013):155–170
Mao Y, Boqin F, Li Z, Yao L (2006) Neural networks based
states: the test data generation, application of data to the automated test oracle for software testing. In: International
tested software and result evaluation. The results were conference on neural information processing. Springer, Berlin,
classified and this was received from the classifier in an pp 498–507
order that was random by default. Thus, they were listed in Pandian AP (2019) Artificial intelligence application in smart
warehousing environment for automated logistics. J Artif Intell
the preferred groups in accordance with their priorities and 1(2):63–72
here the test cases that had the same priority had been listed

123
 Author's personal copy
A weight optimized artificial neural network for automated software test oracle

Shahamiri SR, Kadir WMW, Ibrahim S (2010) A single-network Williams H, Bishop M (2014) Stochastic diffusion search: a
ANN-based oracle to verify logical software modules. In: 2010 comparison of swarm intelligence parameter estimation algo-
2nd international conference on software technology and engi- rithms with RANSAC. Algorithms 7(2):206–228
neering, vol 2, pp V2-272. IEEE
Shahamiri SR, Kadir WMNW, Ibrahim S, Hashim SZM (2011) An Publisher’s Note Springer Nature remains neutral with regard to
automated framework for software test oracle. Inf Softw Technol jurisdictional claims in published maps and institutional affiliations.
53(7):774–788
Shahamiri SR, Wan-Kadir WM, Ibrahim S, Hashim SZM (2012)
Artificial neural networks as multi-networks automated test
oracle. Autom Softw Eng 19(3):303–334

123