Journal of Engineering Research 11 (2023) 100002

Contents lists available at ScienceDirect

Journal of Engineering Research

journal homepage: www.journals.elsevier.com/journal-of-engineering-research

An automated software failure prediction technique using hybrid machine

learning algorithms ]]
]]]]]]
]]

R. Chennappana, , Vidyaathulasiramanb
⁎

a
Department of Computer Science, Periyar University, Salem, India
b
Department of Computer Science, Government Arts and Science College for Women, Bargur, India

A R T I C L E I N F O A B S T R A C T

Keywords: Many sophisticated applications have been emerged in various industries due to the rapid growth of software
Software technology technologies. Especially, the business organizations utilize the services of software-based applications to provide
Prediction a state-of-the-art service. However, fault prediction in a software is a biggest challenge that needs to be ad-
Defect dressed by the industries to improve the growth of their business. Therefore, there is a need for new techniques
Fault finding
to perform fault prediction at an early stage of software life cycle so that software defects can be avoided in later
stage. To overcome the issues in manual prediction, many prediction techniques are available that can predict
the defects automatically. All of the available techniques are based on the pattern learning that finds the fault in
the software based on the previously learned similar patterns. Even though many fault findings techniques are
available, still there are some challenges to achieve the desired effect in its performance. To overcome the issues
in currently available prediction techniques, this paper introduces an efficient software failure prediction
technique using hybrid machine learning algorithms. First part of the work performs feature selection with an
improved fitness function by utilizing genetic algorithm (GA) to optimize the features in the data set. After
selecting the better features, Decision Tree algorithm is used as a classification technique for processing that
features. The work compares the GA-DT based hybrid model with the currently available machine learning
model such as RCSOLDA-RIR and WPA-PSO for the prediction of software failure. The outcome of the experi-
mental analysis shows that the proposed model achieves better accuracy than the currently available model.

Introduction selecting a preprocessing technique for a particular data set is difficult

since many numbers of preprocessing schemes are available. Therefore,
The quality of Industrial applications has been improved greatly due it is very difficult to select an appropriate prediction model from the
to the rapid development of software technology. This ever-expanding number of available models. At present, ensemble-based learning
technology improves the growth of the organization with the help of methods are popular for improving the prediction accuracy [10]. Many
fault free software. Software defect prediction [5] is necessary for the machine learning models are available such as random forest, decision
developers to improve the quality of the software. Even a single defect tree, SVM, Bayesian and neural networks. Among these algorithms,
in software can creates a major problem that leads to loss of the busi- neural network-based algorithms support processing of high dimen-
ness life. Even though manual software testing is utilized in the in- sional data. In this paper, we propose a hybrid machine learning
dustries, they are very complex in nature and requires manpower to technique for the prediction of software defects. This paper is the ex-
perform software testing. However, algorithms such as Software failure tended version of our previous work where we utilized Ruzchika in-
prediction is recently performed by many automated prediction dexive regression (RCSOLDA-RIR) technique for improvising software
schemes [15]. They are useful in selecting appropriate prediction quality. The previous works are focusing only on the better prediction
models and other necessary techniques automatically to predict the of software failures but the time characteristics such as response time
number of defects in the software module. However, there should be an are not considered properly. Only few works are existed [6,7,16] by
appropriate model for a specific data set since the number of para- considering the importance of time factor. To overcome these problems,
meters in each data set is different (Herbold et.al 2018). Similarly, the proposed work introduces an improved cuckoo search algorithm

⁎
Corresponding author.
E-mail address: [email protected] (R. Chennappan).

https://doi.org/10.1016/j.jer.2023.100002
Received 21 September 2022; Received in revised form 12 November 2022; Accepted 21 November 2022
2307-1877/© 2023 The Authors. Published by Elsevier B.V. on behalf of Kuwait University. This is an open access article under the CC BY license (http://
creativecommons.org/licenses/by/4.0/).
R. Chennappan and Vidyaathulasiraman Journal of Engineering Research 11 (2023) 100002

Fig. 1. Architecture of neural network.

that performs the optimization of initial vector and thresholds. The which are not lister here because they all chosen same samples but
remaining sections are organized in such a way that the Related work different learning techniques. From the above literatures we reviewed,
section discusses the related works, The proposed work section in- we understand that most of the work focused only on improving the
troduces the proposed system, Experimental analysis section describes prediction accuracy and other metrics such as F-measure, the problems
the experimental analysis and the work is concluded in Discussion are not well addressed in terms of time and computational overheads.
section. To overcome the above issues, we propose a hybrid machine learning
technique for software defect prediction.

Related work
The proposed work
Many software defect prediction techniques have been emerged for
Software defect prediction using automated tools is a challenging
the past decades to create a reliable and error free software. However,
task in software engineering. Even though many prediction models are
the appropriate model that guarantees the quality of prediction is still
available as discussed in the above sections, each model is proportional
questionable. All the existing works can be divided in to two parts: first,
to one another. One model performs well in improving the accuracy but
all focused on improving the accuracy by selecting combined feature
it exhibits huge computational cost while the other model reduces the
extraction techniques, then hand-craft metrics are developed for the
time and cost but not the other performance metrics. To overcome these
detection of defective characteristics [1]. This section discusses the is-
issues, we propose a hybrid approach where we utilized decision tree
sues and challenges in the existing models. Zheng et al. [17], proposed a
and genetic algorithm along with cuckoo search algorithm to improve
software failure prediction technique to increase the quality of software
the quality of prediction while performing the fault findings of soft-
systems. This work identifies the random patterns from the unbalanced
ware. The work applies genetic algorithm on neural network for better
data of faulty software databases. The proposed work can be used to
feature selection. In addition to this, the work also utilized the im-
create reliable classifiers for industrial applications. Even though the
proved cuckoo search algorithm to perform optimization of initial
work performs well in predicting the defects, the time characteristics
vectors and thresholds. Fig. 1 shows the basic architecture of the neural
are not well addressed. Song et al. [14] identified the critical defect in
network.
the software using neural networks based algorithms. The work pro-
posed an efficient prediction model using Elman neural network. Au-
thors claims that the work improves F-measure values with 50 % of Dataset details
growth. However, the work does not guarantee the security of software
defect prediction. Qiao et al. [12], describes the importance of reliable The work uses various datasets from different open sources. The old
software and the defect prediction is done using deep learning strate- version of data is used for training and new version is used for testing.
gies. Authors claims that the work reduces the error rate by 14 % and Mainly the datasets are collected from the 3 sources [13–15] where
the correlation metrics is increased by 8 %. However, the research fo- dataset 1 has 8 input parameters, dataset 2 has 22 input parameters and
cused only on module level prediction which does not guarantee the dataset 3 has 20 parameters.
efficient fault findings in the level of severity. Aquil [2] introduced a
stacking classifier model for building an efficient software defect pre- Feature selection
diction technique [4] and the work uses 13 datasets for evaluating the
different prediction methods based on ensemble classifiers, and su- Data cleaning, reduction, and filling the missing values are done by
pervised and unsupervised learning methods. Ensemble classifiers data preprocessing technique. After cleaning, dimensionality reduction
achieves higher accuracy than the supervised and unsupervised is performed to select the exact features in the data set. Genetic algo-
learning methods. The proposed STC model improves the F-measure rithm is used for the optimal selection of features. Mostly the datasets of
growth among all the ensemble classifiers. However, time and resources software defect consist of high dimensional features. To improve the
are not properly dealt with while performing the data processing. Kavya performance of classifier, it is important to select relevant features.
K.S, proposed a deep boost classifier which performs the defect finding Genetic algorithm plays an important role in choosing exact features.
process efficiently. They used genetic algorithm for feature selection [3] First it generates initial population to compute the fitness score of the
and SLS for class distribution. Even though the work achieves better features. This score is used to maximize the area under ROC curve.
performance in terms of accuracy, the unique features of datasets are Fitness function playa an important role in GA to identify the optimal
not taken in to account. Apart from the above literatures, still there are solution from the number of available solutions. Fitness function is
many prediction models [8,9,11] are existed for defect prediction calculated using the below formula

2
R. Chennappan and Vidyaathulasiraman Journal of Engineering Research 11 (2023) 100002

Fig. 2. Prediction model.

f = WS f + WAc U + WV xAccuracy (1) Improved Cuckoo search algorithm

S S
x =1 V x=1 f
where U = 1 uf
and f = 1 uf
. This algorithm was introduced by Yang et al. [16] based on the
WAc represents the weight of classification accuracy, f is the fitness behavior of cuckoo birds. This algorithm is mainly used for optimiza-
score, Wf represents the weight of fitness score. Fig. 2 shows the pre- tion and the merits of this algorithm are simple operations, and strong
diction model. optimization capability. This algorithm has only few parameters and at
the same time it performs better optimization than the algorithm such
as swarm optimization algorithm. According to the algorithm, the
Classification
searching of solution can be affected by the probability P of discovery.
Converging to better solution is difficult in large size of P. Also, con-
The datasets of software defect model will not have equal number of
vergence speed is less in small size of P. The basic idea of this algorithm
occurrences and they need to be balanced. If the data are unbalanced,
is that each cuckoo lays only one egg, the nest should be always
then the algorithm will be biased for maximum of classes. The classifier
available, cuckoo can found dumped egg inside the shell with the P
ignores the features that are not balanced and considered them as noise.
€(0,1). The algorithm considers local search and global search. If an
This section performs partitions for the input data based on attribute
optimal solution exists from the available solution, a new solution for
values. Data are splatted into number of leaves. The leaves are assigned
local search can be computed as follows
to specific classes depends on attribute values. Decision tree based
classification is used in the paper. This type of classification follows top- Mii + 1 = Mi j + as H (Pa ) (Mi j Mik ) (3)
down approach based on the greedy algorithm. In this process, the
optimal split is considered as current node and the process is repeated where Mi and
j
represents the solution that are randomly chosen and
Mik
until the following conditions met. H represents the Heaviside function.
If an optimal solution exists for the global search, then it can be
|fA = u |
f (G, A) = f (s ) f (GA = u ) computed by
u U (s )
|A| (2)
Mii + 1 = Mi j + aL (f , ) (4)
The steps involved in the classification model is shown in the fol-
lowing algorithm In the above equation, L represents the behavior of the problem. The
Input: DT Classifier (G, A, S, U) algorithm can be expressed in a following way
Output: defective and non defective labels Step1: Initial values f(m), m-(m1, m2, m3…mn)
Step1: decision tree is formulated with root node Step2: Initial Population Pi, i = 1,2,3…. n
Step2: if no further partition, then Step3: evaluate fitness scores
Step3: consider the current node as leaf l Step4: find the best solution from P
Step4: else Step5: for i ranges from 1 to maxvalue
Step5: ¥ u € U find the best split with f(G, A) Step6: cuckoo nest i = 1,2…. maxvalue in random selection
Step6: label this value with L Step7: choose the best fitness value f1
Step7: for each value of Ui Step8: End for
Step8: construct a sub tree Step8: for j ranges from 1 to n
Step9: Ensure all the labels are connected by edge Step9: opposite nest k = (1, 2…. n) opf1
Step10: End for Step10: End for

3
R. Chennappan and Vidyaathulasiraman Journal of Engineering Research 11 (2023) 100002

Table 1
Details of the projects.

Project description Version Files Defective rate (%)

A template engine based on Java 1.4, 1.5, 1.6.1 214 59

Java based build tool 1.40, 1.50, 1.60, 1.70 350 20
Microsoft format files 1.5, 1.6.1, 17.2 350 50
Logging library 1.0, 1.1, 1.2, 1.3 150 49.9
Integration framework 1.2, 1. V5, 1. V8 700 19
Transforming library 1.5.0, 2.0, 3.1 840 52
XML parser 1.0, 1.1, 1.2, 1.3 399 37

Table 2
Correlation coefficient analysis.

Metrics Maxcc avgcc wmc Amc Rfc Cam moa cbo

Maxcc 1 0.354 1 0.798 0.845 0.378 0.632 0.471

avgcc 0.689 1 0.23 0.362 0.362 0.542 0.435 0.378
wmc 0.397 0.523 0.0486 0.854 0.672 0.362 0.412 0.362
Amc 0.32 0.672 0.547 1 0.652 0.627 0.729 0.512
Rfc 0.362 0.256 0.753 0.632 1 0.372 0.652 0.672
Cam 0.632 0.362 0.378 0.672 0.523 1 0.378 0.652
moa 0.412 0.412 0.2964 0.741 0.423 0.368 1 0.562
cbo -0.022 -0.029 0.765 0.378 0.512 0.465 0.652 1

Step11: if (opf1 < f1) the dataset contains different versions and the overall failure rate
Step12: replace f1 by opf1 identified for the project is 59 %. The work uses 10-fold cross validation
Step13: End if for the evaluation of the proposed model. While performing cross va-
After finding the initial population and thresholds, mean square lidation, the training data is divided into training set and testing set in
function is used as fitness function toe evaluate the convergence con- the ratio of 80:10.
dition. The process is repeated till it satisfies the convergence condition. To measure the performance of the proposed method, the metrics
As in the above condition split criteria can be found using such as F-score, precision, recall and ROC are utilized. The commonly
used metrics such as precision, f-measure and recall are utilized to
|f A = u |
f (s ) f (GA = u ) detect the failures in the software. Precision represents the accurate
u U (s )
|A| (5) prediction from the available positive prediction and it is given as
Now the attribute based difference can be determined using TP
Prediction =
TP + FP (7)
0I 1[M ] = I 2[M }
d (M , I 1, I 2) =
1otherwise (6) where TP is the number of files that are predicted as defective files are
truly defective files, and FP represent the number of files that are
predicted as defective files are not truly defective files. At the same time
Experimental analysis FN is false negative which represent that the files predicted as non
defective files are truly defective files.
The above sections introduced the modeling of the proposed algo- Recall is the another performance metrics which represents the
rithm and it is evident that the model is theoretically feasible. The work number of recalling that they are predicted as positive instances.
uses PROMISE dataset which contains 7 projects and 27 versions. The
TP
implementation of the work is done using MATLAB. Each project in the Recall =
TP + FN (8)
dataset has many versions and the defective rate of the project is 59 %.
Table 1 describes the information related to these projects. Only the metrics such as prediction and recall is not sufficient to
From Table 1, we observed that temporal characteristics are not evaluate the mode. F-measure is added for further evaluation
directly reflected because of the complex nature of the data in each
Precision*recall
version. In order to reduce this, the version granularity is sliced to the F measure = 2*
Precision + recall (9)
data that are existed and if the field is set to 1 if any defects found,
otherwise 0. We used cuckoo search for the selection of parameters with The performance of the proposed hybrid approach is analyzed using
minimum correlation to predict software defects. As in Table 2 the confusion matrix as in Table 3.
correlation between maxcc and avgcc is 0.8 and wmc and rfc is 0.9. From Table 3 we can formulate the accuracy as
Finally, the attributes are given as input to the network. while the al-
ready processed data are utilized for training it is identified that the TruePostive + TrueNegative
Accuracy =
errors are large. Therefore, normalization is important for the processed TP + TN + FP + FN (10)
data to improve the performance of the model. Table 2 shows the
correlation coefficient of the influencing metrics. Table 3
Confusion matrix.

Discussion Actual Defects predicted No defects predicted

Defective files TP FN
The work evaluated the proposed technique in PROMISE dataset
Non defective files FP TN
which contains 7 projects and 27 versions. All the projects included in

4
R. Chennappan and Vidyaathulasiraman Journal of Engineering Research 11 (2023) 100002

Funding and Acknowledgements

This research did not receive any specific grant from funding
agencies in the public, commercial, or not-for-profit sectors.

Declaration of Competing Interest

The authors declare that they have no known competing financial

interests or personal relationships that could have appeared to influ-
ence the work reported in this paper.

References

Fig. 3. Performance of prediction model. [1] F. Alghanim, M. Azzeh, A. El-Hassan, H. Qattous, Software defect density prediction
using deep learning, IEEE Access 10 (2022) 114629–114641.
[2] Aquil, Ishak, Predicting software defects using machine learning techniques,
IJATCSE 9 (4) (2020) 6609–6616.
[3] M. Belouch, S. Elhadaj, M. Idhammad, A hybrid filter-wrapper feature selection
where TP, TN, FP, FN are true positive, true negative, false positive, and method for DDoS detection in cloud computing, Intell. Data Anal. 22 (2018)
false negative respectively. Fig. 3 shows the performance of prediction 1209–1226.
models. [4] S. Herbold, A. Trautsch, J. Grabowski, A comparative study to benchmark cross-
project defect prediction approaches, IEEE Trans. Softw. Eng. 44 (2018) 811–833.
The experiment is conducted to check the accuracy of the compared [5] K.S. Kavya, Y. Prasanth, An ensemble DeepBoost classifier for software defect
models. The models such as RCSOLDA-RIR, WPA- PSO are compared prediction, Int. J. Adv. Trends Comput. Sci. Eng. 9 (2) (2020) 2021–2028.
with the proposed GA-DT approach. From the analysis, it is observed [6] M. Kondo, C.P. Bezemer, Y. Kamei, A.E. Hassan, O. Mizuno, The impact of feature
reduction techniques on defect prediction models, Empir. Softw. Eng. (2019) 1–39.
that the proposed model achieves better accuracy than the other [7] S.C. Kumar, R.S. Sirisati, S. Thonukunuri, Software defect prediction using opti-
models. Different activation function such as ReLu, Tanh, Sigmoid are mized cuckoo search based nature-inspired technique, Smart Computing
used. ReLu function used in the work improves the accuracy greatly due Techniques and Applications. Smart Innovation, Systems and Technologies, 224
Springer, Singapore, 2021.
to the fact that it supports the property of non-linearity which outputs
[8] T. Menzies, R. Krishna, D. Pryor, The promise repository of empirical software
the input directly if it is positive. ReLu is disparate at all points except at engineering data. 〈http://openscience.us/repo〉, North Carolina State University,
0. During the experimental analysis, it is also evident that the improved Department of Computer Science, 2015.
[9] A. Monden, J. Keung, S. Morisaki, Y. Kamei, K. Matsumoto, A heuristic rule re-
cuckoo search performs better in providing higher reliability when we
duction approach to software fault-proneness prediction, in: Proc. APSEC2012,
choose the GA-DT based hybrid approach. 2012, pp. 838–47.
[10] S. Nakariyakul, High-dimensional hybrid feature selection using interaction in-
formation-guided search, Knowl. Based Syst. 145 (2018) 59–66.
Conclusion [11] A. Ouaarab, B. Ahiod, X.S. Yang, Discrete Cuckoo search algorithm for the travel-
ling salesman problem, Neural Comput. Appl. 24 (7–8) (2014) 1659–1669.
[12] Qiao, Song, Qasim, Guo, Deep learning based software defect prediction, Neuro
In this work, GA-DT based hybrid machine learning approach is Comput. 385 (2020) 100–110.
utilized along with cuckoo search algorithm. The work also utilized [13] Software Engineering Data Repository for Research and Education, 〈http://
analytics.jpn.org/SEdata/〉, Software Measurement and Analytics Laboratory,
datasets from different sources to predict the software defects. The work Okayama University, 2018.
improved the performance of prediction process with the help of im- [14] Song, Lv, Hu, He, Software Defect Prediction Based on Elman Neural Network and
proved CS algorithm through optimization. The evaluation of the pro- Cuckoo Search Algorithm, Math. Probl. Eng., 2021.
[15] H. Tong, B. Liu, S. Wang, Software defect prediction using stacked denoising au-
posed work is done through 7 projects and results proved that the
toencoders and two-stage ensemble learning, Inf. Softw. Technol. 96 (2018)
proposed GA-DT based hybrid approach outperforms other algorithms 94–111.
in terms of accuracy. In addition to this, GA-DT with CS algorithm [16] Y. Zhen, W. Liu, Dongsheng, Z. Wei, Parameter estimation of software reliability
achieves good performance in terms of F-measure with an increased model and prediction based on hybrid Wolf pack algorithm and particle swarm
optimization, IEEE Access 8 (2020) 29354–29369.
rate of 59 %. As part of the future work, we plan to increase the projects [17] Zheng, Gai, Yu, Zou, Gao, Software Defect Prediction Based on Fuzzy Weighted
written in different programming languages. Extreme Learning Machine with Relative Density Information, Hindawi, 2020.