Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Integrating Random Forest, MLP and DBN in a

Hybrid Ensemble Model for Accurate Breast
Cancer Detection
Sirisha Yamania, Ziaul Haque Choudhuryb
Vignan’s Foundation for Science Technology and Research (Deemed to be University),
Department of Information Technology, Guntur, AP, India.

Abstract:- Worldwide, breast cancer is the leading cause of breast cancer is crucial for improving patient outcomes
of death among women. Early detection is essential for and reducing mortality rates. Machine learning algorithms
reducing aggressive treatments and increasing survival have been increasingly used for breast cancer classification,
rates. Machine learning algorithms have demonstrated detection, diagnosis, and prognosis prediction in recent years.
their ability to diagnose breast cancer accurately from They have shown great potential for improving the accuracy
medical imaging data. However, no individual algorithm of breast cancer detection and diagnosis, as well as reducing
can consistently provide optimal results. To address this, the time and cost of diagnosis. A growing number of studies
researchers have proposed hybrid ensemble learning have reported promising results using various machine
models that combine multiple approaches. In this study, learning techniques, such as deep learning, decision trees,
we have proposed a hybrid ensemble learning model that random forests, support vector machines, and boosting
combines three powerful algorithms, the Random Forest algorithms [1, 3-6, 9, 11, 14, 16, 18-20].
(RF), Multilayer Perceptron (MLP) and the Deep Belief
Network (DBN) to diagnose breast cancer accurately. The Deep learning, a type of machine learning technique
MLP and DBN algorithm teaches non-linear correlations that has shown remarkable performance in various image and
between features and labels, while the RF algorithm uses signal processing applications, has attracted significant
a random subset of features to create multiple decision attention in breast cancer detection and diagnosis.
trees and combine their predictions. The proposed hybrid Convolutional neural networks (CNNs) are a type of deep
model trains the RF, MLP and DBN models separately on learning algorithm that has been widely used for image
a breast cancer dataset and integrates them using a recognition and classification, including breast cancer
weighted average method for the final prediction. Cross- diagnosis. Several studies have reported the successful
validation is used to establish the optimal weights for the application of CNNs in breast cancer detection and diagnosis,
RF, MLP and DBN models. Our proposed hybrid model achieving high accuracy rates and outperforming traditional
achieves an accuracy rate of 96.5% on a publicly machine learning algorithms [5, 7-9, 14, 20]. Despite the
available breast cancer dataset, outperforming the growing interest in machine learning-based breast cancer
individual RF, MLP and DBN models, which achieved detection and diagnosis, there are still several challenges that
accuracy rates of 93.9%, 91.3% and 97.5 % respectively. need to be addressed. One of the major challenges is the
Our findings suggest that the hybrid ensemble learning limited availability of high-quality annotated data, which is
model is a more reliable and accurate tool for breast essential for developing accurate machine learning models.
cancer identification than individual machine learning Another challenge is the lack of interpretability and
algorithms. This model has significant potential for early transparency of machine learning models, which can hinder
breast cancer identification in clinical settings, leading to their clinical adoption.
better patient outcomes and reduced healthcare costs.
Our research demonstrates the effectiveness of hybrid In recent years, there have been efforts to develop
ensemble learning models in improving breast cancer machine learning models that are more transparent and
identification accuracy. interpretable, such as decision trees and rule-based models.
These models can provide insights into the decision-making
Keywords:- Breast Cancer, Machine Learning, Random process of the algorithm, which can help clinicians
Forest, MLP, DBN, Ensemble Learning, Hybrid Model. understand and trust the results. In addition to breast cancer
detection and diagnosis, machine learning algorithms have
I. INTRODUCTION also been used for breast cancer prognosis prediction.
Prognosis prediction is critical for determining the most
Breast cancer is a significant public health concern appropriate treatment plan for each patient and improving
worldwide. It is the most common cancer among women and their long-term survival. Several studies have reported the
the second leading cause of cancer death among women. successful application of machine learning algorithms in
According to the American Cancer Society, an estimated breast cancer prognosis prediction, achieving high accuracy
281,550 new cases of invasive breast cancer will be rates and outperforming traditional prognostic models [11,
diagnosed in the United States in 2021, and approximately 18, 19].
43,600 women will die from the disease [2]. Early detection

IJISRT23JUL955 www.ijisrt.com 1556

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
disease [36]. Pathological complete response rates following
In conclusion, machine learning algorithms have shown breast cancer neoadjuvant chemotherapy have been used to
great potential in improving breast cancer detection, predict long-term outcomes and select patients for breast
diagnosis, and prognosis prediction. The development of conservation [37]. The 70-gene signature has also been
accurate and transparent machine learning models can help proposed as an aid to treatment decisions in early-stage breast
clinicians make more informed decisions and improve patient cancer [38]. Finally, Burstein et al. [39] proposed a
outcomes. However, further research is needed to overcome framework for estimating the benefits of therapy for early-
the challenges of limited data availability and model stage breast cancer.
interpretability, as well as to validate the performance of
machine learning models in larger and more diverse III. PROPOSED SYSTEM
populations.
The proposed approach in this project is to use an
In this paper, we have organized our work into several ensemble model that combines the strengths of three different
sections. Part 2 presents a thorough literature review of the classifiers: the Random Forest (RF), the Multi-Layer
topic, while Section 3 describes the recommended techniques Perceptron (MLP), and the Deep Belief Network (DBN). The
in detail. We discuss the datasets used in our experiments in RF algorithm is a tree-based method that can handle both
Section 4. The results of our experiments are reported in categorical and continuous data and is good at capturing
Sections 5 and 6, which collectively conclude the paper. nonlinear relationships between variables. On the other hand,
the MLP algorithm is a neural network-based approach that is
II. RELATED WORK particularly effective at modelling high-dimensional and
nonlinear data. The DBN is a type of deep neural network
Breast cancer is a prevalent disease affecting women that can learn hierarchical representations of data. To create
worldwide. According to the American Cancer Society [21], the ensemble model, we trained each of the three classifiers
breast cancer is the most common cancer in women, with on the same dataset independently. We then used a weighted
about 281,550 new cases and 43,600 deaths expected in the average method to combine the predictions of the three
United States alone in 2021. The use of deep learning classifiers, with the weights optimized through cross-
methods for breast cancer diagnosis has gained attention due validation. The idea behind this approach is that by
to their high accuracy and efficiency in detecting cancerous combining the strengths of the three classifiers, we can
tissue. Li and Yu [22] proposed a deep learning-based mitigate their individual weaknesses and improve the overall
diagnosis model using MRI images. In addition, Niknazar et performance of the ensemble model. The RF algorithm is
al. [23] presented a review of the various machine learning good at handling categorical and continuous data, the MLP is
algorithms used for breast cancer detection. Gandomkar et al. effective at modelling high-dimensional and nonlinear data,
[24] discussed the role of radiologists in breast cancer and the DBN can learn hierarchical representations of data.
screening in the age of artificial intelligence.
We evaluated the performance of the ensemble model
Breast cancer screening is an essential tool for early on multiple benchmark datasets and compared it with the
detection and has been shown to reduce mortality rates [25]. individual RF, MLP, and DBN classifiers, as well as other
Khan et al. [26] presented a systematic review and meta- state-of-the-art algorithms. The ensemble model consistently
analysis of machine learning-based breast cancer risk outperformed the individual classifiers and demonstrated
prediction models. Nagi and Mcclymont [27] provided a competitive performance compared to alternative approaches.
comprehensive review of the various machine learning In conclusion, the proposed ensemble approach provides a
techniques used for breast cancer diagnosis. Rizwan et al. practical and versatile method for improving the accuracy of
[28] conducted a systematic review and meta-analysis of the supervised learning tasks. By combining multiple models
various machine learning algorithms used for breast cancer with different strengths, we can overcome the limitations of
detection. Samala et al. [29] proposed a transfer learning individual models and improve overall performance.
deep convolutional neural network that integrates multiple
tasks for the computer-aided diagnosis of breast cancer on
mammograms. Integration of imaging and clinical data has
also been proposed to improve cancer outcomes [30].

Breast cancer is a complex disease with various genetic

factors contributing to its development [31]. Kim and Kim
[32] discussed the role of epigenetics in breast cancer
heterogeneity. Hwang et al. [33] identified genetic factors
associated with breast cancer susceptibility using a
multilocus genome-wide association study. Molecular
portraits of human breast tumors have been identified, with
molecular subtypes associated with distinct clinical outcomes
[34]. The Ki67 index and HER2 status have also been used to
predict the prognosis of patients with luminal B breast cancer
[35]. Treatment of breast cancer depends on the stage of the

IJISRT23JUL955 www.ijisrt.com 1557

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
be expanded to its full depth, which may also cause
overfitting.
 min_samples_split: The min amount of data points
needed to divide a node is referred to as the "minimum
samples split" in decision tree algorithms. Setting a higher
value for this parameter can make the model more
cautious and mitigate the risk of overfitting. By default,
the value is set to 2.
 min_samples_leaf: One can specify the minimum number
of times a leaf node must be created in a decision tree. A
higher value can produce a more cautious model that
mitigates overfitting. Typically, this value is set to 1 by
default.

 max_features: The "max_features" parameter is

responsible for deciding the number of features that each
node should take into account when splitting. To avoid
overfitting and enhance the variety of the ensemble, it is
advisable to decrease the value of this parameter. The
default value of "auto" indicates that all features will be
used for splitting.
Fig 1. Proposed Architectural Diagram  random_state: The seed used by the random number
generator. This parameter is used to maintain
IV. METHODOLOGIES repeatability.

A. Random Forest Here is the formula for the classification of RF:

A well-liked ensemble learning approach called
Random Forest (RF) combines different decision trees to The process for constructing a random forest involves
boost prediction accuracy. To create a final forecast, RF the following steps for each tree in the forest:
builds a lot of decision trees and combines their predictions.  Generate a sample of n observations from the training set
This section will cover the RF approach as well as several using bootstrap sampling technique, where each
frequently used hyperparameters. observation is selected randomly with replacement.
 Randomly select a subset of m characteristics from the
 Methodology total features.
Bagging is a technique in which multiple decision trees  Construct a decision tree by utilizing a bootstrap sample
are generated on randomly selected samples with and the chosen features.
replacement from a dataset. The predictions of all trees are
combined to produce the final prediction in the Random For each test sample:
Forest machine learning algorithm. Although this technique  Aggregate the predictions of all the trees to form a final
helps reduce model variance, complex decision trees can lead prediction.
to overfitting. This is because the model is tuned to the
training data so closely that it cannot accurately predict new, The output of each tree is calculated using the following
unseen data. In order to solve this issue, RF adds two more formula:
sources of randomization. At each node, only a random
subset of features is taken into account for splitting. As a For a given input x, let p(y|t,x) be the probability of
result, the ensemble's diversity increases while the correlation class y given that the input x belongs to the region t of the
between trees decreases. Second, without pruning, each tree tree.
is allowed to reach its full potential. As a result, the training
set becomes overfit; however, this is mitigated by combining The output of the tree is then given by:
predictions from several trees. 𝑓 (𝑥) = 𝑎𝑟𝑔𝑚𝑎𝑥(𝑝(𝑦|𝑡1, 𝑋) + 𝑝(𝑦|𝑡2, 𝑋)
+ ⋯ 𝑝(𝑦|𝑡𝑛, 𝑋))
 Hyperparameters (1)
 n_estimators: This parameter determines the number of where ti represents the region of the i-th tree.
trees present in the forest. A higher value for this
parameter results in a more stable model, but it also The Random Forest algorithm is a robust ensemble
increases the computational cost. By default, this learning technique that integrates the strengths of decision
parameter is set to 100. trees and bagging. It is frequently utilized for classification
 max_depth: This determines the max no of the levels in and regression tasks due to its exceptional accuracy and
the tree. If the value is set to a larger number, the model's robustness. Fine-tuning the hyperparameters of the RF model
complexity will increase, which may result in overfitting. can enhance its performance on specific datasets.
On the other hand, if the value is set to None, the tree will

IJISRT23JUL955 www.ijisrt.com 1558

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
B. MLP (Multi-Layer Perceptron) 𝑎𝑗𝑖 = 𝑔(𝑧𝑗𝑙 )𝑓𝑜𝑟 𝑙 ≥ 2, 𝑗 = 1, … . , 𝑛𝑙
For classification purposes, a common neural network is (3)
the Multi-Layer Perceptron (MLP) method. It is made up of a
number of node layers that are connected to one another
Output: 𝑦̂ = 𝑓(∑𝑗 𝑤𝑗𝐿+1 𝑎𝑗𝐿 + 𝑏 𝐿+1 )
through links between each layer. The input data is converted
by the nodes in each layer using a non-linear activation (4)
function, and the classification result is generated using the
final layer's output. The first step in When using MLP for Where,
classification, before being tested on another set of data, the 𝑧𝑗𝑙 is the weighted input to neuron j in layer L,
model is first trained on a set of training data. To lessen the 𝑖
𝑤𝑗𝑘 is the weight between neuron k in layer l-1
hole between the normal and real result, the model modifies
the weights of connections between nodes in each layer and neuron j in layer L, 𝑎𝑘𝑙−1 s the output of neuron k in layer
during training. Backpropagation is used to determine the l-1,𝑏𝑗𝑙 is the bias of neuron j in layer l, g is the activation
error gradient in relation to each weight, and then alter the function, nl is the number of neurons in layer l, f is the output
weights to reduce error. function,
𝑦̂ is the predicted output, 𝑤𝑗𝐿+1 is the weight between
 Hyperparameters neuron j in the last hidden layer and the output neuron, and
𝑏𝐿+1 is the bias of the output neuron.
 hidden_layer_sizes:
In scikit-learn's MLP Classifier, the boundary stowed
C. DBN (Deep Belief Network)
away layer sizes decide the quantity of hubs in the brain
Deep Belief Networks (DBNs) are a type of artificial
organization's secret layer. By default, a single hidden layer
neural network that can be used for unsupervised feature
with 100 nodes is used. However, this parameter can be
learning, as well as for supervised classification and
customized by passing an integer tuple. The tuple indicates
regression tasks. DBNs are composed of multiple layers of
the number of nodes in each hidden layer, allowing users to
Restricted Boltzmann Machines (RBMs), which are
construct networks with multiple hidden layers. For example,
unsupervised generative models that can learn useful features
setting the parameter to (100, 100) would create a network
from raw input data.
with two hidden layers, each containing 100 nodes.
DBNs consist of an input layer, several hidden layers of
 activation: RBMs, and an output layer. The input layer receives the raw
This option provides the activation function to be input data, while the hidden layers are responsible for
utilized by the nodes in each layer. The activation function is learning increasingly complex and abstract features from the
responsible for adding non-linearity into the model, which input data. The output layer performs the final classification
permits it to learn complex examples in the information. The or regression task. The connections between the layers are
default activation function in scikit-learn is 'relu', but other learned through a process called unsupervised pre-training,
options include 'logistic' and 'tanh'. where each layer is trained in an unsupervised manner to
learn features from the layer below it. Once the RBMs have
 alpha: been trained, the entire network can be fine-tuned using
This parameter controls the regularization strength of supervised learning to improve its performance on the final
the model, which helps to prevent overfitting. It is a scalar task.
value that multiplies the L2 penalty term in the loss function.
Higher values of alpha lead to stronger regularization, which The input layer of a DBN consists of visible units that
can improve generalization performance but may also reduce directly correspond to the input data. If the input data has n
the model's ability to fit the training data. features, then the input layer will have n visible units. The
output layer of a DBN depends on the task being performed.
Empirical experimentation led us to choose the For example, in a classification task with k classes, the output
hyperparameters hidden_layer_sizes=(100, 100), alpha=0.1, layer will have k SoftMax units, one for each class.
and activation='relu' when instantiating the MLPClassifier.
These values have proven to be effective for various The input layer can be represented mathematically as:
classification tasks. To prepare the model on the preparation
information, we utilized the fit() technique, while the 𝑖
predict() method was utilized to assess its performance on the 𝑣1
test data. Ultimately, the model's classification report and 𝑣2
accuracy on the test set were printed to the console. The 𝑣
V= 3 (5)
formula for the multilayer perceptron (MLP) can be .
expressed as follows: .
[.]
𝑣𝑛
𝑧𝑗𝑙 = ∑𝑘 𝑤𝑗𝑘
𝑖
𝑎𝑘𝑙−1 + 𝑏𝑗𝑙 𝑓𝑜𝑟 𝑙 ≥ 2, 𝑗 = 1, … . , 𝑛𝑙
where v is a column vector of visible units, and v[i] is the
(2) activation of the i-th visible unit.

IJISRT23JUL955 www.ijisrt.com 1559

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
The output layer can be represented mathematically as: a. Visualizing the Data
A scatter plot representing the relationship between two
𝑦1 chosen features (mean radius and mean texture) from the
Y=[ 𝑦 ] (6) WDBC dataset is created through data visualization. The plot
2
is color-coded by diagnosis, i.e., malignant or benign, to help
where [y] is a column vector of output units, and y[i] is the understand the correlation between the two features and their
activation of the ith SoftMax unit. relationship with the diagnosis. Further, plotting a histogram
is used to show how the mean radius feature's values are
In between the input and output layers, a DBN can have distributed across the two diagnoses. These visualizations
multiple hidden layers. The number of hidden layers, as well offer crucial insights into the dataset and aid in its analysis
as the number of hidden units in each layer, are and modeling.
hyperparameters that need to be chosen before training the
DBN.

V. EXPERIMENTAL RESULTS

In our research, the dataset for the Wisconsin

Diagnostic Breast Cancer (WDBC) is a well-established and
widely used benchmark dataset for the classification of breast
cancer in the field of machine learning, and we used it in our
research. The 569 samples in the dataset, which represent
breast mass lesions, was sourced from the machine learning
repository of the University of California, Irvine (UCI). For
each sample, 30 features were derived from digitized images
of the breast mass, including the mean, standard deviation,
and worst (the three largest values) of ten distinct cell Fig 3. Diagnoses types and their features
nucleus features that are visible in the picture. These
characteristics included the diagnosis of the breast mass (M = b. Classification Performance Analysis
malignant, B = benign), the radius, texture, perimeter, area, In the classification project, the dataset was parceled
smoothness, compactness, concavity, symmetry, and fractal into preparing and testing sets with a 70:30 proportion. The
dimension. To ensure that there were no missing values or preparation set, involving 398 examples, was utilized to
outliers, the breast cancer dataset underwent a thorough prepare the AI models, while the remaining 171 samples in
cleaning and preprocessing. The dataset had a total of 569 the testing set were reserved to evaluate the models'
samples, consisting of 357 benign and 212 malignant cases, performance on unseen data. The aim was to allow the
thus making it a well-balanced dataset. The study aimed to models to learn from a significant dataset and test their
foster an order model that could precisely foresee the ability to generalize on a separate set. The partitioning was
determination of bosom mass injuries in light of the given done randomly and ensured that the classes were equally
components. To achieve this objective, we divided the represented in both the training and testing sets.
dataset into a training set of 379 samples and a test set of 190
samples using a 2:1 ratio. The data's distribution can be C. Confusion Matrix
found. in Figure 2. A measure called precision is used to analyse how
effectively a classification model is operating. It assesses the
model's potential to avoid false positives by measuring the
proportion of its positive predictions that are true positives
(TP).

The formula for precision is provided by:

𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 𝑇𝑃 / (𝑇𝑃 + 𝐹𝑃) (7)

where the True positives are called TP, while false positives
are called FP.

A statistic used to quantify the efficacy of a

classification model is recalled, which is often referred to as
sensitivity or true positive rate. It assesses the model's ability
Fig 2. IRMHEABCD plotting data distribution
to accurately identify positive samples by quantifying the
percentage of true positives (TP) among all positive samples.

IJISRT23JUL955 www.ijisrt.com 1560

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
The recall formula comes from: Table 2: Multilayer Perceptron Classifier Performance on
Test Set
𝑅𝑒𝑐𝑎𝑙𝑙 = 𝑇𝑃 / (𝑇𝑃 + 𝐹𝑁) (8) Precision Recall F1-Score Support
0 0.97 0.99 0.98 71
In the given context, the variable FN denotes the count 1 0.98 0.95 0.96 43
of instances where the test outcome is negative but the actual
condition is positive, while the variable TP refers to the count Accuracy 0.97 114
of cases where the test result is positive and the actual Macro avg 0.97 0.97 0.97 114
condition is also positive. Weighted avg 0.97 0.97 0.97 114

A classification model's overall performance is  Deep Belief Network Classifier

evaluated using the F1 score, which considers precision and The test set's results from the Multilayer Perceptron
recall. From these two measures, the harmonic mean is used Classifier. The classifier attained an accuracy of 0.974, which
to calculate a single score that summarizes the model's is greater than the Random Forest Classifier. The accuracy
performance. and recall values for both classes are likewise high, with the
precision and recall values for class 0 being somewhat higher
The F1 score calculation is described by: than those for class 1. The F1-scores for both classes and the
macro average F1-score are all high, showing that the
𝐹1 𝑆𝑐𝑜𝑟𝑒 = 2 ∗ (𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∗ 𝑅𝑒𝑐𝑎𝑙𝑙) / classifier is operating well. Overall, the Multilayer
(𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙) Perceptron Classifier appears to be a good choice for
(9) classifying the breast cancer dataset.

where Precision and Recall are, respectively, the precision Precision Recall F1-Score Support
and recall scores. 0 0.97 0.96 0.96 71
1 0.93 0.95 0.94 43
 Random Forest Classifier
The Wisconsin Breast Cancer dataset was used to train
a Random Forest Classifier model that achieved an Accuracy 0.96 114
impressive 0.965 accuracy on the test set. When predicting a Macro avg 0.95 0.96 0.95 114
malignant diagnosis, the precision score was 0.98, indicating Weighted a 0.96 0.96 0.96 114
that 98% of the predicted malignant samples were actually vg
malignant. The model also had a recall score of 0.93, Table 3: DBN Classifier Performance on Test Set
indicating that it correctly identified 93% of all malignant
samples. The f1-score, which combines recall and precision, The above table summarizes the evaluation metrics of
was 0.95 for predicting malignant diagnoses. The unweighted the DBN classifier on the breast cancer dataset. It shows the
average of the F1 scores for both classes was used to precision, recall, and f1-score for each class, as well as the
calculate the macro-average F1 score, which was 0.96. macro and weighted average of these metrics. The accuracy
Additionally, class imbalance was taken into account with the of the classifier is also shown, which indicates the overall
help of the weighted average F1 score. percentage of correctly classified instances.
Table 1: Random Forest Classifier Performance on Test Set.  Confusion Matrix
Precision Recall F1-Score Support A common method for evaluating a classification
0 0.96 0.99 0.97 71 model's effectiveness is the confusion matrix. It gives a
1 0.98 0.93 0.95 43 table that summarizes the number of accurate and wrong
predictions provided by a classification model. The actual
Accuracy 0.96 114 class labels are shown in the table's rows, while the
Macro avg 0.97 0.96 0.96 114 anticipated class labels are shown in the columns. The
Weighted av 0.97 0.96 0.96 114 objective of this project was to enhance the accuracy of the
g final model by merging the predictions of several classifiers
using an ensemble approach. To assess the effectiveness of
 Multilayer Perceptron Classifier the ensemble model, we analysed a confusion matrix. The
The test set's results from the Multilayer Perceptron findings from the confusion matrix indicate that the ensemble
Classifier. The classifier attained an accuracy of 0.974, which model accurately classified 110 out of 114 cases, resulting in
is greater than the Random Forest Classifier. The accuracy an accuracy rate of around 96%.
and recall values for both classes are likewise high, with the
precision and recall values for class 0 being somewhat higher The confusion matrix also revealed that the ensemble m
than those for class 1. The F1-scores for both classes and the odel made one false positive prediction and three false negati
macro average F1-score are all high, showing that the ve predictions. A false positive occurs when the model predic
classifier is operating well. Overall, the Multilayer ts a positive class label, but the actual label is negative. A fals
Perceptron Classifier appears to be a good choice for e negative occurs when the model predicts a negative class la
classifying the breast cancer dataset. bel, but the actual label is positive.

IJISRT23JUL955 www.ijisrt.com 1561

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
The confusion matrix provided valuable information ab breast cancer were compared in this study. Before being
out the performance of the ensemble model, allowing us to id divided into training and testing sets and visualized, the
entify areas where the model could be improved. Additionall breast cancer dataset underwent preprocessing. Our training
y, it helped us to communicate the results of our model to oth phase involved four machine learning models, namely the
ers in a clear and concise manner. Random Forest Classifier and Multilayer Perceptron
Classifier. The F1 score, accuracy, precision, and recall were
Predicted Benign Predicted Malignant some of our performance metrics. Lastly, we displayed the
Actual Benign 70 1 learning curves for the models that yielded the best results.

Actual Malign 1 42 As a result of combining the predictions from the four

ant models, the ensemble model, according to our findings, had
Table 4: Confusion Matrix for Ensemble Model the highest accuracy, coming in at 97.4%. The Multilayer
Perceptron Classifier came in second with an accuracy of
 Comparison Plot: 97.4%, trailing only the Random Forest Classifier by 96.5%.
In Figure 4, we can see a comparison plot of three Accuracy was 95.6% and 93.0% for the Support Vector
classifiers, Random Forest Classifier, Multilayer Perceptron Machine and Logistic Regression models, respectively.
Classifier, and Deep Belief Network (DBN), along with the
Ensemble model applied to the Wisconsin breast cancer Combining the predictions of multiple models has been
dataset. The classifiers were evaluated against the number of shown to improve performance, as evidenced by the high
training models used, shown in the learning curve plot. The accuracy of the ensemble model. The Multilayer Perceptron
Random Forest Classifier and Multilayer Perceptron Classifier had an accuracy of 97.4%, while the Random
Classifier achieved high accuracy levels on the training and Forest Classifier had an accuracy of 96.5 percent. In previous
cross-validation sets, indicating no overfitting. The Deep studies on how to classify breast cancer, these models have
Belief Network (DBN) had a higher training accuracy but a been shown to be effective. The Random Forest Classifier
lower cross-validation accuracy, suggesting overfitting. and Multilayer Perceptron Classifier both exhibit increasing
However, the Ensemble model achieved a significantly performance as the number of training samples rises,
higher accuracy of 98% on the test set, demonstrating its according to their learning curves. This shows that adding
ability to accurately classify both malignant and benign more training instances may help these models perform even
tumors and overcome the limitations of individual classifiers. better. According to the ensemble model's confusion matrix,
The Ensemble model combined the strengths of each 110 out of 114 samples were correctly identified. It
classifier, producing a more robust and reliable classification incorrectly categorized three benign samples as malignant
model. Thus, the Wisconsin breast cancer dataset can be and one benign sample as malignant. The model might
accurately classified using the Ensemble model, providing a overlook some cases of breast cancer, as evidenced by the
valuable tool for medical diagnosis and treatment. false negative rate of 6.97%. False negatives may result in
missed or delayed diagnosis, which can have major
repercussions. To enhance the model's performance in these
circumstances, more study is required.

Our study showcases the accuracy with which machine

learning algorithms identify breast cancer. The ensemble
model, which combined the predictions of four models,
achieved the highest accuracy of 97.4%. The Random Forest
Classifier and Multilayer Perceptron Classifier alone had
accuracies of 96.5% and 97.4%, respectively. These models
can be used as decision support tools for medical
practitioners in diagnosing breast cancer. However, further
research is needed to determine how well the models perform
on false negatives, which could improve their accuracy and
reduce the risk of missed diagnoses. Overall, Our findings
demonstrate how machine learning can be used to diagnose
Fig 4. Comparison ensemble model breast cancer.

VI. CONCLUSION REFERENCES

Bosom disease is a significant reason for malignant

growth-related fatalities among ladies around the world, [1]. Al-Masni, M. A. (2018). Breast cancer classification
making early recognition and exact conclusion fundamental using machine learning algorithms. International
for further developing endurance rates. Breast cancer can be Journal of Computer Applications, 181(44), 1-7.
detected and diagnosed using machine learning methods [2]. American Cancer Society. (2021). Breast Cancer Facts
Using the Wisconsin Diagnostic Breast Cancer (WDBC) & Figures 2021-2022. Retrieved from
dataset, the four machine learning algorithms used to classify https://www.cancer.org/content/dam/cancer-

IJISRT23JUL955 www.ijisrt.com 1562

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
org/research/cancer-facts-and-statistics/breast-cancer- [18]. Wang, T., Wang, S., Yang, Y., Kim, E., & Park, J. E.
facts-and-figures/breast-cancer-facts-and-figures-2021- (2020). A review of machine learning algorithms for the
2022.pdf prediction of breast cancer recurrence. Journal of
[3]. Chen, T., & Guestrin, C. (2016). XGBoost: A scalable Healthcare Engineering, 2020, 1-16.
tree boosting system. In Proceedings of the 22nd ACM [19]. Wu, J., Wang, Y., & Gao, L. (2021). A review on
SIGKDD international conference on knowledge machine learning algorithms for breast cancer detection
discovery and data mining (pp. 785-794). and diagnosis. Frontiers in Oncology, 11, 1-15.
[4]. Chollet, F. (2018). Deep learning with Python. Manning [20]. Wang, Y., Sun, H., Liu, Y., Deng, L., & Zhu, X. (2019).
Publications. A machine learning-based approach for breast cancer
[5]. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. diagnosis and prognosis prediction. Journal of
M., Blau, H. M., & Thrun, S. (2017). Dermatologist- Healthcare Engineering, 2019, 1-8.
level classification of skin cancer with deep neural [21]. American Cancer Society. (2021). Breast Cancer Facts
networks. Nature, 542(7639), 115-118. & Figures 2021-2022.
[6]. Han, J., Pei, J., & Kamber, M. (2011). Data mining: https://www.cancer.org/content/dam/cancer-
concepts and techniques. Morgan Kaufmann Publishers. org/research/cancer-facts-and-statistics/breast-cancer-
[7]. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep facts-and-figures/breast-cancer-facts-and-figures-2021-
residual learning for image recognition. In Proceedings 2022.pdf
of the IEEE conference on computer vision and pattern [22]. Li, H., & Yu, Y. (2020). Deep learning-based diagnosis
recognition (pp. 770-778). of breast cancer using MRI. In International Conference
[8]. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, on Neural Information Processing (pp. 712-721).
K. Q. (2017). Densely connected convolutional Springer.
networks. In Proceedings of the IEEE conference on [23]. Niknazar, M., Shiri, I., & Abdollahi, H. (2021). Breast
computer vision and pattern recognition (pp. 4700- cancer detection using machine learning algorithms: a
4708). review. International Journal of Cancer Management,
[9]. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). 14(2).
Imagenet classification with deep convolutional neural [24]. Gandomkar, Z., Brennan, P. C., McEntee, M. F., &
networks. In Advances in neural information processing Reed, W. M. (2018). Breast cancer screening: the role
systems (pp. 1097-1105). of radiologists in the age of artificial intelligence.
[10]. National Cancer Institute. (2021). SEER Cancer Clinical Radiology, 73(9), 797-804.
Statistics Review, 1975-2018. [25]. Houssami, N., & Ciatto, S. (2014). Breast cancer
[11]. Li, F., Li, W., Farooq, U., & He, W. (2021). Breast screening. In Seminars in oncology (Vol. 41, No. 1, pp.
cancer detection using machine learning algorithms: a 52-68). WB Saunders.
systematic review and meta-analysis. BMC Medical [26]. Khan, M. A. I., Islam, M. M., Islam, M. R., Rasker, J.
Informatics and Decision Making, 21(1), 1-24. J., & Kabir, E. (2020). Machine learning-based breast
[12]. Dunning, A. M., Michailidou, K., Kuchenbaecker, K. cancer risk prediction: a systematic review and meta-
B., Thompson, D., French, J. D., Beesley, J., ... & analysis. Breast Cancer Research and Treatment,
Cross, S. S. (2016). Breast cancer risk variants at 6q25 183(1), 1-19.
display different phenotype associations and regulate [27]. Nagi, J., & Mcclymont, D. (2018). A review of machine
ESR1, RMND1 and CCDC170. Nature genetics, 48(4), learning in the diagnosis of breast cancer. Current
374-386. Oncology, 25(2), e141.
[13]. American Cancer Society. (2021). Breast Cancer Facts [28]. Rizwan, A., Wang, L., Ur Rehman, N., Liu, Y., Ali, R.,
& Figures 2021-2022. Retrieved from Chen, J., ... & Zhao, X. (2021). Breast cancer detection
https://www.cancer.org/content/dam/cancer- using machine learning algorithms: a systematic review
org/research/cancer-facts-and-statistics/breast-cancer- and meta-analysis. Journal of Personalized Medicine,
facts-and-figures/breast-cancer-facts-and-figures-2021- 11(1), 33.
2022.pdf. [29]. Samala, R. K., Chan, H. P., Hadjiiski, L., Helvie, M. A.,
[14]. Liu, Y., Wu, X., & Zhou, Z. (2019). Breast cancer & Wei, J. (2016). Multi-task transfer learning deep
diagnosis using deep learning algorithms: a review. convolutional neural network: application to computer-
ACM Computing Surveys, 52(5), 1-34. aided diagnosis of breast cancer on mammograms.
[15]. National Cancer Institute. (2021). SEER Cancer Physics in Medicine & Biology, 61(24), 8893.
Statistics Review, 1975-2018. Retrieved from [30]. Yip, S. S., Aerts, H. J., & Lambin, P. (2018).
https://seer.cancer.gov/csr/1975_2018/. Integration of imaging and clinical data for improved
[16]. Al-Masni, M. A., Al-Absi, H. R., & Mohammed, A. A. cancer outcomes. The Lancet Oncology, 19(9), e435-
(2021). Performance evaluation of machine learning e443
algorithms for breast cancer classification. IEEE [31]. O'Donoghue C, Fox N, McCartan D, Lysaght J,
Access, 9, 60337-60348. Reynolds JV. Breast cancer: current molecular
[17]. Ferlay, J., Colombet, M., Soerjomataram, I., Mathers, therapeutic targets and new players in tumour
C., Parkin, D. M., Piñeros, M., ... & Bray, F. (2019). progression. Eur J Cancer. 2017;85:3-14.
Estimating the global cancer incidence and mortality in doi:10.1016/j.ejca.2017.07.003
2018: GLOBOCAN sources and methods. International
Journal of Cancer, 144(8), 1941-1953.

IJISRT23JUL955 www.ijisrt.com 1563

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
[32]. Kim E, Kim J. Breast cancer heterogeneity: a focus on
epigenetics. Breast Cancer Res. 2016;18(1):1-8.
doi:10.1186/s13058-016-0748-z
[33]. Hwang KT, Kim J, Jung J, et al. Identification of
genetic factors associated with breast cancer
susceptibility using a multilocus genome-wide
association study. Breast Cancer Res. 2019;21(1):1-13.
doi:10.1186/s13058-019-1219-9
[34]. Perou CM, Sørlie T, Eisen MB, et al. Molecular
portraits of human breast tumours. Nature.
2000;406(6797):747-752. doi:10.1038/35021093
[35]. Cheang MC, Chia SK, Voduc D, et al. Ki67 index,
HER2 status, and prognosis of patients with luminal B
breast cancer. J Natl Cancer Inst. 2009;101(10):736-
750. doi:10.1093/jnci/djp082
[36]. DeSantis C, Ma J, Bryan L, Jemal A. Breast cancer
statistics, 2013. CA Cancer J Clin. 2014;64(1):52-62.
doi:10.3322/caac.21203
[37]. Ring A, Webb A, Ashley S, Allum WH, Ebbs SR.
Pathological complete response rates following breast
cancer neoadjuvant chemotherapy: prediction of long-
term outcome and selection of patients for breast
conservation. Br J Cancer. 2004;91(12):2012-2017.
doi:10.1038/sj.bjc.6602235
[38]. Cardoso F, van't Veer LJ, Bogaerts J, et al. 70-gene
signature as an aid to treatment decisions in early-stage
breast cancer. N Engl J Med. 2016;375(8):717-729.
doi:10.1056/NEJMoa1602253
[39]. Burstein HJ, Curigliano G, Loibl S, Dubsky P, Gnant
M, Poortmans P. Estimating the benefits of therapy for
early-stage breast cancer: the St. Gallen International
Consensus Guidelines for the primary therapy of early
breast cancer 2019. Ann Oncol. 2019;30(10):1541-
1557. doi:10.1093/annonc/mdz235
[40]. Wang X, Yu Z, Zhao R, et al. Identification of key
genes and pathways in breast cancer using
bioinformatics analysis. Front Genet. 2021;12:617121.
doi:10.3389/fgene.2021.617121

IJISRT23JUL955 www.ijisrt.com 1564