Received 23 November 2023, accepted 19 January 2024, date of publication 29 January 2024, date of current version 2 February 2024.

Digital Object Identifier 10.1109/ACCESS.2024.3359418

Automated Brain Tumor Segmentation

and Classification in MRI Using
YOLO-Based Deep Learning
MARAM FAHAAD ALMUFAREH 1 , MUHAMMAD IMRAN 2 ,
ABDULLAH KHAN 2 , MAMOONA HUMAYUN 1 , AND MUHAMMAD ASIM 2
1 Department of Information Systems, College of Computer and Information Sciences, Jouf University, Sakaka 72388, Saudi Arabia
2 Institute of Computer Sciences and Information Technology, Faculty of Management and Computer Sciences, The University of Agriculture, Peshawar,

Peshawar 25130, Pakistan

Corresponding authors: Mamoona Humayun ([email protected]) and Muhammad Imran ([email protected])
This work was supported by the Deputyship for Research and Innovation, Ministry of Education, Saudi Arabia, under Project 223202.

ABSTRACT Recent advancements in image processing and computer vision have brought significant
transformations in healthcare technology, leading to significant improvements in diagnosis accuracy, cost-
effectiveness, and time efficiency. Magnetic Resonance Imaging (MRI) is employed by the radiologist for its
remarkable ability to detect even the most subtle brain abnormalities. This study considers a comprehensive
analysis of the two prominent object identification frameworks, YOLOv5 and YOLOv7, leveraging state-
of-the-art deep learning architectures to classify and detect brain cancers within MRI. The brain tumor
dataset encompasses three distinct classes, including meningiomas, gliomas and pituitary tumors. To ensure
precise segmentation of the tumor regions, the preprocessing phase incorporates advanced mask alignment
techniques. This preprocessed dataset has been used to evaluate the performance of the deep learning models
for brain tumor detection and classification. From the numerical results of YOLOv5, it was noticed that a
recall score of 0.905 for box detection and 0.906 for mask segmentation, with a precision score of 0.94 and
0.936, respectively. At an IoU threshold of 0.5, both box detection and mask segmentation achieve a mAP
of 0.947, whereas, at an IoU threshold of 0.5 to 0.95, they achieve mAPs of 0.666 and 0.657, respectively.
In comparison, YOLOv7 exhibits strong performance with box detection accuracy of 0.936 and a mask
segmentation accuracy of 0.935. The recall score are 0.904 for box detection and mask segmentation is
0.903. Notably, the mAP result at the IoU threshold of 0.5 are 0.94 for box detection and mask segmentation
is 0.941. Over the broader IoU spectrum of 0.5 to 0.95, the mAP was 0.677 for box detection and 0.659 for
mask segmentation. To underscore the novelty of the approach, the performance of the proposed framework
is systematically compared with established methods such as RCNN, Faster RCNN, and Mask RCNN.

INDEX TERMS Brain tumor, deep learning, image processing, MRI, YOLO.

I. INTRODUCTION of cells within the body. The human body normally regu-
Brain cancer is a widespread devastating illness that results lates cell growth and multiplication, generating new cells
in numerous fatalities, even in developed nations. In the through cell division. When an old or damaged cell dies,
United States, the death toll is particularly alarming, with they are promptly replaced by new ones. However, in cases
approximately 20,000 lives lost to this formidable disease [1]. where this well-organized process fails, abnormal or damaged
It is characterized by the uncontrolled growth and spread cells can proliferate inappropriately. These cells can manifest
as tissue masses called tumors. Primary brain tumors, for
The associate editor coordinating the review of this manuscript and instance, originate in the brain and typically remain localized
approving it for publication was Wenming Cao . there [2], [3]. On the other hand, secondary brain tumors

2024 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
VOLUME 12, 2024 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ 16189
 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

begin elsewhere in the body and subsequently spread to the son to perform. Unfortunately, some hospitals and healthcare
brain [4]. centers still lack the expertise to meet these requirements,
Tumors can be classified into malignant and benign [5]. further contributing to the lengthy diagnostic process [9].
Malignant tumors can infiltrate neighboring tissues and The rapid advancements in machine learning and computer
metastasize, forming new tumors in distant body parts. vision have given rise to power solutions known as Convolu-
In contrast, benign tumors do not invade nearby tissues tional Neural Networks (CNNs). These cutting-edge models
and lack the capacity to metastasize. Once removed, benign have successfully addressed complex Computer-Aided Diag-
tumors usually do not regenerate, unlike malignant tumors. nosis (CAD) problems, encompassing recognition, classifica-
Although benign tumors do not spread, they can occasionally tion, segmentation, and even detection [10], [11], [12], [13],
attain significant size and pose severe symptoms or life- [14]. However, many existing CAD solutions for brain tumor
threatening risks. Human brain is a crucial for controlling detection and identification relying on CNNs suffer from
actions and decision-making, acting as the central hub of inefficiency across various platforms and demand substantial
the nervous system. Protecting it from harm is essential and computational resources. The lighter variants of CNN clas-
among potential threats, tumors are a significant concern. sification models have inherent limitations when pinpointing
Specific types of brain tumors, such as meningioma, glioma, the tumor’s exact location precisely [15]. On the other hand,
and pituitary tumors, result from abnormal cell growth. segmentation models can accurately identify the affected area
Meningiomas typically develop in the thin membranes sur- using a mask, thus enabling tumor localization, albeit at the
rounding the brain and are primarily benign [6]. Despite their expense of higher computing costs. This poses challenges
benign nature, they still present a life-threatening disease that when employing ordinary devices, leading to inefficient per-
directly impacts human life. formance and unsatisfactory results.
Meningiomas, which make up 36.1% of all primary Several object detection algorithms have been developed to
tumors, tend to cluster around the brain’s outer and upper enhance the detection of various objects in images [16], [17].
contours. The meninges are the three layers of tissue that Notable algorithms include the Single Shot Multibox Detec-
cover and protect the brain and spinal cord, and these growths tor (SSD) [18], R-CNN [19], and Fast R-CNN [20]. Some
emerge as bumps there. Meningioma may be diagnosed recent literature also explores the use of self-supervised deep
based on its anatomical location, form, and cellular appear- learning models to enhance performance by leveraging aggre-
ance. These tumors develop slowly and might cause serious gated information related to semantics and position [21], [22].
side effects, including convulsions and vision loss. However, However, the You-Only-Look-Once (YOLO) algorithm [23]
gliomas are tumors that develop from the brain’s glial cells has gained considerable attention due to its exceptional object
rather than its neurons. They comprise a large subset of detection system and employs single unified neural network.
tumors with a wide range of symptoms and danger levels. This algorithm revolutionized object detection by treating it
In addition, the pituitary glands at the base of the skull are as a regression problem, estimating bounding box coordinates
surrounded by cells, and tumors may form as aberrant masses and class probabilities directly from pixel-level information.
there. Due to their closeness to the pituitary glands, which It enables the simultaneous prediction of multiple bounding
play a vital role in regulating hormone synthesis and release, boxes and class probabilities, having higher performance in
these tumors can affect many biological systems. terms of both speed and accuracy. As a result, it has found
The ability to successfully prevent and treat brain tumors widespread application in the medical field, such as its ability
depends on thoroughly understanding the disease’s many to accurately categorize and locate abnormalities in brain
phases. Radiologists have used various imaging modalities, images, as demonstrated in this study.
such as X-ray, Magnetoencephalography (MEG), Com- In this work, we employed two variant of YOLO deep
puted Tomography (CT), Ultrasound, Electroencephalog- learning models, namely YOLOv5 and YOLOv7 that pro-
raphy (EEG), and Magnetic Resonance Imaging (MRI), vides a more efficient solution for segmenting and classifying
to examine brain tumors and aid in accurate diagnosis and brain tumors from MRI scans. The main contributions of this
selection of appropriate treatment in recent years. Primary manuscript are as follows:
brain tumors vary significantly in size, location, and other
features, making early diagnosis difficult. Accurate visualiza- 1) Dataset alignment: Figshare brain tumor dataset [24] is
tion of tumors by non-invasive imaging methods relies on the used to train and validate the models. Henceforth, the
absorption characteristics of tissues, making it necessary to dataset will be denoted as the BT dataset throughout
precisely delineate absorption rates for efficient tumor imag- the remainder of the article. Mask alignment scheme
ing [7]. When comparing various imaging options, magnetic has been employed on the images in the dataset and
resonance imaging (MRI) stands out as the gold standard are aligned using segmentation coordinates to ensure a
because it can provide information about the brain in both standardized format, enhancing analysis outcomes and
healthy and diseased states. In the event of anomalies, it aids training for tumor detection.
in pinpointing the precise tumor type [8]. However, analyzing 2) Tumor segmentation and classification: (a). To the best
MRI scan results requires meticulous observation and a high of our knowledge, this study stands as a pioneering
level of proficiency, making it impossible for an ordinary per- effort, being the first to utilize the BT dataset [24]

16190 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

for training both YOLO variants. Additionally, it is that validates extracted features through feedforward NN as
noteworthy to highlight the absence of any documented a classification model. The feature extraction process was
instances of YOLOv7 implementation in the existing accomplished using a pre-trained VGG19 network. Their
literature for brain tumor segmentation and classifica- model is trained and evaluated using BraTS 2015, 2016, and
tion. (b). Furthermore, YOLO V5 has not previously 2018 dataset with 98%, 96%, and 92% respective accuracy.
been utilized with the selected dataset, and its imple- Their work achieved higher accuracy in some cases, but the
mentation has been included to validate the results by model result varies depending on the dataset.
comparing them with the state of the art. A patch-based DNN is proposed in [34] to classify brain
3) Evaluation metrics: To assess the performance of tumors using MR images. The proposed model is eval-
deep learning models in handling multiple-class MRI uated extensively using eight datasets, including BraTS
scans of brain tumors, various evaluation metrics are 2012 to 2015, Ischemic Stroke Lesion Segmentation (ISLES)
employed, including precision, recall, f1-score, and 2015 and 2017, and MICCAI. The model achieved a high
mean Average Precision (mAP) at different thresholds Dice Similarity Coefficient of 99.8% on BraTS 2013 dataset
([email protected] and mAP@ 0.5 to 0.95). These metrics and showed a consistent performance across different modal-
allow for a comprehensive comparison of the perfor- ities and datasets. The authors in [35] have proposed an
mance between the two YOLO variants. automatic system for detecting brain tumors using the seg-
4) Finally, the proposed framework is systematically com- mentation method and DNN. The proposed model used
pared with established methods such as RCNN, Faster MR images for training consuming the BraTS 2013 dataset.
RCNN, and Mask RCNN, to highlight the contribution The model explored a unique method where local and
of this work. global features are consumed for diagnosis. Their model
achieved an 85% dice score, 93% specificity, and 80%
The remaining paper is organized as follows: Section II
sensitivity
presents a systematic literature review, the scope of this
The research work in [36] proposed a deep learning model
paper, and highlights the research gap. Section III presents the
to detect Gliomas type of brain tumor using a hybrid CNN
research methodology for deep learning based on brain tumor
consisting of two and three-path networks. Their model is
detection and classification. This section covers data collec-
evaluated on BraTS 2013 dataset and achieved 0.86 (dice
tion, preprocessing, deep learning models, and performance
score), 0.86 (sensitivity), and 0.91 (specificity). The dataset
analysis methods. Section IV presents the numerical results
contains scans of 20 patients with four modalities resulting in
and discusses the performance achieved by the considered
a total of 80 images. A recent study has proposed a decision
deep learning strategies. In Section V, the performance of
support system (DSS) using a pre-trained Densenet201 deep
the proposed deep learning framework is compared with
learning model for multi-class brain tumor classification [37].
state-of-the art results from the literature and presents future
Two different metaheuristic algorithms are employed for fea-
work. Finally, Section VI serves as the concluding section,
ture selection in the proposed DSS. The model is trained and
summarizing the key finding and insights.
evaluated using BraTS 2018 and BraTS 2019 datasets and
achieves a high accuracy of 95% using the SVM classifier.
II. LITERATURE REVIEW The BT dataset used in this study has been explored by
Timely detection of brain tumors is challenging, but the various researchers for brain tumor classification [38], [39],
evolution of deep learning algorithms has shown promising [40]. For instance, the research work in [38] uses a cus-
results in accurately detecting brain tumors using digital tomized dual suppression encoding and factorized bilinear
images. MRI images, CT scans, and EEG are standard meth- encoding plugged in to standard CNN model. The model is
ods used to detect brain tumors. MRI and CT scans combined heavily tested on different dataset and achieved an accuracy
with deep learning models have shown improved accuracy in of 95%. Similarly, the research in [39] proposed a residual
correctly detecting brain tumors. Deep learning models based network based on transfer learning for the classification of
on CNNs [6], [7], [25], Recurrent Neural Networks [26], brain tumor. The model is checked in terms of accuracy with
Auto-Encoders (AEs) [27], [28], [29], [30], transfer learn- standard models including ConvNet, AlexNet, and VGG 16.
ing and hybrid algorithms [31], [32] are explored heavily The proposed model achieved accuracy of 95% using the BT
by researchers in recent years because of their ability for dataset.
early detection and high accuracy. CNNs, in particular, have Many researchers have applied the YOLO model for object
become a choice of researchers for detecting and classifying detection in medical images because of its speed and perfor-
brain tumors. mance [41], [42], [43], [44]. The YOLOv5 object detection
BrainMRNet [33] used an end-to-end model to detect model, integrated into a portable Microwave Head Imag-
tumors using MR images and CNN, in which dataset con- ing system (MWHI), autonomously classifies and detects
tains 253 images, with 155 tumors and 98 normal images. human brain abnormalities. Using 400 RMW image samples,
The model is evaluated in terms of standard parameters and encompassing non-tumor and tumor instances at various loca-
achieved 96% accuracy, 92% precision, and 96% sensitivity. tions, the YOLOv5l model achieves notable metrics: 96.32%
Similarly, a research work in [25] has proposed a 3D CNN accuracy, 95.17% precision, 94.98% sensitivity, 95.28%

VOLUME 12, 2024 16191

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

TABLE 1. Summary of tools, technologies, and models used for Brain Tumor segmentation and classification.

specificity, 95.53% F1-score and 96.12% mean average pre- In [47], an innovative deep learning approach incorporat-
cision (mAP). Similarly, the research work in [46] has applied ing transfer learning techniques was introduced, resulting in
YOLOv5 using a transfer learning approach to detect the an impressive accuracy rate of 99.68% using ResNet50-v2.
brain using using BraTS 2020 dataset. The proposed model Furthermore, [48] developed an automated tumor identifi-
achieved 82-92% accuracy depending on the YOLO version, cation and segmentation system, achieving a segmentation
with YOLOv5 achieving the highest accuracy. Their work accuracy of 95% by utilizing mask images as labels. The
also suggests the tradeoff between the achieved accuracy and work in [49] proposed a computer-aided system incorpo-
training time. Table 1 summarizes the significant research, rating preprocessing, segmentation, and CNN-based clas-
existing tools, technologies, and models utilized to classify sification, demonstrating improved efficiency compared to
and detect brain tumors. alternative techniques. However, it is worth noting that
The literature review demonstrates the utilization of vari- the provided illustrations portray some inaccurate in the
ous object detection models in conjunction with classification segmentations.
models for detecting and classifying brain tumors. The The research work in [50] employed the Faster R-CNN
accuracy of these models depends on factors such as the com- algorithm for tumor detection, achieving an average preci-
plexity of the underlying algorithm, the dataset utilized, and sion of 77.60%. In [51], a CNN architecture was proposed,
the extracted features. Among various object detection mod- achieving a high accuracy rate of 96.56% for tumor clas-
els, YOLO has exhibited remarkable performance in object sification. Lastly, [52] combined a pre-trained CNN with
detection across diverse domains, including medical imaging. gray-level co-occurrence matrix features, resulting in a %
Lightweight CNN classification models, in particular, have accuracy rate of 96.5%. Previous work’s lack of specific task
limitations in accurately identifying small objects. Segmen- details hinders a comprehensive analysis of overall perfor-
tation models like YOLO address this limitation, albeit at a mance. It becomes challenging to fully evaluate and compare
higher cost. This research aims to provide a balanced solution the effectiveness of the related work and models by omit-
by combining the latest object detection models with classi- ting essential metrics, such as recall or specific detection
fication models to achieve optimal performance. thresholds.

16192 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 1. YOLO-based Brain Tumor Segmentation and Classification Research Flow Block Diagram.

III. PROPOSED METHODOLOGY Algorithm 1 Pseudo Code for Converting .mat to .png Image
Figure 1 outlines the proposed research’s key steps, start- Step 1 → Begin
ing with a thorough review of relevant literature, including Step 2 → Initialize path to the dataset, path to the output
the most recent papers, leading to dataset compilation. Sub- directory and a list of .mat files
sequently, the data undergoes preprocessing and selection. Step 3 → Outer for Loop: for file in files:
The proposed model is trained and evaluated using standard Step 4 → Transform : MATLAB.matimages and masks to
parameters following these steps. .png
Step 5 → Extract : Brain tumor polygonal segmentation
A. DATA COLLECTION coordinates
The Brain Tumor dataset of Southern Medical University, Step 6 → Inner for loop : for index, element in enumerate
Guangzhou, is utilized for this research acquired from [24] (seg_coords):
and [53]. Figure 2 summarizes the dataset, revealing that it Step 7 → Convert : The extracted coordinates to.txt YOLO
comprises 3064 images from 233 patients. All the images in format
the dataset belong to any of the three tumor classes: menin- Step 8 → End : Inner for loop
gioma, glioma, and pituitary, with proportions of 23.11%, Step 9 → End : Outer for loop
30.35%, and 46.54%, respectively. Each slice is composed Step 10 → End
of 512 pixels.

B. DATA PREPROCESSING segmentation coordinates are extracted. This step involves

The dataset is converted from .mat format to extract .png utilizing image processing and analysis techniques to identify
images and masks. Additionally, this study observes that the and extract the coordinates representing the tumor regions
tumor border vector does not accurately align with the tumor (step 5). Subsequently, an inner for loop is entered, iterating
in the images. As a result, the research extends to annotating over the segmentation coordinates (step 6). Each coordinate
the tumor using mask images. Figure 3 depicts the images in the list is converted to the YOLO format. This process
and corresponding masks, which have been transformed from entails transforming the coordinates into normalized values
.mat to .png format. Subsequently, the masks are utilized to relative to the image dimensions and writing them into a .txt
annotate the images using CV2 accurately. file (step 7). Once all the coordinates have been processed, the
Algorithm 1 presents the essential steps for converting .mat inner loop is exited (step 8). Finally, the outer loop is exited,
files to .png files. It begins by initializing the necessary paths completing the conversion and coordinate extraction for all
for the dataset, output directory, and list.MAT files. Next, the .mat files in the dataset (step 9).
an outer for loop iterates over each file in the list (step 3). Algorithm 2 aims to align the segmentation coordinates to
Within the loop, the transformation from MATLAB ‘‘.mat’’ a standardized format to enable further analysis and training
images and masks to .png format is performed (step 4). for tumor detection tasks. It consists of preprocessing and
Following the transformation, the brain tumor polygonal correction steps by performing morphological closing on a

VOLUME 12, 2024 16193

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 2. Distribution of Brain Tumor Slices in the Dataset. FIGURE 3. Brain Tumor Slices and Masks.

binary mask image (step 3) which closes small holes in

masked brain tumor, finding tumor contours (step 4), approx- D. MODEL TRAINING
imating contours with polygons (step 6), and converting the Two variants of the YOLO framework, i.e., YOLOv5 and
coordinates to the YOLO format (step 8). The aligned seg- YOLOv7, are employed in this research work for brain tumor
mented annotations are saved in a .txt file, completing the segmentation and classification. The selection of these vari-
preprocessing and correction step. ants is aimed at facilitating a comprehensive assessment of
their capabilities and gauging their effectiveness in address-
Algorithm 2 Pseudo code for correction of annotations ing the challenge of tumor detection and classification.
Input: Binary Mask Image
Output: Aligned Segmentation Coordinates in .TXT 1) YOLO V5 MODEL
Step 1 → Begin The YOLOv5 is a robust architecture for object detection.
Step 2 → Initialize Provide binary mask image of brain It consists of 225 layers and 7,413,608 parameters. It is a
tumor, list of coordinates deep convolutional neural network type, where various lay-
Step 3 → Perform: Morphological closing C(x, y) = ers extract meaningful features, and object localization is
E(D (f (x, y)), B) achieved within images, as shown in Figure 5.
Step 4 → Find: Contours of tumor in image The model begins with a series of Convolutional (Conv)
Step 5 → for loop: for contour in contours: layers, which learn low-level features from the input images.
Step 6 → Approximate: Contour with polygon These layers are followed by C3 layers, which help in feature
Step 7 → Append: Polygonal coordinates to list of coordi- aggregation and information fusion. The C3 layers, responsi-
nates ble for collecting more general and high-level information,
Step 8 → Convert: List of coordinates to YOLO annotation use larger filters. Down-sampling is achieved using Conv
format layers with stride and kernel size settings to improve the
Step 9 → End: for loop model’s capacity to collect multi-scale features by lowering
Step 10 → End the spatial dimensions of the feature maps. The input pic-
ture is down-sampled so that the model can better identify
The results of Algorithm 2 are illustrated in Figure 4, items across various sizes. A Spatial Pyramid Pooling Fusion
where 4(a) shows sample images before correction, and 4(b) (SPPF) component is also a part of the YOLOv5 design. The
displays the images after modification. It is evident that SPPF component is built to execute pooling operations of
Algorithm 2 effectively aligns and corrects the labels and varying sizes on the feature maps, thereby capturing contex-
coordinates accordingly. tual information at various scales.
This broadens the model understanding of the images
C. DATA SELECTION contents, which improves the accuracy with which it can
After the necessary preprocessing, the entire dataset is split recognize things. The YOLOv5 model last levels are the
into three portions for model training, validation, and testing. Conversion (Conv) and Concatenation (Concat) layers. The
The training set comprises 2144 images, which account for Conv layers aid in the fine-tuning of the features, while the
69.97% of the data. The validation set contains 827 images, Concat layers combine multi-scale feature maps to create
representing 26.99% of the data. The testing set consists of detailed object representations. The YOLOv5 model final
91 images comprising 2.97% of the data. Note that all three layer, called YOLOv5 Segment, performs the actual object
portions of the dataset have additional mask annotations. detection task.

16194 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 4. Tumor Mask Alignment for Correction of Annotations or Label.

FIGURE 5. YOLOv5 Architecture for Brain Tumor Segmentation and Classification.

It generates bounding box predictions for three scales, The pseudo code of the employed model to segments
allowing the model to detect objects of various sizes and meningioma, glioma, and pituitary for the given input
aspect ratios. These bounding boxes are predicted based on images. It starts by initializing the segmenters optimized
anchor boxes as illustrated in Figure 6. weights, image dimensions, and confidence score (step 2).
The mathematical formulas used to predict the bound- Subsequently, each layer is executed, which includes the
ing boxes in YOLO which involves determining the width convolutional layer (step 6), C3 layer (step 8), SPPF (step 10),
and height as offsets from cluster centroids and the center upsample layer (step 12), concatenation layer (step 14), and
coordinates relative to the filter application location using the segment layer (step 16). After processing all the layers,
a sigmoid function [54]. In particular, the network predicts the output is obtained as the segmented image of the any
five coordinates for each bounding box, tx , ty , tw , th and particular class of the brain tumor.
to .If the cell is positioned with an offset from the top left
corner of the image by (Cx , Cy ), and the bounding priors 2) YOLO V7 MODEL
have widths and heights represented as pw , ph , then the YOLOv7 is a refined variant of the YOLO framework, intro-
predictions correspond to bx , by , bw , bh , as depicted in the ducing a range of advancements to achieve highly efficient
Figure 6. and accurate results. It incorporates several enhancements

VOLUME 12, 2024 16195

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

Figure 7 shows the employed YOLOv7 segmentor archi-

tecture for brain tumor segmentation and classification. The
segmentor consists of 82 convolution layers, 15 concate-
nation layers, five max-pooling layers, an SPPCSP layer,
two upsample layers, and a segment layer as the foundation
of the algorithm for brain tumor segmentation. Further, the
segmentor is trained with 2144 images with 512 image size,
64 batch size, and yolov7-seg.pt weights for 100 epochs.
The employed framework accurately segment meningioma,
glioma, and pituitary tumors using using MR images.
The pseudo code of the emoloyed model to segment menin-
gioma, glioma, and pituitary by using the given MR images.
It begins by initializing the segmentor optimized weights,
FIGURE 6. Bounding Box with dimension prior and location
image dimensions, and confidence score (Step 2). Subse-
prediction [54]. quently, each layer is executed based on its respective nature:
the convolutional layer conducts feature extraction by con-
volving input data with filters (step 6), the concatenation layer
Algorithm 3 Pseudo code for YOLOv5 Model for Brain merges feature maps from diverse network stages (step 8), the
Tumor Detection max-pooling layer partitions the input into non-overlapping
Input: Brain Tumor Dataset regions and retains the maximum value within each region
Output: Segment Tumor (Meningioma, Glioma, Pituitary) (step 10), the SPPCSPC layer effectively captures multi-scale
Step 1 → Begin information using the SPP layer and guarantees the proper
Step 2 → Initialize: Segmentor with best weights, image size, flow of gradients and information across different stages of
and confidence value the network using the CSP connection (step 12), the upsam-
Step 3 → Input: Camera or image pling layer enhances the spatial resolution of the feature
Step 4 → for loop: for layer in layers: maps (step 14), and the segment layer generates predictions
Step 5 → if condition: if layer in Conv_layers: for segmentation boundaries and class probabilities for the
Step 6 → Conv(layer) segmented objects (step 16). After processing all the layers,
Step 7 → elseif condition: layer in C3: the output is obtained as the segmented image of the brain
Step 8 → C3 (layer) tumor.
Step 9 → elseif condition: layer in SPPF:
Step 10 → SPPF(layer)
Step 11 → elseif condition: layer in Upsample_layers: Algorithm 4 Pseudo Code for YOLOv7 Model for Brain
Step 12 → Upsample (layer) Tumor Detection
Step 13 → elseif condition:layer in Concat_layers: Input: Brain Tumor Dataset
Step 14 → Concat(layer) Output: Segment Tumor (Meningioma, Glioma, Pituitary)
Step 15 → elseif condition: layer in segment_layers Step 1 → Begin
Step 16 → Segment (layer) Step 2 → Initialize: Segmentor with best weights, image size,
Step 17 → Endif and confidence value
Step 18 → EndFor Step 3 → Input: Camera or image
Step 19 → End Step 4 → for loop: for layer in layers:
Step 5 → if condition: if layer in Conv_layers:
Step 6 → Conv(layer)
Step 7 → elseif condition: layer in Concat_layers:
to improve performance and handle objects across different Step 8 → Concat (layer)
scales effectively. It excels at capturing contextual informa- Step 9 → elseif condition: layer in MP_layers
tion from diverse scales by integrating multi-scale feature Step 10 → MP (layer)
fusion, which enhances object detection accuracy. Addition- Step 11 → elseif condition: layer in SPPCSPC_layer
ally, spatial pyramid pooling enables the model to extract Step 12 → SPPCSPC (layer)
information at multiple scales and effectively handle objects Step 13 → elseif condition:layer in Upsample_layers
of varying sizes. Moreover, it integrates upsampling layers Step 14 → Upsample (layer)
to facilitate the detection of smaller objects by enlarging Step 15 → elseif condition: layer in segment_layers
the feature maps. It employs concatenation operations and Step 16 → Segment (layer)
multiple convolutional layers to iteratively enhance the rep- Step 17 → End if
resentation of objects and elevate the precision of bounding Step 18 → End For
box predictions. Step 19 → End

16196 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 7. YOLOv7 Architecture for Brain Tumor segmentation and classification.

2
S X
B
E. EVALUATION PARAMETERS X obj
2
Precision, Recall, and F1 Score are widely used performance + λobj Ii,j ci − ĉl , (4)
i=0 j=0
evaluation metric, which are discussed below.
where coefficient λco??rd is the weight given to positional
1) CONFUSION MATRIX loss, while coefficient λcls is the weight given to category
The metric provide insights in to the precision and inaccu- loss. The variables x̂ and ŷ are the actual central coordinate
racies of the proposed models. Each element (i, j) within the of the target while ŵ and ĥ are the target width and height.
matrix denotes the proportion of instances that are labeled as If the anchor box located at position (i, j) contains targets,
obj
class ‘‘i’’ while predicted as a class ‘‘j’’. the corresponding value Ii,j is set to 1; otherwise, 0. The
variable pi (C) denotes the probability of the target belonging
2) LOSS to a specific category, and pl (C) is the actual category value.
The loss function optimizes the training phase of the model The lengths of both variables equal the total number of
to minimize the loss and is given as: categories C.
loss = lbox + lcls + lobj (1) 3) PRECISION
where lbox represents the bounding box regression, lcls rep- This metric assess the correctness of the bounding box (bbox)
resents the classification loss, and lobj represents confidence predictions by calculating the proportion of true positive
loss. Equation (2) – (4) presents the loss for each category as detections to the combined count of true positive and false
follows: positive using (5)
S X
B 2 TP
X obj Precision = (5)
lbox = λcoord Ii,j bj (2 − wi × hi ) TP + FP
i=0 j=0 whereas TP and FP are the true positive and false positive
  
j 2
 
j 2
 
j 2
 
j 2 values, respectively.
× xi − x̂i + yi − ŷi + wi − ŵi + hi − ĥi ,
(2) 4) RECALL
B S2 It is determined by calculating the ratio of true positive detec-
obj tions to the total of true positive and false negative using (6)
XX X
lcls = λcls Ii,j pi (c) log (p̂l (c)), (3)
i=0 j=0 c∈classes TP
Recall = (6)
S X
B 2 TP + FN
X noobj
2
lobj = λnoobj Ii,j ci − ĉl whereas TP and FN are the true positive and false negative
i=0 j=0 values, respectively.

VOLUME 12, 2024 16197

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

TABLE 2. Loss performance of the YOLO models.

FIGURE 8. Loss Performance for the YOLOv5 and YOLOv7 for Brain Tumor segmentation and classification.

5) MEAN AVERAGE PRECISION([email protected] & [email protected]:.95) the performance of the deployed models for brain tumors
The metric [email protected] represents the mean Average Precision with A100-SXM4-40GB GPU. The GitHub repositories of
(mAP) at an Intersection over a Union (IoU) threshold of 0.5, YOLOv5 and YOLOv7 are cloned to Google Drive. The mod-
while [email protected]:0.95 stands for the average mAP calculated els are evaluated using a dataset of 827 images of brain tumors
across various IoU thresholds ranging from 0.5 to 0.95 and as discussed in section III (c). Both models have been trained
calculated using (7) using hyper parameters with the stochastic gradient decent
learning algorithm, employing a learning rate of 0.001 over
k=n
1X the course of 100 epochs.
mAP = APk (7)
n
n=1

where n are the number of classes and APk refers to the B. LOSS PERFORMANCE
average precision of class k. Table 2 depicts the training loss of box, segmentation, object,
and classification in brain tumor detection. In box training
6) F1-CONFIDENCE CURVE loss (T/box), YOLOv5 has slightly high than YOLOv7 with
It is a graphical illustrates the correlation between F1 score values of 0.017893 and 0.012063, respectively. In segmen-
and the confidence threshold for object detection. Under- tation training loss (T/seg), YOLOv7 performs better than
standing the tradeoff between precision and recall at different YOLOv5 with 0.017295 compared to 0.014582. YOLOv5
degrees of confidence may be facilitated by examining the also exhibits classification training loss (T/cls) and object
F1-confidence curve. training loss (T/obj) compared to YOLOv7 with values of
0.000391 and 0.005704 for YOLOv5, and 0.000368 and
IV. RESULTS AND DISCUSSION 0.00412 for YOLOv7, respectively. Regarding validation
This section provides all the necessary detail required to loss, YOLOv5 shows slightly higher values for box validation
deploy the YOLO based models for brain tumor segmenta- loss (V/box) and classification validation loss (C/cls). The
tion and classification. This section also outlines an in-depth segment validation loss (V/seg) compared to YOLOv7 is low.
assessment and findings for understanding the outcome of However, YOLOv7 outperforms YOLOv5 in box validation
this study. loss (V/box) and classification validation loss (V/cls) with
lower values. Figure 8 displays the data in Table 2 graphically
A. PRELIMINARY STUDY to facilitate a clear understanding of the employed models.
Google Colab integrated development environment i.e., To maintain brevity, we refrain from presenting additional
Google Colab Pro has been used for training and evaluating results obtained from the analysis.

16198 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 9. Precision confidence curve for YOLO models for Brain Tumor segmentation and classification.

C. PRECISION PERFORMANCE TABLE 3. Precision performance of YOLO models.

The performance evaluation of YOLOv5 and YOLOv7 on

827 labeled boxes and mask images are presented in Table 3.
Meningioma, glioma, and pituitary brain tumors are evalu-
ated for the precision metric. The precision measures provide
the model ability to accurately identify the regions of interest
(ROI) in images. For all the classes, YOLOv5 has a precision
score of 0.94 for box detection and 0.936 for mask detection,
while YOLOv7 has a precision score of 0.936 for box detec-
tion and 0.935 for mask detection. 0.985 for mask detection. For glioma, YOLOv5 has a box
For meningioma, both YOLOv5 and YOLOv7 have the detection precision score of 0.893 and a mask detection pre-
highest precision scores of 0.965 for box detection and cision score of 0.879, while YOLOv7 has a box detection

VOLUME 12, 2024 16199

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 10. Recall-confidence curve for YOLO models for Brain Tumor segmentation and classification.

precision score of 0.962 and a mask detection precision score 0.905 for box detection and 0.906 for mask detection, while
of 0.964, while YOLOv7 has box detection precision score of YOLOv7 has a recall score of 0.904 for box detection and
0.953 and a mask detection precision score of 0.953. 0.903 for mask detection. For meningioma, both YOLOv5
Figure 9 depicts the precision-confidence curves for and YOLOv7 have the highest recall scores of 0.965 for box
YOLOv5 and YOLOv7, showing the exceptional perfor- detection and 0.97 for mask detection. For glioma, YOLOv5
mance of both models; overall, with one exception in the has a box detection recall score of 0.799 and a mask detection
glioma class, which is represented by the orange line. The recall score of 0.801, while YOLOv7 has a box detection
score for 100% precision is at a confidence value of 0.955 and recall score of 0.787 and a mask detection recall score of
0.985 for YOLOv5 and YOLOv7, respectively. 0.785. For the pituitary, both YOLOv5 and YOLOv7 have
high recall scores of 0.951 for box detection and 0.955 for
D. RECALL PERFORMANCE mask detection. Refer to the Table 4, for more results clarity.
Table 4 presents the recall performance evaluation of Figure 9 shows the box and mask recall-confidence curves
YOLOv5 and YOLOv7 on 827 labeled boxes and masks of YOLOv5 and YOLOv7, which depict that both models
images. For all the classes, YOLOv5 has a recall score of have good performance for two classes i.e., meningioma and

16200 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 11. F1-confidence curve for YOLO models for brain tumor segmentation and classification.

TABLE 4. Recall performance of the YOLO models. TABLE 5. Mean average precision ([email protected]).

pituitary. However, the recall-confidence curve for glioma the classes combined, YOLOv5 has a mAP score of 0.947 for
indicates slightly poor performance, highlighted as orange box detection and 0.947 for mask detection, while YOLOv7
line in the Figure 9. has a mAP score of 0.94 for box detection and 0.941 for mask
detection. For meningioma, both YOLOv5 and YOLOv7
E. MEAN AVERAGE PRECISION ([email protected]) have the highest mAP scores of 0.99 for box detection and
Table 5 shows the mean Average Precision (mAP) for the 0.985 for mask detection.
YOLOv5 and YOLOv7 models on 827 labeled boxes and For glioma, YOLOv5 has a box detection mAP score
masks images, evaluated at the IoU threshold of 0.5. For all of 0.872 and a mask detection mAP score of 0.872, while

VOLUME 12, 2024 16201

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 12. Precision-recall curve for YOLO models for brain tumor segmentation and classification.

YOLOv7 has a box detection mAP score of 0.86 and a mask F. MEAN AVERAGE PRECISION ([email protected]:.95)
detection mAP score of 0.862. For pituitary, YOLOv5 has a Table 6 shows the mAP for the YOLOv5 and YOLOv7
box detection mAP score of 0.978 and a mask detection mAP models on 827 labeled boxes and masks images, evaluated
score of 0.978, while YOLOv7 has a box detection mAP score at IoU threshold of 0.5-0.95. The [email protected]:.95 score means
of 0.976 and a mask detection mAP score of 0.975. that the model considers an object detected if it has an IOU
Figure 11 presents the F1-confidence curves for YOLOv5 threshold score between 50- 95%.
and YOLOv7. Both the models perform good for two classes The [email protected] score for all the classes for YOLOv5 has
i.e., meningioma and pituitary. However, the glioma has a low 0.666 for box detection and 0.657 for mask detection, while
curve indicating poor performance relating to other classes. YOLOv7 has [email protected] score of 0.677 for box detection and
Further, it depicts that the ideal box confidence value is 0.659 for mask detection. For meningioma, both YOLOv5
0.639 and 0.53 while mask confidence values are 0.635 and and YOLOv7 have the highest scores of 0.79 for box detec-
0.53 for YOLOv5 and YOLOv7, respectively, to achieve a tion and 0.799 for mask detection. For glioma, YOLOv5 has a
0.92 f1-score. score of 0.546 for box detection and 0.53 for mask detection,

16202 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 13. Confusion Matrix for YOLO models for Brain Tumor segmentation and classification.

TABLE 6. Mean average precision ([email protected]:.95). Figure 14 illustrates the visual representation of the valida-
tion (a) and (c) batches, as well as prediction batches (b) and
(d) for both YOLOv5 and YOLOv7 models. These batches
serve to optimize computational efficiency and enhance
inference speed, enabling the detection of instances across
multiple images. The validation images showcase tumor
detection on the original images; while the predicted images
demonstrate the accuracy of the brain tumor detection after
training on the original images.

while YOLOv7 has a score of 0.553 for box detection and

I. DISCUSSION
0.531 for mask detection. For the pituitary, YOLOv5 has a
score of 0.661 for box detection and 0.642 for mask detection, The performance of the YOLOv5 and YOLOv7 is evaluated
while YOLOv7 has a score of 0.675 for box detection and in segmenting three distinct classes of brain tumors, namely
0.653 for mask detection. meningioma, glioma, and pituitary. The performance of these
Figure 12 demonstrate the precision-recall curves of models relies on several metrics, including the confusion
YOLOv5 and YOLOv7 models. It depicts that both models matrix, precision, recall, F1-curve, and inference criteria. The
perform better for all classes except glioma. The glioma curve finding revealed that both models exhibited superior perfor-
shows a more false positive than meningioma and pituitary. mance across all classes except for glioma, which displayed
an elevated false positives rate of 0.75 and 0.74 for YOLOv5
and YOLOv7, respectively. Notably, YOLOv5 excelled in
G. DETECTION TIME
detecting meningioma, while it also outperformed in glioma
The models are evaluated on a dataset of 91 frames using an and pituitary tumor detection. In contrast, YOLOv7 demon-
image size of 512 pixels. For YOLOv5, the inference time is strated comparable performance across all three classes,
7.0 ms, and the NMS time is 1.2 ms. The Frames Per Second albeit with slightly lower precision scores than YOLOv5.
(FPS) for YOLOv5 is 142.857, indicating that the model can Furthermore, precision scores for box detection are generally
process approximately 143 frames per second on average. For exceeded those for mask detection.
YOLOv7, the inference time is 10.0 ms, and the NMS time is The study also presented the precision-confidence curves,
1.1 ms per image. The FPS for YOLOv7 is 100, indicating which underscored models robust performance, except in
that the model can process approximately 100 frames per the case of glioma. Remarkably, recall scores for all classes
second on average. exceeded 0.78, with the highest recall observed for menin-
gioma in both YOLOv5 and YOLOv7. Based on these
H. CONFUSION MATRIX results, it can be concluded that both models offer excep-
Figure 13 shows the normalized confusion matrix of tional overall performance, with YOLOv7 demonstrates
YOLOv5 and YOLOv7, illustrating that both models perform superior performance across numerous evaluation crite-
well for all classes except for glioma, which exhibits high ria, particularly with regard to [email protected] evidence
false positives with values of 0.75 and 0.74, respectively. in Table 6.

VOLUME 12, 2024 16203

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

FIGURE 14. Validation and prediction batches for tumor segmentation and classification.

V. COMPARATIVE ANALYSIS OF SEGMENTATION of 94.7% at an IoU of 0.5, with a slightly reduced mAP of
MODELS 65.7% in the broader IoU range of 0.5 to 0.95. Similarly,
Table 7 showcase various segmentation models, including YOLOv7 achieved a precision of 93.5% precision, a recall
RCNN, Faster RCNN, and Mask RCNN [49], all of which rate of 90.3%, and an mAP of 94.7% at an IoU of 0.5, with a
employed the same dataset. The RCNN models achieved a corresponding mAP of 65.7% within the IoU range of 0.5 to
commendable high recall rate with 95%, though precision 0.95. This work presents a state-of-the-art implementation of
and mAP remain unspecified. In contrast, YOLOv5 exhibited YOLO based brain tumor detection and classification. How-
a precision of 93.6%, a recall rate of 90.6%, and an mAP ever, it is essential to note that this study relies exclusively on

16204 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

TABLE 7. Comparison of the proposed models.

[5] N. Abiwinanda, M. Hanif, S. T. Hesaputra, A. Handayani, and
T. R. Mengko, ‘‘Brain tumor classification using convolutional neural net-
work,’’ in Proc. World Congr. Med. Phys. Biomed. Eng. Prague, Czech
Republic: Springer, 2019, pp. 183–189.
[6] J. Amin, M. Sharif, A. Haldorai, M. Yasmin, and R. S. Nayak, ‘‘Brain
tumor detection and classification using machine learning: A compre-
hensive survey,’’ Complex Intell. Syst., vol. 8, no. 4, pp. 3161–3183,
Aug. 2022.
[7] Z. N. K. Swati, Q. Zhao, M. Kabir, F. Ali, Z. Ali, S. Ahmed, and
a single dataset, namely the brain tumor dataset for evaluating J. Lu, ‘‘Brain tumor classification for MR images using transfer learning
and fine-tuning,’’ Computerized Med. Imag. Graph., vol. 75, pp. 34–46,
the employed model. Additionally, a limitations of this work Jul. 2019.
lies in the substantial computational resources required to [8] J. Watts, G. Box, A. Galvin, P. Brotchie, N. Trost, and T. Sutherland, ‘‘Mag-
analyze larger datasets with these models. This challenge may netic resonance imaging of meningiomas: A pictorial review,’’ Insights
Imag., vol. 5, no. 1, pp. 113–122, Feb. 2014.
be mitigated by augmenting available computing resources [9] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi,
and amalgamating multiple datasets, including BraTS, and M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, and C. I. Sanchez,
to assess variations in accuracy, training time, and other ‘‘A survey on deep learning in medical image analysis,’’ Med. Image Anal.,
vol. 42, pp. 60–88, Dec. 2017.
pertinent performance metrics. Conversely, transitioning to
[10] A. S. Lundervold and A. Lundervold, ‘‘An overview of deep learning in
more lightweight deep learning models [46] or investigat- medical imaging focusing on MRI,’’ Zeitschrift Für Medizinische Physik,
ing self-supervised learning [22], holds significant potential, vol. 29, no. 2, pp. 102–127, May 2019.
especially in handling diverse data sources. [11] G. Alwakid, W. Gouda, M. Humayun, and N. Z. Jhanjhi, ‘‘Diagnosing
melanomas in dermoscopy images using deep learning,’’ Diagnostics,
vol. 13, no. 10, p. 1815, May 2023.
VI. CONCLUSION [12] M. F. Almufareh, S. Tehsin, M. Humayun, and S. Kausar, ‘‘A transfer learn-
This study presents a comprehensive evaluation of the ing approach for clinical detection support of monkeypox skin lesions,’’
YOLO-based deep learning segmentation and classifications Diagnostics, vol. 13, no. 8, p. 1503, Apr. 2023.
of brain tumors, in particular meningioma, glioma, and pitu- [13] M. Humayun, M. I. Khalil, S. N. Almuayqil, and N. Z. Jhanjhi, ‘‘Frame-
work for detecting breast cancer risk presence using deep learning,’’
itary. Both YOLOv5 and YOLOv7 demonstrated superior Electronics, vol. 12, no. 2, p. 403, Jan. 2023.
performance in segmenting and correctly identifying the [14] A. Khan, J. Bukhari, J. I. Bangash, A. Khan, M. Imran, M. Asim, M.
specific tumor class. These models exhibited remarkable pro- Ishaq, and A. Khan, ‘‘Optimizing connection weights of functional link
neural network using APSO algorithm for medical data classification,’’
ficiency in accurately detecting meningioma, with YOLOv5 J. King Saud Univ.-Comput. Inf. Sci., vol. 34, no. 6, pp. 2551–2561,
excelling in identifying glioma and pituitary tumors. Further- Jun. 2022.
more, YOLOv7 displayed comparable performance across [15] Z.-Q. Zhao, P. Zheng, S.-T. Xu, and X. Wu, ‘‘Object detection with deep
learning: A review,’’ IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 11,
all three tumor categories, with only slight difference in pp. 3212–3232, Nov. 2019.
precision scores compare to YOLOv5. Recall scores for all [16] M. Imran, H. Anwar, M. Tufail, A. Khan, M. Khan, and D. A.
categories consistently exceeded the threshold of 0.78, with Ramli, ‘‘Image-based automatic energy meter reading using deep
the highest recall scores observed for meningioma in both learning,’’ Comput., Mater. Continua, vol. 74, no. 1, pp. 203–216,
2023.
YOLOv5 and YOLOv7. These findings validate the potential [17] G. Alwakid, W. Gouda, and M. Humayun, ‘‘Deep learning-based predic-
of YOLO models in precisely detecting brain tumors, partic- tion of diabetic retinopathy using CLAHE and ESRGAN for enhance-
ularly in the case of meningioma. They also offer valuable ment,’’ in Proc. MDPI, 2023, p. 863.
[18] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu,
insights into the performance characteristics and limitations and A. C. Berg, ‘‘SSD: Single shot MultiBox detector,’’ in Com-
of these models, paving the way for further advancements in puter Vision—ECCV. Amsterdam, The Netherlands: Springer, 2016,
the computer vision and medical field. pp. 21–37.
[19] R. Girshick, J. Donahue, T. Darrell, and J. Malik, ‘‘Rich feature
hierarchies for accurate object detection and semantic segmentation,’’
ACKNOWLEDGMENT
in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014,
The authors extend their appreciation to the Deputyship for pp. 580–587.
Research Innovation, Ministry of Education in Saudi Arabia [20] S. Ren, K. He, R. Girshick, and J. Sun, ‘‘Faster R-CNN: Towards
for funding this research work through the project number real-time object detection with region proposal networks,’’ IEEE
Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149,
223202. Jun. 2017.
[21] W. Zhao, C. Li, W. Zhang, L. Yang, P. Zhuang, L. Li, K. Fan, and H. Yang,
REFERENCES ‘‘Embedding global contrastive and local location in self-supervised
[1] (2023). Key Statistics for Brain and Spinal Cord Tumors. Accessed: learning,’’ IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 5,
May 18, 2023. [Online]. Available: https://www.cancer.org/cancer/types/ pp. 2275–2289, May 2023.
brain-spinal-cord-tumors-adults/about/key-statistics.html [22] W. Zhao, L. Yang, W. Zhang, Y. Tian, W. Jia, W. Li, M. Yang, X. Pan,
[2] S. Lapointe, A. Perry, and N. A. Butowski, ‘‘Primary brain tumours in and H. Yang, ‘‘Learning what and where to learn: A new perspective on
adults,’’ Lancet, vol. 392, no. 10145, pp. 432–446, Aug. 2018. self-supervised learning,’’ IEEE Trans. Circuits Syst. Video Technol., early
[3] E. I. Zacharaki, S. Wang, S. Chawla, D. Soo Yoo, R. Wolf, E. R. Melhem, access, Jul. 26, 2023, doi: 10.1109/TCSVT.2023.3298937.
and C. Davatzikos, ‘‘Classification of brain tumor type and grade using [23] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, ‘‘You only look once:
MRI texture and shape in a machine learning scheme,’’ Magn. Reson. Med., Unified, real-time object detection,’’ in Proc. IEEE Conf. Comput. Vis.
vol. 62, no. 6, pp. 1609–1618, Dec. 2009. Pattern Recognit., Jun. 2016, pp. 779–788.
[4] J. Liu, M. Li, J. Wang, F. Wu, T. Liu, and Y. Pan, ‘‘A survey of MRI-based [24] (2023). Figshare Brain Tumor Dataset. Accessed: May 23, 2023. [Online].
brain tumor segmentation methods,’’ Tsinghua Sci. Technol., vol. 19, no. 6, Available: https://www.kaggle.com/datasets/ashkhagan/figshare-brain-
pp. 578–595, Dec. 2014. tumor-dataset

VOLUME 12, 2024 16205

 M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

[25] A. Rehman, M. A. Khan, T. Saba, Z. Mehmood, U. Tariq, and N. Ayesha, [44] J. George, S. Skaria, and V. V. Varun, ‘‘Using YOLO based deep
‘‘Microscopic brain tumor detection and classification using 3D CNN and learning network for real time detection and localization of lung nod-
feature selection architecture,’’ Microsc. Res. Technique, vol. 84, no. 1, ules from low dose CT scans,’’ Proc. SPIE, vol. 10575, pp. 347–355,
pp. 133–149, Jan. 2021. Feb. 2018.
[26] H. Mohsen, E.-S.-A. El-Dahshan, E.-S.-M. El-Horbaty, and [45] A. Hossain, M. T. Islam, and A. F. Almutairi, ‘‘A deep learning
A.-B.-M. Salem, ‘‘Classification using deep learning neural networks model to classify and detect brain abnormalities in portable microwave
for brain tumors,’’ Future Comput. Informat. J., vol. 3, no. 1, pp. 68–71, based imaging system,’’ Sci. Rep., vol. 12, p.6319, Apr. 2022, doi:
Jun. 2018. 10.1038/s41598-022-10309-6.
[27] C. Baur, S. Denner, B. Wiestler, N. Navab, and S. Albarqouni, [46] T. Shelatkar and U. Bansal, ‘‘Diagnosis of brain tumor using light weight
‘‘Autoencoders for unsupervised anomaly segmentation in brain MR deep learning model with fine tuning approach,’’ in Machine Learning
images: A comparative study,’’ Med. Image Anal., vol. 69, Apr. 2021, and Computational Intelligence Techniques for Data Engineering, vol. 2.
Art. no. 101952. Berlin, Germany: Springer, 2023, pp. 105–114.
[28] K. Munir, H. Elahi, A. Ayub, F. Frezza, and A. Rizzi, ‘‘Cancer diagnosis [47] M. A. Talukder, M. M. Islam, M. A. Uddin, A. Akhter, M. A. J. Pramanik,
using deep learning: A bibliographic review,’’ Cancers, vol. 11, no. 9, S. Aryal, M. A. A. Almoyad, K. F. Hasan, and M. A. Moni, ‘‘An efficient
p. 1235, Aug. 2019. deep learning model to categorize brain tumor using reconstruction and
[29] J. Amin, M. Sharif, N. Gul, M. Raza, M. A. Anjum, M. W. Nisar, fine-tuning,’’ Exp. Syst. Appl., vol. 230, May 2023, Art. no. 120534.
and S. A. C. Bukhari, ‘‘Brain tumor detection by using stacked autoen- [48] L. A. Salman, A. T. Hashim, and A. M. Hasan, ‘‘Automated brain tumor
coders in deep learning,’’ J. Med. Syst., vol. 44, no. 2, pp. 1–12, detection of MRI image based on hybrid image processing techniques,’’
Feb. 2020. Telkomnika, vol. 20, no. 4, pp. 762–771, Aug. 2022.
[30] M. Togaçar, B. Ergen, and Z. Cömert, ‘‘Application of breast cancer [49] T. Liu, Z. Yuan, L. Wu, and B. Badami, ‘‘An optimal brain tumor detection
diagnosis based on a combination of convolutional neural networks, ridge by convolutional neural network and enhanced sparrow search algorithm,’’
regression and linear discriminant analysis using invasive breast can- Proc. Inst. Mech. Eng., H, J. Eng. Med., vol. 235, no. 4, pp. 459–469,
cer images processed with autoencoders,’’ Med. Hypotheses, vol. 135, Apr. 2021.
Feb. 2020, Art. no. 109503. [50] Y. Bhanothu, A. Kamalakannan, and G. Rajamanickam, ‘‘Detection and
[31] G. B. Praveen and A. Agrawal, ‘‘Hybrid approach for brain tumor detection classification of brain tumor in MRI images using deep convolutional
and classification in magnetic resonance images,’’ in Proc. Commun., network,’’ in Proc. 6th Int. Conf. Adv. Comput. Commun. Syst. (ICACCS),
Control Intell. Syst. (CCIS), Nov. 2015, pp. 162–166. Mar. 2020, pp. 248–252.
[32] M. Sharma, G. N. Purohit, and S. Mukherjee, ‘‘Information retrieves from [51] M. M. Badža and M. C. Barjaktarovic, ‘‘Classification of brain tumors from
brain MRI images for tumor detection using hybrid technique K-means and MRI images using a convolutional neural network,’’ Appl. Sci., vol. 10,
artificial neural network (KMANN),’’ in Networking Communication and no. 6, p. 1999, Mar. 2020.
Data Knowledge Engineering, vol. 2. Berlin, Germany: Springer, 2018, [52] O. N. Belaid and M. Loudini, ‘‘Classification of brain tumor by combina-
pp. 145–157. tion of pre-trained VGG16 CNN,’’ J. Inf. Technol. Manag., vol. 12, no. 2,
[33] M. Togaçar, B. Ergen, and Z. Cömert, ‘‘BrainMRNet: Brain tumor pp. 13–25, 2020.
detection using magnetic resonance images with a novel convolu- [53] J. Cheng, W. Huang, S. Cao, R. Yang, W. Yang, Z. Yun, Z. Wang, and
tional neural network model,’’ Med. Hypotheses, vol. 134, Jan. 2020, Q. Feng, ‘‘Enhanced performance of brain tumor classification via tumor
Art. no. 109531. region augmentation and partition,’’ PLoS ONE, vol. 10, no. 10, Oct. 2015,
Art. no. e0140381, doi: 10.1371/journal.pone.0140381.
[34] J. Amin, M. Sharif, M. Yasmin, and S. L. Fernandes, ‘‘Big data analysis
for brain tumor detection: Deep convolutional neural networks,’’ Future [54] J. Redmon and A. Farhadi, ‘‘YOLOv3: An incremental improvement,’’
Gener. Comput. Syst., vol. 87, pp. 290–297, Oct. 2018. 2018, arXiv:1804.02767.
[55] M. Masood, T. Nazir, M. Nawaz, A. Javed, M. Iqbal, and A. Mehmood,
[35] M. Havaei, A. Davy, D. Warde-Farley, A. Biard, A. Courville, Y. Bengio,
‘‘Brain tumor localization and segmentation using mask RCNN,’’ Frontiers
C. Pal, P. M. Jodoin, and H. Larochelle, ‘‘Brain tumor segmentation
Comput. Sci., vol. 15, no. 6, Dec. 2021, Art. no. 156338.
with deep neural networks,’’ Med. Image Anal., vol. 35, pp. 18–31,
Jan. 2017.
[36] S. Sajid, S. Hussain, and A. Sarwar, ‘‘Brain tumor detection and segmen-
tation in MR images using deep learning,’’ Arabian J. Sci. Eng., vol. 44, MARAM FAHAAD ALMUFAREH received the
no. 11, pp. 9249–9261, Nov. 2019. Ph.D. degree in computer sciences from Clare-
[37] M. I. Sharif, M. A. Khan, M. Alhussein, K. Aurangzeb, and M. Raza, mont Graduate University, USA. She has five-plus
‘‘A decision support system for multimodal brain tumor classification years of teaching and administrative experience
using deep learning,’’ Complex Intell. Syst., vol. 8, no. 4, pp. 3007–3020, nationally and internationally. She has an extensive
Aug. 2022. background in teaching, research supervision, and
[38] G. Xiao, H. Wang, J. Shen, Z. Chen, Z. Zhang, and X. Ge, ‘‘Syn- administrative work. She has experience in teach-
ergy factorized bilinear network with a dual suppression strategy for ing advanced Era technological courses, including
brain tumor classification in MRI,’’ Micromachines, vol. 13, no. 1, p. 15, mobile application development (Android), cyber
Dec. 2021. security, net framework programming besides
[39] H. S. Ali, A. I. Ismail, E. M. El-Rabaie, and F. E. A. El-Samie, other undergraduate and postgraduate courses, graduation projects, and the-
‘‘Deep residual architectures and ensemble learning for efficient sis supervisions.
brain tumour classification,’’ Exp. Syst., vol. 40, no. 6, Jul. 2023,
Art. no. e13226.
[40] A. A. Waskita, J. M. Amda, D. S. K. Sihono, and H. Prasetio, ‘‘Effi- MUHAMMAD IMRAN received the Ph.D.
cientNetV2 based for MRI brain tumor image classification,’’ in Proc.
degree (Hons.) in computer science (specifically
Int. Conf. Comput., Control, Informat. Appl. (IC3INA), Oct. 2023,
in the area of cloud computing and data preser-
pp. 171–176.
vation) from the University of Vienna, Austria,
[41] P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, ‘‘A review of YOLO
in 2014. He is currently an Assistant Professor
algorithm developments,’’ Proc. Comput. Sci., vol. 199, pp. 1066–1073,
Jan. 2022. and a Member Postgraduate Advisory Commit-
[42] M. A. Al-masni, M. A. Al-antari, J.-M. Park, G. Gi, T.-Y. Kim, P. Rivera, tee with the Institute of Computer Sciences and
E. Valarezo, M.-T. Choi, S.-M. Han, and T.-S. Kim, ‘‘Simultaneous detec- Information Technology, The University of Agri-
tion and classification of breast masses in digital mammograms via a culture, Peshawar, Pakistan. He has published
deep learning YOLO-based CAD system,’’ Comput. Methods Programs several research articles in international journals,
Biomed., vol. 157, pp. 85–94, Apr. 2018. presented his work at many conferences of international repute, and
[43] H. M. Ünver and E. Ayan, ‘‘Skin lesion segmentation in dermoscopic chaired many conferences. His research interests include cloud comput-
images with combination of YOLO and GrabCut algorithm,’’ Diagnostics, ing, data preservation, provenance, machine learning, and service-oriented
vol. 9, no. 3, p. 72, Jul. 2019. architectures.

16206 VOLUME 12, 2024

M. F. Almufareh et al.: Automated Brain Tumor Segmentation and Classification in MRI

ABDULLAH KHAN received the B.Sc. degree MUHAMMAD ASIM received the B.S. degree
from the University of Malakand, the M.Sc. degree in electrical engineering from CECOS University,
in computer science from the University of Science Pakistan, in 2009, the M.Eng. degree in informa-
and Technology, Bannu, and the Ph.D. degree from tion and communication engineering from Chosun
Universiti Tun Hussein Onn Malaysia, in 2010. University, South Korea, in 2012, under the NIPA
He is currently an Assistant Professor with the Scholarship Program, and the Ph.D. degree in
Faculty of Management and Computer Sciences, information technologies from the University of
The University of Agriculture, Peshawar, Pakistan. Parma, Italy, in 2016. Currently, he is an Assistant
He has published a variety of research papers in Professor with the Institute of Computer Sciences
reputed journals and conferences. His research and Information Technology, ICS/IT, Faculty of
interests include the domain of artificial intelligence, machine learning, and Management and Computer Sciences, The University of Agriculture,
deep learning. Peshawar, Pakistan. His research interests include channel coding schemes,
soft decision-directed iterative receivers in fading channels, reconfigurable
architectures, and computer vision techniques.

MAMOONA HUMAYUN received the Ph.D.

degree in computer sciences from the Harbin Insti-
tute of Technology, China. She has 15 years of
teaching and administrative experience interna-
tionally. She has an extensive teaching, research
supervision, and administrative work background.
She has authored several research papers, super-
vised many postgraduate students, and has an
external thesis examiner to her credit. She has
strong analytical, problem-solving, interpersonal,
and communication skills. Her research interests include cyber security,
wireless sensor networks (WSN), the Internet of Things (IoT), requirement
engineering, global software development, and knowledge management. She
is currently the guest editor and a reviewer of several reputable journals and
conferences around the globe.

VOLUME 12, 2024 16207