Brain Tumor Segmentation using U-Net

1st Mr.Jonak Saha 2nd Mr.Ansh Sandilya 3rd Mr. Adityaa Kumar 7th Ms. Sapna Mishra
Electronics and Comm. Engg. Electronics and Comm. Engg Electronics and Comm. Engg Electrical Engg
NERIST NERIST NERIST IIT DELHI
Nirjuli, India Nirjuji, India Nirjuli, India Delhi, India

6th Mr. Rusni Kima Mangang

Electronics and Comm. Engg
NERIST
Nirjuli, India

Abstract—A brain tumor is regarded as one of the severe more reliable and consistent. These techniques were developed
illnesses that can affect both adults and children.In this paper, in response to the growing need for effective diagnostic tools
we provide a model that addresses the critical need in medical due to the rising frequency of brain tumors. Over 700,000
imaging by providing precise delineation of tumor boundaries.
The U-Net model, effectively captures both the fine details and Americans are estimated by the American Brain Tumor As-
contextual information of images, making it particularly suitable sociation to be living with a primary brain tumor, and 80,000
for biomedical image segmentation tasks. Through comprehen- new cases are identified each year.[3]
sive preprocessing and augmentation techniques, the dataset is Brain tumor segmentation has been approached from several
prepared to train the UNET model, enhancing its ability to angles, with approaches ranging from conventional image pro-
generalize across diverse cases. Results demonstrate the model’s
efficacy in accurately predicting tumor regions, validated by cessing methods to cutting-edge machine learning algorithms.
metrics such as the Jaccard. This work not only contributes to Because brain tumors are complicated and heterogeneous,
the existing body of research in medical image segmentation but early systems that depended on manual thresholding, region
also showcases the potential of deep learning models in improving growth, and clustering algorithms were not well suited to
clinical outcomes through precise and reliable medical imaging handle this complexity. The scalability and effectiveness of
analysis.
Index Terms—Brain tumor MRI scans, Brain tumor segmen- these old procedures in clinical settings were limited since
tation, Deep learning, U-Net architecture. they frequently needed a high degree of physical involvement
and expertise.[4]
I. I NTRODUCTION Convolutional Neural Networks (CNNs) have showed great
Brain tumors are abnormal cell growths, either benign or potential with the emergence of deep learning. The encoder-
malignant, that occur within the brain. They provide serious decoder structure of the Ronneberger et al.-introduced U-Net
health hazards, such as the possibility of neurological damage, architecture has been particularly significant in biological pic-
incapacity, and even death. A number of variables, including ture segmentation since it allows for exact localization while
genetic mutations, environmental conditions, radiation expo- preserving contextual information. A symmetric expanding
sure, and family history, have been linked to the etiology path for accurate localization and a contracting path for context
of brain tumors. Effective treatment and care, which can capture make up the U-Net design. This approach is very use-
entail sophisticated medical procedures like surgery, radiation ful for medical picture segmentation jobs since it can handle
therapy, and chemotherapy, depend on early discovery and tumors of different sizes and forms with effectiveness.[5]
precise diagnosis. As an illustration of the seriousness of Further advancements, such as the nnU-Net, have built upon
this medical disease, the Central Brain Tumor Registry of this foundation to provide self-configuring methods that adapt
the United States (CBTRUS) offers statistical insights on the to various biomedical segmentation tasks. nnU-Net automat-
occurrence and effect of brain tumors.[1] ically configures its architecture, training, and pre-processing
For patients to survive and have a high quality of life, brain pipelines based on the dataset, making it a highly versatile tool
tumors must be diagnosed correctly and promptly. Radiologists for a wide range of medical imaging challenges. Additionally,
must manually segment brain tumors from magnetic resonance models like the DeepMedic and SegNet have also contributed
imaging (MRI) images, which is laborious, error-prone, and to the field by introducing new architectural innovations and
extremely subjective. This underscores the need for brain tu- training strategies that enhance segmentation performance.[6]
mor segmentation techniques that are automated, dependable, In this study, we provide a model for brain tumor segmen-
and effective. By offering accurate and repeatable segmen- tation from MRI images, based on the U-Net architecture.
tations, advanced computational approaches can greatly assist Our method makes use of the U-Net architecture’s potent
radiologists in enhancing diagnosis and treatment planning.[2] feature extraction capabilities, which enable the network to
Automated segmentation techniques can lower variability concentrate on pertinent areas of the input picture and increase
between and across observers, resulting in judgments that are segmentation accuracy—particularly in complicated and di-
verse tumor locations. 2) Bottleneck: The encoder and decoder paths are con-
In this paper, the introduction addresses the importance of nected by the bottleneck layer. It uses two convolutional layers
accurate segmentation and the overview of U-Net architec- without max pooling but with batch normalization and ReLU
ture. The materials and methods section details the datasets, activation.[5]
preprocessing steps, U-Net architecture, training procedures, 3) Expansive Path(Decoder): The decoder path is made up
loss functions, and evaluation metrics. Results and discussions of several upsampling blocks, and their goal is to minimize
present the model’s efficiency, evaluation metrics, and com- the amount of feature mappings while restoring the spatial
parisons with ground truth. The conclusion summarizes the dimensions.[5]
study’s significance, potential applications, and future research • Upsampling layers: Transposed convolution, or decon-
directions. volution, is used by these layers to expand the spacial
dimensions of the feature maps.[5]
II. M ATERIALS AND M ETHODS • Skip Connections: High-resolution features that were
A. Dataset Description lost during downsampling are retained by concatenating
the related feature maps via skip connections from the
We have used two publicly available datasets for our model
encoder path to the decoder path.[5]
training and result evaluation which are as follows :
• Convolutional Blocks: All the blocks have two 2D
1) Dataset–I: The dataset includes 3064 T1-weighted, convolutional layers with the ‘same’ padding, kernel size
contrast-enhanced images from 233 individuals who had pi- of (3, 3), and ReLU activation.
tuitary tumors (930 slices), meningiomas (708 slices), and • Output Layer: Since this is a binary segmentation task,
gliomas (1426 slices).[7] the last layer is a 2D convolutional layer that uses a
2) Dataset–II: The dataset contains 8476 T1, T2, and fluid- sigmoid activation function to create the segmentation
attenuated inversion recovery MRI images. There are four mask and a single filter.[5]
subclasses in each directory: glioma, meningioma, pituitary,
and normal brain MRI images. The subclasses include glioma D. Training Procedure
(649 slices), meningioma (999 slices), pituitary (994 slices),
and no tumour (1595 slices). [8]

B. Data Preprocessing
The preparation and changing of raw data to fit the learning
model is called data preprocessing. This is the first important
step in building a machine learning model. In general, a real-
world data has a format that particularly cannot be used for
machine learning models and may contain noise and missing
data. So, the data needs to be preprocessed to fit it into the
system’s machine learning model. This further improves the
machine learning model’s precision and effectiveness.
The images and masks of our dataset are resized to a fixed Fig. 1: Dataset Division for training,validation and testing
size of 256 × 256.Then, the images and masks are normalised
to bring the pixel values of the images and masks within the The model is optimised using the ’Adam’ optimizer, which
range of [0, 1]. has a learning rate (lr) of 1e-4 (Adam(lr)). Adam is a fast
Stochastic Gradient Descent(SGD) optimizer that uses gradi-
C. The U-Net Architecture ent history to modify learning rates for specific parameters.[9]
1) Contracting route (Encoder): The encoder route is A batch size of 16 is defined for training the model. A
composed of many convolutional blocks, often including a learning rate of 1e-4 is used for the model training. This hyper-
max-pooling layer and two convolutional layers. Following parameter affects how quickly the model learns by regulating
batch normalization, each layer employs ReLU activation. The the step size used in gradient descent optimization. A full run
feature maps’ spatial dimensions are reduced by the max- through the training dataset is referred to as an epoch. A total
pooling layer.[5] number of 500 epochs is used for training the model. The
dataset is divided into 60%-20%-20% for training, validation
• Input layer: An image with a size of (128, 128, 3) is
and testing respectively as shown in Fig. 1.
fed into the model.
• Convolutional Blocks: Each block contains two 2D
E. Loss Function
convolutional layers with kernel sizes of (3,3), ReLU
activation, and padding set to ”same”. A max-pooling The dice-loss function is used for training. This custom
layer is applied after every pair of convolutions with a function implements the Dice coefficient loss, which is a
pool size of (2, 2).[5] widely used measure of the overlap between anticipated and
ground truth masks in segmentation tasks.[10]Eq. (1) used for over 500 epochs. Both losses started to decrease rapidly
the calculating the dice loss is as follows: because the model quickly learned the patterns in the data,
2 ∗ |X ∩ Y | indicating a rapid improvement in prediction. While the two
DiceLoss = 1 − (1) losses will decrease, training loss eventually reaches a plateau,
|X| + |Y |
while validation loss plateaus and persists, which indicates that
where, the model has learned enough to expand appropriately without
X represents the total number of pixels in the predicted competition and that further training will not affect it beyond
segmentation mask. this stage.
Y represents the total number of pixels in the ground truth
segmentation mask. 0.7
Training and Validation Dice Coefficient

F. Evaluation Metrics 0.6

1) Jaccard Score: The Jaccard Score defines the overlap 0.5

between the ground truth mask and predicted mask.[11] It has

Dice Coefficient values

0.4

a range of 0 (perfect overlap) to 1 (no overlap). 0.3

TP + TN 0.2

Jaccard = (2) Training Dice Coefficient

TP + FP + FN + TN 0.1
Validation Dice Coefficient

where TP = True positives, TN = True negatives, FP = False 0

0 50 100 150 200 250 300 350 400 450
EPOCH

positives, FN = False negatives.

2) Precision: This measure shows the percentage of true Fig. 3: Training and Validation Dice Coeff. for Dataset 1.
positives among anticipated positives.[8] A model whose pre-
cision is more suggests that there aren’t many false positives 2) Training and Validation Dice Coefficient: The training
in the prediction. (red) and validation (black) Dice coefficients across 500
TP epochs measures the segmentation task performance of the
P recision =
(3) model which is shown in Fig. 3. At first, both values rise
TP + FP
quickly, indicating quick learning, when the validation coef-
3) Recall: Out of all the positive cases in the ground truth, ficient reaches 0.3 and the training coefficient reaches 0.4.
this metric shows the percentage of true positives the model The validation coefficient fluctuates about 0.5, showing vary-
correctly detected.[11] ing generalization. The validation coefficient stabilizes at or
TP around 0.6, indicating improved performance. During the last
Recall =
(4) stage, the validation coefficient remains stable at 0.62 and the
TP + FN
training coefficient rises marginally to 0.67, indicating the little
where TP = True positives, FN=False negatives
progress.
4) F1 Score: The F1 score, which balances a model’s
capacity to detect real positives and prevent false positives or
negatives, is the harmonic mean of precision and recall.[11]
P recision ∗ Recall
F1 = 2 ∗ (5)
P recision + Recall
III. R ESULTS AND D ISCUSSIONS
A. Trained Model Efficiency (a) Sample output for dataset 1

Training and Validation Loss

1

Training Loss
Validation Loss

0.9

0.8
Loss Coefficient values

0.7

(b) Sample output- dataset 2

0.6

0.5
Fig. 4: Actual Mask Vs Predicted Mask
0.4

0.3
0 50 100 150 200 250 300 350 400 450
B. Data Evaluation
EPOCH

We evaluated the model for test set of both the datasets.

Fig. 2: Training and Validation Loss-Dataset 1 The projected segmentation output vs the ground truth is
displayed in the Fig. 4. The image on the right depicts an
1) Training and Validation Loss: Fig. 2 shows the training MRI scene of a brain, while the two images on the left show
(red line) and validation (black line) loss of machine learning a side-by-side comparison of two segmentation maps relevant
to brain imaging, with particular emphasis on tumor region [2] D. N. Louis, H. Ohgaki, O. D. Wiestler, et al., “The
identification. The manually annotated or known regions of 2007 who classification of tumours of the central ner-
interest (in this example, the segmentation tumor region) are vous system,” Acta neuropathologica, vol. 114, pp. 97–
represented as the ”ground truth” in the Figure. The precise 109, 2007.
locations of the tumor in the brain slice are indicated by the [3] American Brain Tumor Association, Brain tumor statis-
discrete white patches set against a black backdrop in this tics, Accessed: 2024-06-06, 2023. [Online]. Available:
picture. The model’s predicted output for the same MRI brain https://www.abta.org/about-brain-tumors/brain-tumor-
scan is displayed in the right image of Figure 3 (a) and (b). The statistics/.
predicted locations of the tumor in the brain slice are indicated [4] A. Wadhwa, A. Bhardwaj, and V. S. Verma, “A review
by the discrete white patches set against a black backdrop in on brain tumor segmentation of mri images,” Magnetic
this picture. resonance imaging, vol. 61, pp. 247–259, 2019.
[5] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convo-
C. Parameter Evaluated on Test Data.
lutional networks for biomedical image segmentation,”
Table 1 shows the brain tumor segmentation model’s per- in Medical image computing and computer-assisted
formance on two distinct datasets: intervention–MICCAI 2015: 18th international confer-
TABLE I: Analysis of Brain Tumor Segmentation ence, Munich, Germany, October 5-9, 2015, proceed-
ings, part III 18, Springer, 2015, pp. 234–241.
Sl. No. Evaluation Metrics Dataset 1 Dataset 2 [6] F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and
1 Precision 0.93401 0.91052 K. H. Maier-Hein, “Nnu-net: A self-configuring method
2 Jaccard 0.80866 0.80021 for deep learning-based biomedical image segmenta-
3 Train Dice Coeff. 0.65129 0.64055 tion,” Nature methods, vol. 18, no. 2, pp. 203–211,
4 Recall 0.85140 0.83211 2021.
5 F1 0.86943 0.83802 [7] J. Cheng, “brain tumor dataset,” Apr. 2017. DOI: 10 .
6084 / m9 . figshare . 1512427 . v5. [Online]. Available:
When evaluated on the test sets of both the datasets, the https : / / figshare . com / articles / dataset / brain tumor
model produced competitive precision, dice coefficient, jaccard dataset/1512427.
index, recall and F1 scores. This suggests that the model can [8] A. Akter, N. Nosheen, S. Ahmed, et al., “Robust
accurately recognize tumor areas in MRI images. The model clinical applicable cnn and u-net based algorithm for
may find use in clinical settings, according to the positive mri classification and segmentation for brain tumor,”
results. The results are promising which suggest that the model Expert Systems with Applications, vol. 238, p. 122 347,
can be potentially be utilised in clinical applications. 2024.
[9] D. P. Kingma and J. Ba, “Adam: A method for stochas-
IV. C ONCLUSION tic optimization,” arXiv preprint arXiv:1412.6980,
This work is an important step forward in the field of 2014.
segmenting MRI pictures of brain tumors. The study effec- [10] C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and
tively demonstrates how accurate a deep learning method that M. Jorge Cardoso, “Generalised dice overlap as a deep
integrates with the U-Net architecture can be. The model’s learning loss function for highly unbalanced segmen-
high-quality metrics, including a precision of 93.4%, prove to tations,” in Deep Learning in Medical Image Anal-
its remarkable accuracy in segmentation tasks. Additionally, ysis and Multimodal Learning for Clinical Decision
the model showed strong recall, F1, and Jaccard scores, Support: Third International Workshop, DLMIA 2017,
demonstrating its reliability and robustness in the key stage and 7th International Workshop, ML-CDS 2017, Held
of tumor detection: distinguishing tumor regions from healthy in Conjunction with MICCAI 2017, Québec City, QC,
tissues. It facilitates the creation of customized treatment Canada, September 14, Proceedings 3, Springer, 2017,
plans. This innovation offers a number of benefits, from pp. 240–248.
improving patient outcomes right away to accelerating the pace [11] O. Rainio, J. Teuho, and R. Klén, “Evaluation metrics
at which healthcare choices are made. We aim to improve the and statistical tests for machine learning,” Scientific
model in subsequent study to get better results for testing in Reports, vol. 14, no. 1, p. 6086, 2024.
all the publicly available datasets.
R EFERENCES
[1] Q. T. Ostrom, H. Gittleman, J. Xu, et al., “Cbtrus
statistical report: Primary brain and other central ner-
vous system tumors diagnosed in the united states
in 2009–2013,” Neuro-oncology, vol. 18, no. suppl 5,
pp. v1–v75, 2016.