Received 8 January 2023, accepted 23 January 2023, date of publication 30 January 2023, date of current version 2 February 2023.

Digital Object Identifier 10.1109/ACCESS.2023.3240443

Multiple Types of Cancer Classification Using

CT/MRI Images Based on Learning Without
Forgetting Powered Deep Learning Models
MALLIGA SUBRAMANIAN1 , JAEHYUK CHO 2 ,
VEERAPPAMPALAYAM EASWARAMOORTHY SATHISHKUMAR 2,

AND OBULI SAI NAREN 1

1 Department of Computer Science and Engineering, Kongu Engineering College, Erode 638060, India
2 Department of Software Engineering, Jeonbuk National University, Jeonju-si, Jeollabuk-do 54896, South Korea
Corresponding author: Jaehyuk Cho ([email protected])
This work was supported in part by the Korea Environmental Industry and Technology Institute (KEITI) Grant through the Korean
Government (Ministry of the Environment), the Development of the IoT-Based Technology for Collecting and Managing Big Data on
Environmental Hazards and Health Effects, under Project RE202101551; in part by the Institute of Information and Communications
Technology Planning and Evaluation (IITP) Grant through the Korean Government, Ministry of Science and ICT (MIST), Implementation
of 5G-Based Smart Sensor Verification Platform, under Grant 2019-0-00135; and in part by IITP through the Korea Government, MSIT,
Building a Digital Open Laboratory as Open Innovation Platform, under Grant 2021-0-00546.

ABSTRACT Cancer is the second biggest cause of death worldwide, accounting for one of every six deaths.
On the other hand, early detection of the disease significantly improves the chances of survival. The use
of Artificial Intelligence (AI) to automate cancer detection might allow us to evaluate more cases in less
time. In this research, AI-based deep learning models are proposed to classify the images of eight kinds
of cancer, such as lung, brain, breast, and cervical cancer. This work evaluates the deep learning models,
namely Convolutional Neural Networks (CNN), against classifying images with cancer traits. Pre-trained
CNN variants such as MobileNet, VGGNet, and DenseNet are employed to transfer the knowledge they
learned with the ImageNet dataset to detect different kinds of cancer cells. We use Bayesian Optimization
to find the suitable values for the hyperparameters. However, transfer learning could make it so that models
can no longer classify the datasets they were initially trained. So, we use Learning without Forgetting (LwF),
which trains the network using only new task data while keeping the network’s original abilities. The results
of the experiments show that the proposed models based on transfer learning are more accurate than the
current state-of-the-art techniques. We also show that LwF can better classify both new datasets and datasets
that have been trained before.

INDEX TERMS Cancer, convolutional neural network (CNN), pretrained models, Bayesian optimization,
transfer learning, learning without forgetting, VGG16, VGG19, DenseNet, mobile net.

I. INTRODUCTION Any part of the body can develop cancer cells, but the most
The term ’’cancer’’ describes a situation in which the body’s common sites are the lungs, breasts, brain, colon, rectum,
normal cells develop abnormally due to uncontrolled muta- liver, stomach, skin, and prostate. A myriad of factors, includ-
tions. Upon creation, these cells divide erratically and dissem- ing behavioral qualities like a high BMI, cigarette and alcohol
inate throughout the organs. If left untreated, most cancers consumption, physical carcinogens like UV rays and radia-
have the potential to kill us. The leading cause of death tion, etc causes cancer. In addition, discomfort, tiredness, per-
worldwide is cancer, followed by cardiovascular diseases. sistent cough, nausea, breathing problems, bruising, bleeding,
weight loss, muscle pain, and other cancer symptoms are
The associate editor coordinating the review of this manuscript and prevalent. Hence, diagnosing cancer at its earliest stages often
approving it for publication was Wentao Fan . provides the best chance for a cure. Four modalities are

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
10336 VOLUME 11, 2023
M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

available to clinicians who diagnose, stage, and treat human Particularly in the extraction of features, CNN’s perfor-
cancer: physical examination, laboratory tests, imaging tech- mance was exceptional. CNNs excelled at numerous com-
niques, and biopsy. Of these, imaging techniques such as puter vision tasks [4], [6]. CNNs have become the primary
Computed Tomography (CT), Magnetic Resonance İmaging deep learning method for medical image categorization in
(MRI), etc, are capable of 3D detection of cancer anywhere recent years due to their self-learning capabilities. Numerous
in the human body. Figure 1. shows a few types of imaging CNN-based neural network models have been advocated for
techniques. identifying various diseases [6]. The primary objective of
Among various techniques to detect the presence of cancer this study is to develop effective strategies for using CNN
cells, CT and MRI provide a wealth of information on tumor to detect various forms of cancer. For this research, we have
location, size, morphology, and structural changes in the collected CT/MRI images of eight types of cancer, namely
surrounding tissues. Physicians and health practitioners use Acute Lymphoblastic Leukemia (ALL), Brain Cancer, Breast
these images to detect and locate cancer cells in the human Cancer, Cervical Cancer, Kidney Cancer, Lung, and Colon
body. But, visual inspection of MRI /CT scans sometimes Cancer, Lymphoma, and Oral Cancer. Through this work,
results in false positive diagnoses. Consequently, improved we intend to address the following research questions:
computer-assisted diagnosis techniques are essential for reli- RQ1: How can models that have been pre-trained be cus-
ably confirming the presence of cancer cells. tomized to classify the images in a new dataset?
We have finetuned the pre-trained CNN models and then
compared how well they worked for a new dataset with
images of four types of cancer: acute lymphoblastic leukemia,
cervical, kidney, and breast cancer.
RQ2: How effective can fine-tuning the hyperparameters
of different CNN models be?
Given the importance of these values in determining how
well a model performs, we used Bayesian optimization to
select the best values for hyperparameters.
RQ3: Can models use what they’ve learned from one task
or dataset to do something new and excel in both original and
new datasets?
To look into this problem, we used a set of CT/MRI images
of four other types of cancer: lymphoma, lung and colon,
brain tumor, and oral cancer.
In this study, we develop a set of models using pre-trained
CNN architectures to automatically detect and characterize
different types of cancer from CT/MRI images, which have
been acquired from diverse sources such as Kaggle, etc.
To fine-tune pre-trained models for repurposing, we have
retrained a few top feature extraction layers and replaced the
classification layer at the top. In order to determine the ideal
FIGURE 1. Various imaging techniques.
values for the hyperparameters used to train various CNN
architectures, this work also uses Bayesian optimization.
As the use of Artificial Intelligence (AI) in health care The following are the substantial contributions to this
continues to grow, researchers are finding new ways to use work:
deep learning models. Deep learning has been very important i. Acquired CT/MRI images for different types of cancer
for diagnosing, making decisions about, and treating chronic from different sources and preprocessed the images
diseases, especially in cancer research. A study [1] found ii. Investigated the performance of CNN variants using
that deep learning models can find and diagnose cancer as transfer learning, fine-tuning, and exhaustive simulations.
well as or better than pathologists by looking at scans of iii. Explored Bayesian optimization to hyperparameter
tissue. This study suggests that pathologists use more pre- optimization
screening technology to make diagnoses faster. Deep learning iv. Transferred the knowledge acquired by the pro-
models can not only find cancer earlier, but they can also posed models to a standard, challenging dataset comprising
improve detection accuracy. Deep learning based computer CT/MRI scans of another four types of cancer.
vision algorithms specializing in image recognition have We analyze the performance of five CNN architectures
been applied to medical imaging techniques such as CT and using transfer learning over the CT/MRI image dataset.
MRI scans [2], [3], [4], [5]. Several attempts have been In addition, Learning without Forgetting (LwF) is investi-
made to extract image information, including spatial corre- gated as a solution for multitask learning. To our knowledge,
lations, via medical imaging-based deep learning algorithms. no previous work has focused on transfer learning with LwF

VOLUME 11, 2023 10337

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

and Bayesian Optimization for categorizing CT/MRI images findings, this framework has a maximum accuracy of 96.33%
of multiple cancer cells. in determining whether or not tissues contain cancer.
The remainder of the paper is structured as follows: Khan et. al. [13] suggested a CNN-based hierarchical deep
Section II summarizes recent cancer classification efforts learning-based brain tumor classifier and used it to clas-
with CNNs based on deep learning. Section III describes the sify the input into the following four categories: glioma,
materials and procedures utilized in this work. In Section IV, meningioma, pituitary, and non-tumor. This model achieved
the recommended models are explained. The experimental an accuracy of 92.13%, making it superior to the vari-
findings and performance of the proposed models are shown ous approaches currently used to detect and segment brain
in Section V. In this section, the rationale for employing LwF tumors. Alanazi et al. [14] came up with the idea of using
is also discussed. Section VI finally concludes this work. a CNN to improve the accuracy of the automatic detection
of breast cancer. The analysis of hostile ductal carcinoma
tissue zones in whole-slide images was done. In addition, the
II. RELATED WORKS authors explored models that use a variety of CNN architec-
Several traditional approaches for detecting and classifying tures to automatically diagnose breast cancer. They compared
tumors are simple and complicated, requiring more time and the findings of these models to those produced by machine
labor around the clock. Recently, deep learning models have learning techniques. To segment the colon and polyps from
been used to conduct cancer classification. İn this section, CT images, a deep CNN-based residual network technique
we summarize the recent attempts on detecting/ classifying provided by Akilandeswari et al. [15] has been adopted over
a variety of cancer using deep learning models. the 2D CT images. The residual stack block and the short
Deep learning networks such as VGG-16 and ResNet-50 skip nuance have been implemented in the hidden layers to
were utilized by Rezayi et. al. [7] for detecting acute lym- preserve the spatial data. Here, ResNet-enabled CNN is used
phoblastic leukemia. In addition, the authors presented a for thorough segmentation of colon cancer region.
CNN with 10 convolutional layers and six common machine İn an attempt, a classification and detection model was
learning algorithms for dividing leukemia into two groups. developed by Warin et. al. [16] utilizing DenseNet121 and
The authors compared various ML and CNN models in a faster R-CNN. According to the findings of these authors,
detecting this condition and concluded that CNN outper- the above algorithms were proven to have adequate potential
formed ML models. Gunasekara et. al. [8] proposed a three- for identifying malignant tumors in images. Finally, Tufail et.
fold deep learning architecture with classifiers implemented al. [17] described recent works where deep learning has been
with a deep CNN at the first level, and at the second level, used to determine the most effective models for cancer predic-
a region-based CNN (R-CNN) to locate the tumor regions tion tasks. This work focused on analyzing the effect that deep
of interest. The ChanVese segmentation algorithm is used to learning systems have on histopathology images, presented
contour the concentrated tumor boundaries for segmentation a synopsis of state-of-the-art deep learning approaches, and
in the final stage. The average dice score for this model provided future researchers with guidance on how to improve
is 0.92. upon the methods that are now in use.
To further understand how textural, morphological, and From the review of the existing research attempts, we see
graph properties affect categorization classification accuracy, that the performance of the contemporary models, such as
Reshma et. al. [9] conducted a series of studies. An improved ResNet, MobileNet, DenseNet etc is appreciable compared
Genetic Algorithm and a weighted feature selection method to traditional models such as VGGNet, AlexNet etc. We have
in conjunction with CNN have been proposed to detect breast already enumerated the research questions in Section I. These
cancer. Zhao et al. [10] found that adding manual features and research questions have been framed from the following
a vote system can produce outstanding accuracy in cervical points. From the literature study, we find that pretrained CNN
cell categorization even with little amounts of labeled data. models with transfer learning provide a number of advantages
A small number of images are annotated using a clarity func- over traditional CNN models, including faster performance,
tion, and a voting mechanism is used to balance the training fewer data requirements, and reduced training time. Hyper-
data for the algorithm. The proposed method in this research parameters govern the learning process, and as a result, these
[10] has a 91.94% accuracy rate with only a little amount of parameters’ values directly impact other model parameters
labeled data. Pederson et. al. explored the efficacy of CNNs like weights and biases, which in turn affects how well a
to facilitate the distinction of oncocytoma from renal cell model performs. While using transfer learning, the models
carcinoma [11]. The authors suggested a revised version of may forget their previous knowledge when retrained on a new
the ResNet50V2 as a solution. A majority vote of individual task set. Since, we want to retain previous knowledge along
image classifications determined a renal tumor’s benign or with the new task set, we use LwF.
malignant nature, which was set at 51%. By analyzing the To summarize, the attempts to detect cancer tissue presence
histological images of lung and colon tissues, Masud et. from CT/MRI images focused mostly on one or two types of
al. [12] developed a categorization framework that enabled cancer. In the present work, we try to detect multiple kinds of
them to differentiate between five distinct lung and colon cancer. Recent publications suggest that deep learning models
tissues, two benign and three malignant. According to the are used more frequently for cancer detection in CT/MRI and

10338 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

other imaging modalities. However, the use of deep learning As the images from the dataset are of different formats such
architectures still has space for improvement, particularly in as JPEG, JPG, PNG, BMP, NII, and TIF, they have been
terms of training time and hyperparameter adjustment. Our converted to JPG format uniformly. As like formats, the
goal is to build a few classification models that can classify images in the dataset differ in dimensions. So, they have all
different types of cancer cells using a combination of neural been resized to 224∗ 224 (common for all models) and labeled
network techniques such as deep learning, transfer learning, according to their classes.
and hyperparameter tuning. Further, we use LwF to retain
the original network capabilities while training the models on 1) IMAGE AUGMENTATION
new cancer datasets. There are a few challenges when we use deep learning mod-
els for image classification. The more information the deep
III. MATERIALS AND METHODS learning models have, the more features they will be able
A. IMAGE COLLECTION to learn. For deep learning models to learn more features,
The image collection is the process of gathering images from enormous volumes of data are required. Having this much
different sources. The present study focuses on detecting a information is only sometimes simple, though. Additionally,
variety of cancer namely ALL, Brain Cancer, Breast Cancer, sometimes, we may have a lot of data on a certain subject,
Cervical Cancer, Kidney Cancer, Lung and Colon Cancer, but it is frequently unlabeled, which prevents us from using
Lymphoma, and Oral Cancer. CT/MRI images for the var- it to train learning algorithms. Since we don’t have a bal-
ious body parts (tissues) are collected and compiled from anced image dataset for each type of cancer, we used image
Kaggle containing the categories ALL [18], Breast Cancer augmentation, which is a way to make new images from the
[19], Cervical Cancer [20], Kidney Cancer [21], Lung and ones in the training set. Table 2 lists the number of images in
Colon Cancer [22], Lymphoma [23] and Oral Cancer [24]. each category. Rotation, shifting, cropping, blurring, scaling,
In addition, an open-source Brain tumor Image Database [25] flipping horizontal and vertical, and padding are examples of
containing different MRI images with different pathological image augmentation methods to minimize the model’s over-
conditions have been generated. Each type of cancer has a fitting. After augmentation, the dataset consists of around
few categories as listed in Table 1. 130000 images, with 5000 images in each category of all
types of cancer. The number of images in each class after
augmentation is shown in Figure 2.
TABLE 1. Different types of cancer and their categories.
Hence, using augmentation, we have constructed a new
dataset and uploaded it as a public dataset on Kaggle under
the name ‘‘Multi Cancer Dataset’’ [26]. Next, the deep learn-
ing model we train should be tested and evaluated against the
performance. For this, the dataset is split into 70:15:15 ratio
for training, validation, and testing.

FIGURE 2. Number of Images after augmentation.

B. IMAGE PREPROCESSING AND AUGMENTATION

Since the images have been acquired from different sources, C. METHODS AND MODELS
they differ in number, format, and size. Hence, the datasets The development of several CNN architectures for image
must be preprocessed before being used to input any model. recognition applications has resulted in their effective

VOLUME 11, 2023 10339

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

application to various difficult visual imagery challenges. their primary application in image classification, CNNs are
In this study, the ability of five different CNN architec- increasingly used for feature extraction and classification
tures, namely VGG16, VGG19, DenseNet201, MobileNetV3 in many image processing applications. Figure 3 depicts a
(Small) and MobileNetV3 (Large) to classify the types of typical CNN.
cancer from CT/ MRI images has been evaluated. The basic
CNN is explained below.

TABLE 2. Number of actual images in the dataset.

FIGURE 3. Architecture of CNN (Adapted from: https://link.springer.

com/article/10.1007/s00521-022-07246-w).

The first few layers of a CNN are usually responsible for

finding basic things like horizontal, vertical, and diagonal
edges, among other things. The results of these layers are
sent to the layers in the middle, whose job is to pull out
more complex features like corners and edges. As we go
deeper into the network, the layers start recognizing higher-
level things like objects, faces, and other things. Several
CNN variants have been developed and effectively applied to
challenging visual tasks for image recognition applications.
In this study, we choose to develop the proposed models using
pre-trained models such as VGG16, VGG19, DenseNet201,
MobileNetV3 (small), and MobileNetV3 (large) which are
the variants of CNN.

D. TRANSFER LEARNING WITH FINETUNING

For image categorization, CNN is a popular deep learning Transfer learning entails transferring the information of a
technology. It’s also useful for modeling applications that model learned on a big dataset to a smaller dataset. Typi-
require a lot of data and is positioned at the top of its field [27], cally, the pre-trained models are trained on enormous datasets
[28]. A gradient-based strategy was used by Lecun et. al. [29] that serve as a typical benchmark for computer vision tasks,
to solve the problem of categorizing hand-written digits using such as ImageNet [30]. Canziani et. al. [31] investigated
CNNs. With this success came the recognition of language the efficacy of pre-trained computer vision models using the
and activities, and now it is used in a wide range of fields, ImageNet database. The weights derived from the models are
including object tracking and identification. A CNN can be reusable for various computer vision tasks [32]. The justi-
broken into two basic building blocks: a feature extractor fication for employing pre-trained CNN models is that the
and a classifier. The block for feature extraction consists of earliest convolutional layers of the models extract broad, low-
convolutional and pooling layers. The classification layer is level features that are relevant across images, such as edges,
fully connected and flattened. The filters comprising a convo- patterns, and gradients, while the subsequent layers detect
lutional layer indicate lower-dimensional input data charac- unique image properties. In the present study, we categorize
teristics. To construct feature maps, the filters convolute the the dataset under consideration using the models such as
entire input image. The pooling layers are then employed to VGG16, VGG19, DenseNet201, MobileNetV3 (small) and
lower the dimensionality of the feature maps, and the feature MobileNetV3 (large). Because the early layers can acquire
maps are subsequently downsampled. The last convolution general features, we have reused the weights of these layers,
layer flattens the feature map. Finally, two or three hidden but, we retrain a few top layers of the feature extraction (con-
layers and an output layer that uses a classifier to differentiate volution base) component. This is done so that the models
between two or more classes are at the classifier part. Despite can learn the characteristics unique to the datasets. In the

10340 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

classifier part, we added a layer with the number of labels will assist in improving the learning of a particular model by
corresponding to the number of unique classes in the dataset. using the knowledge contained in all of the ‘n’ tasks if there
Then, the output of the convolution base is passed into this are ‘n’ tasks where these ‘n’ tasks or a subset of them are
classifier. related to each other but not exactly identical. When only data
from new tasks are available, LwF aims to train a network
on effectively performing both old and new tasks. With this
method, results from the previous network are retained, and
samples from the new task are used to enhance the accuracy
of the new task. However, this method does not require
the previous task’s images and labels. In this attempt, we
employed CT/MRI images depicting the symptoms of four
types of cancer cells, including lymphoma, lung/colon, brain
tumor, and oral cancer, how LwF determines what they are.
Figure 4 shows a few CT images that bear both old (Task set 1)
and new tasks (Task set 2).
The rationale behind splitting the dataset into two task sets
is to evaluate the models’ performance with and without LwF.
Task set 2 has images of different oral, leukemia, lung/colon,
and brain cancer subclasses. The images of breast cancer,
acute lymphoblastic leukemia, kidney, and cervical cancer are
under task set 1. Since the survivors of the task set 1 types of
cancer are at risk of falling into the types of cancer in task
set 2 [34], [35], [36], we split as above.

F. BAYESIAN OPTIMIZATION FOR HYPERPARAMETER

TUNING
The number of hidden units, dropout, activation function,
weight initialization, and other hyperparameters establish
a neural network’s structure, whereas learning rate, batch
size, epochs, etc. Tuning hyperparameters minimizes a loss
function and improves results. Unlike Grid and Random
FIGURE 4. CT/MRI Images of older and new tasks.
search methods, Bayesian optimization employs previous
iterations of the algorithm. This facilitates Bayesian opti-
mization to choose the optimal combination of the hyperpa-
E. LEARNING WITHOUT FORGETTING(LwF)
rameters for model evaluation [37], [38]. Due to the volume
Even while pre-trained CNNs use transfer learning, they may of data involved and the complexity of computations neces-
forget what they had previously learned when transferring sary, training deep learning models can be time-consuming.
that knowledge to a new task. Transfer learning typically When dealing with issues of this kind, employing Bayesian
disregards the model’s performance on previous tasks. For optimization can significantly assist problem-solving. So,
example, a CNN that has been pre-trained to identify items we employ Bayesian optimization in conjunction with pre-
such as flowers, animals, etc. may not perform well when viously trained models to enhance performance. Fig. 5 illus-
used to classify different animal types. Because the shared trates the proposed flow of the research work.
parameters do not adequately reflect what distinguishes the
new task, feature extraction typically fails when applied to
the new task. If we fine-tune, it may decrease performance for IV. DETAILS OF EXPERIMENTS
previously learnt tasks, as the shared parameters will change We designed experiments to evaluate the performance of the
without providing a new direction for the task-specific pre- fine-tuned models using transfer learning and LwF. In the
diction parameters. To overcome this issue, Learning without following section, we present the details of the same.
Forgetting (LwF) [33], a multitasking approach, has been
developed; it works effectively on new tasks while maintain- A. EXPERIMENTAL PLATFORM
ing the same performance on previous ones. While learning Graphical Processing Units (GPU) were employed to run
new tasks, we frequently use the knowledge we gained from the proposed models due to their high power consumption
the related tasks. This is termed as multitasking learning and requirement for high-performance hardware. Table 3
(MTL). By employing the domain knowledge in the training shows the employed hardware and software configurations.
task of related tasks as an inductive bias, the MTL approach We imported the Keras model architectures and instantiated
to inductive transfer enhances generalization. Formally, MTL them with the ImageNet dataset TABLE 3.

VOLUME 11, 2023 10341

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

FIGURE 5. Proposed workflow.

TABLE 3. Experimental platform. can determine an objective function’s lowest or maximum

value. Among the fine-tuned hyperparameters are optimizers,
learning rate, activation function, number of epochs, batch
size, and the number of neurons. The Bayesian optimization
procedure has been executed twenty times. The number of
epochs for each round of Bayesian optimization is set at 150.
At each stage, we kept track of our performance and losses.
The hyperparameters we deal with and their respective search
spaces are shown in Table 4. Table 4 also presents the tuned
values of hyperparameters for different models. Figure 5
shows the overall steps involved in the proposed workflow.
B. TUNING OF HYPERPARAMETERS
Hyperparameters are crucial in determining training param-
eters and influencing model output when utilizing deep C. METRICS FOR EVALUATION
learning algorithms. This study applies Bayesian optimiza- We evaluated the performance of each model against each of
tion to determine the optimal hyperparameter values while the thirteen classes belonging to four types of cancer using
assuring high accuracy. Using Bayes’ theorem, this method accuracy, precision, recall, and F1-score. To find the values of

10342 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

FIGURE 6. Confusion matrices for task set 1.

VOLUME 11, 2023 10343

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

TABLE 4. Hyperparameters with their search space and fine-tuned values.

TABLE 5. Overall performance of the proposed models.

these metrics, we have used the indices such as True Positive the developed models is presented in Table 6. We used the
(TP), True Negative (TN), False Positive (FP), and False indices TP, FP, TN, and FN to find the values of several
Negative (FN) (1, 2). We use Eq. (1) to Eq. (4) to calculate performance metrics, such as Accuracy, Precision, Recall,
TP, FP, TP, and TN. and F1-score. A supervised learning tool, the confusion
matrix, compares predicted classes to actual classes. Figure 6
tpi = cii (1) shows the confusion matrix for every model that has been
Xn developed.
fpi = cli − tpi (2)
l=1 The diagonals of the confusion matrix show the cor-
Xn rect classifications. The actual classes are shown on
fni = cil − tpi (3)
l=1 the X-axis, while the predicted classes are shown on
Xn Xn the Y-axis. In Fig 6(a), for instance, it is evident that
tni = clk − tpi − fpi − fni (4)
l=1 k=1 MobileNetV3 (Small) incorrectly identified 5 all_benign as
Accuracy = (TP + TN )/(TP + TN + FP + FN ) (5) all_pro.

Recall = TP/(TP + FN ) (6) B. RESULTS OF PROPOSED MODELS FOR TASK SET 2

To demonstrate the power of LwF, we have also run the
Precision = TP/(TP + FP) (7)
proposed models on task set 2, which consists of a dataset
(2 ∗ precision ∗ recall) having 13 unseen classes belonging to the types of cancer
F1score = (8)
(precision + recall) which have not been included in task set 1. The results of
this trial are shown in Table 7.
V. EXPERIMENTAL RESULTS AND FINDINGS When testing task set 2 on the developed models,
In this section, we present the performance of the pro- we obtained the results in Table 7. Since the models have
posed models on both old and new tasks without and with been trained on task set 1, they gave good results for the test
LwF. data set belonging to the task set 1. The features of the task
set 2 are different from those of task set 1, and the models
A. RESULTS OF PROPOSED MODELS FOR TASK SET 1 have learned the values for the task-specific parameters of
We checked out how well the CNN models that were sug- task set 1. So, they could not perform well on the task set 2.
gested worked. Experiments have been carried out using the To improve the performance of the models on the task set 2,
tuned hyperparameters shown in Table 4, which produced we have trained the models using LwF and evaluated the
the best results when the model was put through its train- performance. Figure 7 depicts the confusion matrices that
ing phase. Table 5 shows how well each model works for indicate the performance of the models for task set 2 without
validation and test datasets. The class-wise performance of LwF.

10344 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

TABLE 6. Class wise performance of the proposed models.

VOLUME 11, 2023 10345

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

TABLE 6. (Continued.) Class wise performance of the proposed models.

C. TRANSFER LEARNING USING LwF ON TASK SET 2 VI. FINDINGS AND DISCUSSION
In this section, we show how the LwF compares to proposed The objective of this study is to detect the presence of eight
models on task set 2 of CT/MRI images having symp- types of cancer using CT/MRI images. For this, we developed
toms of four different types of cancer, namely brain can- five models using pre-trained CNN architecture. The exper-
cer, lung/colon, cervical and oral cancer. LwF utilizes only imental results have been provided earlier in this section.
images from task set 2 to train the network but retains the Below, we discuss the results of our investigation.
model’s original capabilities. When integrating LwF with the In this study, several indicators have been employed to
proposed models, the shared parameters (PS) of the feature determine the efficacy of the transfer learning techniques:
extraction layers and the task-specific parameters (PO) of the first, it is determined whether the proposed models could
classification layers for the task set 1, which was used for classify CT /MRI images using the transferred knowledge
training, were maintained; however, the task-specific param- with fine-tuning. Since the weights of pre-trained models
eters (PB) of the new set of images from the task set 2 were were used as-is during feature extraction, the resulting models
modified. These models learned parameters that are effective may be less accurate. A few top layers’ weights have been
on both task sets. We used CT/MRI images from the task retrained during fine-tuning. This allows the models to gain
set 2 for LwF training and retrained the network without task image-specific characteristics and enhances their accuracy.
set 1. We added neurons to the output layer and randomly ini- The performance of the developed models on the task
tialized the weights to retrain the models on the new images. set 1 is shown in Table 5. Among all the proposed mod-
The number of newly added parameters is, therefore, equal els, the MobileNetV3 (small) exhibited the best perfor-
to the number of neurons added to the output layer multiplied mance on this task set. But, while using LwF, MobileNetV3
by the number of neurons in the previously shared layer. This (large) has outperformed the other models. This is because
is a relatively modest number compared to the total network this model uses a technique known as depth-wise separa-
parameters. Reference [33] explains the training procedure. ble convolution, which streamlines the learning process and
The values for accuracy, precision, recall, and F1-score for boosts overall performance. Further, using an appropriate
different models with LwF on CT/MRI images of the task TPU hardware accelerator, this model could improve accu-
set 2 are shown in Table 7, and Figure 8 shows the confusion racy while simultaneously reducing the runtime and power
matrices for these models. consumption.

10346 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

FIGURE 7. Confusion matrices for task set 2 (Without LwF).

VOLUME 11, 2023 10347

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

TABLE 7. Performance of the proposed models task set 2.

10348 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

TABLE 7. (Continued.) Performance of the proposed models task set 2.

Due to the varied architectural design, such as the number outputs on all relevant images. To summarize, in this study,
of layers and convolution blocks, the models used in this work we implemented LwF, a multi-task learning strategy for
exhibited varied performance. MobileNet is a lightweight CNNs that enables CNNs to acquire good performance on
deep neural network with better classification accuracy and new tasks while maintaining performance on previous tasks.
fewer parameters. Dense blocks from DenseNets are used As a result of this work, we understood how LwF could
into MobileNet to further minimize the amount of net- be utilized in situations where the initial training dataset
work parameters and increase classification accuracy. Despite is unavailable. On a new dataset, the efficacy of LwF has
being 32 times smaller than VGG16, MobileNet has higher been validated, and experimental findings demonstrated that
accuracy and is more effective at gathering knowledge. it outperforms fine-tuning. According to the experimental
When we tested how well the proposed models could findings, contemporary models like MobileNet, DenseNet,
transfer knowledge over a new task set 2 CT/MRI images etc. perform significantly better than traditional models like
having symptoms of four different types of cancer, we found VGGNet, AlexNet, etc.
that the models were not very accurate. This is because the
models have yet to be retrained on task set 2. After retraining A. BLACK BOX ISSUES
the model using task set 2, we tested the models again with Deep learning model integration in cancer care could
task set 1 and found that they were less accurate than before. increase the precision and speed of diagnosis, support clinical
As the number of tasks increases, storing and retraining such decision-making, and result in better health outcomes. These
images becomes impossible. Adding new capabilities to a models, however, are regarded as ’’black box’’ models. Black
CNN wipes out the training data for the existing capabilities. boxes are frequently employed in the healthcare sector to
So, we turned to LwF, which retrains the network using task analyze huge amounts of data and make predictions based on
set 2 while keeping the network’s original abilities. Table 8 that data, but patients and healthcare professionals may find
shows that LwF does better than most fine-tuning techniques the lack of openness about how such decisions are made to be
on the task set 2 images. In Table 9, we can see how the concerning. A black box’s predictions or decisions could not
models with fine-tuning and LwF compare to each other. be clear to patients or healthcare professionals, which might
For increased performance on the task set 2, LwF may breed mistrust or muddled thinking. But by expanding such
be an alternative to fine-tuning. Subsequently, the accuracy a black box, explainable AI can be produced. If people can
of the task set 1 will be equivalent to that of the original understand how an AI system arrived at its choice, the system
models provided the models are maintained in such a way is considered to be explainable. This would make it easier to
that task-specific characteristics of task set 1 give identical gain research information in healthcare.

VOLUME 11, 2023 10349

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

FIGURE 8. Confusion matrices for the task set 2 (With LwF).

10350 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

TABLE 8. Performance of the proposed models with LWF on task set 2.

VOLUME 11, 2023 10351

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

TABLE 7. (Continued.) Performance of the proposed models with LWF on task set 2.

TABLE 8. Performance of the models on task set 2 with and without LWF.

TABLE 9. LwF vs fine tuned models. models. The results demonstrate that LwF maintained the
knowledge learned from task set 1 and performed better
than fine-tuned models on the task set 2. From this study,
we understood that LwF could be used where the original
training dataset is unavailable. Of all models, we found that
MobileNetV3 outperformed with a substantial performance
improvement.
To further extend this work, we plan to include more types
of cancer in our task sets and calibrate multitask learning with
VII. CONCLUSION extensive experiments. We will also investigate other types of
Cancer causes one in six deaths worldwide. Therefore, can- imaging techniques that aid in the detection of cancer cells.
cer diagnostic and treatment techniques have received much
investigation. In this attempt, we presented improved CNN
variants for detecting multiple types of cancer. Transfer learn- REFERENCES
ing has been utilized in this study since it does not require vast [1] Top Opportunities for Artificial Intelligence to Improve Cancer Care.
datasets and retrains a few top layers while using little pro- Accessed: Nov. 29, 2021. [Online]. Available: https://healthitanalytics.
com/features/top-opportunities-for-artificial-intelligence-to-improve-canc
cessing effort. In addition, Bayesian optimization is utilized er-care
to choose the most appropriate values for the hyperparameters [2] M. Subramanian, K. Shanmugavadivel, and P. S. Nandhini, ‘‘On fine-
employed throughout the training process. Further, we split tuning deep learning models using transfer learning and hyper-parameters
the images into two task sets, one for fine-tuning the models optimization for disease identification in maize leaves,’’ Neural Comput.
Appl., vol. 34, no. 16, pp. 13951–13968, Aug. 2022.
and another to verify the multitasking ability of the developed
[3] M. Subramanian, ‘‘Hyperparameter optimization for transfer learning of
models. Finally, LwF, a multitask learning method, has been VGG16 for disease identification in corn leaves using Bayesian optimiza-
employed to inculcate multitasking ability into the developed tion,’’ Big Data, vol. 10, no. 3, pp. 215–229, Jun. 2022.

10352 VOLUME 11, 2023

M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

[4] S. Krishnamoorthy, A. Shanthini, G. Manogaran, V. Saravanan, [24] Histopathologic Oral Cancer Detection Using CNNs. Accessed:
A. Manickam, and R. D. J. Samuel, ‘‘Regression model-based feature Dec. 2020. [Online]. Available: https://www.kaggle.com/datasets/ashenafi
filtering for improving hemorrhage detection accuracy in diabetic fasilkebede/dataset
retinopathy treatment,’’ Int. J. Uncertainty, Fuzziness Knowl.-Based Syst., [25] J. Cheng, ‘‘Brain tumor dataset,’’ Figshare. Accessed: Apr. 2017, doi:
vol. 29, no. 1, pp. 51–71, Apr. 2021. 10.6084/m9.figshare.1512427.v5.
[5] S. Roy, T. Meena, and S.-J. Lim, ‘‘Demystifying supervised learning [26] O. S. Naren and M. Subramanian. (2022). Multi Cancer Dataset. [Online].
in healthcare 4.0: A new reality of transforming diagnostic medicine,’’ Available: https://www.kaggle.com/datasets/obulisainaren/multi-cancer
Diagnostics, vol. 12, no. 10, p. 2549, Oct. 2022. [27] Y. Bengio, I. Goodfellow, and A. Courville, ‘‘Deep learning,’’ Nature,
[6] S. Krishnamoorthy, Y. Zhang, S. Kadry, and W. Yu, ‘‘Framework to vol. 521, no. 7553, pp. 436–444, May 2016.
segment and evaluate multiple sclerosis lesion in MRI slices using VGG- [28] Y. LeCun, F. J. Huang, and L. Bottou, ‘‘Learning methods for generic
UNet,’’ Comput. Intell. Neurosci., vol. 2022, pp. 1–10, Jun. 2022. object recognition with invariance to pose and lighting,’’ in Proc. IEEE
[7] S. Rezayi, N. Mohammadzadeh, H. Bouraghi, S. Saeedi, and A. Moham- Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Washington, DC,
madpour, ‘‘Timely diagnosis of acute lymphoblastic leukemia using arti- USA, Jul. 2004, p. 104.
ficial intelligence-oriented deep learning methods,’’ Comput. Intell. Neu- [29] Y. LeCun, L. D. Jackel, L. Bottou, C. Cortes, J. S. Denker, and
rosci., vol. 2021, pp. 1–12, Nov. 2021. H. H. Drucker, ‘‘Learning algorithms for classification: A comparison on
[8] S. R. Gunasekara, H. N. T. K. Kaldera, and M. B. Dissanayake, handwritten digit recognition,’’ Neural Netw., Stat. Mech. Perspective,
‘‘A systematic approach for MRI brain tumor localization and segmen- vol. 261, no. 276, pp. 261–276, 1995.
tation using deep learning and active contouring,’’ J. Healthcare Eng., [30] M. Raghu, C. Zhang, J. Kleinberg, and S. Bengio, ‘‘Transfusion: Under-
vol. 2021, pp. 1–13, Feb. 2021. standing transfer learning for medical imaging,’’ in Proc. Adv. Neural Inf.
[9] V. K. Reshma, N. Arya, S. S. Ahmad, I. Wattar, S. Mekala, S. Joshi, Process. Syst., 2019, pp. 3347–3357.
and D. Krah, ‘‘Detection of breast cancer using histopathological image [31] A. Canziani, A. Paszke, and E. Culurciello, ‘‘An analysis of deep neural
classification dataset with deep learning techniques,’’ BioMed Res. Int., network models for practical applications,’’ 2016, arXiv:1605.07678.
vol. 2022, pp. 1–13, Mar. 2022. [32] J. Chen, J. Chen, D. Zhang, Y. Sun, and Y. A. Nanehkaran, ‘‘Using deep
[10] S. Zhao, Y. He, J. Qin, and Z. Wang, ‘‘A semi-supervised deep learning transfer learning for image-based plant disease identification,’’ Comput.
method for cervical cell classification,’’ Anal. Cellular Pathol., vol. 2022, Electron. Agricult., vol. 173, Jun. 2020, Art. no. 105393.
pp. 1–12, Feb. 2022. [33] Z. Li and D. Hoiem, ‘‘Learning without forgetting,’’ IEEE Trans. Pattern
[11] M. Pedersen, M. B. Andersen, H. Christiansen, and N. H. Azawi, ‘‘Clas- Anal. Mach. Intell., vol. 40, no. 12, pp. 2935–2947, Dec. 2018.
sification of renal tumour using convolutional neural networks to detect [34] When Breast Cancer Metastasizes to the Brain? Accessed:b Feb. 21, 2022.
oncocytoma,’’ Eur. J. Radiol., vol. 133, Dec. 2020, Art. no. 109343. [Online]. Available: https://www.healthline.com/health/breast-cancer/brea
[12] M. Masud, N. Sikder, A.-A. Nahid, A. K. Bairagi, and M. A. AlZain, st-cancer-metastasis-to-brain#overview
[35] Can Cervical Cancer Lead to Kidney Failure? Accessed: Oct. 21, 2022.
‘‘A machine learning approach to diagnosing lung and colon cancer using
[Online]. Available: https://www.healthline.com/health/cervical-cancer/
a deep learning-based classification framework,’’ Sensors, vol. 21, no. 3,
cervical-bcancer-and-kidney-failure
p. 748, Jan. 2021.
[36] J. P. Leone and B. A. Leone, ‘‘Breast cancer brain metastases: The last
[13] A. H. Khan, S. Abbas, M. A. Khan, U. Farooq, W. A. Khan, S. Y. Siddiqui,
frontier,’’ Exp. Hematol. Oncol., vol. 4, no. 1, pp. 1–10, Dec. 2015.
and A. Ahmad, ‘‘Intelligent model for brain tumor identification using
[37] M. I. Sameen, B. Pradhan, and S. Lee, ‘‘Application of convolutional neu-
deep learning,’’ Appl. Comput. Intell. Soft Comput., vol. 2022, pp. 1–10,
ral networks featuring Bayesian optimization for landslide susceptibility
Jan. 2022.
[14] S. A. Alanazi, M. M. Kamruzzaman, M. N. I. Sarker, M. Alruwaili, assessment,’’ Catena, vol. 186, Mar. 2020, Art. no. 104249.
[38] U. Yunus, J. Amin, M. Sharif, M. Yasmin, S. Kadry, and S. Krishnamoor-
Y. Alhwaiti, N. Alshammari, and M. H. Siddiqi, ‘‘Boosting breast can-
thy, ‘‘Recognition of knee osteoarthritis (KOA) using YOLOv2 and clas-
cer detection using convolutional neural network,’’ J. Healthcare Eng.,
sification based on convolutional neural network,’’ Life, vol. 12, no. 8,
vol. 2021, pp. 1–11, Apr. 2021.
p. 1126, Jul. 2022.
[15] A. Akilandeswari, D. Sungeetha, C. Joseph, K. Thaiyalnayaki,
K. Baskaran, R. J. Ramalingam, H. Al-Lohedan, D. M. Al-dhayan,
M. Karnan, and K. M. Hadish, ‘‘Automatic detection and segmentation
of colorectal cancer with deep residual convolutional neural network,’’ MALLIGA SUBRAMANIAN received the Ph.D.
Evidence-Based Complementary Alternative Med., vol. 2022, pp. 1–8, degree from Anna University, Chennai, in 2010.
Mar. 2022. She has more than 26 years of teaching experience.
[16] K. Warin, W. Limprasert, S. Suebnukarn, S. Jinaporntham, and P. Jantana, She has done consultancy project for BPL and
‘‘Automatic classification and detection of oral cancer in photographic offered several courses on the latest technology.
images using deep learning algorithms,’’ J. Oral Pathol. Med., vol. 50, She is working as a Professor with the Department
no. 9, pp. 911–918, Oct. 2021.
[17] A. B. Tufail, Y.-K. Ma, M. K. A. Kaabar, F. Martínez, A. R. Junejo, I. Ullah, of Computer Science and Engineering, Kongu
and R. Khan, ‘‘Deep learning in cancer diagnosis and prognosis prediction: Engineering College, Tamil Nadu, India. Cur-
A minireview on challenges, recent trends, and future directions,’’ Comput. rently, she is guiding four research scholars. She
Math. Methods Med., vol. 2021, pp. 1–28, Oct. 2021. has also guided many U.G. and P.G. projects. She
[18] Acute Lymphoblastic Leukemia (ALL) Image Dataset. Kaggle, has published more than 50 articles in international journals and presented
San Francisco, CA, USA, 2021. more than 60 papers in national and international conferences in her research
[19] F. Spanhol. Breast Cancer Histopathological Database. Accessed: and other technical areas. Her main research interests include network secu-
Nov. 30, 2022. [Online]. Available: https://www.kaggle.com/datasets/anas rity, the IoT, deep learning, and NLP.
elmasry/breast-cancer-dataset
[20] Sipakmed. A New Dataset for Feature and Image Based Classifica
tion of Normal and Pathological Cervical Cells in Pap Smear Images.
Accessed: Oct. 2018. [Online]. Available: https://www.kaggle.com/ JAEHYUK CHO received the Ph.D. degree in com-
datasets/prahladmehandiratta/cervical-cancer-largest-dataset-sipakmed puter science, focusing on mobile and embedded
[21] CT Kidney Dataset: Normal-Cyst-Tumor and Stone. Accessed: Dec. 2021. computing systems, from Chung-Ang Univer-
[Online]. Available: https://www.kaggle.com/datasets/nazmul0087/ct- sity, South Korea, in 2011. He was a Profes-
kidney-dataset-normal-cyst-tumor-and-stone sor at the Department of Electronic Engineering,
[22] A. A. Borkowski, L. B. Thomas, C. P. Wilson, L. A. DeLand, Soongsil University. He was a National Research
and S. M. Mastorides. Lung and Colon Cancer Histopathological and Development Program Project Manager at the
Image Dataset (LC25000). Accessed: Dec. 2019. [Online]. Available: Korea Institute of S&T Evaluation and Planning
https://www.kaggle.com/datasets/andrewmvd/lung-and-colon-cancer-hist
(KISTEP), Seoul. He was a Senior Researcher at
opathological-images
[23] N. V. Orlov, W. W. Chen, D. M. Eckley, T. J. Macura, L. Shamir, and LG CNS, Seoul. He is currently a full-time Pro-
E. S. Jaffe, ‘‘Automatic classification of lymphoma images with transform- fessor with the Department of Software Engineering, Jeonbuk National Uni-
based global features,’’ IEEE Trans. Inf. Technol. Biomed., vol. 14, no. 4, versity, Jeonju, South Korea. His research interests include applied AI, data
pp. 1003–1013, Jul. 2010. process, bigdata of sensors, the IoT, smart city, and SW platform systems.

VOLUME 11, 2023 10353

 M. Subramanian et al.: Multiple Types of Cancer Classification Using CT/MRI Images

VEERAPPAMPALAYAM EASWARAMOORTHY OBULI SAI NAREN was studying at Kongu Engi-

SATHISHKUMAR received the bachelor’s degree neering College, Department of Computer Science
in information technology from the Madras Insti- Engineering as an undergraduate and graduated
tute of Technology, Anna University, in 2013, the with the bachelor’s degree in 2022. He is a Project
master’s degree in biometrics and cyber security Engineer with Soliton Technologies, India.
from PSG College of Technology, in 2015, and the
doctoral degree from Sunchon National Univer-
sity, in 2021. He worked as a Research Associate
at VIT University, from 2015 to 2017. He was
a Postdoctoral Researcher at the Department of
Industrial Engineering, Hanyang University, Seoul, South Korea. In 2021,
he worked as an Assistant Professor at the Department of Computer Science
and Engineering, Kongu Engineering College. He is currently a Postdoctoral
Researcher with the Department of Software Engineering, Jeonbuk National
University, South Korea. He published more than 60 research papers in
reputed journals and conferences. His research interests include data mining,
big data analytics, cryptography, digital forensics, artificial intelligence, and
computational chemistry. He received the South Korea’s prestigious Global
Korean Scholarship for pursuing the doctoral degree. He is a reviewer for
more than 200 journals and has reviewed more than 2000 research articles.
He is currently serving as an Academic Editor for the journals PLOS ONE
and Journal of Healthcare Engineering.

10354 VOLUME 11, 2023