Adam Optimizer
Adam Optimizer
Abstract. Skin cancer is a type of cancer that grows in the skin tissue, which can cause damage
to the surrounding tissue, disability, and even death. In Indonesia, skin cancer is the third
leading for most cancer cases after cervical and breast cancer. The accuracy of diagnosis and the
early proper treatment can minimize and control the harmful effects of skin cancer. Due to the
similar shape of the lesion between skin cancer and benign tumor lesions, physicians consuming
much more time in diagnosing these lesions. The system was developed in this study could identify
skin cancer and benign tumor lesions automatically using the Convolutional Neural Network
(CNN). The proposed model consists of three hidden layers with an output channel of 16,32,
and 64 for each layer respectively. The proposed model uses several optimizers such as SGD,
RMSprop, Adam, and Nadam with a learning rate of 0.001. Adam optimizer provides the best
performance with an accuracy value of 99% in identifying the skin lesions from the ISIC
dataset into 4 classes, namely dermatofibroma, nevus pigmentosus, squamous cell carcinoma,
and melanoma. The results obtained outperform the performance of the existing skin cancer
classification system.
1. Introduction
Skin cancer is a disease caused by changes in the properties of normal skin cells to
become malignant, in which cells will continue to divide into abnormal shapes that are
uncontrolled due to DNA damage. Based on histopathology view, skin cancer has an
irregular structure with cell differentiation in various levels of chromatin, nucleus, and
cytoplasm [1]. Skin cancer is a malignancy disease that is often found in Indonesia in
addition to cervical cancer and breast cancer. Skin cancer found 5.9 to 7.8% for all types
of skin cancer per year. The most common skin cancers in Indonesia are basal cell
carcinoma (65.5%), followed by squamous cell carcinoma (23%), malignant melanoma
(7.9%), and other skin cancers [2]. Even though the number of Malignant Melanoma
incidents is smaller than Basal Cell Carcinoma and Squamous Cell Carcinoma, the death
rate tends to be greater, which causes 75% of deaths from skin cancer [1]. The most
invasive skin cancer is melanoma, which has a high mortality rate, especially if it is not
early detected. Non-melanoma skin cancers, such as basal cell carcinoma and squamous
cell carcinoma are more common but less metastatic, and only partially leads to disability
or death. Accurate diagnosis and early detection of skin cancer can help the healing process,
proper medical treatment, and avoid the worst effects of skin cancer. Therefore, an early
detection system is needed that can facilitate and increase public awareness in identifying
types of skin cancer or other skin disorders such as a benign tumor on the skin that
look very similar to skin cancer.
Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution
of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Published under licence by IOP Publishing Ltd 1
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
The automatic skin disorders classification can help people in identifying skin disorders
that occur and immediately consult with medical personnel to get appropriate medical
treatment. Several related studies based on digital image processing for the detection and
classification of skin cancers were developed as a tool for medical personnel to diagnose skin
disorders more accurately with fast computing time. The previous research [3] was
developed using an edge detection method with K-NN and C-NN algorithms provide an
accuracy of 75% and 75.6% respectively to classify skin disorders that potentially benign
cancer and skin disorders that have the potential to be malignant cancer by using the
International Skin Imaging dataset Collaboration (ISIC). In research [4], using the ISIC
dataset for the conditions of skin cancer and skin benign tumors, an automatic skin disease
classification system was developed based on deep learning with PNASNet-5-Large
architecture which gives the best performance accuracy of 76%. Furthermore, other studies
using CNN for the detection of skin diseases [5] [6] [7] provide performance accuracy of
80.52%, 86.21%, 87.25% respectively. To increase the amount of data and to improve the
performance of the skin cancer detection system, the ISIC data augmentation process was
performed [8]. The best accuracy was obtained by 95.91% using Alexnet. In the study [9],
the CNN method with random regulators gave performance accuracy of 97.49% to
distinguish some lesions of skin disorders such as nevus lesions, carcinomas, and
melanomas. In this study, augmentation data from the ISIC dataset will be used to recognize
the condition of skin cancer lesions or benign tumor lesions that look similar to cancer. The
proposed method in this study uses CNN with various optimizer methods such as SGD,
RMSprop, Adam, and Nadam optimizer to find out which optimizer provides the best
performance.
2.1. Dermatofibroma
Dermatofibromas include the category of benign tumors caused by an overgrowth of a
mixture of various types of cells in the dermis layer of the skin. The skin growth that causes
dermatofibroma usually occurs after experiencing several types of minor trauma to the
skin, such as stab wounds caused by glass splinters or insect bites. Characteristics of
dermatofibromas measuring around 2-3 mm, purplish brown, hard structure, and painful
when pressed [11].
2
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
tumor can continue to grow to be cancer which can spread to bones, tissues, and even
lymph nodes due to the lateness of treatment. The more widespread the spread, the more
difficult the cancer is to be treated.
2.4. Melanoma
Melanoma is a type of skin cancer that is very dangerous. This condition starts with
human skin and can spread to other organs in the body. This disease is a type of skin
cancer originating from melanocyte cells, melanin-producing cells that are usually found
in the skin. Melanoma has an irregular shape and more than one color. Moles affected by
melanoma can feel itchy and can bleed, also, their size can exceed normal moles [14].
3
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
( ) { (1)
3.5. Hyperparameter
Hyperparameter has variable values that remain during the model training process and
can affect the performance of the model trains. In this study, the hyperparameter that
used is an optimizer, such as Stochastic Gradient Descent (SGD), Root Mean
Square Propagation (RMSprop), Adaptive Moment Estimation (Adam), and Nesterov-
accelerated Adaptive Moment Estimation (Nadam). Stochastic Gradient Descent (SGD)
is a repetitive optimization method that functions to optimize the model using better
functions such as differential or subdifferential [18]. SGD uses each training sample as a
new parameter. Root Mean Square Propagation (RMSprop) is widely used in the design
4
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
of deep learning models [19]. This optimizer is an improvisation from Root Propagation
(Rprop). Initially, Rprop cannot be used on files with large amounts of data. The
essence of RMSprop is moving the average gradient at the time of the model. Adam
optimizer is a combination of RMSprop and momentum. This optimizer also uses an
average gradient of weight [20]. The advantage of Adam over other optimizers is efficient
in computing time, consume less memory, and can handle sparse gradients on noisy
problems. Nadam (Nesterov-accelerated Adaptive Moment Estimation) thus combines
Adam and NAG (Nesterov accelerated gradient).
4. System Design
In this study, the dataset used is an augmentation of the ISIC dataset for the conditions of
dermatofibroma, nevus pigmentosus, squamous cell carcinoma, and melanoma.
Dermatofibroma and nevus pigmentosus are benign tumors, while squamous cell
5
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
carcinoma and melanoma are skin cancers. The total amount of augmentation data is
4000 images consisting of 1000 images for each class. The distribution of training data
and validation data were 75% and 25% so that the training data used were 3000 images
and the validation data used were 1000 images.
Based on Figure 5 and Table 1, the resolution of skin images is changed to 128 × 128
pixels as an input of the CNN model which consists of 3 hidden layers. The image is
convoluted using 3 × 3 filters on each hidden layer with the number of output channels on
each layer is 16, 32, 64 respectively. At each layer, the activation process uses Rel-U
activation and Max pooling. The result of Maxpooling reduces the size of the image as
can be seen in Figure 5 and Table 1. After that, the flatten process will change image
features from 3 dimensions became 1 dimension. The last, softmax activation function
will be used to classify the condition of skin image into four classes namely,
dermatofibroma, nevus pigmentosus, squamous cell carcinoma, and melanoma.
(2)
(3)
(4)
(5)
6
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
True Positive (TP) indicates the condition while the data is positive and correctly
predicted as positive. True Negative (TN) indicates the condition where the data is
negative and correctly predicted as negative. False Positive (FP) indicates the conditions
where the data is negative but incorrectly detected as positive. Whereas False Negative
(FN) indicates the conditions where the data is positive, but incorrectly detected as
negative.
Based on the results shown in Figure 7, it shows the performance comparison of each
optimizer used, Adam optimizer provides the best accuracy performance and loss
performance when compared to other optimizers. The performance accuracy using SGD,
RMSprop, and Nadam optimizer for training and validation data can decrease suddenly so
that the loss also rises at certain epochs. This is indicated by the appearance of many spikes
on the accuracy and the loss graph. Whereas, the system model that uses Adam optimizer
7
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
continues to show an increase in accuracy at each iteration (epoch) and the difference in
accuracy between the training data and the validation data is not much different, as well as
for the system loss continues to decrease at each iteration. This condition shows that the
proposed model is not overfitting and the system model used can recognize the condition
of dermatofibroma, nevus pigmentosus, squamous cell carcinoma, and melanoma with
the best accuracy performance of 99% and loss of 0.0346.
Confusion Matrix for system models with Adam optimizer can be seen in Figure 6. It
can be seen that from 1000 validation images used, 990 images were successfully
classified according to their class. The error occurs in four images on nevus pigmentosus
detected as melanoma and six images on melanoma detected as nevus pigmentosus. Other
parameters used to evaluate system
8
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
performance are Precision, Recall, and F1-score, which has a range of values from 0 to 1
(a value of 1 indicates no error). Based on the data shown in Table 2, the value of system
performance parameters obtained is close to 1, so it can be concluded that the CNN
model proposed can classify the conditions of dermatofibroma, nevus pigmentosus,
squamous cell carcinoma and melanoma with high accuracy and provides a minimum
error.
6. Conclusion
In this study, an automatic system was designed to classify the conditions of
dermatofibroma, nevus pigmentosus, squamous cell carcinoma, and melanoma based on
digital image processing. The CNN model used in this study consists of 3 hidden layers,
using 3 × 3 filter sizes with 16, 32, and 64 channel outputs in sequence, a fully connected
layer, and softmax activation. The optimization is performed on the proposed model
using SGD, RMSprop, Adam, and Nadam optimizer. Based on testing that has been
carried out the CNN model proposed with Adam optimizer provides the best
performance in classifying the dataset of skin cancer lesions and benign tumor lesions
with 99% accuracy, loss of 0.0346 and the value of precision, recall, F1- score is almost
1. Based on the performance results, the system shows that the proposed model is
promising to use as an existing tool for medical personnel in determining the diagnosis of
skin cancer or benign tumors. In further research, systems can be developed to classify
the various types of skin cancer and other skin diseases.
References
[1] Buljan M, Bulana V and Sandra S 2008 Variation in Clinical Presentation of Basal Cell Carcinoma
(Kroasia: University Department of Dermatology and Venereology Zagreb Croatia) p 25-30.
[2] Cipto H, Suriadiredja AS. Tumor kulit. Dalam: Menaldi SL, Bramono K, Indriatmi W, editor. Ilmu
penyakit kulit dan kelamin. Edisi ketujuh. Jakarta: Badan Penerbit FKUI; 2016. h.262-276.
[3] Teresia R, Savera, Winsya H, Suryawan and Agung WS 2020 Deteksi Dini Kanker Kulit
Menggunakan K-NN dan Convolutional Neural Network J. JTIIK. 7 2 p 373-378.
[4] Md Ashraful Alam Milton 2018 Automated Skin Lesion Classification Using Ensemble of Deep Neural
Networks in ISIC: Skin Lesion Analysis Towards Melanoma Detection Challenge.
[5] Serban Radu SJ, Loretta Ichim, et al 2019 Automatic Diagnosis of Skin Cancer Using Neural
Networks (Bucharest, Romania: The XIth International Symposium on Advanced Topics in
Electrical Engineering March 28-30).
[6] Nazia Hameed, et al Multi-Class Skin Diseases Classification Using Deep Convolutional Neural
Network and Support Vector Machine.
[7] Xinyuan Zhang, et al 2018 Towards Improving Diagnosis of Skin Diseases by Combining Deep Neural
Network and Human Knowledge BMC Medical Informatics and Decision Making 18(Suppl 2) p
59
[8] Khalid M. Hosny, et al 2019 Classification of skin lesions using transfer learning and
augmentation with Alex-net PLOS ONE.
[9] Marwan AA 2019 Skin Lesion Classification Using Convolutional Neural Network With Novel
Regularizer IEEE Access.
[10] The International Skin Imaging Collaboration (ISIC).Accessed: June 2020. [Online].
Available: https:
//www.isicarchive.com/#!/topWithHeader/onlyHeaderTop/gallery
[11] Mousumi Roy, et al 2016 Dermatofibroma: Atypical Presentations Indian J. Dermatology.
[12] R. Delila Tsaniyah, Aspitriani and Fatmawati “Prevalensi dan Gambaran Histopatologi Nevus
Pigmentosus di Bagian Patologi Anatomi Rumah Sakit Dr. Mohammad Hoesin Palembang,”
Periode 1 Januari 2009-31 Desember 2013.
[13] Syril Keena T. Cutaneous squamous cell carcinoma. Journal of The American Academy of
9
ICETIR 2020 IOP Publishing
IOP Conf. Series: Materials Science and Engineering 982 (2020) 012005 doi:10.1088/1757-899X/982/1/012005
10