Skin cancer detection

Using deep learning algorithms &

Patient's metadata

Supervisor:
Dr. Hossam Moftah
Eng. Mohammed Gamal
Page | 1
 Presented by: Vatican team

Team members Code

Basel Ayman 1800016

Zahwa Khaled 1800022

Mazen Tarek 1800041

Mohammed Hamed 1800046

Mohammed Abd-Elsalam 1800048

Nada Ahmed 1800055

Acknowledgment
We would like to express our gratitude for everyone who helped
us during the graduation project, starting with endless thanks for
our supervisor Dr. Hossam Moftah and Eng. Mohammed Gamal
who did any effort in encouraging us to do a great job, providing
our group with valuable information and advices to be better
each time.

Page | 2
 Table of Contents

I. Introduction/Executive Summary ....................................... 6

1.1 Introduce the project problem .............................................. 7
1.2 Related Discussion ............................................................. 11
II. Previous work ………...…….…........................................ 13
2.1 Related work …………………………………………….. 14
2.2 Comparing Our System with Existing Systems …………. 15
III. Methodology..................................................................... 17
3.1 Exploratory Data Analysis (EDA)...................................... 18
3.2 Data Processing ......................…........................................ 27
3.3 Modeling ………………………........................................ 31
3.4 Results …………………………........................................ 49
3.5 Developing results and their quality …………………….. 43
3.5 Save and convert model …………………………………. 50
3.6 Validation model ……………………………………….... 52
3.7 Developing results and their quality …………………...... 53
IV. Deployment &Technology............................................... 56
4.1 Tools and Environment ………………………………… 57
4.2 Methods ………………………………………………….. 58
4.2 App architecture…………...……………….…………… 62

Page | 3
V. System requirements& Designs …….............................. 65
5.1 Functional Requirements ………………………………... 65
5.2 Non-Functional Requirements …………………………... 66
5.3 System Design ...………………………………………… 67
VI. Application Design ………...……................................... 70
VII Testing & Evaluation …………………….....….…….. 105
VII Conclusion & Future work ………………………….. 112
8.1 Conclusion ………………………………………………113
8.2 Future work and other ideas ……………………………..114
VIII. References …………...…………................................ 116

Page | 4
Abstract
Skin diseases are more common than other diseases. Skin
diseases may be caused by fungal infection, bacteria, allergy, or
viruses, etc. The advancement of lasers and Photonics based
medical technology has made it possible to diagnose the skin
diseases much more quickly and accurately. But the cost of such
diagnosis is still limited and very expensive. So, image
processing techniques help to build automated screening system
for dermatology at an initial stage. The extraction of features
plays a key role in helping to classify skin diseases. Computer
vision has a role in the detection of skin diseases in a variety of
techniques.
Finding out the type of cancer can take months for doctors as it
is a very tedious practice and also requires the use of expensive
devices and contraptions. This would waste a good lot of time
for the patient which could have been utilized by the doctors to
treat the patient in time. It is also monetarily very demanding on
the patient’s part, which could pose a problem if the patient is
not financially settled.

Page | 5
 Chapter 1
Introduction

Page | 6
1.1 Introduction

Diagnosis in dermatology is largely based on visual

inspection of a lesion on the suspicious skin area. Therefore,
diagnostic ability and accuracy depends greatly on the
experience and training of dermatologists or general
practitioners, in areas where dermatological services are not
readily available. When dermatologists get no access to
additional technical support, they have an approximately 65%-
70% accuracy rate in skin cancer diagnosis. If the lesion is
suspicious, the visual inspection is supplemented with different
diagnostic tools (e.g. dermoscopy, confocal microscopy or
optical coherence tomography) providing the ability to explore
the skin in vivo, in depth and at a higher resolution. However,
access to these instruments remains limited due to time,
logistical and cost concerns.

Even when this technical support is feasible, dermatologists

rarely achieve average rates greater than 85%. The situation is
even worse if we consider that there is a shortage of
dermatologists whilst diagnostic accuracy of non-expert
clinicians is sensibly below than what is observed with
dermatologists, reaching estimate rates between 20 and 40%.
Thus, new diagnostic tools assisting dermatologists or general
practitioners to accurately diagnose skin lesions should be
developed, evaluated and optimized.

Page | 7
Analyzing cancers isn't an easy task. It requires intensive
examining. More than 50% of lesions are confirmed through
histopathology (histo), the ground truth for the rest of the cases
is either follow-up examination (follow-up), expert consensus
(consensus), or confirmation by in-vivo confocal microscopy
(confocal). The lack of experts (radiologists) has always been a
bottleneck. Now there are three things that we have to consider
here:
Given the fact that there are a limited number of experts, how
can we make them more efficient? Can we aid them using state
of the art machine learning techniques? If yes, how?
Training of neural networks for automated diagnosis of
pigmented skin lesions is hampered by the small size and lack of
diversity of available dataset of dermatoscopic images. Labelled
data in healthcare is another bottleneck. With the available
limited data, how much can we do?
As Machine Learning Engineers, if we can't help the doctors and
ultimately the society, then what are we good at? Healthcare is a
complicated field and using Machine Learning in this field has
its own advantages and disadvantages. There is a limit to which
we can do things with ML in healthcare but whatever we can do,
it matters!

The first step to identify whether the skin lesion is malignant or

benign for a dermatologist is to do a skin biopsy. In the skin
biopsy, the dermatologist takes some part of the skin lesion and
examines it under the microscope.

Page | 8
The current process takes almost a week or more, starting from
getting a dermatologist appointment to getting a biopsy report.
This project aims to shorten the current gap.

The approach uses Convolutional Neural Network (CNN) to

classify 7 types of skin lesions:

 Actinic keratosis and intraepithelial carcinoma: common

non-invasive variants of squamous cell carcinomas. They
are sometimes seen as precursors that may progress to
invasive squamous cell carcinoma.

 Basal cell carcinoma: a common version of epithelial skin

cancer that rarely metastasizes but grows if it isn’t treated.

 Benign keratosis: contains three subgroups (seborrheic

keratoses, solar lentigo, and lichen-planus like keratoses
(LPLK)). These groups may look different but are
biologically similar.

 Dermatofibroma: a benign skin lesion that is regarded as a

benign proliferation or an inflammatory reaction to minimal
trauma.

 Melanoma: a malignant neoplasm that can appear in

different variants. Melanomas are usually, but not always,
chaotic, and some criteria depend on the site location.

Page | 9
  Melanocytic Nevi: these variants can differ significantly
from a dermatoscopic point of view but are usually
symmetric in terms of distribution of color and structure.

 Vascular Lesions: generally categorized by a red or purple

color and solid, well-circumscribed structures known as red
clods or lacunes.

The overarching goal is to support the efforts to reduce the death

caused by skin cancer. The primary motivation that drives the
project is to use the advanced image classification technology
for the well-being of the people. Computer vision has made
good progress in machine learning and deep learning that are
scalable across domains. With the help of this project, we want
to reduce the gap between diagnosing and treatment. Successful
completion of the project with higher precision on the dataset
could better support the dermatological clinic work. The
improved accuracy and efficiency of the model can aid to detect
skin cancer in the early stages and can help to reduce
unnecessary biopsies.

We aim to make it accessible for everyone and leverage the

existing model and improve the current system. To make it
accessible to the public, we build an easy-to-use mobile app.
The user or dermatologist can upload the patient information
with the skin lesion image. With the image and patient
information as input, the model will analyze the data and return
Page | 10
the results within a split second. Keeping the broader
demographic of people in the vision, we have also tried to
provide the basic info of the skin cancer and the seven skin
lesions , which provides a generalized overview about skin
lesions and steps to use the online tool to get the results.

1.2 Related Discussion

Despite the advancements in technology, however the
inefficiency within the clinical dataset has restricted the utility
of deep learning in biological data. Melanoma is a most usual
skin cancer that showed huge mortality rate. It is estimated that
nearly 9,730 deaths have occurred due to melanoma in 2017.

Basal cell carcinoma commonly referred to as BCC is the most

usual skin cancer, however is commonly not fatal. So it is very
important for both health care services to diagnose the type of
cancer and to develop an efficient method to discriminate
different types of skin cancer will be useful for initial screening.
To improve a classification tool using biological images of 7
different known skin lesions - Dermatofibroma, Melanocytic
nevi, Basal cell carcinoma, Benign keratosis, Actinic keratosis,
Vascular lesions, Melanoma.

Huge efforts should be made to improve an image classification

methods for more precise prediction of lesions. In one of the old
Page | 11
studies, machine-guided diagnostic methods depending on
feature extraction method imaged a considerable diagnostic
capability with some types of skin cancer, which includes
melanoma. However, an Al algorithm would not create precise
diagnoses over a large varieties class of skin cancers. Not very
long back, deep CNN architectures became popular in object
classification going feature learning especially with image data.
Extensive research from ILSVR (ImageNet Large Scale Visual
Recognition Challenge) has depicted that object classification
abilities of CNNs can exceed over that of human diagnosis
abilities.

Many dermatologic studies showed the uses of machine or deep

learning. For example, Liao et ai. Used a CNN based model to
Classify top level 23 Categories Such as, viral infections,
bullous diseases etc. With 23000 images. It showed an accuracy
of 73.1% and 91.0% respectively for rates at which a model
gives Output of the correct label with top-1 and top-5
predictions for a given image. They have used a binary
classification CNN based model which gave an AUC of 0.96 for
carcinoma diagnosis using the above mentioned Edinburgh
dataset with 707 cases and gave AUC of 0.96 for melanoma
diagnosis which has 225 cases.

Page | 12
 Chapter 2
Previous work

Page | 13
2.1 Related work
As we know there are many apps that try to detect skin lesions
much easier. At the start of our project we have inquired about
on some auxiliary assets dependent on this. From these have
access to machine resources sources, we have had the option to
know the current capacities and decided our work plot.

There is an application called Medical Dermatology it’s a one of

the app in this field. It provide availability to make classification
for skin lesion and metadata like gender, age, etc. The drawback
of the app that it has bad interface that doesn’t attract users to
use and less metadata.

Another application called SkinVision it’s a mobile application

published and used in widely range only.it has a good and
creative user interface that help the patient to get good and
satisfy result.

Another application called Aysa it’s a mobile application with a

lot of features.it has the best and most creative user interface and
it is totally free and save patient history with a lot of medical
knowledge questions to reach the best accurate result.

Another application called Visus it’s a mobile application that

provide image classification for 7 types of skin lesion with bad
and not accurate accuracy.
Page | 14
Another application called AI Dermatologist it’s a mobile
application with amazing user interface to classify image lesion
but limited features and not free.

Another application called SmartSkin it’s a mobile application

with bad user interface and limited features and its accuracy not
accurate.

There are also a lot of applications that target skin lesions

classification but not reach the level to rely on it like All Skin
Dermatologist, Skin Diseases, AISkin, Scanoma and more.

2.2 Comparing Our System with Existing Systems

In our mobile application user all data will be stored with local
database so he doesn’t need to share his private data with
informal organizations. We developed malignant melanoma
detection based on dermoscopic images and patient's metadata
using an artificial intelligence (AI) model that will work on low-
resource devices with a good and creative user interface that
help the patient to get good and satisfy result supporting android
and iOS.

Page | 15
For find suitable doctor we provide feature that help you to find
the nearest doctor or hospital in your local area according to
GPS. Also we try to provide a lot of information about skin
cancer and its symptoms especially for 7 lesions with difference
between benign lesions and malignant lesions.

Page | 16
 Chapter 3
Methodology

Page | 17
Brief about the algorithm and its steps

3.1 Exploratory Data Analysis (EDA)

Here we discuss about the different features of the dataset, their
distributions and the count of that types present in the dataset.
This is helpful to analysis the nature of our data and helps us in
the data processing step. First we will see the number of
instances of data present for every possible values of every
feature of data feature wise study of the data. Before jumping
into analysis part let us look at from where the data is collected.
Page | 18
3.1.1 Data Collection
This the HAM10000 ("Human against machine with 10000
training images") dataset. It consists of 10015 dermatoscopy
images which are released as a training set for academic
machine learning purposes and are publicly available through
the ISIC archive. This benchmark dataset can be used for
machine learning and for comparisons with human experts.
https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.
7910/DVN/DBW86T

It has 7 different classes of skin lesion which are listed below:

 Melanocytic Nevi (NV)
 Melanoma (MEL)
 Benign Keratosis-like Lesion (BKL)
 Dermatofibroma (DF)
 Basal Cell Carcinoma (BCC)
 Actinic Keratoses (Akiec)
 Vascular Lesions (Vasc)

Page | 19
Melanocytic nevi, keratosis, vascular and dermatofibroma are
benign lesions
Melanoma, basal cell carcinoma and actinic keratosis are
malignant lesions

Page | 20
3.1.2 Feature wise study

1) Diagnosis

Its seems from the above plot that in this dataset cell type
melanocytic nevi has very large number of instances in
comparison to other cell types

2) Gender

Page | 21
55/45 distribution, dataset matches industry established
distribution

There is a 60/40 split in the Malignant population for males and

females due to the fact that under 50, melanoma occurs more
frequently in women, while above 50, occurs more often in men
and increasingly so moving into 80.

https://www.cancer.net/cancer-types/melanoma/statistics

Page | 22
 3) Age

It seems that there are larger instances of patients having age

from 30 to 60

Page | 23
 4) Site

It seems back, lower extremity, trunk and upper extremity are

heavily compromised regions of skin cancer

5) Technical Validation field (ground truth)

The distribution of its 4 categories which are listed below:

1) Histopathology (Histo): Histopathologic diagnoses of
excised lesions have been performed by specialized
dermatopathologists.

2) Confocal: Reflectance confocal microscopy is an in-vivo

imaging technique with a resolution at near-cellular level ,
and some facial benign with a grey-world assumption of all

Page | 24
 training-set images in Lab-color space before and after
manual histogram changes.

3) Follow-up: If nevi monitored by digital dermatoscopy did

not show any changes during 3 follow-up visits or 1.5 years
biologists accepted this as evidence of biologic benignity.
Only nevi, but no other benign diagnoses were labeled with
this type of ground-truth because dermatologists usually do
not monitor dermatofibromas, seborrheic keratoses, or
vascular lesions.

4) Consensus: For typical benign cases without histopathology

or follow up biologists provide an expert-consensus rating
of authors PT and HK. They applied the consensus label
only if both authors independently gave the same
unequivocal benign diagnosis. Lesions with this type of
ground truth were usually photographed for educational
reasons and did not need further follow-up or biopsy for
confirmation.

Page | 25
3.1.3 Average Color Information
Here we get and normalize all of the color channel information, the
shape of the image array is (450, 600, 3), 3 are the 3 channels: Red, Blue
and Green! Taking the mean across axis= (0, 1) gives the mean for each
3 channels.

Page | 26
3.2 Data Processing

3.2.1 Data Cleaning

Data cleaning is an important step in machine learning. Data
cleaning plays an important role in building a proper model.
Proper data cleaning can make or break the project. There is a
popular belief that "Better data beats fancier algorithms".

A different steps in data cleaning are,

1) Unwanted observations removal
This means deleting duplicate, irrelevant and redundant values
from dataset. Duplicate data arise mostly during data collection.
Irrelevant observations are those observations that does not fit
the certain problem that we are trying to solve. Our data does
not contain any unwanted or duplicate, we removed 57 unknown
values in gender field

2) Handling missing data

This type of data poses tricky problems in machine learning.
This data cannot be ignored or removed from dataset because
this data may contain important features specific to the
corresponding class other than that of missing feature. In this
step we find all the null values in the data and replace them with
mean and mode values of that field. In our data only ‘age’ field

Page | 27
has null data as shown in and they are replaced by the median
values.

3.2.2 Resizing Images

Original images in the dataset are all of same size but the size is
very large. The images are of dimension (450 x 600 x 3) as seen
this will have a huge computation time while training and
TensorFlow cannot handle this.
Let us resize the images by keeping the same size ratio so that
no information is lost. We have resized the images to dimension
(128 x 128x 3).

3.2.3 Data Normalization

Original images are represented in color code format with 3
values of Red, Blue and Green for each pixel and each value
ranging from 0 to 255. So the normalization is applied as
follows, normalized image = (original Image – mean (all
original images) / standard deviation (all original images)

After normalization, each value of color code format changed to

a range of -2 to 2 which is preferred by neural networks.

Page | 28
3.2.4 Data Augmentation
Deep neural networks perform better with large amount of data.
Aim of this step is to create images that depict the features of its
class in every possible angle. This makes sure that at whatever
angle the image may be taken, our trained model can predict it
with more precision.
Different techniques used for this are,
1) Randomly rotate the images in the range from 0 to 180
degrees
2) Randomly zoom images
3) Randomly flip images horizontally
4) Randomly flip images vertically
5) Randomly shift the images horizontally
6) Randomly shift the images vertically
Page | 29
Augmentation of images is done to deal with the problem of
skewed classes, overfitting, and training image scarcity. As can
be seen from the frequency table of classes, the NV class
dominates with approximately 67% of images in training data.
Hence, to balance the distribution various augmentation
techniques are implemented to increase the size of each class

Page | 30
3.3 Modeling
Convolutional Neural Networks are currently the most popular
and effective technique for image classification. In this project,
we aim to develop a CNN model for classifying Skin Lesion
images into 7 different classes of lesions. There are various
CNN models developed by companies like Google and
researchers based on ImageNet dataset. We have explored some
of these models by training them on ISIC 2018 Skin Lesion
Image dataset to verify whether they provide expected accuracy.
Moreover, we have developed custom models based on study of
various research papers. In this report, we have described all the
attempts made to develop a robust CNN model for classification.

We did extensive research to understand the intuition behind

making a robust CNN model especially for Skin Lesion
classification. Most of the research work we reviewed were
performing binary classification to detect whether a Lesion is
Melanoma or not. These models achieved good accuracy due to
binary nature of classification. Some of the research work was
for all the 7 classes of ISIC dataset however, the dataset used to
train was a small subset of 10k images that are available in ISIC
archives. Although the models developed in these papers
achieved good accuracy, they were not generalized model since
the amount of data used to train was less.
As we developed an idea about the purpose of different
convolution layers, number of filters, pooling and dropout, we

Page | 31
were able to tweak proposed and pre-implemented models to
train them for our ISIC 2018 dataset.
The following is a detailed description of different attempts
made to develop an efficient model along with limitations of
each model.

3.3.1 Baseline model (CNN)

CNN model proposed by researchers at Telkom University was
first utilized to develop a CNN model for ISIC 2018 images.
The researchers developed model for 4 different classes of
images while for our dataset, we tweaked the model to work for
7 classes of output layer. We used this model as a base reference
because it was used for skin lesion classification & achieved
good accuracy. Also, it is a simpler model for implementation
and understanding.

The architecture of this model is heuristically based we follow

the convention in famous DCNNs: using the smallest (3x3)
convolutional layers; and double the number of filters in the
output whenever the spatial activation size is halved to maintain
roughly constant hidden dimensions.
To train this model, data augmentation is employed. The
intuition of this method is to transform the training dataset a bit
in each epoch to produce variation and to guarantee that the
model will never see the same image twice.

Page | 32
Learning rate is initialized at 0.01 and Adam optimizer is used.
Learning rate decay is also used so that the learning rate will
halve whenever the validation accuracy plateaus for 3 epochs.
Baseline model is trained for a total of 30 epochs

Page | 33
The model give training accuracy of 78% with loss 0.56 and test
accuracy around 76% with loss 0.61

Show the changes in accuracy and loss training data and

validation data during training process as the epochs progress

Page | 34
Even though there are some spikes in both the validation
accuracy and loss, we can still say that this particular model with
image augmentation is doing well
Training and validation accuracies are much similar now, which
proves that the model is not overfitting
Showing the main classification metrics using classification
report

Computing confusion matrix to evaluate the accuracy of a

classification.

Page | 35
3.3.2 VGG16
Even though there are many DCNNs model achieving better
result on ImageNet than VGG16, we choose to fine-tune
VGG16 given its simplicity. The best performing VGG16 net is
similar except that the third, fourth and fifth convolutional block
has 4 convolutional layers.

Page | 36
To fine-tune VGG16, the top fully-connected layers are
removed, and new fully-connected layers (consisting of: one
global max pooling layers, one fully connected layer with 512
units, one dropout layer with 0.5 rate, one softmax activation
layer for 7 types of skin lesions) for our classification tasks are
added, First, freeze all layers in VGG16, and perform feature
Page | 37
extraction for the newly added FC layers so that the weights for
these layers aren’t completely random and the gradient wouldn’t
be too large when we start fine-tuning. After 3 epochs of feature
extraction, we unfreeze the final convolutional block of VGG16
and start fine-tune the model for 20 epochs. Throughout the
training process, learning rate of0.001 and Adam optimizer are
used. The same data augmentation and learning rate decay
strategy as in baseline model is used.

The model give training accuracy of 86% with loss 0.36 and test
accuracy around 60% with loss 0.78

Page | 38
Show the changes in accuracy and loss training data and
validation data during training process as the epochs progress

We find that the model with image augmentation is doing well

with Training data having high accuracy but we have overfitting
with validation data.

Showing the main classification metrics using classification

report

Page | 39
Computing confusion matrix to evaluate the accuracy of a
classification.

3.3.3 Inception V3
The namesake of Inception v3 is the Inception modules it uses,
which are basically mini models inside the bigger model. The
inspiration comes from the idea that you need to make a
decision as to what type of convolution you want to make at
each layer: Do you want a 3×3? Or a 5×5? The idea is that you
don’t need to know ahead of time if it was better to do, for
example, a 3×3 then a 5×5. Instead, just do all the convolutions

Page | 40
and let the model pick what’s best. Additionally, this
architecture allows the model to recover both local feature via
smaller convolutions and high abstracted features with larger
convolutions.

The larger convolutions are more computationally expensive, so

[4] suggests first doing a 1×1 convolution reducing the
dimensionality of its feature map, passing the resulting feature
map through a Relu, and then doing the larger convolution (in
this case, 5×5 or 3×3). The 1×1 convolution is key because it
will be used to reduce the dimensionality of its feature map.

Page | 41
The model give training accuracy of 80% with loss 0.56and test
accuracy around 76% with loss 0.66

Page | 42
Show the changes in accuracy and loss training data and
validation data during training process as the epochs progress

We find that the model with image augmentation is doing well

with Training data having high accuracy but we have some
overfitting with validation data.

Showing the main classification metrics using classification

report

Page | 43
Computing confusion matrix to evaluate the accuracy of a
classification.

3.3.4 Dense Net

Dense Net is competitive to Inception V3, but Dense Net has
less parameters (approximately 20M compare with approximate
23M of Inception V3). Dense Net 201 has 4 dense blocks. In a
dense block, the lth layer has l inputs, consisting of the feature-
maps of all preceding convolutional blocks, and its own feature-
maps are passed on to all subsequent layers L – l. Each layer
reads the state from its preceding layers and writes to the
Page | 44
subsequent layer. It changes the state but also passes on
information that needs to be preserved. Dense Net architecture
explicitly differentiates between information that is added to the
network and information that is preserved by concatenating
features instead of summing features as in Res Net.

Page | 45
The model give training accuracy of 95% with loss 0.1 and test
accuracy around 85% with loss 0.6

Show the changes in accuracy and loss training data and

validation data during training process as the epochs progress

Page | 46
We find that the model with image augmentation is doing well
with Training data having high accuracy but we have overfitting
with validation data.

Showing the main classification metrics using classification

report

Computing confusion matrix to evaluate the accuracy of a

classification.

Page | 47
Page | 48
3.4 Results

Test test
Model Accuracy Loss Depth Params
accuracy loss
Baseline 11
78% 0.56 76% 0.61 2,124,839
Model layers
23
VGG16 87% 0.36 78% 0.60 14,980,935
layers
Inception 315
80% 0.56 76% 0.66 22,855,463
V3 layers
Dense 711
95% 0.14 85% 0.62 19,309,127
Net 201 layers

When a model is underfit, we are oversimplifying the problem.

If you have pegs with the following shape: square, circle,
triangle, star, and hexagon, making the hole too big such that all
of them fit through would be an example of underfitting. This is
also considered error due to bias. Underfit learning curves are
often indicated by a flat or decreasing training loss until the end
of training, as well as a high training and validation loss.

When a model is overfit, we are overcomplicating the problem.

If you have a hole that only allows for the star shape to pass
through, that would be one way of thinking about overfitting.
Overfit learning curves are often indicated by training loss

Page | 49
continuing to decrease with epochs and validation loss decreases
and increasing again, where there is an increasing difference
between the training and validation loss. In other words, high
validation and low training error.

Noisy movements around training loss can be indicative of an

unrepresentative validation dataset or one that has too few
examples compared to the training dataset.

Best models were decided by lowest validation loss, which

ensures that the model is not already overfitting such that the
training and validation loss start diverging.

Some of the models looked like they could still be further

trained as the training loss was still continuing to decrease
through the end of the epochs.

3.5 Save and convert model

We saved model with best accuracy in H5 format so to used
model in real product like mobile application not in terminal we
decided to convert the saved model to tflite format

Page | 50
Advantages of using a TensorFlow Lite Model for on-device
Machine-Learning:

Latency: As inference is taken on the edge, there’s no round-trip

to a server resulting in low latency.
Data Privacy: Due to inference at edge, the data is not shared
across any net network. So the personal information doesn’t
leave the device resolving any concerns to the data privacy.
Connectivity: As no internet connectivity is required, there are
no connectivity issues.
Model Size: Tensor Flow Lite models are lightweight as the
edge devices are resource-constrained.

Page | 51
Power Consumptions: Efficient inference and lack of network
connectivity lead to low power consumption.

3.6 Validation model

To make sure that the photos entered are skin lesion images we
made a small model based on classification images to skin
image or not skin image. The dataset we used consisted of 8k
images of skin images and a random images of everything

Page | 52
(dogs, cats, trees, sky, wood... etc.). We can develop it using
mode data and better algorithm

We got that idea from an Alveoli application for Sreyom

Sresaan Indian engineer who helped us a lot.
https://github.com/sreyom31

3.7 Developing results and their quality

Combination of the patient's metadata can prevents the

overfitting that occurs in the CNN model using dermoscopic
images only.

We did research to find out the questions about symptoms that if

we ask the patient, it can lead to a better result. We created this
excell fill sheet with help of Dr Mostafa Mahmoud Eid
Page | 53
(Dermatologist at the Oncology Institute in Cairo( and Skin
cancer foundation https://www.skincancer.org/skin-cancer-
information/

Page | 54
The accuracy for CNN model 95% in training and 85% in test.
Combination of the patient's metadata prevents the overfitting
that occurs in the CNN model using dermoscopic images only
and achieve better results.

Page | 55
 Chapter 4
Deployment
&Technology

Page | 56
4.1.1 Tools

Software
Visual code: frame work of the implemented code
Firebase: cloud services from google used to save data

Languages:
Flutter: used in design user interface in mobile apps
Dart programming language: used in developing mobile apps on
multi platforms

Libraries:
Google maps: Google API to add maps to mobile app
Tflite: Flutter plugin for accessing TensorFlow Lite API.
Supports image classification
Image_picker: A Flutter plugin for iOS and Android for picking
images from the image library, and taking new pictures with the
camera.
Firebase_auth: Flutter plugin to allow use of firebase
authentication to
register and login users
Location: Flutter plugin to get current location of the user

Page | 57
4.1.2 Environment
CPU: We used CPU because the mobile application didn’t focus
on graphics or rendering images orvideos

4.2 Methods

The agile lifecycle is a structured series of stages that a product

goes through. It consists of six phases:

Requirements
Stakeholders conduct an overall project assessment to determine
the time and resources required for the development process. At
the same stage, the owner assesses the risks and prioritizes the
various functions depending on their business value.
Page | 58
Design
The software owner meets with the software development team
and introduces them to the requirements outlined in the first
step. The group then discusses the sequence for introducing
functions and identifies the essential tools – the programming
language, syntax libraries, and basic frameworks. At the same
stage, software development teams can prototype the expected
user interface.

Development and coding

After agreeing on the plan with the customer, the team develops
the product itself. The product is delivered in stages, in separate
sprints, each designed to improve the current version of the
product. The initial release is likely to undergo many changes to
provide improved functionality and new features.

Each cycle includes testing, and the final product must also
undergo final testing. For this phase, you can use Scrum and the
Kanban methodology, the development process based on
individual tasks.

Integration and testing

At this point, the product becomes available to consumers, so
the team must conduct a series of tests to ensure that the
software is fully functional. If potential bugs or flaws are found,

Page | 59
the developers will fix them immediately. At this stage, they
also collected consumer feedback.

Implementation and deployment

The software is now fully deployed and available to customers.
This action puts him in the maintenance phase. During this
phase, the software development team provides ongoing support
to keep the system running smoothly and fix any new bugs.
Over time, further iterations are possible to update an existing
product or add other functionality.

Review
That is the last stage of the agile development cycle. After
completing all the previous stages of development, the
development team presents to the owner the result achieved in
meeting the requirements. After that, the agile software
development phases start over – either with a new iteration or
moving to the next stage and scaling Agile.

Advantages of Agile Project Management

 You can deploy software quicker, so your customer can get
value sooner rather than later
 You waste fewer resources because you always work on
up-to-date tasks
Page | 60
  You can better adapt to change and respond faster
 Faster turnaround times
 You can detect and fix issues and defects faster
 You spend less time on bureaucracy and busywork
 There's a big community of Agile practitioners with whom
you can share knowledge
 You can get immediate feedback (which also improves
team morale)
 Developers can improve their skills based on QA feedback
 You don't have to worry about premature optimization
 You can experiment and test ideas because its costs are low

Disadvantages of Agile Project Management

 Documentation tends to get sidetracked, which makes it
harder for new members to get up to speed
 It's more difficult to measure progress than it is in Waterfall
because progress happens across several cycles
 Agile demands more time and energy from everyone
because developers and customers must constantly interact
with each other
 When developers run out of work, they can't work on a
different project since they'll be needed soon
 Projects can become ever-lasting because there's no clear
end
 Scope creep and experience rot
 Clients who work on a specified budget or schedule can't
know how much the project will actually cost, which makes

Page | 61
 for a very complicated sales cycle ("Until iteration ends," is
not something clients like to hear)
 The product lacks overall design, both from a UX and
architecture point of view, which leads to problems the
more you work on the product
 Teams can get sidetracked into delivering new
functionalities at the expense of technical debt, which
increases the amount of unplanned work
 Features that are too big to fit into one or even several
cycles are avoided because they don't fit in nicely into the
philosophy
 You need a long-term vision for the product and actively
work on communicating it
 Products lack cohesion, and the user journey is fragmented
because the design is fragmented. The more time passes,
the more disjointed the software ends up becoming
 Short cycles don't leave enough time for the design
thinking process, so designers have to redevelop the
experience over and over due to negative feedback

4.3 App architecture

In the following example, the specification will display the
range of activity for program users (patient) and how any of
them works for the framework and what the local database does.
Architecture Diagram is extremely relevant for any organization
or program, particularly in organizations that use architecture
Page | 62
diagrams to make a judgment or to take decisions that are not
decided upon by all managers of that business.

Page | 63
 Chapter 5
System requirements
& Designs

Page | 64
5.1 Functional Requirements
Functional Requirement is a summary of the function that the
program will deliver. Describes the software program or its
portion. Function is the input to the software system, its
behavior and outputs. It can be a measurement, data processing,
business method, user experience, or some other unique feature
that determines what role the program is likely to execute.

5.1.1 Login:
The system provides security features through username-
password matching where only authorized user can access the
system. We use firebase authentication to provide maximum
security.
Input:-Username, Password
Output: - Invalid or successfully logged in

5.1.2 Register:
This allows healthy public to register as donor, seeker or blood
Bank itself.
Input: Name, Email Address. Password, Contact Number
Output: - Successfully Registered.

Page | 65
5.1.3 Make Test
This function is for patient, in this function the user can make
request for making skin cancer test using his personal image and
metadata. Once he click this Tflite library make classification
using model which is pretrained

5.1.4 Find Doctor

This function is for patient, in this function the user can send
request for the find doctor in his location. Once he click this
request the google API use his location to search for the nearest
doctors in local area.

5.2 Non-Functional Requirements

The quality attribute of the software system is defined by a non-
functional requirement.
We reflect a collection of criteria used to determine the basic
function of the method. Example, how fast is the website
loaded? In order to maintain the reliability and productivity of
the whole operating program, a non-functional necessity is
necessary. Failure to meet non- functional specifications can
make systems that fail to meet user needs. Non-functional
Specifications enable you to place constraints or limits on the
architecture of the device.

Page | 66
5.2.1 Availability
The app are available at all times, meaning the user can access it
using application.

5.2.2 Reliability
As the app provide the right tools for problem solving it is made
in such a way that the app is reliable in its operations and for
securing the sensitive details

5.2.3 Performance:
Our App didn’t take more than few seconds if there is a good
internet connection.

5.3 System Design:

5.3.1 Use Case Diagram:

Use case diagram shows a graphical representation of the
interaction of device components with users. It helps to identify
the requirements of the system and to provide clearness of
understanding.

Page | 67
L

Page | 68
5.3.2 Activity Diagram:
Activity diagram is structure step-by - step process. Comprising
system flow and user interaction with system is helpful. The
diagram shows how user enters the system interface and
executes actions. This minimizes the risk of in-process
duplication

Page | 69
 Chapter 6
Application Design

Page | 70
6.1.1 Splash screen

Page | 71
6.1.2 Signup

Page | 72
6.1.3 Login

Page | 73
6.2.1 Main page

Page | 74
6.2.2 Test main page

Page | 75
6.2.3 Info for using app

Page | 76
6.2.4 Image selection

Page | 77
6.2.5 Display selected image

Page | 78
6.2.6 Select affected area position

Page | 79
6.2.7 Select affected area position

Page | 80
6.2.8 Select affected area size

Page | 81
6.2.9 Select duration of injury

Page | 82
6.2.10 Select if it is itch or no

Page | 83
6.2.11 Select if having fever or not

Page | 84
6.2.12 Select affected area shape

Page | 85
6.2.13 Select affected area color

Page | 86
6.2.14 Select type of tissue damage

Page | 87
6.2.15 Final result

Page | 88
6.3.1 Find doctor

Page | 89
6.3.2 Find doctor

Page | 90
6.3.3 Find doctor

Page | 91
6.3.4 Find doctor

Page | 92
6.4.1 Diseases main page

Page | 93
6.4.2 Skin cancer info page

Page | 94
6.4.3 Benign Keratosis info page

Page | 95
6.4.4 Vascular Lesion info page

Page | 96
6.4.5 Melanoma info page

Page | 97
6.4.6 Melanocytic Nevus info page

Page | 98
6.4.7 Dermatofibroma info page

Page | 99
6.4.7 Actinic Keratosis info page

Page | 100
6.4.8 Basal Cell Carcinoma info page

Page | 101
6.5.1 History page

Page | 102
6.5.2 History page

Page | 103
6.6 About developer’s page

Page | 104
 Chapter 7
Testing & evaluation

Page | 105
7.1 Testing & Evaluation
Testing is an essential step in this program as we can find issues
in testing, and our framework in general has problems. We
checked every step in our application, and we would check each
step, as I described before to achieve the project goal, our
strategy was gradual and iterative. There are various check
forms that you can use to ensure that code changes function as
intended. However, not everyone’s testing is the same.

7.1.1 Unit testing

Unit testing is very weak, near to the application's source. You
check specific methods and features of the software's classes,
elements or modules. Unit tests are typically cheap to carry out
with a continuous integration system and can be performed quite
quickly.

7.1.2 Integration testing

Integration testing ensure that the program fits together well
with various components or utilities. For example, the
relationship with the database can be checked. Such kinds of
experiments are more difficult to conduct, because certain parts
of the code are required for them to function.

Page | 106
7.1.3 Functional testing
Functional testing rely on the application's market
specifications. You just test the results of an operation and do
not track the system intermediate state during this operation.
Sometimes there is a discrepancy between integration testing
and functional tests, as they both tend to communicate with
multiple elements. The distinction is that an integration test will
only verify whether the database can be queried while a practical
test is required to derive a particular value from the database as
specified by the software specifications.

7.1.4 End-to-end tests

End-to -end testing in a full program environment replicates the
device experience of the applications. It verifies that different
user flows work as expected and can be just as simple as
downloading a website, logging into or verifying email alerts in
more complicated situations.

Page | 107
7.2.1 Login
Test Name Login
Test Scenario Login page form
Test Case Description Empty email or password field
Pre-requisites Email:[email protected]
Password:1234567
Test Input Nothing to write
Execution click on (sign in) button
Message show “please enter
email”,
Password must at least 7 digits
will appear.
check the empty fields in
email
Expected behavior An error toast message
contains “Empty Fields” will
appear
Assumptions Null
Actual Result

Page | 108
7.2.1 Register
Test Name Register
Test Scenario Register page form
Test Case Description Missing fields
Pre-requisites Null
Test Input Nothing to Entered
Execution Click “next” button
Errors on the empty fields will
appear with read text.
Start checking empty fields
and fill them with information
needed.
Expected behavior Focused errors on the empty
fields will
Appear with red labels.
Assumptions Design GUI will be tested in
alone
Scenario or as a test scenario.
Actual Result

Page | 109
7.2.1 Make test
Test Name Make test
Test Scenario Make test page form
Test Case Description Enter an image for no human
skin body
Pre-requisites Valid image for skin
Test Input Nothing to write
Execution - Click on processed
button.
- Warning dialog show
“Please enter an valid
image for human skin
body or continue anyway
”
Expected behavior An error toast message
contains “Invalid image”
Assumptions Null
Actual Result

Page | 110
7.2.2 Find doctor
Test Name Find doctor
Test Scenario Find doctor page form
Test Case Description Enter an invalid location
Pre-requisites Valid location
Test Input Nothing to write
Execution Click on processed button.
Warning dialog show “Please
enter an valid location for”
Expected behavior An error toast message
contains “Invalid location”
Assumptions Null
Actual Result

Page | 111
 Chapter 8
Conclusion & Future Work

Page | 112
8.1 Conclusion
This work involved a novel method involving deep learning
algorithm for skin lesion type identification based on image
data. This introduces a new method to identify lesion type
before medical diagnosis and helps the medical staff to diagnose
the type predicted by the model first. This reduces a lot of time
and effort. This model uses a Convolution Neural Network
(CNN) within a Tensor flow framework. For the specific cancer
types the accuracy ranges from 88% to 100%. And the overall
accuracy of the model is 95%.
Compared to traditional medical diagnosis this method is better
in two ways. First. This method will not be influenced by
diagnostic medical instruments that is error in the instruments
can occur during medical diagnosis and this will not have any
effect in this method if the images are of good quality. With
more and more cancer cases our model becomes more and more
powerful unlike medical instruments which has wear and tear.
Second, this method does not depend on stage of cancer and the
cancer can be detected even in an early stage. But some medical
diagnosis can detect some types of skin cancer only when it
reaches advanced stage and this does not occur with this model.
If among seven types, we add one more class which is non-
malignant cancer type and then train the model, our model can
even detect whether it is malignant or not and also tells the type
of cancer.

Page | 113
This model may have some limitations like the prediction
probability cannot be 100%. But it provides a good basis for
further diagnosis and will be very helpful. The CNN model with
metadata can increase the accuracy of classification in skin
cancer detection even with limited data and is promising for
development as a screening device in remote and low resources
health care.

8.2 Future work and other ideas

In many of the image classification projects correct results
depends mostly on dataset. Our dataset contains 10,015 images,
but most of those images are of only one class Melanocytic
Nevi. It is important to collect a balanced data for better
predictions. Also our dataset has only seven different lesion
types whereas in practical there are many more different types of
skin cancer. One of the future work is to collect a better dataset
with more balanced data with more lesion types. Also our data
only contains cancer images of patients. This model can only tell
different types of cancer, but cannot tell whether the lesion itself
is malignant or not. Along with the 7 classes if we can get
images of non-malignant lesions and add them to our data and
then train the model, we can also detect cancer and also tell the
type of cancer.

Page | 114
The skin cancer detection app places the control in the palm of
your hand to save lives. To developing our app and make it
accessible and more efficient we target a lot of plans:
1- Connect the application to a cloud database so that we can
see a large amount of data and improve the accuracy
2- Allows you to contact the doctor directly from the
application
3- Sending the patient’s data to the personal doctor on an
ongoing basis
4- Send alerts to the patient to perform a periodic examination
to ensure his safety
5- Make our own application that supports different social
groups
6- Adding more diseases and not just 7 to get better results

Page | 115
References
https://github.com/datascisteven/Melanoma-Image-
Classification
https://github.com/ashishpatel26/Skin-Lesions-Detection-Deep-
learning
https://www.kaggle.com/code/jagdmir/siim-melanoma-
classification-modelling/notebook#Some-insights-about-the-
training-and-testing-data
https://www.kaggle.com/code/sid321axn/step-wise-approach-
cnn-model-77-0344-accuracy#Step-11:-Model-Building
https://www.kaggle.com/code/jagdmir/all-you-need-to-know-
about-cnns
https://github.com/Geeky-star/Skin-Cancer-Detection-App
https://github.com/MoatazMaher1998/Skin-Cancer-Detection-
Mobile-App
https://github.com/DefineDan/ml-skin-cancer-classification-app
https://github.com/datascisteven/Melanoma-Image-
Classification
https://github.com/ruoyzhang/Skin_Cancer_Detection_with_Gra
dCam
https://github.com/Tirth27/Skin-Cancer-Classification-using-
Deep-Learning
https://github.com/sj-singh/DL_Melanoma_Detection

Page | 116
https://github.com/SwagatSBhuyan/Skin-Cancer-Classification-
Using-CNN-Deep-Learning-Algorithm
https://github.com/AakashKumarNain/skin_cancer_detection
https://www.skincancer.org/skin-cancer-information/
https://www.skincancerpractice.com.au/skin-cancer/introduction
https://towardsdatascience.com/building-a-skin-lesion-
classification-web-app-16fd2c422b9d
https://www.medrxiv.org/content/10.1101/2020.05.03.20072454
v1.full.pdf

THANK
VATICAN - 2022

Page | 117