DIAGNOSIS OF RETINAL DISEASES WITH

APPEARANCE USING HYBRID ALGORITHM

OF DEEP LEARNING
S Kumar M Livan Kumar J Sai Krishna
B.Tech. student, Dept. of CSE B.Tech. student, Dept. of CSE B.Tech. student, Dept. of CSE
Institute of Aeronautical Engineering Institute of Aeronautical Engineering Institute of Aeronautical Engineering
Hyderabad, India Hyderabad, India Hyderabad, India
[email protected] [email protected] [email protected]
0000-0001-7049-2301 0000-0002-0364-8481 0009-0008-7467-6253

K Rashmi RM Noorullah
Assistant Professor, Dept. of CSE Associate Professor, Dept. of CSE
Institute of Aeronautical Engineering Institute of Aeronautical Engineering
Hyderabad, India Hyderabad, India
[email protected] [email protected]
0000-0002-5251-5685
0000-0001-6742-9432

Abstract— Abstract—Diabetic retinopathy (DR) is a major show any sign until the later stages. Lack of early and
cause of blindness, and it is important to identify symptoms accurate detection of DR further worsens the condition.
early enough to allow appropriate intervention and vision-
saving treatment. For this reason, the early diagnosis of DR is
of crucial importance to ophthalmologists. Recently, many
advances have been made in handling big datasets as the
demand for diagnosis and analysis of large numbers of images
and telemedicine has risen considerably, and with the help of
these developments, computers are now taking over many of
those tasks. Since disease diagnoses can be very subjective,
involving intuition and experience, trained systems based on
complex calculations are gaining popularity. This is a study
that employs hybrid convolutional neural networks (CNNs) for
automated DR screening based on fundus images. Researchers
adopt 2 well-known deep learning architectures (VGGNet and
ResNet) for DR diagnosis, train the models separately, and
argue that combining both nets improves accuracy and
generalization in the diagnosis of DR. This hybrid view will be
able to capture a wider range of features due to ResNet. the
vanishing gradient problem is addressed using (2). The deep
learning-based diagnosis is able to recognize and classify 4
labels of DR. This is a decision function. The weight Fig 1. Microaneurysms, Hemorrhages, and Exudates.
parameters will be trained under several input images and
tested by standard performance metrics: accuracy, sensitivity,
precision and f1-score on a well-known dataset. It is believed
By the time it is detected, the patients might have already
that using the proposed method will improve patient outcomes started losing their sight. [9] When an ophthalmologist
and alleviate the global burden of blindness, avoiding and perceives leakage or exudates shown from the inner-back
detecting DR early to make sure it remains in its early stages, surface, there are several signs such as microaneurysms,
when it is more manageable. hemorrhages and exudates. Conventionally,
ophthalmologists perform thorough examinations in
Keywords— Diabetic retinopathy, VGG16, ResNet50, Residual retrieving signs from the retinal images or fundus
Connection. photographs captured from the victim of this condition. By
doing so, we can diagnose the problem. However,
ophthalmologists could miss signs due to their subjective
I. INTRODUCTION judgment and due to the limited time given for the
Diabetic retinopathy (DR) is one of the most common examination. Furthermore, ophthalmologists are always
complications among diabetic patients and has been a not always available, especially in rural areas. Deep
global health burden over the past three decades. This learning, an advanced machine learning technique
condition destroys the fine tissue on the inner-back surface mimicking the human brain, could help us overcome these
of the eye called the retina, rendering sight incapacitation limitations. [10] When it comes to reviewing medical
if not treated properly. Severe situations can lead to images, the deep learning architecture can explain the
permanent blindness. [1] The worst part is that it might not medical data given. It is known to have a capability of
analyzing a significant amount of data, making it suitable

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

for health domain applications. We tested a computerized method of stacked generalization involving convolutional
system that investigates the images of DR with two deep neural networks (CNN). The results of the proposed hybrid
learning architectures: VGG (16) and ResNet (50). model demonstrate an impressive result of 99.92% accuracy
Looking at existing architectures and inspired by human in binary and 87.45% in multi-class classification. Got
information and knowledge processing, including the results affirm the effectiveness of the proposed illumination
perception and association, processes at different levels of normalization methodology. Enhanced the deep learning
our organism, we were able to detect pictures and start the model, producing superior outcomes compared to several
work to build our models.[3] VGGNet is a deep learning cutting-edge techniques. An eye disease of diabetes that
algorithm capable of extracting high-dimensional features harms the health condition at the rear eye blood vessels,
in each layer from the input image: 13 convolution layers potentially resulting in total blindness and other ocular
and 3 fully connected layers, ending with a single issues.
classification output. Since VGGNet is the Keras MXNet
by-product, it was easy for us to work with it, running on “Diabetic Retinopathy Grading Using ResNet
Google Colab. On the other hand, ResNet boosts the Convolutional Neural Network (2020):” [6] This paper
image recognition quality 5% by introducing skip presents a comprehensive process for handling and grading
connections, also known as residual connections. Here, we retinal fundus images employing the ResNet Convolutional
approach building a hybrid model by combining ResNet Neural Network. Pipeline includes image pre-processing,
and VGGNets, each with its own merits. ResNet is feature extraction, and classification. Gaussian filtering is
renowned for its ability to model the vanishing gradient used to standardize images and enhance quality. The
problem, while VGGNet is well-known for feature convolution architecture is used in the second and third
extraction. Thus, ResNet serves as learning, stages. Transfer learning and fine-tuning Implemented on
understanding, transferring and interpreting the ResNet18 with the Kaggle dataset, addressing the diagnosis
information present in our input, while VGGNet excels in of Diabetic Retinopathy involves five severity levels. The
feature extraction. We hope to create a hybrid deep final model achieved 70% accuracy, achieving a recall of
learning model that outperforms separate models. 50% and a specificity of 88%, the model surpasses others
constructed from the ground up with shorter training
durations. The training procedure addresses the challenge of
II. RELATED WORK imbalanced datasets through two distinct methods, with the
utilization of an imbalanced dataset sampler proving to be
“A Comprehensive Review of Deep Learning an effective solution. The resulting model has the potential
Strategies in Retinal Disease Diagnosis Using Fundus to serve as the central component in the detection and
Images (2022):” [12] The five main eye conditions that are diagnosis of Eye disease in diabetics.
the subject of this study are Eye disease of diabetes, acute
glaucoma, cataract, Eye Wiki (age-related macular “Automatic recognition of severity level for
degeneration), and Terry syndrome. The authors highlight diabetic retinopathy using deep visual features (2017):” [2]
the importance of fundus imaging in identifying ocular Eye disease of diabetic (DR) stands as the root cause in
structures and biomarkers, such as Microaneurysms (MAs), individuals with diabetes. whose prevalence is predicted to
Hemorrhages, and Hard Exudates (HEs). The authors also rise to 7.7% by 2030. Diabetes-related problems like DR can
discuss the challenges and envision prospects in the realm of result in blindness. Diabetes comes in two basic forms: type
diagnosing retinal diseases. [4] Fundus imaging, as a non- I (75–95%) and type 2 (60–60%). Prompt diagnosis and
invasive and cost-effective method, plays a crucial role in ongoing observation can slow down the loss of vision.
this domain for examining the human eye, with fundus Scatter laser photocoagulation has been shown in the
photography being a popular choice Because of its non- diabetic retinopathy study (DRS) to lower risk in 90% of
invasive characteristics, the study underscores the cases, with severe vision loss. To evaluate or classify the
importance of evaluating the existence of these lesions in severity of DR, clinical professionals employ computer-
conjunction with other retinal biomarkers. This is essential aided diagnosis methods and non-mydriatic cameras.
for gaining valuable insights into significant retinal diseases However, the requirement for considerable expert
and facilitating their diagnosis. The authors conclude that knowledge in the domain, and identifying severity levels is
deep learning is a promising tool for detecting and difficult in using both systems. In this work, an automatic
diagnosing retinal diseases using fundus images. recognition technique utilizing deep visual features (DVFs)
for the five-level disease of the diabetic is presented. The
“Diabetic Retinopathy Diagnosis from Fundus system's accuracy was assessed and compared using cutting-
Images Using Stacked Generalization of Deep Models edge methods. The findings show that the SLDR approach is
(2021):” [15] This research presents a methodology to suitable for DR early diagnosis and offers a useful treatment
eliminate color aberrations and irrelevant illuminations in for diabetes type prediction and computer-aided diagnosis
eye fundus images for diabetic retinopathy diagnosis. The systems. In this work, a automatic recognition technique
researchers employ an innovative approach using image utilizing deep visual features
processing and Deep Learning techniques to enhance image
quality. Luminosity normalization is achieved through the Table 1: A summary of the related works presented in this
gray World algorithm, which leads to Color Constancy. The study.
effectiveness of this technique is assessed by using PSNR
and MSE metrics applied to the normalized image. [8]
Additionally, the researchers introduce a computer-aided
diagnostic system based on deep learning, utilizing a unique
 (DVFs) for the five severity levels of DR is presented. The
system's sensitivity, specificity, and area under the receiving
operating curves (AUC) were assessed and compared using
cutting-edge methods. The findings show that the SLDR

Authors Year Method Dataset Acc

(Images)
Qaisar Abbas 2017 Deep visual features 750 94.3%
Muhammad Samer 2020 ResNet50 35122 69.4%
Sallam
Abhishek 2021 VGG-16 5396 74.58%
Deshpande
Harshith Kaushik 2021 Stacked CNN 2471 97.9%
BALLA 2022 A Comprehensive Review of Deep N.A N.A
GOUTAM1 Learning Strategies
approach is suitable for DR early diagnosis and offers a
useful treatment for diabetes type prediction.

III. METHODOLOGY AND MATERIALS

PreProcessing: Since preprocessing is a crucial stage in DR
detection and classification, the DR images were
subsequently preprocessed. The preprocessing procedures
used in this investigation included data augmentation, image
resizing.
DataSet Description: [5] An optical device called a
background camera, which consists of a camera coupled to
a low-power microscope, records retinal fundus images. It
can be applied to both the imaging and illumination of the
retina at the same time. The retina, optic disc, macula, and
posterior pole of the eye were the main areas of the interior Fig 2. The percentage of training and testing data
that were intended to be imaged. The benchmark dataset for
DR diagnosis supplied by EyePACS was obtained through Methodology: A variation of the well-known VGG-Net
a Kaggle competition and utilized in our investigation. A convolutional neural network is the VGG-16. Thirteen
web-based tool called EyePACS was created to assist convolutional layers and three fully linked layers make up
patients with DR problems from a distance without a the layers of the VGG-16 architecture. That means he has to
physician's assistance. It's a platform where medical learn how much 16 diapers weigh. A colour image with
professionals can cooperate and exchange their work, which dimensions of 224 by 224 pixels is fed into the algorithm,
may then be used for research. With the utilization of their which assigns it to one of the 1000 classes. [11] As a result,
open-source retinal database, EyePACS enabled Google a vector of size 1000 is returned, containing the likelihood
and Kaggle to conduct a competition, aimed at addressing of falling into each class. Color filters measuring 3 × 3
DR. Participants were able to contribute to the research by pixels and shifted in steps of one pixel are used in each
opening new areas of study. convolutional layer. To ensure that the input volumes and
output dimensions are the same, the zero padding is set to
[7] We have used 5740 images and divided them as 25% for one pixel. The quantity of filters differs based on the "block"
validation and 75% for training. All images are colored and that the layer is situated in. For every filter, a bias parameter
kept in the original .JPEG format. Since images were not is also added to the convolution product. [14] The purpose
acquired by the same camera and were in different lighting of every convolutional layer is to turn on a ReLU. Stated
conditions with visible illumination variance, color differently, a convolution layer is always followed by a
combination, camera angles, therefore, an image ReLU rectification layer. To prevent cell overlap, the
normalization technique was implemented before feature pooling procedure uses cells with a size of 2X2 pixels and a
extraction. step of 2 pixels.

Table 2: Dataset Divison for training and Testing of the

Hybrid Model:

Class Labels Training Testing Dataset

Dataset
Healthy(NO DR) 1998 536
Unhealthy (DR) 2307 899
Total 4305 1435
 IV. RESULTS AND DISCUSSIONS
Graphical representation of figure 5 & 6 for 5 epochs:
With a high accuracy of 99.41%, the model was able to
classify the great majority of training data samples correctly.
The model's forecast very slightly differed from the real
labels, as seen by the loss value of 0.0133, indicating a high
degree of prediction precision.

Fig 3. VGG Net Architecture.

The Microsoft-developed ResNet50, often known as the

Residual Network, was victorious in the yearly ILSVRC
(ImageNet Large Scale Visual Recognition Competition) in
2015. Every year, a number of teams compete in

figure 5

Fig 3. Res Net Architecture.

the ILSVRC competition to develop the best algorithm for

computer vision problems. ResNet50 has more than 25.6
million parameters and 50 layers deep. Convolution, an
identity block (input = output), and a completely connected
layer make up the set. The input value of the original block figure 6
or signal is matched by identity x. Consequently, the total of
the block's input value and the output values of its internal Graphical representation of figure 7 & 8 for 10 epochs:
layers is the residual block's output value. The model achieved a high accuracy of 99.60%, indicating
its ability to accurately classify the majority of the training
Hybrid Model: dataset. The loss value at this epoch is recorded as 0.0053,
Using Residual connections for hybridisation: [13] Within implying that the model's predictions diverged slightly from
the hybrid VGGNet-ResNet model for the diagnosis of the true labels during training.
diabetic retinopathy, "residual connections" denote a
particular method for combining features that are taken from
these two architectures.
Merging with Residual Connections in your Hybrid Model:
Find the point of convergence: To extract features, select a
particular block in VGGNet. This will be known as block V.
Retrieve features at the same depth as V from the matching
block in ResNet. These attributes will be referred to as J. R.
Instead of directly concatenating V and R features, add them
element-wise. This injects low-level details from V directly
into the higher-level feature representation of R. Put the
combined features of V and R into the deeper ResNet layers
or extra classification head that make up the remaining
layers of the hybrid model. figure 7
 figure 10

figure 8 Graphical representation of figure 11 & 12 for 20 epochs

The model achieved a high accuracy of 99.9%, indicating its
Graphical representation of figure 9 & 10 for 15 epochs ability to accurately classify the majority of the training
The model achieved a high accuracy of 99.89%, indicating dataset. The loss value at this epoch is recorded as 0.0011,
its ability to accurately classify the majority of the training implying that the model's predictions diverged slightly from
dataset. The loss value at this epoch is recorded as 0.0013, the true labels during training.
implying that the model's predictions diverged slightly from
the true labels during training.

figure 11

figure 9

figure 12

Epochs Accuracy Sensitivity Precision F1 score

Table 3. Results:
5 99.41% 99.60% 98.91% 99.25%

The table 3 represents that the model demonstrated great

accuracy, ranging from 99.41% to 100%, across all epochs.
10 99.60% 99.20% 98.90% 99.05% Sensitivity, which measures how well the model can detect
positive cases, continuously stayed over 99% and peaked at
15 99.89% 99.90% 98.90% 98.95%

20 99.90% 99.00% 98.90% 98.95%

99.90% after 15 epochs. Throughout the training epochs, of 74.5%. Harshith Kaushik achieved 97.9% accuracy using
precision, which measures the model's accuracy in a stacked CNN architecture; however, recall and precision
identifying positive predictions, was constant at 98.90%. data were not disclosed. Proposed hybrid model combining
With very minor variations, the F1 score—which is a features of VGGNet and ResNet, achieving exceptional
composite indicator of sensitivity and precision—continued performance with an accuracy of 99.9%, precision of 98.9%,
to be high, ranging from 98.95% to 99.25%. recall of 99.0%, and an F1-score of 98.9%.

Table 4 depicts the use of deep visual features; Qaisar

Abbas was able to achieve 94.3% accuracy and 95.0% recall.
Using the ResNet50 architecture, Muhammad Samer Sallam
achieved 69.4% accuracy, 50.7% precision, 49.2% recall, and
49.5% F1 score. Abhishek Deshpande achieved a recall of
96.0% and an F1-score of 97.9% employing the VGG-16
architecture with perfect precision (100%) and an accuracy

Table 4. Comparative analysis with different architectures

Author Architecture Accuracy Precision Recall F1 -score

Qaisar Abbas Deep visual features 94.3% NA 95.0% NA
Muhammad ResNet50 69.4% 50.7% 49.2% 49.5%
Samer Sallam
Abhishek VGG-16 74.5% 100 96.0% 97.9%
Deshpande
Harshith Kaushik Stacked CNN 97.9% NA NA NA

Proposed Solution Hybrid model 99.9% 98.9% 99.0% 98.9%

(VGGNet + ResNet)

V. CONCLUSION AND FUTURE SCOPE local private data is used to train the models, and the learnt
weights are subsequently imported into a global model.
Conclusion : Retinal diseases are highly intricate, so it is Lightweight Network Architecture: The majority of deep
vital to analyze them in detail to make prompt and definitive learning models created for the diagnosis of retinal diseases
diagnosis. This work was the first to apply a hybrid deep operate effectively, albeit at a significant computational cost.
learning model which combines VGG (which specializes in This is a significant barrier to using these models on mobile
the global pattern structure) and ResNet (which specializes in edge devices. Creating innovative lightweight models to
the structure of local lesions) to enhance our network’s lower the computational parameters while keeping
ability to diagnose retinal diseases. In contrast to isolated performance is another important research avenue in this
VGG or isolated ResNet models which only specialize in area.
global patterns or local lesions, our hybrid model
incorporates the overall disease status. Performance is
considerably improved, and we found that our hybrid model VI. REFERNCES
outperformed both average experts and other conventional
diagnostic methods in detecting retinal disease. The [1] S. Sengupta, A. Singh, H. A. Leopold, T. Gulati,
improved efficiency of pathology diagnosis assists medical and V. Lakshminarayanan, ‘‘Ophthalmic diagnosis
experts to take early intervention, thereby potentially using deep learning with fundus images—A critical
reducing vision damage and save lives. review,’’ Artif. Intell. Med., vol. 102, Jan. 2020,
Art. no. 101758.
[2] Qaisar Abbas, “Automatic recognition of severity
Future Scopes: level for diagnosis of diabetic retinopathy using
deep visual features” International Federation for
Smartphone-based diagnosis of retinal diseases: Most of Medical and Biological Engineering 2017.
the research being done in this area uses high-resolution
[3] M. Badar, M. Haris, and A. Fatima, ‘‘Application
fundoscopy images of the fundus. Researchers have plenty of
of deep learning for retinal image analysis: A
room to create algorithms that can identify patterns in fundus
photos taken using mobile phones. This will support the review,’’ Comput. Sci. Rev., vol. 35, Feb. 2020,
creation of a facility for remote eye screening. Art. no. 100203.
[4] W. L. Alyoubi, M. F. Abulkhair, and W. M.
Federated learning implementation: Most hospitals and Shalash, ‘‘Diabetic retinopathy fundus image
other research institutions are reluctant to share fundus classification and lesions localization system using
photos with third parties because of various data privacy deep learning,’’ Sensors, vol. 21, no. 11, p. 3704,
laws. This exacerbates the issue of data scarcity by limiting May 2021.
the model training to publicly accessible datasets, preventing [5] Abhishek Deshpande , “Automated detection of
the models from receiving training on rich and varied private Diabetic Retinopathy using VGG-16 architecture”
fundus data that are accessible at the hospitals. One can
investigate schemes such as federated learning, in which
 International Research Journal of Engineering and ‘‘Deep neural networks to predict diabetic
Technology (IRJET) 2021. retinopathy,’’ J. Ambient Intell. Humanized
[6] Muhammad Samer Sallam “Diabetic Retinopathy Comput., pp. 1–14, Apr. 2020, doi:
Grading Using ResNet Convolutional Neural 10.1007/s12652-020-01963-7.
Network” 2020 IEEE Conference on Big Data and [12] BALLA GOUTAM , “A Comprehensive Review
Analytics (ICBDA). of Deep Learning Strategies in Retinal Disease
[7] Diabetic Retinopathy Detection | Kaggle. Diagnosis Using Fundus Images” Digital Object
Accessed: Apr. 22, 2022. [Online]. Available: Identifier 10.1109/ACCESS.2022.3178372,
https://www.kaggle.com/c/diabetic- vol.10 , 2022.
retinopathydetection [13] A. Rakhlin, ‘‘Diabetic retinopathy detection
[8] D. H. Wolpert, ‘‘Stacked generalization,’’ Neural through integration of deep learning classification
Netw., vol. 5, no. 2, pp. 241–259, 1992. framework,’’ BioRxiv, Jan. 2018, Art. no. 225508,
[9] J. C. Javitt, J. K. Canner, R. G. Frank, D. M. doi: 10.1101/225508.
Steinwachs, and A. Sommer, ‘‘Detecting and [14] C. Lam, D. Yi, M. Guo, and T. Lindsey,
treating retinopathy in patients with type i diabetes ‘‘Automated detection of diabetic retinopathy using
mellitus: A health policy model,’’ Ophthalmology, deep learning,’’ AMIA Summits Transl. Sci., vol.
vol. 97, no. 4, pp. 483–495, 1990. 2018, no. 1, p. 147, 2018.
[10] D. J. Hemanth, O. Deperlioglu, and U. Kose, ‘‘An [15] HARSHIT KAUSHIK , “Diabetic Retinopathy
enhanced diabetic retinopathy detection and Diagnosis From Fundus Images Using Stacked
classification approach using deep convolutional Generalization of Deep Models” Digital Object
neural network,’’ Neural Comput. Appl., vol. 32, Identifier 10.1109/ACCESS.2021.3101142.
no. 3, pp. 707–721, Feb. 2020.
[11] T. R. Gadekallu, N. Khare, S. Bhattacharya, S.
Singh, P. K. R. Maddikunta, and G. Srivastava,