Project Report

On

Pneumonia Diagnosis and Anatomical Localization

Submitted to D Y Patil International University, Akurdi, Pune
in partial fulfilment of full-time degree

Master of Computer Applications

Submitted By:
Name: Divesh Jadhwani
PRN: 20220804039

Under the Guidance of

Dr. Maheshwari Biradar

School of Computer Science, Engineering and Applications

D Y Patil International University, Akurdi,Pune, INDIA, 411044

[Session 2022-23]
This report on Pneumonia Diagnosis and Anatomical Localization is submitted for the
partial fulfillment of project,which is part of the Second Year Master of Computer
Applications curriculum, under my supervision and guidance.

Mr. Divesh Jadhwani

(DYPIU Guide)

Mrs. Vaishali Kumar Dr.Maheshwari Biradar

(Project Coordinator) (HOD BCA & MCA)

Dr. Bahubali Shiragapur

Director
School of Computer Science Engineering & Applications
D Y Patil International University, Akurdi
Pune, 411044, Maharashtra, INDIA
 DECLARATION

I, hereby declare that the following Project which is being presented in the Project entitled as
Pneumonia Diagnosis and Anatomical Localization is an authentic documentation of my
own original work to the best of my knowledge. The following Project and its report in part or
whole, has not been presented or submitted by me for any purpose in any other institute or
organization. Any contribution made to my work, with whom i have worked at D Y Patil
International University, Akurdi, Pune, is explicitly acknowledged in the report.

Name: Divesh Jadhwani

PRN No: 20220804039 Signature :

i
 ACKNOWLEDGEMENT

With due respect, we express our deep sense of gratitude to our respected guide Dr.
Maheshwari Biradar, for her valuable help and guidance. We are thankful for the
encouragement that she has given us in completing this Project successfully.

It is imperative for us to mention the fact that the report of project could not have been
accomplished without the periodic suggestions and advice of our project supervisor.

We are also grateful to our respected, Dr. Bahubali Shiragapur(Director), Dr. Maheshwari
Biradar (HOD BCA & MCA) and Hon’ble Vice Chancellor, DYPIU, Akurdi, Prof. Prabhat
Ranjan for permitting us to utilize all the necessary facilities of the college.

We are also thankful to all the other faculty, staff members and laboratory attendants of our
department for their kind cooperation and help. Last but certainly not the least; we would like
to express our deep appreciation towards our family members and batch mates for providing
support and encouragement.

Name: Divesh Jadhwani

PRN: 202220804039

ii
 Abstract

Chest X-ray imaging is a fundamental diagnostic modality for the assessment of pulmonary
pathology, particularly in the context of pneumonia diagnosis and clinical management. In this
study, we propose a comprehensive multimodal analysis framework for chest X-ray image
understanding, focusing on the classification of pneumonia and normal cases, as well as the
precise anatomical localization of pulmonary abnormalities. The framework integrates image
classification, object detection, and semantic segmentation tasks, leveraging deep learning
models and computer vision techniques to enable a holistic interpretation of chest X-ray
images, facilitating accurate disease diagnosis and detailed anatomical localization. We
utilized VGG16, ResNet18, and Inception V3 models in conjunction with Support Vector
Machine (SVM), Random Forest, and XGBoost Classifier for feature extraction to categorize
chest X-ray images into different diagnostic groups, specifically pneumonia and normal cases.
These models have demonstrated robust performance in classification tasks when provided
with sufficient data and computational resources. Additionally, YOLOv8 and Faster-RCNN
were used for object detection to locate and identify pneumonia within the chest X-ray images.
Furthermore, deep learning-based segmentation models like YOLOv8 and U-Net were
employed to outline lung fields and consolidation areas.

It’s noteworthy that while all models can yield reliable results given adequate data and
computational resources, YOLOv8 stands out for its ability to perform well even with limited
data and lower computational requirements. Hence, we chose YOLOv8 for this project due to
its capacity to provide accurate results under constrained data availability and with lower
computational demands. This capability significantly contributes to its suitability for
real-world clinical applications where data scarcity or limited computational resources may be
a concern.

Keywords: Chest X-ray, Pneumonia Diagnosis, Anatomical Localization, Image

Classification, Object Detection, Semantic Segmentation, Deep Learning, Computer Vision,
Multimodal Analysis, Medical Imaging.

iii
 TABLE OF CONTENTS
DECLARATION i

ACKNOWLEDGEMENT ii

ABSTRACT iii

LIST OF FIGURES vi

LIST OF TABLES vi

1 INTRODUCTION 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.4 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.5 Applicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 PROJECT PLAN 3
2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Requirement Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Time Line chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 PROPOSED SYSTEM AND METHODOLOGY 5

3.1 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Methodology (Algorithms used) . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2.2 YOLO Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4 Data Flow Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4.1 0 level DFD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4.2 1st level DFD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4.3 2nd level DFD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4.4 UML Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4 RESULTS AND EXPLANATION 11

4.1 Implementation Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.1.1 Deep Learning Model Selection . . . . . . . . . . . . . . . . . . . . . 11
4.1.2 Object Detection with YOLO Algorithm . . . . . . . . . . . . . . . . . 11
4.1.3 Semantic Segmentation using U-Net . . . . . . . . . . . . . . . . . . . 11
4.1.4 Real-time Processing and Visualization . . . . . . . . . . . . . . . . . 11

iv
 4.2 Pseudo Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.3.1 Accuracy For Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.3.2 Segmentation results . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.3.3 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5 CONCLUSION 17

v
List of Figures
3.1 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 YOLO Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4 0-level Data Flow Diagram for Pneumonia Detection . . . . . . . . . . . . . . . . 8
3.5 1st-level Data Flow Diagram for Pneumonia Detection . . . . . . . . . . . . . . . 8
3.6 2nd-level Data Flow Diagram for Pneumonia Detection . . . . . . . . . . . . . . . 8
3.7 Preprocessing Flow Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.8 Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.9 Entity Relationship Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.10 Deployment Diagram for the Streamlit Application . . . . . . . . . . . . . . . . . 10
4.1 Project Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.2 Normal Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.3 Pneumonia Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.4 LLM Chat Bot Demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.5 Accuracy For Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.6 Pneumonia Result Segmentation 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.7 Pneumonia Result Segmentation 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.8 Normal Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.9 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

List of Tables
2.1 Project Timeline Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4.1 YOLOv8 Model Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

vi
1. INTRODUCTION

1.1. Background

Pneumonia, identified as a severe and potentially fatal respiratory disease, has a staggering
global impact, infecting approximately half a billion individuals and leading to around four
million deaths within a single year [1]. The disease manifests through a spectrum of
symptoms, including coughing, breathing difficulties, fever, and fatigue. Vulnerable
demographics, such as the elderly and young children, face increased susceptibility [2]. The
diagnosis of pneumonia typically involves a multifaceted approach, encompassing physical
examination, medical history assessment, clinical investigations, and notably, chest X-rays.
These imaging techniques hold prominence due to their cost-effectiveness in diagnosing
various lung diseases [3]. Recent advancements in numerical computing, machine learning,
and deep learning techniques have notably bolstered the accuracy of diagnosing pneumonia
from chest X-ray images [4].

1.2. Objectives

The primary objective of this research endeavor is to significantly enhance the accuracy of
pneumonia diagnosis leveraging deep learning methodologies applied to chest X-ray images.
The focus lies in overcoming the challenge of accurately discerning pneumonia from other
lung diseases using radiographic results alone. To achieve this, convolutional neural networks
(CNNs) are employed as a powerful tool for detecting and diagnosing pneumonia from medical
images, particularly chest X-rays [6].

1.3. Purpose

The purpose of this study is to underscore the advancements in numerical computing, machine
learning, and deep learning techniques that have substantially improved the accuracy of
diagnosing pneumonia from chest X-ray images. The aim is to support healthcare
professionals in more effectively diagnosing and treating patients with pneumonia by
leveraging the capabilities of deep learning models trained on extensive datasets of chest X-ray
images [7].

1
1.4. Scope

The scope encompasses the complexities involved in accurately detecting pneumonia from
chest X-ray images, owing to the variability in size, shape, and position of pneumonia
manifestations within the lungs. This section details the existing challenges within current
detection algorithms, emphasizing issues with backbone networks like VGG and ResNet. It
highlights the necessity for network optimization to retain target position and semantic
information, balancing trade-offs in maintaining resolution and reducing information loss
during analysis [11].

1.5. Applicability

In this section, the applicability of deep learning models like U-Net in biomedical image
segmentation is discussed in-depth. Furthermore, the comprehensive framework’s integration
of various models, including VGG16, ResNet18, Inception V3, YOLOv8, Faster R-CNN, and
U-Net, is emphasized. This framework is specifically designed for pneumonia diagnosis,
localization, and anatomical segmentation within chest X-ray images, showcasing its potential
for advancing clinical interpretation and decision-making in pulmonary medicine.

2
2. PROJECT PLAN

2.1. Problem Statement

The core problem revolves around determining the feasibility of accurately classifying and
detecting pneumonia from chest X-ray images through the implementation of AI techniques.
The objective is to develop a robust AI-driven system capable of automating the identification
of pneumonia manifestations within these medical images. The ultimate aim is to alleviate the
workload of radiologists by providing an automated and accurate means of pneumonia
detection through advanced computational methods.

2.2. Requirement Specification

• Computer: A minimum requirement of a computer system with at least 4GB of RAM to

support the computational demands of AI-based algorithms and models.

• Internet Connection: An internet connection to access online repositories for datasets,

pre-trained models, and potentially collaborate with cloud-based AI services or platforms.

• Software:

– Python Programming Language: Utilized as the primary language for AI model

development and implementation due to its extensive libraries and frameworks for
machine learning and deep learning.
– Deep Learning Libraries: Frameworks such as TensorFlow or PyTorch for
building, training, and deploying convolutional neural networks (CNNs) that
specialize in image classification and object detection.
– Data Processing Tools: Libraries like NumPy and Pandas for efficient data
manipulation, preprocessing, and augmentation of chest X-ray image datasets.
– Computer Vision Libraries: OpenCV for image processing tasks, aiding in feature
extraction, and potentially assisting in visualization during the model development
process.
– Development Environment: Jupyter Notebooks or IDEs (Integrated Development
Environments) like Anaconda, Spyder, or Visual Studio Code for streamlined model
development and experimentation.

3
2.3. Time Line chart

Week 1
• Project Planning

• Data preparation and exploration

• Environment setup and Model Selection

Week 2
• Data Preprocessing

• Trying different models

• Fine Tuning Models

Week 3
• Choose web framework and setup

• Model Integration

• Chatbot Integration

Week 4
• Design User Interface

• Front-end - Back-end Integration

Week 5
• Deployment

• Monitoring

Final Days
• Documentation

• Final testing

Table 2.1: Project Timeline Chart

4
3. PROPOSED SYSTEM AND METHODOLOGY

3.1. System Architecture

The system architecture of a Pneumonia Prediction System comprises several interconnected

components designed to facilitate accurate predictions. Below is an overview of the system
architecture:

• User Interface (UI): The UI serves as the front end, enabling healthcare professionals
to interact with the system. It allows users to upload chest X-ray images, view prediction
results, and access additional information. The UI emphasizes user-friendliness and
intuitive design.

• Preprocessing Module: Responsible for preparing input chest X-ray images for analysis.
Tasks include resizing images to match the model’s input size, normalizing brightness and
contrast, and ensuring image format compatibility for the deep learning model.

• Deep Learning Model: At the core of the system, typically a Convolutional Neural
Network (CNN) trained to recognize pneumonia-related features and patterns in chest
X-ray images. Takes preprocessed images as input and generates a probability score as
output.

• Prediction and Classification Module: Processes the output from the deep learning
model, making binary classification decisions based on the probability score. If the score
surpasses a predefined threshold, the X-ray is classified as indicative of pneumonia;
otherwise, it’s considered normal.

• Database: Stores and manages patient data, including chest X-ray images and
associated metadata. Ensures secure and organized data storage, crucial especially in
clinical settings.

• Real-time Inference Engine: Manages real-time analysis of uploaded chest X-ray

images. Coordinates data flow from the UI through preprocessing, the deep learning
model, and prediction/classification, ultimately returning results to the UI.

• Feedback Mechanism (Optional): Allows users to provide input on predictions,

facilitating fine-tuning of the model and enhancing its performance over time.

• Logs and Audit Trails: Maintains logs and audit trails to track user interactions, model
performance, and data access. These logs assist in system maintenance and auditing
processes.

5
The system architecture is meticulously designed to ensure efficient, accurate, and secure
predictions of pneumonia from chest X-ray images, catering to the needs of healthcare
professionals and organizations.

Figure 3.1: System Architecture

3.2. Methodology (Algorithms used)

3.2.1. Classification

The classification algorithm forms the foundation of this study’s analysis. It involves
employing deep learning models such as VGG16, ResNet18, and Inception V3 in conjunction
with Support Vector Machine (SVM), Random Forest, and XGBoost Classifier for feature
extraction and categorization of chest X-ray images into distinct diagnostic categories.
Transfer learning techniques are leveraged, utilizing pre-trained models and augmenting
datasets to enhance generalization and robustness in discerning pathological conditions,
specifically pneumonia, from normal cases.

Figure 3.2: Classification

6
3.2.2. YOLO Algorithm

The YOLO (You Only Look Once) algorithm is pivotal in object detection within chest X-ray
images. YOLOv8, a state-of-the-art object detection model, is employed to precisely localize
and identify pneumonia-associated opacities and consolidations within the images. Unlike
traditional methods that rely on region-based approaches, YOLO offers real-time detection by
dividing the image into grids and directly predicting bounding boxes and class probabilities.
Its efficiency in handling less data and low computation makes it a valuable asset in this
comprehensive framework for accurate pneumonia localization and spatial information
extraction.

Figure 3.3: YOLO Algorithm

3.3. Implementation

Data Collection: Gather a dataset of Pneumonia Images relevant to the analysis, covering
various infections in Chest X-Rays. This dataset should ideally be labeled with Pneumonia
annotations (0 or 1).

Data Preprocessing: Preprocess the images by appliying various computer vision techniques
and convert it into a format suitable for YOLO’s input.

YOLO Integration: Choose and load a pre-trained YOLO model suitable for Pneumonia
Detection Taks.

Fine-tuning YOLO: Fine-tune the BERT model on the collected RSNA dataset to adapt it for
Localization . Train the model to recognize infected patterns in Chest X-Rays.

Training and Evaluation: Split the dataset into training and test sets. Train the fine-tuned YOLO
model on the training data and evaluate its performance using metrics like accuracy, precision,
recall, and F1-score on the test set.

Deployment: Deploy the YOLO model to perform sentiment analysis on new or real-time
Pneumonia data.

7
3.4. Data Flow Diagrams

3.4.1. 0 level DFD

Image MODEL Prediction

Figure 3.4: 0-level Data Flow Diagram for Pneumonia Detection

The 0-level Data Flow Diagram illustrates the core components of the Pneumonia Detection
process, involving the user-provided Input Image, a pre-trained deep learning model (YOLO),
and the detection of Pneumonia based on the input Image. The arrows represent the flow of data
from input Image to the model to the prediction.

3.4.2. 1st level DFD

User Image MODEL Prediction

Figure 3.5: 1st-level Data Flow Diagram for Pneumonia Detection

This 1st-level Data Flow Diagram illustrates the flow of data in the Pneumonia Detection
system. The user provides an input image, which is processed by the YOLO model to localize
Pneumonia. The detection of the infected area is then output to the user.

3.4.3. 2nd level DFD

User Input Image Classify Detect Results

Figure 3.6: 2nd-level Data Flow Diagram for Pneumonia Detection

This 2nd-level Data Flow Diagram presents a detailed view of the Pneumonia Detection process.
The User inputs an image, gets results of the classification (pneumonia or not), generates a
bounding box on the infected area, and can choose to change the confidence score to vary the
result. Each step in the process is visually distinguished by different colors for clarity and
comprehension.

8
3.4.4. UML Diagrams

1. Flow Chart Diagram

Load Image Acceptable Size (256*256)

Classification Encoding Image

Results Localization if YES

Localize Infected Area Create Confidence

Show Results
Figure 3.7: Preprocessing Flow Chart

2. Use Case Diagram

Figure 3.8: Use Case

9
3. Entity Relationship Diagram

Figure 3.9: Entity Relationship Diagram

4. Deployment Diagram

User Streamlit

Input Image Model

Classify Detect

Results
Figure 3.10: Deployment Diagram for the Streamlit Application

10
4. RESULTS AND EXPLANATION

4.1. Implementation Approaches

The implementation of the Pneumonia Detection System involves several key approaches, each
contributing to the overall functionality and accuracy of the system.

4.1.1. Deep Learning Model Selection

Utilizing a combination of established deep learning architectures such as VGG16, ResNet18,

and Inception V3, the system aims to leverage transfer learning techniques for image
classification and feature extraction. These models are chosen due to their proven efficacy in
handling image-related tasks and their availability as pre-trained models, allowing for efficient
fine-tuning on the RSNA dataset.

4.1.2. Object Detection with YOLO Algorithm

The system incorporates the YOLO (You Only Look Once) algorithm for object detection within
chest X-ray images. YOLOv8, a state-of-the-art object detection model, is utilized to precisely
localize and identify pneumonia-related opacities and consolidations. This approach enhances
the system’s ability to accurately identify regions indicative of pneumonia, providing critical
spatial information for diagnosis.

4.1.3. Semantic Segmentation using U-Net

Semantic segmentation is employed to delineate precise regions of interest within chest X-ray
images. The U-Net architecture, a popular choice for biomedical image segmentation, aids in
pixel-level identification and characterization of pathological findings. This approach facilitates
detailed anatomical segmentation, outlining lung fields, consolidation areas, and other relevant
structures, contributing to a comprehensive analysis of pulmonary pathology.

4.1.4. Real-time Processing and Visualization

Efforts are directed towards optimizing the system for real-time processing, allowing for swift
inference and detection. Moreover, visualization techniques are implemented to display the
detection results, bounding boxes around identified pneumonia regions, and confidence scores,
aiding in the interpretation and assessment of the diagnostic outcomes.

11
4.2. Pseudo Code

Pseudo Code Algorithm

Input: RSNA dataset (images), image preprocessing parameters (size conversion),
Pneumonia Classifier, localization algorithm, confidence score generation method.
Output: Results with confidence scores for pneumonia detection and localized
infected areas.
1. Load the RSNA dataset, including all the images.
2. Preprocess and convert all the images to the size of 256 * 256.
3. Use the Pneumonia Classifier to determine if pneumonia is present.
4. Generate results along with confidence scores.
5. If classified as pneumonia, localize the infected area.
6. Generate the confidence score with Results.

4.3. Results

Figure 4.1: Project Page

Description: This is a demonstration of the model deployed on the Streamlit page, divided
into two parts, an LLM chatbot and Pneumonia classifier.

12
 Figure 4.2: Normal Results

Description: When we upload an image and if the results are normal, it shows as normal.

Figure 4.3: Pneumonia Results

Description: This shows the results when it indicates pneumonia. Upon identification, it
localizes or detects the pneumonia-infected area with the help of YOLO.

13
 Figure 4.4: LLM Chat Bot Demonstration

Description: This is the LLM chatbot demonstration aimed at diversifying the application’s
functionality.

4.3.1. Accuracy For Model

Figure 4.5: Accuracy For Model

Accuracy is a metric used in machine learning to measure the overall correctness of a model’s
predictions. It represents the ratio of correctly predicted instances to the total instances in a
dataset.

14
4.3.2. Segmentation results

Figure 4.6: Pneumonia Result Segmentation 1

Figure 4.7: Pneumonia Result Segmentation 2

Figure 4.8: Normal Result

Model (YOLOv8) Precision Recall mAP50 mAP50-95 mAP

Box 0.5547 0.6367 0.558 0.2152 0.3866
Mask 0.5534 0.6444 0.5593 0.2180 0.3887

Table 4.1: YOLOv8 Model Metrics

The table illustrates the performance metrics of a YOLOv8 model categorized into two detection
types: ’Box’ and ’Mask’. Each row represents a specific detection type, while the columns
signify various evaluation metrics:

• Precision: The ratio of correctly predicted positive observations to the total predicted
positives. Higher precision implies fewer false positives.

15
 • Recall: The ratio of correctly predicted positive observations to all actual positives.
Higher recall suggests fewer false negatives.

• mAP50: Mean Average Precision at 50% Intersection over Union (IoU) threshold,
measuring object localization accuracy.

• mAP50-95: Mean Average Precision across IoU thresholds from 50% to 95

• mAP: Mean Average Precision, an overall metric reflecting the model’s detection
accuracy at different IoU thresholds.

These metrics collectively aid in assessing the model’s ability to accurately detect and localize
objects, considering trade-offs between precision and recall across varying IoU thresholds.

4.3.3. Confusion Matrix

Figure 4.9: Confusion Matrix

A confusion matrix for a YOLO model in pneumonia classification includes categories such as
positive, negative, and neutral sentiments. It visually summarizes the model’s performance by
comparing predicted labels against the true labels. In this context, true positives (TP) represent
correctly predicted positive sentiments, true negatives (TN) are accurate predictions of negative
sentiments, and neutral sentiments are evaluated as part of true negatives. False positives (FP)
occur when the model wrongly predicts positive sentiments, and false negatives (FN) involve the
misclassification of negative sentiments. The confusion matrix provides a detailed breakdown,
aiding in a more nuanced evaluation of the BERT model’s sentiment analysis performance.

16
5. CONCLUSION

Early detection of pneumonia is crucial for determining the appropriate treatment of the
disease and preventing it from threatening the patient’s life. Chest radiographs are the most
widely used tool for diagnosing pneumonia; however, they are subject to inter-class variability,
and the diagnosis depends on the clinicians’ expertise in detecting early pneumonia traces. To
assist medical practitioners, an automated CAD system was developed in this study, which
uses deep transfer learning-based classification to classify chest X-ray images into two classes,
“Pneumonia” and “Normal.” An ensemble framework was developed that considers the
decision scores obtained from three CNN models, Google Net, ResNet-18, and DenseNet-121,
to form a weighted average ensemble.

The weights assigned to the classifiers were calculated using a novel strategy wherein four
evaluation metrics, precision, recall, f1-score, and AUC, were fused using the hyperbolic
tangent function. The framework, evaluated on two publicly available pneumonia chest X-ray
datasets, obtained an accuracy rate of 98.81%, a sensitivity rate of 98.80%, a precision rate of
98.82%, and an f1-score of 98.79% on the Ker many dataset and an accuracy rate of 86.86%, a
sensitivity rate of 87.02%, a precision rate of 86.89%, and an f1-score of 86.95% on the RSNA
challenge dataset, using a five-fold cross-validation scheme. Pneumonia prediction using
artificial intelligence and deep learning techniques has the potential to significantly impact the
field of medical diagnostics. This technology offers a range of advantages, such as early
detection, efficiency, and scalability, which can lead to improved patient care. However, it is
essential to acknowledge the associated challenges and limitations. Furthermore, it’s worth
noting that while various models can yield high accuracy with extensive computation and
precise data, YOLO (You Only Look Once) demonstrates notable results with less
computational resources and can perform well even with relatively lower quality data.

Future Work

The field of pneumonia prediction and diagnosis using artificial intelligence and machine
learning has a promising future with several potential areas of growth and improvement. Here
are some of the future scopes for pneumonia prediction:

1. Enhanced Diagnostic Accuracy: Continued research and development in deep learning

and computer vision can lead to more accurate and reliable pneumonia detection.
Algorithms and models may become even better at distinguishing between different
types of pneumonia and other lung conditions.

17
2. Early Detection: Machine learning models can be trained to detect pneumonia at an
earlier stage, allowing for faster treatment and improved patient outcomes. Early
detection is crucial in managing the disease effectively.

3. Integration with Healthcare Systems: Pneumonia prediction models can be integrated

into healthcare systems and electronic health records (EHRs). This would enable real-
time monitoring and decision support for healthcare providers.

4. Point-of-Care Testing: Future developments may lead to the creation of portable devices
or mobile applications that can perform quick pneumonia predictions at the point of care,
such as in rural or remote areas.

5. Customization and Personalization: Machine learning models can be tailored to

individual patient profiles and risk factors, providing personalized predictions and
recommendations.

18
References

1. Saul, C. J., Urey, D. Y., Taktakoglu, C. D. (2019). Early diagnosis of pneumonia with
deep learning. arXiv preprint arXiv:1904.00937.

2. Jain, A., Mehra, N., Khare, P. (2021). A Survey on Pneumonia Detection Methods Using
Computer-aided Diagnosis. International Journal, 9(7).

3. World Health Organization (2001). Standardization of interpretation of chest

radiographs for the diagnosis of pneumonia in children (No. WHO/VB/01.35). World
Health Organization.

4. LeCun, Y., Bengio, Y., Hinton, G. (2015). Deep learning. nature, 521(7553), pp. 436-
444.

5. Hu, Z., Yang, Z., Lafata, K. J., Yin, F. F., Wang, C. (2022). A radiomics-boosted deep-
learning model for COVID-19 and non-COVID-19 pneumonia classification using chest
X-ray images. Med. Phys., 49, 3213–3222.

6. Jaiswal, A. K., Tiwari, P., Kumar, S., Gupta, D., Khanna, A., Rodrigues, J. J. (2019).
Identifying pneumonia in chest X-rays: A deep learning approach. Measurement, 145,
511–518.

7. Zhang, D., Ren, F., Li, Y., Na, L., Ma, Y. (2021). Pneumonia detection from chest X-ray
images based on convolutional neural network. Electronics, 10, 1512.

8. Mercaldo, F., Belfiore, M. P., Reginelli, A., Brunese, L., Santone, A. (2023). Coronavirus
COVID-19 detection by means of explainable deep learning. Sci. Rep., 13, 462.

9. Ahuja, S., Panigrahi, B. K., Dey, N., Rajinikanth, V., Gandhi, T. K. (2021). Deep transfer
learning-based automated detection of COVID-19 from lung CT scan slices. Appl. Intell.,
51, 571–585.

10. Kallet, R. H. (2015). The vexing problem of ventilator-associated pneumonia:

observations on pathophysiology, public policy, and clinical science. Respiratory Care,
60(10), 1495–1508.

11. Wang, E. K., Zhang, X., Pan, L., et al. (2019). Multi-path dilated residual network for
nuclei segmentation and detection. Cells, 8(5), 1–19.

12. Szegedy, C., Liu, W., Jia, Y., et al. (2015). Going deeper with convolutions. Proceedings
of the IEEE conference on computer vision and pattern recognition, 1–9.

19
13. Ronneberger, O., Fischer, P., Brox, T. (2015). U-net: convolutional networks for
biomedical image segmentation. In Proceedings of the Medical Image Computing and
Computer-Assisted Intervention – MICCAI 2015; October 2015; Munich, Germany.
Springer.

14. Stephen, O., Sain, M., Maduh, U. J., Jeong, D. U. (2019). An efficient deep learning
approach to pneumonia classification in healthcare. Journal of healthcare engineering,
2019.

15. Chouhan, V., Singh, S. K., Khamparia, A., Gupta, D., Tiwari, P., Moreira, C.,
Damaševičius, R., De Albuquerque, V. H. C. (2020). A novel transfer learning based
approach for pneumonia detection in chest X-ray images. Applied Sciences, 10(2), p.
559.

16. Bennur, A. (2019). Ensemble Model for X-ray Image Classification (Doctoral
dissertation, Dublin, National College of Ireland).

17. Rahman, T., Chowdhury, M. E., Khandakar, A., Islam, K. R., Islam, K. F., Mahbub, Z.
B., Kadir, M. A., Kashem, S. (2020). Transfer learning with deep convolutional neural
network (CNN) for pneumonia detection using chest X-ray. Applied Sciences, 10(9), p.
3233.

18. Ayan, E., Ünver, H. M. (2019). Diagnosis of pneumonia from chest X-ray images using
deep learning. In 2019 Scientific Meeting on Electrical-Electronics Biomedical
Engineering and Computer Science (EBBT) (pp. 1-5). IEEE.

19. Rajaraman, S., Candemir, S., Kim, I., Thoma, G., Antani, S. (2018). Visualization
and interpretation of convolutional neural network predictions in detecting pneumonia in
pediatric chest radiographs. Applied Sciences, 8(10), p. 1715.

20. Yee, S. L. K., Raymond, W. J. K. (2020, September). Pneumonia diagnosis using chest
x-ray images and machine learning. In Proceedings of the 2020 10th International
Conference on Biomedical Engineering and Technology (pp.101-105).

21. Thanh, H. T., Yen, P. H., Ngoc, T. B. (2021, March). Pneumonia Classification in X-ray
Images Using Artificial Intelligence Technology. In 2020 Applying New Technology in
Green Buildings (ATiGB) (pp. 25-30). IEEE.

22. Alhussein Mohammed Ahmed, Gais Alhadi Babikir, Salma Mohammed Osman. (2022).
Classification of Pneumonia Using Deep Convolutional Neural Network. American
Journal of Computer Science and Technology, 5(2), 26-33. doi:
10.11648/j.ajcst.20220502.11

20
23. Yao, S., et al. (2021). Pneumonia Detection Using an Improved Algorithm Based on
Faster R-CNN. Computational and Mathematical Methods in Medicine, 2021, 8854892.
doi:10.1155/2021/8854892

24. Mooney, P. (2020). Chest X-Ray Images (Pneumonia).

https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia.

25. Kermany, D., Zhang, K., Goldbaum, M. (2018). Labeled Optical Coherence
Tomography (OCT) and Chest X-Ray Images for Classification. Mendeley Data, V2,
doi: 10.17632/rscbjbr9sj.2

26. RSNA Pneumonia detection challenge (2018).

https://www.kaggle.com/c/rsna-pneumonia-detection-challenge

27. Reshan, M. S. A., Gill, K. S., Anand, V., Gupta, S., Alshahrani, H., Sulaiman, A.,
Shaikh, A. (2023). Detection of Pneumonia from Chest X-ray Images Utilizing
MobileNet Model. Healthcare, 11, 1561. doi: 10.3390/healthcare11111561

21