ISBN 979-8-3503-1512-7

Deep Learning Model To Detect And Classify Bone Fracture

In X-Ray Images
Imrankhan M#1, Varalakshmi I#2, Arthi J#3, Pavithra P#4, Reshma J#5
#1,2AssistantProfessor, Department of Computer Science and Engineering
Manakula Vinayagar Institute of Technology, Puducherry, India.
#3,4,5UG Scholars, Department of Computer Science and Engineering

Manakula Vinayagar Institute of Technology, Puducherry, India.

#[email protected]
2023 International Conference on System, Computation, Automation and Networking (ICSCAN) | 979-8-3503-1512-7/23/$31.00 ©2023 IEEE | DOI: 10.1109/ICSCAN58655.2023.10394986

Abstract– The skeletal system of the human body is very engineering, and analysis, while deep learning models offer advanced
important and bones are vital to it. A support and facilitate body tools for image analysis, predictive modeling, and NLP to support
movement. To detect fractures in bones, doctors use X-ray images, healthcare providers and researchers in improving patient care and
but this manual technique is laborious and prone to mistakes. advancing the field of orthopedics and bone health. Also this, is the
Therefore, an automated approach for diagnosing bone fractures overarching field that encompasses various techniques and
is required. Power electronic device modeling is a frequent use of methodologies for working with data to extract valuable insights and
deep neural networks, or DNNs. The proposed system, provides a solve real-world problems.
deep neural network model using ResNet 50 to distinguish
between bones that are healthy and damaged. But when faced with
B. Deep Learning
little data, our deep learning model has a tendency to over fit. The
data augmentation techniques is used to increase the data set in Deep learning is a specialized sub-category of machine
order to overcome this problem. This stratergy uses 5-fold cross- learning that specifically deals with neural networks composed of
validation and our model is able to discriminate objects with multiple layers. Deep learning excels at handling complex,
92.44% classification accuracy. unstructured data like images, text, and audio by automatically
learning hierarchical features and representations. It includes neural
Keywords--Bone Fracture, Detection Classification, ResNet50, network architectures such as convolutional neural networks (CNNs)
Deep Learning, NLP. and recurrent neural networks (RNNs) , as well as more recent
advancements like transformers. While deep learning is a powerful and
I. INTRODUCTION important component of machine learning, it is just one of many
The many kinds of bones that make up the human body act as techniques within the larger field of data science [3]. Deep learning
the body's structural support system and shield critical organs like the techniques, particularly in the context of natural language processing
heart, brain, and lungs. These bones are prone to breaking, usually as (NLP), can be applied to improve the diagnosis, management, and
a result of trauma like falls or auto accidents. The human body is made research related to bone fractures. While deep learning models are
up of 206 different types of bones that differ in size, form, and primarily associated with computer vision tasks, such as identifying
properties. Because their bones are weaker, elderly people are more fractures in X-ray images, they can also enhance NLP applications in
likely to suffer from bone fractures. Broken bones must be properly this domain.
treated in order to heal [1].Bone fractures are frequent and can be
caused by a number of things that put too much pressure on the bones. C. NLP
Although minor fractures can be difficult to detect manually because
this technique is time consuming with a wide margin of error, doctors Natural Language Processing(NLP) is a multidisciplinary field
commonly use X-ray pictures to diagnose bone fractures. As a result, that draws upon techniques from computer science, linguistics, and
a computer-based system must be created to speed up and enhance the statistics. Natural Language Processing (NLP) is primarily a text-
precision of bone fracture diagnostics. This field is utilizing modern based technology, and it is not used directly for the detection of bone
deep learning technologies, such as those employed in power fractures in X-rays. Instead, the detection of bone fractures in X-ray
electronics and medical imaging [2]. The computer-based approach images is typically performed through computer vision and image
diagnoses bone fractures based on potentially noisy X-ray pictures. In processing techniques. However, NLP can play a complementary role
humans, oblique, complex, spiral, green stick, com minuted, and in the broader context of managing and documenting the diagnostic
transverse fractures are among the most common forms. To identify a process[4].
fracture and determine its exact location, medical practitioners
frequently use imaging methods such as ultrasound, computed
tomography, and electromagnetic radiation to establish the existence
III. PROPOSED METHOD AND TECHNIQUES USED
of an injury. The suggested method presents a novel technique that per-
processing the images by removing edges and noise using the ResNet-
50 algorithm for edge detection. After extracting features from the About 20,355 x-ray pictures, representing three distinct bone
bone picture, deep learning (DL) methods are used to train the system types elbow, hand, and shoulder are included in our data. We divided
for classification. our photographs into three categories: 70% training, 20% validation,
and 10% test, after loading all the images into data frames and giving
each image a label. The first step in the procedure is preparing the X-
II. RELATED WORK ray pictures [5], which could involve actions like data augmentation
and horizontal flipping to improve the dataset's quality. A ResNet50,
which neural network model is used in the following stage to classify
A. Data Science
bone types. The model loads one of three different models after
determining the particular bone type present in the image. These
Data science and deep learning work together to enhance the models have all been trained to recognize fractures in various types of
diagnosis, treatment, and overall management of bone fractures. Data bones. The presence of a fracture in the specified bone is subsequently
science provides the framework for data collection, feature ascertained using these specific models. This method uses the
Manakula Vinayagar Institute of Technology
Puducherry, India. IEEE ICSCAN2023
Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on August 22,2024 at 07:24:31 UTC from IEEE Xplore. Restrictions apply.
 ISBN 979-8-3503-1512-7

powerful image classification powers of ResNet50 (Fig.1) to professionals by accurately and efficiently analyzing X-ray pictures.
determine the kind of bone and then applies a customized model to The algorithm can identify whether the prediction shows a negative
each bone to detect the presence or absence of a fracture. The result, indicating no fracture, or a positive result, indicating the
algorithm may swiftly and accurately identify patients by using this presence of a bone fracture. The tool provides the user with the
two-step procedure to evaluate x-ray pictures effectively and outcomes of both bone fracture detection and bone type classification,
accurately. The prediction can be interpreted by the algorithm as either simplifying interpretation. The ability of this algorithm to identify
a positive result, indicating the presence of a bone fracture or negative bone fractures could be very helpful to medical practitioners, as it
result, indicating the absence of a bone fracture. The program will would enhance patient diagnosis and care. Its rapid and accurate X-ray
present the user with the outcomes of the classification of bone type image analysis expedites the diagnosis process, allowing patients to
and detection of bone fracture, making interpretation simple [7]. receive the necessary care on schedule.(Fig.2)
Medical practitioners may find this approach to be of considerable use
in identifying bone fractures and in enhancing patient diagnosis and
care. Its quick and reliable x-ray image processing helps expedite the
process of diagnosing and assist patients in receiving the right care.

Fig.1 ResNet 50 model architecture

Fig.2 Block Diagram for Fracture Identification
A. Proposed Method
This project's main objective is to develop a computerized system If you are writing a conference paper about using the ResNet50
that can recognize and classify fractures of bones in X-ray pictures. algorithm to detect bone fractures in X-ray pictures, you must include
This will be accomplished by creating a model using deep learning that a clear and comprehensive description of your study approach. A
can precisely identify and locate X-ray fractures in bones pictures of recommended framework for the methods section is provided below:
the shoulder, elbow, and hand. For fractured images, the system should
perform multi-class classification to determine the type and severity of • Collecting and reprocessing data
the fracture. This could include classifying fractures as simple, Describe the data set used in detail, including the number of X-ray
comminuted, greenstick, or other relevant classifications. Provide images, where they came from, and any actions done to improve or
precise localization of the fracture within the X-ray image to assist annotate the data.Describe in further detail how the data set was
medical professionals in treatment planning. This requires not only divided into categories for fracture and nonfracture.
detecting the presence of a fracture but also marking the specific area
of the bone affected. It is user-friendly interface for medical • Division of Data
professionals to interact with the system, allowing for easy input of X- Describe the method used to separate the data set into subgroups
ray images and clear visualization of the results [6]. The goal is to for testing, validation, and training. Include the percentage distribution
streamline the diagnostic process in orthopedic radiology, ensuring of each subset and the reasoning for the division.
that healthcare professionals receive accurate, informative, and
standardized reports to guide patient care. • Enhancement of Data and Initial Processing
Analyze how the data was augmented (e.g., rotated, scaled, or
B. Recommendation System flipped) to increase the diversity of the training data set. Draw
attention to any initial processing stages, such as re-sizing,
The 20,355 or so X-ray pictures in our data set are divided into
normalizing, or reducing noise in the image. Overview of the
three different bone types: elbow, hand, and shoulder. We separated
ResNet50 Architecture: Present the ResNet50 model, emphasizing its
our data set into three groups after integrating these photos into data
per-trained nature and appropriateness for the picture classification
frames and labeling each image: 70% for training, 20% for validation,
problem. Explain how the fine-tuning process worked, outlining the
and 10% for testing. The approach starts by perprocessing the X-ray
precise layers that were changed and the reasoning behind them.
images and augmenting the data, which involves flipping the images
horizontally. The next stage uses a ResNet50 neural network to C. Picture input:
classify the kind of bone that is seen in each picture. Following the
An image, typically expressed as a grid of pixels, is fed into
prediction of the bone type, a customized model appropriate for that
ResNet50.
type of bone is loaded from a set of three models, each of which has
been trained to recognize fractures in various forms of bone. The D. Layers Convolution:
purpose of these models is to determine if there is a bone fracture. This A series of convolution layers process the input image, applying
method makes use of ResNet50's powerful image classification skills filters to the image to efficiently extract features like textures, edges,
to determine the kind of bone, and then it applies a specialized model and patterns.
to fracture detection [9] in that particular bone. By using this two-step
procedure, the algorithm can precisely and quickly diagnose medical
Manakula Vinayagar Institute of Technology
Puducherry, India. IEEE ICSCAN2023
Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on August 22,2024 at 07:24:31 UTC from IEEE Xplore. Restrictions apply.
 ISBN 979-8-3503-1512-7

E. Remaining Blocks:
Residual blocks, which have a unique architecture that includes
skip connections, are used to build ResNet50. These connections
enable the network to effectively learn residual features, which makes
training very deep networks easier.

F. Pooling Layers:
These layers reduce the spatial dimensions of the feature maps by
acting after each pair of convolution layers. This method lowers Fig.4 Elbow fracture
computational complexity while highlighting important information.
G. Completely Networked Layers: Generally speaking, a fracture of the shoulder bone refers to a break or
crack in one or more of the bones. Any of these bones can sustain a
The last layer is made up of completely linked layers that are
fracture, which can range in severity from little hairline fractures to
mostly in charge of managing categorization jobs and producing
large breaks.A medical professional would usually do a physical
predictions based on the features that have been collected. ResNet50
examination and arrange imaging tests, such as X-ray scans, to confirm
is essentially a deep neural network made specifically for analyzing
a shoulder bone fracture. To properly plan the appropriate course of
images. To extract and understand picture properties. It uses residual
therapy, these tests can yield extensive information regarding the
blocks in conjunction with convolutional and pooling layers. Finally,
nature and location of the fracture.(Fig.2).
these learned characteristics are used by the last fully connected layers
to perform powerful picture categorization tasks.

IV. IMPLEMENTATION

Detecting and classifying normal versus fractured bones in the

hand, elbow, and shoulder for a project involves the use of medical
imaging and deep learning techniques using resnet50 algorithm.
Gather a diverse data set of medical images, including X-rays of the
hand, elbow, and shoulder. The data set should include images of
normal bones of elbow, shoulder and hand. Any break or fracture in
one or more hand bones the fingers, thumb, or palm bones, for
example is referred to as a hand bone fracture. Numerous factors,
including falls, sports injuries, auto accidents, and direct trauma to the
hand, can result in hand bone fractures. Identifying the symptoms and
Fig.2 Shoulder fracture
indicators of a hand bone fracture and getting medical help for a
suitable diagnosis and course of treatment are usually the first steps in
detecting one(Fig.1). Preprocess the images to enhance the quality and standardize the
format. This may include re sizing, cropping, and adjusting image
contrast. Extract relevant features from the images that can aid in
fracture detection and classification. This might involve by Identifying
bone edges and contours(Fig.3). Measuring the intensity of bone
structures. Calculating region-based statistics. Choose a deep learning
architecture for image analysis. Common choices include ResNet50 .
Train the model on the preprocessed data set, providing labeled
examples for normal and fractured cases. For classification, ensure
that the data set is labeled with the specific type of fracture (e.g., hand,
elbow, or shoulder).

Fig.1 Hand fracture

An elbow fracture is defined as a break or crack in any one of the three

bones that comprise the elbow joint. The two forearm bones are the
ulna, radius, and humerus (upper arm bone). Elbow fractures can
happen in several places on these bones and range in severity from
little cracks to large breaks [11]. These fractures may be the
consequence of underlying medical disorders that weaken the bones,
or they may be the result of severe events such as direct hits to the
elbow, sports accidents, or falls. In most cases, a combination of
clinical evaluation and medical imaging is used to detect an elbow
bone fracture. Physical therapy is often recommended as part of the Fig.3 Contour Identification
rehabilitation process to restore strength and function to the elbow
joint.(Fig.4).

Manakula Vinayagar Institute of Technology

Puducherry, India. IEEE ICSCAN2023
Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on August 22,2024 at 07:24:31 UTC from IEEE Xplore. Restrictions apply.
 ISBN 979-8-3503-1512-7

V. RESULT AND ANALYSIS

When the experimental results are contrasted with those of

classical edge detectors, such as canny edge detectors, it becomes
evident that whereas the former can reliably and precisely recognize
the structure of the bones silently, they are less successful in telling the
fracture apart facets. The created bone fracture detection system
system's performance is shown to be 78% accurate when it is tested
using a different set of X-ray images. The system should precisely
identify the presence, type, and extent of bone fractures in addition to
identifying the stage. It should also produce thorough reports with
patient information, the fracture site, recommendations for
rehabilitation, treatment plans, and a digitally signed medical
disclaimer from a doctor. Our suggested approach, which makes use
of a bone fracture segmentation algorithm, helps diagnose whether or Fig.2.Bone Loss
not a bone fracture exists by accurately detecting the bone structure
and, in the event of noise, the fracture edges. A detailed report is also
provided. The impact on clinical workflow, the efficiency of report
creation, and the accuracy of fracture identification should be the main
areas of analysis for project results, specifically number 20,355. For B. Fracture Prediction
continuous improvement, healthcare experts' input and constant
• ELBOW:
observation are crucial.
Creating a graph representation of the elbow's X-ray images, with
A. Bone Part Prediction each node representing an image and the edges capturing relationships
To determine whether there are fractures in every given X-ray between the images, is a common step in the graph-based methods for
image, the data-set needs to be meticulously tagged. Make a structure elbow bone fracture detection [10]. Fractures can be located and
for a graph to show the information. In the graph, every X-ray picture identified using graph-based models, including graph neural networks
represents a node. On the basis of connections or similarities between (GNNs) applied to this representation.(Fig.3)
images, edges connecting nodes may be formed. photos of the exact
same anatomical location or photos with comparable features, for
instance, can be connected. Take significant features out of the X-ray
pictures. Conventional methods involve the automatic extraction of
pertinent picture information, like bone density, alignment, or fracture
presence, utilizing convolutional neural networks (CNNs). Establish
weighted edges that indicate how similar or unlike the X-ray images
are from one another to establish relationships between the nodes. This
stage could entail graph distance computations or picture
embeddings.(Fig.1 )Graph data is processed by Graph Neural
Networks (GNNs). GNNs are ideally suited for this task because they
are made to operate with graph-structured data. In order to better
understand the spatial correlations between X-ray images, GNN layers
can compile data from nearby nodes.

Fig.3.Elbow Accuracy

Develop the GNN models to determine if an elbow X-ray image has a

fracture. To produce predictions, the model can leverage information
propagated across the graph and features collected from the
photos.(Fig.4).

Fig.2.Bone Accuracy

To determine if an X-ray image is likely to show a bone fracture, train

a prediction model, usually on top of the GNN layers. The features
taken out of the X-ray pictures and the data that moves along the graph
can be utilized by this model.(Fig.2)

Fig.4.Elbow Loss

Manakula Vinayagar Institute of Technology

Puducherry, India. IEEE ICSCAN2023
Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on August 22,2024 at 07:24:31 UTC from IEEE Xplore. Restrictions apply.
 ISBN 979-8-3503-1512-7

• HAND:

Detecting hand bone fractures using a graphbased approach

involves constructing a graph representation of X-ray images of the
hand and utilizing graph neural networks (GNNs) or similar models
for fracture detection. Establish the edges and their weights in the
graph to represent the similarity or dissimilarity between images. This
can involve calculating image embeddings or other measures of
similarity.(Fig.5).

Fig.7.Shoulder Accuracy

To achieve these predictions, the model leverages information

propagated across the graph and features derived from the
photos(Fig.8).

Fig.5.Hand Accuracy

Evaluate the model's performance using conventional evaluation

metrics, such as FI score, accuracy, precision, recall, and area under
the receiver operating characteristic curve (AUC-ROC) (Fig.6), along
with validation datasets.

Fig.7.Shoulder Loss

VI. CONCLUSION
Developing a system that uses deep learning to identify and
categorize bone fractures is the main goal of this project. Utilizing X-
ray pictures of both healthy and injured bones was part of the
investigation. First, a dataset including one hundred photos was
gathered. The dataset was expanded, ultimately consisting of 20,355
photos, in order to address overfitting problems in deep learning. Its
performance requires more validation on a bigger dataset in order to
be fully evaluated. The study described in the abstract aims to develop
an automated approach for diagnosing bone fractures. The authors
Fig.6.Hand Loss have created a deep neural network model to distinguish between
healthy and damaged bones. However, when faced with little data, the
deep learning model tends to overfit. To overcome this problem, the
• SHOULDER: authors have used data augmentation techniques to increase the data
set. This stratergy uses 5-fold cross-validation and our model is able to
Detecting shoulder bone fractures using a graph-based approach
discriminate objects with a 92.44% classification accuracy. The future
involves representing X-ray images of the shoulder as a graph, where
use of this study could be in developing an automated system for
each node corresponds to an image, and relationships are established
diagnosing bone fractures that can be used in hospitals and clinics. This
between these nodes. Graph neural networks (GNNs) or similar
system could help doctors save time and reduce errors in diagnosing
models can then be used to analyze the graph for fracture detection.
bone fractures.
Establish the graph's edges and give each one a weight to indicate how
similar or different the images are from one another. Techniques like
graph distance computations, picture embeddings, or other similarity
REFERENCES
metrics may be used in the generation of these edges(Fig.7).
[1] McBee and P Morgan, "Deep learning in radiology", Academic radiology,
vol. 25, no. 11, pp. 1472-1480, 2018.
[2] D. H. Kim and T. MacKinnon, "Artificial intelligence in fracture
detection: transfer learning from deep convolutional neural networks",
Clinical radiology, vol. 5, pp. 439445, 2018.
[3] Data Set: (Fracture Bone), [online] Available: https://www.iiests.ac.in/.

Manakula Vinayagar Institute of Technology

Puducherry, India. IEEE ICSCAN2023
Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on August 22,2024 at 07:24:31 UTC from IEEE Xplore. Restrictions apply.
 ISBN 979-8-3503-1512-7

[4] W. Abbas, S.M. Adnan, M.A. Javid, F. Majeed, T. Ahsan, S.S. Hassan et
al., Lower leg bone fracture detection and classification using faster
RCNN for X-rays images, in 2020 IEEE 23rd International Multitopic
Conference (INMIC) (IEEE, 2020), pp. 1–6 .
[5] A.Y. Yang, L. Cheng, M. Shimaponda-Nawa, H.Y. Zhu, Long-bone
fracture detection using artificial neural networks based on line features
of X-ray images, in 2019 IEEE symposium series on computational
intelligence (SSCI) (IEEE, 2019), pp. 2595–2602 .
[6] V. D. Vegi, S. L. Patibandla, S. SKavikondala, Z.Basha(2016),
Computerized Fracture Detection System using x-ray Images,
international Journal of Control Theory and Applications, vol. 9, pp. 615-
621.
[7] B,Swathika.,K,Anandhanarayanan., B, Baskaran and bR,Govindaraj.
(2015), ―Radius Bone Fracture Detection UsingMorphological Gradient
Based Image Segmentation Technique‖, International Journal of Computer
Science and InformationTechnologies, Vol.6, Issue No.2, pp. 1616-1619.

[8] http://gwyddion.net/documentation/userguide-en/wavelettransform.html .

[9] Kim, D.H.; MacKinnon, T. Artificial intelligence in fracture detection:

Transfer learning from deep convolutional neural networks. Clin. Radiol.
2018, 73, 439–445.
[10] Azizkhani, R.; Yazdi, Z.H.; Heydari, F. Diagnostic accuracy of
ultrasonography for diagnosis of elbow fractures in children. Eur. J.
Trauma Emerg. Surg. 2021, 184, 1–8.
[11] R.B.; Koskinen, S.K.; Pudas, T.K.; Kajander, S.; et al. Deep learning
accurately classifies elbow joint effusion in adult and pediatric
radiographs. Sci. Rep. 2022, 12, 11803.

Manakula Vinayagar Institute of Technology

Puducherry, India. IEEE ICSCAN2023
Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on August 22,2024 at 07:24:31 UTC from IEEE Xplore. Restrictions apply.