0% found this document useful (0 votes)
32 views14 pages

Quantum Algorithm For Quicker Clinical Prognostic Analysis: An Application and Experimental Study Using CT Scan Images of COVID-19 Patients

Articulo Cientifico

Uploaded by

titoshosito2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
32 views14 pages

Quantum Algorithm For Quicker Clinical Prognostic Analysis: An Application and Experimental Study Using CT Scan Images of COVID-19 Patients

Articulo Cientifico

Uploaded by

titoshosito2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227

https://doi.org/10.1186/s12911-021-01588-6

RESEARCH ARTICLE Open Access

Quantum algorithm for quicker


clinical prognostic analysis: an application
and experimental study using CT scan images
of COVID‑19 patients
Kinshuk Sengupta1,2,3* and Praveen Ranjan Srivastava2,3

Abstract
Background: In medical diagnosis and clinical practice, diagnosing a disease early is crucial for accurate treatment,
lessening the stress on the healthcare system. In medical imaging research, image processing techniques tend to be
vital in analyzing and resolving diseases with a high degree of accuracy. This paper establishes a new image classifica-
tion and segmentation method through simulation techniques, conducted over images of COVID-19 patients in India,
introducing the use of Quantum Machine Learning (QML) in medical practice.
Methods: This study establishes a prototype model for classifying COVID-19, comparing it with non-COVID pneu-
monia signals in Computed tomography (CT) images. The simulation work evaluates the usage of quantum machine
learning algorithms, while assessing the efficacy for deep learning models for image classification problems, and
thereby establishes performance quality that is required for improved prediction rate when dealing with complex
clinical image data exhibiting high biases.
Results: The study considers a novel algorithmic implementation leveraging quantum neural network (QNN). The
proposed model outperformed the conventional deep learning models for specific classification task. The perfor-
mance was evident because of the efficiency of quantum simulation and faster convergence property solving for an
optimization problem for network training particularly for large-scale biased image classification task. The model run-
time observed on quantum optimized hardware was 52 min, while on K80 GPU hardware it was 1 h 30 min for similar
sample size. The simulation shows that QNN outperforms DNN, CNN, 2D CNN by more than 2.92% in gain in accuracy
measure with an average recall of around 97.7%.
Conclusion: The results suggest that quantum neural networks outperform in COVID-19 traits’ classification task,
comparing to deep learning w.r.t model efficacy and training time. However, a further study needs to be conducted
to evaluate implementation scenarios by integrating the model within medical devices.
Keywords: Medical imaging and analysis, Artificial intelligence, Quantum neural networks, Medical informatics

Background
In the clinical trial and drug discovery process, the role
of statistical analytics and machine learning has been
shown to be significant, especially in biological imag-
*Correspondence: [email protected] ing and analysis, commonly used at various stages, from
1
Microsoft Corporation, New Delhi, India preclinical R&D to clinical trials, solving problems like
Full list of author information is available at the end of the article

© The Author(s) 2021. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit http://​creat​iveco​mmons.​org/​licen​ses/​by/4.​0/. The Creative Commons Public Domain Dedication waiver (http://​creat​iveco​
mmons.​org/​publi​cdoma​in/​zero/1.​0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 2 of 14

Table 1 Empirical research for detecting COVID-19 using deep ­learninga


Model proposed Study Dataset size Training Model performance
samples
sufficiency

MODE (Multi-objective differential evolu- Singh et al. [47] 1000 + CT images +  +  + Accuracy—90.6%
tion) based CNN
UNET +  + Chen et al. [44] 46,000 + CT images +  +  + Accuracy—95.24%
Sensitivity—100%
Specificity—93.55%
Stacked Two CNN three dimensional for Xu et al. [43] 19,000 + CT Images with COVID-19, 1175 +  +  + Accuracy—86.70%
classification and VNET for Segmenta- healthy samples
tion
COVNet + ResNet 50 for classification and Li et al. [35] 4000 + CT Samples +  +  + Sensitivity—90.0%
U-Net for segmentation Specificity—96.0%
Transfer Learning + ResNet 50 for classifi- Jin et al. [10] 1100 + total samples with 730 positive +  + AUC—0.991
cation and UNet +  + (3D) for segmenta- samples Sensitivity—97.4%
tion Specificity—92.2%
Inception with Transfer Learning tech- Wang et al. [32] 450 + CT scans of confirmed COVID-19 + Accuracy—82.9%
nique Sensitivity—84.0%
Specificity—80.5%
Neural Networks with ResNET 50, atten- Song et al. [42] 750 + Images + Accuracy—86.0%
tion technique and Feature Pyramid F-Score—87.0%
Network Sensitivity—93.0%
Deep Conv Net(2D) on ResNet-50 for clas- Gozes et al. [41] 50 + patients’ samples + AUC—0.996
sification and UNet for segmentation Sensitivity—98.2%
Specificity—92.2%
VBNet neural network to Shan et al. [13] 200 + CT scan samples + Dice Coef.—91.6%
Segment COVID-19 infection regions in
CT scans
2D CNN Jin et al. [10] 970 CT Scan samples + Accuracy—94.0%
AUC—0.979
SVM + Wavelet transformation Barstugan et al. [39] 150 CT Scan Samples + Accuracy—99.68%
Deep CNN(3D) for classification and U-Net Zheng et al. [46] 500 + Samples + AUC-ROC—0.959
for segmentation
DCNN Heinrich et al. [31] 500 + Samples + Dice Coef.—71.0%
CNN-LSTM Islam et al. [60] 4000 + X-ray Samples +  +  + AUC—0.992
Sensitivity—99.3%
Specificity—98.9%
VGG-19-RNN Zabirul Islam et al. [59] 6000 + x-ray samples(sample with CoViD, +  +  + Accuracy—99.9%
pneumonia and normal cases) AUC—99.9%
Recall -99.8%
Ensemble DCCNs Singh [1] 6000 + (sample with CoViD, tuberculosis, +  +  + Accuracy—99.2%
pneumonia)
a
Refer to Abbreviations for detailed nomenclature

sputum detection [1], image augmentation [2] and other diverse COVID-19 diagnosis research work using deep
applications, such as nucleus counting [3]. In the recent learning and traditional machine learning methods as
past, substantial research work have been proposed stud- shown in Table 1. Currently, with evolving COVID-19
ying various classical machine learning and deep learning mutants it is now becoming extremely important to lev-
methods applied to an image that assists scientists and erage faster and accurate solutions for clinical discovery,
medical practitioners in analyzing and seeing inorganic prompting therefore our study to understand the evolu-
growth or accumulation of tissues, cells, and subcellular tion in terms of offering medical imaging solutions for
components in CT scans, along with a more technology- factor detection of mutant variants [7].
oriented solution in the space of wearable technology There has been active research in biomedical image
[4] and tele-health care services to discover COVID-19 analysis using deep learning methods, whereby deep
[5]. An example of detecting brain tumors through deep learning seems to have outperformed most com-
learning methods has been studied by researchers [6] and puter vision problems for instance [8]. Nevertheless,
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 3 of 14

Table 2 Previously studied applications of machine learning in drug discovery and medical diagnosis
Description of study Author Methodsa

Skin cancer detection Kadampur and Al Riyaee [9] DNN


Protein structure prediction Torrisi et al. [18] DL-CNN, DL-RNN
Cuneiform Dehydration Method for Medical Diagnosis Baranov [50] Image Filtering, thresholding, Gaussian blur
Quantitative structure–activity relationship analysis in drug discovery Uesawa [51] Deep learning
Quantum chemical properties analysis Gilmer et al. [52] Message passing neural network (MPNN)
Predicting compound property and activity Mayr et al. [27] Multitask DNN
Predicting pharmacological properties of drugs and for drug repurpos- Aliper et al. [53] DNN
ing leveraging transcriptomic data from the LINCS project
Automatic molecular structure learning Merkwirth and Lengauer [54] DNN and RNN
Lusci et al. [55]
Method to model drug induced liver injury (DILI) Xu et al. [56] UGRNN
Neural fingerprints of the compound Duvenaud et al. [57] Graph CNN
Predicting the ligand–protein interactions Gomes et al. [48] CNN, DNN
Predicting the reactions and retrosynthetic analysis Liu et al. [36] Neural sequence to sequence model and
Monte-Carlo tree search
Drug discovery with on short learning Altae-Tran et al. [58] LSTM
Visual Screening from protein–ligand complex Pereira et al. [49] DNN
Facilitating probe selection for gene-expression arrays Tobler et al. [3] Naïve Bayes, neural nets
a
Refer to Abbreviations for detailed nomenclature

computer vision techniques have shown vast opportuni- applications being evaluated to solve problems in the
ties in numerous application areas, especially in medi- fields of optimizing deep learning or machine learning
cal research and healthcare [9]. Medical imaging does tasks, finance [15], drug discovery [16], along with help-
provide better visibility than standard medical records’ ing in shedding light on various clinical research [17].
data assessment, such as solving for Diabetic Retinopa- Table 2 enlists extant literature that has dealt with drug
thy [10]. High-resolution images analyzed can provide discovery.
any growth details on actuals, on a day-to-day basis, Although, there have been other studies that have
helping a medical practitioner to evaluate the situation deliberated upon the success of employing deep learning
quickly and provide a better treatment. It is apparent to in drug discovery [16] and MRI image analysis for brain
be mentioned that the success of leveraging deep learn- tumors, and for detecting and segmenting pneumonia
ing over traditional machine learning methods have traces using classical machine learning models [6] or
been studied along with wide area of application in the leveraging deep learning in biomedical image segmenta-
medical domain [11]. Moreover, recent developments of tion applications [18]. The core purpose of this paper is
quantum computing, vis a vis its application of quantum to evaluate and provide empirical evidence for applying
algorithm in varied domains, has now opened up new Quantum algorithms in medical imaging and drug dis-
research areas for further optimizing classical machine covery problems.
learning problems [12]. In fact, recently, researchers from
Massachusetts Institute of Technology (MIT) created an Quantum machine learning
algorithm to overcome the challenges of developing com- The recent developments of Quantum Enhanced Learn-
putationally efficient and performing algorithms in order ing [19], fusing AI and ML to obtain significant optimal
to solve several medical imaging problems [13]. solutions for boosting algorithmic performance has given
The domain of medical science needs significant devel- rise to a new area of research termed ‘Quantum Machine
opment for making sense of an analysis generated from Learning’ (QML), which has effectively evolved from the
an image. Previous studies dealing with this topic, have theory of quantum computing. The concept behind lever-
discussed the varied applications of machine learning, aging quantum computing for machine learning tasks is
deep learning, and quantum algorithms in drug discov- to inherently achieve solution parallelism [20], achieved
ery and screening process, thereby solving problems for optimal constraint solving, using Moore’s law [21].
that include compound property and activity prediction, Quantum algorithms are centered on the concept of
using multitask DNN on 12,000 compounds [14]. Impor- Boolean algebra (e.g., OR, AND, and NOT gates) and
tantly, Quantum is a new paradigm today, with multiple quantum physics. The data storage layout is established
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 4 of 14

Fig. 1 Execution block diagram of classical machine learning/deep learning versus quantum machine learning algorithm designing (refer to
Table 4. for algorithmic details on QML)

from Quantum bit (Qb) or qubits1 that depends on the- The need for quantum algorithms
oretical foundations of electron spin [22]. Importantly, QML is considered as one of the future areas of research
quantum methods in addition can translate other than 1’s in deep learning algorithms. The two key tasks QML can
or 0’s such as complex information or negative values. A better perform when compared to classical deep learning
typical model development flow diagram is demonstrated techniques include:
in Fig. 1, explaining the basic control flow difference of
classical machine learning versus Quantum machine 1. Optimization [23] and Gibbs Sampling [24].
learning algorithms. 2. Enhance learning algorithms like Bayesian networks
[25], Tensors, and search.

QML’s ability to deal with large-scale biased datasets


yield faster complexity factors for major classical com-
puting and machine learning tasks, consuming thereby
1
Refer to Abbreviations for detailed nomenclature.
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 5 of 14

less space and time. Effectively, it uses quantum anneal- researchers are leveraging includes quantum techniques
ers and tunneling for loss function minimization tasks, in Quantum Simulation [28], applied around nano-
solving thereby complex problems of finding super-local tech, bio-medical imaging, physical chemistry, and with
minima, and a close approximation of global minima. quantum systems tasks, such as search [29], which fur-
In fact, multiple methods within the QML strategy are ther provides polynomial speed, as compared to classical
based on fast quantum algorithms for linear algebra, algorithms for other varied scenarios.
and semi-definitive or constraint-specific problems like
optimization, as in the case of neural network models
for weight adjustment, during both search and optimiza- Method
tion tasks, similar to gradient descent-based optimizer. The section discusses the methodology followed for
Assuming the task is to optimize a linear function of building the model, provide details on overall data col-
MxM matrix(X), over a parallel space with constraints(c), lection process followed, key modeling process imbibed
the solver has runtime complexity of O(c(c2 + nω + cns) for the study, and analysis performed with quantum
logO(1)(cnR/ϵ)) [26]. Notably, herein, ϵ is denoted as networks.
an approximation factor, while s denotes sparsity, and
R represents a range that is bound to yield an optimal
Quantum neural network model
matrix(X). Notably, these proven methods outperform
classical optimization methods, yielding thereby com- This study conducted an experimental analysis with a
plexity of O (ncs(Rr/ϵ)4 + ns(Rr/ϵ)7) as proposed by new variant of a learning model to further take advan-
Arora and Kale [27]. The proposed optimizers in turn, tage of quantum computing devices to perform learning
tend to improve the overall solution convergence for any tasks with quantum data [30]. We assumed that Quan-
machine learning problems. volutional neural network or Quantum neural network
This paper looks to address two major research ques- (QNN) would solve classical deep learning problems
tions, while evaluating the application of QML in specific to be computationally faster from the design paradigm.
practice, specifically focusing upon medical image diag- Figure 2a illustrates the QNN architecture, which would
nostics and/or drug discovery, help in understanding the Quantum network design
methodology. Further, for simulation, we benchmarked
the QNN model across other studies from extant lit-
RQ1 Are quantum algorithms suited for large-scale
erature. The rationale behind this exercise was to help
classification problems in medical image diagnostics
in exploring varied application scenarios in the medical
dealing and industrial applications?
image analysis task that is presented in subsequent sec-
tions. Notably, the process of designing QNN has been
RQ2 Can quantum algorithms outperform classifica-
described in Table 3 and has been elaborated upon in the
tion or segmentation tasks in comparison with classical
subsequent sections.
deep learning methods w.r.t model efficacy, biased train-
The proposed system is illustrated below in Fig. 2b.
ing, and inferencing performance on high-resolution
From a practical implementation perspective, the model
clinical image data?
accepts input image rescaled to 4 × 4 size before being
fed into the Unitary matrix to obtain features at different
The research questions would further provide sup-
channel. Further the features were leveraged to create a
port to exemplify the application of quantum theory in
quantum circuit model, thereafter, followed by compiling
optimizing deep learning techniques to achieve superior
the model using a loss function and optimizer using Ten-
performance in solution convergence and quality of the
sorFlow Keras model utility library.
model. Another important aspect to emphasize on sup-
porting production deployment is selecting appropri-
ate quantum hardware for training, while deploying the Data and pre‑processing
model for real-time inferencing in health informatics This section discusses the data collection process and the
applications, which is discussed later in this paper under pre-processing activities that were conducted during the
the experiment section. experiments. Notably herein, the relative transformation
measures were required for modeling a QML algorithm.
Application of quantum machine learning Additionally, this section consists of two sub-sections;
The foundation of QML targets to solve research foun- the first, describes the data collection process, along with
dation problems in mathematical analysis to generalize the larger data schema, while the second, discusses the
quantum to improve classical learning tasks with poten- affirmative steps that are taken for pre-processing in pre-
tial optimization to speed of execution. Some of the task’s modeling stages.
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 6 of 14

Fig. 2 a QNN Architecture [31]. b The proposed model

Table 3 Algorithm design stages for quanvolutional neural network

Stage 1: An input image with small region of interest is embedded into a quantum circuit. An example of a 2 × 22 × 2 square region
Stage 2: A quantum computation, associated with a unitary matrix(Ua) in Fig. 3, is performed on the system. A Cirq could generate the unitary, most
quantum operations have a unitary matrix representation applied to the gate, operations and circuit that represents an object
Stage 3: The system is then quantified by obtaining the list of classical expected values
Stage 4: Similar to the classical convolution layer, each expected value is mapped to a different channel of a single output pixel
Stage 5: The process is iteratively executing across different regions of the image. A full input image scan is viable by re-positioning an output object
positioned a multi-channel image
Stage 6: The quantum convolution layer would additionally abide to quantum or classical layers
a
Refer to Abbreviations for detailed nomenclature

Data collection a significant amount of training dataset for building


The model development and verification for quantum an efficient model [32], thus a through process was
deep learning-based image classification would require followed for sampling the image files to eliminate any
a large sample set for the quantum machine to per- representational biases. Hereafter, this study combined
form. As a pre-requisite step, learning models require data shared by semanticscholar.com, along with the
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 7 of 14

Table 4 Dataset
Dataset description Data statistics Source

CT scans for COVID-19 349 CT images of 216 patients https://​github.​com/​UCSD-​AI4H/​COVID-​C T


SIRM COVID-19 database Sample < 50 images https://​www.​sirm.​org/​en/​2020/​03/​31/​COVID-​19-​case-4/
Radiopedia COVID dataset Sample < 50 images https://​radio​pedia.​com
Eurorad dataset Sample < 50 images https://​www.​euror​ad.​org/​case/​16689
Center for artificial intelligence in medicine More than 5000 + sample images of patients https://​aimi.​stanf​ord.​edu/​resou​rces/​COVID​19#​data
and imaging
Total samples selected ~ 10,000 +

Fig. 3 a Sample CT scan image illustrating small to medium patches forming with each week’s diagnosis. b Sample CT scan image of CoViD-19
diagnosed

research work done by Chen et al. [33], and Jin et al. search process ensured that the data resolutions were
[34]. consistent across all the sources. The preferred resolu-
The data collection process adopted a strategy to tion range of images were chosen (256 × 256 and 448 ×
collect CT scan samples of varied age groups, ranging 448), because image resolution does play a vital role in
from 20–30, 30–45, 45–60, and above 60 years of age, deep learning space, and often, high-resolution images
with both positive and negative samples. Key sources do go on to impact model training performance and
were identified, based on represented data statistics efficacy to a great extent [35].
(Table 4), along with other open dataset sources from Figures 3a, b and 4, adopted from Shi et al. [36];
Microsoft open research database, Google dataset Li et al. [37]; and Hani et al. [38] represent classical
search, Stanford, and MIT datasets. Notably, the data CoViD-19 and non-COVID-19 (influenzas and virus
pneumonia) scans. Based on this, we discuss the overall
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 8 of 14

Fig. 4 CT scan of two patients of 45-year and 48-years of age with influenza virus pneumonia and Goodpasture syndrome shows bilateral
ground-glass opacities in contrast to COVD-19 patients (Hani et al. 2020)

finding that was observed from CT scans taken through impacts the image segmentation process in the overall
a specific time duration. modeling situation.
The sample data illustrated in Fig. 3a is identified as
CoViD-19 positive. A significant growth in building Handling representation and measuring bias in image
patches was observed in the lungs over 10 days during dataset
quarantine (day 5 and day 15 scan). The scan of day 20 The section discusses the impact of measuring bias that
showed the formation of a dense mucus that was concen- depicts the systematic value distortion, which takes place
trated across the lungs. Figure 3b illustrates mucus, seg- when an issue with a specific device is utilized to visu-
mented across a small patch growth across two weeks of alize and observe an image quality from a training per-
supervision. An evaluation dataset of non-CoViD-19 suf- spective. Importantly, this type of bias is hard to replicate
fering from pneumonia and influenza was also sourced with sampling technique, and thus requires a manual
for validation of the model results. review of the colored images, being used for training [40].
This study further leverages upon bootstrapping resam-
Pre‑processing and normalization pling technique [41] with different ratios to assemble the
Importantly, the datasets that were used come from mul- required representation of the dataset for experiments.
tiple sources; the process of data normalization included
comparing the homogeneity of data sources, while fur- Experiments
ther calibrating the images to the required scale for In this section, the data pre-processing, model imple-
modeling. The study leveraged upon color models of an mentation, and evaluation methods have been explained.
abstract mathematical model, describing the way colors The experiment processes involved choosing a base
can effectively be represented as tuples of numbers that model for initial trials and develop the same using the
are useful in viewing conditions. Once the image was data collected. Based on various performance criteria
thoroughly analyzed, the dataset was normalized using compared between QNN, QCNN,2 Hybrid CNN with a
erosion and dilation [39], leveraging upon OpenCV single filter and Hybrid CNN with multiple filters (Fig. 5)
library, a morphological transformation method, primar- from the simplicity of circuit design and performance
ily used for handling noise, or detecting intensity col- measurement, QNN was chosen for remaining bench-
lisions. Further, image de-noising (Buades et al. 2011) marking during the trials. Furthermore, the experiments
and scaling was done, using Python-OpenCV library, were performed using TensorFlow Quantum (TFQ), and
which in turn, was implemented to the entire dataset for a python framework for QML development. Notably,
standardization, with the help of fast Nl Means Denois- we leveraged upon D-wave Leap and TensorFlow Quan-
ing function for colored images, where the source image tum Framework as a platform for training and evaluat-
input of 8-bit 3-channel images were provided with tem- ing the experimental setup. The estimated wait time for
plate window size of 7 pixels and 21-pixel, and hColor of problem submission was 1–10 s on a 2041 qubits system,
10 in order to remove the colored noise; post this, they
were kept into consideration for the completion of the
de-noising process. Notably herein, de-noising generally
2
Refer to Abbreviations.
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 9 of 14

Fig. 5 A quantum CNN: hybrid Convolution with multiple quantum filters

under 13.5 qubit temperature (mK). Detailed specifica- processing stage. The overall image calibration steps per-
tions of the platform may be referred from D-wave and formed are stated as below,
TensorFlow.3
TFQ’s core focus area is generally on quantum data, 1. Input raw data using Keras
and a hybrid quantum-classical model. Various compo- 2. Filtering the dataset to only 3 s and 6 s
nents that need to be followed to build a quantum circuit 3. Downscales the images to fit in a quantum hardware.
within the TensorFlow environment have been described 4. Treating and removing contradictory examples
below, 5. Convert binary images to Cirq circuits
6. Convert the Cirq circuits to a TensorFlow quantum
a. Circuit—Cirq is used to design the quantum circuit circuits
(Fig. 6). Cirq4 is a python framework for writing,
optimizing quantum circuits executing in quantum In QML, a pixel is represented as a qubit, wherein each
hardware. stage would actually depend on the pixel value. The pro-
b. Pauli Sum—the linear combinations of tensor prod- cess of encoding the data into the Quantum circuit was
ucts of Pauli operators5 defined in Cirq is represented iterated at multiple threshold values, in the range [0.5,
by Pauli sum, operations like circuits, create batches 0.6, 0.7]. A circuit at 0.5 threshold is represented in Fig. 7,
of operators of varying size are of such type. and which effectively is a form of 2-layer circuit design
for binary classification problems. In terms of model
The experiments performed, involved various fea- development, various iterations were performed to
ture engineering and transformation stages applied to
design. Since the quantum machine cannot handle the
large size of the image, we re-scaled it to 4 × 4 at the data

3
https://​www.​dwave​sys.​com; https://​www.​tenso​rflow.​org/​quant​um/​conce​pts

4
Refer to Abbreviations for detailed nomenclature.
5
Fig. 6 A quantum circuit
Refer to definition of pauli operators in nomenclature section.
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 10 of 14

Fig. 7 The circuit from the training samples in the first iteration of the 2-layer circuit

optimize the general performance of the model through a Table 6 Loss score and hinge accuracy
range of hyper-parametrization testing at various epochs.
Epoch Loss Hinge accuracy Validation loss Val hinge
Finally, the experiment encompassed creating a 2-lay- accuracy
ered model (Fig. 7) fitting the data-circuit size, including
1/10 0.6566 0.7534 0.3870 0.8160
both preparation and readout operations. In fact, this
method could possibly be compared to running a small 2/10 0.3568 0.8263 0.3348 0.8311
recurring neural network across pixels. Notably herein, 3/10 0.3281 0.8497 0.3269 0.8579
each layer uses n instances of the same gate, with each 4/10 0.2994 0.9061 0.2894 0.8769
of the data qubits acting on the readout qubit. Addition- 5/10 0.2707 0.9542 0.2594 0.8978
ally, the model building process further used hinge loss 6/10 0.2707 0.9582 0.2293 0.9188
as a loss function, along with adaptive learning rate opti- 7/10 0.2133 0.9586 0.1993 0.9397
mization (ADAM) optimizer instead of stochastic gradi- 8/10 0.1872 0.9582 0.1692 0.9607
ent descent-based optimizer, which was computationally 9/10 0.1872 0.9582 0.1692 0.9607
inexpensive, and possibly even easier to implement. The 10/10 0.1821 0.9692 0.1691 0.9657
experimentation process was conducted with different
epoch and batch sizes, and the results are presented in
the following sections; notably, the overall model param- Evaluation criteria
eters are shown in Table 5 below. This paper presents validation loss and validation accu-
racy as evaluation criteria for the QNN model, whereby
the selected metric is ‘hinge loss’ for the experiment, as
the problem formulation alludes to a binary classification
problem [42]. Importantly, the ‘hinge loss’ represents the
Table 5 QNN parameter difference in prediction from actuals. Moreover, since
Parameter(s) Value validation loss is not used to update weights in general,
it possibly serves as the right measure of any neural net-
Layer PQC work model. A hinge loss6 i.e., l(y) is calculated by com-
Output shape (None, 1) paring prediction (y) with the actual target for prediction
Param 32 (t), followed by subtracting the value from 1, while com-
Model Sequential puting thereafter the maximum value between 0 and the
Loss function Hinge result of the earlier computation.
Optimizer ADAM
Evaluation metrics Hinge accuracy 6
Refer to Abbreviations for detailed nomenclature.
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 11 of 14

Table 7 Confusion matrix the consideration are loss and hinge accuracy shown in
Actual Values Table 6 and confusion matrix shown in Table 7.
The change in epoch lowers the loss score, while
improving the overall hinge accuracy. The change in
Positive Negative
accuracy score sees a significant lift after the third epoch,
and gradually improves, reducing thereby the loss to
Positive 874 26 0.1559; notably, the percentage score of the change of
Predicted Values

loss is shown in the analysis in Fig. 8. The overall preci-


sion of the implemented model is 97.11%, whereas recall
Negative 20 580 is 97.76% respectively.
While comparing the traditional deep learning model
with QNN (Fig. 9), the latter obtained a 2.92% lift from
Total Evaluation Sample
the 2D CNN model, trained for classification problems in
1500 the CT scan data set of COVID-19 patients. The analysis
further describes the efficiency of the overall system to
scale classification models, whereby the total time to train
the model with 9500 images was observed to be 52 min
on quantum optimized hardware, while compared to an
0.7 experiment using CNN on K80 GPU Instance.
0.6
0.5
Loss Score

0.4 Discussion
0.3 The study does entail certain limitations; for instance, the
Loss
0.2 dataset and the training time comparison were limited to
Val Loss available samples. The images collected were limited to
0.1
0 CT-Scan with a focus on identifying discoverable patches
1 2 3 4 5 6 7 8 9 10 denoting COVID-19 infection. Further, a detailed study
Epoch is required to incorporate signals that may occur in a CT-
Fig. 8 Change in loss per epoch (training and validation) Scan image tending to non-COVID signals to make the
model more robust on detection. Further, this study is
limited to a minimum viable solution model that would
possibly need additional research to take the present ver-
100 96.92 sion of the model into readily deployable services mode,
94.01
95 90.6 within the ambit of the biomedical device ecosystem
90 86.7 86.01 from an end-to-end technology implementation stand-
85 82.9 point, supporting thereby large-scale usage in clinical
80 trials.
75 The crucial point of discussion from a future research
MODE Stacked Transfer NN with 2D CNN QNN standpoint would be around how to leverage the power
CNN Learning Resnet of quantum algorithms on hardware and localize it to
% Accuracy biomedical devices for seamless analysis. This study did
Fig. 9 Comparison of various DL models versus QNN demonstrate a substantial advantage to overall medical
imaging problems, using quantum learning techniques,
while also implementing classical learning models in the
context for performance and efficacy in improved model
Results implementations. The model demonstrates robustness
To conclude the experimentation process and bench- while comparing the overall recall value, as any incor-
marking with other relevant methods adopted for rect misclassification of CoViD-19 infected patient could
detecting COVID-19 patients, the model built, used a lead to reduction in the overall significances of the pre-
9500-training dataset, encompassing an evaluation con- dicted outcome further deferring accurate medical diag-
ducted over 1500 validation sample sets. The efficacy nosis when compared to a wrongly classified patient with
and the performance evaluation criteria are based on pneumonia or other viral infection that may show similar
experiments performed with three, five and ten epochs strains.
and similar batch sizes. The key metrices taken into
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 12 of 14

While the study also discussed the aspects of hardware is a basic unit of quantum information representing information as 0 or 1;
requirements for training and evaluation of such models, CoViD-19: Coronavirus disease.

significant research is still under process, whereby there Acknowledgements


has been an attempt to launch an economical cloud plat- Not Applicable.
form for quantum hardware simulation and modeling.
Authors’ contributions
Further, this study provides scope for new development KS staged the overall conceptualization of the problem, performed data
area of edge-quantum computing, and opens up research collection and experimentation. Dr. PRS validated key metrics and revised the
dialogues around faster diagnostics, and easy interpreta- manuscript from a quality perspective. Both authors have read and approved
the final manuscript.
tion of quantum algorithms in the medical world.
Funding
Not Applicable.

Conclusion Availability of data and materials


In this study, we proposed a quantum neural learning The datasets generated and/or analyzed during the current study are available
in the GitHub repository, https://​bit.​ly/​3xGUS​8Q. The dataset does not consist
model to classify patients with COVID-19 infection,
of any confidential data.
leveraging upon computed tomography scan images in
medical diagnosis. The suggested model attained optimal
Declarations
degree of model efficacy during an experimental com-
parison, yielding 96.92% of accuracy overall, leveraging Ethics approval and consent to participate
Not Applicable. The data leveraged in the study were obtained from open-
9500 + CT-Scan sample images. Additionally, the overall
source databases, no additional approval is required.
computation time for training the model recorded was
52 min, with the entire sample, along with the inferenc- Consent for publication
Not applicable.
ing time recorded, which was a minute per image. This
overall model training time was significantly less as com- Competing interests
pared to classical CNN model building with similar sam- KS is full-time Data Scientist at Microsoft Corporation India. All other authors
declare they have no conflict of interest.
ples, using quantum hardware. Our results thereby yield
not only a significant lift in the overall accuracy, but also Author details
optimizes upon the execution time. The model could be 1
Microsoft Corporation, New Delhi, India. 2 Department of Information System,
Indian Institute of Management, Rohtak, India. 3 City Southern Bypass, Sunaria,
further deployed in clinical trials and medical diagnoses,
Rohtak, Haryana 124010, India.
which have a significant impact on overall decision sup-
port for treating patients with early symptoms. Moreo- Received: 4 September 2020 Accepted: 18 July 2021
ver, a medical practitioner could also leverage upon our
framework for quicker diagnostics, helping him/her to
follow-up with the right treatment, and thereby save a
life. References
1. Niu J, Shi Y, Cai M, Cao Z, Wang D, Zhang Z, Zhang X. Detection of sputum
The impact of this paper not only quantifies the abil- by interpreting the time-frequency distribution of respiratory sound sig-
ity of QML, but also would help clinical scientists build nal using image processing techniques. Bioinformatics. 2017;34(5):820–7.
diagnostic tools applied to drug discovery and disease https://​doi.​org/​10.​1093/​bioin​forma​tics/​btx652.
2. Bloice M, Roth P, Holzinger A. Biomedical image augmentation using
identification problems with much faster analytical capa- Augmentor. Bioinformatics. 2019;35(21):4522–4. https://​doi.​org/​10.​1093/​
bility using quantum hardware. The study further pro- bioin​forma​tics/​btz259.
vides prospects to evaluate quantum algorithms for more 3. Shu J, Liu J, Zhang Y, Fu H, Ilyas M, Faraci G, et al. Marker controlled super-
pixel nuclei segmentation and automatic counting on immunohisto-
complex problems pertaining to image segmentation. chemistry staining images. Bioinformatics. 2020;36(10):3225–33. https://​
doi.​org/​10.​1093/​bioin​forma​tics/​btaa1​07.
4. Milon Islam Md, Saifuddin Mahmud LJ, Muhammad Md, Islam R, Noorud-
Abbreviations din S, Ayon SI. Wearable technology to assist the patients infected with
DL: Deep learning; CNN: Convolutional neural network; QNN: Quantum neural novel coronavirus (COVID-19). SN Comput Sci. 2020;1(6):320.
network; QCNN: Quantum convolutional neural network; ADAM: Adaptive 5. Ullah SMA, Milon Islam Md, Mahmud S, Sheikh Nooruddin SM, Raju
learning rate optimization; MODE: Multi-objective differential evolution; TU, Rezwanul Haque Md. Scalable telehealth services to combat novel
RNN: Recurring neural networks; AUC-ROC: Area under the curve-receiver coronavirus (COVID-19) pandemic. SN Comput Sci. 2020;2(1):18.
operating curve; DNN: Deep neural network; LSTM: Long short-term memory; 6. Lather M, Singh P. Investigating brain tumor segmentation and detection
CT: Computed tomography; CXR: Chest x-ray; Unitary Matrix(U): In Quantum techniques. Procedia Comput Sci. 2020;167:121–30. https://​doi.​org/​10.​
mechanics, the Hermitian adjoint of a matrix is denoted by a dagger (†) 1016/j.​procs.​2020.​03.​189.
and the equation is represented by U†U = UU† = I, comparable with linear 7. Staff S. Covid-19: Double mutant strain could be considered a variant of
algebra equation U*U = UU* = I, when composite square matrix U is unitary, if concern, says top scientist (2021). Retrieved 15 April 2021, from https://​
conjugate transpose U* is inverse; Pauli Sum: Calculate and simplify the sum of scroll.​in/​latest/​992323/​covid-​19-​double-​mutant-​strain-​could-​be-​consi​
Pauli operators (I, X, Y, Z) according to Pauli algebra rules; Circuit: It represents dered-a-​varia​nt-​of-​conce​rn-​says-​top-​scien​tist
Quantum model, a sequence of quantum gates for performing a series of 8. Zhou S, Greenspan H, Davatzikos C, Duncan J, Van Ginneken B, Madab-
computations; Hinge Loss Formula: L(y) l(y) = max(0,1 − t*y); Qubit: A qubit hushi A, Prince J, Rueckert D. Summers R. A review of deep learning in
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 13 of 14

medical imaging: imaging traits, technology trends, case studieswith 33. Chen J, Wu L, Zhang J, Zhang L, Gong D, Zhao Y et al. Deep learning-
progress highlights, and future promises. 2020. based model for detecting 2019 novel coronavirus pneumonia on high-
9. Gao J, Yang Y, Lin P, Park DS. Computer vision in healthcare applications. J resolution computed tomography: a prospective study; 2020. https://​doi.​
Healthc Eng. 2018; org/​10.​1101/​2020.​02.​25.​20021​568
10. Porwal P, Pachade S, Kokare M, Deshmukh G, Son J, Bae W, et al. IDRiD: 34. Jin S, Wang B, Xu H, Luo C, Wei L, Zhao W et al. AI-assisted CT imaging
diabetic retinopathy—segmentation and grading challenge. Med Image analysis for COVID-19 screening: building and deploying a medical AI
Anal. 2020;59: 101561. https://​doi.​org/​10.​1016/j.​media.​2019.​101561. system in four weeks; 2020. https://​doi.​org/​10.​1101/​2020.​03.​19.​20039​354
11. Altaf F, Islam SMS, Akhtar N, Janjua NK. “Going deep in medical image 35. Sabottke C, Spieler B. The effect of image resolution on deep learning
analysis: Concepts, methods, challenges and future directions”. IEEE in radiography. Radiol Artif Intell. 2020;2(1):e190015. https://​doi.​org/​10.​
Access. 2019;7:99540–72. 1148/​ryai.​20191​90015
12. Acampora G. Quantum machine intelligence. Quantum Mach Intell. 36. Shi H, Han X, Jiang N, Cao Y, Alwalid O, Gu J, et al. Radiological findings
2019;1(1–2):1–3. https://​doi.​org/​10.​1007/​s42484-​019-​00006-5. from 81 patients with COVID-19 pneumonia in Wuhan, China: a descrip-
13. Kiani B, Villanyi A, Lloyd S. Quantum medical imaging algorithms; 2020. tive study. Lancet Infect Dis. 2020;20(4):425–34. https://​doi.​org/​10.​1016/​
arXiv.org. Retrieved 23 May 2020, from https://​arxiv.​org/​abs/​2004.​ s1473-​3099(20)​30086-4.
02036. 37. Li L, Qin L, Xu Z, Yin Y, Wang X, Kong B, et al. Artificial intelligence distin-
14. Mayr A, Klambauer G, Unterthiner T, Hochreiter S. Deeptox: toxicity guishes COVID-19 from community acquired pneumonia on chest CT.
prediction using deep learning. Front Environ Sci. 2016. https://​doi.​ Radiology. 2020. https://​doi.​org/​10.​1148/​radiol.​20202​00905.
org/​10.​3389/​fenvs.​2015.​00080. 38. Hani C, Trieu N, Saab I, Dangeard S, Bennani S, Chassagnon G, Revel M.
15. Orús R, Mugel S, Lizaso E. Quantum computing for finance: overview COVID-19 pneumonia: a review of typical CT findings and differential
and prospects. Rev Phys. 2019;4: 100028. https://​doi.​org/​10.​1016/j.​ diagnosis. Diagn Interv Imaging. 2020;101(5):263–8. https://​doi.​org/​10.​
revip.​2019.​100028. 1016/j.​diii.​2020.​03.​014.
16. Chen H, Engkvist O, Wang Y, Olivecrona M, Blaschke T. The rise of deep 39. Sreedhar K. Enhancement of images using morphological transforma-
learning in drug discovery. Drug Discovery Today. 2018;23(6):1241–50. tions. Int J Comput Sci Inf Technol. 2012;4(1):33–50. https://​doi.​org/​10.​
https://​doi.​org/​10.​1016/j.​drudis.​2018.​01.​039. 5121/​ijcsit.​2012.​4103.
17. Solenov D, Brieler J, Scherrer JF. The potential of quantum computing 40. Hu X, Wang H, Vegesana A, Dube S, Yu K, Kao G, et al. Crowdsourcing
and machine learning to advance clinical research and change the detection of sampling biases in image datasets. Proc Web Conf. 2020.
practice of medicine. Mo Med. 2018;115(5):463–7. https://​doi.​org/​10.​1145/​33664​23.​33800​63.
18. Rizwan I, Haque I, Neubert J. Deep learning approaches to biomedical 41. Wanyan T, Zhang J, Ding Y. Azad A, Wang Z, Glicksberg B. Bootstrapping
image segmentation. Inform Med Unlocked. 2020;18: 100297. https://​ your own positive sample: contrastive learning with electronic health
doi.​org/​10.​1016/j.​imu.​2020.​100297. record data (2021). [online] arXiv.org. <https://​arxiv.​org/​abs/​2104.​02932​
19. Dunjko V, Taylor J, Briegel H. Quantum-enhanced machine learning. v1> Accessed 7 July 2021.
Phys Rev Lett. 2016;117(13). https://​doi.​org/​10.​1103/​physr​evlett.​117.​ 42. Liang S, Sun R, Lee JD, Srikant R. Adding one neuron can eliminate all
130501 bad local minima. In: 32nd Conference on Neural Information Processing
20. Laumann T, Snyder A, Mitra A, Gordon E, Gratton C, Adeyemo B, Systems (NeurIPS 2018), Montréal, Canada; 2018
Gilmore A, Nelson S, Berg J, Greene D, McCarthy J, Tagliazucchi E, Laufs 43. Singh D, Kumar V, Vaishali et al. Classification of COVID-19 patients from
H, Schlaggar B, Dosenbach N, Petersen S. On the stability of BOLD fMRI chest CT images using multi-objective differential evolution–based con-
correlations. Cerebral Cortex. 2016. volutional neural networks. Eur J Clin Microbiol Infect Dis. 2020. https://​
21. Moore G. Cramming more components onto integrated circuits, doi.​org/​10.​1007/​s10096-​020-​03901-z
Reprinted from Electronics, volume 38, number 8, April 19, 1965, 44. Xu X, Jiang X, Ma C, Du P, Li X, Lv S et al. Deep learning system to screen
pp.114 ff. IEEE Solid-State Circuits Soc Newslett. 2006;11(3):33–35. coronavirus disease 2019 pneumonia; 2020. arXiv.org. Retrieved 22 May
22. Yanofsky N. An introduction to quantum computing. Proof Comput 2020, from https://​arxiv.​org/​abs/​2002.​09334​v1.
Agency. 2011. https://​doi.​org/​10.​1007/​978-​94-​007-​0080-2_​10. 45. Buades A, Coll B, Morel J. Non-Local means denoising. Image Process
23. Imtiyaz S. Quantum adiabatic evolution for global optimization in big Line. 2011;1:208–212.
data; 2020. arXiv.org. Retrieved 23 May 2020, from https://​arxiv.​org/​ 46. Song Y, Zheng S, Li L, Zhang X, Zhang X, Huang Z et al. Deep learning
abs/​1805.​11479. enables accurate diagnosis of novel coronavirus (COVID-19) with CT
24. Chowdhury AN, Somma RD. Quantum algorithms for Gibbs sampling images; 2020. https://​doi.​org/​10.​1101/​2020.​02.​23.​20026​930
and hitting-time estimation. Quantum Info Comput. 2017;17:1–2. 47. Shan F, Gao Y, Wang J, Shi W, Shi N, Han M et al. Lung infection quanti-
25. Moreira C, Wichert A. Quantum-like Bayesian networks for modeling fication of COVID-19 in CT images with deep learning; 2020. arXiv.org.
decision making. Front Psychol. 2016. https://​doi.​org/​10.​3389/​fpsyg.​ Retrieved 22 May 2020, from https://​arxiv.​org/​abs/​2003.​04655.
2016.​00011. 48. Barstugan M, Ozkaya U, Ozturk S. Coronavirus (COVID-19) classification
26. Lee YT, Sidford A, Wong SC-W. A faster cutting plane method and its using CT images by machine learning methods; 2020. arXiv.org. Retrieved
implications for combinatorial and convex optimization. In: Proceed- 22 May 2020, from https://​arxiv.​org/​abs/​2003.​09424​v1.
ings of 2015 IEEE 56th Annual Symposium on Foundations of Com- 49. Heinrich MP, Blendowski M, Oktay O. TernaryNet: faster deep model infer-
puter Science (FOCS), Berkeley, CA. New York: IEEE; 2015, p. 1049–1065 ence without GPUs for medical 3D segmentation using sparse and binary
27. Arora S, Kale S. A combinatorial, primal-dual approach to semidefinite convolutions. Int J CARS. 2018;13:1311–20. https://​doi.​org/​10.​1007/​
programs. In: Proceedings of of the 39th Annual ACM Symposium on s11548-​018-​1797-4.
Theory of Computing, San Diego, CA. New York: ACM; 2007, p. 227–236 50. Islam MM, Islam MZ, Asraf A, Ding W. Diagnosis of COVID-19 from X-rays
28. Georgescu I, Ashhab S, Nori F. Quantum simulation. Rev Modern Phys. using combined CNN-RNN architecture with transfer learning; 2020
2014;86(1):153–85. https://​doi.​org/​10.​1103/​revmo​dphys.​86.​153. 51. Zabirul Islam M, Islam MM, Asraf A. A combined deep CNN-LSTM network
29. de Lacy K, Noakes L, Twamley J, Wang J. Controlled quantum for the detection of novel coronavirus (COVID-19) using X-ray images.
search. Quantum Inf Process. 2018;17(10). https://​doi.​org/​10.​1007/​ Inform Med Unlocked 2020; 20:100412
s11128-​018-​2031-6 52. Singh D, Kumar V, Kaur M. Densely connected convolutional networks-
30. Sentís G, Monràs A, Muñoz-Tapia R, Calsamiglia J, Bagan E. Unsupervised based COVID-19 screening model. Appl Intell. 2021
classification of quantum data. Phys Revi X. 2019;9(4). 53. Kadampur M, Al Riyaee S. Skin cancer detection: applying a deep learning
31. Henderson M, Shakya S, Pradhan S, Cook T. Quanvolutional neural net- based model driven architecture in the cloud for classifying dermal cell
works: powering imagerecognition with quantum circuits. 2019. [online] images. Inform Med Unlocked. 2020;18: 100282. https://​doi.​org/​10.​1016/j.​
arXiv.org. Available at: [Accessed 21 July 2021]. imu.​2019.​100282.
32. Vabalas A, Gowen E, Poliakoff E, Casson A. Machine learning algorithm 54. Baranov M. Image processing of biological liquids films for medical diag-
validation with a limited sample size. PLoS ONE. 2019;14(11): e0224365. nostics. J Electron Sci Technol. 2020. https://​doi.​org/​10.​1016/j.​jnlest.​2020.​
https://​doi.​org/​10.​1371/​journ​al.​pone.​02243​65. 100027.
Sengupta and Srivastava BMC Med Inform Decis Mak (2021) 21:227 Page 14 of 14

55. Uesawa Y. Quantitative structure–activity relationship analysis using deep 59. Islam MM, Azmat Ullah SM, Mahmud S, Taslim Uddin Raju SM. Breathing
learning based on a novel molecular image input technique. Bioorg Med aid devices to support novel coronavirus (COVID-19) infected patients.
Chem Lett. 2018;28(20):3400–3. https://​doi.​org/​10.​1016/j.​bmcl.​2018.​08.​ SN Comput Sci. 2020;1(5):274.
032. 60. Islam MM, Karray F, Alhajj R, Zeng J. A review on deep learning tech-
56. Gilmer J, Schoenholz S, Riley P, Vinyals O, Dahl G. Neural Message Passing niques for the diagnosis of novel coronavirus (COVID-19). IEEE Access.
for Quantum Chemistry. arXiv.org. Retrieved 22 May 2020, from https://​ 2021;9:30551–72.
arxiv.​org/​abs/​1704.​01212.
57. Aliper A, Plis S, Artemov A, Ulloa A, Mamoshina P, Zhavoronkov A. Deep
learning applications for predicting pharmacological properties of Publisher’s Note
drugs and drug repurposing using transcriptomic data. Mol Pharm. Springer Nature remains neutral with regard to jurisdictional claims in pub-
2016;13(7):2524–30. https://​doi.​org/​10.​1021/​acs.​molph​armac​eut.​6b002​ lished maps and institutional affiliations.
48.
58. Merkwirth C, Lengauer T. Automatic generation of complementary
descriptors with molecular graph networks. J Chem Inform Model.
2005;45(5):1159–68. https://​doi.​org/​10.​1021/​ci049​613b.

Ready to submit your research ? Choose BMC and benefit from:

• fast, convenient online submission


• thorough peer review by experienced researchers in your field
• rapid publication on acceptance
• support for research data, including large and complex data types
• gold Open Access which fosters wider collaboration and increased citations
• maximum visibility for your research: over 100M website views per year

At BMC, research is always in progress.

Learn more biomedcentral.com/submissions

You might also like