Proteomics Clinical Apps - 2020 - Leite - Radiomics and Machine Learning in Oral Healthcare
Proteomics Clinical Apps - 2020 - Leite - Radiomics and Machine Learning in Oral Healthcare
Proteomics Clinical Apps - 2020 - Leite - Radiomics and Machine Learning in Oral Healthcare
www.clinical.proteomics-journal.com
1. Introduction
The increasing storage of information, data, and forms of knowledge has led
to the development of new technologies that can help to accomplish complex Technology is advancing rapidly, some-
times even surpassing scientific advance-
tasks in different areas, such as in dentistry. In this context, the role of
ments and validation. Nowadays new
computational methods, such as radiomics and Artificial Intelligence (AI) tools may allow to integrate multiunit in-
applications, has been progressing remarkably for dentomaxillofacial puts leading to a more efficient approach
radiology (DMFR). These tools bring new perspectives for diagnosis, to enable solving complex problems.[1]
classification, and prediction of oral diseases, treatment planning, and for the This is surely true in oral healthcare,
evaluation and prediction of outcomes, minimizing the possibilities of human where the increasing storage of infor-
mation, data, and forms of knowledge
errors. A comprehensive review of the state-of-the-art of using radiomics and
have led to the development of new tech-
machine learning (ML) for imaging in oral healthcare is presented in this nologies and methods of human inter-
paper. Although the number of published studies is still relatively low, the actions with machines. In this context,
preliminary results are very promising and in a near future, an augmented artificial intelligence (AI), machine learn-
dentomaxillofacial radiology (ADMFR) will combine the use of ing (ML), and deep learning (DL) have
actually become commonly used terms
radiomics-based and AI-based analyses with the radiologist’s evaluation. In
in several areas of modern daily life,
addition to the opportunities and possibilities, some challenges and such as in medicine and dentistry.[2–4]
limitations have also been discussed for further investigations. ML and its subclass DL are tech-
niques that enable computer systems
to improve with experience and data.
Where ML uses hand-designed features, DL achieves even
greater power by learning its features.[5] Radiomics relates to
Prof. A. F. Leite both, as it is the study that aims to extract quantitative features
Department of Dentistry from medical images for improved decision support.[6] Thus,
Faculty of Health Sciences these radiomic features can serve ML applications.
University of Brasília A comprehensive review of the state-of-the-art of using ra-
Brasília 70910-900, Brazil
E-mail: [email protected]; andrefl[email protected] diomics and ML for imaging in oral healthcare is presented in
Prof. A. F. Leite, Dr. K. de F. Vasconcelos, Prof. R. Jacobs this paper. Initially, the concepts of AI, ML, and DL are described.
Omfsimpath Research Group Second, an overview of how these techniques have permeated the
Department of Imaging and Pathology health sciences is given. Finally, the article highlights the promis-
Biomedical Sciences ing applications provided by radiomics and DL in dentomaxillo-
KU Leuven and Dentomaxillofacial Imaging Department
University Hospitals Leuven
facial radiology (DMFR). In addition, opportunities, challenges,
Leuven 3000, Belgium and limitations of these techniques are also discussed for oral
E-mail: [email protected] healthcare application.
H. Willems
Relu, Innovatie-en incubatiecentrum KU Leuven
Leuven 3000, Belgium
Prof. R. Jacobs 2. Concepts of AI, ML, and DL
Department of Dental Medicine
Karolinska Institutet Intelligence is defined as the ability to acquire and apply knowl-
Huddinge 17177, Sweden edge and skills. Artificial is an adjective referring to the fact that
it does not appear in nature. AI could hence be described as the
The ORCID identification number(s) for the author(s) of this article non-biological ability to accomplish complex tasks.[7] Examples
can be found under https://doi.org/10.1002/prca.201900040 of AI applications include image processing, chess-playing sys-
© 2020 The Authors. Proteomics – Clinical Applications published by tems, and processing of formal languages. More specifically, it is
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim. This is an open
access article under the terms of the Creative Commons
able to recognize patterns in an imaging exam to detect a disease.
Attribution-NonCommercial-NoDerivs License, which permits use and AI has a hierarchical relationship with ML and DL, as shown in
distribution in any medium, provided the original work is properly cited, Figure 1.
the use is non-commercial and no modifications or adaptations are ML is a technique that enables computer systems to learn
made. patterns from large datasets, opposed to AI where knowledge
DOI: 10.1002/prca.201900040 can also be explicitly programmed into the algorithm.[5–7] If ML
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (1 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
Figure 1. Diagram showing how DL is a kind of ML, which is in turn a kind Holger Willems is the CEO
of AI. Each section of this diagram contains the concept description and and co-founder of relu, an AI
some examples of AI technology. start-up in medical imaging
focussed on dental. He holds a
Master of Engineering: energy
(KU Leuven), He taught applied
linear algebra at the KU Leuven
and worked in the medical R&D
department at Materialise.
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (2 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
Figure 3. Example of a CNN used to predict dental disease based on information extracted from a panoramic radiograph.
Several types of deep neural networks exist. In practice, both re- cal, and environmental monitoring, advanced imaging, and be-
current neural networks (RNNs) and convolutional neural net- havioral tracking. Consequently, these transformations may im-
works (CNN) are used.[5,14] RNNs deal with sequential input data, prove risk assessment, diagnostic, and prognostic capabilities for
including speech and language. CNNs are specialized to deal with several diseases. In this context, AI tools may help to extract more
data with a grid-like topology, such as 2D and 3D images.[15] and better information from the patient in order to achieve accu-
While imitating neural connectivity patterns in the human vi- rate outcomes at lower health costs.[20–22]
sual cortex, the basic architecture of a CNN consists of a one or With the increasing availability of digital medical information
more convolutional layer, a pooling layer, and a fully connected in the form of electronic health records, and the fast development
layer.[16] The moment that training data enter the input layer, the of big data analytic methods, AI might assist health profession-
learning process begins. The data are then sequentially passed to als with difficult decisions in complex clinical situations.[9,23] In
neurons in the next layer until the output is reached. This process this way, the worldwide interest in AI applications is high and
is called forward propagation. This generated output is then com- growing rapidly.
pared with the ground truth and their difference is computed. The improvement of CNNs strengthens its use in healthcare.
These errors are back-propagated through the neural network, A previous study has demonstrated that a trained CNN was able
and the weights of the connections between the neurons are up- to differentiate abnormal from normal electroencephalographies
dated to minimize this error. After the training phase, including (EEGs). In this study, 7.671 EEGs were used for training the neu-
multiple iterations of forward and back propagation, the perfor- ral network, and 851 were used for its testing. The reference
mance of the CNN is assessed by an unseen test data set.[5,17] This test was established by two experts in EEG. Regarding the per-
process is illustrated in Figure 3. formance, the area under the curve was 0.924 for the developed
Although the artificial neural networks were introduced in model.[24]
the 1940s,[5] initially these were extremely limited in its ability Recent studies have demonstrated that CNNs may increase
to solve problems, mainly due to the overfitting problems with diagnostic accuracy or prediction of various diseases, such
training of deep architecture, lack of computing power, and pri- as cardiovascular diseases,[25,26] metabolic bone diseases,[27,28]
marily the insufficient data to train the computer system.[16] Re- neurological diseases,[29,30] ophthalmological diseases,[31] in-
cent substantial advances in computing power, improved data fectious NTHL1 diseases,[32,33] as well as several benign and
storage facilities, and new DL algorithms allow computers to malignant tumors.[34–36] One of the most promising applications
perform an increasing number of tasks that have historically of CNNs is analyzing radiological images. As radiomics inves-
not been possible.[18] Nowadays, there are multiple DL frame- tigates the extraction of features from radiological images, CNN
works available, the most promising being Tensorflow, PyTorch, can support and co-develop these techniques. A recent paper
Sonnet, Keras, and MXNet.[19] Among the various current appli- even demonstrates CNN outperforming radiomic analysis for
cations of AI, there is an increase in the use of AI tools in the multiparametric breast magnetic resonance imaging (MRI).[37]
health area, where CNN takes up an important role in the pro- Interaction of DL CNN and radiomics is the topic of next
cessing of medical images. section.
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (3 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (4 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
Table 1. Recent studies that discussed the possibilities and perspectives for applying AI tools in different fields of Dentistry.
Periodontics Classification and control of dental plaque;[47] segmentation of gingival diseases;[48] automated evaluation of periodontal pockets;[49] diagnosis
and prediction of periodontally compromised teeth[50] imaging study; detection of malodor.[51]
Orthodontics Automated identification of cephalometric landmarks[52] imaging study; prediction of growth and mandibular morphology in class I, II and III
patients[53] imaging study; analysis of biological markers for orthodontic tooth movement;[54] understanding of aetiopathogenesis of
craniofacial diseases;[54] automated identification of craniofacial syndromes;[54] prediction of treatment and outcomes models;[54] genetic risk
assessment of orofacial cleft;[55] identification of epigenetic changes in the normal and abnormal craniofacial development;[54] analysis of
mandibular condyles and temporomandibular joint disorder[56] imaging study.
Restorative dentistry Evaluation of lifespan of dental restorations;[3] improving accuracy of caries diagnosis[57] imaging study.
Endodontics Location of minor apical foramen in cadavers;[58] characterization of gene expression of radicular cyst and periapical granuloma;[59] detection of
vertical root fractures in intact and endodontically treated teeth[60] imaging study; evaluation of periapical lesions (bone repair) after
treatment[61] imaging study; assessment of root morphology on radiographs[62] imaging study.
Forensic application Automated determination of skeletal and dental ages[63,64] imaging studies; autopsy with AI-driven robotics methods[65]
Craniomaxillofacial Differentiation of different jaw tumors[66] imaging study; prediction of the occurrence of bisphosphonate-related osteonecrosis of the jaws
Surgery associated with dental extraction[17] imaging study; assessment of the impact of orthognathic treatment on facial attractiveness and estimated
age;[67] autonomous AI-driven robotics for performing surgeries and biopsies.[65,68]
virtual surgery planning, radiation therapy planning, and ra- in oral healthcare may allow rendering more patient-specific
diomic analysis.[5,18] diagnosis and personalized treatment planning.[20]
Another positive application of CNN consists of the possi- The most investigated applications of DL algorithms in DMFR
bility of optimizing the images, generating images of better were for the detection, classification, or diagnosis of diseases
quality, with low radiation dose and with little production of or anatomical structures, such as classification of teeth and
scattering or artifacts.[87] Such AI optimization algorithms seem mandibular morphology; differentiation of jaw tumors; and
promising and could be applied on different imaging modal- detection of root fractures, Sjögren’s syndrome, maxillary si-
ities, especially for correcting technical positioning errors on nusitis, calcified carotid atheroma’s, caries, and periodontal
panoramic radiographs or for enhancing quality on computed to- diseases.[50,52,53,57,60–62,64,66,77–85] However, the studies were hetero-
mographic images.[72–74] Toward the future, larger studies should geneous and mainly based on 2D images (intraoral or extraoral
be carried out to verifying the proposed optimization, which radiographs). Therefore, several possibilities still exist concern-
can be surely helpful to reduce radiation dose, scattering, and ing the use of AI tools in DMFR. On the other hand, some chal-
artifacts. lenges and limitations need to be addressed to optimize radiomic
The combination of radiomic and AI tools in dentistry are and ML techniques for dental care.[88] Opportunities and chal-
still limited to the evaluation of osteoporosis status on den- lenges for DMFR are addressed in the last two sections.
tal panoramic radiographs.[75,76] Some authors have evaluated
the diagnostic performance of three deep CNN-based computer-
assisted systems for detecting osteoporosis on 1268 dental
6. Augmented DMFR: A New Perspective
panoramic radiographs. In this study, the grey level of a region
of interest was the input to the networks. Experienced oral and As in medical radiology, the possibilities and perspectives of the
maxillofacial radiologists were considered as the reference test. use of radiomics-based and AI-based analyses combined with the
For the training and validation phases of the study, 1068 radio- radiologist’s analysis will revolutionize the DMFR,[89] as shown in
graphs were used. For the testing phase, 200 radiographs were Figure 5.
selected. Regarding the performance, the areas under the curve Augmented DMFR (ADMFR) will be a powerful auxiliary tool
values obtained were 0.976, 0.999, and 0.998 for the three differ- for achieving the precise and personalized oral healthcare. This
ent neural networks.[75] new approach will improve diagnostic capabilities, with better
Several radiomic features may be extracted from 2D or 3D accuracy and predictive models for several oral diseases. DL
images, based on size and shape; descriptors of the relationship may correlate radiomic features with other clinical and genetic
between image pixels or voxels, gray-level occurrence matrix, data. New imaging biomarkers will be defined. The predictive
run-length matrix; descriptors of histograms of image intensity; power of the imaging modalities will be expanded including out-
textures extracted from filtered images; and complex fractal come factors and prognosis associated with each kind of selected
features, among others. These imaging features may aid in the treatment. Automated segmentation of dental and craniofacial
early detection of malignant or many bone diseases. They can structures will also improve the evaluation of tissue and organs,
also be used to predict treatment response to therapies, including disease extent, and burden.[90,91]
oncotherapy, and to estimate functional parameters.[18–37,39–41] Concerning image acquisition, new protocols may be used or
The combination of these imaging features with other clinical created, as AI tools may optimize images, reducing the radiation
and genetic data may improve the capacity of detecting and dose, and correcting scatter and artifacts. The AI-driven manage-
predicting diagnosis and outcomes. Integrated data processing ment and mining of large imaging databases may also impact
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (5 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
Table 2. Brief summary of the most recent and relevant studies concerning AI applied in DMFR.
Application [Reference] Imaging modality Method Aim Remarks on dataset size Main conclusions
Year
Image segmen- [69] 2018 Panoramic CNN and 2D To test a novel method for Small (14 images) New method performed better
tation radiographs coupled shape automatic teeth than state-of-the-art
model segmentation methods.
[70] 2019 CBCT CNN and RNN To develop a fully-automated Small (50 images with high New method showed superior
image analysis for mandible variability) efficacy compared to
and anatomical landmark state-of-the art methods.
segmentation
[71] 2018 CT CNN To develop an automated Small (10 images) New method was promising
method of mandible and accurate.
segmentation
Image [72] 2018 Panoramic CNN To estimate and correct large (5166 images) CNN-based auto-positioning
optimization radiographs positioning errors of method for dental arch was
patients’ dental arch effective in providing
reconstructed panoramic
images of stable diagnostic
quality
[73] 2018 CT CNN To develop a neural network Moderate (52 images) CNN method yielded
for CT image high-resolution images
super-resolution based on low-resolution
images
[74] 2019 CBCT and micro-CT CNN To develop neural networks for Large (5680 cross-sectional New method showed better
images resolution enhancement slices of 13 teeth + test et of result than state-of-the-art
1824 slices of 4 teeth) reconstruction-based super-
resolution approaches in
terms of both quality metrics
and image-segmentation-
based-analysis.
Radiomics [75] 2018 Panoramic CNN To evaluate the performance of Large (1069 images for CNN showed high agreement
radiographs a CNN for detecting training and validation; 200 with experienced oral and
osteoporosis images in test database) maxillofacial radiologists in
detecting osteoporosis.
[76] 2018 Panoramic CNN To develop a deep Octuplet Large (55 000 images in 286 New method outperformed all
radiographs Siamese Network (OSN) to texture categories used for other state-of-the-art
learn and fuse training dataset; 56 images methods in osteoporosis
discriminative features for without and 52 images with categorization.
osteoporosis condition osteoporosis as test
prediction database)
Detection, [62] 2018 Panoramic CNN To evaluate the performance of Large (760 images from DL system showed high
classification radiographs a DL system for 400 patients) accuracy in differential
or diagnosis classification of the root diagnosis of single or extra
morphology of mandibular root in first molars.
first molars
[77] 2018 Dental periapical CNN To develop a neural network Small dataset (32 teeth) The method was effective in
radiographs for recognizing and recognizing and classifying
classifying teeth position the 32 teeth position
[78] 2019 Panoramic CNN To develop a neural network Large (1352 with test set The performance of the
radiographs for detecting and numbering comprised of 222 images) proposed computer-aided
teeth diagnosis solution was
comparable to the level of
experts.
[66] 2018 Panoramic CNN To differentiate jaw tumors Large (training on 400 images Sensitivity, specificity, accuracy,
radiographs (ameloblastoma and and the test set of 100 and diagnostic time using
keratocystic odontogenic images) CNN were comparable to
tumor) that of manual diagnosis by
oral maxillofacial specialists.
(Continued)
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (6 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
Table 2. Continued.
Application [Reference] Imaging modality Method Aim Remarks on dataset size Main conclusions
Year
[61] 2018 Dental periapical CNN To classify dental treatment Large (196 images) Automated approach to
radiographs qualities of periapical classify dental treatment
lesions yielded good accuracy
results that was comparable
to expert-level dentists.
[52] 2017 Lateral CNN To develop an end-to-end DL Large (300 images) Automated approach to
cephalometric system for cephalometric identify anatomical
radiographs landmark detection landmarks was successful
and comparable to dental
experts.
[64] 2017 CBCT CNN To develop an automated Moderate (52 images) By increasing the number of
method for classifying tooth training samples by rotation
types and intensity transformation
(data augmentation), a high
accuracy of 91.0% was
achieved.
[79] 2017 Contrast-enhanced CNN To evaluate the accuracy of DL Moderate (45 cases with 127 CNN yielded diagnostic
CT image classification for histologically proven accuracy similar to those of
diagnosis of lymph node positive cervical lymph the radiologists.
metastasis nodes and 314 histologically
proven negative lymph
nodes)
[80] 2018 Contrast-enhanced CNN To develop a neural network Large (training dataset The model could accurately
CT for detecting head and neck comprised of 2875 predict extra nodal extension
cancer nodal metastasis and CT-segmented lymph nodes. and nodal metastasis
extra nodal extension The test dataset of 131
images)
[81] 2019 Panoramic CNN To apply DL methodology to Moderate (65 images for Atheromas were detected with
radiographs detect calcified carotid training data) a sensitivity of 75%, a
atheroma specificity of 80%, and an
accuracy of 83%.
[82] 2018 Panoramic CNN To differentiate normal sinus Large (Training data consisting The diagnostic performance of
radiographs from inflamed, and to verify of 400 images with healthy the DL system for maxillary
the accuracy of the method sinuses and 400 with sinusitis was comparable to
for diagnosing maxillary inflamed sinuses. Testing dental experts and higher
sinusitis. dataset with 60 healthy and than dental residents.
60 inflamed sinuses images)
[60] 2017 Dental periapical CNN To design a neural network Large (Training data The designed neural network
radiographs and and test its efficacy to comprised 240 images) could be used as a proper
CBCT diagnose vertical root model for the diagnosis of
fractures inintact and vertical fractures, especially
endodontically treated teeth. on CBCT images.
[83] 2017 Dental periapical CNN To develop an artificial neural Large (200 images for training The CNN demonstrated high
radiographs (ex network for vertical root and testing) accuracy measurements for
vivo teeth) fracture detection being used as a model for
vertical root fracture
detection
[85] 2019 CT CNN To estimate the performance Large (400 images for training The performance of the DL
of a DL system for detection and 100 for testing) system was comparable to
of Sjögren’s syndrome the experienced radiologists,
but the unexperienced
radiologists showed lower
accuracy to detect the
disease.
(Continued)
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (7 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
Table 2. Continued.
Application [Reference] Imaging modality Method Aim Remarks on dataset size Main conclusions
Year
[57] 2018 Dental periapical CNN To evaluate the efficacy of CNN Large (2400 images for The diagnostic accuracies of
radiographs algorithms for detection and training, 600 images for premolar, molar, and both
diagnosis of dental caries testing) premolar and molar models
were 89.0% (80.4–93.3),
88.0% (79.2–93.1), and
82.0% (75.5–87.1),
respectively.
[50] 2018 Dental periapical CNN To develop a CNN system and Large (1392 images for The method was as effective as
radiographs to evaluate the usefulness training; 348 images for experienced periodontists
and accuracy for diagnosis testing) for positively diagnosing
and prediction of and predicting periodontally
periodontally compromised compromised teeth.
teeth
[84] 2017 Lateral CNN To develop an AI Moderate §96 images for The success rates of the
cephalometric decision-making model for training and 60 for testing) classifiers were 93% for the
radiographs the diagnosis of extractions diagnosis of extraction vs
for orthodontic purpose, nonextraction and 84% for
and also to evaluate the the detailed diagnosis of the
validity and accuracy of this extraction patterns in total.
model.
[53] 2017 Lateral CNN and To predict the mandibular Training data comprised 229 CNN showed a high
cephalometric Support morphology through images predictability ability of the
radiographs Vector craniomaxillary variables in mandibular morphology in
Machine patients with skeletal class I, the three skeletal
II and III classifications.
AI, artificial intelligence; CBCT, cone beam computed tomography; CNN, convolutional neural network; CT, computed tomography; DL, deep learning; RNN, recurrent neural
network.
Figure 5. ADMFR: new perspectives and possibilities when combining radiologist’s analyses with AI and radiomics tools.
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (8 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
the daily workflow by pre-analyzing and prioritizing cases. This tween 10 and 5166 images (mean 1053 images). This heterogene-
technique can also facilitate the radiologist’s reports, enabling ity is mainly related to different tasks and target accuracy.[86] For
the association of words, images, quantitative parameters, and overcoming this limitation, many studies that used small sample
minimizing errors. Therefore, ADMFR will add value improving sizes have used some strategies, such as data augmentation and
diagnosis, surgery, and treatment planning, predicting diseases transfer learning.
and outcomes, and consequently creating an accurate robust Data augmentation strategy is generally used for the training
clinical decision support process. Furthermore, radiologists may data to reduce the risk of overfitting. By generating different im-
experience a more efficient workflow with automated sugges- ages from the original images, such as cropped images with dif-
tions for complex case processing and prediction of surgical ferent locations, noise-added images, mirrored images, among
complications. others, this technique aims to increase the size of the dataset.[15]
As these DL algorithms have been modifying radiology prac- Furthermore, transfer learning is also a widely applied technique
tice, the process of learning in radiology will be similarly on trained networks over smaller datasets. Knowledge gained by a
affected.[92] The way that our students and faculty members ac- pretrained network in one large dataset (generally using nonmed-
cess and interact with information and with the physical world ical images) is transferred to another dataset with a completely
will continue to change and evolve. Students and practitioners different kind of data (such as medical images). The pre-trained
will inhabit a future in which AI and virtual reality might steer network may be used as a feature extractor, or by fine-tuning the
personalized educational experiences.[93] last fully connected layers. They let the model learn some basic
Many e-learning tools have increasingly been used in den- low level features of the image from a large dataset, but use the
tal curricula to support traditional teaching, such as computer- high level features to be more specific to the training data.[98]
assisted learning, web-based learning, and mobile-enhanced DL studies often require a large amount of data because
learning. However, the number of studies that have used e- the features are learned directly from the data via an end-to-
learning methods in DMFR is still scarce.[94] In fact, this use of end process.[92] Furthermore, during training for diagnostic AI,
technology is particularly relevant in radiology, where visual in- datasets should also be carefully composed and adequately la-
formation, central to its practice, and also the complexity and the beled by experts to render the prediction robust and accurate.[21]
large volume of data, can be presented and reinforced in innova- Further studies should be conducted on large datasets in order
tive ways. to verify whether CNNs could assist in detection and predic-
More recently, some adaptive learning platforms (ALPs) have tion of early-stage of dental and osseous changes that are almost
been developed for medical education.[95,96] Based on DL algo- invisible to the human eye, and process the data quickly and
rithms, these platforms are able to analyze students’ performance accurately.
in real time and suggest specific content (videos, games, texts, One should also realize that high variability of imaging proto-
etc.) for the needs of each one. Behind this virtual learning en- cols may impede comparisons among different studies, and the
vironment, with contents grouped and delivered in a variety of development of multi-institutional large databases. Another in-
ways, there is an algorithm capable of suggesting a student the trinsic challenge is related to the establishment of the ground
way he tends to learn better. Therefore, these platforms may help truth (gold standard) for validating classifications or predictive
teachers, managers, and educational networks to give more au- models resulted from AI studies.[21] As this ground truth serves
tonomy to students and to customize the learning process.[97] for training and validating the quality of the AI, it should be care-
fully considered by experts and this might require validation it-
self. A further issue that might need to advance in the future is
7. Challenges and Limitations of ADMFR the computational speed; which needs to be clinically realistic to
enable its use in clinical practice.
Although the results of previous AI research have been extremely To overcome these challenges and limitations, it is encouraged
promising, the studies are still preliminary. Some general and in- that radiologists and AI experts work together, as such to result
trinsic challenges exist. In general, the mathematical processes in robust and clinically applicable use of radiomics and AI in
underlying AI hinder the understanding of outcomes by radi- Dentistry.
ologists, i.e., it is hard to explain, which affects their opinion
about the use of these computer-aided techniques in daily prac-
tice. However, the radiologist’s work is prone to subjectivity, vari- 8. Conclusions
ations across observers, and the adverse effect of fatigue. Hence,
mathematical processes should not be considered as a threat for DL, the most promising and active field of AI, improves with a
the DMFR specialty and dental practice. On the other hand, it steady pace. This technique is exceptionally versatile to perform
is an opportunity for improvement, as radiomics and ML tech- tasks as it improves with experience and data. Therefore, oppor-
niques might add value to the radiologist’s current practice surely tunities arise in the whole medical field. Research in dentistry
in case of integrated processing of multiple imaging and clinical remains preliminary, but shows great promise.
features fitting a specific diagnostic task.[41] AI and radiomics could support specialists in decision-
Moreover, some intrinsic challenges exist to increase the usage making, by extracting in a consistent way the most relevant fea-
of AI in DMFR, similar to medical imaging.[21] The main chal- tures from vast amounts of data. Another use case for DL is to
lenge is the availability of big data concerning dental images. An- make recommendations to specialists, to both improve the ac-
other challenge is to calculate a minimum sample size to perform curacy of a correct diagnosis and treatment, and its speed. AI
a specific DL-task. In a recent scoping review, datasets varied be- could improve each single stage of dental practice and hence its
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (9 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
outcomes. This would unlock personalized Dentistry, enabling [20] H., Slavkin, J. Public Health Dent. 2018 in press, https://doi.org/10.
the best possible outcome for each individual. 1111/jphd.12296.
To reach this goal, more interdisciplinary research is needed [21] J. Thrall, X. Li, Q. Li, C. Cruz, S. Do, K. Dreyer, J. Brink, J. Am. Coll.
to develop AI tools that are explainable and robust. Once these Radiol. 2018,15, 504.
[22] T. Litman, APMIS 2019, 127, 386.
challenges are addressed, human–machine interaction could
[23] E. Shortliffe, M. Sepúlveda, JAMA 2018, 320, 2199
become mainstream in dental practice, challenging on its turn
[24] K. van Leeuwen, H. Sun, M. Tabaeizadeh, A. Struck, M. van Putten,
the way we should educate the current generation of students. M. Westover, Clin. Neurophysiol. 2018, 130, 77.
[25] Y. Kim, J. Lee, I. Yu, H. Song, I. Baek, J. Seong, H. Jeong, B. Kim, H.
Nam, J. Chung, O. Bang, G. Kim, W. Seo, Stroke 2019, 50, 1444.
Acknowledgements [26] F. PaN, P. He, F. Chen, J. Zhang, H. Wang, D. Zheng, Int. J. Med.
Inform. 2019, 128, 71.
The authors are grateful to the Fundação de Apoio à Pesquisa
[27] C. Kruse, Curr. Osteoporosis Rep. 2018, 16, 320.
do Distrito Federal (FAP-DF) for the financial support (protocol #
[28] U. Ferizi, S. Honig, G. Chang, Curr. Opin. Rheumatol. 2019, 31, 368.
23106.013588/2019-05) for this study.
[29] N. T. Duc, S. Ryu, M. N. I Qureshi, M. Choi, K. H Lee, B. Lee, Neu-
roinformatics. 2019 https://doi.org/10.1007/s12021-019-09419-w
[30] Z. Tang, K. V. Chuang, C. DeCarli, L. W. Jin, L. Beckett, M. J. Keiser, B.
Conflict of Interest N. Dugger, Nat. Commun. 2019, 10, 2173.
[31] Y. Fujinami-Yokokawa, N. Pontikos, L. Yang, K. Tsunoda, T. Yoshitake,
The authors declare no conflict of interest.
T. Iwata, H. Miyata, K. Fujinami, J. Ophthalmol. 2019, 2019, 1691064.
[32] N. Lüneburg, N. Reiss, C. Feldmann, P. van der Meulen, M. van de
Steeg, T. Schmidt, R. Wendl, S. Jansen, Stud. Health Technol. Inform.
Keywords 2019, 260, 192.
[33] A. Khalighifar, E. Komp, J. Ramsey, R. Gurgel-Gonçalves, A. Peterson,
CBCT, machine learning, oral health, radiomics
J. Med. Entomol. 2019, 56,1404. in press.
[34] D. Ardila, A. Kiraly, C. Bharadwaj, B. Choi, J. Reicher, L. Peng, D. Tse,
Received: July 10, 2019
M. Etemadi, W. Ye, G. Corrado, D. Naidich, S. Shetty, Nat. Med. 2019,
Revised: December 9, 2019
25, 954.
Published online: January 29, 2020
[35] A. Dascalu, E. David, E. Bio. Medicine 2019, 43, P107.
[36] C. Wang, C. Hamm, L. Savic, M. Ferrante, I. Schobert, T. Schlachter,
M. Lin, J. Weinreb, J. Duncan, J. Chapiro, B. Letzen, Eur. Radiol. 2019,
29, 3348.
[37] D. Truhn, S. Schrading, C. Haarburger, H. Schneider, D. Merhof,
[1] H. S. Slavkin, J. Dent. Educ. 2012, 76, 28. C. Kuhl, Radiology. 2019, 290, 290.
[2] J. P. Beregi, M. Zins, J. P. Masson, P. Cart, J. M. Bartoli, B. Silberman, [38] C. Liew, Eur. J. Radiol. 2018, 102, 152.
F. Boudghene, J. F. Meder, Diagn. Interv. Imaging. 2018, 99, 677. [39] P. Lambin, R. Leijenaar, T. Deist, J. Peerlings, E. de Jong, J. van Tim-
[3] W. J. Park, J. B. Park, Eur. J. Dent. 2018, 12, 594. meren, S. Sanduleanu, R. Larue, A. Even, A. Jochems, Y. van Wijk,
[4] F. Pesapane, M. Codari, F. Sardanelli, Eur. Radiol. Exp. 2018, 2, 35. H. Woodruff, J. van Soest, T. Lustberg, E. Roelofs, W. van Elmpt, A.
[5] I. Goodfellow, Y. Bengio, A., Courville, Deep Learning, MIT Press, Dekker, F. Mottaghy, J. Wildberger, S. Walsh, Nat. Rev. Clin. Oncol.
Cambridge, MA 2016. 2017, 14, 749.
[6] European Society of Radiology, Insights Imaging. 2019, 10, 44. [40] M. Avanzo, J. Stancanello, I. Naqa, Physica Medica 2017, 38, 122.
[7] M. Tegmark, Life 3.0: Being Human in the Age of Artificial Intelligence, [41] F. Lucia, D. Visvikis, M. Desseroit, O. Miranda, J. Malhaire, P. Robin,
Vintage, New York, NY 2017. O. Pradier, M. Hatt, U. Schick, Eur. J. Nucl. Med. Mol. Imaging 2018,
[8] B. Erickson, P. Korfiatis, Z. Akkus, T. Kline, RadioGraphics 2017, 37, 45, 768.
505. [42] S. Jha, E. Topol, JAMA 2016, 316, 2353.
[9] F. Jiang, Y. Jiang, H. Zhi, Y. Dong, H. Li, S. Ma, Y. Wang, Q. Dong, H. [43] D. Shen, G. Wu, H. Suk, Annu. Rev. Biomed. Eng. 2017, 19, 221.
Shen, Y. Wang, Stroke. Vasc. Neurol. 2017, 2, 230. [44] M. Chen, R. Ball, L. Yang, N. Moradzadeh, B. Chapman, D. Larson,
[10] D. Hashimoto, G. Rosman, D. Rus, O. Meireles, Ann. Surg. 2018, 268, C. Langlotz, T. Amrhein, M. Lungren, Radiology 2018, 286, 845.
70. [45] P. Lakhani, D. Gray, C. Pett, P. Nagy, G. Shih, J. Digit. Imaging. 2018,
[11] G. Chartrand, P. M. Cheng, E. Vorontsov, M. Drozdzal, S. Turcotte, C. 31, 283.
J. Pal, S. Kadoury, A. Tang, RadioGraphics 2017, 37, 2113. [46] W. Zhu, Y. Huang, L. Zeng, X. Chen, Y. Liu, Z. Qian, N. Du, W. Fan, X.
[12] L. Jones, D. Golan, S. Hanna, M. Ramachandran, Bone. Joint. Res. Xie, Med. Phys. 2018. https://doi.org/10.1002/mp.13300.
2018, 7, 223. [47] S. Imangaliyev, V. van der Veen, C. Volgenant, B. Loos, B. Keijser, W.
[13] M. Mupparapu, C. Wu, Y. Chen, Quintessence. Int. 2018, 49, 687. Crielaard, arXiv 2017, 1705.09193.
[14] S. Anwar, M. Majid, A. Qayyum, M. Awais, M. Alnowami, M. Khan, J. [48] A. Rana, G. Yauney, L. Wong, O. Gupta, A. Muftu, P. Shah, presented
Med Syst. 2018, 42, 226. at the IEEE-NIH Special Topics Conference on Healthcare Innova-
[15] K. Yasaka, H. Akai, A. Kunimatsu, S. Kiryu, O. Abe, Jpn. J. Radiol. 2018, tions and Point-of-Care Technologies. Bethesda, 2017.
36, 257. [49] M. Reddy, S. Shetty, R. Shetty, V. Vannala, Sk Shakeel, J. Investig. Clin.
[16] J. Lee, S. Jun, Y. Cho, H. Lee, G. Kim, J. Seo, N. Kim, Korean. J. Radiol. Dent. 2019, 10, e12423.
2017, 18, 570. [50] J. Lee, D. Kim, S. Jeong, S. Choi, J. Periodontal. Implant. Sci. 2018, 48,
[17] D. Kim, H. Kim, W. Nam, H. Kim, I. Cha, Bone 2018, 116, 207. 114.
[18] M. McBee, O. Awan, A. Colucci, C. Ghobadi, N. Kadom, A. Kansagra, [51] M. Nakhleh, M. Quatredeniers, H. Haick, Oral Diseases 2018, 24,
S. Tridandapani, W. Auffermann, Acad. Radiol. 2018, 25, 1472. 685.
[19] B. Erickson, P. Korfiatis, Z. Akkus, T. Kline, K. Philbrick, J. Digit. Imag- [52] H. Lee, M. Park, J. Kim, Proc. SPIE 10134, Med. Imaging, 2017,
ing. 2017, 30, 400. https://doi.org/10.1117/12.2255870.
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (10 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim
18628354, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/prca.201900040 by Harisankar Binod , Wiley Online Library on [12/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
www.advancedsciencenews.com www.clinical.proteomics-journal.com
[53] T. C. Niño-Sandoval, S. V. Guevara Pérez, F. A. González, R. A. Jaque, [75] J. S. Lee, S. Adhikari, L. Liu, H. G. Jeong, H. Kim, S. J. Yoon, Den-
Forensic. Sci. Int. 2017, 281, 187.e1.e1. tomaxillofac. Radiol. 2018, 20170344. https://doi.org10.1259/dmfr.
[54] V. Allareddy, S. Rengasamy Venugopalan, R. P. Nalliah, J. L. Caplin, 20170344
M. K. Lee, V. Allareddy, Orthod. Craniofac. Res. 2019, 22, 8. [76] P. Chu, C. Bo, X. Liang, J. Yang, V. Megalooikonomou, F. Yang, B.
[55] S. J. Zhang, P. Meng, J. Zhang, P. Jia, J. Lin, X. Wang, F. Chen, X. Wei, Huang, X. Li, H. Ling, Conf. Proc. IEEE. Eng. Med. Biol. Soc. 2018.
Genomics, Proteomics Bioinf. 2018, 16, 354. [77] K. Zhang, J. Wu, H. Chen, P. Lyu, Comput. Med. Imaging. Graph. 2018,
[56] J. Bianchi, J. R. Gonçalves, A. C. O. Ruellas, J. B. Vimort, M. Yatabe, https://doi.org/10.1016/j.compmedimag. 2018.07.001.
B. Paniagua, P. Hernandez, E. Benavides, F. N. Soki, L. H. S. Cevi- [78] D. V. Tuzoff, L. N. Tuzova, M. M. Bornstein, A. S. Krasnov, M. A.
danes, Dentomaxillofac. Radiol. 2019, 48, 20190049. [Epub ahead of Kharchenko, S. I. Nikolenko, M. M. Sveshnikov, G. B. Bednenko, Den-
print]. tomaxillofac. Radiol. 2019, 48, 20180051.
[57] J. H. Lee, D. H. Kim, S. N. Jeong, S. H. Choi, Journal of Dentistry 2018, [79] Y. Ariji, M. Fukuda, Y. Kise, M. Nozawa, Y. Yanashita, H. Fujita, A.
77, 106. Katsumata, E. Ariji, Oral. Surg. Oral. Med. Oral. Pathol. Oral. Radiol.
[58] M. A. Saghiri, F. Garcia-Godoy, J. L. Gutmann, M. Lotfi, K. Asgar, J. 2019, 127, 458.
Endod. 2012, 38, 1130. [80] B. H. Kann, S. Aneja, G. V. Loganadane, J. R. Kelly, S. M. Smith, R.
[59] O. Poswar Fde, L. C. Farias, C. A. Fraga, W. Jr Bambirra, M. Brito- H. Decker, J. B. Yu, H. S. Park, W. G. Yarbrough, A. Malhotra, B. A.
Júnior, M. D. Sousa-Neto, S. H. Santos, A. M. de Paula, M. F. Burtness, Z. A. Husain, Sci. Rep. 2018, 8, 14036.
D’Angelo, A. L. Guimarães, J. Endod. 2015, 41, 877. [81] L. Kats, M. Vered, A. Zlotogorski-Hurvitz, I. Harpaz, Int. J. Comput.
[60] M. Johari, F. Esmaeili, A. Andalib, S. Garjani, H. Saberkari, Dentomax- Dent. 2019, 22, 163.
illofacial Radiol. 2017,46, 20160107. [82] M. Murata, Y. Ariji, Y. Ohashi, T. Kawai, M. Fukuda, T. Funakoshi,
[61] J. Yang, Y. Xie, L. Liu, B. Xia, Z. Cao, C. Guo, IEEE 42nd Annual Com- Y. Kise, M. Nozawa, A. Katsumata, H. Fujita, E. Ariji, Oral Radiology
puter Software and Applications Conference (COMPSAC), Tokyo, 2019, 35, 301.
2018. [83] S. Kositbowornchai, S. Plermkamon, T. Tangkosol, Dental Traumatol-
[62] T. Hiraiwa, Y. Ariji, M. Fukuda, Y. Kise, K. Nakata, A. Katsumata, H. ogy 2013, 29, 151.
Fujita, E. Ariji, Dentomaxillofacal Radiol. 2019, 48, 20180218. [84] S. K. Jung, T. W. Kim, Am. J. Orthod. Dentofacial. Orthop. 2016, 149,
[63] J. De Tobel, P. Radesh, D. Vandermeulen, P. W. Thevissen, J. Forensic 127.
Odontostomatol. 2017, 2, 42. [85] Y. Kise, H. Ikeda, T. Fujii, M. Fukuda, Y. Ariji, H. Fujita, A. Katsumata,
[64] Y. Miki, C. Muramatsu, T. Hayashi, X. Zhou, T. Hara, A. Katsumata, E. Ariji, Dentomaxillofac. Radiol. 2019, 48, 20190019.
H. Fujita, Comput. Biol. Med. 2017, 80, 24. [86] F. Schwendicke, T. Golla, M. Dreher, J. Krois, J. Dent. 2019, 5, 103226.
[65] S. O’Sullivan, S. Leonard, A. Holzinger, C. Allen, F. Battaglia, N. Neve- [87] Y. Jiang, C. Yang, P. Yang, X. Hu, C. Luo, Y. Xue, L. Xu, X. Hu, L. Zhang,
jans, F. W. B. van Leeuwen, M. I. Sajid, M. Friebe, H. Ashrafian, H. J. Wang, K. Sheng, T. Niu, Phys. Med. Biol. 2019, 64, 145003. [Epub
Heinsen, D. Wichmann, M. Hartnett, Int. J. Med. Robot 2019, e2020. ahead of print].
https://doi.org/10.1002/rcs.2020. [88] J. J. Hwang, Y. H. Jung, B. H. Cho, M. S. Heo, Imaging. Sci. Dent. 2019,
[66] W. Poedjiastoeti, S. Suebnukarn, Health Inform. Res. 2018, 24, 49, 1.
236. [89] V. S. Parekh, M. A. Jacobs, Expert Review of Precision Medicine and Drug
[67] R. Patcas, D. A. J. Bernini, A. Volokitin, E. Agustsson, R. Rothe, R. Development 2018, 4, 59.
Timofte, Int. J. Oral. Maxillofac. Surg. 2019, 48, 77. [90] D. W. Kim, S. Lee, S. Kwon, W. Nam, I. H. Cha, H. J. Kim, Sci. Rep.
[68] S. P. Tarassoli, Ann. Med. Surg. 2019, 41, 53. 2019, 9, 6994.
[69] A. Wirtz, S. G. Mirashi, S. Wesarg, Proceedings of the 21st Interna- [91] R. B. Parikh, A. Gdowski, D. A. Patt, A. Hertler, C. Mermel, J. E. Bekel-
tional Conference on Medical Image Computing and Computer As- man, Am. Soc. Clin. Oncol. Educ. Book. 2019, 39, e53.
sisted Intervention, Part IV; Granada, Spain. 2018. [92] S. H. Tajmir, T. K. Alkasab, Academic Radiology 2018, 25, 747.
[70] N. Torosdagli, D. K. Liberton, P. Verma, M. Sincan, J. S. Lee, U. Bagci, [93] N. Y. Karimbux, J. Dent. Educ. 2018, 82, 439.
IEEE Trans. Med. Imaging 2018, 38, 919. [94] G. N. Santos, A. F. Leite, P. T. Figueiredo, N. M. Pimentel, C. Flores-
[71] J. Egger, B. Pfarrkirchner, C. Gsaxner, L. Lindner, D. Schmalstieg, J. Mir, N. S. de Melo, E. N. Guerra, G. De Luca Canto, J. Dent. Educ.
Wallner, Conf. Proc. IEEE. Eng. Med. Biol. Soc. 2018, 656. 2016, 80, 1126.
[72] X. Du, Y. Chen, J. Zhao, Y. Xi, Conf. Proc. IEEE. Eng. Med. Biol. Soc. [95] T. Taveira-Gomes, A. Saffarzadeh, M. Severo, M. J. Guimarães, M. A.
2018, 2615. Ferreira, BMC Medical Education 2014, 14, 143.
[73] J. Park, D. Hwang, K. Y. Kim, S. K. Kang, Y. K. Kim, J. S. Lee, Phys. Med. [96] A. Menon, S. Gaglani, M. R. Haynes, S. Tackett, Med. Teach. 2017, 39,
Biol. 2018, 63, 145011. 975.
[74] J. Hatvani, A. Horváth, J. Michetti, A. Basarab, D. Kouamé, M. [97] P. Brusilovsky, C. Peylo, Int. J. Artif. Intell. Educ. 2003, 13, 156.
Gyöngy, IEEE Transactions on Radiation and Plasma Medical Sciences [98] M. Maqsood, F. Nazir, U. Khan, F. Aadil, H. Jamal, I. Mehmood, O.
2019, 3, 120. Song, Sensors. 2019, 19, 2645.
Proteomics Clin. Appl. 2020, 14, 1900040 1900040 (11 of 11) © 2020 The Authors. Proteomics – Clinical Applications published by
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim