Vinayahalingam et al.

BMC Oral Health (2023) 23:643 BMC Oral Health

https://doi.org/10.1186/s12903-023-03362-8

RESEARCH Open Access

Intra‑oral scan segmentation using deep

learning
Shankeeth Vinayahalingam1,2,3, Steven Kempers1,2, Julian Schoep4, Tzu‑Ming Harry Hsu5, David Anssari Moin4,
Bram van Ginneken6, Tabea Flügge7*, Marcel Hanisch4,3† and Tong Xi1†

Abstract
Objective Intra-oral scans and gypsum cast scans (OS) are widely used in orthodontics, prosthetics, implantol‑
ogy, and orthognathic surgery to plan patient-specific treatments, which require teeth segmentations with high
accuracy and resolution. Manual teeth segmentation, the gold standard up until now, is time-consuming, tedious,
and observer-dependent. This study aims to develop an automated teeth segmentation and labeling system using
deep learning.
Material and methods As a reference, 1750 OS were manually segmented and labeled. A deep-learning approach
based on PointCNN and 3D U-net in combination with a rule-based heuristic algorithm and a combinatorial search
algorithm was trained and validated on 1400 OS. Subsequently, the trained algorithm was applied to a test set con‑
sisting of 350 OS. The intersection over union (IoU), as a measure of accuracy, was calculated to quantify the degree
of similarity between the annotated ground truth and the model predictions.
Results The model achieved accurate teeth segmentations with a mean IoU score of 0.915. The FDI labels
of the teeth were predicted with a mean accuracy of 0.894. The optical inspection showed excellent position agree‑
ments between the automatically and manually segmented teeth components. Minor flaws were mostly seen
at the edges.
Conclusion The proposed method forms a promising foundation for time-effective and observer-independent teeth
segmentation and labeling on intra-oral scans.
Clinical significance Deep learning may assist clinicians in virtual treatment planning in orthodontics, prosthetics,
implantology, and orthognathic surgery. The impact of using such models in clinical practice should be explored.
Keywords Deep learning, Artificial intelligence, Intra-oral scan, Computer-assisted planning, Digital imaging

† 6
Marcel Hanisch and Tong Xi contributed equally to this work. Department of Radiology, Radboud University Nijmegen Medical
Centre, Nijmegen, the Netherlands
*Correspondence: 7
Charité – Universitätsmedizin Berlin, corporate member of Freie
Tabea Flügge Universität Berlin and Humboldt-Universität Zu Berlin, Department of Oral
[email protected]
1 and Maxillofacial Surgery, Hindenburgdamm 30, 12203 Berlin, Germany
Department of Oral and Maxillofacial Surgery, Radboud University
Nijmegen Medical Centre, Nijmegen, the Netherlands
2
Department of Artificial Intelligence, Radboud University, Nijmegen, the
Netherlands
3
Department of Oral and Maxillofacial Surgery, Universitätsklinikum
Münster, Münster, Germany
4
Promaton Co. Ltd, 1076 GR Amsterdam, The Netherlands
5
MIT Computer Science & Artificial Intelligence Laboratory, 32 Vassar St,
Cambridge, MA 02139, USA

© The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecom‑
mons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 2 of 9

Introduction Material and methods

In recent years, the development of digital dentistry Data
has revolutionized the dental field [1]. 3D virtual treat- In the present study 1750 3D scans (875 maxilla, 875
ment planning and subsequent computer-aided design/ mandible) from 875 patients were randomly collected
computer-aided manufacturing of occlusal splints, from different clinics in the Netherlands. The accu-
surgical guides, and prothesis are increasingly being mulated 3D scans (intra-oral scan and gypsum casts
implemented in the clinical workflow [2–4]. One com- scan) were acquired with 3Shape Trios Move, 3Shape
monly used imaging technique within the scope of vir- D500 (3shape, Copenhagen, Denmark), DW 3Series + ,
tual treatment planning is the intra-oral scan, which DW 7Series, DW 3Series, and DW 5Series (Dental
provides a 3D mesh of the dentition [1]. Wings, Montreal, Canada). This study was conducted
These 3D meshes (OS) are widely used in orthodon- in accordance with the code of ethics of the World
tics, prosthetics, implantology, and orthognathic sur- Medical Association (Declaration of Helsinki) and the
gery to plan patient-specific treatments, which require ICH-GCP. The Institutional Review Board, Commissie
teeth segmentations with high accuracy and resolution Mensgebonden Onderzoek Radboudumc, Nijmegen,
[3]. Teeth segmentations aim to separate and classify The Netherlands approved the study and granted the
the 3D mesh of the dental arch into different teeth fol- approval that informed consent was not required as all
lowing the FDI standard so that each individual tooth image data were anonymized and de-identified before
position can be rearranged and realigned accordingly. analysis (decision no. 2021–13253).
Manual teeth segmentation, the gold standard up
until now, is time-consuming, tedious, and observer- Data annotation
dependent [5]. To be able to implement digital models The OS were mesh-wise annotated (teeth and gingiva)
as a clinical standard, fully-automated segmentation of by different clinicians independently and in duplicate
teeth with high accuracy is required [6]. This remains using the brush mode in Meshmixer (Autodesk, San
challenging due to the positional variations, shape Rafael, United States). Each triangle surface could only
alterations, size abnormalities, and differences in the belong to one of the two classes. All segmented and
number of teeth between individuals [6]. labeled OS were subsequently reviewed and revised
Recently, artificial intelligence (AI) and more specifi- by two different clinicians (MH, DM). Each of the cli-
cally deep learning (e.g. convolutional neural network nicians and reviewers was instructed and calibrated
(CNN)) has shown superior segmentation performance in the segmentation task using a standardized proto-
compared to geometry-based approaches, mainly due col before the annotation and reviewing process. The
to task-oriented extraction and fusion of local details definitive dataset was constructed from all annotated
and semantic information [7]. meshes.
In dentistry, CNNs have been successfully applied to The training boxes were calculated based on the
detect carious lesions [8], periodontal lesions [9], cysts mesh-wise annotation. For each tooth in the OS, the
[10], and tumors [11] and even surpassed the detection training box is determined by computing the minimum
performance of experienced clinicians in certain condi- 3D bounding box around the tooth’s points.
tions [12]. Further deep learning based applications are
the difficulty assessment of endodontic treatment [13], The model
prediction of extraction difficulty for mandibular third The OS detection, segmentation, and labeling process
molars [14], skeletal classification [15], soft tissue pre- included three parts: the detection module, the seg-
diction [16], and root morphology evaluation [17]. mentation module, and the labeling algorithm (Fig. 1).
The capability of CNNs to automatically segment
teeth on OS(s) were explored in different studies [6, The detection module
18–23]. However, these CNNs are black boxes and The detection module was comprised of two different
lack interpretability [24]. Clinicians and patients dem- CNNs: 1). PointCNN [27] and 2). 3D-Unet [28].
onstrate reticence in confiding and adopting AI sys- PointCNN is an architecture tailored for point cloud
tems, which are not transparent, understandable, and processing tasks, operating on unordered point sets.
explainable [25, 26]. For this reason, this study aimed This architecture incorporates a learnable permutation
to develop an explainable detection, segmentation, and invariant operation that efficiently gathers and aggre-
FDI labeling system using deep learning as a funda- gates local features from neighboring points, facili-
mental basis for improved and more automated treat- tating effective feature learning while preserving the
ment planning in dentistry. inherent structure of the point cloud. The 3D-Unet is
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 3 of 9

Fig. 1 The workflow of detection, segmentation and labeling process

a modified version of the U-net architecture. It consists The labeling algorithm

of an encoder, which down-samples the input volume The N detected teeth from the model were assigned to
to capture hierarchical features, skip connections to C = 32 FDI numbers. This was carried out by filling in an
preserve spatial information, and a decoder, which up- assignment matrix E ∈ {0, 1}N ×C from a mathematical
samples the feature. perspective. The solution space was immense; hence, effi-
An OS was uniformly downsampled to 30,000 vertices. cient heuristics were required to reduce the space effec-
The PointCNN acted as an initial feature extractor on tively. For this reason, a penalty function f (E) and an
the downsampled OS. The PointCNN encodes an OS to associated exploration strategy space  were formulated.
a point cloud where each vertex is represented with 256 The resulting assignment E = argminf (E) would be the
E∈�
features. This downsampled point cloud is transformed one assignment that minimized the penalty.
to a Cartesian grid by max pooling the features of all The post-processing was carried out in multi-
points in one grid cell. The distributed surface points on ple stages, each refining upon the previous assign-
the entire grid domain were fed forward to the 3D-Unet. ment, exploring the assignments that were similar to
In this stage, the model estimated the bounding box the existing one. Prior to post-processing, the center
dimensions and its central position. The final aggregated of mass (COM) of each detection n , COMn , was cal-
bounding box proposals were used as inputs for the seg- culated by extracting the center of the associated
mentation task [21]. segmentation mask. The mean of all COMs was rep-
resented by COM⊙ , the axial component of which,
COM z⊙ , roughly acted as a watershed between two half
The segmentation module jaws. The COMs are used extensively in subsequent
The points pertaining to a detected tooth were extracted penalty calculations.
from the OS by expanding the tooth’s bounding box and As a first stage, E was greedily assigned to minimize
uniformly sampling 8192 points within the expanded vol- E 1 = arg min f1 (E),
ume. A PointCNN was used in the segmentation mod- E∈�Greedy
ule. Each point located inside the 3D bounding box was f1 (E) = f11 (E) + 12 f 12 (E) + 13 f13 (E)
binary classified as a tooth or gingiva.
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 4 of 9

feature normalization, flips, and rotations around the

  
= max enc − 1, 0
c n z-axis were employed on the training set.

   
    
enc · 1 COM zn < COM z⊙ + enc · 1 COM z⊙ < COM zn +13
  
+12 max 1 − enc , 0 ,
n c∈ Upper Jaw c∈ Lower Jaw n c

where f11 wished to have all FDI numbers assigned to The detection module was trained over 180 epochs
an unique object, f12 aimed to have detections assigned with a learning rate decay of 0.8 while the segmenta-
to the right jaw, and f13 reduced the count of unassigned tion module was trained for 50 epochs with a learning
detections to a minimum. ’s were weights, and were set rate decay of 1. The applied batch size was one for the
at 12 = 0.1 and 13 = 0.01. For the second stage, a per- detection module with 30,000 vertices and batch size
mutated space of E 1 was explored where the assigned three for the segmentation module with 8192 vertices.
detections remained assigned in each jaw while having Weight decay of 0.0001 and early stopping were applied
a possible permutation of FDI numbers (i.e., c enc stays for both modules. Both modules used the Adam opti-
constant ∀n). This step encourages the FDI numbers to mizer at a learning rate of 0.001. No momentum or gradi-
become sorted. ent clipping were applied. The binary cross-entropy loss
E 2 = arg min f2 (E) is minimized, where function was applied for the segmentation module. The
E∈�Permutation (E 1 )
 
en1 c1 · en2 c2 · 1 COM xn1 > COM xn2 ⊕ (c1 > c2 )
  
f2 (E) =
n1 ,n2 c1 ,c2 ∈UpperJaw

 
en1 c1 · en2 c2 · 1 COM xn1 > COM xn2 ⊕ (c1 > c2 )
  
+
n1 ,n2 c1 ,c2 ∈LowerJaw

In the formula, COM x ( x went from left to right for the detection module used a multi-task loss function consist-
patient) was enforced to grow monotonically while the ing of binary cross-entropy loss and IoU loss. The model

NVIDIA ® V100 Tensor Core GPU 16G.

FDI number increased. ⊕ denotes exclusive or. was implemented in TensorFlow 1.8 and trained on an
Finally, the sorted relationship in E 2 was retained, but
allowed insertion/removal of blank assignments and
minimize E 3 = arg min f3 (E), where Statistical analysis
E∈�Sorted (E 2 )
   The model predictions on the test set were compared to
the expert annotations. Object detection, instance seg-
 
f3 (E) = en1 c1 · en2 c2 · COMn1 − COMn2  − Dc1 c2 .
n1 ,n2 c1 ,c2 ∈UpperJaw

mentation and FDI labeling metrics were reported as

The purpose of the penalty was to minimize the dif-
follows for the test set: accuracy = TP+TN +FP+FN , preci-
TP+TN
ference between the distance of a pair of teeth and their
sion = TP+FP , recall = TP+FN and intersection over union
TP TP
corresponding predetermined distance parameter. The
(IoU) = TP+FP+FN
TP
. TP, TN, FP and FN denote true posi-
distance, Dc1 c2 was a prior matrix based on the training
tives, true negatives, false positives and false negatives,
dataset that represented the mean of distances (in mil-
respectively [5].
limeters) across the whole set.
The resulting assignment after three stages of refine-
Results
ment, E 3, would then be used for subsequent analysis.
The model achieved high detection accuracies on the test
set (350 OS(s)) with a precision of 0.994, recall of 0.988,
Model training
and average bounding box IoU of 0.806 (Table 1). The
The annotated 3D meshes were randomly split into three
bounding box IoU for individual teeth ranged from 0.718
sets of OS(s), 1224 for training (612 patients), 176 for val-
to 0.873. The detection model had, in total, 54 missed
idation (88 patients), and 350 for testing (175 patients).
detections and 29 false-positive detections.
The validation set was used to evaluate the model con-
Considering a successful detection, the model achieved
vergence during training, while the hold-out test set was
teeth segmentations with an average IoU score of 0.915
used to evaluate the model performance after training.
(Table 2). The segmentation IoU, recall, precision and
Data augmentation techniques such as shuffle points,
accuracy for individual teeth ranged from 0.792 to
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 5 of 9

Table 1 Precision, recall, and Intersection over Union (IoU) of the Table 2 Accuracy, precision, recall, and Intersection over Union
detections (IoU) of the OS segmentations
Tooth Precision Recall IoUBoundingBox Tooth Accuracy Precision Recall IoUMask

11 1.000 1.000 0.848 11 0.997 0.935 0.990 0.926

12 1.000 0.994 0.806 12 0.998 0.923 0.992 0.916
13 0.988 0.942 0.831 13 0.998 0.931 0.991 0.923
14 0.983 1.000 0.847 14 0.997 0.935 0.993 0.929
15 0.942 1.000 0.819 15 0.998 0.941 0.992 0.933
16 0.994 1.000 0.863 16 0.997 0.961 0.987 0.948
17 0.969 0.976 0.810 17 0.996 0.946 0.959 0.909
18 0.263 0.833 0.801 18 0.998 0.966 0.971 0.939
21 1.000 1.000 0.836 21 0.997 0.931 0.988 0.921
22 1.000 1.000 0.778 22 0,997 0.916 0.993 0.910
23 0.959 0.959 0.810 23 0.997 0.911 0.993 0.905
24 0.989 1.000 0.843 24 0.997 0.937 0.992 0.929
25 0.926 0.994 0.821 25 0.997 0.937 0.992 0.929
26 0.983 0.994 0.873 26 0.997 0.955 0.989 0.945
27 0.969 0.992 0.821 27 0.995 0.940 0.935 0.881
28 1.000 1.000 0.718 28 0.997 0.880 0.983 0.867
31 1.000 1.000 0.742 31 0.996 0.899 0.989 0.890
32 1.000 1.000 0.796 32 0.997 0.919 0.990 0.909
33 0.988 0.988 0.796 33 0.997 0.927 0.991 0.919
34 0.965 1.000 0.814 34 0.997 0.932 0.993 0.926
35 0.868 0.986 0.800 35 0,997 0.937 0.992 0.931
36 0.994 1.000 0.818 36 0.994 0.959 0,965 0.926
37 0.943 0.935 0.765 37 0.990 0.941 0.887 0.839
38 0.429 0.500 0.824 38 0.998 0.955 0.992 0.948
41 0.994 0.988 0.727 41 0.997 0.906 0.989 0.896
42 0.994 1.000 0.764 42 0.997 0.918 0.989 0.908
43 0.976 0.982 0.754 43 0.996 0.915 0.991 0.907
44 0.988 0.994 0.801 44 0.997 0.933 0.992 0.926
45 0.900 0.994 0.812 45 0.997 0.940 0.991 0.932
46 0.994 0.994 0.814 46 0.994 0.958 0.974 0.933
47 0.943 0.951 0.767 47 0.989 0.935 0.876 0.824
48 0.750 1.000 0.743 48 0.990 0.934 0.847 0.792

0.948, 0.847 to 0.993, 0.880 to 0.966, and 0.989 to 0.998, Discussion

respectively. The field of AI in dentistry is rapidly advancing and
The optical inspection (Figs. 2 and 3) showed excellent holds great potential for significant contributions to
position agreements between the automatically and man- dental practices in the near future [26–29]. Chen et al.
ually segmented teeth components. Minor flaws were categorized AI systems into three types: pre-appoint-
mainly seen cervically, and the lowest segmentation and ment, inter-appointment, and post-appointment
detection accuracies were seen for the third molars. systems (30). These systems can aid in patient manage-
The FDI labels of the teeth were predicted with an ment by analyzing their needs and risks before appoint-
accuracy of 0.894 (Table 3). The accuracy range for indi- ments, assisting in diagnosis, treatment planning, and
vidual teeth was between 0.6 and 1. Figure 4 illustrates outcome prediction during appointments, and sup-
the confusion matrices for the upper and lower jaw. porting labor work such as prosthodontics design and
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 6 of 9

Fig. 2 Overview of mandible teeth segmentations; left: manual segmentation; middle: automatic segmentation; right: overlay; one of the two
detection errors is illustrated

Fig. 3 Overview of maxillary teeth segmentations; left: manual segmentation; middle: automatic segmentation; right: overlay
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 7 of 9

Table 3 Accuracy of the FDI numeration Zanjani et al. proposed a volumetric anchor-based
Tooth Accuracy
region proposal network for teeth point cloud detec-
tion and segmentation with a mean IoU of 0.98 [21].
11 0.944 Cui et al. applied a two-stage network architecture
12 0.943 for tooth centroid extraction using a distance-aware
13 0.944 voting scheme and segmentation with an F1-score of
14 0.947 0.942 [20]. Similarly, Hao et al. proposed a two-mod-
15 0.945 ule approach. The segmentation module generated a
16 0.902 fine-grained segmentation, whereas the canary module
17 0.797 autocorrected the segmentation based on confidence
18 0,800 evaluation. Hao et al. reported a mean IoU of 0.936 and
21 0.938 0.942 for mandible and maxillary teeth, respectively [6].
22 0.938 The number of studies reporting the classification and
23 0.944 semantic labeling accuracies of each tooth is yet limited
24 0.913 [18, 19]. Tian et al. employed a 3D CNN using a sparse
25 0.926 voxel octree for teeth classification with an accuracy of
26 0.871 0.881 [18]. Ma et al. proposed a deep learning network to
27 0.873 predict the semantic label of each 3D tooth model based
28 0.600 on spatial relationship features. The proposed SRF-Net
31 0.850 achieved a classification accuracy of 0.9386 [19].
32 0.879 It is important to recognize that the performance of
33 0.892 deep learning models relies heavily on factors such as the
34 0.898 dataset, hyperparameters, and architecture involved [8].
35 0.931 One key obstacle to reproducing and validating previous
36 0.849 results is the restricted accessibility of the datasets used,
37 0.843 stemming from privacy concerns. Furthermore, the con-
38 1.000 siderable variation in training and test sets sizes across
41 0.847 different studies makes it difficult to draw direct compar-
42 0.884 isons. The lack of clarity regarding data representative-
43 0.916 ness further compounds the issue.
44 0.918 Moreover, attempting to reproduce complex compu-
45 0.941 tational pipelines based solely on textual descriptions
46 0.905 without access to the source code becomes a subjective
47 0.914 and challenging task (31). The inadequate description of
48 0.667 training pipelines, essential hyperparameters, and cur-
rent software dependencies undermines the transpar-
ency and reproducibility of earlier findings. Given these
treatment evaluation after appointments [18]. Particu- limitations, it’s essential to approach any direct compari-
larly, 3D treatment planning can be time-consuming son of previous segmentation and labeling results with
and laborious, but with the help of automated assis- caution [5].
tance, it can become more time-efficient, leading to Even though previous studies achieved remarkable
a more cost-effective 3D treatment planning process results, the models are regarded as black boxes lacking
[6]. In this study, the researchers evaluated the per- explicit declarative knowledge representation. Generat-
formance of a deep learning model for automating 3D ing the underlying explanatory structures is essential in
teeth detection, segmentation, and FDI labeling on 3D the medical domain to provide clinicians with a trans-
meshes. parent, understandable, and explainable system [29]. The
In dentistry, different studies have applied deep learn- current study made the results re-traceable on demand
ing models for segmentation on 3D meshes [6, 20–23]. using a hierarchical three-step plug-and-play pipeline.
Lian et al. introduced a mesh-based graph neural net- This pipeline allows clinicians to verify the immediate
work for teeth segmentation with an F1-score of 0.981 results of each module before proceeding further. In case
[23]. Zhao et al. used a graph attentional convolution the detection module fails to detect a tooth, the clinician
network with a local spatial augmentation module for can correct the mistake immediately and proceed to the
segmentation and achieved a mean IoU of 0.871 [22]. subsequent module. This stop-and-go approach ensures
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 8 of 9

Fig. 4 Confusion Matrices show the agreement between actual and predicted classes to indicate labeling accuracy, and brighter cells signify
a higher class agreement. The left and right matrices display the model performance in the maxilla and mandible, respectively

an efficient workflow while maintaining high precision and automated alignment of intra-oral scans and cone-
and explainability. Another advantage of this plug-and- beam computed tomography.
play pipeline is the interchangeability of the different The proposed model is currently clinically used for
modules. The detection and segmentation modules can orthodontic treatment planning. The constant error
be exchanged with alternative model architectures with- reductions and adaptions to real-world cases will fur-
out much difficulties. ther enhance the current model. The successful imple-
The segmentation IoU scores ranged between 0.792 mentation of this approach in daily clinical practice will
and 0.948. Furthermore, each tooth was classified and also further reduce the risks of limited robustness, gen-
labeled with an accuracy between 0.6 and 1. The low- eralizability, and reproducibility.
est segmentation and labeling accuracies were seen for
third molars. Hierarchical concatenation of different
deep learning models and post-processing heuristics Conclusion
have the disadvantage that the errors in the different In conclusion, our proposed method achieved accurate
modules are cumulative. In other words, inaccuracies teeth segmentations with a mean IoU score of 0.915.
in the detection module will affect the segmentation The FDI labels of the teeth were predicted with a mean
module and the FDI labeling algorithm. However, this accuracy of 0.894. This forms a promising foundation
shortcoming can be neglected if the pipeline is interac- for time-effective and observer-independent teeth seg-
tively used with the clinicians. mentation and labeling on intra-oral scans.
Although our proposed model has achieved clinically
Acknowledgements
applicable results, it has some limitations. Wisdom None.
teeth, supernumerary teeth, or crowded teeth impede
the segmentation and labeling accuracies. Most failure Authors’ contributions
Shankeeth Vinayahalingam: Conceptualization, Method, Investigation, Formal
cases are related to rare or complicated dental mor- Analysis, Software, Funding acquisition, Writing – original draft. Steven
phologies [6, 7, 18–20]. Without real-world integration, Kempers: Validation, Visualization, Data curation, Writing – review &; editing.
deep learning models are bound to the limits of the Julian Schoep: Software, Method, Formal Analysis, Writing – review &; editing.
Tzu-Ming Harry Hsu: Software, Method, Formal Analysis, Writing – review
training set and validation set. Furthermore, extensive &; editing. David Anssari Moin: Investigation, Validation, Resources, Project
model comparisons are required to choose the optimal administration, Funding acquisition, Supervision, Writing – review &; editing.
model architectures for the respective modules (e.g., Bram van Ginneken: Investigation, Validation, Supervision, Writing – review &;
editing. Tabea Flügge: Investigation, Validation, Supervision, Writing – review
Point-RCNN for the detection module). Future stud- &; editing. Marcel Hanisch: Investigation, Validation, Supervision, Writing
ies should focus on further automation of 3D treat- – review &; editing. Tong Xi: Investigation, Validation, Supervision, Writing –
ment planning steps, such as automated crown design review &; editing.
Vinayahalingam et al. BMC Oral Health (2023) 23:643 Page 9 of 9

Funding 14. Yoo JH, Yeom HG, Shin W, Yun JP, Lee JH, Jeong SH, et al. Deep learning
Open Access funding enabled and organized by Projekt DEAL. This research based prediction of extraction difficulty for mandibular third molars. Sci
is partially funded by the Radboud AI for Health collaboration between Rep. 2021;11(1):1954.
Radboud University and Radboudumc, and the Innovation Center for Artificial 15. Yu HJ, Cho SR, Kim MJ, Kim WH, Kim JW, Choi J. Automated skeletal clas‑
Intelligence (ICAI). sification with lateral cephalometry based on artificial intelligence. J Dent
Res. 2020;99(3):249–56.
Availability of data and materials 16. Ter Horst R, van Weert H, Loonen T, Berge S, Vinayahalingam S, Baan F,
The datasets used and/or analyzed during the current study are available from et al. Three-dimensional virtual planning in mandibular advancement
the corresponding author upon reasonable request. surgery: Soft tissue prediction based on deep learning. J Craniomaxillofac
Surg. 2021;49(9):775–82.
17. Lahoud P, EzEldeen M, Beznik T, Willems H, Leite A, Van Gerven A, et al. Arti‑
Declarations ficial intelligence for fast and accurate 3-dimensional tooth segmentation
on cone-beam computed tomography. J Endod. 2021;47(5):827–35.
Ethics approval and consent to participate 18. Tian S, Dai N, Zhang B, Yuan F, Yu Q, Cheng X. Automatic classification and
This study was conducted in accordance with the code of ethics of the World segmentation of teeth on 3d dental model using hierarchical deep learn‑
Medical Association (Declaration of Helsinki) and the ICH-GCP. The approval ing networks. IEEE Access. 2019;7:84817–28.
of this study was granted by the Commissie Mensgebonden Onderzoek 19. Ma Q, Wei GS, Zhou YF, Pan X, Xin SQ, Wang WP. SRF-net: spatial relation‑
Radboudumc, Nijmegen, The Netherlands, which also approved that informed ship feature network for tooth point cloud classification. Comput Graph
consent was not required as all image data were anonymized and de-identi‑ Forum. 2020;39(7):267–77.
fied before analysis (decision no. 2021–13253). 20. Cui ZM, Li CJ, Chen NL, Wei GD, Chen RN, Zhou YF, et al. TSegNet: An
efficient and accurate tooth segmentation network on 3D dental model.
Consent for publication Med Image Anal. 2021;69:101949.
Not applicable. 21. Zanjani FG, Pourtaherian A, Zinger S, Moin DA, Claessen F, Cherici T, et al.
Mask-MCNet: tooth instance segmentation in 3D point clouds of intra-
Competing interests oral scans. Neurocomputing. 2021;453:286–98.
The authors declare no competing interests. 22. Zhao Y, Zhang LM, Yang CS, Tan YY, Liu Y, Li PC, et al. 3D Dental model
segmentation with graph attentional convolution network. Pattern
Recogn Lett. 2021;152:79–85.
Received: 7 April 2023 Accepted: 26 August 2023 23. Lian C, Wang L, Wu TH, Wang F, Yap PT, Ko CC, et al. Deep multi-scale
mesh feature learning for automated labeling of raw dental surfaces from
3d intraoral scanners. IEEE Trans Med Imaging. 2020;39(7):2440–50.
24. Poon AIF, Sung JJY. Opening the black box of AI-Medicine. J Gastroenterol
Hepatol. 2021;36(3):581–4.
References 25. Amann J, Blasimme A, Vayena E, Frey D, Madai VI, Consortium PQ. Explain‑
1. Mangano F, Gandolfi A, Luongo G, Logozzo S. Intraoral scanners in den‑ ability for artificial intelligence in healthcare: a multidisciplinary perspec‑
tistry: a review of the current literature. BMC Oral Health. 2017;17(1):149. tive. Bmc Med Inform Decis. 2020;20(1):1–9.
2. Jheon AH, Oberoi S, Solem RC, Kapila S. Moving towards precision 26. Kempers S, van Lierop P, Hsu TH, Moin DA, Berge S, Ghaeminia H, et al.
orthodontics: an evolving paradigm shift in the planning and delivery of Positional assessment of lower third molar and mandibular canal using
customized orthodontic therapy. Orthod Craniofac Res. 2017;20:106–13. explainable artificial intelligence. J Dent. 2023;133:104519.
3. Baan F, Bruggink R, Nijsink J, Maal TJJ, Ongkosuwito EM. Fusion of intra- 27. Li YY, Bu R, Sun MC, Wu W, Di XH, Chen BQ. PointCNN: Convolution On X
oral scans in cone-beam computed tomography scans. Clin Oral Investig. -Transformed Points. Adv Neur In. 2018;31.
2021;25(1):77–85. 28. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O, editors. 3D
4. Stokbro K, Aagaard E, Torkov P, Bell RB, Thygesen T. Virtual planning in U-Net: learning dense volumetric segmentation from sparse annotation.
orthognathic surgery. Int J Oral Maxillofac Surg. 2014;43(8):957–65. International conference on medical image computing and computer-
5. Vinayahalingam S, Goey RS, Kempers S, Schoep J, Cherici T, Moin DA, et al. assisted intervention; Springer. 2016.
Automated chart filing on panoramic radiographs using deep learning. 29. Chen YW, Stanley K, Att W. Artificial intelligence in dentistry: current appli‑
J Dent. 2021;115:103864. cations and future perspectives. Quintessence Int. 2020;51(3):248–57.
6. Hao J, Liao W, Zhang YL, Peng J, Zhao Z, Chen Z, et al. Toward clinically 30. Haibe-Kains B, Adam GA, Hosny A, Khodakarami F. Massive Analysis
applicable 3-dimensional tooth segmentation via deep learning. J Dent Quality Control Society Board of D, Waldron L, et al. Transparency and
Res. 2022;101(3):304–11. reproducibility in artificial intelligence. Nature. 2020;586(7829):E14–E6.
7. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey 31. Holzinger A, Biemann C, Pattichis CS, Kell DB. What do we need to
on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88. build explainable AI systems for the medical domain? arXiv preprint
8. Vinayahalingam S, Kempers S, Limon L, Deibel D, Maal T, Hanisch M, et al. arXiv:171209923. 2017.
Classification of caries in third molars on panoramic radiographs using
deep learning. Sci Rep. 2021;11(1):12609.
9. Krois J, Ekert T, Meinhold L, Golla T, Kharbot B, Wittemeier A, et al. Deep Publisher’s Note
learning for the radiographic detection of periodontal bone loss. Sci Rep. Springer Nature remains neutral with regard to jurisdictional claims in pub‑
2019;9(1):8495. Ready
lished mapsto and
submit your research
institutional ? Choose BMC and benefit from:
affiliations.
10. Lee J-H, Kim D-H, Jeong S-N. Diagnosis of cystic lesions using panoramic
and cone beam computed tomographic images based on deep learning • fast, convenient online submission
neural network. Oral Dis. 2020;26(1):152–8. • thorough peer review by experienced researchers in your field
11. Fu Q, Chen Y, Li Z, Jing Q, Hu C, Liu H, et al. A deep learning algorithm • rapid publication on acceptance
for detection of oral cavity squamous cell carcinoma from photographic
images: a retrospective study. EClinicalMedicine. 2020;27:100558. • support for research data, including large and complex data types
12. Schwendicke F, Rossi JG, Göstemeyer G, Elhennawy K, Cantu AG, Gaudin • gold Open Access which fosters wider collaboration and increased citations
R, et al. Cost-effectiveness of Artificial Intelligence for Proximal Caries • maximum visibility for your research: over 100M website views per year
Detection. J Dent Res. 2021;100(4):369–76.
13. Qu Y, Lin Z, Yang Z, Lin H, Huang X, Gu L. Machine learning models for At BMC, research is always in progress.
prognosis prediction in endodontic microsurgery. J Dent. 2022;118:103947.
Learn more biomedcentral.com/submissions