CT-based Deep Learning Segmentation of Ovarian Cancer and The Stability of The Extracted Radiomics Features
CT-based Deep Learning Segmentation of Ovarian Cancer and The Stability of The Extracted Radiomics Features
CT-based Deep Learning Segmentation of Ovarian Cancer and The Stability of The Extracted Radiomics Features
Contributions: (I) Conception and design: Y Wang, P Cao, EYP Lee; (II) Administrative support: EYP Lee; (III) Provision of study materials or
patients: M Wang, L Han, EMF Wong, G Ho, TPW Lam, EYP Lee; (IV) Collection and assembly of data: Y Wang, M Wang; (V) Data analysis and
interpretation: Y Wang, M Wang, EYP Lee; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.
Correspondence to: Dr. Lujun Han. Department of Medical Imaging, State Key Laboratory of Oncology in South China, Guangdong Key Laboratory
of Nasopharyngeal Carcinoma Diagnosis and Therapy, Sun Yat-sen University Cancer Center, Guangzhou 510060, China. Email: hanlj@sysucc.org.cn;
Dr. Elaine Y. P. Lee. Department of Diagnostic Radiology, Room 406, Block K, Queen Mary Hospital, University of Hong Kong, 102 Pokfulam
Road, Hong Kong, China. Email: eyplee77@hku.hk.
Background: Radiomics analysis could provide complementary tissue characterization in ovarian cancer
(OC). However, OC segmentation required in radiomics analysis is time-consuming and labour-intensive.
In this study, we aim to evaluate the performance of deep learning-based segmentation of OC on contrast-
enhanced CT images and the stability of radiomics features extracted from the automated segmentation.
Methods: Staging abdominopelvic CT images of 367 patients with OC were retrospectively recruited. The
training and cross-validation sets came from center A (n=283), and testing set (n=84) came from centers B
and C. The tumours were manually delineated by a board-certified radiologist. Four model architectures
provided by no-new-Net (nnU-Net) method were tested in this task. The segmentation performance
evaluated by Dice score, Jaccard score, sensitivity and precision were compared among 4 architectures. The
Pearson correlation coefficient (ρ), concordance correlation coefficient (ρc) and Bland-Altman plots were used
to evaluate the volumetric assessment of OC between manual and automated segmentations. The stability of
extracted radiomics features was evaluated by intraclass correlation coefficient (ICC).
Results: The 3D U-Net cascade architecture achieved highest median Dice score, Jaccard score, sensitivity
and precision for OC segmentation in the testing set, 0.941, 0.890, 0.973 and 0.925, respectively. Tumour
volumes of manual and automated segmentations were highly correlated (ρ=0.944 and ρc =0.933). 85.0% of
radiomics features had high correlation with ICC >0.8.
Conclusions: The presented deep-learning segmentation could provide highly accurate automated
segmentation of OC on CT images with high stability of the extracted radiomics features, showing the
potential as a batch-processing segmentation tool.
Keywords: Ovarian cancer (OC); computed tomography; deep learning; automated segmentation; radiomics
^ ORCID: Yiang Wang, 0000-0003-2656-1511; Mandi Wang, 0000-0002-8467-148X; Peng Cao, 0000-0002-1116-1171; Elaine Y. P. Lee,
0000-0002-0627-5297.
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
Quantitative Imaging in Medicine and Surgery, Vol 13, No 8 August 2023 5219
Submitted Oct 23, 2022. Accepted for publication May 22, 2023. Published online Jun 13, 2023.
doi: 10.21037/qims-22-1135
View this article at: https://dx.doi.org/10.21037/qims-22-1135
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
5220 Wang et al. CT-based deep learning segmentation of ovarian cancer
Exclusion:
• Significant artefacts precluding
evaluation of the pelvic region (n=8)
• Incomplete coverage of OC on the
CT images (n=40)
Figure 1 Flow diagram of patient inclusion and exclusion criteria. OC, ovarian cancer; CT, computed tomography.
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
Quantitative Imaging in Medicine and Surgery, Vol 13, No 8 August 2023 5221
32×512×512 32×80×160×160
64×256×256 64×40×80×80
128×128×128 128×20×40×40
256×64×64 256×10×20×20
480×32×32 320×5×10×10
Max pooling 480×16×16 Max pooling
Up-convolution Up-convolution
Skip connection Skip connection 320×5×5×5
Conv2D-IN-LeakyReLU 480×8×8 Conv3D-IN-LeakyReLU
32×48×192×192
64×24×96×96
3D- 3D-
128×12×48×48 lowres fullres
256×6×24×24
320×6×12×12 Segmentation with Image with
Max pooling Up/down sampling
low resolution full resolution
Up-convolution
Segmentation with 3D U-Net with
Skip connection 320×6×6×6 full resolution Cropping
low/full resolution
Conv3D-IN-LeakyReLU
Figure 2 Four model architectures generated by nnU-Net in this study. (A) 2D; (B) 3D-lowres; (C) 3D-fullres; (D) 3D-cascade. Conv2D,
2D convolutional layer; Conv3D, 3D convolutional layer; IN, instance normalization; LeakyReLU, Leaky rectified linear unit.
and 3D-fullres architecture respectively. The second stage target slice thickness was the largest slice thickness of the
can be implemented after the training of all five 3D-lowres training cases. Global intensity normalization was applied
models in the first stage have completed. The batch size was to all the CT images before training.
32 for 2D model and 2 for 3D models. The input patch size
for 2D, 3D-lowres/3D-cascade, and 3D-fullres architecture
Training details
was 512×512, 80×160×160, and 48×192×192 respectively.
The loss function was defined as the sum of Dice loss
and cross-entropy loss. The optimizer used in this study
Image preprocessing
was stochastic gradient descent with initial learning rate
Dataset properties, such as voxel spacings, image shape, of 0.01 and momentum of 0.99. Data augmentation
modality and intensity, were first automatically collected methods used were rotation, scaling, gamma correction
by the nnU-Net pipeline. These properties were then used and mirroring. Each model was trained for 1,000 epochs
to configure resampling and normalization strategy, batch by default to ensure its convergency. For each type of
size, patch size etc. As the voxel spacing of CT images from network architectures (i.e., 2D, 3D-lowres, 3D-fullres, and
three centers were different and anisotropic (Table 1), all 3D-cascade), the final predicted segmentation was made
CT images and corresponding masks were resampled to the based on the average result of five models, which were
same voxel spacing (1.492×1.492×5 mm for 3D-lowres, and trained using different dataset divisions for cross-validation.
0.736×0.736×5 mm for 2D and 3D-fullres). Third-order One network architecture with the highest average Dice
spline interpolation was used for in-plane resampling, and score computed in the cross-validation set was selected as
nearest-neighbor interpolation was used for out-of-plane the recommended architecture, and used for volumetric
resampling to suppress resampling artefacts. The target assessment and radiomics stability analysis.
sampling pixel spacing automatically selected by the nnU- The whole process was performed on an Intel Xeon Gold
Net pipeline was the 10th percentile pixel spacing of the 5217 central processing unit and a NVIDIA Tesla V100
training cases for 3D-lowres model, and the median pixel graphics processing unit card. The program was written in
spacing was selected for 2D and 3D-fullres models. The Python 3.7, with PyTorch library version 1.8.1 and nnUNet
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
5222 Wang et al. CT-based deep learning segmentation of ovarian cancer
library version 1.6.6. plots were used to evaluate the correlation or agreement
of volumes manually delineated by the radiologist as the
ground truth and predicted segmentation provided by
Performance evaluation
selected nnU-Net model. All tests with a P value <0.05
The evaluation metrics for the segmentation performance were regarded as statistically significant. All the hypothesis
includes Dice score, Jaccard score, sensitivity and precision, tests were performed using Scipy (version 1.6.3), a Python
defined in previous studies (19,20). To further investigate library. The developed codes of OC segmentation can
the possible influence of clinicopathological factors on be found in the online repository (https://github.com/
the auto-segmentation performance, the Dice scores in HKUCaoLab/segment_OC).
the testing set were compared between two histological
subtypes: high grade serous carcinoma (HGSC) and non-
Results
HGSC, FIGO (International Federation of Gynecology
and Obstetrics) stages and testing centers. For the selected Patient characteristics
network architecture, we investigated the accuracy of
tumour volume assessments in the testing set. The The patient’s median age was 51 years (range, 18–90 years), and
tumour volume was calculated as number of voxels inside median tumour volume of manual delineations was 375 cm3
the tumour multiplied by the voxel volume based on (range, 6–32,435 cm3) based on the resampled images. The
the resampled images. In total 1,218 radiomics features dataset consisted of patients with all stages of OC: FIGO
defined in the Pyradiomics template (https://github. stage I (n=76, 21.1%), stage II (n=43, 11.9%), stage III
com/AIM-Harvard/pyradiomics/blob/master/examples/ (n=180, 50.0%) and stage IV (n=61, 16.9%). The majority
exampleSettings/exampleCT.yaml) were extracted with of cases were HGSC (n=240, 65.4%). The pathological
Pyradiomics library version 3.0.1 to keep the repeatability of characteristics are summarized in Table 2. The mean
feature extraction. These radiomics features were extracted Kappa value for intra-rater reliability was 0.886±0.104,
from three image types: original, wavelet filter (including 8 demonstrating high consistency between the two
combinations of either high or low pass filter in each spatial delineations.
dimensions) and Laplacian of Gaussian (LoG) filter (sigma
=1.0, 2.0, 3.0, 4.0 and 5.0). For the original image type, the Segmentation performance
radiomics features include First Order, Shape, Gray Level
Cooccurence Matrix, Gray Level Run Length Matrix, The results of OC segmentation performance in the testing
Gray Level Size Zone Matrix, and Gray Level Dependence set are summarized in Table 3. Representative segmentations
Matrix features. For each type of wavelet or LoG filter, in the testing set predicted by four models were shown in
all the aforementioned features were included except for Figure 3. The highest median Dice score, Jaccard score and
Shape-based feature. The intraclass correlation coefficients sensitivity among the testing set were 0.941, 0.890 and 0.973
(ICCs) of radiomics features between the ground truth achieved by 3D-cascade. The highest median precision
and predicted segmentation in the testing set were used to among the testing set was 0.938 achieved by both 2D and
evaluate the stability of the extracted radiomics features. An 3D-fullres models. Dice score, Jaccard score, and sensitivity
ICC >0.8 was regarded as marker of high stability, as widely among four nnU-Net architectures were significantly
defined in other studies (21-23). different (P=0.028, P=0.028, and P<0.001, respectively),
while precision was not (P=0.800). The Dice score,
Jaccard score, sensitivity and precision of 3D-cascade were
Statistical analysis
significantly higher than 2D (P=0.013, P=0.013, P<0.001
Evaluation metrics for segmentation performance, age and P<0.001 respectively).
and tumour volume were represented as median (range). There were 4 types of relationship between the ground
The difference was compared using Mann-Whitney U test truth and automated segmentation: good agreement (Figure
for two categories, and Kruskal–Wallis test for multiple 4A), area underestimated by automated segmentation
categories, considering that some subsets did not follow (Figure 4B), area overestimated by automated segmentation
the normal distribution. Pearson correlation coefficient (ρ), (Figure 4C) and mixture of both underestimated and
concordance correlation coefficient (ρc) and Bland-Altman overestimated area (Figure 4D). It was also noted that
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
Quantitative Imaging in Medicine and Surgery, Vol 13, No 8 August 2023 5223
Tumour volume* (cm3) 360 (6–32,435) 343 (7–3,389) 652 (27–3,660) 375 (6–32,435)
A B
Manual
2D
3D-lowres
3D-fullres
3D-cascade
Figure 3 Representative examples of segmentation results predicted by the four models (A,B). The area inside the red ROI represents
manually delineated tumour by radiologist as ground truth, and the area inside the green/orange/pink/blue ROI represents the segmentation
predicted by 2D/3D-lowres/3D-fullres/3D-cascade model. The 2D model failed to identify the tumour in (B). ROI, region of interest.
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
5224 Wang et al. CT-based deep learning segmentation of ovarian cancer
A B
C D
Figure 4 Representative examples of segmentation results predicted by the 3D-cascade model (Dice score for the whole tumour: A. 0.963, B.
0.869, C. 0.930 and D. 0.881). The area inside the red ROI represents manually delineated tumour by radiologist as ground truth, and the
area inside the blue ROI represents the predicted segmentation. ROI, region of interest.
physiological corpus luteal cyst, pelvic ascites, pelvis and ρc =0.933, Figure 6B). Figure 5B shows a representative
peritoneal and nodal metastases, and the presence of slice of the outlier identified in Figure 6B.
bilateral ovarian masses affected the performance of the
automated segmentation resulting in low Dice scores due
Radiomics features stability
to incorrect segmentation of these pelvis masses or areas
(Figure 5). Most of the radiomics features extracted from segmentation
3D-cascade was the selected network architecture with predicted by the 3D-cascade model were stable compared
the highest average Dice score in the cross-validation set with radiomics features extracted from manual delineation,
among the 4 tested nnU-Net architectures. The differences in that 85.0% of the radiomics features had high correlation
of Dice scores between histological types, FIGO stages, with ICCs >0.8 (Figure 7).
validation and testing sets, as well as two external testing
sets for the 3D-cascade model are presented in Table 4. The
Discussion
Dice score of HGSC was significantly lower than non-
HGSC (HGSC: Dice =0.919; non-HGSC: Dice =0.958; In this study, deep learning-based models (i.e., nnU-Net)
P<0.001), and the Dice score of cross-validation set was were used to segment primary OC on contrast-enhanced
significantly lower than testing set (validation: Dice =0.892; CT images. Among the tested nnU-Net architectures,
testing: Dice =0.941; P<0.001). the 3D-cascade performed best in the testing set with
the highest median Dice score of 0.941. In addition, the
automated segmented volume was highly concordant to
Volumetric assessment
the manually segmented volume by radiologist with high
The 3D-cascade nnU-Net model underestimated the stability of the extracted radiomics features.
tumour volume by 75.29 cm3 on average (Figure 6A). High There was no significant difference in the Dice scores
correlation and concordance were found between manually of 3D-cascade model between the two external testing sets
segmented volumes and predicted tumour volumes (ρ=0.944 and FIGO stages, which implied the generalizability of
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
Quantitative Imaging in Medicine and Surgery, Vol 13, No 8 August 2023 5225
A1 A2
B C
D E
Figure 5 Low Dice scores predicted by the 3D-cascade model due to incorrect segmentation of the right corpus luteal cyst, Dice score 0.760
(A1, A2); ascites, Dice score 0.498 (B); peritoneal metastasis, Dice score 0.501 (C); enlarged pelvic sidewall lymph nodes, Dice score 0.584
(D); contralateral ovarian mass with bilateral involvement, Dice score 0.371 (E). The area inside the red ROI represents manually delineated
tumour by radiologist as ground truth, and the area inside the blue ROI represents the predicted segmentation. ROI, region of interest.
our model in different external datasets and FIGO stages. was found for FIGO stage I–II (Table 4).
In contrast, the median Dice score of HGSC was found Strong correlation and high concordance were found
significantly lower than non-HGSC. This might result between manually segmented volumes as ground truth
from the differences in the complexity and heterogeneity and predicted tumour volumes. The change of difference
of HGSC and non-HGSC tumours, and the different in volume was not observed when the average volume
distribution of HGSC and non-HGSC tumours in the of ground truth and predicted segmentation changed,
training and testing sets. With the latter, the lower Dice implying that the performance of volume assessment was
score for HGSC could lead to a lower average Dice score independent of tumour volume. However, our model
for cross-validation results than testing results. Similarly, wrongly segmented tissues with similar density to that
the imbalanced distribution of each FIGO stage could of OC, for example in physiological corpus luteal cyst
result in such a significant difference. We speculate that that gave rise to more complex appearance with central
the higher Dice score on testing set might also come from rim enhancement on CT. The presence of pelvic ascites
higher proportion of FIGO stage I–II, as higher Dice score adjacent to a relatively less complex ovarian tumour could
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
5226 Wang et al. CT-based deep learning segmentation of ovarian cancer
Table 4 Dice scores of different histological types, FIGO stages, and two testing sets for the 3D-cascade model
Subgroups Dice score P value
II 0.947 (0.897–0.978)
IV 0.940 (0.914–0.981)
A B
+SD1.96: −7.6e+02
Difference in volume, cm3
5000 y=1.079x+8.636
500 ρ=0.944; P<0.001
4000 ρc=0.933
Figure 6 Bland-Altman plot (A), and correlation/concordance analysis (B) for the tumour volumes manually segmented by a radiologist
(ground truth), and automatically segmented by the best-performed 3D-cascade model in the testing set. SD, standard deviation; ρ, Pearson
correlation coefficient. ρc, concordance correlation coefficient.
80 1000 the less distinct borders between the two, or being mistaken
Number of features
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
Quantitative Imaging in Medicine and Surgery, Vol 13, No 8 August 2023 5227
this quantitative information was also retained by applying the highest performance metrics achieved by 3D-cascade
global normalization to all images during training and model. The 3D-cascade model provided accurate volume
testing. assessment of OC on CT images and ensured stability of
Different from conventional deep learning-based the extracted radiomic features, showing potential use in
segmentation methods, nnU-Net focused on building quantitative radiomics analysis.
a systemized and generalized processing pipeline for all
kinds of biomedical segmentation tasks instead of network
Acknowledgments
architecture (16). The effectiveness of such improvements
was confirmed by a series of studies. Zhao et al. reported Funding: This work was supported by the Health and
a Dice score of 0.92 for the segmentation of brain Medical Research Fund, Hong Kong (grant number
hemorrhage on CT images (20). Huo et al. reported Dice 08192106).
scores of 0.968 and 0.877 in segmenting whole breast
and fibroglandular tissue on dynamic contrast-enhanced
Footnote
magnetic resonance images (24). Our results further
confirmed that nnU-Net could adapt to our task and dataset Conflicts of Interest: All authors have completed the ICMJE
well with high performance, and could become a convenient uniform disclosure form (available at https://qims.amegroups.
out-of-the-box tool for further quantitative analysis of com/article/view/10.21037/qims-22-1135/coif). EL reports
OC. To further evaluate its potential clinical or research that this work was supported by the Health and Medical
utility, we investigated the stability of the radiomics features Research Fund, Hong Kong (grant number 08192106). The
based on tumour segmentation derived from 3D-cascade other authors have no conflicts of interest to declare.
model and showed that 85.0% of the radiomics features
achieved ICC >0.80, similar to Caballo et al., but with Ethical Statement: The authors are accountable for all
more radiomics features tested in our study (25). The aspects of the work in ensuring that questions related
performance of our trained segmentation model may have to the accuracy or integrity of any part of the work are
promising role in radiomics analysis with the potential of appropriately investigated and resolved. The study was
saving time and labour in tumour segmentation of OC. conducted in accordance with the Declaration of Helsinki
There were several limitations in this study. First, (as revised in 2013). The study was approved by the
histological subtypes and FIGO stages were imbalanced Institutional Review Board of the University of Hong
in each center. This may lead to low Dice scores in those Kong/Hospital Authority Hong Kong West Cluster (No.
infrequent subtypes or FIGO stages. Second, the proportion UW 20-251), the Hong Kong East Cluster Research
of histological subtypes and FIGO stages were different Ethics Committee (No. HKECREC-2020-040), and the
in the 3 centers. This might lead to the difference of Institutional Review Board of the Sun Yat-sen University
segmentation performance between validation and testing Cancer Center (No. YB2018-52), and individual consent for
set, and limit the robustness of these trained models. Third, this retrospective analysis was waived.
the training, validation and testing sets were relatively small,
which result in a wide range of evaluation metrics on the Open Access Statement: This is an Open Access article
testing set. The small testing set may limit the verification distributed in accordance with the Creative Commons
of model robustness on different types of OC. Continuous Attribution-NonCommercial-NoDerivs 4.0 International
effort will be made to increase the sample size and further License (CC BY-NC-ND 4.0), which permits the non-
improve the accuracy and generalizability of the tumour commercial replication and distribution of the article with
segmentation task. Fourth, the specific task in this study was the strict proviso that no changes or edits are made and the
to segment OC, hence it may not be generalizable to other original work is properly cited (including links to both the
tumour types or other tasks on abdominopelvic CT images. formal publication through the relevant DOI and the license).
See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
Conclusions
References
The deep learning-based models showed the potential to
provide high performance automated segmentation with 1. Stewart C, Ralyea C, Lockwood S. Ovarian Cancer: An
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
5228 Wang et al. CT-based deep learning segmentation of ovarian cancer
Integrated Review. Semin Oncol Nurs 2019;35:151-6. U-Net and Its Variants for Medical Image Segmentation:
2. Javadi S, Ganeshan DM, Qayyum A, Iyer RB, Bhosale A Review of Theory and Applications. Ieee Access
P. Ovarian Cancer, the Revised FIGO Staging System, 2021;9:82031-57.
and the Role of Imaging. AJR Am J Roentgenol 14. Oktay O, Schlemper J, Folgoc LL, Lee MJ, Heinrich MP,
2016;206:1351-60. Misawa K, Mori K, McDonagh SG, Hammerla NY, Kainz
3. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images B, Glocker B, Rueckert D. Attention U-Net: Learning
Are More than Pictures, They Are Data. Radiology Where to Look for the Pancreas. arXiv preprint 2018; doi:
2016;278:563-77. https://arxiv.org/abs/1804.03999.
4. Danala G, Thai T, Gunderson CC, Moxley KM, Moore 15. Zhang ZX, Liu QJ, Wang YH. Road Extraction by Deep
K, Mannel RS, Liu H, Zheng B, Qiu Y. Applying Residual U-Net. Ieee Geoscience and Remote Sensing
Quantitative CT Image Feature Analysis to Predict Letters 2018;15:749-53.
Response of Ovarian Cancer Patients to Chemotherapy. 16. Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-
Acad Radiol 2017;24:1233-9. Hein KH. nnU-Net: a self-configuring method for deep
5. An H, Wang Y, Wong EMF, Lyu S, Han L, Perucho learning-based biomedical image segmentation. Nat
JAU, Cao P, Lee EYP. CT texture analysis in histological Methods 2021;18:203-11.
classification of epithelial ovarian carcinoma. Eur Radiol 17. Ye X, Guo D, Ge J, Di X, Lu Z, Xiao J, Yao G, Lu L, Jin D,
2021;31:5050-8. Yan S. Anatomy Guided Thoracic Lymph Node Station
6. Wang M, Perucho JAU, Hu Y, Choi MH, Han L, Delineation in CT Using Deep Learning Model. Int J
Wong EMF, Ho G, Zhang X, Ip P, Lee EYP. Computed Radiat Oncol Biol Phys 2021;111:e120-e1.
Tomographic Radiomics in Differentiating Histologic 18. Heidenreich JF, Gassenmaier T, Ankenbrand MJ, Bley TA,
Subtypes of Epithelial Ovarian Carcinoma. JAMA Netw Wech T. Self-configuring nnU-net pipeline enables fully
Open 2022;5:e2245141. automatic infarct segmentation in late enhancement MRI
7. Rizzo S, Botta F, Raimondi S, Origgi D, Fanciullo C, after myocardial infarction. Eur J Radiol 2021;141:109817.
Morganti AG, Bellomi M. Radiomics: the facts and the 19. Taha AA, Hanbury A. Metrics for evaluating 3D medical
challenges of image analysis. Eur Radiol Exp 2018;2:36. image segmentation: analysis, selection, and tool. BMC
8. Jiang H, Diao Z, Yao YD. Deep learning techniques Med Imaging 2015;15:29.
for tumor segmentation: a review. J Supercomput 20. Zhao X, Chen K, Wu G, Zhang G, Zhou X, Lv C, Wu
2022;78:1807-51. S, Chen Y, Xie G, Yao Z. Deep learning shows good
9. Jin J, Zhu H, Zhang J, Ai Y, Zhang J, Teng Y, Xie C, reliability for automatic segmentation and volume
Jin X. Multiple U-Net-Based Automatic Segmentations measurement of brain hemorrhage, intraventricular
and Radiomics Feature Stability on Ultrasound Images extension, and peripheral edema. Eur Radiol
for Patients With Ovarian Cancer. Front Oncol 2021;31:5012-20.
2021;10:614201. 21. Leijenaar RT, Carvalho S, Velazquez ER, van Elmpt
10. Rundo L, Beer L, Ursprung S, Martin-Gonzalez P, WJ, Parmar C, Hoekstra OS, Hoekstra CJ, Boellaard R,
Markowetz F, Brenton JD, Crispin-Ortuzar M, Sala Dekker AL, Gillies RJ, Aerts HJ, Lambin P. Stability of
E, Woitek R. Tissue-specific and interpretable sub- FDG-PET Radiomics features: an integrated analysis
segmentation of whole tumour burden on CT images of test-retest and inter-observer variability. Acta Oncol
by unsupervised fuzzy clustering. Comput Biol Med 2013;52:1391-7.
2020;120:103751. 22. Pavic M, Bogowicz M, Würms X, Glatz S, Finazzi T,
11. Liu J, Wang S, Linguraru MG, Yao J, Summers RM. Riesterer O, Roesch J, Rudofsky L, Friess M, Veit-
Tumor sensitive matching flow: A variational method to Haibach P, Huellner M, Opitz I, Weder W, Frauenfelder
detecting and segmenting perihepatic and perisplenic T, Guckenberger M, Tanadini-Lang S. Influence of inter-
ovarian cancer metastases on contrast-enhanced abdominal observer delineation variability on radiomics stability in
CT. Med Image Anal 2014;18:725-39. different tumor sites. Acta Oncol 2018;57:1070-4.
12. Wadhwa A, Bhardwaj A, Singh Verma V. A review on brain 23. van Griethuysen JJM, Fedorov A, Parmar C, Hosny A,
tumor segmentation of MRI images. Magn Reson Imaging Aucoin N, Narayan V, Beets-Tan RGH, Fillion-Robin JC,
2019;61:247-59. Pieper S, Aerts HJWL. Computational Radiomics System
13. Siddique N, Paheding S, Elkin CP, Devabhaktuni V. to Decode the Radiographic Phenotype. Cancer Res
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135
Quantitative Imaging in Medicine and Surgery, Vol 13, No 8 August 2023 5229
© Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2023;13(8):5218-5229 | https://dx.doi.org/10.21037/qims-22-1135