European Radiology

https://doi.org/10.1007/s00330-020-07325-3

IMAGING INFORMATICS AND ARTIFICIAL INTELLIGENCE

A preliminary investigation of radiomics differences

between ruptured and unruptured intracranial aneurysms
Chubin Ou 1,2 & Winston Chong 3 & Chuan-Zhi Duan 2 & Xin Zhang 2 & Michael Morgan 1 & Yi Qian 1

Received: 23 March 2020 / Revised: 7 August 2020 / Accepted: 18 September 2020

# European Society of Radiology 2020

Abstract
Objectives Prediction of intracranial aneurysm rupture is important in the management of unruptured aneurysms. The application
of radiomics in predicting aneurysm rupture remained largely unexplored. This study aims to evaluate the radiomics differences
between ruptured and unruptured aneurysms and explore its potential use in predicting aneurysm rupture.
Methods One hundred twenty-two aneurysms were included in the study (93 unruptured). Morphological and radiomics features
were extracted for each case. Statistical analysis was performed to identify significant features which were incorporated into
prediction models constructed with a machine learning algorithm. To investigate the usefulness of radiomics features, three
models were constructed and compared. The baseline model A was constructed with morphological features, while model B was
constructed with addition of radiomics shape features and model C with more radiomics features. Multivariate analysis was
performed for the ten most important variables in model C to identify independent risk factors. A simplified model based on
independent risk factors was constructed for clinical use.
Results Five morphological features and 89 radiomics features were significantly associated with rupture. Model A, model B,
and model C achieved the area under the receiver operating characteristic curve of 0.767, 0.807, and 0.879, respectively. Model C
was significantly better than model A and model B (p < 0.001). Multivariate analysis identified two radiomics features which
were used to construct the simplified model showing an AUROC of 0.876.
Conclusions Radiomics signatures were different between ruptured and unruptured aneurysms. The use of radiomics features,
especially texture features, may significantly improve rupture prediction performance.
Key Points
• Significant radiomics differences exist between ruptured and unruptured intracranial aneurysms.
• Radiomics shape features can significantly improve rupture prediction performance over conventional morphology-based
prediction model. The inclusion of histogram and texture radiomics features can further improve the performance.
• A simplified model with two variables achieved a similar level of performance as the more complex ones. Our prediction model
can serve as a promising tool for the risk management of intracranial aneurysms.

Electronic supplementary material The online version of this article

(https://doi.org/10.1007/s00330-020-07325-3) contains supplementary
material, which is available to authorized users.

* Yi Qian
[email protected]

1
Faculty of Medicine and Health Sciences, Macquarie University,
Sydney, New South Wales, Australia
2
National Key Clinical Specialty/Engineering Technology Research
Center of Education Ministry of China, Guangdong Provincial Key
Laboratory on Brain Function Repair and Regeneration,
Neurosurgery Institute, Department of Neurosurgery, Zhujiang
Hospital, Southern Medical University, Guangzhou, Guangdong,
China
3
Monash Medical Centre, Monash University, Clayton, Victoria,
Australia
 Eur Radiol

Keywords Intracranial aneurysm . Radiomics . Rupture . Stroke . Machine learning

Abbreviation This study aims to evaluate the radiomics differences be-

CTA Computed tomography angiography tween ruptured and unruptured aneurysms and explore its po-
GLCM Gray-level co-occurrence matrix tential use in predicting aneurysm rupture.
GLDM Gray-level dependence matrix
GLRLM Gray-level run length matrix
GLSZM Gray-level size zone matrix Methods
NGTDM Neighboring gray tone difference matrix
Data collection

This study was approved by the local institution ethics commit-

Introduction tees and patients’ informed consent was obtained. From our
database, 1031 consecutive cases between 2008 and 2018 were
Intracranial aneurysms are present in 3–7% of the population reviewed. The selection criteria included a confirmed diagnosis
[1]. Though the rupture rates of aneurysms are low, the rupture of one or more intracranial aneurysms. The exclusion criteria
of aneurysms can cause mortality rate as high as 65% [2–4]. included incomplete record of follow-up, cases treated immedi-
Surgical or endovascular treatments for aneurysms are effec- ately after diagnosis and insufficient image quality for analysis.
tive but still carry the risk of complications [3]. Given the high In the current study, unruptured aneurysms were defined as
prevalence and catastrophic consequence of rupture, identifi- aneurysms which remained unruptured and asymptomatic for
cation of rupture-prone aneurysms is of vital importance. at least 2 years during follow-up. Ruptured aneurysms were
Morphology and hemodynamics have been shown to be asso- defined as aneurysms which ruptured during yearly follow-up.
ciated with aneurysm rupture [5–10]. There are other risk Since aneurysm morphologies differ significantly before and
factors such as hypertension [11], blood lipid level [12], alco- after rupture [25], only the pre-rupture images of ruptured aneu-
hol consumption, and smoking [13–15]. Based on these risk rysms were analyzed. One hundred twenty-two aneurysms (93
factors, various risk evaluation methods have been proposed. unruptured, 29 ruptured) were selected.
The PHASES score is among the most quoted which is de-
rived based on several large cohort studies [16]. However, the Acquisition of morphological features and radiomics
prediction performance still needs to be improved [17]. features
Artificial intelligence has been applied to the detection, risk
management, and treatment planning of intracranial aneu- The overall workflow of the current study is shown in Fig. 1.
rysms [18]. Radiomics is an emerging analysis technique CTA images were acquired from GE computerized tomograph-
which extracts features from images and analyzes them in a ic (CT) scanners (GE Healthcare). All images were examined
data-driven approach. It has been proved to be useful in many by two independent interventional neuroradiologists.
fields including cancer prognosis, radiotherapy, and assess- Conventional morphological parameters were measured direct-
ment of cancer genetics. Recently, it has been found that shape ly from 3-dimensional CTA images according to the definitions
features derived from radiomics can help predict intracranial in previous study [26], as shown in Table 1. The inter-rater
aneurysm rupture and embolization outcome from digital sub- reliability for morphology measurement was assessed by
traction angiography (DSA) images [19, 20]. However, the intraclass correlation and was found to be in good agreement
invasiveness and radiation make DSA less frequently used between the two neuroradiologists (ICC = 0.865 95% 0.826–
for follow-up of unruptured aneurysms [21]. Moreover, shape 0.904). All images were resampled to same resolution
features included in previous studies are only a small fraction (0.33 mm × 0.33 mm × 0.5 mm). The aneurysm region was
of the abundant features that can be derived from images. segmented using 3D Slicer (version 4.10.2; http://www.slicer.
There are still many features, such as first-order histogram org) by the two neurosurgeons. The segmentation mask and the
and second-order texture features, which remain unexplored. original images were then input into the PyRadiomics module
Machine learning algorithms are designed to analyze complex in Python for extraction of radiomics features. A total of 107
relationships of underlying data. There has been growing in- radiomics features were computed for the 3-dimensional region
terest in the use of machine learning to predict aneurysm rup- of interest (see details in Supplemental Table 1), which can be
ture using features from morphologies and hemodynamics categorized into three major classes: shape descriptors, first-
[19, 22–24]. However, there is still no study applying order histogram descriptors, and second-order texture pattern
radiomics-based machine learning on CTA to predict aneu- descriptors. The definitions of radiomics features can be found
rysm rupture. from documentation online [27].
Eur Radiol

Fig. 1 Flowchart of the overall methodology

Construction of prediction models constructed with incremental inclusion of features. Model A

was the baseline model constructed using conventional mor-
To determine the effectiveness of radiomics features in phological parameters measured directly from images. Model
predicting aneurysm stability, three prediction models were B was constructed with addition of shape features derived
from radiomics. Model C was constructed with further inclu-
sion of first-order histogram features and second-order texture
Table 1 Baseline characteristics of ruptured and unruptured aneurysms features. All variables were first screened by statistical analy-
Ruptured (29) Unruptured (93) p sis. Only variables with statistical significance at the level of
p < 0.05 in the univariate analysis were selected as input var-
Age 54.20 ± 13.10 55.75 ± 10.37 0.513 iables into machine learning models. Continuous variables
Female 12 60 0.566 were normalized to the range of 0 to 1 before training. Since
Size (mm) 9.73 ± 7.55 5.03 ± 2.99 < 0.001 our dataset size is relatively small, we used repeated ten-fold
Neck width (mm) 5.23 ± 2.32 4.27 ± 1.47 0.046 cross-validation to reduce overfitting, which has been proved
Parent artery diameter (mm) 2.73 ± 0.68 3.08 ± 0.81 0.048 to be stable and representative even for small dataset [28]. In
Size ratio 3.61 ± 2.69 1.71 ± 1.00 < 0.001 ten-fold cross-validation, the whole dataset was divided into
Aspect ratio 2.03 ± 1.88 1.12 ± 0.53 0.005 ten folds of equal size. All the folds except one were used to
Type (bifurcation) 16 33 0.059 train the machine learning model. The trained model was then
Irregular shape 10 17 0.066 evaluated on the held-out fold (also termed validation fold)
Multiplicity 15 61 0.179 which was not used in the training. This process was iterated
Location 0.027 until each fold had been used as validation fold and the aver-
ICA 4 35 age performance was calculated. To further avoid overopti-
MCA 10 24 mistic results for a certain partition, the above cross-validation
PComA 4 20 process was repeated for 100 times, each time with a new
AComA 8 11 random division of the data into ten folds.
BA 3 2 We adopted three machine learning algorithms: general-
ized logistic regression, LASSO regression, and ridge
 Eur Radiol

regression in our training procedures. LASSO and ridge re- was compared with model C in terms of area under the receiv-
gression are developed from logistic regression with the use of er operating characteristic curve (AUROC).
regularization. Regularization can shrink the less important
features’ coefficients to zero and remove redundant features. Statistical analysis
This is particularly helpful when we deal with a large number
of features compared with the number of training samples. All features were compared between ruptured and unruptured
The model training and evaluation workflow are illustrated cases using univariate analyses. For binary or categorical fea-
in Fig. 2. Model A, model B, and model C were compared tures, Fisher exact test or chi-square test was performed. For
in terms of the area under the receiver operating characteristic continuous features, they were first examined with the
curve (AUROC). We also presented the comparison in terms Shapiro-Wilk test to determine normality, followed by the
of the area under the precision-recall curve (AUPRC), which Student t test or Mann-Whitney U test. A p value < 0.05 was
is more informative than ROC when evaluating binary classi- considered statistically significant. Comparison between multi-
fier on imbalanced data [29]. The sensitivity and specificity of ple groups was corrected by Bonferroni correction. Multivariate
each model were determined at the optimal cutoff point on the logistic regression was performed with a backward stepwise
AUROC curve, defined as the point at which the Youden method. Variable collinearity was assessed by Pearson’s corre-
index (J = sensitivity + specificity − 1) reaches its maximum. lation test. The comparison of different models’ performance in
Since the consequence of missing a rupture-prone aneurysm is the 100 repeats was examined by Wilcoxon signed-ranks test as
more severe, we have also presented precision at sensitivity suggested by a previous study [30]. Statistical analyses were
equal to 100%. The importance of each variable was ranked performed using SPSS 24.0 (IBM Corp).
by the average magnitude of coefficients in the 100 repeats.
Having too many features can increase the model complex-
ity and limit its practical use in clinical settings. Therefore, a Results
simplified model was also derived. First, the collinearity of the
top ten most important features in model C was examined. The statistics of the unruptured and ruptured aneurysms are
Linearly independent features were input to multivariate re- presented in Table 1. Aneurysm size, neck width, parent artery
gression to identify independent and significant variables. A diameter, location, size ratio, and aspect ratio were significant-
simplified model using only independent and significant var- ly different between the two groups. For the 107 radiomics
iables was trained and evaluated again using the method pre- features, 89 of them showed significant difference between
viously described. The performance of the simplified model the two groups, which include shape descriptors, first-order

Fig. 2 Schematics of training and

evaluation procedures for models
A, B, and C
Eur Radiol

histogram descriptors, and texture descriptors. A heat map size ratio, and aspect ratio were the three most important var-
was constructed to show the association between radiomics iables. For model B, sphericity, location on ICA, and surface-
features and aneurysm rupture status, as shown in Fig. 3. For to-volume ratio were the important variables. For model C,
details of each feature, please see Supplemental Table 1. gray-level co-occurrence matrix maximum probability
To investigate whether radiomics features are useful in (GLCM Max Probability), location on ICA, and gray-level
predicting aneurysm stability in addition to aneurysm mor- co-occurrence matrix maximal correlation coefficient
phology, three models based on different features were con- (GLCM MCC) were the three most important variables. It
structed. Model A (baseline model) achieved an average should be noted that location of ICA and AComA appeared
AUROC of 0.767 (95% CI 0.754–0.779). Model B showed as important features in all three models. In all three models,
an average AUROC of 0.807 (95% CI 0.795–0.818), which is ICA was associated with negative coefficient (decreased risk)
significantly better than model A (p < 0.001). The difference and AComA was associated with positive coefficient (in-
between model A and model B is the inclusion of radiomics- creased risk).
derived shape features. Model C showed the best AUROC The correlations between model C’s top ten features are
among the three models at 0.879 (95% CI 0.871–0.888), sig- shown in Supplemental Fig. 1. We observed that some
nificantly better than model B (p < 0.001). The AUPRC of the radiomics features were linearly dependent on each other.
three models were 0.593 (95% CI 0.576–0.610), 0.631 (95% For linearly correlated features, we only selected the ones with
CI 0.615–0.649), and 0.763 (95% CI 0.748–0.777), respec- factors of greatest importance. As a result, only four features
tively. Similarly, model C was significantly better than model (ICA, AComA, GLCM Max Probability, and GLSZM Zone
B and model A (p < 0.001), which was consistent with the Percentage) were input into multivariate analysis. Multivariate
result of AUROC. The performances of the models were regression analysis showed that GLCM Max Probability and
summarized in Table 2. If a high sensitivity threshold is GLSZM Zone Percentage were independent and significant
chosen (~ 100%), models A, B, and C will have a positive variables (p < 0.05) while location being AComA was mar-
predictive value of 0.388, 0.441, and 0.518, respectively. ginally significant (p = 0.067). The multivariate analysis re-
The difference between model B and model C is the inclusion sults were shown in Table 3. The simplified model construct-
of first-order features and second-order texture features. The ed with the two radiomics features (GLCM Max Probability
three ROC curves are shown in Fig. 4. Our results suggested and GLSZM Zone Percentage) achieved an AUROC of 0.876
that the inclusion of radiomics features can significantly improve (95% CI 0.866–0.885), which indicates that the simplified
prediction performance. It also showed that model C is both model performed as well as model C. The simplified model
more sensitive and specific compared with the other 2 models. had a ROC curve almost identical to that of model C. There is
The ten most important variables in each model and their no statistically significant difference between the performance
importance are shown in Fig. 5. For model A, aneurysm size, of simplified model and model C.

Fig. 3 Heat map of statistically significant radiomics features selected for machine learning. Each row and column correspond to one normalized
radiomics feature and one aneurysm, respectively
 Eur Radiol

Table 2 Summary of performances of models A, B, C and the simplified model

AUROC AUPRC Sensitivity Specificity Precision @100% sensitivity

Model A 0.767 (0.754–0.779) 0.593 (0.576–0.610) 62.4% 77.0% 38.8%

Model B 0.807 (0.795–0.818) 0.631 (0.615–0.649) 67.6% 77.0% 44.1%
Model C 0.879 (0.871–0.888) 0.763 (0.748–0.777) 71.6% 88.0% 51.8%
Simplified model 0.876 (0.866–0.885) 0.766 (0.753–0.781) 70.3% 88.0% 50.9%

Discussion radiomics features derived from CTA images can predict an-
eurysm stability with high discrimination (AUROC = 0.879).
In this study, we have shown the significant differences in Various aneurysm morphological parameters have been
radiomics between ruptured and unruptured aneurysms. We suggested to be related to rupture including size, size ratio,
further demonstrated that a machine learning model based on aspect ratio, shape regularity, and locations [2–6, 9, 26].

Fig. 4 ROC characteristic curves

(a) and precision-recall curves (b)
of models A, B, and C. Model A
was constructed with morphology
features. Model B was construct-
ed with morphology features and
radiomics shape features. Model
C was constructed with morphol-
ogy features, radiomics shape
features, first-order histogram
features, and second-order texture
features. The black hollow dots
on the curves indicate the thresh-
old determined by Youden index
Eur Radiol

Fig. 5 Top ten features and feature coefficients (feature importance) in models A, B, and C

Model A derived from morphological parameters showed a three most important variables to predict aneurysm stability.
similar result where size, size ratio, and aspect ratio were the Model B was constructed with morphological features plus
 Eur Radiol

Table 3 Multivariate analysis of the top 10 important variables in that these texture patterns may indirectly represent the hemo-
model C
dynamics within the aneurysm. Hemodynamics parameters
OR (95% CI) p Value have been associated with aneurysm rupture. However, most
studies of aneurysm hemodynamics employed computer sim-
GLCM Max Probability 7.365 (1.289–42.084) 0.025 ulations which are based on assumed properties of flow [5–8].
GLSZM Zone Percentage 0.001 (0.000–0.012) 0.001 In contrast, radiomics features are derived from images, which
AComA 3.605 (0.915–14.20) 0.067 are more objective and realistic.
ICA 0.647 (0.153–2.742) 0.554 The simplified model achieved a similar level of perfor-
mance as model C. This can be explained by the fact that
many radiomics features were correlated to each other and
shape descriptors derived from radiomics. Statistical analysis redundant, as suggested by a 2018 study [33]. Removing re-
showed that most of these shape descriptors were significantly dundant features can simplify the model, improve its interpret-
related to aneurysm rupture, similar to the result obtained by ability, and facilitate practical use in clinical setting. In the
Liu et al [19]. Conventionally, size, size ratio, and aspect ratio current study, machine learning models required input of
have been associated with rupture risk. A question may be radiomics features, which were extracted from images using
raised as to whether these shape descriptors are just redundant PyRadiomics. Since the preprocessing of images is not trivial
surrogates of conventional aneurysm morphologies or wheth- to doctors, to make the model easier to use in clinical settings,
er they encompass extra information not contained in conven- a web-based calculator for a simple risk scoring scale can be
tional morphological parameters. By comparing models A derived from machine learning models, as illustrated in previ-
and B, we found that the additional use of radiomics-derived ous work [34].
shape descriptors can better predict aneurysm rupture. Liu et al have previously shown that using shape features
We further examined whether the inclusion of more derived from radiomics can predict aneurysm rupture [19].
radiomics features can further improve the prediction model. The difference between our work and previous work is mainly
By comparing models C and B, we found that including these in three aspects. First, previous work is based on DSA images.
descriptors significantly improves the prediction performance We used CTA images which are more commonly used in the
(0.879 vs 0.807, p < 0.001). monitoring of untreated aneurysms. Second, our study was
In multivariate analysis, AComA appeared as marginally based on mid-term follow-up. For unruptured aneurysms, they
significant with OR = 3.605 (p = 0.067), which agrees with pre- have remained stable for at least 2 years. For aneurysms that
vious study indicating AComA as an independent risk factor ruptured during follow-up, since post-rupture aneurysm mor-
[2]. Only GLCM Max Probability (OR = 7.365) and GLSZM phology may change [25], in this study, only pre-rupture im-
Zone Percentage (OR = 0.001) remained as independent and ages within 1 year before rupture were used, which makes it
significant variables. In ruptured group, GLCM Max more accurate and applicable to real-world situation. Third,
Probability was significantly higher (p < 0.001) and GLSZM we examined extra features (first-order histogram and second-
Zone Percentage was significantly lower (p < 0.001) than their order texture features) in our model and we proved that these
counterpart in the unruptured group. The gray-level co-occur- extra features could improve the model performance signifi-
rence matrix describes the probability distribution for neighbor- cantly. Besides radiomics, deep learning has also shown
ing voxels to have similar intensity levels. A higher value of promise in aneurysm detection on CTA images [35].
GLCM Max Probability indicates that a certain texture pattern Radiomics integrated with deep learning may provide a new
is more salient in the aneurysm region. The gray-level size zone method in the risk management of intracranial aneurysms.
matrix quantifies gray-level zones (region of similar voxel in-
tensities) in images. A lower value of GLSZM Zone Percentage
indicates fewer zones (region of similar voxel intensities) in the Limitations
aneurysm region, which also means higher heterogeneity.
Therefore, these two texture descriptors suggested that This study has several limitations. Although we believe our
rupture-prone aneurysms were associated with higher heteroge- study has included a relatively large number of follow-up
neity in voxel intensity and appeared to show certain texture aneurysms, from statistical point of view, the number of cases
patterns. Two typical images of aneurysms with heterogenous included in the current study is relatively small and may not
pattern and two typical images of aneurysms with homoge- fully represent the variability among ruptured aneurysms. We
neous patterns are shown in Supplemental Fig. 2. did not include some recognized clinical risk factors such as
The texture patterns within the aneurysm region are likely hypertension and hyperlipidemia, which should be considered
to be caused by the uneven distribution of contrast. in future study. The follow-up period of 2 years is relatively
Inhomogeneous contrast pattern has been suggested to be re- short, and thus, our result may not be fully applicable for
lated to turbulent flow [31, 32], which has led us to propose predicting the long-term stability of aneurysms. As this is
Eur Radiol

preliminary study, further validation using prospective, multi- 4. Korja M, Kivisaari R, Rezai Jahromi B, Lehto H (2017) Natural
history of ruptured but untreated intracranial aneurysms. Stroke. 48:
center design with larger number of cases recruited should be
1081–1084. https://doi.org/10.1161/STROKEAHA.116.015933
performed before its use in clinical practice. 5. Xiang J, Natarajan SK, Tremmel M et al (2011) Hemodynamic-
morphologic discriminants for intracranial aneurysm rupture.
Stroke. 42:144–152. https://doi.org/10.1161/STROKEAHA.110.
592923
Conclusions 6. Varble N, Tutino VM, Yu J et al (2018) Shared and distinct rupture
discriminants of small and large intracranial aneurysms. Stroke. 49:
856–864. https://doi.org/10.1161/STROKEAHA.117.019929
There were significant radiomics differences between rup- 7. Cebral JR, Mut F, Weir J, Putman C (2011) Quantitative character-
tured and unruptured aneurysms. Machine learning models ization of the hemodynamic environment in ruptured and
based on radiomics can be used to predict aneurysm rupture. unruptured brain aneurysms. AJNR Am J Neuroradiol 32:145–
Shape features from radiomics can significantly improve the 151. https://doi.org/10.3174/ajnr.A2419
8. Takao H, Murayama Y, Otsuka S et al (2012) Hemodynamic dif-
prediction power compared with conventional morphological ferences between unruptured and ruptured intracranial aneurysms
features. The incorporation of histogram and texture features during observation. Stroke. 43:1436–1439. https://doi.org/10.1161/
may further improve the prediction performance. These STROKEAHA.111.640995
models showed promise in being applied to a decision support 9. Zhang X, Karuna T, Yao ZQ et al (2019) High wall shear stress
beyond a certain range in the parent artery could predict the risk of
system.
anterior communicating artery aneurysm rupture at follow-up. J
Neurosurg 131:868–875. https://doi.org/10.3171/2018.4.
Funding Funding was provided through the National Health and Medical JNS173179
Research Council (NHMRC) Project (Grant ID: APP1157566). 10. Miura Y, Ishida F, Umeda Y et al (2013) Low wall shear stress is
independently associated with the rupture status of middle cerebral
Compliance with ethical standards artery aneurysms. Stroke. 44:519–521. https://doi.org/10.1161/
STROKEAHA.112.675306
Guarantor The scientific guarantor of this publication is Yi Qian who 11. Tada Y, Wada K, Shimada K et al (2014) Roles of hypertension in
works in Macquarie University. the rupture of intracranial aneurysms. Stroke. 45:579–586. https://
doi.org/10.1161/STROKEAHA.113.003072
12. Can A, Castro VM, Dligach D et al (2018) Lipid-lowering agents
Conflict of interest The authors of this manuscript declare no relation- and high HDL (high-density lipoprotein) are inversely associated
ships with any companies whose products or services may be related to with intracranial aneurysm rupture. Stroke. 49:1148–1154. https://
the subject matter of the article. doi.org/10.1161/STROKEAHA.117.019972
13. Can A, Castro VM, Ozdemir YH et al (2018) Alcohol consumption
Statistics and biometry No complex statistical methods were necessary and aneurysmal subarachnoid hemorrhage. Transl Stroke Res 9:13–
for this paper. 19. https://doi.org/10.1007/s12975-017-0557-z
14. Can A, Castro VM, Ozdemir YH et al (2017) Association of intra-
Informed consent Written informed consent was waived by the cranial aneurysm rupture with smoking duration, intensity, and ces-
Institutional Review Board. sation. Neurology. 89:1408–1415. https://doi.org/10.1212/WNL.
0000000000004419
Ethical approval Approval for this study was obtained from the local 15. Juvela S (2019) Growth and rupture of unruptured intracranial an-
Institutional Review Board of Macquarie University. eurysms. J Neurosurg 131:843–851. https://doi.org/10.3171/2018.
4.JNS18687
Methodology 16. Greving JP, Wermer MJ, Brown RD Jr et al (2014) Development of
• retrospective the PHASES score for prediction of risk of rupture of intracranial
• cross sectional study aneurysms: a pooled analysis of six prospective cohort studies.
• performed at one institution Lancet Neurol 13:59–66
17. Bijlenga P, Gondar R, Schilling S et al (2017) PHASES score for
the management of intracranial aneurysm: a cross-sectional popu-
lation-based retrospective study. Stroke. 48:2105–2112
18. Shi Z, Hu B, Schoepf UJ, Savage RH et al (2020) Artificial intel-
References ligence in the management of intracranial aneurysms: current status
and future perspectives. AJNR Am J Neuroradiol 41(3):373–379
1. Li MH, Chen SW, Li YD et al (2013) Prevalence of unruptured 19. Liu QL, Jiang P, Jiang YH et al (2019) Prediction of aneurysm
cerebral aneurysms in Chinese adults aged 35 to 75 years: a cross- stability using a machine learning model based on PyRadiomics-
sectional study. Ann Intern Med 159:514–521. https://doi.org/10. derived morphological features. Stroke. 50:2314–2321. https://doi.
7326/0003-4819-159-8-201310150-00004 org/10.1161/STROKEAHA.119.025777
2. Morita A, Kirino T, Hashi K et al (2012) The natural course of 20. Zhang Y, Ma C, Liang S et al (2018) Morphologic feature elonga-
unruptured cerebral aneurysms in a Japanese cohort. N Engl J tion can predict occlusion status following pipeline embolization of
Med 366:2474–2482. https://doi.org/10.1056/NEJMoa1113260 intracranial aneurysms. World Neurosurg 119:934–940. https://doi.
3. Wiebers DO, Whisnant JP, Huston J 3rd et al (2003) Unruptured org/10.1016/j.wneu.2018.08.007
intracranial aneurysms: natural history, clinical outcome, and risks 21. Thompson BG, Brown RD Jr, Amin-Hanjani S et al (2015)
of surgical and endovascular treatment. Lancet. 362:103–110. Guidelines for the management of patients with unruptured intra-
https://doi.org/10.1016/s0140-6736(03)13860-3 cranial aneurysms: a guideline for healthcare professionals from the
 Eur Radiol

American Heart Association/American Stroke Association. Stroke. 29. Saito T, Rehmsmeier M. The precision-recall plot is more informa-
46(8):2368–2400 tive than the ROC plot when evaluating binary classifiers on imbal-
22. Liu J, Chen Y, Lan L et al (2018) Prediction of rupture risk in anced datasets. PloS One. 2015;10(3)
anterior communicating artery aneurysms with a feed-forward arti- 30. Demšar J (2006) Statistical comparisons of classifiers over multiple
ficial neural network. Eur Radiol 28:3268–3232. https://doi.org/10. data sets. J Mach Learn Res 7(Jan):1–30
1007/s00330-017-5300-3 31. George E, Giannopoulos AA, Aghayev A et al (2016) Contrast
23. Kim HC, Rhim JK, Ahn JH et al (2019) Machine learning applica- inhomogeneity in CT angiography of the abdominal aortic aneu-
tion for rupture risk assessment in small-sized intracranial aneu- rysm. J Cardiovasc Comput Tomogr 10:179–183. https://doi.org/
rysm. J Clin Med 8:683. https://doi.org/10.3390/jcm8050683 10.1016/j.jcct.2015.11.006
24. Detmer FJ, Chung BJ, Mut F et al (2018) Development and internal 32. Aghayev A, Giannopoulos AA, Gronsbell J et al (2018) Common
validation of an aneurysm rupture probability model based on pa- first-pass CT angiography findings associated with rapid growth
tient characteristics and aneurysm location, morphology, and hemo- rate in abdominal aorta aneurysms between 3 and 5 cm in largest
dynamics. Int J Comput Assist Radiol Surg 13:1767–1779. https:// diameter. AJR Am J Roentgenol 210:431–437. https://doi.org/10.
doi.org/10.1007/s11548-018-1837-0 2214/AJR.17.18094
25. Skodvin TØ, Johnsen LH, Gjertsen Ø, Isaksen JG, Sorteberg A 33. Berenguer R, Pastor-Juan MD, Canales-Vázquez J et al (2018)
(2017) Cerebral aneurysm morphology before and after rupture: Radiomics of CT features may be nonreproducible and redundant:
nationwide case series of 29 aneurysms. Stroke. 48:880–886. influence of CT acquisition parameters. Radiology. 288:407–415.
https://doi.org/10.1161/STROKEAHA.116.015288 https://doi.org/10.1148/radiol.2018172361
26. Dhar S, Tremmel M, Mocco J et al (2008) Morphology parameters
34. Chen T, Li X, Li Y et al (2019) Prediction and risk stratification of
for intracranial aneurysm rupture risk assessment. Neurosurgery.
kidney outcomes in IgA nephropathy. Am J Kidney Dis 74(3):300–
63:185–197. https://doi.org/10.1227/01.NEU.0000316847.64140.
309
81
27. Zwanenburg A, Leger S, Vallières M, Löck S (2016) Image bio- 35. Park A, Chute C, Rajpurkar P et al (2019) Deep learning–assisted
marker standardisation initiative - feature definitions. In eprint diagnosis of cerebral aneurysms using the HeadXNet model.
arXiv:1612.07003 JAMA Netw Open 2(6):e195600
28. Molinaro AM, Simon R, Pfeiffer RM (2005) Prediction error esti-
mation: a comparison of resampling methods. Bioinformatics 21: Publisher’s note Springer Nature remains neutral with regard to jurisdic-
3301–3307. https://doi.org/10.1093/bioinformatics/bti499 tional claims in published maps and institutional affiliations.