ORIGINAL ARTICLE

NOVEL HEAD AND NECK CANCER SURVIVAL ANALYSIS

APPROACH: RANDOM SURVIVAL FORESTS VERSUS COX
PROPORTIONAL HAZARDS REGRESSION
Frank R. Datema, MD,1 Ana Moya, PhD,2 Peter Krause, PhD,3 Thomas Bäck, PhD,3 Lars Willmes,4
Ton Langeveld, MD, PhD,5 Robert J. Baatenburg de Jong, MD, PhD,1 Henk M. Blom, MD, PhD6
1
Department of Otorhinolaryngology–Head and Neck Surgery, Erasmus Medical Center, Rotterdam,
The Netherlands. E-mail: [email protected]
2
WAZ Mediengruppe GmbH, Essen, Germany
3
Divis Intelligent Solutions GmbH, Dortmund, Germany
4
Intulion Solutions GmbH, Dortmund, Germany
5
Department of Otorhinolaryngology–Head and Neck Surgery, Leiden University Medical Center, The Netherlands
6
Department of Otorhinolaryngology, Haga Ziekenhuis, The Hague, The Netherlands

Accepted 22 October 2010

Published online 14 February 2011 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/hed.21698

bidity are very important covariates in the model, whereas sex,

Abstract: Background. Electronic patient files generate an
mild comorbidity, and a supraglottic larynx tumor are less
enormous amount of medical data. These data can be used
important. A discrepancy arose regarding the importance of
for research, such as prognostic modeling. Automatization of
M1 classification (see Discussion).
statistical prognostication processes allows automatic updating
Conclusion. Both approaches delivered similar error rates.
of models when new data is gathered. The increase of power
The Cox model gives a clinically understandable output on
behind an automated prognostic model makes its predictive
covariate impact, whereas RSF becomes more of a ‘‘black
capability more reliable. Cox proportional hazard regression is
box.’’ RSF complements the Cox model by giving more insight
most frequently used in prognostication. Automatization of a
and confidence toward relative importance of model covari-
Cox model is possible, but we expect the updating process to
ates. RSF can be recommended as the approach of choice in
be time-consuming. A possible solution lies in an alternative
automating survival analyses. V C 2011 Wiley Periodicals, Inc.
modeling technique called random survival forests (RSFs). RSF
Head Neck 34: 50–58, 2012
is easily automated and is known to handle the proportionality
assumption coherently and automatically. Performance of RSF Keywords: Random survival forests; Cox regression;
has not yet been tested on a large head and neck oncological squamous cell carcinoma; survival prediction; relative
dataset. This study investigates performance of head and neck importance
overall survival of RSF models. Performances are compared to
a Cox model as the ‘‘gold standard.’’ RSF might be an interest-
ing alternative modeling approach for automatization when per-
An important question that the newly diagnosed
formances are similar. patient with head and neck cancer will most likely ask
Methods. RSF models were created in R (Cox also in his or her physician is: ‘‘How are my survival chances?’’
SPSS). Four RSF splitting rules were used: log-rank, conserva- The answer to this question is not that simple because
tion of events, log-rank score, and log-rank approximation. it is based on the impact and interaction of multiple fac-
Models were based on historical data of 1371 patients with pri- tors. Roughly, these factors can be divided into 3 main
mary head-and-neck cancer, diagnosed between 1981 and categories: tumor-specific, patient-specific, and treat-
1998. Models contain 8 covariates: tumor site, T classification, ment-specific. Even for the most experienced head and
N classification, M classification, age, sex, prior malignancies, neck surgeon, it remains a difficult task to determine
and comorbidity. Model performances were determined by the impact and relevance of each applicable covariate
Harrell’s concordance error rate, in which 33% of the original on overall survival.
data served as a validation sample.
To aid the physician in this survival prediction di-
Results. RSF and Cox models delivered similar error rates.
lemma, there are statistical survival analyses. The
The Cox model performed slightly better (error rate, 0.2826).
The log-rank splitting approach gave the best RSF perform-
most popular and broadly used survival analysis is the
ance (error rate, 0.2873). In accord with Cox and RSF models, Cox proportional hazards regression (Cox model). Our
high T classification, high N classification, and severe comor- most up-to-date survival model is a Cox model contain-
ing 8 covariates: tumor location, tumor size (T classifi-
cation), regional metastasis (N classification), and
Correspondence to: F. R. Datema distant metastasis (M classification), sex, age at diagno-
Conflict of interest: none. sis, prior malignancies, and comorbidity severity (Adult
V
C 2011 Wiley Periodicals, Inc. Comorbidity Evaluation-27 [ACE-27]). This Cox model

50 Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012
is based on the historical data of 1371 Dutch patients The second purpose of this study was to gain fur-
with primary head and neck squamous cell carcinoma ther insights into the relative importance of each
and is validated internally by use of a bootstrap proce- model covariate according to each modeling
dure. The predictive accuracy of the model is illustrated technique.
by a Harrell’s concordance index (C-index) of 0.73.1
To further improve individualized prognosis pre- MATERIALS AND METHODS
diction, we need to enhance the performance of our
Cox model. We believe that this can be achieved in Cox Proportional Hazards Regression (Cox
multiple ways. The fist method is addition of covari-
Model). The Cox model is the most general of
ates with a significant impact on overall survival.
regression models because it is not based on any
This method was recently illustrated by the addi-
assumptions concerning the nature or shape of the
tion of comorbidity as a covariate in our Cox model,
underlying survival distribution. The Cox model can
which resulted in a modest but significant accuracy
be written as:
improvement of 3.0%.1,2 The second method is
extension of the original database to increase power
hðt; xÞ ¼ h0 ðtÞ
HR ¼ h0 ðtÞ
expðX1 b1 þX2 b2 þ::þXm bm Þ
and to create a more heterogeneous study popula-
tion. This is a worthwhile but time-consuming
effort, especially in a single-center study setting. Consider the model to consist of 2 parts. First, the
Perhaps a solution lies in the introduction and baseline hazard h0(t) describes the hazard (risk of
implementation of electronic patient files (EPFs). dying) at a specified point in time for a reference
When an EPF is organized to collect standardized patient with all covariate values equal to 0.
medical data that can automatically be exported into Second, effect parameters (b1, b2, : : :, bm) describe
a study database, it can greatly assist in prognostic how the hazard varies in response to the models’
modeling. First of all, a process of extracting patterns covariates (X1, X2, : : :, Xm) and this is expressed as
from data, called ‘data mining’ allows extrapolation of (X1b1, X2b2, : : :, Xmbm). The model can be linearized
relevant covariates hidden in the EPF-generated by dividing both sides of the equation with h0(t) and
database in an automated fashion. Second, by autom- then taking the natural logarithm of both sides. Then
atization of the survival analysis itself, periodic feed- the model is written as:
back on model performance can be given after the
log hðt; xÞ ¼ log h0 ðtÞ þ X1 b1 þ X2 b2 þ    þ Xm bm
addition of newly identified covariates and additional
patient data.
Automatization of the generally used Cox model is Because the functional form of the baseline hazard is
possible but will require substantial subjective input not given but determined from the data, the Cox
from the user when it is applied to a dataset in which model is called semiparametric. A parametric form
covariates are highly interrelated. Perhaps other sur- for the effect of the covariates on the resultant haz-
vival analyses that deliver accurate predictive models ard, however, is assumed as follows:
are more suited for automatization. In this article, we
therefore explore a relatively new survival analysis 1. A multiplicative relationship between the underly-
from data mining, called random survival forests ing hazard function and the log-linear function of
(RSFs). the covariates exists. This assumption is called the
RSF is derived from the Random Forest Modeling proportionality assumption. In practical terms, it
technique, introduced by Breiman in 2001.3 RSF is assumed that any 2 subjects have hazard func-
proved to be useful in the determination of risk fac- tions whose ratio is a constant proportion that
tors for disease-free survival of patients with breast depends on the covariates and not on the time.
cancer and uncovered highly complex interrelation- 2. There is a log-linear relationship between the in-
ships between covariates in a coronary artery dependent covariates and the underlying hazard
study.4,5 Furthermore, RSF is a survival analysis that function.
can easily be automated. To our knowledge, it is the 3. The effect of the covariates is the same at all
first time that RSF has been applied to a large times.
historical head and neck cancer dataset and is, there-
fore, considered to be a novel approach. In this study, the Cox model was created in SPSS
for Windows (version 16.0) and in R (version 2.10.0).
The 8 covariates for the Cox model were significant in
Study Objectives. The main purpose of this study was prior univariate analyses. Tumor location, tumor size
to test the predictive performance of RSF models and (T classification), regional metastasis (N classification),
compare them with the performance of the Cox model distant metastasis (M classification), age at diagnosis,
as a ‘‘gold standard.’’ When RSF delivers a better or sex, prior tumors, and comorbidity severity (ACE-27
comparable predictive performance, it can be the sur- grade) were considered categorical covariates with
vival analysis of choice for automatization. their reference category set as that covariate with the

Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012 51
 FIGURE 1. Graphical presentation of the basic Random Survival Forest (RSF) algorithm. OOB, out-of-bag data.

best prognosis. Age at diagnosis was considered a con- and is populated by cases with similar survival. In
tinuous covariate. the RSF algorithm implemented in this paper, the
As in a multiple regression, the p value was used tree reaches a saturation point when a terminal node
as a criterion to decide whether a covariate was stat- (the most extreme node in a saturated tree) has at
istically significant (p  .01) or not (p > .01). least 1 death with unique survival times. By grouping
The Z-statistic was used to determine which cova- hazard estimates from terminal nodes, a cumulative
riates were most informative. The Z-value marks the hazard function for the tree can be calculated. In this
ratio of each regression coefficient to its SE within study, 1000 trees were grown in a first trial to gener-
the Cox model. High positive Z-values indicate in- ate the ensemble cumulative hazard. Results were
formative covariates. later compared to a replication process with 100 trees
to reduce computation speed. The ensemble cumula-
tive hazard function is produced by the average over
Random Survival Forests. RSFs is an ensemble tree all trees (Figure 1).5,6
method for the analysis of right censored survival Version 3.6.0 of the RSF R-package provides 4
data. In RSF, randomization is introduced in 2 forms. splitting rules to choose from.6 (1) The log-rank split-
First, a randomly selected bootstrap sample (approxi- ting rule splits the nodes by maximization of the log-
mately 67% of the original data) is used for growing rank test statistic. The larger the value, the greater
the tree. This bootstrap sample can be seen as the the difference between the 2 groups and the better the
root of the tree. Second, the root is split into 2 daugh- split is.7,8 (2) The conservation-of-events splitting rule
ter nodes by using a splitting rule on a randomly splits the nodes by finding daughter nodes closest to
selected covariate. The split is the best when survival the conservation-of-events principle. This principle
difference between the daughter nodes is maximized. states that the sum of the estimated cumulative haz-
Eventually, as the number of tree nodes increases ard function over the observed time points must equal
with every split, and dissimilar cases become sepa- the total number of deaths. The value is small if the 2
rated, each node in the tree becomes homogeneous groups are well separated ( Naftel, Blackstone, and

52 Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012
Turner, unpublished notes). (3) The log-rank score lished in 1969 and contains patient, treatment, and
splitting rule, which splits the nodes using a standar- follow-up data for each patient with cancer diagnosed
dized log-rank statistic.9 (4) The log-rank approxima- in the Leiden University Medical Center. Trained
tion splitting rule. It splits the nodes by using an oncological data managers store these data and safe-
approximation of the log-rank test to reduce guard an up-to-date TNM classification by staging or
computations.8,10 restaging the disease according to the most up-to-date
In this study, RSF models were created with the 4 Union Internationale Contre le Cancer manual (for
mentioned splitting rules. Analyses were performed this study, the fifth edition was most up-to-date at the
with the 8 covariates simultaneously. The same refer- last moment of follow-up). Prior malignancies were
ence categories were used as in the Cox model. coded in the presence of all preceding malignant
The importance of each model covariate was tumors except for basal cell and squamous cell carci-
determined by applying the model on the data that noma of the skin. Comorbidity severity was coded
was not used in building the model. These data (the according to the ACE-27 manual. An overall ACE-27
remaining 33% of the original data) are called out-of- grade 0 corresponds with no comorbidity, grade 1
bag data (OOB-data). High positive importance values with mild comorbidity, grade 2 with moderate comor-
indicate informative covariates, whereas small posi- bidity, and grade 3 with severe comorbidity.11 We
tive or negative importance values indicate noninfor- chose not to score prior malignancies as ACE-27
mative covariates.5,6 comorbidity because this factor is a separate covariate
in the Cox model. The impact of prior malignancies
Model Performance. The quality of a predictive sur- on overall survival would be unjust when scored
vival model is reflected by its prediction error. Har- twice. In this article, we therefore use the term
rell’s concordance index (C-index) can be used to ‘‘adjusted’’ ACE-27. Based on the therapeutic nil hy-
determine the model’s quality. The C-index is related pothesis, the type of treatment was not considered a
to the area under the receiver operating characteris- prognostic factor for our model.12–15
tic curve. It estimates the probability that, in a ran-
domly selected pair of cases, the case that fails (dies)
first had the worst predicted outcome. The Harrell’s RESULTS
concordance error rate is computed as 1 minus the
C-index. Error rates are between 0 and 1, with 0.5 Explorative Data Analyses. The mean age was 62.6
corresponding to a procedure doing no better than years old with an SD of 12.0 years. The majority of
random guessing. A value of 0.0 reflects perfect patients were men (1088; 79.4%). A minority of
accuracy.11 patients had a preceding malignancy outside the head
For both the Cox model and the RSF models, the and neck area (133; 9.7%). Comorbidity was found in
mean and SD of Harrell’s concordance error rate was 500 patients (36.5%) of whom 76 (5.5%) had severe
calculated by using 100 and 10 independent replica- comorbidity. Most tumors were located in the oral
tions. Models were fit on their bootstrap data and the cavity (280; 20.4%) and glottic larynx (442; 32.2%).
prediction error was estimated using the correspond- Further distribution of tumor locations and the TNM
ing OOB-data. In RSF models, the number of trees classification can be found in Table 1.
was equal to 1000 in the first test and 100 in the The mean follow-up time was 12.3 years and the
second. median follow-up time was 5.3 years. During follow-
up, 1048 patients (76.4%) eventually died. The
Head and Neck Cancer Data. For this study, the remaining 323 patients (23.6%) were alive at last fol-
data of 1662 patients diagnosed with primary head low-up.
and neck squamous cell carcinoma at the Leiden Uni-
versity Medical Center between 1981 and 1998 were
available. Patients with esophageal cancer and sub- Cox Proportional Hazards Regression Analysis. Based
glottic cancer were not included because the prognosis on the given p values, the covariate sex does not
of these patients is poor and the number of incom- impact overall survival. Furthermore, based on the
plete TNM classifications in this group was relatively used reference categories, no significant impact on
large. Patients with carcinoma in situ were not overall survival was found from mild comorbidity
included because the prognosis of these patients is (ACE-27 grade 1) and from tumors located in the
exceptionally good. One patient was excluded because nasopharynx, glottic larynx, and supraglottic larynx
there was no follow-up data. The final study popula- (Table 2).
tion consisted of 1371 patients with histologically pro- For the remaining significant covariates, exp(B)
ven primary squamous cell carcinoma of the lip, oral can be interpreted as a multiplicative effect on the
cavity, oropharynx, nasopharynx, hypopharynx, glot- hazard of death. For example, holding all covariates
tic larynx, or supraglottic larynx. The TNM classifica- constant, an additional year of age increases the
tion was obtained from the hospital-based oncological monthly hazard of death by a factor of 4% (exp[B] ¼
documentation. Oncological documentation was estab- 1.04). Similarly, a patient with moderate comorbidity

Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012 53
 model only the log-rank splitting rule was able to con-
Table 1. Demographic data and tumor data of baseline study
population (n ¼ 1371). firm this finding. The glottic larynx is an insignificant
covariate in the Cox model, whereas the conservation
No. (%) of events splitting rule ranked it as 1 of the most im-
Tumor location portant covariates. The conservation of events split-
Lip 123 (9.0) ting rule and log-rank score splitting rule ranked
Oral cavity 280 (20.4) moderate comorbidity (ACE-27 grade 2) as unimpor-
Oropharynx 152 (11.1)
Nasopharynx 41 (3.0)
tant, whereas the Cox model identified it as
Hypopharynx 137 (10.0) important.
Glottic larynx 442 (32.2)
Supraglottic larynx 196 (14.3)
T classification
Cox Model Performance versus Random Survival
T1 516 (37.6) Forrest Model Performance. The Cox model showed
T2 369 (26.9) a slightly better performance than all 4 RSF
T3 208 (15.2) approaches with a Harrell’s Concordance error rate (1
T4 278 (20.3)
N classification
minus C-index) of 0.2826. The performance of each
N0 964 (70.3) RSF approach was very similar with the best error
N1 145 (10.6) rate (0.2873) obtained by the log-rank splitting rule
N2 180 (13.1) with 1000 trees (Table 3). To reduce computation
N3 82 (6.0)
M classification
speed, a second RSF test with 100 trees was per-
M0 1354 (98.8) formed. We observed a slight increase in error rates
M1 17 (1.2) (Table 3).
Sex
Male 1088 (79.4)
DISCUSSION
Female 283 (20.6)
Comorbidity (adjusted) ACE-27 Although at first glance RSF seems to be an unusual
Grade 0: none 782 (57.0) procedure to evaluate right-censored survival data,
Grade 1: mild 239 (17.4)
Grade 2: moderate 185 (13.5)
Grade 3: severe 76 (5.5)
Table 2. Results of the Cox model.
No data 89 (6.5)
Prior malignancies Covariate B Exp (B) Z-value p value 95% CI
Yes 133 (9.7)
No 1238 (90.3) Age 0.038 1.039 11.804 <.01 1.032–1.045
Sex
Abbreviation: ACE-27, Adult Comorbidity Evaluation-27.
Female (RC) – 1.000 – – –
Male 0.066 1.068 0.789 .430 0.907–1.258*
has an increased hazard of 39% (exp[B] ¼ 1.39) com- Tumor location
pared to its reference category (RC). Lip (RC) – 1.000 – – –
Based on the p value and Z-value of each covari- Hypopharynx 0.628 1.874 3.417 <.01 1.307–2.686
Oral cavity 0.471 1.602 2.836 <.01 1.157–2.218
ate, age at diagnosis, T4 classification, N3 classifica- Oropharynx 0.495 1.641 2.757 <.01 1.154–2.334
tion, M1 classification, and severe comorbidity are the Glottic larynx 0.020 1.020 0.128 .855 0.907–1.258*
most important covariates (Table 2). Supraglottic larynx 0.266 1.305 1.555 .125 0.933–1.825*
Nasopharynx 0.211 1.235 0.801 .425 0.737–2.068*
T classification
Random Survival Forrest Analyses. Figure 2, T1 (RC) – 1.000 – – –
Figure 3, Figure 4, and Figure 5 present a graphical T2 0.289 1.335 3.204 <.01 1.119–1.592
T3 0.437 1.549 4.022 <.01 1.251–1.916
output of how each RSF model ranks its covariates by T4 0.708 2.029 6.820 <.01 1.656–2.487
level of OOB-importance, based on 1000 trees. Each N classification
RSF approach shows a slightly different ranking N0 (RC) – 1.000 – – –
order. However, similarities are present. Very impor- N1 0.311 1.365 2.878 <.01 1.104–1.686
N2 0.611 1.843 5.833 <.01 1.500–2.263
tant covariates in nearly all 4 RSF approaches were:
N3 0.873 2.395 6.450 <.01 1.837–3.122
age at diagnosis, T4 classification, N3 classification, M classification
and N2 classification. Unimportant covariates in M0 (RC) – 1.000 – – –
nearly all 4 RSF approaches were: sex, mild comor- M1 1.903 6.707 6.645 <.01 3.826–11.759
bidity (grade 1), tumor location, supraglottic larynx, Prior tumors 0.548 1.730 5.097 <.01 1.401–2.136
Comorbidity
and nasopharynx. The remaining covariates had ACE-27 Grade 0 (RC) – 1.000 – – –
smaller positive importance values with somewhat ACE-27 Grade 1 0.057 1.059 0.648 .524 0.891–1.259*
different ranking within each RSF approach. ACE-27 Grade 2 0.332 1.394 3.624 <.01 1.165–1.669
These findings were generally comparable with ACE-27 Grade 3 0.827 2.286 6.409 <.01 1.775–2.943
the results from the Cox model, with some exceptions. Abbreviations: B, beta; Exp(B), multiplicative factor on the hazard; Z-value, ratio of
M1 classification is a very important and significant regression coefficient to its SE; 95% CI, 95% confidence interval; RC, reference
category.
covariate in the Cox model, whereas in the RSF *Not statistically significant.

54 Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012
FIGURE 2. Out-of-bag data (OOB-data) covariate importance values according to log-rank splitting rule. ACE-27, Adult Comorbidity
Evaluation-27.

considerable empirical evidence has shown random- always the concern whether associations between
ized ensembles to be highly effective. Because stand- covariates and hazards have been modeled appropri-
ard survival analyses often rely on restrictive ately. RSF is known to handle this problem coher-
assumption, such as proportional hazards, there is ently and automatically.3,5 Despite this advantage,

FIGURE 3. Out-of-bag data (OOB-data) covariate importance values according to conservation of events splitting rule. ACE-27, Adult
Comorbidity Evaluation-27.

Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012 55
FIGURE 4. Out-of-bag data (OOB-data) covariate importance values according to log-rank score splitting rule. ACE-27, Adult Comor-
bidity Evaluation-27.

RSF was not able to deliver a model with a substan- historical dataset. The covariates for these 1371 con-
tially better C-index than the Cox model in this study. secutive patients were collected with a strong
This could be the result of the size and content of our assumption that they all impact overall survival

FIGURE 5. Out-of-bag data (OOB-data) covariate importance values according to log-rank approximation splitting rule. ACE-27, Adult
Comorbidity Evaluation-27.

56 Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012
 the original data (M1 classification, n ¼ 17; 1.2%), it is
Table 3. Harrell’s concordance error rate (1 minus C-index).
possible that some RSF approaches are unable to iden-
Repetitions ¼ 100 Repetitions ¼ 10 tify it as an important covariate, especially because the
Mean (ER) SD (ER) Mean (ER) SD (ER) level of importance is computed on the OOB-data sam-
ple, which is approximately 33% of the original data. In
Cox model 0.2826 0.0120 0.2788 0.0114
RSF No. of trees ¼ 1000 No. of trees ¼ 100
this study, the log-rank approach, which delivered the
Log-rank 0.2873 0.0005 0.2889 0.0014 lowest RSF error rate, did identify M1 classification as
Conservation of events 0.3079 0.0006 0.3099 0.0015 an important covariate. A second important finding
Log-rank score 0.2951 0.0007 0.2986 0.0020 was that despite comparable error rates, the relative
Log-rank approximation 0.2958 0.0006 0.2996 0.0013
importance ranking in the 4 RSF approaches was some-
Abbreviations: ER, error rate; RSF, Random Survival Forest. what different, especially for the different tumor loca-
tions. This might make someone question the true level
based on prior univariate regression analyses. Per- of importance for these model covariates.
haps RSF performs better as a data-mining instru-
ment on a large dataset that does not have covariates CONCLUSIONS
with a predetermined impact on the model output The Cox model delivered the best performance of all
variable. We will test this hypothesis when our EPF approaches. The performance of the 4 RSF
is able to generate and export large head and neck approaches were, however, almost similar. Therefore,
oncological datasets with a substantial amount of both methods can be recommended in right-censored
covariates. Nevertheless, the results of this study con- head and neck cancer survival analyses. There are
firm what is generally found: RSF produced accurate some considerations when using RSF. First, a Cox
ensemble models with performances comparable to model clearly represents the impact of each covariate
the Cox model. on overall survival and its relationship with other
In this study, the log-rank splitting approach per- covariates. RSF does not give hazard ratios and p val-
formed better than the other 3 RSF approaches. Users ues and perhaps becomes more of a ‘‘black box’’ when
are, however, encouraged to try all 4 RSF approaches. interpreting the model due to the sheer number of
The log-rank approximate approach has the fastest trees generated. Second, the relative importance
computation, followed by the conservation of events ranking of model covariates slightly differed within
rule. The log-rank approximate approach, therefore, each RSF approach. This might make someone ques-
might be the preferred method in settings in which tion the true level of importance of the respective
computational speed is essential. model covariates. Third, when a covariate is insuffi-
Comparing RSF to the Cox model, it can be said ciently represented in the original data, RSF can
that a Cox model is more capable of extracting pat- miss it as an important covariate.
terns and relationships hidden deep into medical The advantage of RSF is that it is much better
datasets, whereas in the RSF model the ensemble suited for automatization of survival analysis than a
tree methods become more of a ‘‘black box’’ when Cox model because it requires less input from the
interpreting the model due to the sheer number of user in data settings where covariates are highly
trees generated. interrelated.
A predictive model not only has a certain perform- We will consider RSF a suited explorative ‘‘data-
ance, but also indicates the relative importance of its mining’’ tool for large (future EPF generated) datasets
covariates. In the Cox model, we used the ratio of the with covariates that have an as yet undetermined
regression coefficient to its SE (Z-value) to rank cova- effect on the model output variable. Based on the
riates. In RSF, the OOB-data importance values were results of these analyses, RSF can assist in building
used to rank covariates. From both methods, we Cox models with highly significant covariates with a
learned that age at diagnosis, a high T classification, higher level of confidence than with a single method.
a high N classification, and severe comorbidity (ACE-
27 grade 3) are very informative covariates. The cova- Acknowledgment. We thank E. W. Steyerberg,
riates, sex, mild comorbidity (ACE-27 grade 1), tumor Department of Public Health, Erasmus Medical
location nasopharynx, glottic larynx, and supraglottic Center, for his input and expertise on prognostic
larynx were less informative. For the remaining cova- modeling.
riates, it can be said that they have intermediate to
low importance values.
Discrepant findings between methods were pres- REFERENCES
ent. Where M1 classification is a very informative
1. Datema FR, Ferrier MB, van der Schroeff MP, Baatenburg de
covariate in the Cox model, it was an unimportant Jong RJ. Impact of comorbidity on short-term mortality and
covariate in 3 of 4 RSF approaches. Every clinician will overall survival of head and neck cancer patients. Head Neck
agree that presence of distant metastasis is correlated 2010;32:728–736.
2. Baatenburg de Jong RJ, Hermans J, Molenaar J, Briaire JJ, le
with a very poor prognosis. It is important to realize Cessie S. Prediction of survival in patients with head and neck
that when a covariate is insufficiently represented in cancer. Head Neck 2001;23:718–724.

Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012 57
 3. Breiman L. Random forests. Machine Learning 2001;45:5–32. 10. Cox DR, Oakes D. Analysis of Survival Data. Chapman and
4. Imran KO, Mevlut T, Füsun T. The comparisons of random sur- Hall; 1988.
vival forests and Cox regression analysis with simulation and an 11. Harrell FE Jr, Califf RM, Pryor DB, Lee KL, Rosati RA. Evalu-
application related to breast cancer. Expert Syst Appl ating the yield of medical tests. JAMA 1982;247:2543–2546.
2009;36:8582–8588. 13. Piccirillo JF, Costas I. The impact of comorbidity on outcomes. ORL J
5. Ishawaran H, Kogalur UB, Blackstone EH, Lauer MS. Random Otorhinolaryngol Relat Spec 2004;66:180–185.
survival forests. Ann Appl Stat 2008;2:841–860.
13. Feinstein AR. Clinical biostatistics. XIV. The purposes of prognos-
6. Ishawaran H, Kogalur UB. Random Survival Forest 3.0.1. R-package
tic stratification. Clin Pharmacol Ther 1972;13:285–297.
2007 (online available at: http://cran.r-project.org).
7. Sinisi SE, van der Laan MJ. Regression trees for censored data. 14. Feinstein AR. Clinical biostatistics. XV. The purposes of prognos-
Biometrics 1988;44:35–47. tic stratification. I. Clin Pharmacol Ther 1972;13:442–457.
8. Leblanc M, Crowley J. Survival trees by goodness of split. 15. Feinstein AR. Clinical biostatistics. XVI. The process of prognos-
J Amer Stat Assoc 1993;88:457–467. tic stratification. 2. Clin Pharmacol Ther 1972;13:609–624.
9. Hothorn T, Lausen B. On the exact distribution of maximally 16. Feinstein AR. Clinical biostatistics. XVII. Synchronous partition
selected rank statistics. Comput Stat Data Anal 2003;43:121– and bivariate evaluation in predictive stratification. Clin Phar-
137. macol Ther 1972;13:755–768.

58 Novel Head and Neck Cancer Survival Analysis Approach HEAD & NECK—DOI 10.1002/hed January 2012