HHS Public Access

Author manuscript
Kidney Int. Author manuscript; available in PMC 2015 November 25.
Author Manuscript

Published in final edited form as:

Kidney Int. 2014 March ; 85(3): 659–667. doi:10.1038/ki.2013.349.

Derivation and validation of the renal angina index to improve

the prediction of acute kidney injury in critically ill children
Rajit K. Basu1,2, Michael Zappitelli3, Lori Brunner1, Yu Wang4, Hector R. Wong2, Lakhmir S.
Chawla5, Derek S. Wheeler1,2, and Stuart L. Goldstein1,6
1Center for Acute Care Nephrology, Department of Pediatrics, Cincinnati Children’s Hospital
Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, USA
Author Manuscript

2Division of Critical Care, Department of Pediatrics, Cincinnati Children’s Hospital Medical Center,
University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
3Division of Pediatric Nephrology, Department of Pediatrics, Montreal Children’s Hospital, McGill
University, Montreal, Quebec, Canada
4Divisionof Biostatistics and Epidemiology, Department of Pediatrics, Cincinnati Children’s
Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
5Division
of Anesthesiology and Critical Care Medicine, George Washington University,
Washington, DC, USA
6The Heart Institute, Department of Pediatrics, Cincinnati Children’s Hospital Medical Center,
University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
Author Manuscript

Abstract
Reliable prediction of severe acute kidney injury (AKI) has the potential to optimize treatment.
Here we operationalized the empiric concept of renal angina with a renal angina index (RAI) and
determined the predictive performance of RAI. This was assessed on admission to the pediatric
intensive care unit, for subsequent severe AKI (over 200% rise in serum creatinine) 72 h later
(Day-3 AKI). In a multicenter four cohort appraisal (one derivation and three validation),
incidence rates for a Day 0 RAI of 8 or more were 15–68% and Day-3 AKI was 13–21%. In all
cohorts, Day-3 AKI rates were higher in patients with an RAI of 8 or more with the area under the
curve of RAI for predicting Day-3 AKI of 0.74–0.81. An RAI under 8 had high negative
predictive values (92–99%) for Day-3 AKI. RAI outperformed traditional markers of pediatric
severity of illness (Pediatric Risk of Mortality-II) and AKI risk factors alone for prediction of
Author Manuscript

Day-3 AKI. Additionally, the RAI outperformed all KDIGO stages for prediction of Day-3 AKI.
Thus, we operationalized the renal angina concept by deriving and validating the RAI for

Correspondence: Rajit K. Basu, Division of Critical Care Medicine-MLC 2005, Cincinnati Children’s Hospital Medical Center, 3333
Burnet Avenue, Cincinnati, Ohio 45229, USA. [email protected].
DISCLOSURE
The remaining authors declared no competing interests.
SUPPLEMENTARY MATERIAL
Supplement A. Derivation of the renal angina index.
Supplement B. Renal angina index separated by known or unknown baseline creatinine.
Supplement C. Severity of illness and renal angina index comparisons.
Supplementary material is linked to the online version of the paper at http://www.nature.com/ki
 Basu et al. Page 2

prediction of subsequent severe AKI. The RAI provides a clinically feasible and applicable
Author Manuscript

methodology to identify critically ill children at risk of severe AKI lasting beyond functional
injury. The RAI may potentially reduce capricious AKI biomarker use by identifying patients in
whom further testing would be most beneficial.

Keywords
acute kidney injury; biomarkers; pediatrics; renal angina

Approximately 10% of all children admitted to an intensive care unit (ICU) develop acute
kidney injury (AKI), and this rate increases up to 82% with increasing patient severity of
illness.1,2 Increasing AKI severity, characterized by serum creatinine (SCr)- and urine
output (UOP)-based stratifications of AKI, is associated with increased mortality in adults3
and children.4 Even small increases in SCr (0.3 mg/dl) reflect significant kidney damage and
Author Manuscript

are associated with poor patient outcome.5,6 The well-recognized limitations of SCr for real-
time accurate AKI diagnosis have prevented timely therapeutic interventions.7 Thus,
extensive research efforts have been expended to find earlier, more sensitive biomarkers for
AKI.

Several AKI biomarkers have demonstrated promising results for the identification and
prediction of AKI in children. However, most have been validated only in the
cardiopulmonary bypass (CPB) setting, where demographic homogeneity, lack of
comorbidities, and a known onset and duration of ischemic injury provide an ideal
biomarker validation environment.8,9 Demographic heterogeneity likely contributes to the
poor discriminatory performance of these biomarkers in non-cardiac pediatric intensive care
unit (PICU) patients (area under the curve (AUC) values range from 0.54 to 0.78).10–13 We
Author Manuscript

previously found that children with persistent AKI at PICU admission (AKI after 48 h) were
at the highest risk for requiring renal replacement therapy (RRT).2 Identifying patients at
risk for severe and long-lasting AKI in the PICU, and as importantly, identifying patients
unlikely to be at risk, is imperative as risk stratification could allow more judicious AKI
biomarker assessment to drive therapeutic intervention, increasing their predictive
performance and cost-effectiveness.14,15 Along these lines, the recent 10th Acute Dialysis
Quality Initiative Conference (ADQI-X) issued a directive to use combinations of
biomarkers to identify and differentiate functional AKI (‘pre-renal’ or ‘reversible’) from
kidney damage (persistent).16

The context-based disparity of biomarker efficacy for acute coronary syndrome provides
important lessons for the AKI field; troponin demonstrates suboptimal efficacy with
Author Manuscript

capricious, undirected use.17,18 Although the ability to detect and subsequently

expeditiously treat myocardial infarction was augmented with the discovery and
incorporation of troponin into the clinical context of cardiac angina, repeated evidence
highlights the erosion of troponin performance when measured in patients at low
demographic and/or clinical risk of myocardial infarction from coronary disease.17–23 In
addition, independent of troponin, the absence of cardiac angina carries high negative
predictive value (NPV) for the diagnosis of a heart attack.24,25

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 3

To that end, we recently proposed the empiric clinical model of renal angina to identify
Author Manuscript

which critically ill patients would be at the greatest risk of AKI.26 Using patient
demographic factors and early signs of injury, renal angina aims to delineate patients at risk
for subsequent severe AKI (AKI beyond the period of functional injury) versus those at low
risk (Figure 1a). In the current study, we operationalize renal angina fulfillment by deriving
an index (renal angina index: RAI) and, in separate derivation and validation cohorts, test
the hypotheses that: (1) renal angina fulfillment using a RAI threshold improves prediction
of subsequent severe AKI over severity of illness or risk factors alone and (2) RAI
prediction of AKI outperforms currently used clinical thresholds for early signs of kidney
injury.

RESULTS
Group characteristics
Author Manuscript

Demographics for each cohort (C1 (n = 144): derivation; C2–C4 (n = 118, 108, and 214,
respectively): validation) are shown in Table 1. Other than the absence of transplant patients,
there were no significant demographic differences between C1 and C3. C4 patients were
more severely ill (Pediatric Risk of Mortality II (PRISM-II) score27) and had higher use of
inotropy and mechanical ventilation than the other cohorts. The overall incidence of the
subsequent severe AKI outcome 72–96 h from PICU admission (Day-3 AKI) in the cohorts
was 10–20% (C1: 19%, C2: 10.2%, C3: 10.2%, and C4: 13.6%). The optimal RAI cutoff for
fulfillment of renal angina (ANG(+), defined by RAI ≥8) was derived by studying patients
from cohort 1 (Supplementary A online).

Derivation cohort (C1)—Cincinnati sepsis #1

Day 0 (PICU admission day) ANG(+) occurred in 51/144 (35%) of patients. Compared with
Author Manuscript

ANG(−) (RAI <8) patients, ANG(+) patients had higher Day-3 AKI rates, longer PICU
length of stay (LOS), higher RRT provision, and higher hospital mortality rates (Table 2).
Day 0 RAI predicted Day-3 AKI with an AUC of 0.77 (95% confidence interval (CI) =
0.68–0.86). RAI <8 had a high NPV of 92% (95% CI = 85–97%) (Table 3).

Validation cohorts (C2–C4)—Montreal retrospective, prospective, and Cincinnati sepsis #2

Day 0 ANG(+) occurred in 15.3% (C2), 35.2% (C3), and 67.8% (C4) of patients. ANG(+)
patients had significantly higher Day-3 AKI rates than ANG(−) patients in all cohorts (Table
2). Day 0 RAI predicted Day-3 AKI with an AUC between 0.74 and 0.81 and RAI <8 had an
NPV ≥95% for all three cohorts (Table 3). In addition, RRT provision rates were higher,
PICU LOS was longer, and mortality was higher in ANG(+) than in ANG(−) patients (Table
Author Manuscript

2).

RAI prediction of AKI—by creatinine clearance and/or fluid overload criteria

Both the predictive variable (RAI) and the outcome variable (AKI) were broken down by
composite factors of kidney injury. The discrimination of RAI for Day-3 AKI by change in
creatinine clearance from baseline (ΔeCCl) resulted in AUC values consistently superior to
the discrimination by percent fluid overload (FO). Although FO did not perform as well for
prediction as ΔeCCl, the AUC for RAI for Day-3 AKI improved when RAI incorporated

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 4

both day of admission ΔeCCl and FO (Table 4). The AUC values for RAI prediction of
Author Manuscript

Day-3 AKI were not different for whichever outcome criterion was used for outcome (UOP
or ΔeCCl).

RAI and baseline creatinine

Approximately 25% (35/144) of patients in C1 required an imputed baseline creatinine
owing to a lack of a baseline creatinine from which to compute ΔeCCl on the day of
admission for the RAI calculation. Only 9 of these 35 were ANG(+) on the day of admission
and only 1 had Day-3 AKI (this patient was ANG(+)). For the remaining 109 patients who
had a known baseline creatinine for calculation of ΔeCCl, the RAI discrimination for Day-3
AKI was greater than for PRISM-II values and similar to the cohort as a whole
(Supplementary B online).
Author Manuscript

RAI versus KDIGO stage

RAI prediction of Day-3 AKI was superior to simply having Kidney Diseases Improving
Global Outcomes (KDIGO) stage 1 injury on the day of admission; fulfillment of renal
angina demonstrated higher positive predictive value (PPV), NPV and a higher Youden’s
index28 for severe subsequent AKI than KDIGO stage 1. ANG(+) demonstrated similar
predictive efficacy for Day-3 AKI compared with KDIGO stages 2–3, but had higher
sensitivity, higher NPV, and higher Youden’s index (Table 5).

RAI versus pediatric illness severity

Significant differences in PRISM-II scores between ANG(+) and ANG(−) patients were
analyzed by risk tranche and found to be attributable to weighting of patients in each group
(Supplementary C online). On the basis of the univariate associations observed for patients
Author Manuscript

with or without Day-3 AKI in both C1 and C4 (cohorts with similar admission diagnoses),
we constructed a multivariable predictive model for Day-3 AKI using fulfillment of ANG,
patient age, and PRISM-II score. Independent predictors of Day-3 AKI were as follows:
ANG(+) (odds ratio (OR) = 3.91, 95% CI = 1.89–8.2), age (OR = 1.01 for 1-year increase in
age, 95% CI = 1.02–1.11), and PRISM-II score (OR = 1.04 for 1-unit increase in score, 95%
CI = 1.0–1.1) (Supplementary C online). When compared directly, RAI outperformed
PRISM-II for the prediction of AKI outcome measured either by UOP, change in creatinine,
or the worse metric of the two (Table 4).

DISCUSSION
Renal angina fulfillment identifies children at the highest risk of suffering subsequent severe
AKI. For a clinician, the ability to predict the presence of severe AKI 3 days in advance
Author Manuscript

carries obvious benefit. The percentage of children suffering from severe AKI at 72 h
(Day-3 AKI) in each cohort is indicative of the extent of the AKI burden in the PICU. The
concomitant comorbidities of ANG(+)-associated AKI (increased duration of mechanical
ventilation, inotropy, RRT use, and mortality) are clear.

AKI biomarkers need to demonstrate the appropriate balance of diagnostic performance and
cost-effectiveness to gain widespread acceptance leading to implementation at the bedside.

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 5

Indiscriminate testing for any condition (myocardial infarction, stroke, kidney injury, and so
Author Manuscript

on) in every patient (regardless of size, age, and comorbidities) will render any biomarker
virtually useless. Avoidance of such shotgun testing is only possible by providing direction
(clinical context) for biomarker use. We have used the performance of troponin levels to
detect acute coronary syndrome to provide an apt example of directed, optimized biomarker
testing. Troponin measured in patients who exhibit cardiac angina, a combination of clinical
signs and known coronary disease risk factors, allows practitioners to rule in myocardial
infarction. In this select, risk-stratified population, troponin has great specificity and PPV.
When measured in patients without cardiac angina, troponin loses performance.
Unfortunately, unlike a heart attack, AKI does not carry an easily identifiable physical
prodrome such as cardiac angina. Simply put, a kidney attack29 does not ‘hurt’. Therefore,
to optimize performance, clinicians must seek novel ways of directing AKI biomarker use.

We suggest that renal angina fulfills this prediction need, as it outperformed signs of injury
Author Manuscript

alone for prediction of severe subsequent AKI. The derivation of the RAI (Figure 1b,
Supplementary A online) was based on available AKI epidemiology reported in select
pediatric populations: children admitted to the ICU carry increased risk over the general
population (4.5–10%),1,30 children receiving bone marrow transplantation have ~3× risk
(11–21%),31 and those who are intubated and on vasopressor support carry nearly 5× risk
versus the general ICU population (51%).2 The ‘signs of injury’ (i.e., kidney pain) in the
RAI include ΔeCCl and FO. The significance of FO on PICU morbidity has been recently
documented, causing deleterious effects on oxygenation32 and increasing morbidity and
mortality in children with lung injury.33 When the RAI was derived based on either of the
individual variables (ΔeCCl or FO) in isolation, the discrimination for Day-3 AKI remained
robust, particularly for ΔeCCl (Table 4). Although FO for RAI calculation did not perform
as well as ΔeCCl, FO alone was equal to PRISM-II scores for prediction of Day-3 AKI
Author Manuscript

(Supplementary C online). More importantly, the prediction for Day-3 AKI when the worse
value of FO or ΔeCCl was used improved over using only ΔeCCl (0.73–0.75). The RAI
AUC values for Day-3 AKI as classified by ΔeCCl, UOP, or the worse of the two metrics
were nearly identical (Table 4). Renal angina outperformed early signs of injury as stratified
by the KDIGO stages. ANG(+) demonstrated a markedly more robust predictive
performance compared with KDIGO stage 1 (higher Youden’s index, PPV and NPV) and
was at least equal to, if not slightly superior to, KDIGO stages 2 and 3 (Table 5). It should
be noted that the improvement over KDIGO stage 1 is the value of the RAI—the
identification of the at-risk patient who requires further investigation and for whom an AKI
biomarker may be most beneficial. The overweight smoker with sub-sternal chest pain, jaw
claudication, and ST-segment changes on electrocardiogram (i.e., risk + KDIGO 2–3) gets a
troponin as a confirmatory test while being prepped and draped in the interventional
Author Manuscript

catheterization suite. A patient with cardiac risk factors, and mild chest pressure, benefits
from troponin as a discriminatory test, as the result affects clinical decision-making.

Renal angina outperforms severity of illness scores for prediction of severe subsequent AKI.
As mentioned earlier, FO used alone for calculation of the RAI was on par with PRISM-II
discrimination of Day-3 AKI. When either eCCl or the worse metric (either eCCl or FO)
was used for RAI calculation, the index scores’ AUC values far surpassed PRISM-II scores’
values (Table 4 and Supplementary C online). Analysis of RAI per risk tranche revealed that

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 6

the PRISM-II score skewed toward those patients in the very high-risk tranche, likely
Author Manuscript

leading to a significant difference in the overall PRISM-II scores between the ANG(+) and
ANG(−) patients. The PRISM-II score consistency within the individual tranches taken
together with the comparison with renal angina simultaneously highlights the limitations of
traditional severity of illness scoring systems for AKI34 and underscores the utility of renal
angina for AKI prediction. The implied benefit of RAI having greater advocacy than
PRISM-II at identifying patients at risk for AKI is the application of RAI in clinical trials of
AKI. Identification of patients at a higher AKI risk using RAI stratification could
theoretically guide the enrollment for a novel AKI biomarker or therapy trial, which could
ultimately guide treatment strategy.

Our goal with the derivation of the RAI was to develop a simple score, which is easily
calculable and can be used at the bedside of a critically ill patient. Although published
prediction scores for organ failure, severity of illness, or mortality admittedly use more
Author Manuscript

rigorous statistical methodologies for derivation, these models are constrained by a constant
need of recalibration, potential over-fitting, and are intended to be used in population
analyses, not for single patients. Thus, although our derivation of the RAI was semi-empiric
and, by comparison, simple, we propose that this simplicity (and, based on this study,
apparent validity) will enhance its use in clinical care and in future research. Another
important point is that children admitted to the PICU most often do not have clearly
identifiable risk factors for AKI (as a preoperative cardiac surgery adult patient may have);
the RAI uses easily identifiable criteria within the first day of PICU admission to predict the
highly clinically relevant outcome of subsequent severe AKI. Finally, previous studies of
adult AKI prediction scores have had extremely large and comprehensive administrative
databases available for deriving and validating scores; such databases are nonexistent in the
PICU population. We propose that future research should aim at constructing and validating
Author Manuscript

the use of large, multicenter PICU databases, to enable more refined and rigorous evaluation
of AKI risk factors and evidence of injury, which will allow for validation and/or calibration
of the RAI score if needed.

The performance of AKI biomarkers for diagnosis and prediction varies depending on the
risk of AKI in the population studied. Candidate biomarkers including neutrophil gelatinase-
associated lipocalin (NGAL), kidney injury molecule-1 (KIM-1), interleukin-18 (IL-18), and
liver-type fatty acid–binding protein (L-FABP), initially discovered via proteomic data in
murine models of kidney ischemia,35 demonstrate good-to-excellent predictive performance
(AUC = 0.75–0.95) for AKI in children after CPB.36–38 A recent study from our center
demonstrated that a clinical model of patient age and CPB duration yielded AUC values of
0.72 and 0.74 to predict a 50% SCr rise at 2 and 6 h after bypass.9 In our current study, the
Author Manuscript

renal angina clinical model on Day 0 of PICU admission in all four cohorts studied yielded a
similar predictive precision (moderate-to-good predictive range, with AUC values ranging
from 0.73 to 0.81) for persistent severe AKI on Day 3. We also suggest that for ANG(−)
children our model provides a powerful predictive tool to guide care. The NPVs >92% in all
four cohorts and >95% in three of the cohorts indicate that Day 0 ANG(−) patients have a
very low likelihood of having a 200% rise in SCr or prolonged oliguria on PICU Day 3.
These data suggest that AKI biomarkers should not be obtained to predict severe AKI in
Day 0 ANG(−) children, given the low likelihood of Day-3 AKI. For instance, although not

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 7

tested in the current study, ANG(−) children with elevated Day 0 SCr may likely have a
Author Manuscript

fluid responsive state (previously termed pre-renal azotemia), and therefore could likely
receive significant volume without developing the severe FO consistently associated with
poor outcomes in critically ill children.32,33,39 In the CPB study conducted at our center, the
prediction of AKI severity improved after urinary biomarkers were added alone and in
combination; we also predict that inclusion of available plasma AKI biomarkers (i.e.,
directed biomarker testing) will improve upon the renal angina clinical model. Ideally,
through a simple calculation of the RAI, a clinician can identify ANG fulfillment or absence
in any patient on admission and then appropriately allocate the use of an AKI biomarker test
to those in whom the test may yield the greatest predictive benefit. We suggest that renal
angina allows biomarker testing to be appropriately directed. Directed use is a logical
method that can be used to derive biomarker panels that classify and phenotype AKI types—
the exact directive that ADQI-X set forth in the recent consensus statement.16 Such
Author Manuscript

optimization eliminates shotgun use of AKI biomarkers, both cost prohibitive and
nonscientific.15

We acknowledge that given our limited data in pediatrics, particularly in the general PICU,
which cares for wide-ranging and heterogeneously ill children, an admission change in SCr
from baseline (to any degree) carries an unknown prognosis. In our derivation cohort,
roughly 25% of the patients did not have a baseline creatinine in our database, and we
imputed baseline values for these patients based on height and normal creatinine clearance
for age. The only effect this manipulation carried on our results was to potentially
underestimate the change in eCCl from baseline (affecting both ANG diagnosis and AKI
outcome diagnosis; Table 4); the imputed baseline creatinine levels were higher than
baseline creatinine levels observed in comparable patients (based on size and age). Finally,
although data from the adult AKI literature suggests that the outcome of AKI as measured
Author Manuscript

by both creatinine and UOP is worse than creatinine-only AKI, this is unknown in
pediatrics. There are, in fact, few studies that examine UOP-pRIFLE (pediatric-modified
RIFLE),2 and this limitation is outlined in the recent worldwide KDIGO AKI
guidelines.40–42 In our study, we found no significant differences in RAI prediction of AKI
when the outcome was based on creatinine or UOP.

Our study has several strengths. We examined four independent cohorts with 100–200
critically ill children each, which in aggregate is a pediatric large AKI study. Our study
includes a wide-ranging array of patient age, background history, and illness severity. The
multicenter nature of our study overcomes single-center bias for the initiation of mechanical
ventilation, inotropic support, and RRT provision. In addition, we have provided evidence
that RAI outperforms severity of illness scores, risk factors, and degrees of injury alone for
Author Manuscript

prediction of subsequent severe AKI. Our study may be useful in addressing the limitations
and problems that are encountered in pediatric AKI study.43 As we suggested earlier, the use
of renal angina to stratify patients for enrollment in biomarker or therapy trials may create
the uniformity required to properly analyze AKI in pediatrics. Such stratification and
implementation of the ADQI-X directive of AKI classification may identify patients who
have significant creatinine change as a result of functional injury, which was the case in
Cohort 1 (Table 5 shows a low PPV for KDIGO 2–3 on admission).

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 8

There are several potential limitations to our study. The retrospective analyses of several
Author Manuscript

cohorts is subject to the inherent limitations of data extraction, correlative analyses, and
conclusions associated with data not collected with the exclusive purpose of testing our
hypothesis. The number and the diverse range of illness in our patient population overcome
some of these limitations by giving us a larger sample of patients who are at varying degrees
of risk. Although the specificity of early changes in either creatinine clearance or % FO for
persistent AKI is limited, this composite aspect of RAI incorporates clinical context to
provide usable bedside information (e.g., when a doubling of SCr may reflect reversible
AKI vs. persistent AKI). Although there seemed to be a paucity of high-risk patients in our
cohorts (history of transplant), a substantial percentage of transplant patients were stratified
as very high risk given the need for inotropy and mechanical ventilation. The apparent
higher incidence of AKI in the high risk group versus the very high risk group is therefore
slightly misleading. Patients with transplants have an increased risk of AKI, and we expect
Author Manuscript

that with future study of renal angina, supported by larger pediatric population database
study, more patients will fall directly in the high-risk tranche, allowing for refinement of the
RAI score. We did not test the RAI on ICU admission days other than Day 0 or to predict
AKI persistence after ICU Day 3 (primarily owing to the most logical predictive benefit for
a clinician and sample size limitation), and acknowledge that the RAI performance may be
better or worse than the clinical context we tested.

We conclude that risk stratification using renal angina will aid with the prediction of
persistent severe AKI and that renal angina prodrome, in conjunction with AKI biomarker
measurement, may reliably differentiate patients who will be responsive to appropriate
restorative therapy from those will progress to severe subsequent AKI. We believe that renal
angina is a clinical adjunct that will lead to the optimization of AKI biomarker performance
across the wide-ranging heterogeneity that exists across the general pediatric PICU
Author Manuscript

population.

MATERIALS AND METHODS

All patient data
We used data collected from four separate PICU cohorts from Cincinnati Children’s
Hospital Medical Center (CCHMC) and Montreal Children’s Hospital (MCH). Each cohort
is described below. For all cohorts, patients with preexisting chronic kidney disease or
immediately after cardiac surgery were excluded (three patients were excluded from C1
because of baseline requirement of RRT). Each site’s Institutional Review Board waived the
need for informed consent for the purposes of this study, but informed consent was obtained
for C3 and C4 as part of their original study (see below). All patients in the study (all
Author Manuscript

cohorts) were in the age range of 28 days to 25 years.

Individual cohort descriptions

‘Derivation’ cohort (C1): this was a retrospective cohort of patients admitted to the CCHMC
PICU from 2009 to 2011 with an International Classification of Diagnosis (ICD-9) code of
‘sepsis’ or ‘septic shock’, as per international consensus guidelines for the diagnosis of

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 9

sepsis.44 The RAI was first derived and validated in this cohort, as described in
Author Manuscript

Supplementary A online.

‘Validation’ cohorts (C2, C3, and C4): two separate observational cohorts of patients
admitted to the MCH PICU were included as validation cohorts. The retrospective cohort
(C2) consisted of all eligible children admitted to the MCH PICU from 2004 to 2007 for at
least 4 days. The original study for C2 was performed to evaluate FO in critically ill children
and has not previously been reported. UOP data were unavailable. The prospective cohort
(C3) consisted of children admitted to the MCH PICU for at least 2 days from 2007 to 2010.
These patients were recruited into a prospective AKI biomarker study and have not
previously been reported. Both databases were reviewed in detail to ensure that relevant data
for this report were appropriately collected, followed by extraction of necessary variables
and analysis. The C4 cohort included patients admitted to 17 different PICUs across the
United States from the years 2006–2011 and enrolled in a study investigating the genomic
Author Manuscript

profile of children with sepsis.45 Inclusion criteria of this study required an ICD-9 code of
‘septic shock’. Patients from C4, called ‘CCHMC sepsis 2’ for convenience, were not
included in the C1 cohort above.

Data obtained
Obtained data were denoted as follows: Day 0 was the first calendar day of PICU admission.
Day 3 consisted of the time period between 72 and 96 h after PICU admission. Baseline data
included demographic information, admission diagnoses, PRISM-II scores,27 and first SCr
of the PICU admission. Day 0 data included for determination of RAI included the use of
vasopressors/inotropy and the use of invasive mechanical ventilation (yes or no). Calculated
variables for the determination of RAI included % FO46 and changes in kidney function
(based on estimated creatinine clearance (eCCl)). Percent FO on Day 0 was determined by
Author Manuscript

assessing the first 8 h of admission in the ICU on Day 0. The time frame of 8 h was felt to be
beyond the generally accepted window of ‘early goal-directed therapy’ (EGDT) of
resuscitation,47 allowing time for some diuresis, and also was sufficiently shorter than an
actual full day. The electronic health record was reviewed for the lowest SCr up to 3 months
before PICU admission to establish a reference eCCl. If no SCr was available, a reference
eCCl of 120 ml/min per 1.73 m2 was used.48 A baseline creatinine was also imputed based
on the eCCl and the patient height using the Schwartz correction. UOP was recorded hourly
as ml/kg/h in 8-h blocks (except in C2, no urine output available).

Renal angina
All patients were classified on Day 0 as fulfilling criteria for renal angina (i.e., being
ANG(+) vs. ANG(−)) using the RAI. An RAI score of ≥8 demonstrated the highest
Author Manuscript

Youden’s index28 and the highest NPV (Supplementary A online), and thus ANG(+) was
defined as an RAI score ≥8.

KDIGO stage comparison

RAI scores for each patient in C1 were compared with the KDIGO AKI stages (1 or 2
and3)40 for prediction of outcome.

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 10

Outcomes
Author Manuscript

The primary outcome was the presence of severe AKI 72 h after PICU admission (Day-3
AKI), denoted as ‘subsequent severe AKI’. Day 3 was chosen because of the following
reasons: most PICU patients develop AKI within this time frame; there is enough time to
develop the severe or persistent AKI outcome; and theoretically, 3 days surpasses the time
frame of what would be considered reversible (or functional) AKI. Severe AKI was defined
by the KDIGO AKI classification stage ≥2: SCr of 200% baseline (a decrease in eCCl of
≥50% from baseline) or ≤0.5 ml/kg/h of UOP for ≥8 h.40 We chose KDIGO stage ≥2 as the
primary outcome as it is associated with mortality and morbidity in multiple pediatric
studies.4 The higher of the KDIGO strata (UOP or ΔeCCl) was used. For C2 (retrospective
MCH), only SCr data were available. Secondary outcomes were PICU LOS, use of RRT,
and in-hospital mortality.
Author Manuscript

Statistical analysis
All statistical analyses were performed using STATA version 12 (StataCorp, College
Station, TX) and SAS version 9.3 (SAS Institute, Cary, NC). Continuous variables were
reported as median with interquartile range and compared using the Mann–Whitney test.
Categorical variables were summarized using frequency and proportion and compared by
chi-square or Fisher’s exact tests. An RAI cutoff of ≥8 was used to analyze the predictive
performance of RAI (sensitivity, specificity, NPV, and PPV). As Youden’s index should not
be used as the sole index of performance, the cutoff value of ≥8 was chosen on the basis of
most superior performance based on Youden’s and highest NPV (to optimize the ability of
RAI to serve as a ‘screening test’). Multivariate regression was performed by comparing
variables carrying univariate associations with the outcome and a P-value <0.20
(Supplementary C online). AUC values were calculated for each prediction model. In all
Author Manuscript

analyses, a P-value <0.05 was considered statistically significant.

Supplementary Material
Refer to Web version on PubMed Central for supplementary material.

ACKNOWLEDGMENTS
We thank the following investigators who contributed biological samples and patient data for the database
supporting cohort 4: Natalie Z. Cvijanovich (Children’s Hospital Oakland), Mark Hall (Nationwide Children’s
Hospital), Geoffrey L. Allen (Children’s Mercy Hospital), Neal J. Thomas (Hershey Children’s Hospital), Robert J.
Freishtat (Children’s National Medical Center), Nick Anas (Children’s Hospital of Orange County), Keith Meyer
(Miami Children’s Hospital), Paul A. Checchia (Texas Children’s Hospital), Richard Lin (The Children’s Hospital
of Philadelphia), Michael T. Bigham (Akron Children’s Hospital), Anita Sen (Morgan Stanley Children’s Hospital),
Jeffrey Nowak (Children’s Hospital and Clinics of Minnesota), Michael Quasney (Children’s Hospital of
Author Manuscript

Wisconsin), Jared W. Henricksen (St Christopher’s Hospital for Children), Arun Chopra (CS Mott Children’s
Hospital), Sharon Banschbach (CCHMC), Eileen Beckman (CCHMC), Kelli Harmon (CCHMC), Patrick Lahni
(CCHMC), and Thomas P. Shanley (CS Mott). Cohort 4 patients were enrolled in a study supported by the
following grants from the National Institutes of Health: RO1GM064619, RC1HL100474, and R01GM099773. The
REDCap software was used for data collection and was supported by the following grant: UL1-RR026314-01
NCRR/NIH.

LSC is a consultant for Alere, Abbott, and Astute; and SLG is a consultant for Gambro.

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 11

REFERENCES
Author Manuscript

1. Schneider J, Khemani R, Grushkin C, et al. Serum creatinine as stratified in the RIFLE score for
acute kidney injury is associated with mortality and length of stay for children in the pediatric
intensive care unit. Crit Care Med. 2010; 38:933–939. [PubMed: 20124891]
2. Akcan-Arikan A, Zappitelli M, Loftis LL, et al. Modified RIFLE criteria in critically ill children
with acute kidney injury. Kidney Int. 2007; 71:1028–1035. [PubMed: 17396113]
3. Srisawat N, Hoste EE, Kellum JA. Modern classification of acute kidney injury. Blood Purif. 2010;
29:300–307. [PubMed: 20130395]
4. Slater MB, Anand V, Uleryk EM, et al. A systematic review of RIFLE criteria in children, and its
application and association with measures of mortality and morbidity. Kidney Int. 2012; 81:791–
798. [PubMed: 22258324]
5. Chertow GM, Burdick E, Honour M, et al. Acute kidney injury, mortality, length of stay, and costs
in hospitalized patients. J Am Soc Nephrol. 2005; 16:3365–3370. [PubMed: 16177006]
6. Zappitelli M, Bernier PL, Saczkowski RS, et al. A small post-operative rise in serum creatinine
predicts acute kidney injury in children undergoing cardiac surgery. Kidney Int. 2009; 76:885–892.
Author Manuscript

[PubMed: 19641482]
7. Goldstein SL. Acute kidney injury biomarkers: renal angina and the need for a renal troponin I.
BMC Med. 2011; 9:135. [PubMed: 22189039]
8. Kwiatkowski DM, Goldstein SL, Krawczeski CD. Biomarkers of acute kidney injury in pediatric
cardiac patients. Biomark Med. 2012; 6:273–282. [PubMed: 22731900]
9. Krawczeski CD, Goldstein SL, Woo JG, et al. Temporal relationship and predictive value of urinary
acute kidney injury biomarkers after pediatric cardiopulmonary bypass. J Am Coll Cardiol. 2011;
58:2301–2309. [PubMed: 22093507]
10. Herrero-Morin JD, Malaga S, Fernandez N, et al. Cystatin C and beta2-microglobulin: markers of
glomerular filtration in critically ill children. Crit Care. 2007; 11:R59. [PubMed: 17519026]
11. Washburn KK, Zappitelli M, Arikan AA, et al. Urinary interleukin-18 is an acute kidney injury
biomarker in critically ill children. Nephrol Dial Transplant. 2008; 23:566–572. [PubMed:
17911094]
12. Zappitelli M, Washburn KK, Arikan AA, et al. Urine neutrophil gelatinase-associated lipocalin is
Author Manuscript

an early marker of acute kidney injury in critically ill children: a prospective cohort study. Crit
Care. 2007; 11:R84. [PubMed: 17678545]
13. Wheeler DS, Devarajan P, Ma Q, et al. Serum neutrophil gelatinase-associated lipocalin (NGAL)
as a marker of acute kidney injury in critically ill children with septic shock. Crit Care Med. 2008;
36:1297–1303. [PubMed: 18379258]
14. Al-Ismaili Z, Palijan A, Zappitelli M. Biomarkers of acute kidney injury in children: discovery,
evaluation, and clinical application. Pediatr Nephrol. 2011; 26:29–40. [PubMed: 20623143]
15. Glassford NJ, Eastwood GM, Young H, et al. Rationalizing the use of NGAL in the intensive care
unit. Am J Respir Crit Care Med. 2011; 184:142. [PubMed: 21737595]
16. Endre ZH, Kellum JA, Di Somma S, et al. Differential diagnosis of AKI in clinical practice by
functional and damage biomarkers workgroup statements from the tenth Acute Dialysis Quality
Initiative Consensus Conference. Contrib Nephrol. 2013; 182:30–44. [PubMed: 23689654]
17. Lim W, Whitlock R, Khera V, et al. Etiology of troponin elevation in critically ill patients. J Crit
Care. 2010; 25:322–328. [PubMed: 19781898]
18. Stein R, Gupta B, Agarwal S, et al. Prognostic implications of normal (<0.10 ng/ml) and borderline
Author Manuscript

(0.10 to 1.49 ng/ml) troponin elevation levels in critically ill patients without acute coronary
syndrome. Am J Cardiol. 2008; 102:509–512. [PubMed: 18721503]
19. Meyer M, Fink C, Roeger S, et al. Benefit of combining quantitative cardiac CT parameters with
troponin I for predicting right ventricular dysfunction and adverse clinical events in patients with
acute pulmonary embolism. Eur J Radiol. 2012; 81:3294–3299. [PubMed: 22819132]
20. Agewall S, Giannitsis E, Jernberg T, et al. Troponin elevation in coronary vs. non-coronary
disease. Eur Heart J. 2011; 32:404–411. [PubMed: 21169615]

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 12

21. Agewall S, Tjora S. Physical exertion may cause high troponin levels. Tidsskr Nor Laegeforen.
2011; 131:2226. [PubMed: 22085943]
Author Manuscript

22. Tjora S, Gjestland H, Mordal S, et al. Troponin rise in healthy subjects during exercise test. Int J
Cardiol. 2011; 151:375–376. [PubMed: 21764153]
23. Ammann P, Fehr T, Minder EI, et al. Elevation of troponin I in sepsis and septic shock. Intensive
Care Med. 2001; 27:965–969. [PubMed: 11497154]
24. Bassan R, Pimenta L, Scofano M, et al. Probability stratification and systematic diagnostic
approach for chest pain patients in the emergency department. Crit Pathw Cardiol. 2004; 3:1–7.
[PubMed: 18340129]
25. Bassan R, Pimenta L, Scofano M, et al. Accuracy of a neural diagnostic tree for the identification
of acute coronary syndrome in patients with chest pain and no ST-segment elevation. Crit Pathw
Cardiol. 2004; 3:72–78. [PubMed: 18340143]
26. Goldstein SL, Chawla LS. Renal angina. Clin J Am Soc Nephrol. 2010; 5:943–949. [PubMed:
20299370]
27. Pollack MM, Ruttimann UE, Getson PR. Pediatric risk of mortality (PRISM) score. Crit Care Med.
1988; 16:1110–1116. [PubMed: 3048900]
Author Manuscript

28. Bohning D, Bohning W, Holling H. Revisiting Youden’s index as a useful measure of the
misclassification error in meta-analysis of diagnostic studies. Stat Methods Med Res. 2008;
17:543–554. [PubMed: 18375457]
29. Kellum JA, Bellomo R, Ronco C. Kidney attack. JAMA. 2012; 307:2265–2266. [PubMed:
22572776]
30. Bailey D, Phan V, Litalien C, et al. Risk factors of acute renal failure in critically ill children: a
prospective descriptive epidemiological study. Pediatr Crit Care Med. 2007; 8:29–35. [PubMed:
17251879]
31. Michael M, Kuehnle I, Goldstein SL. Fluid overload and acute renal failure in pediatric stem cell
transplant patients. Pediatr Nephrol. 2004; 19:91–95. [PubMed: 14634863]
32. Arikan AA, Zappitelli M, Goldstein SL, et al. Fluid overload is associated with impaired
oxygenation and morbidity in critically ill children. Pediatr Crit Care Med. 2012; 13:253–258.
[PubMed: 21760565]
33. Valentine SL, Sapru A, Higgerson RA, et al. Fluid balance in critically ill children with acute lung
Author Manuscript

injury. Crit Care Med. 2012; 40:2883–2889. [PubMed: 22824936]

34. Basu RK, Chawla LS, Wheeler DS, et al. Renal angina: an emerging paradigm to identify children
at risk for acute kidney injury. Pediatr Nephrol. 2012; 27:1067–1078. [PubMed: 22012033]
35. Devarajan P, Mishra J, Supavekin S, et al. Gene expression in early ischemic renal injury: clues
towards pathogenesis, biomarker discovery, and novel therapeutics. Mol Genet Metab. 2003;
80:365–376. [PubMed: 14654349]
36. Mishra J, Dent C, Tarabishi R, et al. Neutrophil gelatinase-associated lipocalin (NGAL) as a
biomarker for acute renal injury after cardiac surgery. Lancet. 2005; 365:1231–1238. [PubMed:
15811456]
37. Parikh CR, Mishra J, Thiessen-Philbrook H, et al. Urinary IL-18 is an early predictive biomarker
of acute kidney injury after cardiac surgery. Kidney Int. 2006; 70:199–203. [PubMed: 16710348]
38. Han WK, Waikar SS, Johnson A, et al. Urinary biomarkers in the early diagnosis of acute kidney
injury. Kidney Int. 2008; 73:863–869. [PubMed: 18059454]
39. Goldstein SL, Somers MJ, Baum MA, et al. Pediatric patients with multi-organ dysfunction
syndrome receiving continuous renal replacement therapy. Kidney Int. 2005; 67:653–658.
Author Manuscript

[PubMed: 15673313]
40. Group KW. Kidney Disease Improving Global Outcomes; clinical practice guideline for acute
kidney injury. Kidney Int Suppl. 2012; 2:19–33.
41. Han SS, Kang KJ, Kwon SJ, et al. Additional role of urine output criterion in defining acute kidney
injury. Nephrol Dial Transplant. 2012; 27:161–165. [PubMed: 21712492]
42. Cruz DN, Bolgan I, Perazella MA, et al. North East Italian Prospective Hospital Renal Outcome
Survey on Acute Kidney Injury (NEiPHROS-AKI): targeting the problem with the RIFLE
Criteria. Clin J Am Soc Nephrol. 2007; 2:418–425. [PubMed: 17699446]

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 13

43. Endre ZH, Pickering JW. Acute kidney injury clinical trial design: old problems, new strategies.
Pediatr Nephrol. 2013; 28:207–217. [PubMed: 22639043]
Author Manuscript

44. Goldstein B, Giroir B, Randolph A. International pediatric sepsis consensus conference: definitions
for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med. 2005; 6:2–8. [PubMed:
15636651]
45. Wong HR, Salibury S, Xiao Q, et al. The pediatric sepsis biomarker risk model. Crit Care. 2012;
16:R174. [PubMed: 23025259]
46. Sutherland SM, Zappitelli M, Alexander SR, et al. Fluid overload and mortality in children
receiving continuous renal replacement therapy: the prospective pediatric continuous renal
replacement therapy registry. Am J Kidney Dis. 2010; 55:316–325. [PubMed: 20042260]
47. Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for
management of severe sepsis and septic shock: 2012. Crit Care Med. 2013; 41:580–637. [PubMed:
23353941]
48. Zappitelli M, Parikh CR, Akcan-Arikan A, et al. Ascertainment and epidemiology of acute kidney
injury varies with definition interpretation. Clin J Am Soc Nephrol. 2008; 3:948–954. [PubMed:
18417742]
Author Manuscript
Author Manuscript
Author Manuscript

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 14
Author Manuscript
Author Manuscript

Figure 1. Renal angina

(a) The renal angina construct. The juxtaposed graphs depict risk of acute kidney injury
(AKI) versus decrease in estimated creatinine clearance from baseline (↓eCCl) and increase
in % intensive care unit (ICU) fluid overload (% ↑ICU FO). There are three risk groups
defined for the pediatric ICU population (tranches): very high risk (intubated + presence of
at least one vasopressor or inotrope), high risk (history of solid organ or bone marrow
transplant), and moderate risk (ICU admission). The construct is created such that less sign
of injury (estimated creatinine clearance (eCCl) change or FO change) is required for the
higher risk tranches to fulfill renal angina (solid red slope line). (Adapted with permission
Author Manuscript

from Goldstein and Chawla.26) (b) The renal angina index. On the basis of existing pediatric
AKI literature, tiered AKI risk strata were assigned point values for ‘risk’ and ‘signs’ of
injury. The worse parameter between change in eCCl from baseline and % FO was used to
yield an injury score. The full description of the derivation appears in Supplementary A
online. The resultant renal angina index score can range from 1 to 40. A cutoff of ≥8 is used
to determine renal angina fulfillment.
Author Manuscript

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 15

Table 1

Demographic and clinical data for all cohorts

Author Manuscript

Cohort 1: CCHMC sepsis 1 Cohort 2: MCH pro Cohort 3: MCH retro Cohort 4: CCHMC sepsis 2
derivation validation 1 validation 2 validation 3
N 144 118 108 214
Very high risk 34 19 27 184
High risk 32 0 0 9
Moderate risk 78 99 81 21
Age, years 3.8 (1.2, 12.5) 3.0 (0.2, 11.7) 1.5 (0.3, 10.6) 2.2 (0.8, 5.9)
Male, n (%) 83 (57.6) 74 (62.7) 64 (59.2) 134 (62.6)
PRISM-II 11 (7, 18) 6 (4, 10) 7 (4, 10) 14 (8, 21)*
Transplant, n (%) 39 (27.1) 0 0 9 (4.2)
Inotropy, n (%) 56 (38.9) 23 (19.5) 28 (25.9) 214 (100)*
Author Manuscript

MV, n (%) 69 (47.9) 87 (73.7) 83 (76.9) 184 (85.9)*

Day-3 AKI, n (%) 28 (19.4) 12 (10.2) 11 (10.2) 29 (13.6)
PICU LOS, days 5 (3, 13) 6 (4, 8) 9 (6, 13) 13 (8, 24)*
RRT, n (%) 13 (9.0) 3 (2.5) 3 (2.8) N/A
Mortality, n (%) 13 (9.0) 7 (5.9) 4 (3.7) 23 (10.7)

Abbreviations: AKI, acute kidney disease; CCHMC, Cincinnati Children’s Hospital; LOS, length of stay; MCH, Montreal Children’s Hospital;
MV, mechanical ventilation; N/A, not available; PICU, non-cardiac pediatric intensive care unit; PRISM-II, Pediatric Risk of Mortality score; pro,
prospective; retro, retrospective; RRT, renal replacement therapy.

Descriptive characteristics for each cohort of patients are listed above. Transplant refers to solid organs and bone marrow. Day-3 AKI refers to
KDIGO stage 2 or 3 at Day 3 of PICU admission. Age, PRISM-II, and LOS are expressed as median (interquartile range).
*
P-value <0.05 Cohort 4 versus Cohort 1.
Author Manuscript
Author Manuscript

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Author Manuscript Author Manuscript Author Manuscript Author Manuscript

Table 2

Demographics of each cohort by fulfillment of renal angina

Cohort 1: CCHMC sepsis 1 Cohort 2: MCH pro Cohort 3: MCH retro Cohort 4 : CCHMC sepsis 2
Basu et al.

derivation validation 1 validation 2 validation 3

ANG(−) ANG(+) P ANG(−) ANG(+) P ANG(−) ANG(+) P ANG(−) ANG(+) P

N (%) 93 (65) 51 (35) 100 (84.7) 18 (15.3) 70 (64.8) 38 (35.2) 69 (32.2) 145 (67.8)
Age (years) 3.1 (1, 11) 5.4 (2, 14) 0.088 6.0 (0, 12) 5.9 (0.4, 10) 0.90 5.6 (0, 11.7) 4.5 (0, 10.7) 0.37 3.8 (1.6, 6.8) 1.7 (0.5, 5) <0.001
Male, n (%) 53 (56.9) 30 (58.8) 0.997 62 (62.0) 12 (67.7) 0.71 44 (62.9) 20 (52.6) 0.30 40 (57.9) 94 (44) 0.413
PRISM-II 10 (5, 16) 15 (8, 22) 0.016 6.8 (1, 12) 7.9 (3, 13) 0.41 6.2 (1, 12) 8.9 (3, 15) 0.004 10 (5, 15) 16 (10, 24) <0.001
Day-3 AKI, n (%) 7 (7.5) 21 (41.2) <0.001 5 (5.0) 7 (38.9) <0.001 0 (0) 11 (28.9) <0.001 2 (2.9) 27 (19) 0.003
LOS, days 5 (2, 10) 9 (4, 15) 0.011 7 (2, 11) 6 (4, 9) 0.66 12.3 (4, 20) 10.1 (3, 17) 0.24 11 (8, 16) 15 (7, 27) 0.032
RRT, n (%) 4 (4.3) 9 (17.6) 0.02 2 (2.0) 1 (5.6) 0.39 0 (0) 3 (7.9) 0.04 N/A N/A
Mortality, n (%) 4 (4.3) 9 (17.6) 0.02 6 (6.0) 1 (5.6) 1 3 (4.3) 1 (2.6) 0.56 0 (0) 23 (16) <0.001

Abbreviations: AKI, acute kidney disease; ANG, renal angina; CCHMC, Cincinnati Children’s Hospital; LOS, length of stay; MCH, Montreal Children’s Hospital; N/A, not available; PRISM-II, Pediatric
Risk of Mortality score; pro, prospective; retro, retrospective; RRT, renal replacement therapy.

Select descriptive characteristics and outcomes for each cohort of patients are listed above. Day-3 AKI refers to KDIGO stage 2 or 3 at Day 3 of PICU admission. Data are expressed as medians with
interquartile ranges in parentheses. P-values compare ANG(−) versus ANG(+) for each individual cohort.

Kidney Int. Author manuscript; available in PMC 2015 November 25.

Page 16
 Basu et al. Page 17

Table 3

Performance of the renal angina index for prediction of subsequent severe AKI
Author Manuscript

Cohort 1: CCHMC Cohort 2: MCH pro Cohort 3: MCH retro Cohort 4: CCHMC sepsis 2
sepsis 1 derivation validation 1 validation 2 validation 3
ANG(+), n (%) 51 (35) 18 (15) 38 (35) 145 (68)
Day-3 AKI, n (%) 28 (19) 12 (10) 11 (10) 29 (13)
Sensitivity, % (95% CI) 75 (55–89) 58 (28–85) 91 (59–100) 93 (76–99)
Specificity, % (95% CI) 73 (64–81) 90 (82–95) 71 (61–80) 36 (33–37)
PPV, % (95% CI) 40 (27–55) 39 (17–64) 26 (13–43) 18 (15–19)
NPV, % (95% CI) 92 (85–97) 95 (89–98) 99 (92–100) 97 (90–99)
AUC, (95% CI) 0.77 (0.68–0.86) 0.74 (0.59–0.88) 0.81 (0.71–0.91) 0.80 (0.75–0.86)

Abbreviations: AKI, acute kidney disease; ANG, renal angina; AUC, area under the curve; CCHMC, Cincinnati Children’s Hospital; CI,
confidence interval; MCH, Montreal Children’s Hospital; NPV, negative predictive value; PPV, positive predictive value; pro, prospective; retro,
retrospective.
Author Manuscript

The performance of the renal angina index (RAI) for prediction of severe AKI is shown above. For each patient in each cohort, an RAI was derived
(a score of ≥8 was considered fulfillment of renal angina). The predictive performance of fulfillment of ANG on day 0 for the presence of Day-3
AKI was evaluated, which comprised the following: sensitivity, specificity, PPV, and NPV. The absolute value of the RAI (range 1–40) was used
to derive the AUC receiver operating characteristic. ANG(+)refers to patients who fulfilled angina. Sensitivity, specificity, NPV, PPV, and AUC
are listed with 95% CI.
Author Manuscript
Author Manuscript

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Basu et al. Page 18

Table 4

Renal angina index performance broken down by individual criterion

Author Manuscript

Day-3 AKI outcome

UOP Cr Worse
RAI
  ΔeCCl 0.81 (0.71–0.90) 0.73 (0.59–0.88) 0.78 (0.69–0.87)
  FO 0.57 (0.44–0.71) 0.63 (0.49–0.76) 0.60 (0.49–0.71)
  Worse 0.78 (0.68–0.88) 0.75 (0.62–0.89) 0.77 (0.68–0.86)
Illness score
  PRISM-II 0.65 (0.52–0.79) 0.61 (0.45–0.79) 0.66 (0.54–0.79)

Abbreviations: ΔeCCl, estimated change in creatinine clearance from baseline; AKI, acute kidney disease; Cr, creatinine; FO, fluid overload; RAI,
renal angina index; UOP, urine output.
Author Manuscript

The area under the curve (AUC) values were calculated for prediction of Day-3 AKI for the RAI broken down by individual components (ΔeCCl,
FO, or the ‘worse’ of the two). The AUC values demonstrate discriminatory superiority of ΔeCCl-derived RAI over FO-derived RAI for Day-3
AKI by all metrics of outcome used (UOP, Cr, or the ‘‘worse’’ variable). Including the FO metric for RAI improved the AUC for AKI measured
by creatinine at Day 3 from 0.73 (RAI derived solely from ΔeCCl) to 0.75 (RAI derived from ‘worse’). Renal angina outperforms severity of
illness scores for prediction of Day-3 AKI. For patients in cohort 1, the discrimination of day of admission RAI for Day-3 AKI was compared
against Pediatric Risk of Mortality-II (PRISM-II) scores. Although the performance of FO-derived RAI is not as robust as ΔeCCl-derived RAI, it is
comparable to PRISM-II and in some cases (Day-3 AKI outcome measured by creatinine clearance change (Cr)) improved. AUC values are
expressed with 95% confidence intervals.
Author Manuscript
Author Manuscript

Kidney Int. Author manuscript; available in PMC 2015 November 25.

 Author Manuscript Author Manuscript Author Manuscript Author Manuscript

Table 5

Fulfillment of renal angina outperforms KDIGO stages of AKI for prediction of subsequent severe AKI

Youden’s
Basu et al.

N Sensitivity Specificity PPV NPV index

KDIGO 1 25 21 (8–41) 84 (76–90) 24 (9–45) 82 (73–88) 5
KDIGO 2–3 24 46 (28–66) 91 (84–95) 54 (33–74) 87 (80–93) 37
ANG(+) 52 75 (55–89) 73 (64–81) 40 (27–55) 92 (85–97) 48

Abbreviations: AKI, acute kidney disease; ANG, renal angina; KDIGO, Kidney Diseases Improving Global Outcomes; NPV, negative predictive value; PPV, positive predictive value; RAI, renal angina
index.

Renal angina outperforms signs of early injury alone for prediction of Day-3 AKI. On the day of admission, patients in cohort 1 were assessed for ANG(+) by RAI ≥ 8 and compared with KDIGO stage 1 or
KDIGO stages 2 and 3. The results demonstrate that ANG(+) is superior to KDIGO stage 1 for prediction of severe subsequent AKI and as effective as KDIGO stages 2 and 3. Both sensitivity and negative
predictive value for ANG(+) are higher than for KDIGO stages 2 and 3. Results are shown as percentages with 95% confidence intervals.

Kidney Int. Author manuscript; available in PMC 2015 November 25.

Page 19