Catheterization and Cardiovascular Interventions 87:302–309 (2016)

PEDIATRIC AND CONGENITAL HEART DISEASE

Editor’s Choice

CRISP: Catheterization RISk Score for Pediatrics: A

Report from the Congenital Cardiac Interventional Study
Consortium (CCISC)
David G. Nykanen,1* MD, Thomas J. Forbes,2 MD, Wei Du,3 PhD,
Abhay A. Divekar,4 MBBS, MD, Jaxk H. Reeves,5 PhD, Donald J. Hagler,6 MD,
Thomas E. Fagan,7 MD, Carlos AC Pedra,8 MD, PhD, Gregory A. Fleming,9 MD, MS,
Danyal M. Khan,10 MD, Alexander J. Javois,11 MD, Daniel H. Gruenstein,12 MD,
Shakeel A. Qureshi,13 MD, Phillip M. Moore,14 MD, and David H. Wax,15 MD,
For the Congenital Cardiac Interventional Study Consortium (CCISC)

Additional Supporting Information may be found in the online Hospital, Oak Lawn, IL; Carlos A. Pedra, MD, PhD, Instituto Dante Pazza-
version of this article. nese de Cardiologia, Sao Paulo, Brazil; Daniel H. Gruenstein, MD, Univer-
1
Arnold Palmer Hospital for Children and the University of sity of Minnesota Amplatz Children’s Hospital, Minneapolis, MN; Danyal
Central Florida College of Medicine, Department of Pediatrics/ M. Khan, MD, Miami Children’s Hospital, Miami, FL; David F. Wax, MD,
Cardiology, Orlando, FL Ann and Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL;
2
Children’s Hospital of Michigan, Department of Pediatrics/ David G. Nykanen, MD, Arnold Palmer Hospital for Children, Orlando,
Cardiology, Detroit, Michigan FL; Dennis W. Kim, MD, PhD, Children’s Healthcare of Atlanta, Atlanta,
3
Wayne State University, Department of Pediatrics, Pharma- GA; Donald J. Hagler, MD, Mayo Clinic, Rochester, MN; Gregory A.
cology, Detroit, Michigan Fleming, MD, MS, Duke University, Durham, NC; James A. Hill, MD,
4
University of Iowa Children’s Hospital, Department of Pedia- University Hospitals Case Medical Center, Cleveland, OH; Joseph N. Gra-
trics/Cardiology, Iowa City, Iowa ziano, MD, Phoenix Children’s Hospital, Phoenix, AZ; Luis E. Alday, MD,
5
University of Georgia, Department of Statistics, Athens, Georgia Sanatorio Allende, Cordoba, Argentina; Makram R. Ebeid, MD, University
6
Mayo Clinic, Department of Pediatrics/Cardiology, Rochester, of Mississippi Medical Center, Jackson, MS; Phillip Moore, MD, MBA,
Minnesota University of California San Francisco, San Francisco, CA; Shakeel A. Qur-
7
University of Colorado, Department of Pediatrics/Cardiology, eshi, MD, Evelina London Children’s Hospital, London, UK; Thomas E.
Denver Fagan, MD, Children’s Hospital Colorado, Denver, CO; Thomas J. Forbes,
8
Instituto Dante Pazzanese De Cardiologia and Hospital Do MD, Children’s Hospital of Michigan, Detroit, MI; Walter Mosquera
Corac¸a~o Da Associac¸a ~o Sanato rio Sırio, Department of Pedia- Alvarez, MD, Fundacion Valle del Lili, Cali, Colombia
trics/Cardiology, Sa ~o Paulo, Brazil
9
Duke University, Department of Pediatrics/Cardiology, Dur- The editor’s choice video can be viewed online at http://onlinelibrary.
ham, North Carolina wiley.com/journal/10.1002/(ISSN)1522-726X/homepage/cci_editor_
10
Miami Children’s Hospital, Department of Pediatrics/Cardiol- s_choice_papers_and_videos.htm.
ogy, Miami, Florida
11 Conflict of interest: Nothing to report.
Advocate Children’s Hospital and the University of Illinois,
Department of Pediatrics/Cardiology, Oak Lawn, Illinois Contract grant sponsor: Festival of Trees (FOT), The DeSeranno
12
University of Minnesota Amplatz Children’s Hospital, Depart- Foundation, Cold Heading Foundation, David and Marie Quint Charita-
ment of Pediatrics/Cardiology, Minneapolis, Minneapolis ble Gift Fund as well as industry support from AGA, Siemens, Gore,
13
Evelina London Children’s Hospital, London, Department of Cook, Medtronic, NuMED, Atrium, BBraun, Cordis, and Arrow.
Paediatrics/Cardiology, United Kingdom
14
University of California San Francisco, Department of Pedia- *Correspondence to: David Nykanen, MD, The Heart Center at
trics/Cardiology, San Francisco, CA Arnold Palmer Hospital for Children, 92 West Miller Street MP
15
Ann and Robert H. Lurie Children’s Hospital, Department of 307, Orlando, FL 32806. E-mail: [email protected]
Pediatrics/Cardiology, Chicago, Illinois
Received 26 April 2015; Revision accepted 2 October 2015
CCISC Risk Registry Participating Principal Investigators, Participating
Institution: Abhay A. Divekar, MBBS, MD, University of Iowa Children’s DOI: 10.1002/ccd.26300
Hospital, Iowa City, IA; Alejandro Peirone, MD, Hospital Privado de Cor- Published online 2 November 2015 in Wiley Online Library
doba, Cordoba, Argentina; Alexander J. Javois, MD, Advocate Children’s (wileyonlinelibrary.com)

C 2015 Wiley Periodicals, Inc.

V
 Risk of Pediatric Cardiac Catheterization 303

Objectives: We sought to develop a scoring system that predicts the risk of serious
adverse events (SAE’s) for individual pediatric patients undergoing cardiac catheteriza-
tion procedures. Background: Systematic assessment of risk of SAE in pediatric cath-
eterization can be challenging in view of a wide variation in procedure and patient
complexity as well as rapidly evolving technology. Methods: A 10 component scoring
system was originally developed based on expert consensus and review of the existing
literature. Data from an international multi-institutional catheterization registry (CCISC)
between 2008 and 2013 were used to validate this scoring system. In addition we used
multivariate methods to further refine the original risk score to improve its predictive
power of SAE’s. Results: Univariate analysis confirmed the strong correlation of each
of the 10 components of the original risk score with SAE attributed to a pediatric car-
diac catheterization (P < 0.001 for all variables). Multivariate analysis resulted in a modi-
fied risk score (CRISP) that corresponds to an increase in value of area under a
receiver operating characteristic curve (AUC) from 0.715 to 0.741. Conclusion: The
CRISP score predicts risk of occurrence of an SAE for individual patients undergoing
pediatric cardiac catheterization procedures. VC 2015 Wiley Periodicals, Inc.

Key words: COMP – complications; pediatric catheterization/intervention

INTRODUCTION and a point value of 0, 1, or 2 was respectively

assigned to each level for each of the 10 variables.
There have been several reports suggesting risk fac-
Where multiple factors contribute to a variable, a
tors for adverse events associated with pediatric cardiac
threshold strategy was employed wherein only the
catheterization procedures. Prior to 2008, most reports
highest scoring factor was used to assign the 0, 1, or 2
involved single-center or limited multi-institutional
points. Anatomic diagnosis utilizing the STS/EACTS
experiences and primarily reflect procedures under-
short list definitions, procedures and physiologic pa-
taken in centers within North America [1–3]. A signifi-
rameters were similarly divided into three levels of
cant challenge to these investigations is the
perceived increasing risk (Supplementary Information
retrospective analysis of databases with relatively low
appendix I–III). A serious adverse event (SAE) was
rate of adverse event occurrence in a diverse popula-
defined as any adverse event causing mortality, perma-
tion with variable patient and procedural characteris-
nent morbidity, need for further interventions, or
tics. The Congenital Cardiac Catheterization Project on
extended length of stay (Table II).
Outcomes (C3PO) study group in the United States
The points assigned to each variable as well as the
began a project in 2006 to better define adverse events
total score (original 20 point risk score) were calculated
and risk adjustment in the current catheterization era
and recorded in the CCISC open unrestricted multi-
[4–10]. This effort has yielded important information
institutional registry whose participating institutions are
with respect to the risk associated with procedure type
characterized by differing volumes, locations and aca-
and developed the multivariate CHARM model for risk
demic affiliations from North America, Europe, and
adjustment of outcomes. As a concurrent effort begin-
Latin America. A list of participating centers is provided
ning in late 2007, the CCISC group has sought to de-
(on-line appendix IV). Patient data were submitted by
velop a pre-catheterization risk scoring system to be
each site via a secure password-protected website. The
applied to individual pediatric patients undergoing car-
server for the CCISC is housed and maintained at the
diac catheterization procedures.
Medical School Information System at Wayne State
University School of Medicine. Data entry was de-
METHODS
identified, so that there was no way to electronically
link specific patient data back to the site from the
A 20-point scoring system consisting of 10 discrete CCISC database application. All local centers obtained
variables (Table I) was developed based on consensus approval for the project with their local Institutional
opinion of pediatric interventional cardiologists. For Review Boards or Research Ethics Committees.
the purposes of this report, this score is referred to as
the original 20-point risk score. The scoring system
was developed, reviewed and accepted at four consecu- Statistical Considerations
tive meetings of the participating CCISC interventional Univariate chi-square analysis was utilized to assess the
cardiologists over a 2-year period. Each variable was correlation of each of the 10 variables in the original 20-
classified into three levels of perceived increasing risk point risk score (Table I), as well as other potential risk
Catheterization and Cardiovascular Interventions DOI 10.1002/ccd.
Published on behalf of The Society for Cardiovascular Angiography and Interventions (SCAI).
304 Nykanen et al.

TABLE I. Calculation of the Risk Score for the Original 20-Point Risk Score
Assigned Points 0 1 2
Patient Status/Timing (X1) Elective Emergent/Urgent Post-operative
Age (X2) > 1 year 30 days–1 year < 30 days
Weight (X3) > 10 kg 2.5–10 kg < 2.5 kg
Inotropic Support (X4) None Yes–Stable Yes–Unstable or ECMO
Respiratory Status(X5) Own Airway Stable on ventilator Respiratory failure on mechanical
or known difficult/unusual ventilation
airway
Systemic illness/failure (X6) none Medically controlled Uncontrolled or > 1 organ system
or 1 organ system failure failure
ASA Score (X7) 1 or 2 3 4 or 5
Physiologic Category (X8) Category 1 Category 2 Category 3
Pre-Catheterization Diagnosis (X9) Category 1 Category 2 Category 3
Procedure Risk Category (X10) Category 1 Category 2 Category 3
ASA, American Society of Anesthesia; ECMO or CPS, Extracorporeal Membranous Oxygenator/mechanical cardiopulmonary support.

factors with SAE occurrence. Procedure type (defined as 0 quartile range: 0.7–10.4 years), and the mean and me-
- diagnostic, 1- interventional, or 2- hybrid in the registry) dian weights were 23.1 kg and 14.5 kg (interquartile
was added to the potential risk factors investigated, as the rage: 7–32.5kg).
C3PO study group [7] demonstrated it to be an important
predictor. The 10 original variables (X1–X10), as well as
procedure type (X11), were then used in a multivariate logis- Adverse Events
tic regression analysis to develop a model for Catheteriza- A total of 1072 SAEs were reported and there were
tion RISk in Pediatrics (CRISP) score. A backward 665 of the total of 14,790 procedures where at least one
stepwise model selection procedure was used to select the SAE was reported (4.5%). A description and frequencies
final subset of variables used in the CRISP score model. of the encountered SAE’s is detailed in Table III. The
In order to determine if the newly developed CRISP most commonly encountered SAE was the need for an
score performs better than the ASA score and the origi- unplanned increase in hemodynamic support. Death
nal 20-point risk score, we fitted each of the three deemed related to the procedure occurred in 12 (0.08%)
scores with a simple logistic regression model to com- of procedures. Unplanned transfusion occurred in 94
pare their ability to predict SAE: (0.64%) of procedures and in 14 (0.09%) following dis-
charge. Re-bleeding at the site of access after achieving
lnðP=ð1-PÞÞ5b01b1ðrisk scoreÞ (11) hemostasis severe enough to require transfusion
occurred in 19 (0.13%) of procedures. An emergent sur-
where P is the probability of a patient developing an gical procedure was required in 69 (0.47%). Arrhythmia
SAE using the respective risk model. requiring intervention occurred in 87 procedures. These
The three risk score models were then compared were managed medically in 27, with temporary pacing
using: (1) N2LL ¼ 2log Likelihood (an assessment in 13 and with DC cardioversion or defibrillation in 47.
for model fit), (2) Akaike’s Information Criteria Of the 75 instances of device migration during the proce-
defined as AIC ¼ N2LLþ(2*k) which is a best model dure, 44 were retrieved with transcatheter techniques, 13
fit statistic, adjusting for k (where k ¼ number of pa- required open surgical removal, one removed via cut-
rameters), (3) Schwarz’s Bayes Information Criteria, down, and the remaining 17 were not removed. Devices
defined as BIC ¼ N2LLþ(ln(N)*k) (where N ¼ Sample that migrated post-discharge were removed via catheter
size), and (4) AUC ¼ Area under the receiver operating procedure in 4/11, removed surgically in 4/11 and left in
characteristic curve [11]. situ in 3/11. Complete heart block occurred in 36 proce-
dures. Twenty-five of these resolved in the catheteriza-
tion laboratory. Of the remaining 11, 9 resolved while 2
RESULTS
required permanent pacemaker implants. Access arterial
From 2008 to 2013, 18,564 procedures were entered compromise was reported in 104 procedures. These were
into the database from 20 centers. The number of oper- managed with heparin infusions in 89, thrombolytic ther-
ators at each center ranged from 1–5. This study only apy in 6, with transcatheter embolectomy in 3 and with
included the results of the 14,790 procedures under- surgery in 6. Fifty of 60 vascular injuries (excluding
taken in patients less than 18 years of age. The mean access site) were managed with observation alone whilst
and median ages were 5.8 years and 3.7 years (inter- 10 were sent to surgery.
Catheterization and Cardiovascular Interventions DOI 10.1002/ccd.
Published on behalf of The Society for Cardiovascular Angiography and Interventions (SCAI).
 Risk of Pediatric Cardiac Catheterization 305

TABLE II. Significant Adverse Events

 Access site re-bleeding requiring transfusion
 Air embolism requiring further treatment including hyperbaric therapy
 Airway compromise, unanticipated. (Failed extubation, accidental extubation during procedure, need of intubation during/after the procedure, post-
extubation airway obstruction requiring observation/intervention, extended need of mechanical ventilation
 Anaphylactic reaction
 Arrhythmia, hemodynamically unstable, requiring pharmacologic intervention
 Arterial compromise requiring surgical or transcatheter intervention
 Arteriovenous fistula requiring surgical or transcatheter intervention
 Atrioventricular valve regurgitation significantly increased and related to the procedure
 Balloon rupture causing vascular damage (requiring surgical or transcatheter intervention)
 Bleeding/hematoma at access site requiring re-visit
 Brachial plexus injury
 Cardiac arrest/bronchospasm with induction
 Cardiac arrest, unexpected, within 24 hr of procedure
 Cardiac perforation
 Complete heart block that is unresolved at the end of procedure
 Device migration requiring open surgical removal, removal via cut down or transcatheter retrieval
 Device migration, post-procedure
 Hemopericardium
 Hematoma resulting in extended length of stay
 Hemothorax
 Infection (systemic/endocarditis), death related to procedural complication
 Myocardial infarction
 Pericardial effusion requiring surgical intervention or pericardial drainage
 Pseudoaneurysm requiring surgical or transcatheter intervention
 Pulmonary compromise (pulmonary hemorrhage/hemoptysis or pulmonary edema
 Pulmonary embolus requiring intervention
 Renal compromise
 Retained device, unintentional
 Retroperitoneal hematoma
 Second organ system injury requiring extended hospitalization or therapeutic intervention
 Seizure (new) within 48 hr of procedure
 Separate procedure required emergently/urgently
 Stent malposition (surgical or transcatheter intervention)
 Stroke
 Systemic embolism
 Transfusion, unanticipated
 Unanticipated escalation of hemodynamic support, CPS/ECMO
 Vascular injury (i.e. dissection, intimal tear, aneurysm) requiring surgical or transcatheter intervention
 Venous compromise requiring surgical or transcatheter intervention
 Wire embolization/retention requiring transcatheter retrieval or removal via cut down or open surgical removal
 Other post-discharge complications not listed which is serious, requires further intervention, increases length of stay or causes revisit or hospitaliza-
tion (adjudicated by principal investigators)

Risk Score Model X2ab ¼1 if age score¼1 or 2; 0 if age score¼0

Table IV lists the variables that were demonstrated X3ab¼1 if weight score¼1 or 2; 0 if weight score¼0
to be highly predictive of SAE in univariate analysis. X4b¼1 if inotropic score¼2; 0 if inotropic score¼0 or 1
These include each of the 10 variables (X1–X10) X6b¼1 if systemic illness¼2; 0 if systemic illness score¼0 or 1
included in the original 20-point risk score, as well as X8a¼ 1 if physiologic score¼1; 0 if physiologic score¼0 or 2
the procedure type (X11). All 11 variables significantly X8b¼ 1 if physiologic score¼ 2; 0 if physiologic score¼0 or 1
predicted SAE independently, but not all variables X9ab¼1 if diagnosis score¼1 or 2; 0 if diagnosis score¼0
were included in the final risk score model due to the X10a¼ 1 if procedure score ¼ 1, 0 if procedure score ¼ 0 or 2
inter-correlation of these variables. A backward model X10b¼1 if procedure risk score¼2; 0 if procedure risk
selection method yielded the following model with the score¼0 or 1
regression coefficients rounded to the nearest integers. X11ab¼1 if procedure type¼interventional or hybrid; 0 if pro-
cedure risk score=diagnostic
CRISP52X2ab 12X3ab 12X4b 13X6b 1X8a (see Table I for the original 0-2 point definition)
14X8b 12X9ab 1X10a 13X10b 13X11ab The multivariable model selection procedure used
where to define the CRISP score model determined that
Catheterization and Cardiovascular Interventions DOI 10.1002/ccd.
Published on behalf of The Society for Cardiovascular Angiography and Interventions (SCAI).
306 Nykanen et al.

TABLE III. Reported Serious Adverse Events (SAE)a TABLE IV. Chi-Square Analysis of SAE by Patient/Procedural
Characteristics (N 5 14,790)
SAE N %
Characteristic % SAE (%) P-value
Increase in hemodynamic support 134 0.91
Unanticipated increase 110 0.74 Patient Status/Timing (X1)
in hospitalization Elective 91% 4% < 0.001
Arterial compromise requiring 104 0.70 Emergent/Urgent 7% 12%
surgical or transcatheter Post-operative 2% 8%
intervention Patient Age (X2) < 0.001
Unplanned transfusion 94 0.64 > 1 year 71% 3%
Arrhythmia requiring pharmacologic 87 0.59 30 days – 1 Year 22% 7%
intervention < 30 days 7% 12%
Device migration 75 0.51 Patient Weight (X3) < 0.001
Emergent procedure required 69 0.47 > 5 kg 86% 3%
Vascular injury requiring surgical or 60 0.40 2.5–5.0 kg 13% 11%
transcatheter intervention < 2.5 kg 1% 12%
(excluding access vessel) Inotropic Support (X4) < 0.001
Post-discharge SAE 41 0.28 None 93% 4%
Complete Heart Block 36 0.24 Yes - Hemodynamically stable 6% 8%
Secondary organ injury 29 0.19 Yes - Unstable or mechanical 1% 21%
Cardiac Arrest within 24 hr 25 0.17 cardiopulmonary support
Cardiac Perforation 24 0.16 Respiratory Status (X5) < 0.001
Hemopericardium 21 0.14 Own Airway 92% 4%
Cardiac Arrest/bronchospasm 21 0.14 Stable on Ventilator or known 7% 9%
at induction difficult/unusual airway
Re-bleed requiring transfusion 19 0.13 Respiratory Failure on 1% 12%
Stroke 14 0.09 mechanical ventilation
Post-discharge transfusion 14 0.09 Systemic Illness/Failure (X6) < 0.001
Death related to procedure 12 0.08 None 70% 4%
Venous Compromise requiring surgical 11 0.07 Medically controlled or 26% 5%
or transcatheter intervention 1 organ system failure
Post-discharge device migration 11 0.07 Uncontrolled or < 1 organ 4% 13%
Unplanned mechanical cardiopulmonary 11 0.07 system failure
support ASA Score (X7) < 0.001
Hemothorax 9 0.06 1 or 2 67% 3%
Anaphylaxis 7 0.05 3 25% 6%
Seizure 7 0.05 4 or 5 8% 9%
Renal compromise 6 0.04 Physiologic Score (X8) < 0.001
Unplanned device retention 5 0.03 Category 1 67% 3%
Pulmonary embolism 4 0.03 Category 2 23% 5%
Pseudoaneurysm 3 0.02 Category 3 10% 12%
AV Fistula 3 0.02 Pre-Catheterization Diagnosis (X9) < 0.001
Retroperitoneal hematoma 2 0.01 Category 1 50% 3%
Systemic embolism requiring surgery 2 0.01 Category 2 46% 6%
Infection 2 0.01 Category 3 4% 5%
Total 1072 Procedure Risk Category (X10) < 0.001
a Category 1 86% 4%
There were 1072 SAE’s reported in 665 of 14,790 (4.5%) procedures.
Category 2 12% 8%
Category 3 2% 15%
Procedure Type (X11) < 0.001
Diagnostic 38% 3%
timing (X1), pre-catheterization airway status (X5),
Interventional 61% 5%
and ASA score (X7) did not significantly add to the Hybrid 1% 12%
predictive power of the model. Table V illustrates
the point values assigned to each variable in the
CRISP model, resulting in a total possible score Not surprisingly the incidence of SAE increased
ranging from 0 to 21. The comparisons of the three with increasing CRISP score (Table VII). No patient
scores are summarized in Table VI and demonstrate received a score of 20. There is also a progressive
the CRISP score to be superior, as it has the lowest decrease in the absolute number of catheterizations rep-
N2LL, as well as lowest AIC and BIC (best model resented as CRISP score increases. A CRISP score of
fit statistics, adjusted for # of parameters and sample exactly 1 appears relatively under-represented in the
size), and highest percent agreement measures by dataset. Procedures receiving a score of 1 were repre-
AUC. sented predominantly by physiologic score (n ¼ 616,
Catheterization and Cardiovascular Interventions DOI 10.1002/ccd.
Published on behalf of The Society for Cardiovascular Angiography and Interventions (SCAI).
 Risk of Pediatric Cardiac Catheterization 307

TABLE V. CRISP Score Point Assignment TABLE VII. Frequency of SAE by CRISP Score
Patient Clinical Characteristics Points assigned CRISP SCORE N SAE (%)
Age (X2) 0 1367 1.0
>1 year 0 1 305 1.3
30 days–1 year 2 2 1297 0.9
<30 days 2 3 4163 2.0
Weight (X3) 4 1017 3.1
>5kg 0 5 1442 3.8
2.5–5 kg 2 6 1641 5.7
<2.5 kg 2 7 861 5.1
Inotropic support (X4) 8 636 6.9
None 0 9 639 8.3
Yes-Stable 0 10 505 15.1
Yes-Unstable/ECMO 2 11 380 11.1
Systemic illness/organ failure (X6) 12 159 14.5
None 0 13 170 22.9
Medically controlled/1 organ failure 0 14 94 8.5
Uncontrolled/> 1 organ failure 3 15 21 33.3
Physiologic Category(X8) 0 16 63 33.3
Category 1 17 9 44.4
Category 2 1 18 10 40.0
Category 3 4 19 8 37.5
Pre-Cath Diagnosis (X9) 20 0 0.0
Category 1 0 21 3 100.00
Category 2 2 Total 14790 4.5
Category 3 2
Procedure Category (X10)
Category 1 0 DISCUSSION
Category 2 1
Category 3 3 A diverse, often physiologically complex patient pop-
Procedure type (X11) ulation makes the assessment of risk in pediatric cardiac
Diagnostic 0 catheterization difficult, especially as there is a rela-
Interventional 3 tively low incidence of reported serious adverse events.
Hybrid 3
In this project only adverse events resulting death or
Total CRISP Score ¼________________. disability, an unanticipated increase in length of stay or
Possible Risk Score ¼ 0–21.
a need for medical, surgical or transcatheter manage-
ment were captured. Comparison with the existing liter-
TABLE VI. Comparison of Risk Score Models in Prediction of ature must consider variations in the definition of a
SAE
captured event [5,7,8,12–14]. For example, the incidence
Risk Model N2LL AIC BIC AUC of access vessel compromise represented in this report
ASA 5307 5311 5326 0.612 should be interpreted cautiously as there is considerable
Original Risk Score 5036 5058 5142 0.715 practice variation in the diagnosis and management of
CRISP 4903 4925 5009 0.741
the weak or absent pulse following catheterization [15].
AIC, Akaike’s Information Criteria; ASA, American Society of Anesthe- The study of mortality in studies of pediatric cardiac
sia Score; AUC, Area under the receiver operator curve; BIC, Schwarz’s catheterization is difficult, especially in a registry-based
Bayes Information Criteria; N2LL, 2log Likelihood (an assessment for
model fit).
report. Definitions in many remain paradoxically difficult.
In this registry, the supervising cardiologist who has an
inherent bias determined the attribution of mortality to the
65.0%) and age (n ¼ 330, 34.9%). The only other pro- procedure. Other indices such as all cause 30-day mortal-
cedure represented was one in which the weight ity, mortality during the same hospital admission or
received 1 point, indicating a procedure for an infant within 30 days of discharge, mortality associated with sal-
with low weight for age. The occurrence of only one vage catheterization, and others. This difficulty of attribu-
risk parameter in isolation is frequent but occurs rela- tion and the practicality of applying it to risk of future
tively less than predicted. Due to the relatively low procedures is a problem that may be addressed by having
number of procedures, for practical purposes, we pro- all mortality within a specifically defined period of the
pose grouping as CRISP groups 1-5 with an observed procedure reviewed by an unbiased analyst. Even though
SAE risk of 1%, 3%, 5%, 14%, and 37% respectively, the incidence of death in this and other studies is low, it
rounded to the nearest whole percent. (Table VIII). must continue to be interpreted with caution.
Catheterization and Cardiovascular Interventions DOI 10.1002/ccd.
Published on behalf of The Society for Cardiovascular Angiography and Interventions (SCAI).
308 Nykanen et al.

TABLE VIII. Incidence of SAE by Proposed Risk Category ture, our model predicts that other variables may over-
Risk Category CRISP Score N SAE SAE (%) shadow age and weight in determining the risk of the
CRISP 1 0 to 2 2969 29 1.0 procedure in young children. As demonstrated by other
CRISP 2 3 to 5 6622 171 2.6 experiences and confirmed most recently by the C3PO
CRISP 3 6 to 9 3777 235 6.2 study group, the complexity of the procedure and underly-
CRISP 4 10 to 14 1308 188 14.4 ing diagnosis remained strong contributors [5,7,13]. In
CRISP 5 15 or more 114 42 36.8
addition, the patient’s physiologic status was also an im-
portant contributor to the CRISP model. The CRISP score
The C3PO group has recently developed and vali- also supports a difference between diagnostic and inter-
dated the Congenital Heart Disease Adjustment for Risk ventional or hybrid catheterizations which have been pre-
Method (CHARM) to allow for better comparisons in viously reported to contribute to the risk possibly due to
analysis of data across institutions and practitioners many of these procedures being innovative or undertaken
[4,7]. Given the concurrent development of this pre- for unusual circumstances [3,8].
procedural risk score and the C3PO group’s efforts in For practical purposes we propose that at present the
risk adjustment it is not surprising that many features scores be further grouped into five risk categories from
are similar. The primary objective of this endeavor was CRISP 1 to CRISP 5 (Table VIII). Table V in combination
to utilize a contemporary multi-centered registry to vali- with the definitions provided by on-line appendices I–III
date a predictive scoring system for pediatric cardiac provide a tool that can be utilized to assign a CRISP score
catheterization. Arguably some of the features of the to any catheterization in the pediatric population prior to the
physiologic parameters were determined during the pro- procedure. This in concert with the type of SAE observed
cedure however these parameters can be predicted non- (Table III) may provide valuable information to the operator
invasively. Future study applied to resource planning for for pre-catheterization counseling and preparation.
procedures and patient safety may be enhanced by anal- There are limitations associated with a registry-based
ysis of specific subsets of the registry to provide insight study. The quality of the data is only as accurate as the
into the number and type of adverse event encountered attention to detail inherent in data entry and the vigilance
in specific subsets of patients. of investigators to ensure that all events are captured. As
The original 20-point score proved to be a valid predic- participation in the registry was voluntary, there was no
tor of SAE’s. Reducing each of the variables to ordinal cat- provision for site visits to randomly audit procedures for
egorical data was intentional in an effort to develop a accuracy. Ten Percent of submissions were reviewed for
relatively simple means of assessing risk for patients and completeness and consistency however there was no audit
procedures with considerable variation. The CRISP score to ensure that each center reported all procedures and all
represents a more robust predictor, confirmed by the agree- SAE’s. Underreporting underestimates the true incidence
ment measure AUC and this may be utilized to plan and of SAE’s in pediatric catheterization and misrepresents the
prepare for available equipment, personnel and support in true risk of a procedure. While it is re-assuring that the
advance of the procedure. Comparison between centers or observed risk is consistent with that of the published litera-
individual operators to identify best practices can be facili- ture the same vulnerability exists in those reports as well.
tated, recognizing that confidence intervals will be broader It is for this reason that only serious adverse events were
where procedure volumes are lower. Whether such use captured, as these are readily identified outcomes and they
results in a lower incidence of SAE’s remains speculative. are perceived to have an important impact on patient out-
The assignment of ordinal scores was arbitrary and fur- come. Unlike the previous reports in the literature, the cur-
ther analysis demonstrated that not all variables were rent study included, having an unanticipated increase in
equal in predicting risk. Three variables were eliminated. length of stay as an SAE. This proved to be a sensitive and
Airway status prior to catheterization failed to contribute easily identified indicator of SAE occurrence. As an unan-
significantly, which is likely due to differences in institu- ticipated increase in length of stay is most commonly asso-
tional preferences rather than being a surrogate for risk. ciated with the occurrence of a defined SAE it improves
Similarly, all procedures has an ASA score assigned by capture of events. However, without a formal systematic
the cardiologist or the anesthesiologist attending the pro- audit there remains a potential for missed events, espe-
cedure, purported to reflect risk, however this did not con- cially following patient discharge. An audit provides an
tribute to the final model perhaps reflecting large opportunity for follow-up quality projects and improves
variation in ASA assignment. Timing of the procedure generalization of observations, especially where surveil-
was not as strong a predictor as other covariates. Patient lance and follow-up may differ between centers. Data entry
age and weight contributed similarly to increased risk in was facilitated with a web-based tool that was designed to
the CRISP model regardless of whether the patient was an allow raw data entry to be accomplished easily. To limit
infant or neonate. While this seems contrary to the litera- errors, assignment of points contributing to the risk score
Catheterization and Cardiovascular Interventions DOI 10.1002/ccd.
Published on behalf of The Society for Cardiovascular Angiography and Interventions (SCAI).
 Risk of Pediatric Cardiac Catheterization 309

was calculated from the variables entered. All entries were 4. Bergersen L, Gauvreau K, Foerster SR, Marshall AC,
examined for discrepancies prior to data lock and discrep- McElhinney DB, Beekman RH, 3rd Hirsch R, Kreutzer J,
Balzer D, Vincent J, Hellenbrand WE, Holzer R, Cheatham JP,
ancies adjudicated by two investigators (DGN, TJF). Moore JW, Burch G, Armsby L, Lock JE Jenkins KJ. Catheteri-
Another important limitation of the data is that there was zation for congenital heart disease adjustment for risk method
no systematic assessment of the success of any procedure (CHARM). JACC Cardiovasc Interv 2011;4:1037–1046.
performed. To date, there is limited standardized assess- 5. Bergersen L, Gauvreau K, Jenkins KJ, Lock JE. Adverse event
ment of success of a procedure and likely will be the sub- rates in congenital cardiac catheterization: A new understanding
of risks. Congen Heart Dis 2008;3:90–105.
ject of future studies. This study focused exclusively on 6. Bergersen L, Gauvreau K, Lock JE, Jenkins KJ. A risk adjusted
the pediatric population defined as less than 18 years of method for comparing adverse outcomes among practitioners in
age. The findings of this study are not generalizable to the pediatric and congenital cardiac catheterization. Congen Heart
adult population with congenital heart disease where other Dis 2008;3:230–240.
co-morbidities may have greater influence. With these lim- 7. Bergersen L, Gauvreau K, Marshall A, Kreutzer J, Beekman R,
Hirsch R, Foerster S, Balzer D, Vincent J, Hellenbrand W,
itations in mind, we believe the data to be representative of Holzer R, Cheatham J, Moore J, Lock J, Jenkins K. Procedure-
a multi-institutional experience. type risk categories for pediatric and congenital cardiac catheter-
ization. Circ Cardiovasc Interv 2011; 4:188–194.
8. Bergersen L, Marshall A, Gauvreau K, Beekman R, Hirsch R,
CONCLUSIONS Foerster S, Balzer D, Vincent J, Hellenbrand W, Holzer R,
Cheatham J, Moore J, Lock J, Jenkins K. Adverse event rates in
This investigation assessed a risk stratification score congenital cardiac catheterization—A multi-center experience.
using a registry based multi-institutional international Catheter Cardiovasc Interv 2010;75:389–400.
database of pediatric cardiac catheterizations. The pro- 9. Holzer RJ, Gauvreau K, Kreutzer J, Moore JW, McElhinney
posed scoring system was validated, but further analy- DB, Bergersen L. Relationship between procedural adverse
events associated with cardiac catheterization for congenital
ses of the variables resulted in a more accurately heart disease and operator factors: Results of a multi-
weighted scoring system defined as the Catheterization institutional registry (C3PO). Catheter Cardiovasc Interv 2013;
Risk in Pediatrics (CRISP) score. We suggest that this 82:463–473.
score can be assigned prior to catheterization to pro- 10. Learn CP, Holzer RJ, Daniels CJ, Torres AJ, Vincent JA,
vide a risk assessment (CRISP) that accounts for spe- Moore JW, Armsby LB, Landzberg MJ, Bergersen L. Adverse
events rates and risk factors in adults undergoing cardiac cathe-
cific patient and procedural characteristics. terization at pediatric hospitals—results from the C3PO. Cathe-
ter Cardiovasc Interven 2013;81:997–1005.
11. Burnbaum KP, Anderson DR. Model Selection and Multimodal
ACKNOWLEDGMENTS Inference: A Practical Information-Theoretic Approach. 2nd ed.
New York, NY: Springer-Verlag; 2002.
The authors would like to acknowledge the important 12. Backes CH, Cua C, Kreutzer J, Armsby L, El-Said H, Moore
contribution of the nursing and technical staff involved JW, Gauvreau K, Bergersen L, Holzer RJ. Low weight as an
with data collection and entry at each of the investiga- independent risk factor for adverse events during cardiac cath-
tional sites. Nancy Sullivan RN and Diasuke Kobayashi eterization of infants. Catheter Cardiovasc Interv 2013;82:
MD have provided invaluable leadership and assistance 786–794.
13. Lin CH, Hegde S, Marshall AC, Porras D, Gauvreau K, Balzer
with coordination of investigational sites, communica- DT, Beekman RH, 3rd, Torres A, Vincent JA, Moore JW,
tion and maintaining and updating the database. Holzer R, Armsby L and Bergersen L. Incidence and manage-
ment of life-threatening adverse events during cardiac catheteri-
REFERENCES zation for congenital heart disease. Pediatric Cardiol 2014;35:
140–8.
1. Cassidy SC, Schmidt KG, Van Hare GF, Stanger P, Teitel DF. 14. Rhodes JF, Asnes JD, Blaufox AD, Sommer RJ. Impact of low
Complications of pediatric cardiac catheterization: A 3-year body weight on frequency of pediatric cardiac catheterization
study. J Am College Cardiol 1992;19:1285–1293. complications. Am J Cardiol 2000; 86:1275–1278, A9.
2. Mehta R, Lee KJ, Chaturvedi R, Benson L. Complications of 15. Glatz AC, Shah SS, McCarthy AL, Geisser D, Daniels K,
pediatric cardiac catheterization: A review in the current era. Xie D, Hanna BD, Grundmeier RW, Gillespie MJ, Rome JJ.
Catheter Cardiovasc Interv 2008;72:278–785. Prevalence of and risk factors for acute occlusive arterial
3. Vitiello R, McCrindle BW, Nykanen D, Freedom RM, Benson injury following pediatric cardiac catheterization: A large
LN. Complications associated with pediatric cardiac catheteriza- single-center cohort study. Catheter Cardiovasc Interv 2013;
tion. J Am College Cardiol. 1998;32:1433–1440. 82:454–462.

Catheterization and Cardiovascular Interventions DOI 10.1002/ccd.

Published on behalf of The Society for Cardiovascular Angiography and Interventions (SCAI).