Our CDR derivation and validation studies. These two prospective,
cross-sectional, strictly observational studies were conducted in accordance
with established standards for CDR development at 18 sites in the U.S. and
Canada. Ten sites participated in both
studies. In both studies, doctors were
kept blinded to the CDR’s final form and content. The two studies used the same inclusion and
exclusion criteria, data forms, methods, and a priori definitional criteria for
AHT. To minimize circular reasoning,
these criteria contained no references to any of the clinical variables
considered for inclusion in the CDR. Eligible
patients were children less than three years old who were hospitalized for
intensive care of symptomatic, acute, closed, traumatic, cranial or
intracranial injuries confirmed on initial head CT or MRI. Patients were excluded if their head injuries
resulted from collisions involving motor vehicles or if initial neuroimaging
revealed clear evidence of pre-existing brain malformation, disease, infection,
or hypoxia-ischemia.
The CDR derivation study was conducted
between February 2010 and August 2011 at 14 PICU sites. The goal was to
identify a cluster of reliable predictor variables that would perform
effectively as an AHT screening tool. To
this end, physician-investigators [1] captured historical, clinical, and
radiological data regarding 209 eligible patients; [2] identified 13 highly
discriminating (p ≤ 0.001) and
highly reliable (κ = 0.68-0.95) clinical variables readily available at or
near the time of PICU admission; [3] calculated each variable’s isolated
predictive qualities (sensitivity, specificity, predictive values, likelihood
ratios, post-test probabilities); [4] applied a recursive partitioning
algorithm designed to penalize missed cases of AHT; and [5] identified four
clusters of these 13 variables that, alone or in combination, detected AHT with
sensitivity ≥ 0.92. These four
clusters of predictor variables became our four candidate CDRs.
The CDR validation study was conducted
between March 2012 and July 2013 at 14 PICU sites. The goal was to verify the candidate CDRs’ AHT
screening performance in a new, equivalent, patient population. Physician-investigators [1] captured
equivalent historical, clinical, and radiological data for 291 additional
patients; [2] confirmed the AHT screening performance of all four candidate
CDRs in a new, equivalent, patient population; and [3] determined that a
four-variable CDR demonstrated the optimal combination of simplicity and AHT
screening performance. All measures of this four-variable CDR’s
screening performance matched or exceeded equivalent results from the completed
derivation study. Most importantly, the CDR demonstrated sensitivity of 0.96 in
both studies.
Secondary analyses. We also
conducted multiple secondary analyses
of the captured data regarding the 500 acutely head-injured PICU patients in
our derivation and validation study datasets.
Our CDR has been broadly validated. Analysis of
the CDR’s screening performance in novel patient cohorts revealed that it would
have performed with high sensitivity (0.95 to 0.98) in different geographical
regions, in large and small PICUs (0.94 to 0.99), and in the same PICUs over
time (0.97 to 0.99).
Current AHT screening and evaluation practices
demonstrate site-by-site disparities. The percentage of eligible patients who underwent both skeletal survey and
retinal examination varied from 33% to 100% across the 18 participating sites. The percentage of eligible patients who underwent at least one of these abuse
evaluations varied from 43% to 100%, and the percentage who underwent neither
abuse evaluation varied from 0% to 57%.
Non-clinical factors appear to impact doctors’ decisions to
launch or forgo child abuse evaluations in their young, acutely head-injured
patients. Exploratory analyses
identified four non-clinical factors associated with significant differences (p < .05) in the percentages of high or low risk patients evaluated for
child abuse. These factors included [1]
caregiver denial of any head trauma, [2] patients of minority race/ethnicity,
[3] patients not yet cruising or walking prior to PICU admission, and [4]
patient age <6 months at the time of PICU admission.
Applied as a directive decision tool, the CDR is
highly sensitive—no matter how we define AHT. Gold standard definitional criteria for AHT do not exist. As defined by our a priori criteria, the
prevalence of AHT in our combined patient population was 0.44. The CDR would have correctly identified
(categorized as high risk) 96% of patients who met these criteria. Applying
several modified (more inclusive and less inclusive) AHT criteria to
iteratively re-sort the patient population, our resulting calculations of AHT
prevalence ranged from 0.28 to 0.52. Subsequent
analyses revealed that the CDR would have correctly identified (categorized as
high risk) at least 96% of AHT patients—no matter how we defined AHT. Applied at the time of PICU admission, the CDR
would also have correctly identified 98% of patients ultimately diagnosed with
probable or definitive AHT, and 99% of patients whose completed skeletal
surveys and/or retinal exams revealed corroborating findings of abuse.
Applied as an assistive prediction tool, the CDR’s
patient-specific estimates of abuse probability are good predictors of the
results of completed abuse evaluations. To calculate an evidence-based,
patient-specific, estimate of abuse probability for every patient, we [1]
divided the patient population (N =
500) into its 16 subpopulations defined by each patients’ specific combination
of the CDR’s four predictor variables, [2] calculated the percentage of
patients within each subpopulation who met our a priori criteria for AHT, and
[3] assigned that same value—expressed as an estimate of abuse probability—to
every patient within that same subpopulation. The resulting patient-specific
estimates of abuse probability correlated positively and very strongly (R = 0.708) with the overall diagnostic
yields of the patients’ completed abuse evaluations (the percentage of patients
within each subpopulation whose completed skeletal survey and/or retinal exam
revealed corroborating findings of abuse). This correlation increased significantly (R = 0.946) when we eliminated the single
patient subpopulation that contained only one patient.
Applied accurately and consistently as a directive
decision tool, the CDR can improve AHT screening accuracy. To estimate
the CDR’s potential impact on AHT screening accuracy in PICU settings, we
applied mean (patient-specific) estimates of abuse probability to predict
additional positive abuse evaluations (cases of missed AHT) among high risk and
low risk patients lacking skeletal survey and/or retinal exam, and then used
these predictions to extrapolate and compare “current PICU screening for AHT” vs. “AHT screening guided by the CDR.” Our results suggest that, applied accurately
and consistently as a directive AHT screening tool, the CDR could [1] increase
AHT detection (from 87% to 96%), [2] increase the overall diagnostic yield of
patients’ completed abuse evaluations (from 49% to 56%), [3] reduce (mean)
evidence-based estimates of abuse probability (estimates of missed AHT) among
patients not evaluated thoroughly for abuse (from 0.19 to 0.07), and [4] reduce
unnecessary abuse evaluations of patients with non-AHT (from 67% to 60%).
Our CDR could decrease AHT-associated acute health
care costs in PICU settings. Applying average skeletal survey, retinal exam,
and visit costs obtained from the MarketScan Commercial Claims and Encounters database, we estimate that accurate and
consistent CDR application as a directive AHT screening tool could reduce the
average cost per correctly identified child with AHT from $458 to $389—a
reduction of 15.1%. Applying published
data on re-injury rates in children with missed AHT and the costs of emergency
department/clinic visits and hospital re-admissions resulting from re-injury,
we estimate a 72.4% reduction in total AHT-associated acute health care costs
stemming from the anticipated reduction in missed AHT cases.
Our preliminary assessment of CDR “acceptability”
among physicians yielded favorable results. The Ottawa
Acceptability of Decision Rules Instrument (OADRI) is a validated 12-item
survey that evaluates decision rule acceptability among clinicians. The OADRI’s
final score is calculated as the mean of all 12 items, resulting in a score
that ranges from 0 to 6 with 6 being greatest
acceptability. To estimate the CDR’s
baseline acceptability among potential users, we distributed a brief,
voluntary, anonymous survey to pediatric intensivists and child abuse
pediatricians who participate in the dedicated listservs of the PALISI research network and the Ray E. Helfer Society. Survey participants were
provided a one-page document that summarized the CDR’s derivation, validation,
and potential clinical impact. Participants were asked to answer the OADRI’s
12 questions “as if they were considering using the rule.” A total of 30 pediatric intensivists, 51 child
abuse pediatricians, and 2 other physicians completed the survey. The mean
OADRI score for the total sample (N=
83) was 4.86 (SD =1.32). A clear majority of participants endorsed
items indicating that the CDR was “easy to use”, “easy to understand”,
“useful”, “clear”, and “will benefit patients”. Though not generalizable, the results suggest
that many physicians will actively consider the CDR’s application as an AHT
screening tool.