Category: Clinical Decision Rules

The Febrile Infant Step-by-Step

You’ve heard of the Philadelphia Criteria. You’ve heard of the Rochester Criteria. But – Step-by-Step?

This is an algorithm developed by European emergency physicians to identify low-risk infants who could be safely managed without lumbar puncture nor empiric antibiotic treatment. After retrospectively validating their algorithm on 1,123 patients, this is their prospective validation in 2,185 – looking for IBI or “Invasive Bacterial Infection” as their primary outcome.

The easiest way to summarize their algorithm and results is by this figure:

Step by Step

Sensitivity and specificity, respectively, were as follows:

  • Rochester – 81.6% and 44.5%
  • Lab-score – 59.8% and 84.0%
  • Step-by-Step – 92.0% and 46.9%

The authors attribute 6 of the 7 missed by Step-by-Step to evaluation early in the disease process – presentation within 2 hours of onset of fever.

Their algorithm is reasonable at face validity, and could be incorporated into a protocol with close follow-up to re-evaluate those early in their disease process. We still have, however, a long way to go regarding specificity.

“Validation of the “Step-by-Step” Approach in the Management of Young Febrile Infants”
http://www.ncbi.nlm.nih.gov/pubmed/27382134

Next Up in Syncope Prediction

The Great White North is the land of clinical decision instruments.  Canadian Head, Canadian C-Spine, Ottawa Ankle, Ottawa SAH, the list goes on – and now, from the same esteemed group: the Canadian Syncope Risk Score.

The vast majority of patients with syncope have unrevealing initial and, if admitted, in-house evaluation.  That said, any physiologic interruptions in the ability to perfuse the brain portend a poor prognosis greater than the general background radiation.  These authors performed an observational study over the course of four years to prospectively derive a decision instrument to support risk-stratification for syncope.

There were 4,030 patients enrolled and eligible for analysis based on 30-day follow-up, and 147 of these suffered a “serious adverse event”.  They identified 43 candidate predictors for prospective collection, and ultimately this resulted in a multivariate logistic regression predictive model with 9 elements.  Scores range from -3, with a 0.4% estimated risk for SAE, to 11, with an 83.6% estimated risk for SAE.  Useable confidence intervals, however, were mostly scores <5.

There are a few things I would quibble with regarding this study.  The “serious adverse event” definition is rather broad, and includes 30-day events for which the underlying pathology was not present or necessarily preventable at the initial visit.  For example, a patient with a subsequent encounter for a GI bleed or a case of appendicitis fit their criteria of SAE.  This would diminish the instrument’s apparent sensitivity without materially improving its clinical relevance.  Then, there is the oddity of incorporating the final ED diagnosis into the scoring system – where a provisional diagnosis of “vasovagal syncope” is -2, and a diagnosis of “cardiac syncope” is +2.  The authors explicitly defend its inclusion and the methods behind it – but I feel its subjectivity coupled with widespread practice variation will impair this rule’s generalizability and external validation.

Finally, the last issue with these sorts of “rules”: “high risk” is frequently conflated to mean “admit to hospital”.  In many situations close to the end-of-life, the protective effect of hospitalization and medical intervention vanishes – and may have little or no value.  This sort of stratification should be applied within the appropriate medical and social context, rather than simply triggering admission.

“Development of the Canadian Syncope Risk Score to predict serious adverse events after emergency department assessment of syncope”
http://www.ncbi.nlm.nih.gov/pubmed/27378464

Perpetuating the Flawed Approach to Chest Pain

Everyone has their favored chest pain accelerated diagnostic risk-stratification algorithm or pathway these days.  TIMI, HEART, ADAPT, MACS, Vancouver, EDACS – the list goes on and on.  What has become painfully clear from this latest article, however, is this approach is fundamentally flawed.

This is a prospective effectiveness trial comparing ADAPT to EDACS in the New Zealand population.  Each “chest pain rule-out” was randomized to either the ADAPT pathway – using modified TIMI, ECG, and 0- and 2-hour troponins – or the EDACS pathway – which is its own unique scoring system, ECG, and 0- and 2-hour troponins.  The ADAPT pathway classified 30.8% of these patients as “low risk”, while the EDACS classified 41.6% as such.  Despite this, their primary outcome – patients discharged from the ED within 6 hours – non-significantly favored the ADAPT group, 34.4% vs 32.3%.

To me, this represents a few things.

We are still have an irrational, cultural fear of chest pain.  Only 11.6% of their total cohort had STEMI or NSTEMI, and another 5.7% received a diagnosis of “unstable angina”.  Thus, potentially greater than 50% of patients were still hospitalized unnecessarily.  Furthermore, this cultural fear of chest pain was strong enough to prevent acceptance of the more-aggressive EDACS decision instrument being tested in this study.  A full 15% of low-risk patients by the EDACS instrument failed to be discharged within 6 hours, despite their evaluation being complete following 2-hour troponin testing.

But, even these observations are a digression from the core hypothesis: ADPs are a flawed approach.  Poor outcomes are such the rarity, and so difficult to predict, that our thought process ought be predicated on a foundation that most patients will do well, regardless, and only the highest-risk should stay in the hospital.  Our decision-making should probably be broken down into three steps:

  • Does this patient have STEMI/NSTEMI/true UA?  This is the domain of inquiry into high-sensitivity troponin assays.
  • Does the patient need any provocative testing at all?  I.e., the “No Objective Testing Rule”.
  • Finally, are there “red flag” clinical features that preclude outpatient provocative testing?  The handful of patients with concerning EKG changes, crescendo symptoms, or other high-risk factors fall into this category.

If we are doing chest pain close to correctly, the numbers from this article would be flipped – rather than ~30% being discharged, we ought to be ~70%.

“Effectiveness of EDACS Versus ADAPT Accelerated Diagnostic Pathways for Chest Pain: A Pragmatic Randomized Controlled Trial Embedded Within Practice”

Putting Children to the Flame

Many readers here are students, trainees, or otherwise academic-affiliated, and have limited exposure to the world of community practice.  In these settings, frequently, our pediatric exposure is supervised by clinician-educator sub-specialists in Pediatric Emergency Medicine.  We see the very best evidence translated into acute care of children in the Emergency Department.

The real world is a little different.

These two articles describe the shortcomings of advanced imaging practice in community pediatric settings – in the diagnosis of appendicitis, and in the evaluation of closed head injury.

In the appendicitis article, the authors compare two settings both staffed by PEM physicians – an academic medical center with in-house pediatric surgical coverage, and a community center with consultation available only by phone.  Each site had similar rates of appendicitis diagnoses – 4.7% vs. 4.0% at the academic and community site, respectively.  The academic site, however, evaluated fewer patients with abdominal pain with blood work, and then fewer still of those went on to advanced imaging.  Then, of those receiving advanced imaging, the rates were 10.8% CT at the academic center vs. 28.1% CT at the community center.  Ultrasound however, was employed in 16.6% of cases at the academic center versus 6.5% at the community practice.  Nearly all this difference, however, seemed to be made up of patients admitted to the hospital without any operative intervention.  The obvious reality, then:  radiation in lieu of observation.

The second article here describes the neuroimaging (CT or MRI) of patients evaluated following trauma, along with their ultimate disposition.  Of 2,679 patients reviewed, there were 94 patients with important non-surgical, trauma-related diagnoses, and an additional 16 patients who required neurosurgical intervention.  These authors, however, based on GCS estimates recorded and the distribution of outcomes in the PECARN study, estimate the prevalence of entry criteria into appropriate scanning would have obviated >2000 of these scans.  While I believe they are probably mis-applying the evidence and overstating the inappropriateness of CT, the rarity of serious diagnoses suggests at least a majority of these CTs probably could have been avoided.

In short, we’re still doing too many CTs on children.  Some of the contributing issues are systems based, and some are related to practice re-education.  More ultrasound and more observation, please – and less nuking of children.

“Imaging for Suspected Appendicitis: Variation Between Academic and Private Practice Models”
https://www.ncbi.nlm.nih.gov/pubmed/27050738

“Neuroimaging Rates for Closed Head Trauma in a Community Hospital”

Informatics Trek III: The Search For Sepsis

Big data!  It’s all the rage with tweens these days.  Hoverboards, Yik Yak, and predictive analytics are all kids talk about now.

This “big data” application, more specifically, involves the use of an institutional database to derive predictors for mortality in sepsis.  Many decision instruments for various sepsis syndromes already exist – CART, MEDS, mREMS, CURB-65, to name a few – but all suffer from the same flaw: how reliable can a rule with just a handful of predictors be when applied to the complex heterogeneity of humanity?

Machine-learning applications of predictive analytics attempt to create, essentially, Decision Instruments 2.0.  Rather than using linear statistical methods to simply weight a small handful of different predictors, most of these applications utilize the entire data set and some form of clustering.  Most generally, these models replace typical variable weighted scoring with, essentially, a weighted neighborhood scheme, in which similarity to other points helps predict outcomes.

Long story short, this study out of Yale utilized 5,278 visits for acute sepsis and a random forest model to create a training set and a validation set.  The random forest model included all available data points from the electronic health record, while other models used up to 20 predictors based on expert input and prior literature.  For their primary outcome of predicting in-hospital death, the AUC for the random forest model was 0.86 (CI 0.82-0.90), while none of the rest of the models exceeded an AUC of 0.76.

This still simply at the technology demonstration phase, and requires further development to become actionable clinical information.  However, I believe models and techniques like this are our next best paradigm in guiding diagnostic and treatment decisions for our heterogenous patient population.  Many challenges yet remain, particularly in the realm of data quality, but I am excited to see more teams engaged in development of similar tools.

“Prediction of In-hospital Mortality in Emergency Department Patients with Sepsis: A Local Big Data Driven, Machine Learning Approach”
http://www.ncbi.nlm.nih.gov/pubmed/26679719

Welcome to Yesterday, Have You Heard of PERC?

I usually like these sorts of articles regarding the yield and utilization of CT pulmonary angiograms.  They’re fun to dissect, useful to marvel at the inefficiency of our usage, and finally to feed my editorial hyperbole.  But, not this time.

This is a retrospective study from the University of Michigan comprising six months of CTPA data from 2013.  These authors reviewed charts on 602 consecutive patients and calculated modified Wells and PERC for each, and describe the appropriateness and yields of various cohorts.

Rather than detail these statistics and outcomes – other than to note their overall yield of 61 positives reported out of 602 scans – I’d rather just focus on the 108 patients scanned who were PERC negative.  PERC has been around since 2004, and it’s been percolating into various guidelines and evidence-based algorithms since.  Hello, it’s 2015: why are almost 20% of CTs at an academic medical center PERC-negative?

The authors state two PERC-negative patients had positive CT findings; given the pretest probability, I wouldn’t be surprised if one or both were ultimately false-positives.  Come on, man.

“CT Pulmonary Angiography: Using Decision Rules in the Emergency Department”
http://www.ncbi.nlm.nih.gov/pubmed/26435116

Clinicians or Statistics for Pediatric Abdominal Injury

PECARN is a wonderful thing.  Any individual pediatric facility sees a handful of children.  A handful, however, multiplied by 20, becomes potentially practice-changing.

And, this is an article further exploring the PECARN pediatric abdominal trauma prediction instrument, comparing its utility to typical clinician judgment.  As part of the initial derivation study, the surveyors asked each clinician to rate the likelihood of intra-abdominal injury, stratified <1%, 1-5%, 6-10%, 11-50%, or >50%.  Turns out, clinician judgement wasn’t too bad.

  • Of 9,252 children with <1% chance of injury requiring intervention, 35 (0.4%) had injuries identified.
  • Of 1,793 between 1-5% chance, 40 (2.2%).
  • Of 506 between 6-10% chance, 33 (6.5%).
  • Of 281 between 11-50% chance, 59 (21.0%).
  • Of 81 greater 50% chance, 36 (41.4%).

The problem with these data?  5,318 CTs were performed to identify 203 significant injuries, including 3,016 in those with <1% chance.

The prediction rule was both better and worse.  It was more sensitive than clinician judgment, but also less specific.  For an endeavor attempting to decrease CT utilization in children, it’s still not quite clear where this fits in – and whether using it in a fashion similar to PERC or D-dimer wouldn’t necessarily increase imaging.  It may, as these authors discuss, have more value in Emergency Departments without the same level of comfort managing traumatically injured children, as it may yet in face reduce imaging in that context.

“Comparison of Clinician Suspicion Versus a Clinical Prediction Rule in Identifying Children at Risk for Intra-abdominal Injuries After Blunt Torso Trauma”

More Discharges With HEART

Although, the observed improvements are probably more a result of their preposterously high initial admit rate.

The HEART score, already evangelized in multiple venues, is a tool for risk-stratifying chest pain patients in the Emergency Department.  Its advantage over other, competing scores such as GRACE and TIMI, is its specific derivation intended for use in the Emergency Department.  This trial, of note, is one of the first to do more than just observationally report on its effectiveness.  These authors randomized patients to the “HEART Pathway” or “usual care”.  The HEART Pathway was a local decision aid, combining the HEART score and 0- and 3-hour troponin measurements.  Patients with low-risk HEART scores (0 to 3) were further recommended to treating clinicians for discharge from the Emergency Department without additional testing.  The primary outcome was rate of objective cardiac testing, along with other secondary outcomes related to resource utilization.  Patients were also followed for 30-day MACE, with typical endpoints for cardiovascular follow-up.

With 141 patients each arm, the cohorts were generally well-balanced – specifically with regard to TIMI score >1 and accepted cardiovascular comorbidities.  Stunningly, 78% of the usual care cohort was hospitalized at the index visit.  Thus, the mere 60% hospitalized in the HEART pathway represented a massive improvement – and, such difference likely played a role in the 57% vs. 68% reduction in objective cardiac testing within 30 days.  17 patients suffered MACE, all at the index visit – and, even though the trial was not powered for safety outcomes, none occurred in the “low risk” patients of the HEART cohort.

The authors go on to state strict adherence to the HEART pathway could have eked out an additional 6% reduction in hospitalization.  Certainly, in a nearly 80% admit rate environment, scaling back to a 54% rate is an important reduction.  But, considering only 6% suffered an adjudicated MACE, there remains a vast gulf between the number hospitalized and the number helped.  Some non-MACE patients probably derived some benefit from their extended healthcare encounter as a result of better-tailored medical management, or detection of alternate diagnoses, but clearly, we can do better.

“The HEART Pathway Randomized Trial – Identifying Emergency Department Patients With Acute Chest Pain for Early Discharge”

Might High-Sensitivity Troponin Out-Perform PESI?

Risk-stratification of patients diagnosed with acute pulmonary embolism is generally considered a valuable enterprise.  High-risk patients are reasonable to observe as an inpatient for deterioration leading to thrombolysis or other invasive procedures, while low-risk patients can be obviated the costs and risks of an inpatient stay.  The Pulmonary Embolism Severity Index is in use in many settings to make such a determination – calibrated for maximum sensitivity to detect adverse events.

Cardiac troponin has been similarly used for risk-stratification – but mainly for determining “high-risk” and the spectrum of submassive PE, as many patients with negative conventional troponins still progress to poor outcomes.  This study evaluates the utility of the highly-sensitive troponin – threshold of detection 0.012 ng/mL – for risk-stratification.

Based on retrospective review of 298 consecutive patients with acute PE, these authors found about half had a detectable hsTnI, while the remainder were below the limit of detection.  With regards to “hard events” as a primary outcome – death, CPR, or thrombolysis – no patient with an undetectable troponin had such an event in the hospital.  Conversely, 15 (9%) patients with a detectable hsTnI suffered a serious outcome.  Interestingly, based on a rough evaluation of the Kaplan-Meier survival curves, even patients stratified as intermediate, high, or very high risk by PESI were still event-free if their hsTnI were negative – while a small number of patients low-risk by PESI had events, but only with positive hsTnI.

This is not the most robust evaluation of such risk-stratification, and there’s clearly some oddities in the chart review, given an odd spate of low-risk patients deteriorating between days 10 and 20.  However, it may be the case the hsTnI does as-good or better at risk-stratifying than our current tools – and may be considered for inclusion into future ones.

“The Prognostic Value of Undetectable Highly Sensitive Cardiac Troponin I in Patients With Acute Pulmonary Embolism”
http://www.ncbi.nlm.nih.gov/pubmed/25079900

(Failing to) Identify Severe Sepsis at Triage

This is the holy grail of predictive health informatics in Emergency Medicine – instant identification of serious morbidity, with the theoretical expectation of outcomes improvement due to early intervention.

And, more than almost any condition, accurate early identification of severe sepsis remains elusive.

This is an observational evaluation of the “Australian Triage Scale” in combination with infectious keywords as a tool to identify and manage patients with severe sepsis.  Patients were enrolled at presentation to the Emergency Department, and ultimately followed from triage through their ICU stay – where a clinical diagnosis of severe sepsis was used as the gold standard for outcomes. However, of the 995 patients triaged through the Emergency Department and ultimately diagnosed with severe sepsis, only 534 were identified at triage.  The authors present various diagnostic characteristics for each level of the ATS with regards to acuity, and the AUCs for sensitivity and specificity range from 0.457 to .567 (where 0.5 is basically a coin-flip).  So, the authors’ presented rule-based mechanism is nearly as likely to be incorrect as correct.  I’m not exactly certain how they came to the conclusion “the ATS and its categories is a sensitive and moderately accurate and valid tool”, but I tend to disagree.

These data are consistent with our a priori expectation for these sorts of tools.  The patients who trigger such rules are generally so obviously severe sepsis such rule-based notifications occur after clinician identification, and are simply redundant and alarm fatigue.  Conversely, patients with severe sepsis going undiagnosed upon initial presentation do so because of their atypical nature – and thus tend to fall outside rigid, rule-based constructs.  E.g., computers are not physicians … yet.

“Identification of the severe sepsis patient at triage: a prospective analysis of the Australasian Triage Scale”
http://www.ncbi.nlm.nih.gov/pubmed/25504659