Category: Clinical Decision Rules

D-Dimer, It’s Not Just a Cut-Off

It’s certainly simpler to have a world where everything is black or white, right or wrong, positive or negative. Once upon a time, positive cardiac biomarkers meant acute coronary syndrome – now we have more information and shades of grey in between. The D-dimer, bless its heart, is probably like that, too.

This is a simple study that pooled patients from five pulmonary embolism studies to evaluate the diagnostic performance characteristics of the D-dimer assay. Conventional usage is simply to deploy the test as a dichotomous rule-out – a value below our set sensitivity threshold obviates further testing, while above consigns us to the bitter radiologic conclusion. These authors, perhaps anticipating a more sophisticated diagnostic strategy, go about trying to calculate interval likelihood ratios for the test.

Using over 6,000 patients as their substrate for analysis, these authors determine the various likelihood ratios for D-dimer levels between 250 ng/mL and greater than 5,000 ng/mL, and identify intervals of gradually increasing width, starting at 250 and building up to 2,500. Based on logistic regression modeling, the fitted and approximate iLR range from 0.0625 for those with D-dimer less 250 ng/mL and increasing to 8 for levels greater than 5,000. Interestingly, a D-dimer between 1,000 and 1,499 had an iLR of roughly 1 – meaning those values basically have no effect on the post-test likelihood of PE.

The general implication of these data would be to inform more precise accounting of the risk for PE involving the decision to proceed to CTPA. That said, with our generally inexact tools for otherwise estimating pretest likelihood of disease (Wells, Geneva, gestalt), these data are probably not quite ready for clinical use. I expect further research to develop more sophisticated individual risk prediction models, for which these likelihood ratios may be of value.

“D-Dimer Interval Likelihood Ratios for Pulmonary Embolism”
https://www.ncbi.nlm.nih.gov/pubmed/28370759

The Failing Ottawa Heart

Canada! So many rules! The true north strong and free, indeed.

This latest innovation is the Ottawa Heart Failure Risk Scale – which, if you treat it explicitly as titled, is accurate and clinically interesting. However, it also masquerades as a decision rule – upon which it is of lesser standing.

This is a prospective observational derivation of a risk score for “serious adverse events” in an ED population diagnosed with acute heart failure and potential candidates for discharge. Of these 1,100 patients, 170 (15.5%) suffered an SAE – death, myocardial infarction, hospitalization. They used the differences between the groups with and without SAEs to derive a predictive risk score, the elements of which are:

• History of stroke or TIA (1)
• History of intubation for respiratory distress (2)
• Heart rate on ED arrival ≥110 (2)
• Room are SaO2 <90% on EMS or ED arrival (1)
• ECG with acute ischemic changes (2)
• Urea ≥12 mmol/L (1)

This scoring system ultimately provided a prognostic range from 2.8% for a score of zero, up to 89.0% at the top of the scale. This information is – at least within the bounds of generalizability from their study population – interesting from an informational standpoint. However, they then take it to the next level and use this as a potential decision instrument for admission versus discharge – projecting a score ≥2 would decrease admission rates while still maintaining a similar sensitivity for SAEs.

However, the foundational flaw here is the presumption admission is protective against SAEs – both here in this study and in our usual practice. Without a true, prospective validation, we have no evidence this change in and its potential decrease in admissions improves any of many potential outcome measures. Many of their SAEs may not be preventable, nor would the protections from admission be likely durable out to the end of their 14-day follow-up period. Patients were also managed for up to 12 hours in their Emergency Department before disposition, a difficult prospect for many EDs.

Finally, regardless, the complexity of care management and illness trajectory for heart failure is not a terribly ideal candidate for simplification into a dichotomous rule with just a handful of criteria. There were many univariate differences between the two groups – and that’s simply on the variables they chose to collect The decision to admit a patient for heart failure is not appropriately distilled into a “rule” – but this prognostic information may yet be of some value.

“Prospective and Explicit Clinical Validation of the Ottawa Heart Failure Risk Scale, With and Without Use of Quantitative NT-proBNP”

http://onlinelibrary.wiley.com/doi/10.1111/acem.13141/abstract

Outsourcing the Brain Unnecessarily

Clinical decision instruments are all the rage, especially when incorporated into the electronic health record – why let the fallible clinician’s electrical Jello make life-or-death decisions when the untiring, unbiased digital concierge can be similarly equipped? Think about your next shift, and how frequently you consciously or unconsciously use or cite a decision instrument in your practice – HEART, NEXUS, PERC, Well’s, PECARN, the list is endless.

We spend a great deal of time deriving, validating, and comparing decision instruments – think HEART vs. TIMI vs. GRACE – but, as this article points out, very little time actually examining their performance compared to clinician judgment.

These authors reviewed all publications in Annals of Emergency Medicine concerned with the performance characteristics of a decision instrument. They identified 171 articles to this effect, 131 of which performed a prospective evaluation. Of these, the authors were able to find only 15 which actually bothered to compare the performance of the objective rule with unstructured physician assessment. With a little extra digging, these authors then identified 6 additional studies evaluating physician assessment in other journals relevant to their original 171.

Then, of these 21 articles, two favored the decision instrument: a 2003 assessment of the Canadian C-Spine Rule, and a 2002 neural network for chest pain. In the remainder, the comparison either favored clinician judgment or was a “toss up” in the sense the performance characteristics were similar and the winner depended on a value-weighting of sensitivity or specificity.

This should not discourage the derivation and evaluation of further decision instruments, as yes, the conscious and unconscious biases of human beings are valid concerns.  Neither should it be construed from these data that many common decision instruments are of lesser value than our current usage places in them, only that they have not yet been tested adequately. However, many of these simple models are simply that – and the complexity of many clinical questions will at least favor the more information-rich approach of practicing clinicians.

“Structured Clinical Decision Aids Are Seldom Compared With Subjective Physician Judgment, and are Seldom Superior”
http://www.annemergmed.com/article/S0196-0644(16)31520-7/fulltext

Ottawa, the Land of Rules

I’ve been to Canada, but I’ve never been to Ottawa. I suppose, as the capital of Canada, it makes sense they’d be enamored with rules and rule-making. Regardless, it still seems they have a disproportionate burden of rules, for better or worse.

This latest publication describes the “Ottawa Chest Pain Cardiac Monitoring Rule”, which aims to diminish resource utilization in the setting of chest pain in the Emergency Department. These authors posit the majority of chest pain patients presenting to the ED are placed on cardiac monitoring in the interests of detecting a life-threatening malignant arrhythmia, despite such being a rare occurrence. Furthermore, the literature regarding alert fatigue demonstrates greater than 99% of monitor alarms are erroneous and typically ignored.

Using a 796 patients sample of chest pain patients receiving cardiac monitoring, these authors validate their previously described rule for avoiding cardiac monitoring: chest pain free and normal or non-specific ECG changes. In this sample, 284 patients met these criteria, and none of them suffered an arrhythmia requiring intervention.

While this represents 100% sensitivity for their rule, as a resource utilization intervention, there is obviously room for improvement. Of patients not meeting their rule, only 2.9% of this remainder suffered an arrhythmia – mostly just atrial fibrillation requiring pharmacologic rate or rhythm control. These criteria probably ought be considered just a minimum standard, and there is plenty of room for additional exclusion.

Anecdotally, not only do most of our chest pain patients in my practice not receive monitoring – many receive their entire work-up in the waiting room!

“Prospective validation of a clinical decision rule to identify patients presenting to the emergency department with chest pain who can safely be removed from cardiac monitoring”
http://www.cmaj.ca/content/189/4/E139.full

The Chest Pain Decision Instrument Trial

This is a bit of an odd trial. Ostensibly, this is a trial about the evaluation and disposition of low-risk chest pain presenting to the Emergency Department. The authors frame their discussion section by describing their combination of objective risk-stratification and shared decision-making in terms of reducing admission for observation and testing at the index visit.

But, that’s not technically what this trial was about. Technically, this was a trial about patient comprehension – the primary outcome is actually the number of questions correctly answered by patients on an immediate post-visit survey. The dual nature of their trial is evident in their power calculation, which starts with: “We estimated that 884 patients would provide 99% power to detect a 16% difference in patient knowledge between decision aid and usual care arms”, which is an unusual choice of beta and threshold for effect size – basically one additional question correct on their eight-question survey. The rest of their power calculation, however, makes sense “… and 90% power to detect a 10% difference in the proportion of patients admitted to an observation unit for cardiac testing.” It appears the trial was not conducted to test their primary outcome selected by their patient advocates designing the trial, but in actuality to test the secondary outcomes thought important to the clinicians.

So, it is a little hard to interpret their favorable result with respect to the primary outcome – 3.6 vs 4.2 questions answered correctly. After clinicians spent an extra 1.3 minutes (4.4 vs 3.1) with patients showing them a visual aid specific to their condition, I am not surprised patients had better comprehension of their treatment options – and they probably did not require a multi-center trial to prove this.

Then, the crossover between resource utilization and shared decision-making seems potentially troublesome. An idealized version of shared decision-making allows patients to participate in their treatment when there is substantial individual variation between the perceived value of different risks, benefits, and alternatives. However, I am not certain these patients are being invited to share in a decision between choices of equal value – and the authors seem to express this through their presentation of the results.

These are all patients without known coronary disease, normal EKGs, a negative initial cardiac troponin, and considered by treating clinicians to otherwise fall into a “low risk” population. This is a population matching the cohort of interest from Weinstock’s study of patients hospitalized for observation from the Emergency Department, 7,266 patients of whom none independently suffered a cardiac event while hospitalized.  A trial in British Columbia deferred admission for a cohort of patients in favor of outpatient stress tests.  By placing a fair bit of emphasis on their significant secondary finding of a reduction in observation admission from 52% to 37%, the authors seems to indicate their underlying bias is consistent with the evidence demonstrating the safety of outpatient disposition in this cohort.  In short, it seems to me the authors are not using their decision aid to help patients choose between equally valued clinical pathways, but rather to try and convince more patients to choose to be discharged.

In a sense, it represents offering patients a menu of options where overtreatment is one of them.  If a dyspneic patient meets PERC, we don’t offer them a visual aid where a CTPA is an option – and that shouldn’t be our expectation here, either.  These authors have put in tremendous effort over many years to integrate many important tools, but it feels like the end result is a demonstration of a shared decision-making instrument intended to nudge patients into choosing the disposition we think they ought, but are somehow afraid to outright tell them.

“Shared decision making in patients with low risk chest pain: prospective randomized pragmatic trial”
http://www.bmj.com/content/355/bmj.i6165.short

The Machine Can Learn

A couple weeks ago I covered computerized diagnosis via symptom checkers, noting their imperfect accuracy – and grossly underperforming crowd-sourced physician knowledge. However, one area that continues to progress is the use of machine learning for outcomes prediction.

This paper describes advances in the use of “big data” for prediction of 30-day and 180-day readmissions for heart failure. The authors used an existing data set from the Telemonitoring to Improve Heart Failure Outcomes trial as substrate, and then applied several unsupervised statistical models to the data with varying inputs.

There were 236 variables available in the data set for use in prediction, weighted and cleaned to account for missing data. Compared with the C statistic from logistic regression as their baseline comparator, the winner was pretty clearly Random Forests. With a baseline 30-day readmission rate of 17.1% and 180-day readmission of 48.9%, the C statistic for the logistic regression model predicting 30-day readmission was 0.533 – basically no predictive skill. The Random Forest model, however, achieved a C statistic of 0.628 by training on the 180-day data set.

So, it’s reasonable to suggest there are complex and heterogenous data for which machine learning methods are superior to traditional models. These are, unfortunately, pretty terrible C statistics, and almost certainly of very limited use for informing clinical care. As with most decision-support algorithms, I would be curious also to see a comparison with a hypothetical C statistic for clinician gestalt. However, for some clinical problems with a wide variety of influential factors, these sorts of models will likely become increasingly prevalent.

“Analysis of Machine Learning Techniques for Heart Failure Readmissions”
http://circoutcomes.ahajournals.org/content/early/2016/11/08/CIRCOUTCOMES.116.003039

Don’t CTPA With Your Gut Alone

Many institutions are starting to see roll-out of some sort of clinical decision-support for imaging utilization. Whether it be NEXUS, Canadian Head CT, or Wells for PE, there is plenty of literature documenting improved yield following implementation.

This retrospective evaluation looks at what happens when you don’t obey your new robot overlords – and perform CTPA for pulmonary embolism outside the guideline-recommended pathway. These authors looked specifically at non-compliance at the low end – patients with a Wells score ≤4 and performed with either no D-dimer ordered or a normal D-dimer.

During their 1.5 year review period, there were 2,993 examinations and 589 fell out as non-compliant. Most – 563 – of these were low-risk by Wells and omitted the D-dimer. Yield for these was 4.4% positivity, compared with 11.2% for exams ordered following the guidelines. This is probably even a high-end estimate for yield, because this includes 8 (1.4%) patients who had subsegmental or indeterminate PEs but were ultimately anticoagulated, some of whom were undoubtedly false positives. Additionally, none of the 26 patients that were low-risk with a normal D-dimer were diagnosed with PE.

Now, the Wells criteria are just one tool to help reinforce gestalt for PE, and it is a simple rule that does not incorporate all the various factors with positive and negative likelihood ratios for PE. That said, this study should reinforce that low-risk patients should mostly be given the chance to avoid imaging, and a D-dimer can be used appropriately to rule-out PE in those where PE is a real, but unlikely, consideration.

“Yield of CT Pulmonary angiography in the emergency Department When Providers Override evidence-based clinical Decision support”
https://www.ncbi.nlm.nih.gov/pubmed/27689922

From Way Too Many CTs to Many CTs

I am always keen to hear reports of successful imaging reduction interventions – and, even moreso, in trauma. The typical, modern, approach to trauma involves liberal use of advanced imaging – almost to the point of it being a punch line.

This single-center before-and-after report details their experiences between 2006 and 2013. Before 2010, there was no specific protocol regarding CT in trauma – leading to institutional self-examination in the setting of rampant overuse. After 2010, the following protocol was in effect:

trauma algorithm

There isn’t much besides good news presented here. Their primary imaging use outcome, abdominopelvic CT, decreased from 76.7% to 44.6% of all presentations. This was related to an increase in mean ISS for those undergoing CT. When free fluid from non-traumatic causes was individually accounted for, the rate of positivity of these CT rose from 12.3% to 17.5%. Finally, mortality was unchanged – 3.1% vs. 2.7%.

No doubt, any reduction in imaging will miss some important findings. The net counterbalancing effect, however, is likely a massive reduction in costs and harms from further evaluation of false-positives, renal contrast injury, and radiation. And, after all, they’re still performing CTs on nearly half their patients!

“Effect of an Institutional Triaging Algorithm on the Use of Multidetector CT for Patients with Blunt Abdominopelvic Trauma over an 8-year Period”

http://pubs.rsna.org/doi/abs/10.1148/radiol.2016152021

Shaking Out Stroke Mimics

In a world of continued aggressive guideline- and pharmaceutical-sponsored expansion of stroke treatment with thrombolytics, this article fills and important need – better codifying the predictors of stroke mimics. While other editorials espouse the need to be fast without being sure, this is frankly irresponsible medicine – and, in resource-constrained environments, unsustainable.

These authors at two academic centers performed a retrospective clinical and imaging review of 784 patients evaluated for potential acute cerebral ischemia. Patients were excluded if they had signs of acute stroke on initial non-contrast imaging, and if they did not subsequently undergo MRI. Based on review of the totality of clinical information for each patient, 41% of this cohort were deemed stroke mimics. The authors scoring system, then derived 6 variables – and 3 or more were present, the chance of stroke mimic being cause of the current presentation was 87.2%. Their criteria:

  • Absence of facial droop
  • Age <50 y/o
  • Absence of atrial fibrillation
  • SBP <150 mm Hg
  • Presence of isolated sensory deficit
  • History of seizure disorder

When the rate of tPA administration to stroke mimics is ~15%, and 30-40% of patients evaluated for stroke are stroke mimics – there is a lot of waste and potential harm occurring here. These authors suggest the use of this score could potentially halve these errant administrations for 94% sensitivity, or cut errant administrations down to 2% with 90% sensitivity. Considering the patients for which stroke/stroke mimic is an ambiguous diagnosis, it is reasonably likely the symptoms are of lesser severity – and in the range for which tPA is of most tenuously “proven” value. While their rule has not been prospectively validated, some of these elements certainly have face validity, and can be incorporated into current practice at least as a reminder.

“FABS: An Intuitive Tool for Screening of Stroke Mimics in the Emergency Department”

http://stroke.ahajournals.org/content/early/2016/08/04/STROKEAHA.116.013842.abstract

The Extra Head CTs in Trauma, Estimated

In the world of academia and residency training, the spirited debate in trauma is usually regarding the merits of the “pan-scan” – and whether we can all agree it is probably safe to reduce costs and resource utilization by selective scanning. In community practice, it’s about picking up the needle in a haystack – and, hence, preventing the innumerable unnecessary CTs.

This is a retrospective review using electronic health record data to estimate the number of potentially unnecessary head CTs in the setting of trauma. These authors pulled records for all patients for whom a head CT was obtained, and for whom recorded EHR values suggested an encounter for trauma. This cohort was then evaluated for appropriateness of a CT by retrospectively determining the presence of high-risk or exclusion criteria for the Canadian CT Head Rule.

Among 27,240 patients extracted, 11,432 (42.0%) were “discordant” with the CCHR by structured EHR content. However, upon manual review of the chart narrative, the structured EHR content misclassified the CCHR recommendation 12.2% (95% CI 5.6-18.8%) of the time. Thus, the authors then estimate approximately 36.8% (95% CI 34.1-39.6%) of CT head for trauma in a community setting is inappropriate.

This is probably a reasonable research strategy, warts and all. Due to EHR limitations, they actually only filtered for 3 of the 5 high-risk criteria – basilar skull fracture and open skull fracture are such rare findings in their cohort the impact on overall results would be negligible. Then, Kaiser is probably more aggressive at minimizing CT use than the general community ED population, as routine quality improvement monitors individual and group rates of CT usage.

Bottom line: at least a third of head CTs for trauma in the community can probably be obviated by use of validated criteria.

“Computed Tomography Use for Adults with Head Injury: Describing Likely Avoidable ED Imaging based on the Canadian CT Head Rule”

http://www.ncbi.nlm.nih.gov/pubmed/27473552