Category: Clinical Decision Rules

ADAPTing & Improving

By far, the most promising of the publications to yet emerge from the ADAPT cohort – 1,974 patients evaluated for acute chest pain in the Emergency Department – is this re-analysis and decision instrument.

The original ADAPT publication, despite over 80% of the patients having no major cardiac event at 30 days, was only able to identify 20% of patients as “low-risk”.  The HEART Score and the Vancouver Chest Pain Rule improve on this, but only incrementally.  This publication, however, improves the identification of a low-risk cohort to nearly 50%.  By incorporating and weighting 37 different predictor variables, then adding a layer of expert review and acceptability evaluation, these authors ultimately arrive at the “Emergency Department Assessment of Chest Pain Score (EDACS)”.  Using age, gender, history, and symptoms, when combined with negative ECG and 0 and 2 hour troponins, a score of 16 constitutes a breakpoint for a decision instrument with ~99% sensitivity and ~55% specificity for MACE at 30 days.

As far as decision instruments go, it’s relatively reasonable – although, certainly, nothing you’d be able to keep in your head.  Scores for age range from +2 to +20, while four different symptoms and signs have varying positive and negative values.  However, in the age of computerized decision-support, at least, mildly complex rules are not as burdensome as they once were.

I would like to see, at least, prospective validation in a North American population – but this appears to be a lovely step forward.

“Development and validation of the Emergency Department Assessment of Chest pain Score and 2 h accelerated diagnostic protocol“
http://onlinelibrary.wiley.com/doi/10.1111/1742-6723.12164/abstract

HIAT-2 for Prognosticating Outcomes After Endovascular Interventions in Stroke: Proving Once Again Sick People are Sick

A guest post by Rory Spiegel (@CaptainBasilEM) who blogs on nihilism and the art of doing nothing at emnerd.com.

Since the publication of IMS-3, SYNTHESIS and MR RESCUE in the NEJM earlier this year, proponents of endovascular interventions for acute ischemic stroke have hypothesized why these trials were universally negative. Among the various explanations, one of the most prominent was that the trial designs selected the wrong patients. In order to fully highlight the benefits of endovascular therapy it must be performed on large vessel occlusions with ischemic but viable brain tissue down stream. The authors of a recently published study in STROKE have suggested a solution for this very problem. With the publication of their derivation and validation cohorts of the second generation of the HIAT score, HIAT-2 score, they postulate this decision rule will help predict which patients will benefit from endovascular interventions. The second revision of the HIAT score incorporates the original 3 factors, age, NIHSS and blood glucose at presentation in addition to the ASPECT score of the initial non-contrast CT. As opposed to the initial iteration of HIAT (1 point allotted for each of the 3 factors), HIAT-2 is a 10-point scale (10 being most severe) with age, ASPECT score, NIHSS and blood glucose level relatively weighted in descending order of influence.

The score was retrospectively derived and validated from two prospectively gathered databases of patients who underwent endovascular treatment for acute ischemic stroke. Older patients, with increased NIHSS and more ischemic changes on their initial CT fared far worse than their younger less critically presenting counterparts. In patients with a HIAT-2 scores of 5 or greater, 80% had a mRS of >4 at 90 days. A HIAT-2 score of >7 resulted in 100% of patients having a mRS of >4 at 90 days. The authors conclude that since patients with a high HIAT-2 score had poor 90-day outcomes, endovascular interventions should be limited to patients with a HIAT-2 score of 5 or less.

Never mind that the HIAT-2 score is only moderately capable of identifying those with a poor outcome after endovascular intervention (AOC is only 0.73), the more egregious logical fallacy committed by these authors was to conclude that the HIAT-2 score will differentiate those who will benefit from an endovascular intervention from those who will not. HIAT-2 does nothing of the sort. It is merely a predictor of prognosis at 90 days. Older patients, with more severe strokes at presentation (both clinically and radiologically) will have a worse prognosis no matter what interventions are performed. Interestingly, if the HIAT-2 score is utilized as proposed by its authors, it would exclude the patients who were originally hypothesized to benefit from these endovascular interventions. Patients with symptoms severe enough (usually NIHSS 10 or greater) to be large vessel occlusions are the types of strokes, which lend themselves to endovascular interventions. These are the types of infarcts the HIAT-2 score would exclude from endovascular treatment options.

Finally, in the IMS-3 trial though the HIAT-2 score was not specifically measured, age, NIHSS and the ASPECT score were all recorded. 31% of the cohort was younger than 65, over half the cohort was ASPECT 8,9, or 10 and greater than 80% had a NIHSS at presentation of 19 or less. No benefit of endovascular treatment over tPA was seen in any of the subgroups that are the most heavily weighted components of the HIAT-2 score.

The HIAT-2 score would require prospective validation before it can be used clinically, but I argue that in its current form, HIAT-2 is unable to answer the question for which it was conceived. At present the best evidence we have demonstrates endovascular interventions are no better than tPA (and some would argue tPA is no better than placebo). More high quality trials are needed to identify if there is a subgroup of patients who may benefit from endovascular interventions but pseudoscience and logical fallacies do nothing to augment the knowledge we have gained so far.

“Optimizing Prediction Scores for Poor Outcome After Intra-Arterial Therapy in Anterior Circulation Acute Ischemic Stroke”
www.ncbi.nlm.nih.gov/pubmed/23929748

HIAT-2 for Prognosticating Outcomes After Endovascular Interventions in Stroke: Proving Once Again Sick People are Sick

A guest post by Rory Spiegel (@CaptainBasilEM) who blogs on nihilism and the art of doing nothing at emnerd.com.

Since the publication of IMS-3, SYNTHESIS and MR RESCUE in the NEJM earlier this year, proponents of endovascular interventions for acute ischemic stroke have hypothesized why these trials were universally negative. Among the various explanations, one of the most prominent was that the trial designs selected the wrong patients. In order to fully highlight the benefits of endovascular therapy it must be performed on large vessel occlusions with ischemic but viable brain tissue down stream. The authors of a recently published study in STROKE have suggested a solution for this very problem. With the publication of their derivation and validation cohorts of the second generation of the HIAT score, HIAT-2 score, they postulate this decision rule will help predict which patients will benefit from endovascular interventions. The second revision of the HIAT score incorporates the original 3 factors, age, NIHSS and blood glucose at presentation in addition to the ASPECT score of the initial non-contrast CT. As opposed to the initial iteration of HIAT (1 point allotted for each of the 3 factors), HIAT-2 is a 10-point scale (10 being most severe) with age, ASPECT score, NIHSS and blood glucose level relatively weighted in descending order of influence.

The score was retrospectively derived and validated from two prospectively gathered databases of patients who underwent endovascular treatment for acute ischemic stroke. Older patients, with increased NIHSS and more ischemic changes on their initial CT fared far worse than their younger less critically presenting counterparts. In patients with a HIAT-2 scores of 5 or greater, 80% had a mRS of >4 at 90 days. A HIAT-2 score of >7 resulted in 100% of patients having a mRS of >4 at 90 days. The authors conclude that since patients with a high HIAT-2 score had poor 90-day outcomes, endovascular interventions should be limited to patients with a HIAT-2 score of 5 or less.

Never mind that the HIAT-2 score is only moderately capable of identifying those with a poor outcome after endovascular intervention (AOC is only 0.73), the more egregious logical fallacy committed by these authors was to conclude that the HIAT-2 score will differentiate those who will benefit from an endovascular intervention from those who will not. HIAT-2 does nothing of the sort. It is merely a predictor of prognosis at 90 days. Older patients, with more severe strokes at presentation (both clinically and radiologically) will have a worse prognosis no matter what interventions are performed. Interestingly, if the HIAT-2 score is utilized as proposed by its authors, it would exclude the patients who were originally hypothesized to benefit from these endovascular interventions. Patients with symptoms severe enough (usually NIHSS 10 or greater) to be large vessel occlusions are the types of strokes, which lend themselves to endovascular interventions. These are the types of infarcts the HIAT-2 score would exclude from endovascular treatment options.

Finally, in the IMS-3 trial though the HIAT-2 score was not specifically measured, age, NIHSS and the ASPECT score were all recorded. 31% of the cohort was younger than 65, over half the cohort was ASPECT 8,9, or 10 and greater than 80% had a NIHSS at presentation of 19 or less. No benefit of endovascular treatment over tPA was seen in any of the subgroups that are the most heavily weighted components of the HIAT-2 score.

The HIAT-2 score would require prospective validation before it can be used clinically, but I argue that in its current form, HIAT-2 is unable to answer the question for which it was conceived. At present the best evidence we have demonstrates endovascular interventions are no better than tPA (and some would argue tPA is no better than placebo). More high quality trials are needed to identify if there is a subgroup of patients who may benefit from endovascular interventions but pseudoscience and logical fallacies do nothing to augment the knowledge we have gained so far.

“Optimizing Prediction Scores for Poor Outcome After Intra-Arterial Therapy in Anterior Circulation Acute Ischemic Stroke”
www.ncbi.nlm.nih.gov/pubmed/23929748

Time to Move to the HEART Score

A couple posts ago I mentioned it was time for the TIMI Risk Score for UA/nSTEMI to go the way of the dodo for evaluation of chest pain in the Emergency Department.  It wasn’t derived from an Emergency Department population, doesn’t have great predictive skill in identifying very-low-risk patients, and includes nonsensical elements (did you take an aspirin within the last 7 days?).

Alternatively, we have the HEART score: History, ECG, Age, Risk factors, Troponin.  This was derived – like the Wells score – from the elements of clinical gestalt, and ought to at least make better intuitive sense than the occasionally frizzy outputs from multivariate logistic regression.  It was initially derived and refined retrospectively, and this represents the prospective validation study.  These authors prospectively enrolled 2,440 patients from 10 centers in the Netherlands and followed them for a primary endpoint of a major adverse cardiac event (AMI, PCI, CABG, death) for six weeks.  They also collected the variables of interest necessary to calculate TIMI and GRACE risk scores for comparison of c-statistic.

Obviously, I’m recommending the HEART score because it outperformed the others – the c-statistic for HEART was 0.83, 0.75 for TIMI, and 0.70 for GRACE.  Most importantly, for the Emergency Department, it was superior at the low-end of the spectrum.  For the 34% of the population that was TIMI 0-1, 23/811 (2.8%) had 6-week MACE.  14.0% had GRACE 0-60, and 10/335 (2.9%) had MACE.  For HEART, 36.4% were 0-3 and ultimately 15/870 (1.7%) had MACE.

Even though there are 2,400 patients in this study, there are few enough in each individual category that confidence intervals for each predictive bucket are still relatively wide.  Then, you can still have a HEART score in the “very low risk” 0-3 range with a troponin >3x the normal limit and an abnormal EKG – which is seemingly counterintuitive.  They also don’t compare their rule to clinical judgment, so we can’t measure the performance of the rule in actual decision-making.

A couple other studies have either prospectively or retrospectively validated these findings with reasonable consistency.  It isn’t perfect – but it’s better than TIMI or GRACE – and it’s what I currently use to support my shared decision-making discussions at disposition of the appropriate chest pain cohort.

http://www.heartscore.nl/en/

“A prospective validation of the HEART score for chest pain patients at the emergency department”
http://www.ncbi.nlm.nih.gov/pubmed/23465250

HINTS vs. ABCD2 in Dizziness

Dizziness in the Emergency Department sends everyone down their favorite diagnostic algorithm, with outcomes ranging from utterly benign to impending permanent disability.  I’ve covered the repurposing of the ABCD2 score for risk-stratification in dizziness before, showing it had some utility in predicting posterior circulation stroke.

However, unsurprisingly, these authors demonstrate examination maneuvers specifically targeted at evaluating cerebellar function outperform risk-stratification.  The HINTS (head impulse, nystagmus type, test of skew) evaluation compared with the ABCD2 (age, blood pressure, clinical features, duration, diabetes) in a convenience sample of 190 prospectively collected patients with acute vestibular syndrome.  Of these 190 patients, 124 had a central cause for their vertigo (stroke, hemorrhage, space-occupying lesion).  The sensitivity and specificity of the ABCD2 score in predicting a central lesion was 58.1% and 60.6%, respectively, while the HINTS score resulted in 96.8% and 98.5%, respectively.

It’s a bit of a straw-man comparison – considering the ABCD2 score was never designed to detect posterior circulation stroke, only to affect probability estimates for cerebrovascular disease.  The prevalence of disease in this sample probably also leads to an overestimation of the specificity of the HINTS exam, but it has otherwise been found to have very good test characteristics.

Visit EMCrit for more information and video footage of the HINTS test, if you’re not already using it.

“HINTS Outperforms ABCD2 to Screen for Stroke in Acute Continuous Vertigo and Dizziness”
http://www.ncbi.nlm.nih.gov/pubmed/24127701

Ultrasound First For Pulmonary Embolism?

Akin to the ultrasound first for appendicitis protocol currently in use, the authors of a recently published study in CHEST propose using ultrasound before CTPA in patients where the diagnosis of pulmonary embolism is being considered.

Their protocol consisted of bedside thoracic and lower extremity ultrasounds to identify either a confirmatory DVT or an alternative diagnosis that would account for the patient’s current presentation. In both the ED and inpatient settings,  ICU physicians evaluated this protocol’s performance in 100 patients. The 54 patients who were determined, due to an alternative diagnosis found on ultrasound, not to require further testing, none of them were found to have a pulmonary embolism on confirmatory CT. Of the remaining 40 patients (42%) whose ultrasound revealed no convincing alternative diagnosis or lower extremity DVTs, 12 were found to have pulmonary embolisms on their confirmatory CTPA. The authors conclude that though further studies are needed, an ultrasound first strategy will reduce the number of CTs obtained to rule out pulmonary embolism.

Though I am not opposed to the utilization of ultrasound as a bedside tool, using it to rule out pulmonary embolisms is a flawed paradigm. The proposed protocol is not one which rules out PE, it in fact does just the opposite. This protocol takes advantage of the high specificity of ultrasound for the diagnosis of pneumonia, pulmonary edema, and DVT. It employs the strategy of ruling in an alternative diagnosis or a lower extremity DVT. If no convincing diagnosis is discovered the patient will then move on to the more traditional rule out strategy of CTPA. This study essentially uses bedside ultrasound to address the  two most heavily weighted criteria on the Well’s Score, “an alternative diagnosis that is less likely than pulmonary embolism” and  “signs and symptoms of deep venous thrombosis”. In no way is this protocol fatally flawed. It has the potential to add a great deal to clinical decision making. Unfortunately it does not address the more serious epidemic in the current management of pulmonary embolisms. That is the egregious over-testing and subsequent over-diagnosis of pulmonary embolism in the ultra low risk patient.

“Ultrasound Assessment of Pulmonary Embolism in Patients Receiving Computerized Tomography Pulmonary Angiography”
journal.publications.chestnet.org/article.aspx?articleid=1763837

For more nihilism, emergency medicine and the art of doing nothing see emnerd.com and @CaptainBasilEM

Ultrasound First For Pulmonary Embolism?

Akin to the ultrasound first for appendicitis protocol currently in use, the authors of a recently published study in CHEST propose using ultrasound before CTPA in patients where the diagnosis of pulmonary embolism is being considered.

Their protocol consisted of bedside thoracic and lower extremity ultrasounds to identify either a confirmatory DVT or an alternative diagnosis that would account for the patient’s current presentation. In both the ED and inpatient settings,  ICU physicians evaluated this protocol’s performance in 100 patients. The 54 patients who were determined, due to an alternative diagnosis found on ultrasound, not to require further testing, none of them were found to have a pulmonary embolism on confirmatory CT. Of the remaining 40 patients (42%) whose ultrasound revealed no convincing alternative diagnosis or lower extremity DVTs, 12 were found to have pulmonary embolisms on their confirmatory CTPA. The authors conclude that though further studies are needed, an ultrasound first strategy will reduce the number of CTs obtained to rule out pulmonary embolism.

Though I am not opposed to the utilization of ultrasound as a bedside tool, using it to rule out pulmonary embolisms is a flawed paradigm. The proposed protocol is not one which rules out PE, it in fact does just the opposite. This protocol takes advantage of the high specificity of ultrasound for the diagnosis of pneumonia, pulmonary edema, and DVT. It employs the strategy of ruling in an alternative diagnosis or a lower extremity DVT. If no convincing diagnosis is discovered the patient will then move on to the more traditional rule out strategy of CTPA. This study essentially uses bedside ultrasound to address the  two most heavily weighted criteria on the Well’s Score, “an alternative diagnosis that is less likely than pulmonary embolism” and  “signs and symptoms of deep venous thrombosis”. In no way is this protocol fatally flawed. It has the potential to add a great deal to clinical decision making. Unfortunately it does not address the more serious epidemic in the current management of pulmonary embolisms. That is the egregious over-testing and subsequent over-diagnosis of pulmonary embolism in the ultra low risk patient.

“Ultrasound Assessment of Pulmonary Embolism in Patients Receiving Computerized Tomography Pulmonary Angiography”
journal.publications.chestnet.org/article.aspx?articleid=1763837

For more nihilism, emergency medicine and the art of doing nothing see emnerd.com and @CaptainBasilEM

hsTnI – All Promise, No Proof

At some point, it is true – there are no “bad” tests, only “bad” applications and interpretations of those tests.  One of those tests, as supported by Abbott Laboratories, is the high-sensitivity troponin.  You may also know this test as the “low-specificity” troponin – as, barring small improvements in the assay, a more sensitive test for the same biomarker is bound to result in decreased specificity.

This article describes the populations of ADAPT and APACE for whom high-sensitivity troponins are available.  These trials were part of a prospective derivation of an “accelerated diagnostic protocol,” in which low-risk patients (TIMI 0 or 1) with normal ECGs and two negative hsTnI two hours apart were found to be eligible for discharge from the Emergency Department.  With an approximately 14% of 30-day MACE (mostly nSTEMI) in each cohort, the authors strategy is reasonable:  only ~0.7% of patients meeting these three criteria eventually met a primary endpoint.

Conversation about this article led to this tweet by the primary author:

@EMManchester @240minDoc Precision of hs #troponin assays now shown to have great advantage. ED patients with possible ACS and this assay 1
— Louise Cullen (@louiseacullen) November 7, 2013

… except it isn’t entirely true.  The missing key to this statement is precisely what the “great advantage” entails.  These authors, sponsored by Abbott Laboratories, do not show this diagnostic strategy utilizing hsTnI is in fact superior to the same strategy using conventional troponins.  Quick back-of-napkin math shows the ADAPT conventional troponin cohort using this same strategy gives statistically similar results.  This critique led to the final tweet from the primary author:

@emlitofnote @EMManchester @240minDoc OK have to wait til part II where I can show you larger pop and more narrow CIs.
— Louise Cullen (@louiseacullen) November 8, 2013

Yes, with sufficient statistical power, there will likely be a reproducible difference between the different troponin assays.  When millions of patients are evaluated for chest pain every year, there may be a few for whom this improved sensitivity is clinically significant.  However, it is far more likely this increased sensitivity will end up referring additional patients for testing – resulting in increased costs and harms from overdiagnosis.  This is not the fault of the test – but, rather, simply that we don’t yet know the clinical significance of all small troponin elevations, and there is no appropriate algorithm for managing them in current practice.

I actually like what these authors are doing – using a rapid rule-out plus risk-stratification to safely discharge patients from the Emergency Department.  However, they’re selling hsTnI without proving it’s superior, in this strategy, to conventional troponin testing.  Then, as tests become more sophisticated, our interpretation of them needs to as well – and studies such as these need to do more than simply describe a “minimal-risk” cohort, but also provide useful guidance on the rest of the grey area troponin elevations.

Finally, I’d also like to finally see the TIMI score retired for use in the Emergency Department.  Please. Make it die.

“Validation of High-Sensitivity Troponin I in a 2-Hour Diagnostic Strategy to Assess 30-Day Outcomes in Emergency Department Patients With Possible Acute Coronary Syndrome”

Strep Throat? Stay Home!

NBC News covered this useful-seeming innovation last week – a predictive score to help patients decide whether their sore throat might be caused by Group A Strep.  It seems a quite reasonable proposition on the surface – if patients can receive guidance on their pretest likelihood of disease, they might rather not seek unnecessary medical care.  For the 12 million physician visits every year for sore throat, putting a dent in this would account for sizable cost savings.

This study describes retrospective development of a “Home Score” for use by patients, based on a MinuteClinic database of 71,000 sore throat presentations for which a strep swab was performed.  The authors split the data into a derivation and a validation set, and produced a complex mathematic scoring system, from 1 to 100, based off age, fever, cough, and biosurveillance data.  Using a score of 10 as a cut-off, the validation set sensitivity was 99%, specificity was 1%, and the prevalence data used resulted in a validation negative predictive value of 87%.  This NPV, the authors say, is the important number regarding advising patients whether they ought to seek care for GAS.

There are a few issues with this derivation, of course.  First of all, the derivation population is subject to selection bias – as only patients with strep swabs are included.  Then, the MinuteClinic data has to be generalizable to the remaining adult population.  The use of the Home Score also depends on the availability of biosurveillance data for their specialized algorithm.  Finally, their NPV cut-off of 90% would theoretically obviate clinic visits for only 230,000 of the 12 million patients seeking care for sore throat – a large drop, but only a drop in the bucket, nonetheless.

And, the elephant in the room: Group A Strep doesn’t need antibiotics in the United States.  The likelihood of adverse reactions to treatment of GAS exceeds the chance of benefit – whether progression to peritonsilar abscess or rheumatic fever is considered.  A few folks on Twitter chimed in to echo this sentiment when this story was discussed:

@embasic @DrLeanaWen @MDaware @NBCNewsHealth just need to redesign app to say “no you don’t” regardless of sx
— Anand Swaminathan (@EMSwami) November 10, 2013

There are legitimate reasons to visit a physician for sore throat – but, in the U.S., nearly all uncomplicated pharyngitis can safely stay home, GAS or not.

“Participatory Medicine: A Home Score for Streptococcal Pharyngitis Enabled by Real-Time Biosurveillance”
http://www.ncbi.nlm.nih.gov/pubmed/24189592

A Plea For Strep Throat Simplicity

A recent article published in the BMJ delightfully demonstrates that regardless of how we choose to overcomplicate the treatment of acute pharyngitis, our patients will all do just fine. The authors do have to be commended for their ambition as they attempt to supplant the mighty Centor’s claim as the clinical tool to guide antibiotic use. Not as self-aggrandizing as Centor, they named their rule FeverPAIN, an acronym for its five components. Their ambition is even more noteworthy as they endeavored to describe this rule’s derivation, validation, revision, bootstrap validation and clinical implementation all in one paper.
Despite the authors’ attempts to woo us with promises of bootstrapping and stepwise logistic regression, this study was essentially a pragmatic trial examining the effectiveness of three treatment strategies to guide antibiotic utilization in patients with acute pharyngitis. Patients (3 years or older) were randomized into one of three groups, either delayed antibiotics,  antibiotics as determined by the FeverPAIN score, or antibiotics as determined FeverPAIN score and a positive rapid antigen test.
Overall all three groups did well. The average duration of symptoms was 4 days. Both the clinical score group and clinical score + antigen test group had on average one day fewer symptoms than the delayed antibiotic group, all while receiving 10% fewer courses of antibiotics. Initially this finding seems counterintuitive. If more antibiotics were given in the delayed antibiotic group, how then did their symptoms last longer than those in the clinical score groups? Unless of course antibiotics played no role in this difference.
Its important to keep in mind that although this was a randomized trial, it was not blinded. Patients in both the clinical score group and the clinical score + antigen group had an experience in which the doctors spent time medically evaluating them and in some cases even running a “test”. Whereas in the delayed antibiotic group the patients had a less fulfilling experience, instructed to go home and a prescription for antibiotics would be waiting for them if they did not improve. Seemingly the prior two groups had a far stronger meaningful response than those in the delayed antibiotic group, demonstrating it is not antibiotics but rather talking to your patients and explaining the expected course of the disease that makes a difference on symptom burden. Interestingly the rapid antigen test added very little to reduction in antibiotic use and had no effect of length of symptoms.
This trial suffers from what is known as the Pollyanna Effect. If everyone will do well regardless of the intervention they receive, then it is almost impossible to show a clinically relevant superiority of one treatment strategy over another. More importantly with the incidence of rheumatic fever being so low it is considered clinically irrelevant. The question is not which of these strategies is most effective for the treatment of acute pharyngitis but rather is any treatment necessary at all? 
“Clinical score and rapid antigen detection test to guide antibiotic use for sore throats: randomized controlled trial of PRISM (primary care streptococcal management)”. www.ncbi.nlm.nih.gov/pubmed/24114306

Nihilsm, Emergency Medicine and the art of doing nothing at emnerd.com