A qSOFA Trifecta

There’s a new sepsis in town – although, by “new” it’s not very anymore. We’re supposedly all-in on Sepsis-3, which in theory is superior to the old sepsis.

One of the most prominent and controversial aspects of the sepsis reimagining is the discarding of the flawed Systemic Inflammatory Response Syndrome criteria and its replacement with the Quick Sequential Organ Failure Assessment. In theory, qSOFA replaces the non-specific items from SIRS with physiologic variables more closely related to organ failure. However, qSOFA was never prospectively validated or compared prior to its introduction.

These three articles give us a little more insight – and, as many have voiced concern already, it appears we’ve just replaced one flawed agent with another.

The first article, from JAMA, describes the performance of qSOFA against SIRS and a 2-point increase in the full SOFA score in an ICU population. This retrospective analysis of 184,875 patients across 15 years of registry data from 182 ICUs in Australia and New Zealand showed very little difference between SIRS and qSOFA with regard to predicting in-hospital mortality. Both screening tools were also far inferior to the full SOFA score – although, in practical terms, the differences in adjusted AUC were only between ~0.69 for SIRS and qSOFA and 0.76 for SOFA. As prognostic tools, then, none of these are fantastic – and, unfortunately, qSOFA did not seem to offer any value over SIRS.

The second article, also from JAMA, is some of the first prospective data regarding qSOFA in the Emergency Department. This sample is 879 patients with suspected infection, followed for in-hospital mortality or ICU admission. The big news from this article is the AUC for qSOFA of 0.80 compared with the 0.65 for SIRS or “severe sepsis”, as defined by SIRS plus a lactate greater than 2mmol/L. However, at a cut-off of 2 or more for qSOFA, the advertised cut-off for “high risk”, the sensitivity and specificity were 70% and 79% respectively.

Finally, a third article, from Annals of Emergency Medicine, also evaluates the performance characteristics of qSOFA in an Emergency Department population. This retrospective evaluation describes the performance of qSOFA at predicting admission and mortality, but differs from the JAMA article by applying qSOFA to a cross-section of mostly high-acuity visits, both with and without suspected infection. Based on a sample of 22,350 ED visits, they found similar sensitivity and specificity of a qSOFA score of 2 or greater for predicting mortality, 71% and 74%, respectively. Performance was not meaningfully different between those with and without infection.

It seems pretty clear, then, this score doesn’t hold a lot of value. SIRS, obviously, has its well-documented flaws. qSOFA seems to have better discriminatory value with regards to the AUC, but its performance at the cut-off level of 2 puts it right in a no-man’s land of clinical utility. It is not sensitive enough to rely upon to capture all patients at high-risk for deterioration – but, then, its specificity is also poor enough using it to screen the general ED population will still result in a flood of false positives.

So, unfortunately, these criteria are probably a failed paradigm perpetuating all the same administrative headaches as the previous approach to sepsis – better than SIRS, but still not good enough. We should be pursuing more robust decision-support built-in to the EHR, not attempting to reinvent overly-simplified instruments without usable discriminatory value.

“Prognostic Accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection Admitted to the Intensive Care Unit”

http://jamanetwork.com/journals/jama/article-abstract/2598267

“Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department”

http://jamanetwork.com/journals/jama/fullarticle/2598268

“Quick SOFA Scores Predict Mortality in Adult Emergency Department Patients With and Without Suspected Infection”

http://www.annemergmed.com/article/S0196-0644(16)31219-7/fulltext

Shenfu!

I will readily admit I am stepping outside the bounds of my expertise with this post – with respect to the “shenfu injection” and its effects on physiology. The authors describe shenfu as “originated from Shenfu decoction, a well-known traditional Chinese formulation restoring ‘Yang’ from collapse, tonifying ‘Qi’ for relieving desertion”. More specifically, from a physiologic standpoint: “Ginsenosides and aconite alkaloids are the main active ingredients in Shenfu. Ginsenosides are the determinant contributor to the vasodilator benefit of Shenfu, whereas the alkaloids play a vital role in the cardiac electrophysiological effect of Shenfu by blocking ion channels”. In China, a pharmacologic shenfu distillate is used routinely to treat sepsis and septic shock as a 100mL daily injection – and this is a placebo-controlled trial endeavoring to demonstrate its efficacy.

At face value, the trial appears reasonable – a targeted enrollment of 160 patients with a goal of detecting a 20% difference in mortality at 28-days, based on an expected overall mortality of 40%. Their primary outcome, however, were the co-primary outcomes of “length of ICU stay, the duration of vasopressor use, illness severity, and the degree of organ dysfunction.” A proper study, of course, has a single primary outcome – and, considering the study was powered for a mortality difference, this patient-oriented outcome probably ought to have been made primary.

Regardless, from the results presented here, it is reasonable to suggest this is promising and worthy of additional evaluation. Several outcomes – ICU LOS, APACHE II score, and duration of vasopressor us – reached statistical significance favoring the intervention. The mortality outcome did not meet statistical significance with the intervention at 20.5% and the placebo at 27.8%. However, an absolute mortality improvement of 7.3% is nothing to sneeze at – and I would be happy to see more work performed to replicate or generalize these results.

“Shenfu injection for improving cellular immunity and clinical outcome in patients with sepsis or septic shock”

https://www.ncbi.nlm.nih.gov/pubmed/28029485

The Magic Bacterial Divining Rod

Antibiotic overuse is a real issue.  In modern countries, despite obsessing over antibiotic stewardship, we are still suckers for the excessive use of both narrow-spectrum antibiotics for ambulatory patients and broad-spectrum antibiotics for the critically ill.  In less resource-capable areas, the tests used to stratify patients as potentially bacterial or viral exceed the cost of the antibiotics – also leading down the path to overuse.

This breathless coverage, featured in Time, the AFP, and proudly advertised by Stanford Medicine, profiles a new panel of tests that is destined to bring clarity.  Rather than relying simply on a single biomarker, “our test can detect an infection anywhere in the body by ‘reading the immune system’”.

They used retrospective genetic expression cohorts from children and adults with supposedly confirmed non-infectious or infectious etiologies to derive and validate a scoring system to differentiate the underlying cause of sepsis.  They then further trim their model by eliminating infants and predominately healthy patients from outpatient cohorts.  Ultimately, they then test their model on a previously uncharacterized whole blood sample from 96 pediatric sepsis patients and report an AUC for viral vs. bacterial sepsis of 0.84, with a -LR of 0.15 and +LR of 3.0 for bacterial infections.  At face value, translated to a presumed clinical setting with a generally low prevalence of bacterial infection complicating SIRS, this is an uninspiring result.

However, these authors rather focus their discussion and press releases around the -LR of 0.10 and +LR of 2.34 produced as part of their ideal validation cohort, trumpeting its superiority over the -LR for procalcitonin of 0.29 as “three-fold improvement”.  This is, of course, nonsense, as the AUC from that same procalcitonin meta-analysis was 0.85, and these authors are simply cherry-picking one threshold and performance characteristic for their comparison.

Now, that’s hardly to say this is not novel work, and their confusion matrices showing clustering of non-infected SIRS vs. bacterial sepsis vs. viral sepsis are quite lovely.  Their approach is interesting, and very well could ultimately outperform existing strategies.  However, their current performance clearly does not match the hype, and they are miles away from a meaningful validation.  Furthermore, the sort of nano-array assay required is neither fast enough to be clinically useful nor likely to be produced cheaply enough to be used in some of the resource-poor settings they claim to be addressing.

It makes for a nice headline, but it’s better consigned to the “Fantasy/Science Fiction” shelf of your local bookstore for now.

“Robust classification of bacterial and viral infections via integrated host gene expression diagnostics”
http://stm.sciencemag.org/content/8/346/346ra91

Severe Sepsis … or ß-Agonist

As our sepsis overlords entrenched new “quality measures” and other protocol-driven resuscitation requirements in our Emergency Departments, this article serves as a lovely reminder of the importance of staying cognitively engaged.

Lactate levels can be elevated by metabolic and microcirculatory derangements related to the spectrum of sepsis – but also other, non-infectious causes.  These include hepatic disease, multiple toxodromes, and multiple medications – one of the most commonly used being beta-agonist therapy for obstructive airways.  This very simple study examines the physiologic changes in healthy volunteers receiving 10mg of nebulized albuterol, as compared with nebulized saline.  Placebo volunteers had no change in lactate or placebo.  Albuterol receiving volunteers had an average increase in lactate of 0.77 mmol/L and an average decrease in potassium of 0.5 mEq/L.  Lactate increases, however, were highly variable – ranging from 0.04 to 2.02 mmol/L.

These data aren’t perfectly generalizable to the critically or pseudo-critically ill, but they’re a reasonable starting point for a gross estimate.  They’re also justification for reconsideration of potentially inappropriate therapies for an intermediate-range lactate that obstinately refuses to clear – in the context of receiving multiple rounds of nebulizers.

At the very least, it’s a reminder of the various exceptions to our protocols we need to consider to prevent costly and avoidable harms.

“The Effect of Nebulized Albuterol on Serum Lactate and Potassium in Healthy Subjects”
https://www.ncbi.nlm.nih.gov/pubmed/26857949

Tying Procalcitonin to Critical Care

It has been hard, over the years, to truly identify a role for procalcitonin.  Generally speaking, its best niche seems to be as a sort of C-reactive protein on steroids – a non-specific infectious or inflammatory marker with better sensitivity than WBC.  This has led to some usage in zero-miss contexts such as neonatal sepsis, as well as a potential role in antibiotic stewardship.

These authors, many of which are supported by the manufacturers of the procalcitonin assay, evaluate its predictive power in the setting of pneumonia hospitalization, attempting to risk-stratify patients for the combined endpoint of vasopressor support or invasive ventilation.  Their goal, they say, is to use procalcitonin levels to better inform level-of-care decisions – both escalated and de-escalated – at the time of hospital admission.

They analyzed 1,770 patients from a prior pneumonia study for whom banked serum samples were adequate for procalcitonin measurement, 115 of whom met their combined critical illness endpoint.  They report risk of critical illness increased approximately linearly with procalcitonin from 4% when procalcitonin was undetectable, to 22.4% when procalcitonin was 10ng/mL or above.  The AUC for procalcitonin alone was 0.69, as compared to WBC at 0.54.  Then, they further go on to add usage of procalcitonin in conjunction with other risk-stratification scores – ATS minor criteria, PSI, and SMART-COP – provided additional discriminatory information.

This could be a potentially useful and interesting application of procalcitonin – except they don’t really make any comparisons to other available tools, other than a straw man comparison with WBC.  Would the venerable CRP have a similar AUC?  Or, better yet, a lab we already use nearly ubiquitously to detect occult severe sepsis – a lactic acid level?  The authors do not present any specific discussion of alternative approaches – of which their friends at BioMerieux probably appreciate.

“Procalcitonin as an Early Marker of the Need for Invasive Respiratory or Vasopressor Support in Adults with Community-Acquired Pneumonia”
https://www.ncbi.nlm.nih.gov/pubmed/27107491

Informatics Trek III: The Search For Sepsis

Big data!  It’s all the rage with tweens these days.  Hoverboards, Yik Yak, and predictive analytics are all kids talk about now.

This “big data” application, more specifically, involves the use of an institutional database to derive predictors for mortality in sepsis.  Many decision instruments for various sepsis syndromes already exist – CART, MEDS, mREMS, CURB-65, to name a few – but all suffer from the same flaw: how reliable can a rule with just a handful of predictors be when applied to the complex heterogeneity of humanity?

Machine-learning applications of predictive analytics attempt to create, essentially, Decision Instruments 2.0.  Rather than using linear statistical methods to simply weight a small handful of different predictors, most of these applications utilize the entire data set and some form of clustering.  Most generally, these models replace typical variable weighted scoring with, essentially, a weighted neighborhood scheme, in which similarity to other points helps predict outcomes.

Long story short, this study out of Yale utilized 5,278 visits for acute sepsis and a random forest model to create a training set and a validation set.  The random forest model included all available data points from the electronic health record, while other models used up to 20 predictors based on expert input and prior literature.  For their primary outcome of predicting in-hospital death, the AUC for the random forest model was 0.86 (CI 0.82-0.90), while none of the rest of the models exceeded an AUC of 0.76.

This still simply at the technology demonstration phase, and requires further development to become actionable clinical information.  However, I believe models and techniques like this are our next best paradigm in guiding diagnostic and treatment decisions for our heterogenous patient population.  Many challenges yet remain, particularly in the realm of data quality, but I am excited to see more teams engaged in development of similar tools.

“Prediction of In-hospital Mortality in Emergency Department Patients with Sepsis: A Local Big Data Driven, Machine Learning Approach”
http://www.ncbi.nlm.nih.gov/pubmed/26679719

The Utility of Urinalysis in Young Infants

When faced with the diagnostic evaluation of the young, febrile infant fewer than three months of age, the definitive tool for sepsis from urinary tract infection has traditionally been urine culture.  This stems from uncertainty over the adequacy of urinalysis sensitivity for serious bacterial infection, i.e., those truly bacteremic from a urinary source.

This is an analysis of a multicenter database of infants with bacteremia and urinary tract infection, as measured by isolation of the same pathologic organism from both blood and urine.  The key numbers:

  • Trace or greater leukocyte esterase: 97.6% (94.5-99.2) sensitive and 93.9% (87.9-97.5) specific.
  • Pyuria, >3 WBC/HPF: 96% (92.5-98.1) sensitive and 91.3% (84.6-95.6) specific.
  • Pyuria or any LE: 99.5% (98.5-100) sensitive and 87.8% (80.4-93.2) specific.

These are pretty impressive statistics, and differ significantly from the prior supposed sensitivity of the UA in young infants.  These authors postulate the problem with prior study has been its over-reliance on urine culture, and the resulting false positives.  If this seems a reasonable interpretation of the evidence, it has substantial ramifications for the diagnostic evaluation of young infants.  Importantly, it has the potential for obviating invasive procedures and unnecessary over-treatment.

I would like to see independent confirmation of these authors’ findings, but, considering this study required 15 years to produce the 276 patients analyzed in this paper, this may be the best evidence we see for awhile.

“Diagnostic Accuracy of the Urinalysis for Urinary Tract Infection in Infants, 3 Months of Age”
http://www.ncbi.nlm.nih.gov/pubmed/26009628

A Window Into Your EHR Sepsis Alert

Hospitals are generally interested in detecting and treating sepsis.  As a result of multiple quality measures, however, now they are deeply in love with detecting and treating sepsis.  And this means: yet another alert in your electronic health record.

One of these alerts, created by the Cerner Corporation, is described in a recent publication in the American Journal of Medical Quality.  Their cloud-based system analyzes patient data in real-time as it enters the EHR and matches the data against the SIRS criteria.  Based on 6200 hospitalizations retrospectively reviewed, the alert fired for 817 (13%) of patients.  Of these, 622 (76%) were either superfluous or erroneous, with the alert occurring either after the clinician had ordered antibiotics or in patients for whom no infection was suspected or treated.  Of the remaining alerts occurring prior to action to treat or diagnose infection, most (89%) occurred in the Emergency Department, and a substantial number (34%) were erroneous.

Therefore, based on the authors’ presented data, 126 of 817 (15%) of SIRS alerts provided accurate, potentially valuable information.  Unfortunately, another 80 patients in the hospitalized cohort received discharge diagnoses of sepsis despite never triggering the tool – meaning false negatives approach nearly 2/3rds the number of potentially useful true positives.  And, finally, these data only describe patients requiring hospitalization – i.e., not including those discharged from the Emergency Department.  We can only speculate regarding the number of alerts triggered on the diverse ED population not requiring hospitalization – every asthmatic, minor trauma, pancreatitis, etc.

The lead author proudly concludes their tool is “an effective approach toward early recognition of sepsis in a hospital setting.”  Of course, the author, employed by Cerner, also declares he has no potential conflicts of interest regarding the publication in question.

So, if the definition of “effective” is lower than probably 10% utility, that is the performance you’re looking it with these SIRS-based tools.  Considering, on one hand, the alert fatigue, and on the other hand, the number of additional interventions and unnecessary tests these sorts of alerts bludgeon physicians into – such unsophisticated SIRS alerts are almost certainly more harm than good.

“Clinical Decision Support for Early Recognition of Sepsis”
http://www.ncbi.nlm.nih.gov/pubmed/25385815

SIRS – Insensitive, Non-Specific

In what is almost certainly news only to quality improvement administrators, this newly published work out of Australia and New Zealand confirms what most already knew: the Systemic Inflammatory Response Syndrome criteria are only modestly associated with severe sepsis.

This is a retrospective evaluation of 13 years of data from the Australia and New Zealand Intensive Care Society Adult Patient Database, comprising routinely collected quality-assurance data.  Of 1,171,797 patients admitted to adult ICUs, 109,663 were identified as having both an infection and organ failure – the general, clinical definition of severe sepsis.  First, the good news:  over the 13 year study period, mortality dropped substantially – from over 30% down to close to 15%.  Then, the bad news:  12.1% of patients in the severe sepsis cohort manifested 0 or 1 SIRS criteria.  Mortality was lower in SIRS-negative severe sepsis, but hardly trivial at 16.1% during the study period, compared with 24.5% in the SIRS-positive patients.

So, the traditional SIRS-criteria definition of severe sepsis, previously thought to have at least sensitivity at expense of specificity will miss 1 in 8 patients with organ failure and an underlying infection.  Considering only approximately 1/3rd of patients with two or more SIRS criteria in the Emergency Department have an underlying infection, the utility of these criteria is substantially less reliable than previously thought.  Sadly, I’m certain many of you are suffering under SIRS criteria-based alerts in your Electronic Health Record – and, if such alerts are introducing cognitive biases by decreased vigilance and alert fatigue, it ought to be obvious we’re simply harming ourselves and patients.

“Systemic Inflammatory Response Syndrome Criteria in Defining Severe Sepsis”
http://www.nejm.org/doi/full/10.1056/NEJMoa1415236

Early Goal-Directed Waste For Sepsis

First there was ProCESS.  Then there was ARISE.  Now there is ProMISe.

If the prior two trials hadn’t already been celebrated and dissected, there would be much more to write regarding this one.  This, like the others, randomized patients to Early Goal-Directed Therapy for severe sepsis versus “usual care”.  This, like the others, found the basic components of resuscitation – intravenous fluids and early antibiotics – are far more important than the specific targets and protocols enshrined by Rivers et al.

These authors screened 6,192 patients to randomize 1,260.  Half had refractory hypotension, and the mean lactate levels were 7.0 and 6.8 in the EGDT and usual care arms.  Patients were enrolled within 6 hours of presentation and randomized within 2 hours of meeting inclusion criteria, with the EGDT arm receiving catheter insertion capable of SCVO2 monitoring within ~1 hour.   EGDT protocol was adhered to for 6 hours following enrollment.

As expected, randomization produced some divergence in treatment due to the EGDT protocol.  The EGDT cohort received more frequent red cell transfusions during both the protocolized period and subsequent care.  Likewise, dobutamine use in the EGDT arm exceeded usual care.  However, some differences occurred outside of the protocol.  EGDT arm patients were more likely to be admitted to an ICU setting, more likely to receive any sort of central line, more likely to receive invasive blood pressure monitoring, and more likely to be placed on vasopressors.  The remaining treatment – crystalloid resuscitation, colloid resuscitation, and other transfusions were similar.

And, finally, 90-day mortality was similar: 29.5% EGDT vs. 29.2% usual care.

A financial analysis found EGDT was more costly, but the result did not reach statistical significance.  However, the cost analysis was performed using different financial models that may not be generalizable to the billing structure in the United States.  The difference in ICU admission and length-of-stay alone certainly has important ramification both from a cost and a resource utilization standpoint.

So, finally, we have the publication of the last of the triumvirate of EGDT trials.  If there were any lingering doubts (hopes?) regarding the necessity of the most resource-intensive interventions, they ought to be laid to rest.  However, as with each of these negative trials, it is important to acknowledge the role of Rivers’ work in aggressively seeking, recognizing, and treating severe sepsis.  Even as we discard the components of his protocol, the main thrust of his work has saved many, many lives.

“Trial of Early, Goal-Directed Resuscitation for Septic Shock”
http://www.nejm.org/doi/full/10.1056/NEJMoa1500896