Predicting Treatment Failure in AOM

Like most infectious diseases, acute otitis media generally breaks down into three cohorts. There are viral infections, for which early antimicrobial therapy is virtually, by definition, unhelpful. Then, there are true bacterial infections – many of which resolve without substantial morbidity regardless of antimicrobial treatment, and those which require antimicrobial therapy to prevent such. The trick, and where modern medicine typically fails miserably, is rapidly predicting into which of these cohorts a patient may fall – a conundrum leading to the epidemic of antibiotic overuse.

This is a secondary analysis of a pediatric AOM trial, first published in the New England Journal of Medicine, looking at which patients were more likely to potentially fail conservative treatment. The intervention arm received amoxicillin/clavulanate, and treatment failure occured in 31.7% of children – vastly favoring the antibiotic arm – 44.9% vs. 18.6%. In theory, this exaggerated treatment effect might help better illuminate any small predictors – but, unfortunately, with only 319 patients, meaningful statistical significance on this data dredge is hard to come by. Worse still, the best predictor of treatment failure (or, really, lack thereof)? A peaked tympanogram (A and C curves) – you know, because we’re all routinely measuring tympanometry. Grossly bulging tympanic membranes were predictive of treatment failure, which has some face validity, at least – but, again, this is as compared between severe, moderate, and mild, which requires pneumatic otoscopy to differentiate.

The question here primarily concerns: can you take away good conclusions from bad data? The magnitude of the treatment effect seen in this trial far exceeded the treatment effect expected from antibiotics in other trials. And, consistent with that questionable generalizability, their findings reflect the stringent criteria determining their diagnosis of AOM. Then, they are relying upon their misguided definition for treatment failure, which relies on otoscopic signs, the same ones that will be colinear with worsened disease on initial examination. Unfortunately, the net result of all of this meandering is essentially no clinically useful insight. Considering the limitations the examination of the screaming ill toddler, more pragmatic approaches are necessary.

“Prognostic Factors for Treatment Failure in Acute Otitis Media”

http://pediatrics.aappublications.org/content/early/2017/08/04/peds.2017-0072

I&D Alone or With Antibiotics for the Little Guys

Most physicians provide adjunctive antibiotic therapy for large abscesses following incision and drainage – the sorts where you need a bucket and a hose. Less clear has been the small abscess – but, in the age of MRSA, the fear factor has led many to cover these, regardless. Recent evidence suggests there is a small absolute benefit to antibiotic use and clinical cure, with an NNT around 14, along with other apparent benefits regarding re-infection and spread to household contacts. These trials, however, still enrolled patients with abscesses much larger than typically encountered in routine practice.

This trial is specifically designed to break the glass on “smaller skin abscesses” – just like in the title! What does small mean to these authors? It means a suppurative cavity of 5cm in diameter, or, up to the size of a cupcake:
abscess cakeSo, before we even start, we can see we may end up with issues regarding generalizability to many of the abscesses we encounter in the Emergency Department.

This trial is comprised of three arms – clindamycin, trimethoprim-sulfamethoxazole, and placebo – and enrolled 786 patients in an attempt to detect a 10% difference between arms while accounting for 20% attrition rate. The primary outcome was test of cure at 10 days after therapy, with a variety of secondary outcomes, including new infections at one month and treatment-related adverse events.

The winner, if one can be crowned, was not placebo. At the test of cure visit in the intention-to-treat population – and likewise, the population that could be evaluated – placebo lagged behind both clindamycin and TMP-SMX by approximately a 12% absolute magnitude of difference. Recurrent infections at the same site, or another site, were lowest in the clindamycin group at 6.8% – and similar between TMP-SMX and placebo, at 13.5% and 12.4%, respectively. However, clindamycin was implicated in the highest rate of adverse events, at 21.9%, compared with TMP-SMX and placebo, at 11.1% and 12.5%, respectively. Most of the difference in adverse events can be attributed to diarrhea illness, although clostridium difficile was not isolated in any cases. There was one case of systemic hypersensitivity reaction thought to be related to TMP-SMX.

There were two main drivers for the difference in test of cure between the placebo cohort and the two antibiotic cohorts, and these were use of rescue antibiotics during the follow-up period and new infections at another site. The use of rescue antibiotics is not necessarily a reliable measure of treatment failure, but it is still reasonable to suggest this difference would not arise by chance alone, despite the small sample. Regarding generalizability to practice, the minority of abscesses were cupcake-sized, but these were still fairly substantial infections. The median size of the abscess was about 2.2cm in diameter, with surrounding erythema of 5.9cm in greatest dimension.

The takeaway, then, hinges on the generalizability of their population to your individual patient. If these are “smaller” skin abscesses, then I wager the bulk of my abscess encounters are for “tinier” abscesses. I doubt this changes much current practice with regard to antibiotics, or antibiotic selection, for those treating abscesses in the 2+cm range, but I expect the differences in cure rates shrink for smaller lesions. It falls within the realm of acceptable practice variation to weigh the harms of antibiotic use with the chance of recurrence or new infection for those lesions.

“A Placebo-Controlled Trial of Antibiotics for Smaller Skin Abscesses”

http://www.nejm.org/doi/full/10.1056/NEJMoa1607033

Double Coverage, Cellulitis Edition

The Infectious Disease Society Guidelines are fairly reasonable when it comes to cellulitis. Non-suppurative cellulitis – that is to say, without associated abscess or purulent drainage – is much less likely to be methicillin-resistant s. aureus. The guidelines, therefore, recommend monotherapy with a ß-lactam, typically cephalexin. Conversely, with a suppurative focus, trimethoprim-sulfamethoxazole monotherapy is an appropriate option. However, it’s reasonable to estimate current practice involves prescribing both agents somewhere between one fifth and a quarter of cases – presumably both wasteful and potentially harmful. This trial, therefore, examines this practice by randomizing patients to either double coverage or cephalexin plus placebo.

The short answer: no difference. The rate of clinical cure was a little over 80% of both cohorts in the per-protocol population. Of those with follow-up and treatment failure, over half progressed to abscess or purulent drainage on re-evaluation – and about two-thirds were cultured out as s. aureus. There was no reliable evidence, however, co-administration of TMP-SMX prevented this progression.

The really fun part of this article, however ties into the second line of their abstract conclusion:

“However, because imprecision around the findings in the modified intention-to-treat analysis included a clinically important difference favoring cephalexin plus trimethoprim-sulfamethoxazole, further research may be needed.”

This hedging stems from the fact 17.8% were excluded from the enrolled cohort for inclusion in the per-protocol analysis – and, depending on the modified intention-to-treat analysis definition, there was actually up to a 7.3% difference in failure rate favoring double coverage (76.2% vs 69.0%). This resulted from almost twice as many patients in the cephalexin monotherapy cohort taking <75% of antimicrobial therapy, missing follow-up visits, or other protocol deviations.

The best Bayesian interpretation of this finding is probably – and this is where frequentism falls apart – simply to ignore it. The pre-study odds of dramatic superiority of double coverage are low enough, and the outcome definition for the modified intention to treat cohort in question is broad enough, this finding should not influence the knowledge translation of this evidence. Stick with the IDSA soft-tissue guidelines – and one antibiotic at a time, please.  It is important to recognize – and educate patients – that about 1 in 6 may fail initial therapy, and these failures to not necessarily reflect inappropriately narrow antibiotic coverage nor therapeutic mismanagement.

“Effect of Cephalexin Plus Trimethoprim-Sulfamethoxazole vs Cephalexin Alone on Clinical Cure of Uncomplicated Cellulitis”
http://jamanetwork.com/journals/jama/article-abstract/2627970

Blood Cultures Save Lives and Other Pearls of Wisdom

It’s been sixteen years since the introduction of Early Goal-Directed Therapy in the Emergency Department. For the past decade and a half, our lives have been turned upside-down by quality measures tied to the elements of this bundle. Remember when every patient with sepsis was mandated to receive a central line? How great were the costs – in real, in time, and in actual harms from these well-intentioned yet erroneous directives based off a single trial?

Regardless, thanks to the various follow-ups testing strict protocolization against the spectrum of timely recognition and aggressive intervention, we’ve come a long way. However, there are still mandates incorporating the vestiges of such elements of care –such as those introduced by the New York State Department of Health. Patients diagnosed with severe sepsis or septic shock are required to complete protocols consisting of 3-hour and 6-hour bundles including blood cultures, antibiotics, and intravenous fluids, among others.

This article, from the New England Journal, looks retrospectively at the mortality rates associated with completion of these various elements. Stratified by time-to-completion following initiation of the 3-hour bundle within 6 hours of arrival to the Emergency Department, these authors looked at the mortality associations of the bundle elements.

Winners: obtaining blood cultures, administering antibiotics, and measuring serum lactate
Losers: time to completion of a bolus of intravenous fluids

Of course, since blood cultures are obtained prior to antibiotic administration, these outcomes are co-linear – and they don’t actually save lives, as facetiously suggested in the post heading. But, antibiotic administration was associated with a fraction of a percent of increased mortality per hour delay over the first 12 hours after initiation of the bundle. Intravenous fluid administration, however, showed no apparent association with mortality.

These data are fraught with issues, of course, relating to their retrospective nature and the limitations of the underlying data collection. Their adjusted model accounts for a handful of features, but there are still potential confounders influencing mortality of those who received their bundle completion within 3 hours as compared to those who did not.  The differences in mortality, while a hard and important endpoint, are quite small.  Earlier is probably better, but the individual magnitude of benefit will be unevenly distributed around the average benefit, and while a delay of several hours might matter, minutes probably do not.  The authors are appropriately reserved with their conclusions, however, only stating these observational data support associations between mortality and antibiotic administration, and do not extend to any causal inferences.

The lack of an association between intravenous fluids and mortality, however, raises significant questions requiring further prospective investigation. Could it be, after these years wandering in the wilderness with such aggressive protocols, the only universally key feature is the initiation of appropriate antibiotics? Do our intravenous fluids, given without regard to individual patient factors, simply harm as many as they help, resulting in no net benefit?

These questions will need to be addressed in randomized controlled trials before the next level of evolution in our approach to sepsis, but the equipoise for such trials may now exist – to complete our journey from Early Goal-Directed to Source Control and Patient-Centered.  The difficulty will be, again, in pushing back against well-meaning but ill-conceived quality measures whose net effect on Emergency Department resource utilization may be harm, with only small benefits to a subset of critically ill patients with sepsis.

“Time to Treatment and Mortality during Mandated Emergency Care for Sepsis”

http://www.nejm.org/doi/full/10.1056/NEJMoa1703058

Symptoms Over Science

There’s a reason general primary care has evolved to diagnose and treat uncomplicated urinary tract infections over the phone: the patient is the authority, not any test we order.

We’ve tried relying upon some constellation of the urinalysis, the urine microscopic examination, and, finally, the urine culture. Each of these has limitations, although, in many settings, the culture result has been the gold standard. However, this culture result, some quantification of the number of colony-forming units, is also somewhat of an arbitrary diagnostic – an arbitrary numerical cut-off must be used, with its own implications for sensitivity and specificity.

This brief clinical microbiology article evaluates the urine culture as a gold standard for the diagnosis of UTI by comparing it with polymerase chain reaction-based methods for measuring the presence of pathogenic bacteria. Based on 86 asymptomatic women and 220 general practice women complaining of UTI symptoms, these authors compared the number of positive culture results with positive PCR results. Of this sample, 149 had positive cultures for e. coli, while 211 patients had positive PCR for e. coli. Finally, combining the culture results – which identified other pathogens, as well – with the PCR for e. coli, 216 of 220 symptomatic women had pathogenic bacteria identified. In the control cohort, there were similar numbers of positive culture and PCR results – ~10% in each, which these authors feel accurately reflects the general rate of asymptomatic bacteruria in the general population.

These data correlate nicely with similar findings demonstrating a negative urine culture does not exclude clinical improvement while on antibiotics, and thus the reasonable conclusion we ought simply treat appropriate symptomatic patients without specifically relying on testing.

“Women with symptoms of a urinary tract infection but a negative urine culture: PCR-based quantification of Escherichia coli suggests infection in most cases”
http://www.clinicalmicrobiologyandinfection.com/article/S1198-743X(17)30209-4/abstract

Vitamin C for Sepsis

This is just a quick post in response to a tweet – and hype-machine press-release – making the rounds today.

This covers a before-and-after study regarding a single-center practice change in an intensive care unit where their approach to severe sepsis was altered to a protocol including intravenous high-dose vitamin C (1.5g q6), intravenous thiamine (200mg q12), and hydrocortisone (50mg q6). Essentially, this institution hypothesized this combination might have beneficial physiologic effects and, after witnessing initial anecdotal improvement, switched to this aforementioned protocol. This report describes their outcomes in the context of comparing the treatment group to similar patients treated in the seven months prior.

In-hospital mortality for patients treated on the new protocol was 8.5%, whereas previously treated patients were subject to 40.4% mortality. Vasopressor use and acute kidney injury was similarly curtailed in the treatment group. That said, these miraculous findings – as they are exhorted in the EVMS press release – can only be considered as worthy of further study at this point. With a mere 47 patients in both treatment groups, a non-randomized, before-and-after design, and other susceptibilities to bias, these findings must be prospectively confirmed before adoption. When considered in the context of Ioannidis’ “Why Most Published Research Findings Are False”, caution is certainly advised.

I sincerely hope prospective, external validation will yield similar findings – but will likewise not be surprised if they do not.

“Hydrocortisone, Vitamin C and Thiamine for the Treatment of Severe Sepsis and Septic Shock: A Retrospective Before-After Study”
https://www.ncbi.nlm.nih.gov/pubmed/27940189

Making Urine Cultures Great Again

As this blog covered earlier this month, the diagnosis of urinary tract infection – as common and pervasive as it might be – is still fraught with diagnostic uncertainty and inconclusive likelihood ratios. In practice, clinicians combine pretest likelihood, subjective symptoms, and the urinalysis to make a decision regarding treatment – and invariably err on the side of over-treatment.

This is an interesting study taking place in the Nationwide Children’s Hospital network regarding their use of urine cultures. In retrospect, these authors noted only half of patients initially diagnosed with UTI had the diagnosis ultimately confirmed by contemporaneous urine culture. Their intervention, then, in order to reduce harm from adverse effects of antibiotics, was to contact patients following a negative urine culture result and request antibiotics be stopped.

This tied into an entire quality-improvement procedure simply to use the electronic health record to accurately follow-up the urine cultures, but over the course of the intervention, 910 patients met inclusion criteria. These patients were prescribed a total of 8,648 days of antibiotics, and the intervention obviated 3,429 (40%) of those days. Owing to increasing uptake of the study intervention by clinicians, the rate of antibiotic obviation had reached 61% by the end of the study period.

There are some obvious flaws in this sort of retrospective reporting on a QI intervention, as there was no reliable follow-up of patients included. The authors report no patients were subsequently diagnosed with a UTI within 14 days of being contacted, but this is based on only 46 patients who subsequently sought care within their healthcare system within 14 days, and not any comprehensive follow-up contact. There is no verification or antibiotics actually being discontinued following contact. Then, finally, antibiotic-free days are only a surrogate for a reduction the suspected adverse events associated with their administration.

All that said, this probably represents reasonable practice. Considering the immense frequency with which urine cultures are sent and antibiotics prescribed for dysuria, the magnitude of effect witnessed here suggests a potentially huge decrease in exposure to unnecessary antibiotics.

“Urine Culture Follow-up and Antimicrobial Stewardship in a Pediatric Urgent Care Network”
http://pediatrics.aappublications.org/content/early/2017/03/14/peds.2016-2103

The Solution to Dilution is ….

Do we order a lot of urinalyses? Does the sun rise in the east? Does a bear ….

For a test we order with great frequency, there is actually quite a bit of complexity in its interpretation. The combination of symptoms, clinical context, the balance between sample contamination and presence of white blood cells, of nitrites and/or leukocyte esterase, and so on, can make it a relatively tricky test to interpret. The gold standard remains a urine culture.

Now – if you haven’t been already – you probably ought to be taking into account the urine specific gravity, as well.

This retrospective analysis of 14,971 children for whom paired urinalyses and urine cultures were available describes the test characteristics of WBCs/hpf, LE, and nitrites as stratified by urine specific gravity. There are a lot of numbers in this article – a “zillion” to be precise – across eighteen dense tables of +LR/-LR, sensitivity/specificity, and PPV/NPV, but the basic gist of the matter is: variations in urine concentration diminish the value of the test in different ways. As urine specific gravity increases, it becomes more likely a patient will not have a positive urine culture despite having typically diagnostic amounts of WBCs/hpf, +LE, and/or +nitrites. Likewise, with dilute urine, a lower threshold for WBCs/hpf may be needed to have adequate sensitivity.

Just one more layer to consider in this frequently used test of under-appreciated complexity.

“The Importance of Urine Concentration on the Diagnostic Performance of the Urinalysis for Pediatric Urinary Tract Infection”
https://www.ncbi.nlm.nih.gov/pubmed/28169050

A qSOFA Trifecta

There’s a new sepsis in town – although, by “new” it’s not very anymore. We’re supposedly all-in on Sepsis-3, which in theory is superior to the old sepsis.

One of the most prominent and controversial aspects of the sepsis reimagining is the discarding of the flawed Systemic Inflammatory Response Syndrome criteria and its replacement with the Quick Sequential Organ Failure Assessment. In theory, qSOFA replaces the non-specific items from SIRS with physiologic variables more closely related to organ failure. However, qSOFA was never prospectively validated or compared prior to its introduction.

These three articles give us a little more insight – and, as many have voiced concern already, it appears we’ve just replaced one flawed agent with another.

The first article, from JAMA, describes the performance of qSOFA against SIRS and a 2-point increase in the full SOFA score in an ICU population. This retrospective analysis of 184,875 patients across 15 years of registry data from 182 ICUs in Australia and New Zealand showed very little difference between SIRS and qSOFA with regard to predicting in-hospital mortality. Both screening tools were also far inferior to the full SOFA score – although, in practical terms, the differences in adjusted AUC were only between ~0.69 for SIRS and qSOFA and 0.76 for SOFA. As prognostic tools, then, none of these are fantastic – and, unfortunately, qSOFA did not seem to offer any value over SIRS.

The second article, also from JAMA, is some of the first prospective data regarding qSOFA in the Emergency Department. This sample is 879 patients with suspected infection, followed for in-hospital mortality or ICU admission. The big news from this article is the AUC for qSOFA of 0.80 compared with the 0.65 for SIRS or “severe sepsis”, as defined by SIRS plus a lactate greater than 2mmol/L. However, at a cut-off of 2 or more for qSOFA, the advertised cut-off for “high risk”, the sensitivity and specificity were 70% and 79% respectively.

Finally, a third article, from Annals of Emergency Medicine, also evaluates the performance characteristics of qSOFA in an Emergency Department population. This retrospective evaluation describes the performance of qSOFA at predicting admission and mortality, but differs from the JAMA article by applying qSOFA to a cross-section of mostly high-acuity visits, both with and without suspected infection. Based on a sample of 22,350 ED visits, they found similar sensitivity and specificity of a qSOFA score of 2 or greater for predicting mortality, 71% and 74%, respectively. Performance was not meaningfully different between those with and without infection.

It seems pretty clear, then, this score doesn’t hold a lot of value. SIRS, obviously, has its well-documented flaws. qSOFA seems to have better discriminatory value with regards to the AUC, but its performance at the cut-off level of 2 puts it right in a no-man’s land of clinical utility. It is not sensitive enough to rely upon to capture all patients at high-risk for deterioration – but, then, its specificity is also poor enough using it to screen the general ED population will still result in a flood of false positives.

So, unfortunately, these criteria are probably a failed paradigm perpetuating all the same administrative headaches as the previous approach to sepsis – better than SIRS, but still not good enough. We should be pursuing more robust decision-support built-in to the EHR, not attempting to reinvent overly-simplified instruments without usable discriminatory value.

“Prognostic Accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection Admitted to the Intensive Care Unit”

http://jamanetwork.com/journals/jama/article-abstract/2598267

“Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department”

http://jamanetwork.com/journals/jama/fullarticle/2598268

“Quick SOFA Scores Predict Mortality in Adult Emergency Department Patients With and Without Suspected Infection”

http://www.annemergmed.com/article/S0196-0644(16)31219-7/fulltext

Shenfu!

I will readily admit I am stepping outside the bounds of my expertise with this post – with respect to the “shenfu injection” and its effects on physiology. The authors describe shenfu as “originated from Shenfu decoction, a well-known traditional Chinese formulation restoring ‘Yang’ from collapse, tonifying ‘Qi’ for relieving desertion”. More specifically, from a physiologic standpoint: “Ginsenosides and aconite alkaloids are the main active ingredients in Shenfu. Ginsenosides are the determinant contributor to the vasodilator benefit of Shenfu, whereas the alkaloids play a vital role in the cardiac electrophysiological effect of Shenfu by blocking ion channels”. In China, a pharmacologic shenfu distillate is used routinely to treat sepsis and septic shock as a 100mL daily injection – and this is a placebo-controlled trial endeavoring to demonstrate its efficacy.

At face value, the trial appears reasonable – a targeted enrollment of 160 patients with a goal of detecting a 20% difference in mortality at 28-days, based on an expected overall mortality of 40%. Their primary outcome, however, were the co-primary outcomes of “length of ICU stay, the duration of vasopressor use, illness severity, and the degree of organ dysfunction.” A proper study, of course, has a single primary outcome – and, considering the study was powered for a mortality difference, this patient-oriented outcome probably ought to have been made primary.

Regardless, from the results presented here, it is reasonable to suggest this is promising and worthy of additional evaluation. Several outcomes – ICU LOS, APACHE II score, and duration of vasopressor us – reached statistical significance favoring the intervention. The mortality outcome did not meet statistical significance with the intervention at 20.5% and the placebo at 27.8%. However, an absolute mortality improvement of 7.3% is nothing to sneeze at – and I would be happy to see more work performed to replicate or generalize these results.

“Shenfu injection for improving cellular immunity and clinical outcome in patients with sepsis or septic shock”

https://www.ncbi.nlm.nih.gov/pubmed/28029485