JAMA, Integrity, Accessibility, and Social vs. Scientific Peer Review

Yesterday, I posted regarding a JAMA Clinical Evidence series article involving procalcitonin measurement to guide antibiotics stewardship.  This is an article I read, raised concerns regarding other negative trials in the same spectrum, and depressingly noted conflict-of-interest with each of the three authors.

Graham Walker, M del Castillo-Hegyi, Javier Benitez and Chris Nickson picked up the blog post, spread it through social media and Twitter, and suggested I write a formal response to JAMA for peer-reviewed publication.  My response – I could put time into such a response, but what would JAMA's motivation be to publish an admission of embarrassing failure of peer-review?  And, whatever response they published would be sequestered behind a paywall – while BRAHMS/ThermoFisher continued to happily reprint away their evidence review from JAMA.  Therefore, I will write a response – but I will publish it openly here, on the Internet, and the social peer review of my physician colleagues will determine the scope of its dissemination based on its merits.

Again, this JAMA article concerns procalcitonin algorithms to guide antibiotic therapy in respiratory tract infections.  This is written by Drs. Schuetz, Briel, and Mueller.  They each receive funding from  BRAHMS/ThermoFisher for work related to procalcitonin assays (www.procalcitonin.com).  The evidence they present is derived from a 2012 Cochrane Review – authored by Schuetz, Mueller, Christ-Crain, et al.  The Cochrane Review was funded in part by BRAHMS/ThermoFisher, and eight authors of the review declare financial support from BRAHMS/ThermoFisher.

The Cochrane Review includes fourteen publications examining the utility of procalcitonin-based algorithms to initiate or discontinue antibiotics.  Briefly, in alphabetical order, these articles are:

• Boudama 2010 - Authors declare COI with BRAHMS.  This is a generally negative study with regards to the utility of procalcitonin.  Antibiotic use was reduced, but mortality trends favored standard therapy and the study was underpowered for this difference to reach statistical significance (24% mortality in controls, 30% mortality in procalcitonin-guided at 60 days).

• Briel 2008 - Authors declare COI with BRAHMS.  This study is a farce.  These ambulatory patients were treated with antibiotics for such "bacterial" conditions as the "common cold", sinusitis, pharyngitis/tonsilitis, otitis media, and bronchitis.  

• Burkhardt 2010 - Authors declare COI with BRAHMS.  Yet another ambulatory study randomizing patients with clearly non-bacterial infections.

• Christ-Crain 2004 - Authors declare COI with BRAHMS.  Again, most patients received antibiotics unnecessarily via poor clinical judgement, for bronchitis, asthma, and "other".

• Christ-Crain 2006 - Authors declare COI with BRAHMS.  This is a reasonably enrolled study of community-acquired pneumonia patients.

• Hochreiter 2009 - Authors declare COI with BRAHMS.  This is an ICU setting enrolling non-respiratory infections along with respiratory infections.  These authors pulled out the 47 patients with respiratory infections.

• Kristofferson 2009 - No COI declared.  Odd study.  The same percentage received antibiotics in each group, and in 42/103 cases randomized to the procalcitonin group, physicians disregarded the procalcitonin-algorithm treatment guidelines.  A small reduction in antibiotic duration was observed in the procalcitonin group.

• Long 2009 - No COI declared.  Unable to obtain this study from Chinese-language journal.

• Long 2011 - No COI declared.  Most patients were afebrile.  97% of the control group received antibiotics for a symptomatic new infiltrate on CXR compared with 84% of the procalcitonin group.  85% of the procalcitonin group had treatment success, compared with 89% of the control group.  Again, underpowered to detect a difference with only 81 patients in each group.

• Nobre 2008 - Authors declare COI with BRAHMS.  This is, again, an ICU sepsis study – with 30% of the patients included having non-respiratory illness.  Only 52 patients enrolled.

• Schroeder 2009 - Authors declare COI with BRAHMS.  Another ICU sepsis study with only 27 patients, of which these authors pulled only 8!

• Schuetz 2009 - Authors declare COI with BRAHMS.  70% of patients had CAP, most of which was severe.  Criticisms of this study include critique of "usual care" for poor compliance with evidence supporting short-course antibiotic prescriptions, and poor external validity when applied to ambulatory care.

• Stolz 2007 - Authors declare COI with BRAHMS.  208 patients with COPD exacerbations only.

• Stolz 2009 - Authors declare COI with BRAHMS.  ICU study of 101 patient with ventilator-associated pneumonia.

So, we have an industry-funded collation of 14 studies – 11 of which involve relevant industry COI.  Most studies compare procalcitonin-guided judgement with standard care – and, truly, many of these studies are straw-man comparisons against sub-standard care in which antibiotics are being prescribed inappropriately for indications in which antibiotics have no proven efficacy.  We also have three ICU sepsis studies included that discard the diagnoses other than "acute respiratory infection" – resulting in absurdly low sample sizes.  As noted yesterday, larger studies in ICU settings including 1,200 patients and 509 patients suggested harms, no substantial benefits, and poor discriminatory function of procalcitonin assays for active infection.  

Whether the science eventually favors procalcitonin, improved clinical judgement, or another biological marker, it is a failure of the editors of JAMA to publish such deeply conflicted literature.  Furthermore, the traditional publishing system is configured in such a fashion that critiques are muted compared with the original article – to the point where I expect this skeptical essay to reach a far greater audience and have a greater effect on practice patterns via #FOAMed than through the traditional route.