…but we don’t know which half. This highly entertaining study dredged the New England Journal of Medicine for the last decade, asking a simple question: does new literature confirm or refute current practice?
They identified 1,344 articles concerning a specific medical practice. Of these, 363 tested an established medical practice. 38% confirmed current practice, 40% rejected current practice, and 22% were inconclusive. Examples of rejected medical practice included:
- Primary rhythm control strategy in patients with atrial fibrillation.
- The use of aprotinin during cardiac surgery.
- Cyclooxygenase 2 inhibitors due to cardiovascular events.
- Tight glycemic control vs. more permissive standards.
- Benefit of stenting for patients with stable coronary disease.
…and so on. It’s a fascinating list – and this is just one journal.
Of further interest, the majority of articles concerning specific medical practice concerned the development of a new medical drug or intervention, and most of these were positive. I expect we shall see half of those similarly rejected by follow-up investigation in the next decade….
“A Decade of Reversal: An Analysis of 146 Contradicted Medical Practices”
I kind of halfway knew that.
50% of the time it works every time.
What do you think of Skeptical scalpels discussion of surgical article "refuting" established practice?
The article in question is here: http://skepticalscalpel.blogspot.com/2013/08/discrediting-paper-about-discredited.html
I would hardly say he "discredits" the paper through his discussion. Out of 11 surgical papers named in the original paper, Skeptical Scalpel only takes issue with two of them. With the hernia mesh paper, there may be significant technical differences and recurrence rates between the different mesh sizes. However, citing continued usage of laproscopic repair in no way invalidates the original paper – physicians persisting in questionable or erroneous practice in no way validate said practice. As for the pre-operative biliary drainage – I don't have enough expertise to comment. As he mentions, one paper is simply one paper in one journal, and the practice could be confirmed in another study in another journal. Of course, then, some of the papers confirming established practice in the NEJM could be paired with other papers in specialty journals refuting the practice. It's less important whether each individual paper is completely defensible in its refutation, and more important just to describe the ongoing controversy regarding many practices we might have previously thought unquestioned.
My point was that if two of the so-called discredited practices really were not discredited by each having a single negative paper published in the NEJM, then how many other discredited practices really were not discredited?
There are others in addition to the two ones I cited. For example, discredited practice #100 says that as of 2008, steroids have been proven ineffective in septic shock. Yet the 2012 Surviving Sepsis Guidelines (http://www.survivingsepsis.org/Guidelines/Documents/Hemodynamic%20Support%20Table.pdf) recommend hydrocortisone for patients in septic shock who are not stabilized with fluids and vasopressors.
A single paper in the NEJM should not necessarily be relied upon for instituting or discontinuing a practice.
I absolutely agree with practices not being truly "discredited" by a single paper in a single journal. But, they are still significant, high-profile publications that question existing practice, within the bounds of the external validity of each paper. Placed into larger context, each paper may be an erroneous outlier or part of a spectrum of refinement and critique of an existing practice. I think it's still a valuable exercise and powerful statement to peel back the illusion of certainty regarding much of what we do, rather than focus on the details of each specific citation.
Steroids in sepsis is a great example: downgraded since CORTCUS but not forgotten. Perhaps need to establish more of a graded slope instead of guideline = truth, to take uncertainty and evidence vs. experience into account.