As I covered last year, half of what you’ve been taught in medicine is wrong – we just don’t know which half.
And, it turns out, sometimes even the same authors taking a second look at the same data as before, can come up with new – and wildly different – conclusions.
This is a review of 37 randomized-controlled trials published after 1966 paired with 37 “re-analyses” of the same data. These trials span the entire medical domain, from mycophenolate therapy after cardiac transplantation to homeopathy for fibrosis. Of these 37 re-analyses, 32 of them involved authors from the original research group. These re-analyses differed by changing statistical techniques, outcome definitions, or other study interpretation methods.
Following re-analysis, 13 (35%) changed the original conclusions – either suggesting more, fewer, or even entirely different patients should be treated. The implication regarding the reliability of our evidentiary basis for medical practice is obviously profound – if even just the original authors and data can result in conflicting conclusions.
In his editorial, Harlan Krumholz argues the solution is clear: open the data. Independent verification of findings – whether by erasing bias or undesired mathematical pathology – is critical to ensuring the most complete understanding of the evidence base. If our highest duty is to our patients, we must break down the barriers created by self-interest and institutional policies in order to promote data sharing – and serve patients by improving the clarity and transparency of medical practice.
“Reanalyses of Randomized Clinical Trial Data”
http://www.ncbi.nlm.nih.gov/pubmed/25203082