It is an oft-repeated pseudoaxiom that half of your medical knowledge is wrong – we just don’t yet know which half. This hyperbole is founded, in part, in the work of Ioannidis and examinations of refutations of clinical trials. This study represents another log to toss onto our conflagaration of uncertainty.
These authors noted a recent analysis of large clinical trials funded by the National Heart, Lung, and Blood Institute showed these trials had mostly neutral results. Neutral trials, of course, are still valuable – they frequently help inform efforts to prevent unnecessary or low-value care. However, the prior analysis only evaluated studies published after year 2000, the year trials began being registered in ClinicalTrials.gov. This new analysis extended the review window back to 1975, to see if this trend of predominantly null outcomes was a historical trend, or whether the requirement to publicly pre-specify a primary outcome might have had some effect.
Not terribly surprisingly, there is a relatively clear difference in the frequency of null outcomes in the post-ClinicalTrials.gov period compared with prior:
The authors’ quite reasonable conclusion: prior trials, not having the requirement to pre-specify a primary outcome, were probably more likely to retroactively promote a positive outcome as the primary outcome of a study.
Should all these primary outcomes prior to 2000 be re-tested? Considering all these prehistoric trials noted ties to industry, it’s probably a fair suggestion – and, certainly, it fuels our further skeptical re-examination of established medical practices.
“Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time”
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0132382