As with last week’s coverage of the updated Cochrane Systematic Review for tPA in acute ischemic stroke, the key question is: what’s new?
The first pooled meta-analysis, published in The Lancet in 2004, included NINDS, ECASS I, ECASS II, and ATLANTIS. It was subsequently updated in 2010 to add ECASS III and EPITHET. Now, these authors have decided to add IST-3.
I am actually a huge fan of individual-patient meta-analyses. Depending on the data availability, the similarity of trial protocols, and other issues associated with heterogeneity, this is the gold-standard for aggregating data and increasing power. Individual-patient analyses also allow for more reliable exploration of subgroup effects not otherwise possible through regular meta-analyses or systematic reviews.
But, at the crux of it, a meta-analysis is only as good as the included trials – and this is a topic much debated over the last twenty years. Entertainingly, the 2014 publication includes this bland statement:
Role of the funding source
The funders had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data and responsibility for the decision to submit for publication.
Yes, the funding source had nothing to do with the study design, excepting all the folks receiving speaker fees and honoraria – and the fact the original idea and refinements to the approach were contributed by one of the authors who is an employee Boehringer Ingelheim:
KRL has received speaker fees from and has served on the data monitoring committee of trials for Boehringer Ingelheim; his department has received research grant support from Genentech. GA has received research grant support from Lundbeck, fees for consultancy and advisory board membership from Lundbeck, Covidien, Codman, and Genentech, fees for acting as an expert witness, and owns stock in iSchemaView. EB is employed by Boehringer Ingelheim. SD has received honoraria from Boehringer Ingelheim, EVER Pharma, and Sanofi and has received fees for consultancy and advisory board membership from Boehringer Ingelheim and Sanofi. GD has received research grant support from the NHMRC (Australia) and honoraria from Pfizer and Bristol-Myers Squibb. JG has received fees for consultancy and advisory board membership from Lundbeck. RvK has received speaker fees and honoraria from Penumbra and Lundbeck. RIL has received honoraria from Boehringer Ingelheim. JMO has received speaker fees from Boehringer Ingelheim. MP has received travel support from Boehringer Ingelheim. BT has received honoraria from Pfizer. DT has received speaker fees and fees for consultancy and advisory board membership from Boehringer Ingelheim and Bayer. JW has received research grant support from the UK Medical Research Council and from Boehringer Ingelheim to the University of Edinburgh for a research scanner bought more than 10 years ago. WW has received research grant support from the UK Medical Research Council. PS has received honoraria for lectures which were paid to the department from Boehringer Ingelheim. KT has received research grant support from the Ministry of Health, Labour, and Welfare of Japan, and speaker fees from Mitsubishi Tanabe Pharma. WH has received research grant support from Boehringer Ingelheim, and speaker fees and fees for consultancy and advisory board membership from Boehringer Ingelheim.
The same level of COI was present in previous versions – including employees of the sponsor as authors – but, interestingly, at least the 2004 version explicitly acknowledges a critical issue:
Role of the funding source
For the ATLANTIS trials, Genentech provided full support for the study and Genentech employees participated to some extent in study design, data collection, data analysis, and data interpretation, writing of the report, and in the decision to submit the manuscript for publication. For the ECASS trials, Boehringer Ingelheim provided full support. Employees of Boehringer Ingelheim participated in study design, in data collection, data analysis, data interpretation, writing of the report, and in the decision to submit the report for publication.
Nothing has changed. If you trusted the data then, you trust the data now – and vice-versa.
So, what is new? If anything, what’s new is worse than preceded it. The authors have nearly doubled the cohort for analysis – by the inclusion of a decade-long trial crippled by the bias introduced by an open-label, mostly unblinded design. Despite the massive resources invested in conducting it, unfortunately, IST-3 is too flawed for inclusion – due to the unfortunate likelihood any small positive signals regarding tPA are certain to be exaggerated. And, simply put, that’s where the astute reader ought to stop reading this publication. There’s no point in trying to interpret their results, to fuss over the heterogeneity between trials, missing baseline characteristics for their many subgroup analysis, or whether the trials stopped early for harm or futility – ATLANTIS – are properly acknowledged. The authors also omit several planned secondary analyses described in their statistical protocol – although, considering the garbage-in/garbage-out nature of this work, it’s of debatable importance.
The last decade of prospective research – ECASS III and IST-3 – has done nothing but degrade the quality of evidence describing tPA in acute ischemic stroke. If there is, indeed, anyone left on the fence regarding the pro/con tPA debate, this effort ought move the needle zero to none. Very early treatment with tPA probably benefits a properly selected subset of patients with acute ischemic stroke. The rest – whether increasing age, high-or-low NIHSS, specific stroke syndromes, or time-dependent factors – have much smaller, if any, chance of benefit exceeding chance of harm. Until we have unbiased evidence, we’ll never truly know how to best select patients for this therapy – and neurologists will continue to lament low treatment rates, while Emergency Physicians continue to reject pro-tPA clinical policies. Only new, independent data has a chance to substantially change our approach to acute ischemic stroke.
“Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials”
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(14)60584-5/abstract
Meta-analyses are also only as good as the trials EXCLUDED — as for example here, the trials with STK … not to mention any data that's been suppressed or hidden. (That never happens!)
I won't nitpick your generally excellent piece to debate the unproven (and, I believe, unlikely) claim that "very early treatment with tPA probably benefits a properly selected subset of patients," but I will make one other point: IST-3 is, as you say, deeply flawed … but that surely doesn't mean we should ignore it. When a HUGELY (pro-treatment) biased study can't find any benefit in its primary outcome (and in truth in any reliably measured outcome) — and when that trial is by far the largest ever done — I don't see how that shouldn't greatly move the needle in an (even more) negative direction.
Thank you Ryan for this insightful post. It is always daunting commenting on your posts. Now even more so following the gigantic shadow cast by Dr. Hoffman.
I agree with everything you and Dr. Hoffman have said. I would just like to highlight the subtle harms demonstrated in both the 2014 and 2010 Lancet meta-analyses. The most recent analysis demonstrated a 1.4% absolute increase in 90-day mortality in the patients randomized to alteplase. Though failing to reach statistical significance, the confidence interval came very close to crossing zero (0.99-1.25). This is very similar to the mortality difference (1.8%) demonstrated in the 2010 Lancet meta-analysis. It is important to note that the recently published Cochrane analysis of all 27 trials investigating thrombolytics for acute CVA found the very same absolute harm (1.5%) to be statistically significant when examined in a larger cohort. The failure to reach statistical significance in the Lancet meta-analyses seems more to be a comment on the lack of statistical power rather than an endorsement of alteplase’s safety.
I absolutely agree, regarding the exclusion of non-alteplase trials from their evaluation of thrombolytic therapy. The systematic review reported separate results for all-agents vs. alteplase-alone. A reasonable argument is made regarding the observed recanalization rates, physiologic half-lives, and fibrin specificity of the various agents to claim they ought be analyzed separately, but an equally reasonable argument can be made they are only discarded as a matter of convenience. If tPA trials were even less convincing than they currently are, if the streptokinase or desmoteplase trials showed more substantially encouraging results, there's no doubt our trialists would reverse their argument to include them, rather than exclude.