This is a rather dangerous article for many reasons. Firstly, it’s published in a high-impact journal and received a fair bit of coverage in the news media. Secondly, it concludes its discussion by suggesting this ought to be adopted as a standardized rule for the evaluation of acute headache – this isn’t just a descriptive study on features of subarachnoid hemorrhage, it’s been given an official-sounding title, the “Ottawa SAH Rule”. Because of this, there’s significant potential for the rule described here to be adopted as widespread practice.
Therefore – it better be nearly perfect.
This is a prospective cohort from 10 university-affiliated Canadian hospitals. They looked at non-traumatic headaches reaching maximal intensity within 1 hour not part of a recurrent headache syndrome, and found 132 patients with SAH out of 2,131 assessed. They specifically gathered information on three previously-derived prediction rules and found none of them were 100% sensitive – so they chose the required elements from each to reach 100% (95% CI 97.2-100) sensitivity. The cost of this 100% sensitivity? Degeneration of specificity from the 28-35% in the three individual rules down to 15% (95% CI 13.8-16.9) in the final rule. The authors observed application of the derived rule would have decreased investigations for SAH from 84% of the enrolled cohort down to 74% of the enrolled cohort, and thusly their rule is superior to routine clinical practice by maintaining 100% sensitivity while decreasing resource utilization.
I think their inclusion criteria are fine – a rapid-onset, severe, atraumatic headache is the classical population of interest. Patients without this feature have such a low incidence of SAH that it’s unreasonable to evaluate for it. Their outcome measures, unfortunately, were a little softer. The positive diagnoses are reasonable – CT proven SAH or positive lumbar puncture with a source feature on cerebral angiography. However, only 82% underwent CT and 39% underwent LP, with a six month telephone follow-up – and a small number were lost to follow-up. Many would argue that CT alone is not sufficient for ruling out SAH, and 6-month survival is a limited proxy. This weakens its claim for 100% sensitivity.
Then, of course, a 15% specificity is awful. This isn’t necessarily a criticism of the authors, but more a recognition of the limitation of distilling diverse clinical data into concise decision instruments. 19 different patient features were significantly different between the SAH and no-SAH groups; reducing this to just 6 features discards so much information that an instrument designed for a complex clinical prediction is bound to fail. There were 1,694 false positives by the rule compared with 132 true positives. If this rule is applied without the strict exclusion criteria specified in the publication, there may be a huge number of inappropriate investigations.
Then, the rate of investigation comparison is also probably invalid. These institutions underwent specific 1-hour orientation to the study being performed and were actively involved in gathering clinical data for the study. I’m certain the 84% rate of investigation observed was conflated by the ongoing research at hand. The previous Perry study from 2011 had an evaluation rate of 57%, so it’s hard for me to believe the statistics from the current publication.
Finally, the kappa values for inter-observer agreement were rather mixed. Based on only sixty cases where two physicians evaluated the same patient, four of the six final rule elements had kappas between 0.44 and 0.59, representing only moderate agreement. This is a significant threat to the internal validity of the underlying data in support of their rule.
Overall, yes – the elements they identify through their observational cohort are likely to capture most cases of SAH. However, the limitations of this study and the poor specificity make me reluctant to buy in completely – and certainly not adopt it as a standardized “rule”.
“Clinical Decision Rules to Rule Out Subarachnoid Hemorrhage for Acute Headache”