With probably underpowered derivation and validation, a model that seems to overfit the data, and incorporating an impractical and questionable cardiac biomarker – despite a lovely continuous predictive function – this instrument is doomed in its current form.
This is the Manchester Acute Coronary Syndromes (MACS), a prospectively derived and validated risk-stratification instrument. These authors identify an 8 variable decision instrument based on 698 patients at Manchester Infirmary – including hsTnT, heart-type fatty-acid binding protein, ECG changes, diaphoresis, vomiting, radiation to right shoulder, worsening angina, and hypotension – and then validate it on 463 patients from Stepping Hill Hospital. In the validation, 27.0% of patients were ultimately classified as “very low risk” with 98% sensitivity (95% CI 93.0% to 99.8%) for 30-day MACE, and the authors feel this tool could reduce unnecessary admissions.
My favorite feature from this study is the derivation of a continuous function for prediction of 30-day outcomes. The authors state an AUC of 0.92 for the function predicting MACE, which suggests potential as a useful tool for discussing individualized risks with patients. Rather than simply dichotomize a “very low risk” cohort, the predictive function could help aid shared decision-making conversations with patients.
However, the utilization of fatty-acid binding protein is questionable. These same authors presented work favoring H-FABP with an AUC for diagnosis of AMI of 0.86, but compared it against a troponin assay with an AUC of 0.70. A response to that same article notes the authors probably made inappropriate comparisons, and modern conventional troponin assays and/or high-sensitivity troponin assays have AUCs >0.90. It’s not clear what, or how much, additional value this biomarker adds to this study – and its inclusion essentially obviates the generalizability of the tool. No rapid, automated H-FABP assay is available suitable for use in an ED context. It is also unfortunate the corresponding author declares conflict-of-interest with the manufacturers of the assays used.
Interestingly, as well, the authors focus only on the “very low risk” group when an odd thing happens in their validation population – the “low risk” group actually had fewer MACE than the “very low risk” group (1.2% vs. 1.6%). This is a substantial reversal of the derivation population (5.8% vs. 0.4%), suggesting the attempted validation reveals their model may be overfitting the data. The authors state the second site validation is a strength regarding combating overfitting, but do not mention this inconsistency in the outcomes.
And, finally, I’m not entirely certain what question this study was designed to answer. The focus on a “very low risk” cohort in the discussion doesn’t entirely match the study design – it seems other studies focused on outcomes in low risk chest pain have specifically excluded patients whose presentation is clearly AMI on initial presentation. The inclusion of the entire spectrum of disease, while valuable for their general model, dilutes the strength of their “very low risk” conclusions, as evidenced by wide confidence intervals around the sensitivity for MACE.
As the authors note in their discussion, additional work needs to be done to compare their model with other risk-stratification tools. And, for anyone to use this tool other than the authors, the H-FABP needs to be dropped. At the least, however, it fits in nicely with another recent critique – that many “low risk” and “very low risk” patients do not require observation and immediate provocative testing, where the false-positives and resource expenditures are simply preposterous.
“The Manchester Acute Coronary Syndromes (MACS) decision rule for suspected cardiac chest pain: derivation and external validation”
http://heart.bmj.com/content/early/2014/04/29/heartjnl-2014-305564