A new US study has revealed that a widely used, federal hospital safety metric is “fundamentally flawed” when applied to emergency stroke care, potentially creating incentives that may discourage hospitals from performing lifesaving thrombectomy procedures, according to University of California Los Angeles (UCLA; Los Angeles, USA) researchers.
The study—published in the Journal of NeuroInterventional Surgery—examined Patient Safety Indicator (PSI) 04, a ‘failure-to-rescue’ measure developed by the US Agency for Healthcare Research and Quality (AHRQ) to track deaths following treatable complications in surgical patients. Researchers analysed data from the Nationwide Inpatient Sample covering 73,580 stroke thrombectomy procedures between 2016–2019 in the USA, along with detailed reviews of consecutive cases at UCLA.
Despite acknowledging that the metric is appropriate for elective procedures performed on relatively healthy patients, the study found PSI 04 to be inappropriate for endovascular mechanical thrombectomies.
“This metric was designed to identify preventable deaths but, when applied to emergency stroke care, it’s flagging unavoidable complications of severe strokes rather than problems with the procedure itself,” said Melissa Marie Reider-Demer (UCLA Health, Los Angeles, USA), the study’s first author. “The unintended consequence is that hospitals providing excellent stroke care to the sickest patients may appear to have poor safety records.”
PSI 04 is triggered when patients develop any of five complications—pneumonia, blood clots, sepsis, shock/cardiac arrest or gastrointestinal bleeding—after a procedure, and subsequently die in the hospital. The metric is used nationally for public reporting, hospital quality ratings, and pay-for-performance programmes, by the Centers for Medicare and Medicaid Services (CMS) and influential organisations like the Leapfrog Group, as noted in a UCLA press release.
The UCLA team analysed both national data and detailed case reviews to assess the metric’s appropriateness for stroke care. Their findings revealed that complications covered by PSI 04 occurred in 20.5% of stroke thrombectomy patients nationally, which is between one and three orders of magnitude higher than all of the 17 other PSIs (median, 0.1%). Additionally, the rate for stroke procedures was significantly higher than the 14.3% rate for all surgical procedures combined—and, among the 18 federal PSIs, PSI 04 had by far the highest event rate for all procedures, suggesting the metric may be “fundamentally problematic”.
At UCLA’s comprehensive stroke centre (CSC), researchers also examined every thrombectomy case flagged by PSI 04 between 2016–2018. An expert panel of neurointerventionists and neurologists reviewed each case and found that all patient deaths were related to complications linked to the severe presenting stroke rather than the thrombectomy procedure. Furthermore, thrombectomy procedures accounted for 7.2% of neurosurgical PSI 04 flags despite representing only 1.5% of neurosurgical procedures. The researchers conclude that there was “not a single case” representing an “actual preventable safety concern”.
The study authors’ claim that the PSI 04 metric is flawed—when applied to stroke thrombectomy—is predicated on two key points:
- The complications it tracks are common consequences of severe strokes themselves, not the procedure, and patients arriving with massive strokes are at high risk for pneumonia, blood clots and other complications regardless of treatment
- Stroke patients are already critically ill before the procedure, unlike patients undergoing elective surgeries—and, even when complications arise, these severely ill patients have far less resilience to survive them compared to relatively healthy surgical patients
“We’re essentially penalising hospitals for trying to save patients who are already dying from stroke,” Reider-Demer added. “These procedures give severely affected patients their only chance at survival or functional recovery, but the current metric makes it look like the hospitals are providing poor care.”
The researchers go on to warn that inappropriate safety metrics can create harmful incentives. As detailed by UCLA’s recent release, previous research has shown that public reporting of surgical mortality rates led some heart surgeons to ‘cherry-pick’ healthier patients to protect their performance ratings, limiting access for the sickest patients who need care most.
“There’s a real concern that hospitals might be discouraged from performing thrombectomy on the most severe stroke patients, or that stroke centres with high volumes of critically ill patients could be unfairly penalised in quality ratings and reimbursement,” commented Jeffrey Saver (UCLA Health, Los Angeles, USA), the present study’s senior author.
This issue has become more pressing as recent clinical trials have expanded thrombectomy to patients with more severe strokes caused by larger infarcts, who have high mortality rates even with intervention—rates that are, however, still lower compared to without intervention.
The US CMS has proposed revising PSI 04 to exclude patients with acute conditions like stroke coded as the principal reason for admission, with implementation planned for fiscal year 2027. In Saver’s view, this revision addresses important shortcomings.
“This revision makes sense from a clinical perspective,” he said. “The current metric doesn’t identify preventable events in stroke care and has the potential to mislead the public about hospital quality while creating incentives that could harm the sickest patients.”
