Contact: Robert Neal, 617-432-0448 (bneal@warren.med.harvard.edu) Don Gibbons, 617-432-0442 (dgibbons@warren.med.harvard.edu)
Study Finds Physician Report Card Data Not Used By Physicians
BOSTON -- Published reports evaluating the performance of cardiac surgeons in Pennsylvania have little or no effect on the referral practices of a vast majority of cardiologists in that state, Harvard Medical School researchers report in the July 25 issue of The New England Journal of Medicine.Despite the neutral impact on referrals, the physician "report cards" may discourage cardiac surgeons from taking on risky cases for coronary artery bypass graft surgery (CABG) for fear of lowering their averages. These numbers have been widely publicized.
"Publicly released performance reports on quality can be an important lever for quality improvement," says Arnold Epstein, professor of medicine and health care policy at Harvard. "The Pennsylvania report is the state of the art in many ways. Yet our data suggest that there is still a credibility gap with clinicians. Closing that gap is vital for these reports to reach their full potential." Epstein coauthored the report with Eric Schneider, who was a research fellow in medicine at Harvard and now is at the Institute for Health Care Research and Policy at Georgetown University.
The findings arrive at a time when insurers and health-care consumers are putting individual doctors, along with hospitals and health plans, under increasing scrutiny and pressure to prove quality of care. Several states -- including Pennsylvania, New York, and California -- have already adopted statewide grading systems, and others are planning to implement them.
"The genie is out of the bottle," says Epstein. "Physician report cards will happen with greater or lesser speed. And in ways that are more positive or less positive. The medical community needs to be part of this process to try and make it as positive as possible."
The researchers polled 279 cardiologists and 58 cardiac surgeons about their use of Pennsylvania1s state-issued Consumer Guide to Coronary Bypass Graft Surgery. Since 1992, the consumer guide has listed annual mortality rates for all hospitals and surgeons providing CABG in the state of Pennsylvania. Mortality rates were adjusted to allow for variation of risk in the cases.
Only 10% of physicians said they found that the consumer guide's risk-adjusted mortality rates were "very important" in assessing a cardiac surgeon's performance. The majority said they never discussed the guide with patients who were candidates for a CABG. Of the cardiologists polled, 87% reported that the guide had little or no impact on their referral practices.
The most potentially disturbing finding of our survey of cardiovascular specialists is their belief that access to care has decreased for severely ill patients who need CABG,2 write Schneider and Epstein.
A majority (59%) of cardiologists said it was "more difficult" or "much more difficult" to find a referral for a severely ill patient in need of CABG surgery. An even larger majority (63%) of cardiac surgeons reported that they were less willing to take severely ill patients who need coronary artery bypass surgery.
Cardiac surgeons may be reluctant to treat the severely ill due to the very public nature of the evaluation systems. Results of the Pennsylvania physician performance reports are published annually in the lay press.
The analogous report in New York State has been in The New York Times, where individual doctors at both ends of the spectrum were publicly named. "These are public data that can have an obvious impact on a surgeon's or hospital's reputation," says Schneider. "This is a very powerful intervention."
Physicians attributed the minimal impact on referral practices to three limitations in the data. To begin with, mortality was thought to be an incomplete measure of quality of care. "I think they1re right about that. Many things other than a surgeon's quality of care can affect that outcome," says Epstein.
Though mortality rates were adjusted on a case-by-case basis according to the risk entailed, many physicians worried that the risk-adjustment methods were inadequate. "Risk adjustment will never be perfect. But the risk adjustment we have available for coronary artery bypass surgery is probably as good as we get in medicine right now. It's been a highly studied condition," says Epstein.
Those polled were also concerned that hospitals and physicians may have biased the data. "I don't think the issue here is fraud," says Schneider. "There is a strong incentive for hospitals and physicians to document all comorbidities."
"Despite these limitations, the Pennsylvania guide and those in New York are impressive prototypes for future programs," according to Epstein. "These really are the gold standard for what we have."
The researchers' study focuses on the impact of the consumer guide on physicians1 perceptions, rather than on patients' perceptions. "Our sense, certainly mine as an internist, was that the real decision-maker here, along with the patient, is the cardiologist," says Epstein.
Schneider and Epstein are currently interviewing patients who underwent CABG surgery at hospitals evaluated by the Pennsylvania guide to see whether they knew about the guide and also to see if their perceptions jibe with the consumer guide ratings.
So this study is part of a broader effort directed at this issue, says Epstein.
News Release