Infection data may not be comparable across hospitals, study shows

A new study has found that some kinds of infection data may not be comparable across hospitals, and may not be suitable for use as a performance measure.

Published in the leading US policy journal Milbank Quarterly, the research found huge variability in how English hospitals collected, recorded and reported their rates of central line infections to a patient safety programme. The study was funded by the Health Foundation, a major UK charitable foundation aiming to improve quality of care.

"Central line infections occur in tubes used in treating seriously ill patients. These infections are largely preventable, and hospitals need to be able to monitor how well they are doing in controlling them," said study author Professor Mary Dixon-Woods from University of Leicester. "But because hospitals don't use the same methods to generate the data, using their reported rates to produce league tables of performance or to impose financial sanctions, as happens in the US, is probably not appropriate."

Researchers found most variability arose for what they called "mundane" reasons. These included challenges in setting up data collection systems, different practices in sending blood samples for analysis, and difficulties in deciding the source of infections.

"Although hospitals were given clear, standardised definitions to use, many laboratories did not have the tools to make the most definitive assessment of where infections were coming from in the patient's body," said Julian Bion, Professor of Intensive Care Medicine at University of Birmingham. "That meant they had to use clinical judgement to decide whether any infection was due to a central line. Once you are relying on judgement, you will get variation."

The study dismissed 'gaming' as the explanation for the variations the researchers found. "Some previous studies have reported deliberate manipulation of performance data to 'look good'", said Professor Dixon-Woods. "We found very little evidence of this. We did find that doctors were fine with making a clinical decision that a patient might have a central line infection for purposes of treating them, but they wanted better evidence if they were going to report an infection externally. That meant that sometimes a patient who was treated as having an infection while in hospital was not counted as having an infection when data were reported. Doctors also varied in how many blood samples they sent off for laboratory analysis".

Professor Bion commented that these findings have important lessons for those involved in improving the quality of patient care. He said: "The study shows that if we are going to use data produced by hospitals to guide quality improvement, data collection systems must be carefully designed and operated, fully integrated with clinical priorities, and impose minimum burden. We need data that clinicians and organizations believe in. And until we have that, we need to be very careful about comparing performance across hospitals. We certainly need to avoid basing pay-for-performance schemes on central line data, because it's now becoming clear that these schemes have many unintended consequences."

Dr Elaine Maxwell, Assistant Director at the Health Foundation, commented: " This research highlights the very real difficulties in measuring safety in healthcare. Adverse events such as infections are increasingly being used in performance management. This study demonstrates that accurate data collection is more complex than may have at first been imagined. The Health Foundation is continuing to support the development of reliable and sensitive safety measures to reflect sustained improvement."

Notes

• The full reference for this paper is: Dixon-Woods M, Leslie M, Bion JF, Tarrant C. What Counts? An ethnographic study of infection data reported to a patient safety program. Milbank Quarterly 2012; 90: 548-591 http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0009.2012.00674.x/abstract
• The study was conducted by Mary Dixon-Woods, (University of Leicester), Myles Leslie (Johns Hopkins University), Julian Bion (University of Birmingham), and Carolyn Tarrant (University of Leicester).
• The study is published in Milbank Quarterly. Ranked as one the leading US academic journals in health policy, Milbank Quarterly has been published for 80 years.
• Central lines, also known as central venous catheters, are narrow tubes inserted into large veins, with the tip lying close to the heart. They are most frequently used in intensive care units (ICUs) in the care of seriously ill patients. CVCs provide a route through which pathogens can enter the bloodstream and cause infections that can result in death or serious complications. Research evidence strongly indicates that many CVCBSIs can be avoided through compliance with specific evidence-based standards of practice (mainly relating to hygiene) when inserting, and managing patients with, CVCs.
• This study was funded by the Health Foundation, an independent charity working to continuously improve the quality of healthcare in the UK. For more information visit www.health.org.uk

About the Health Foundation:

• We want the UK to have a healthcare system of the highest possible quality – safe, effective, person-centred, timely, efficient and equitable. We believe that in order to achieve this, health services need to continually improve the way they work.
• We are here to inspire and create the space for people, teams, organisations and systems to make lasting improvements to health services.
• Working at every level of the healthcare system, we aim to develop the technical skills, leadership, capacity, knowledge, and the will for change, that are essential for real and lasting improvement.
• For more information visit www.health.org.uk
• You can follow the Health Foundation on Twitter: www.twitter.com/healthfdn

Contacts

1. Professor Mary Dixon-Woods, Department of Health Sciences, University of Leicester, email md11@le.ac.uk, tel +44 (0)116 2297262

2. Professor Julian Bion, University Dept Anaesthesia & ICM, University of Birmingham PA: Claire Price: c.s.price@bham.ac.uk, tel: +44 (0)121 627 2060

---

---