Anthony Delaney from the Royal North Shore Hospital, Australia, Chip Doig, from the University of Calgary, Canada, and colleagues, studied 139 reports of meta-analyses addressing critical care topics published between January 1994 and December 2003. They assessed whether data from the reports met any of nine of the criteria listed in the Overview Quality Assessment Questionnaire(OQAQ), the only instrument available to researchers to grade the quality of articles. The reports were given an overall score on a scale of 1-to 7 according to the results of this assessment, with a score of 1 to 4 indicating major flaws in the meta-analysis and a score of 5 or more indicating only minimal or minor flaws.
Delaney et al.'s results show that 69% of the reports score below 4 and the average score was 3.3. The most common flaws were failure to report whether a comprehensive literature search was conducted and failure to report how bias in the inclusion of studies was avoided, with only 35% of studies fulfilling these criteria.
Before the QUOROM statement, the proportion of reports adequately reporting the validity of the studies included in the meta-analysis was 39%, but it increased to 52% after publication of the statement.
Delaney et al. found similar results in other areas such as emergency medicine, anaesthesia and general surgery.
Physicians view meta-analyses as a highly reliable resource. This study shows that critical care clinicians should carefully evaluate meta-analyses before considering applying the results of these studies in their clinical practice.
Article:
A Systematic Evaluation of the Quality of Meta-analyses in the Critical Care Literature.
Anthony Delaney, Sean M Bagshaw, Andre Ferland, Braden Manns, Kevin B Laupland, Christopher J Doig
Critical Care, 2005 9:R575-R582 (9 September 2005)
Journal
Critical Care