Without accessible and usable reports, research cannot help patients or their doctors. In a Viewpoint published Online First and in an upcoming edition of The Lancet, Sir Iain Chalmers, James Lind Library, Oxford, UK, and Professor Paul Glasziou, University of Oxford, UK, discuss the avoidable waste that occurs in medical research—which leads to many reports not fit for purpose.
The authors look at the causes and degree of waste occurring at four successive stages: the choice of research questions; the quality of research design and methods; the adequacy of publication practices; and the quality of research reports.
They discuss how for choice of research questions, public funding of research is correlated only modestly with disease burden, if at all. In an analysis of 344 studies, only nine compared researchers' priorities with those of patients or doctors—and these nine studies revealed dramatic mismatches. For example, priorities of patients with osteoarthritis of the knee and the clinicians looking after them favoured more rigorous evaluation of physiotherapy and surgery, and assessment of educational and coping strategies. Only 9% of patients wanted more research on drugs, yet over 80% of randomised controlled trials in patients with osteoarthritis of the knee were drug evaluations. And this pattern is not restricted to commercial research—controlled trials funded by the UK Medical Research Council and British medical research charities were much more likely to be drug trials than were trials commissioned by what is now the National Institute for Health Research (NIHR).
In terms of quality of research design, the authors say: "New research should not be done unless, at the time it is initiated, the questions it proposes to address cannot be answered satisfactorily with existing evidence." New research is often inadequate as it fails to address other fundamental issues. For example, in a sample of 234 clinical trials in the major medical journals, concealment of treatment allocation sequence was either inadequate (18%) or unclear (26%).
Much discussion in recent years has centred on the inadequacy of publishing practices, namely biased under-publication and over-publication of research. The authors say the public has become aware of this issue thanks to exposure of suppressed evidence about serious adverse effects of treatments. Also, studies with disappointing results are less likely to be reported promptly, and are less likely to proceed from abstracts to full reports. The authors say: "The problem of biased under-reporting of research results mainly from decisions taken by research sponsors and researchers, not from journal editors rejecting submitted reports." They add that public access to full results of all research remains an aspiration, and one that is resisted by some research sponsors and researchers.
Finally, biased or unusable reports of research can be common. Some reports do not include adverse events, or change their primary outcomes between the protocol for the research and the published report. But even in the absence of bias, a report must still be appropriate for use by the end-user. The authors discuss two possible ways to improve studies. First, if clinicians are to use evidence showing beneficial effects of an intervention , the intervention needs to be adequately described—especially when it is not a drug—eg, setting up a stroke unit or details of a low fat diet. Second, evidence must be set in the context of updated systematic reviews. Yet among the world's major medical journals, The Lancet is alone in requiring reports of new research to be preceded by and to conclude with reference to systematic reviews of other relevant evidence.
The authors say: "Since research must pass through all four stages...the waste is cumulative. If the losses estimated in the figure apply more generally, then the roughly 50% loss at stages 2, 3, and 4 would lead to a greater than 85% loss, which implies that the dividends from tens of billions of dollars of investment in research are lost every year because of correctable problems."
They conclude with a number of recommendations for each of the four areas*, including: increasing involvement of clinicians and patients in shaping the research agenda; that all journals require new studies to be set in the context of existing published research; support of timely open access to full results on completion of research; supplementing peer review of studies with review by methodologists and end users; and supporting free access repositories—separate from any publications—so that clinicians and researchers have details of the treatments, test, or instruments studied.
Sir Iain Chalmers, James Lind Library, Oxford, UK T) +44 (0) 1865 517636 E) ichalmers@jameslindlibrary.org
For full Viewpoint see: http://press.thelancet.com/waste.pdf
Note to editors: *For full recommendations, see panel, page 3 of full Viewpoint
Journal
The Lancet