Is peer review biased? Female health researchers who applied for grants from Canada's major health research funder were funded less often than male counterparts because of potential bias, and characteristics of peer reviewers can also affect the result, found a study in CMAJ (Canadian Medical Association Journal).
Applicants who had not been previously funded also received lower scores, making them less likely to be funded.
Between 2012 and 2014, 11 624 applications were submitted to the Canadian Institutes of Health Research (CIHR) open operating grant competitions. Two-thirds (66%) of applicants were male and 69% were aged 40 years or older. Almost two-thirds of applications (64%) were in basic science, with the remainder from applied science (16.6% clinical, 8.1% health services and 11.3% in population health).
The study, by researchers from CIHR and McGill University, looking at reviewer characteristics, including gender, previous success rates with grants, experience, scientific domain, conflict of interest and more, found that these traits did introduce bias into peer review of grant applications. This bias resulted in lower scores that could place the application in the non-fundable range.
CIHR's annual investment in health research is about $1 billion a year as of 2018.
Previous studies have found inconsistent evidence of bias, but few studies have analyzed whether reviewer characteristics could potentially bias applications.
"This study confirmed many of the suspected biases in the peer review of operating grant applications and identified important characteristics of peer reviewers that must be considered in application assignment," writes Dr. Robyn Tamblyn, Scientific Director, CIHR -- Institute of Health Services and Policy Research, and a senior scientist at the Research Institute of the McGill University Health Centre, Montreal, Quebec. "By measuring and controlling for scientific excellence of the applicant, we were able to examine how applicant, application and reviewer characteristics may unduly influence the assessment of operating grant applications."
The researchers also found that reviewer expertise influenced the application rating, as reviewers with high expertise rated previously successful applicants higher than less experienced applicants.
"We found lower scores for applied science applications, gender inequities in application scores that favoured male applicants who had past funding success rates equivalent to female applicants, particularly in the applied sciences," write the authors. "Conflicts on the panel, male reviewers only, reviewers with all high expertise, and those whose own research was exclusively in the same scientific domain as the applicant's conferred positive benefits in application rating."
They suggest that training of reviewers, policy change and monitoring may help address these biases.
"These findings are important, as securing less funding slows career progression for women and reduces opportunities for publishing and other forms of collaboration, which are criteria for professional advancement," writes Rosemary Morgan, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, in a related commentary http://www.cmaj.ca/lookup/doi/10.1503/cmaj.180188 with coauthors. "To understand why this occurs, we must recognize that gender bias within the grant review process is a manifestation of historical and systemic gender bias within academia."
###
The study was funded by the Canadian Institutes of Health Research.
"Assessment of potential bias in research grant peer review in Canada" is published April 23, 2018.
Journal
Canadian Medical Association Journal