After a steady increase between 1985 and 1997, the overall performance of postgraduate students taking the Part 1 examination of the Membership of the Royal Colleges of Physicians of the United Kingdom, MRCP (UK), which is a part of higher specialist training, showed a decline from 1997 until 2001. Performance on specific exam questions that were repeated across the years was 14.1% lower in 2001 than in 1996.
"There has been a decline in the performance of graduates from UK medical schools taking the MRCP(UK) Part 1 examination. The reasons for this are not clear, but the finding has implications for medical education, and further studies are needed of performance in other postgraduate and undergraduate examinations" write Professor Chris McManus from University College London, UK, and colleagues from the Royal Colleges of Physicians, the authors of the study.
"The role of postgraduate medical examinations is to set standards of practice and thus to assure the public and the medical profession that doctors have the knowledge and expertise required to diagnose and treat patients, and to progress in their medical careers", explain the authors.
McManus et al assessed the performance of students on 'marker questions' – questions that were repeated in different MRCP (UK) examinations held between 1985 and 2001, and which allow performance to be compared from one year to another. Overall, the ability of examination candidates increased from 1985 to 1997, after which it declined sharply: on a graphical representation of the results, the curve has a clear 'dog-leg' shape, which rises until 1997 and then drops sharply in the subsequent years.
In the second part of the study, the researchers compared the performance in an examination that took place in 1996 and another one that was held in 2001. The results show that fewer candidates who passed the examination in 2001 gave correct answers than had equivalents in 1996 and the candidates' mean score decreased from 69.4 in 1996 to 59.6 in 2001.
"The 'dog-leg' in performance [in] 1997 was not an artefact of changed Examination Regulations, mix of UK and overseas candidates, or time from qualification until taking the Examination", write the authors. "Study 2 confirmed that performance in 2001 was significantly worse than in 1996, that the poorer performance was found in graduates of UK medical schools, and that candidates passing the Examination in 2001 performed less well than those passing in 1996."
Reliance in the past on 'norm-referencing' for setting the pass mark, when a fixed proportion of candidates was allowed to pass at each examination, meant that candidates passing the examination at the end of this period had a somewhat lower standard. Norm referencing was abandoned by the MRCP(UK) in 2002 in favour of 'criterion-referencing', in which examiners determine the pass mark by assessing the desirable knowledge based on individual questions. Thus the standard required to pass all components of the examination is now fixed and independent of the standard of the candidates sitting at that diet.
The 'dog-leg' shaped graph can be viewed at this URL: http://www.biomedcentral.com/manuscript/login/afl.asp?txt_afl_filename=C5958/1669550796734273_FIG_2.JPG
This press release is based on the article:
Changes in standard of candidates taking the MRCP(UK) Part 1 examination, 1985 to 2002: Analysis of marker questions
I C McManus, J Mollon, O L Duke, J A Vale
BMC Medicine 2005 3:13 (18 July 2005)
This article is available free of charge, according to BMC Medicine's Open Access policy at: http://www.biomedcentral.com/1741-7015/3/13/abstract
Journal
BMC Medicine