MEDEV

School of Medical Sciences Education Development

Introduction

This workshop was the third in a series designed by Kathy Boursicot and Trudie Roberts for the Subject Centre for Medicine, Dentistry and Veterinary Medicine.The first workshop was designed to disseminate good practice in the design and implementation of Objective Structured Clinical Examinations (OSCEs) for the assessment of clinical skills.The second was a practical guide to enhance the consistency of marking in OSCEs by training examiners.The feedback from these four workshops (each was conducted twice) indicated a desire in the academic community for some practical guidance on setting standards in examinations.

With the current climate of accountability and transparency requirements of standards of competence in medicine and other professional course, the setting of passing standards (the pass mark or cut score) in examinations has become a crucial issue, both for internal and external quality assurance purposes.

We designed this workshop for people new to standard setting, who want an introduction to the academic principles underlying different contemporary best practice methods. This workshop was developed to distil the enormous quantity of published work on standard setting into practical methodologies.

Participants working in groups had the opportunity to apply three different standard setting methods to genuine examination material, and then review the outcomes of the different methods. This experiential workshop provided participants with the reality of applying standard setting processes, involving scrutiny of examination material and using their professional experience, to make judgements about standards.

Participants from medicine, dentistry, veterinary medicine, nursing, midwifery, veterinary nursing and examination management came from across the UK, and spanned undergraduate and postgraduate sectors. The participants were organised into multiprofessional groups so that they could share commonalities and explore differences across professional education. At the end of the workshop, the strategies and practicalities of introducing formal standard setting methodologies for use in people's home institutions were discussed.

Both workshops were conducted at St Bartholomew's Hospital in London. 47 particpants have attended to date and two more have been arranged (27 September and 30 November 2006), to accommodate the waiting list.

Workshop Learning Objectives for Participants

• Develop the ability to critically analyse standard setting challenges for institutional assessments
• Gain a working knowledge of the principles of standard setting
• Understand the differences between norm and criterion-based standard setting methods
• Acquire practical experience of 3 different standard setting methods
• Develop the ability to critically analyse the defensibility and outcomes for a particular standard setting activity
• Select appropriate standard setting methods for different situations
• Plan strategies for the introduction standard setting in their own institutions

Benefits for the Higher Education Academy Subject Centre for Medicine, Dentistry and Veterinary Medicine and the disciplinary community

• Support for the mission of Subject Centres, in particular, to promote and facilitate faculty and professional development
• Dissemination of good practice in standard setting for undergraduate assessment
• A common framework for meeting QAA and professional body requirements in relation to standard setting