MEDEV

School of Medical Sciences Education Development

Introduction

Following a successful application to the FDTL4 funding program, the UMAP project began as a consortium of five medical schools (Manchester, Leeds, Liverpool, Sheffield and Newcastle) in January 2003. Here Andrea brings us up to date on progress so far.

The generation of a robust bank of valid and reliable MCQ and EMQ questions written in accordance with a pre-determined inter-curricular matrix remains the principal aim of the project and this has required heavy resource investment.

Resources have been allocated to develop an effective workshop format based on pilot workshops and the advice of external consultants.

This format has been implemented across the five consortium sites with great success, and has, to date,generated 1200 assessment items.

Quality assurance

Although far less visible, the embedding and quality assurance aspects of the project remain the most significant, and at times the most difficult, of the project aims. Rather than imposing a process in each medical school, the project has chosen to merge forces with existing examination preparation structures at each site.

By working closely with members of the examination preparation teams in Manchester to produce their January 2004 examination, the project has been able to develop strategies in liaison with the remaining four schools for the roll out of equivalent systems in time for Summer 2004.

Current activity

As UMAP moves towards inviting new partner schools to join the consortium, it is essential that the project uses its existing experience in quality assurance and embedding to offer a high quality service to future partners.

We will continue to monitor the quality of assessment items on three levels:

• At the item writing level, each of our workshops are high in education content and guide attendees through principles of question construction.
• During question review panel and exam preparation stages, all contributors are required to have an awareness of these same principles by which the questions are written.
• Data generated by student performance in high stakes examinations based on UMAP questions is collated and analysed in order to provide information on question performance.

As this information builds, it becomes possible to select questions on the basis of their predicted performance in further examinations.

Extending the partnership

Liaison with medical schools across the UK has been underway for some time. Expressions of interest have been invited and substantial discussion has taken place between founder schools on how to best progress partnership extension plans. The project has also run a series of six regional question writing workshops to disseminate and discuss practice.

The primary basis for partnership will be question quality. The success of the project depends entirely on the availability of a quality assured bank of questions. To ensure the longevity of the project, it will be a requirement of partnership that each school should make a contribution commitment.

This contribution would be on two levels, in order to conform with the aims set out above:

• Regular question contributions would be required, most likely via an educational workshop format derived in whole or part from the UMAP format.
• A commitment would be required from each new partner to develop embedding and review strategies embracing the principles central to UMAP’s success.

The Future

At the end of 2005, when the funding for the project comes to an end, it is intended that the question bank will continue. Resourcing its continuation is a topic under discussion.

In oder to discuss these points, and many further practicalities we held an open meeting with interested partners in late 2004. This meeting took place on the afternoon of October 14th 2004 at the Radisson SAS Hotel, Manchester Airport.