Mini project update: BEME systematic review & effectiveness of self-assessment in clinical education
Introduction
Self-assessment is being increasingly relied upon as a formative tool both in undergraduate and postgraduate clinical education. Yet it is a skill that is seldom tested and is rarely taught.
The ability to accurately self-assess our own learning needs is fundamental to self-directed lifelong learning and continued professional competence. Though the cornerstone of most appraisal systems and a frequently used means of evaluating learning interventions, the impact of self-assessment on learning behaviour or clinical practice is not clearly understood. This is despite a review of self-assessment concluding that self-assessment skills remain underdeveloped during health professions training.
Validity
In reviewing the validity of selfassessment Gordon (1) found that only 13 out of over 1000 articles focusing on self-assessment had explored methods to improve this skill. On further review these 13 studies2 were found to be methodologically weak, often had small samples in restricted clinical situations or evaluated methods which had been designed or advocated by the authors themselves.
We aim to review the work of Gordon and also to see if this evidence base has changed since 1991. Our Best Evidence Medical Education (3) (BEME) review group has eight members from Edinburgh, Newcastle and Leeds Universities and NHS Education for Scotland.
Our review team
Brian McKinstry, Jan Illing, Iain Colthart, Alex Haig, Alison Evans, Gellisse Bagnall, Heather Peacock, Helen Allbutt, Rachel Adams and Susan Hrisos.
Methods
The EPOC register, Medline and CINAHL were searched using a standard search strategy for studies published from 1991 onwards. To include all significant grey literature, hand-searching was undertaken of conference proceedings, literature held by leading authors in the field and relevant but unreliably indexed journals. Studies meeting the inclusion criteria were retrieved and their abstracts considered for full paper review by pairs of reviewers (below).
Full text versions of included papers will be reviewed by two reviewers with data abstracted into an electronic, web-based coding sheet designed by the group for the review. Data synthesis will be guided by the nature of the papers included in the review. The revision is expected to be descriptive as we did not anticipate retrieving sufficient data for standard multivariate analyses.
Progress
With the support of the Subject Centre the review group has met on eight occasions to date. Our early meetings were largely taken up with agreeing the scope of the review, reaching a group consensus on the review definition of self-assessment and ensuring group clarity of the review research question.
We have spent considerable time in the careful development of a robust coding sheet, which we each have piloted aginst ten papers and refined as necessary. The search procedures indentified 194 papers and the abstracts of these studies were screened against the inclusion criteria for full paper review. 120 (62%) papers were identified for full text reviews.
Dissemination
We expect to complete analysis of the data by winter and aim to produce a full report of the group’s findings.
Our aim is to publish the full report on the BEME website followed by further publications in peer reviewed journals. Leading up to publication, the review will be presented at the ASME and AMEE conferences.
Significance of the review
Self assessment is the cornerstone of current continuous professional development and appraisal programmes. It is essential that the tools we use to perform this function are effective. We would hope that the results of this review will inform a wider audience about existing successful methods or tools and to identify research needs in this area.
For more information: brian.mckinstry@ed.ac.uk
References
