Why are MBChB teaching staff reluctant to give negative feedback to students or fail low-achieving students? A qualitative study to inform the development of a theory-based questionnaire and staff training
N.B. The information below is authored by the mini-project applicants, not by staff of MEDEV. This text represents the views and opinions of the mini-project team only, not those of MEDEV or its affiliates.
Principal investigator
Jennifer Cleland,Full list of project partners
Dr Jennifer Cleland 1
Dr Charlotte Rees 2
Dr Lynn Knight 2
Dr Sue Tracey 1
Professor Christine Bond 1
Professor Malcolm Laing 3
1 Department of General Practice and Primary Care, University of Aberdeen, Aberdeen
2 Peninsula Medical School, Plymouth
3 Associate Dean, Inverness Campus, University of Aberdeen
Background
We cannot rely on low-achieving students accurately identifying their own learning needs. These students regard themselves as competent even after they have had significantly greater problems with summative assessments than their year mean (Cleland et al., Med Teach. 2005; 27: 504-8); low-achieving students are unable to accurately self-assess the quality of their own work (Langendyk. Med Educ., 2006 40, 173-179); or their personal and professional behaviours (Rees & Shepherd. Med. Educ. 2005; 20: 30-39); and weak students are less interested in feedback than better students (Sinclair & Cleland, submitted to Med. Educ.).
Thus, the onus is on staff to identify which students should not progress without support. Not reporting poor students means these students are likely to ultimately fail, or become weak or incompetent doctors (Challis, Fleet & Batstone, G. Med. Teach. 1999; 21: 582-585). Conversely, early identification of poor performance, coupled with intervention (e.g., Sayer et al. Med Ed, 2002, 36, 643-), may help avoid or minimize such situations. However, although ratings for clinical performance account for a large proportion of summative grade in medicine (Kassebaum & Eaglen. Acad. Med. 1999; 74: 842-849), the reliability and validity of this form of evaluation has been questioned. Staff seem unwilling to fail poor students: clinical evaluations are not always consistent with the evaluator's judgement of the student's performance (e.g., Hatala & Norma. Acad. Med. 1999; 74: S118-S120). Concerns are often not recorded formally, or addressed (Tooth et al. 1989. Med. Educ. 1989; 23, 416-421) and very often it is the same medical students who perform poorly as their training progresses (e.g., Sayer et al. Med Ed, 2002, 36, 643-).
Clinical teachers rate unwillingness to record negative evaluations as the single most important problem with evaluation (Cohen, Henry & Dodd. Med. Teach. 1990; 12: 265-272). Dudek et al. (Acad. Med. 2005; 80: S84-S87) identified four reasons why clinical supervisors do not fail poor students: lack of documentation, lack of knowledge as to what to document, anticipating an appeal process, and lack of remediation options. Lack of confidence in their assessment ability is another possible factor (Albanese. Acad. Med. 1999; 74: 652-658). Although use of the MiniCEX is gaining popularity for placement assessments (Holmboe et al. Acad. Med 2003; 78: 826-830), it is likely that faculty ratings of student clinical performance will remain a key component of assessment of clinical assessment while on placements in the foreseeable future.
To the best of our knowledge there have been no previous investigations of staff reluctance to give negative evaluations of UK undergraduate medical student performance when on hospital or GP clinical attachments, or placements. In the current climate, it is essential that we explore factors impacting on supervisors' willingness to record negative evaluations, with the ultimate aim of increase the validity of undergraduate medical student placement ratings.
Proposed activities
Method
A literature review has identified that only a few studies, and no UK studies, have explored why medical faculty staff are unwilling to fail medical students, or provide negative feedback. What are the beliefs and attitudes of staff towards these issues, and how can we support them in appropriate evaluation?
This is a Phase I proposal (http://www.mrc.ac.uk/prn/pdf-mrc_cpr.pdf) which aims to delineate the factors impacting on staff decision making and how different factors inter-relate. Qualitative methodology (focus groups) will be used to collect data. The interaction they facilitate is important in drawing forth data on relevant issues and exposing divergence. They are recognised as being a valuable method for investigating what people think and why (Kitzinger. BMJ. 1995; 311: 299-302).
Separate focus groups will be carried out with community-based teaching staff, hospital-based teaching staff and Faculty staff (involved with MBChB management). Six focus groups, three per school, will be carried out in convenient locations, with 6-10 people in each group. Participants in each group will be selected purposively in order to reflect broad areas of clinical practice, level of teaching responsibility, etc, so that all opinions perhaps associated with these variables can be exposed. This data will allow us to examine if different people have different beliefs and attitudes about failing students/providing negative feedback.
A semi-structured interview schedule will be developed for structuring the discussion within the focus groups, to collect data on:
· teaching staff intentions to fail students or give negative feedback,
· their attitudes towards doing so,
· the subjective norms associated with these behaviours (perceived social pressures to undertake the behaviour or not),
· their perceived control over these behaviours (how easy or difficult this will be), and
· staff knowledge of procedures for failing students.
The first four of these factors map onto the theory of planned behaviour model (TPB: Ajzen. Org. Beh. Human Dec. Proc.. 1991; 50: 179-211). A recent meta-analysis of TPB studies indicate that it can account for around 27% of the variance in peoples' behaviour, and 39% of intentions (Armitage & Conner. Brit. J. Soc Psych. 2001 ; 40 : 471-499).
Data will also be collected on process-related themes such as humour, question-asking, interruption, dramatic devices such as metaphor, etc, to give a richer picture of the topic.
Analysis
The focus groups will be audio-taped with consent, and transcribed verbatim for analysis. Framework analysis (Richie & Spencer. 1994. In Bryman and Burgess (eds) Analysing qualitative data London: Routledge) will be used as the questions are specific and explicit, our sample is pre-designed, and our primary concern is with description and interpretation of what is happening in a particular setting. Data from two focus groups will be analysed independently by a researcher and project partners. Any discrepancies in coding will be discussed and consensus reached to generate a coding structure, then used to code the entire data set.
Process
The applicants will ensure progress via regular tele-conferencing meetings. Focus groups will be carried out by experienced researchers in Aberdeen and Peninsula.
Proposed outcomes
Objectives
· To explore, via focus groups, the reasons why undergraduate teaching staff may be reluctant to (1) fail low-achieving medical students, and (2) give negative feedback
· To investigate, in the same focus groups, what interventions teaching staff feel might make (1) and (2) easier
· To explore the above in two purposively-sampled UK medical schools: Aberdeen (an established medical school, Scotland) and Penninsula (a new medical school, South-West England).
Anticipated outcomes
· A paper suitable for publication in a peer-reviewed medical education journal
· A presentation suitable for a peer-reviewed medical education conference
· The collection of data for the basis of a theory of planned behaviour (TPB) questionnaire (see below).
Benefits
This work progresses a programme of research at the University of Aberdeen looking at factors influencing academic achievement in the MBChB. JC received LTSN/HEA funding for the study "Failing finals is often a surprise for the student but not the teacher: identifying difficulties and supporting students with academic difficulties" (Final report submitted in 2003, paper published in Med. Teach. 2005; 27: 504-508). An associated study, looking at the impact of debt on undergraduate medical student performance and stress is currently in press (Medical Teacher). This study advances this programme of research by exploring factors influencing tutor behaviour in regard to failing, or not failing, low-achieving students.
Disciplinary community
· Collaboration across two medical schools is key to this proposal.
· Comparison between Aberdeen and Peninsula may be particularly informative as there are some key differences across curricula, which may influence responses to our research questions (e.g., Clinical Skills training starts in Y1 at Peninsula, Y2 at Aberdeen; basic science is integrated with clinical teaching at Peninsula whereas at Aberdeen basic science is covered in Y1 then clinical skills introduced in Y2).
· Exploring the research questions in a purposive sample of UK medical schools facilitates the comprehensive collection of data: findings are more likely to be pertinent to all UK medical schools.
· We intend to use the data collected as the basis for items in a questionnaire based on the theory of planned behaviour (TPB: Ajzen. Org. Beh. Human Dec. Proc.. 1991; 50: 179-211) to inform a needs-based training programme. The next stage in this project would involve seeking separate funding to pilot the TPB questionnaire and carry out factor analysis and test-retest reliability with appropriate sample sizes (Guadagnoli & Velicer. Psychol. Bull. 1988; 103: 265-275; Francis et al. 2004. Constructing questionnaires based on the theory of planned behaviour: A manual for health services researchers. http://www.rebeqi.org/?pageID=34&ItemID=72])
· If analysis indicates that the questionnaire is robust, the final stage of the proposed programme of research would be an electronic questionnaire survey of UK medical faculty staff in a representative sample of UK medical schools. This will enable collection and comparison of national data.
This project involves MBChB teaching staff only. Ethical permission for the study will be sought.
Proposed expertise
Dr Jennifer Cleland is a Clinical Lecturer, School of Medicine, University of Aberdeen. She is involved in overall management of the MBChB, and academic lead for clinical communication. She collaborates locally and nationally on medical education research and developing evidence-based teaching. She is PI on an Asthma UK £172, 000 project, and collaborates with colleagues on several other patient and professional education/training projects. She has published and presented papers using qualitative and quantitative research methodology in journals including BJGP, Medical Education and Medical Teacher, and was recently involved in a project involving TPB questionnaire development and application (in submission).
Dr Charlotte Rees is a Senior Lecturer in Medical Education, Peninsula Medical School, University of Exeter. She was the founding academic lead for human sciences, communication skills and PPD at Peninsula Medical School. Her current roles include medical education evaluation and research. As PI, she has just successfully completed a British Academy funded study with Dr Knight examining service user involvement in medical education (Rees et al. in press; Wilkinson et al. under review; Knight & Rees in preparation). This project employed focus group methodology, framework and process analysis and was underpinned by psychological theories of behavioural explanation. Her current writing has employed the TPB to help explain medical students' and doctors' professional and unprofessional behaviours (Rees & Knight under review).
Dr Lynn Knight is a Lecturer in Clinical Education, Peninsula Medical School, Universities of Exeter and Plymouth. Past and present research projects include investigating clinicians' perceptions of clinical teaching, exploration of user involvement in medical education, utilising qualitative methodology. Findings from these projects are currently in press in journals such as Social Science and Medicine, Medical Humanities and Advances in Health Science Education. She holds two grants totalling £15,599 from the Nuffield Foundation and Support for Science. Lynn has further papers involving the TPB with Dr Charlotte Rees under review.
Dr Sue Tracey is a Clinical Lecturer, Department of General Practice and Primary Care, Teaching Co-ordinator, Aberdeen and Inverness, and a GP, Inverness. Her research interest is undergraduate medical education, specifically curriculum development and supporting low-achieving students.
Professor Christine Bond is Head of General Practice Teaching in the Department of General Practice and Primary Care, University of Aberdeen. She is currently involved in a project developing and evaluating inter-professional learning. As well as her expertise as an educationalist, She is also an established primary care Health Services Researcher with well over a 100 peer reviewed publications. She has wide experience of both qualitative and quantitative methods, including the application of TBP to predict and influence professional behaviours.
Professor Malcolm Laing is Associate Dean in Medical Education for the University of Aberdeen's Inverness campus. He has particular interests in curriculum development and formative assessment. He has a number of previous and current collaborations on the theme of failing students, remediation tools and the role of formative assessment. His current main interest is in the creation of an appropriate comprehensive support structure for medical students experiencing academic, personal or health-related difficulties.
Expertise of grant holder and project team
Dr Jennifer Cleland is a Clinical Lecturer, School of Medicine, University of Aberdeen. She is involved in overall management of the MBChB, and academic lead for clinical communication. She collaborates locally and nationally on medical education research and developing evidence-based teaching. She is PI on an Asthma UK £172, 000 project, and collaborates with colleagues on several other patient and professional education/training projects. She has published and presented papers using qualitative and quantitative research methodology in journals including BJGP, Medical Education and Medical Teacher, and was recently involved in a project involving TPB questionnaire development and application (in submission).
Dr Charlotte Rees is a Senior Lecturer in Medical Education, Peninsula Medical School, University of Exeter. She was the founding academic lead for human sciences, communication skills and PPD at Peninsula Medical School. Her current roles include medical education evaluation and research. As PI, she has just successfully completed a British Academy funded study with Dr Knight examining service user involvement in medical education (Rees et al. in press; Wilkinson et al. under review; Knight & Rees in preparation). This project employed focus group methodology, framework and process analysis and was underpinned by psychological theories of behavioural explanation. Her current writing has employed the TPB to help explain medical students' and doctors' professional and unprofessional behaviours (Rees & Knight under review).
Dr Lynn Knight is a Lecturer in Clinical Education, Peninsula Medical School, Universities of Exeter and Plymouth. Past and present research projects include investigating clinicians' perceptions of clinical teaching, exploration of user involvement in medical education, utilising qualitative methodology. Findings from these projects are currently in press in journals such as Social Science and Medicine, Medical Humanities and Advances in Health Science Education. She holds two grants totalling £15,599 from the Nuffield Foundation and Support for Science. Lynn has further papers involving the TPB with Dr Charlotte Rees under review.
Dr Sue Tracey is a Clinical Lecturer, Department of General Practice and Primary Care, Teaching Co-ordinator, Aberdeen and Inverness, and a GP, Inverness. Her research interest is undergraduate medical education, specifically curriculum development and supporting low-achieving students.
Professor Christine Bond is Head of General Practice Teaching in the Department of General Practice and Primary Care, University of Aberdeen. She is currently involved in a project developing and evaluating inter-professional learning. As well as her expertise as an educationalist, She is also an established primary care Health Services Researcher with well over a 100 peer reviewed publications. She has wide experience of both qualitative and quantitative methods, including the application of TBP to predict and influence professional behaviours.
Professor Malcolm Laing is Associate Dean in Medical Education for the University of Aberdeen's Inverness campus. He has particular interests in curriculum development and formative assessment. He has a number of previous and current collaborations on the theme of failing students, remediation tools and the role of formative assessment. His current main interest is in the creation of an appropriate comprehensive support structure for medical students experiencing academic, personal or health-related difficulties.
Similar work
To the best of our knowledge, no other projects exist in this area.
Contact details
Amount awarded: 5,000
MEDEV project contact: Gillian Brown
Reports and resources
