N01.8 The newsletter of the Higher Education Academy Subject Centre for Medicine, Dentistry and Veterinary Medicine, July 2005

Issues and news on learning and teaching in medicine, dentistry and veterinary medicine

Print: ISSN 1740-8768

Online: ISSN 1479-523X

In this issue

• CETL focus: four CETLs outline their plans
• Project update: WILeN, progress files in dentistry
• 6 mini-project reports
• The effect of class on psychometric assessment, and RLOs in veterinary medicine

Welcome!

Welcome to the eighth edition of 01, the newsletter of the Subject Centre for Medicine, Dentistry and Veterinary Medicine.We hope that you like our new print which was changed in April to reflect the Academy branding.

We are still working on changes to the website, at the new URL www.medev.ac.uk We are devastated to report some staffing changes in the Subject Centre.We’d like to thank Ross Hobson, Paul Hollands and Gemma Robinson for all their hard work, and wish them every success in the future.

Thank you also to everyone who submitted articles for the newsletter, we have had to hold some of your articles over to the next edition where there will also be a special focus on veterinary education.

Thank you for your patience and, as always, we welcome feedback!

Megan Quentin-Baxter

Contents

• Problem based learning: a system or just a method?, Prof Peter Dangerfield, University of Liverpool and Prof Charles Engel, Institute of Education
• Mini project update: Enhancing clinical learning, R. Graham Chadwick, University of Dundee
• Hull York Medical School: The first two years, Dr Jean McKendree, Hull York Medical School
• Mini project update: Objects for OSCEs, Vicki Dale, University of Glasgow and Gill McConnell, University of Edinburgh
• Workshop report: Examiner training for OSCEs, Dr Katharine Boursicot, Barts and The London, Queen Mary’s School of Medicine and Dentistry and University of Cambridge, and Prof Trudie Roberts, University of Leeds Medical School
• Workshop report: Interprofessional assessment, Marian Traynor, Queen’s University Belfast
• CETL focus: Assessment and learning in practice settings, Prof Trudie Roberts, University of Leeds Medical School
• CETL focus: Enquiry based learning (EBL), Prof Paul O’Neill, University of Manchester
• CETL focus: Healthcare professional education, Dr Pauline Pearson, University of Newcastle
• CETL focus: Developing professionalism in medical students, Prof Anne Garden, University of Liverpool
• Project focus: A progress file learning system, Prof Liz Davenport, Barts and The London, Queen Mary’s School of Medicine and Dentistry
• FDTL4 update: Web-based interprofessional learning, Dr Giuseppe Cannavina, University of Sheffield
• The learning styles of medical school entrants, Vikki Haley and Claire Smith, University of Southampton
• Mini project update: The effect of social class on psychometric assessments, Prof Mary Ann Lumsden, University of Glasgow
• Workshop report: Supporting students who struggle, Dr Jo Brown and Dr Dason Evans, Barts and The London, Queen Mary’s School of Medicine and Dentistry
• Workshop report: Formal appraisal of undergraduates, Prof Deborah Murdoch-Eaton, University of Leeds
• Mini project update: The effect of reflection on undergraduate learning, Dr Andy Grant, Cardiff University
• Dental students and nurses: Investigating gender, differences in communication and working styles, Prof Ruth Freeman, Queen’s University, Belfast
• Subject Centre workshop programme 2005
• Funding opportunities/Upcoming events

Problem-based learning: a system or just a method?

Peter Dangerfield and Charles Engel report on a Subject Centre funded workshop that explored the elements which would constitute a coherent system of professional education, based on the principles which underpin PBL.

Professor Peter Dangerfield, Faculty of Medicine, University of Liverpool and Professor Charles Engel, School of Lifelong Learning and International Development, Institute of Education, University of London

We have all read about the virtues – and reservations – of evidence–based clinical practice.

We have also been encouraged to practise evidence–based education in our respective disciplines. You will be interested to read ‘More nearly certain’1 where Evans writes most elegantly about certainty of medical interventions in the Year 2055:

You will also have put uncertainty on the pedestal where it belongs – as an exciting and necessary stimulus to medical progress, to be exploited for its true worth, not feared.

If uncertainty of the effectiveness of clinical intervention will continue, so surely would educational interventions in our respective disciplines.

Meta reviews

However, how scientific are we in the rigour of reporting our research in relation to educational interventions?

Do papers on, for example, Problem-Based Learning (PBL) offer the reader sufficient detail that would make it possible to replicate that interventional programme?

Unfortunately, the papers which are examined in meta reviews of the literature hardly ever provide adequate information on who managed the intervention, the nature of the intervention, for which audience, under which conditions, in order to achieve which specific outcomes2.

Even when the experimental process has been made explicit, the actual use of what is claimed to be PBL has not.

The danger is that what is claimed to be PBL may contribute to consigning the potentially valuable approach to oblivion as just another passing fashion.

Criteria search

We thought, therefore, that it would be worthwhile to undertake an exploration of the elements which would constitute a coherent system of professional education which is based on the principles which underpin PBL.

The hope is that this would yield a set of criteria which curriculum planners, researchers, authors and their publishers could apply productively. With this aim in mind we invited educators in medicine, dentistry and veterinary science to join us in a workshop.

Their experience in a consistent application of PBL would help us to begin to explore the components of a coherent system of education, based on PBL. The workshop examined responses which had been assembled in response to a set of e-mailed questions (see below).

We revised the responses and added further components, in order to use this assembly of some forty potential components as the basis for a Delphi consultation. Each component and any additional ones would be amplified with an explanatory paragraph, if desirable with the addition of a more detailed paper and occasionally with an illustrative example.

The iterative process of the Delphi consultation aims to enrich this exploration and to attain a general consensus for the construct of a system of education, based on PBL.

Emailed questions

• What should PBL enable students to do, in order to foster their personal and collaborative studies?
• Which abilities and skills does PBL aim to achieve for the students by the end of the students’ exposure to PBL? Here we focused on generic abilities and skills, not on profession-specific abilities and skills.
• In the design of the curriculum what should be included, in order to support the aims and implementation of PBL?
• Which further additional components of a coherent educational system, based on PBL should be explored?

The guide

The final outcome will be a guide to help those who wish to consider the development of a PBL curriculum, those who wish to amend their existing application of PBL, those who undertake research in PBL, and editors of educational journals who wish to be assured by their referees that submitted papers include essential information to support the authors’ conclusions. This guide would naturally be open to discussion and further development. At present the core team, Peter Dangerfield, Charles Engel, Gillian Maudsley and David Taylor, hope that their eighteen foundation collaborators will wish to continue their participation in this initiative.

We intend to invite additional respondents from the three disciplines. We trust that the guide will be of interest to other professions, such as architecture, engineering, health sciences and law, for the development of their own guides. We keep our fingers crossed that several Subject Centres and the Higher Education Academy, as well as other sponsors will be able to support this venture.

This programme may be seen as a case study of a specific approach to the consensual development of higher education and to agreement of explicit educational criteria which may be applicable in bench marking and quality assurance.

For further information please contact spine92@liv.ac.uk or charles.engel@lineone.net

References

1. Evans, I. (2005). More nearly certain, Journals of the Royal Society of Medicine, 98: 195-196
2. Newman, M. (2003). A pilot systematic review and meta analysis on the effectiveness of Problem Based Learning, Subject Centre for Medicine, Dentistry and Veterinary Medicine, ISBN 0701701587
3. Print: ISSN 1740-8768 Online: ISSN 1479-523X

Mini project update: Enhancing clinical learning: an evaluation of a novel approach

A potential solution to the difficulties of clinical teaching in dentistry is currently under evaluation at Dundee Dental School and Hospital. Here Graham Chadwick reports on recent activity.

R Graham Chadwick, Clinical Senior Lecturer and Honorary Consultant in Restorative Dentistry, University of Dundee

It is well known that watching a clinical procedure being undertaken either in real time or upon playback is a valuable teaching aid for both teacher and learner1,2.

Traditionally this has been achieved by either small groups of students observing a qualified dentist treating a patient or the dental teacher watching a student carry out a procedure.

Neither situation is ideal as the confined space of the oral cavity compromises the view. Furthermore it is incredibly labour intensive to watch a student perform an entire operative procedure from start to finish.

Although observing a procedure at predefined interim stages (such as cavity preparation, lining and restoration) is commonly used to optimise the use of staff resources only snapshots of performance are gleaned. Valuable opportunities for learner support are thus missed and the teacher has little idea of how the end point was achieved.

Solution

A potential solution currently under evaluation in Dundee Dental School and Hospital is that of a head mounted video camera system.

This is mounted upon a headband worn by the student or staff operator. Illumination of the operating area is provided by two bright white LED headlamps.

Laser beams

A small red laser beam allows the wearer to target the camera upon the area of work. For portability this is all battery powered and a radiofrequency transmitter, incorporated in the device, permits real time images to be sent to a receiver for display on either a monitor or capture on a recording device for later playback.

This arrangement obviates the need for trailing cables. For all members of the dental team we are currently seeing if this technology:

• Improves the level of detail observed when observing dental procedures being undertaken
• Improves the level of feedback to learners
• Provides better learner support.

Student evaluation

'Excellent teaching aid for students – normally I can’t see what is happening.’

‘The camera allowed me to see what was happening which is near impossible standing by the operator.’

Monitors

Subject to a satisfactory evaluation it is envisaged that in the future the activities of an entire teaching clinic may be viewed by the teachers on a bank of monitors and verbal feedback given by means of a radio earpiece and chairside visits as required. It will also make demonstrations of dental procedures more accessible to a large group of students who all will be able to view it from the operators perspective.

Evaluation

The evaluation process has raised a number of challenges in its design.

A single questionnaire has been formulated that is both suitable for all members of the dental team to complete from the perspectives of either device wearer or viewer.

It is constructed so that the evaluation of the usefulness of this technology is kept separate from suggestions for technical modifications. This is important for our perceptions influence our evaluation of service quality3.

The results of the full evaluation will reveal if wearing the camera, to have their work observed, is acceptable and helpful to the students.

Feedback upon equipment design will be fed back to the development team. This is important for design is an iterative process.

It is intended to hold a Subject Centre funded open day in Dundee towards the end of the year where delegates will have access to the full evaluation and be given the opportunity to try the device for themselves.

Acknowledgements

The ongoing work reported here would not have been possible without the input of Mark Tulley, Patrick Carena and Bryan Hunter of the Medical Physics Department of Ninewells Hospital, Dundee supported also by the engineering students Marco Steinberg and Jan Hoppner of the Technische University of Ilmenau, Germany.

They turned the initial concept into reality. We are also grateful for some financial support for evaluation from the Subject Centre.

For further information contact r.g.chadwick@dundee.ac.uk

References

1. Britto L.R.,Veazey W.S. and Manasse G.R. (2004) Personal video monitor as an accessory to dental operating microscopes. Quintessence Int 35: 151-154
2. Kardash K. and Tessler M.J. (1997) Videotape feedback in teaching laryngoscopy. Can J Anaesth 44: 54-58
3. Chadwick R.G. (2002) Quality assurance by service quality – an industrial approach to monitoring course delivery. British Dental Journal 192: 285-288

Hull York Medical School: The first two years, Dr Jean McKendree, Learning Resources Manager and Lead on Personal and Professional Development, Hull York Medical School

Hull York Medical School (HYMS) is one of the new UK schools, opening its doors for the first students in Autumn 2003. The Medical Education Unit, led by Professor John Cookson, has attempted from the outset to apply an ‘evidence-based’ approach to educational practice, building on the good practice of other UK schools as well as drawing lessons from the literature.

Phase 1 curriculum: years 1 and 2

Like many other medical schools, HYMS has designed an integrated curriculum in which students learn basic sciences, clinical skills, and patient-centred care as a coherent package. The school takes a Problem- Based Learning (PBL) approach encouraging students, under the guidance of facilitators, to develop their skills and confidence for independent and team-based learning.

Some features of HYMS are perhaps more unusual in UK schools. HYMS placements are split 50:50 between community and hospital with students attending placements from the first weeks of the course.

Students begin each week in the ‘Virtual GP Surgery’ where virtual patients, many of whom return several times throughout the year, present scenarios that relate to learning topics identified in the PBL sessions. The students then meet ‘simulated patients’ to practice communication and physical examination skills. At the end of each week, the students attend placements for a half day in Year 1 and a full day in Year 2 to see real patients that tie in with the learning outcomes for the week.

Another feature of HYMS is a commitment to teaching evidencebased medicine and the appropriate management of resources. These subjects form two of the seven explicit themes addressed each week in HYMS:

• Life sciences
• Clinical sciences
• Clinical techniques and skills
• Evidence-based decision-making
• Person-centred care and skills
• Population health and medicine
• Managing resources.

Each theme has a designated ‘guardian’ contributing to the curriculum and assessments to ensure that all are represented and explicitly tied to the learning outcomes.

A final feature that makes HYMS Phase 1 somewhat different is the delivery of the programme on two campuses. HYMS has put a great deal of effort into ensuring that the curriculum is equivalent for the students, who are randomly allocated to the two campuses. We deliver plenaries via videolink and the HYMS Virtual Learning Environment is heavily used for posting information and allowing discussion among students and staff in any location.

Data from the first set of exams indicate no campus difference and no difference among PBL groups. Papers are planned at ASME and elsewhere.

Phase 2 curriculum: years 3 and 4

The next challenge for HYMS is rolling out Phase 2: Years 3 and 4. Students will effectively be full time in clinical placements in hospitals, general practice and the community.

HYMS will maintain the 50:50 split in placements between acute hospitals and primary and community care – an ambition for training never attempted in the UK before. There will still be a single curriculum delivered in a variety of settings with HYMS defining how placements will achieve the specified outcomes.

Currently, Phase 2 has a series of eight eight-week attachments, each followed by an associated student selected component (SSC). One issue is how possible or desirable it is to maintain a specialty focus when half the placement is within primary care, first consider a health profession. The mentoring relationships will continue until the mentees enter HE.

Another initiative, developed by Jo Cross and Andy Kardasz, Curriculum and Assessment Manager, is based on the work of Dr Sarah Pearce at Durham. This project provides year 12 students a taste of health related careers with a problem based case with its rich mix of symptomatology. Hospital practice is becoming ever more specialised, to the benefit of patients, but students need to acquire broad competencies. HYMS’ current solution is to create a ‘partnership’ for each group of students in which two hospital specialists in relevant subjects and two general practitioners form a teaching team.

Widening participation

HYMS was set up with the goal of recruiting more doctors for an underserved region. Thus, we also have an active programme for Widening Participation led by Dr Jane Adam, Associate Dean for Admissions, and Jo Cross, WP Officer. An ‘e-mentoring project’ in association with the Brightside Trust links HYMS with schools across the region. HYMS students receive special training in e-mentoring and provide support via a ‘blog’ for students in year 10, when pupils may study.

It aims to raise aspirations by making clear connections between the school curriculum, HE, and application to the workplace. The project targets AS and AVCE students from colleges who follow a virtual patient case relating to aspects of their science curriculum. This links to a practical session based at HYMS and a hospital visit all tied together via the case study.

HYMS also contributes at both campuses to the Aim Higher Healthcare Strand, hosts open days with tours of the campuses and activities led by HYMS students, participates in the York ‘Science Trail’, and runs sessions in York and Hull HEFCE Summer Schools.

Future plans for HYMS include collaboration with the National Science Learning Centre which is situated right next door in York.

We feel that the first two years at HYMS have been a great success reflected, we believe, in our ranking as number 2 in the Guardian league table as well as in our reviews by the GMC and external examiners. We hope to build on this exciting foundation and look forward to continuing the sharing of ideas and activities with others.

For further information, visit www.hyms.ac.uk or contact jean.mckendree@hyms.ac.uk

Mini project update:Objects for OSCEs, Vicki Dale, University of Glasgow and Gill McConnell, University of Edinburgh

Work on this project began in December 2003 as a pilot study, funded by the Subject Centre, to document the development of a limited number of reusable learning objects (RLOs) for veterinary education, and in particular RLOs targeted at the veterinary benchmark ‘Day 1competencies’ for new graduates, with the potential for use in Objective Structured Clinical Examinations (OSCEs).

During the period of the project, we have consulted with CETIS (www.cetis.ac.uk), and local academic and library staff. An important first step was to obtain permission from authors to re-work existing CAL packages.

Twenty-four RLOs have been created from these packages:

• Anaesthetic Circuits
• Diagnostic Procedures in Dermatology
• General Pathology Test and Tutorial.

Work is continuing to identify the potential uses of these objects, prior to creating more RLOs from other CLIVE CAL programs.

Creating the objects

In our last article (in 01.5), we described how we were making different versions of objects available, for example re-creating parts of Authorware instructional packages as HTML pages.

As previously noted, this overcomes the problem of having to regularly update Authorware plug-ins for browsers, however re-creating an RLO is a time-consuming process.

The time taken to extract new objects from Authorware packages was anywhere between an hour and several days - it takes about an hour to extract a small unit of instruction when all the text and graphics are already embedded in the document, and the extracted unit is kept in Authorware format.

For packages with external text and images, these components have had to be pasted in before the Authorware unit can be published for web delivery. This takes about two and a half hours per object on average. When extracting objects from the original CAL packages, some re-programming is usually needed to maintain functionality within the subunit. Interface design is often improved for both aesthetic and functional reasons. Recreating an RLO as a series of HTML pages generally takes a little longer, if only because new design and presentation issues arise, however the result is easier to search, print and update.

Around fifty objects are now available for use within the Intralibrary repository (see ‘Repositories’ below), which will be uploaded to the Virtual Learning Environments of the two schools – Moodle in Glasgow and WebCT and EEVeC in Edinburgh.

Adding RLOs to Moodle and Web CT is very straightforward – the content and manifest are packaged as a zip file in RELOAD (www.reload.ac.uk), and uploaded as a SCORM activity within an individual course. As yet, there is no direct RLO import facility in EEVeC, but RLOs can be submitted through the developers.

Quality control

Maintaining motivation to re-purpose ‘old’ materials (up to ten years old in cases) can be difficult. Although it takes longer, it always seems more exciting to create new materials from scratch. Packaging and cataloguing satisfactorily can take several iterations and in the worst cases two to four hours per RLO. However, peer review of content and learning design by teachers and educational technologists and evaluation by end-users is critical so that RLOs maintain educational value.

Repositories

This is a joint Glasgow-Edinburgh collaboration, a continuation of the partnership arising out of the CLIVE project. The University of Edinburgh uses a repository called Intralibrary, to which veterinary objects from Glasgow and Edinburgh have been added arising from this project. The University of Glasgow did have access to the LRC3 repository, but its license has expired. This exercise has flagged the importance of having a shared, long-term repository between the schools. The JORUM repository, to be launched later this year, offers this possibility, at least within the UK (www.jorum.ac.uk).

Metadata

For this project, we have, for convenience, taken the decision to add metadata (LOM 6.2) to the objects as they are uploaded to the Intralibrary repository, rather than within RELOAD. The information provided within Intralibrary includes mandatory and recommended fields – title, description, identifier, language, keywords, aggregation level, lifecycle, contributors, format, and copyright restrictions.

The RLO is then classified (discipline as defined by the Library of Congress, and educational level by those set by the Scottish Curriculum Qualification Framework). The metadata are comparable with the profiles offered in RELOAD. Much more detailed metadata could be added, however cataloguing is time consuming and unpopular with developers. Vocabularies for veterinary keywords are not nationally agreed, so this is an area for future work.

Conclusions

Staff and students, when asked, like the idea of having a searchable repository of RLOs, and readily acknowledge potential benefits. However, working with multiple pieces of software to produce and catalogue RLOs is likely to deter all but the enthusiast and the developer.

Repurposing is a quick (if tedious) way of generating RLOs, although time/cost issues mean that at present, it is really only worthwhile cataloguing high quality materials. Once a repository is populated, it is then quite easy to use. Out of date content will remain an issue, however the ability for any user to add comments to RLOs may be a powerful way to inform use.

The four stage process to recreating an object:

• Sub-division of CAL programs
• Packaging into IMS/SCORM RLOs (we used RELOAD)
• Uploading into repositories and adding metadata (if not done in the previous step)
• Downloading and importing into VLEs for testing

For further information please contact v.dale@vet.gla.ac.uk or g.mcconnell@ed.ac.uk

Workshop report: Examiner training for OSCEs, Dr Katharine Boursicot, Senior Lecturer in Medical Education, Barts and The London, Queen Mary’s School of Medicine and Dentistry, and Associate Dean for Assessment, University of Cambridge School of Clinical Medicine; Trudie Roberts, Professor of Medical Education and Director of the Medical Education Unit, University of Leeds Medical School

Following previous Subject Centre funded workshops on ‘Using OSCEs for the assessment of clinical competence’ Kathy and Trudie developed a new workshop on training examiners for OSCEs.They developed it over several years in their own institutions as part of quality assurance processes to enhance the consistency of examiner scoring in OSCEs.

The Subject Centre offered them the opportunity to run a workshop on a national level and share their experience with the wider academic community.

While many institutions have introduced the use of OSCEs in both formative and summative assessment procedures, the issue of examiner training remains underdeveloped. Examiner inconsistency in the conduct of stations and in marking are important contributors to the error factors affecting the reliability of OSCEs. The student perception of fairness is also affected by variability in the way different examiners conduct stations. It is therefore vital to train examiners to behave as consistently as possible. Additionally, the QAA and the GMC are seeking evidence of appropriate examiner training, especially at the level of graduating, summative examinations.

This workshop was intended to provide a model for training examiners for OSCEs, so that this assessment procedure is conducted to the highest standards.

Who attended?

The workshops were attended by 20-25 people on each occasion – the mix included clinicians and non-clinicians involved in undergraduate medical, dental and veterinary education as well as nurses, midwives, physiotherapists and veterinary nurses.

Participants came from all over the UK and we even had a participant from Singapore and interest from Myanmar!

What we did

• Principles of OSCEs (presentation)
• Role of examiners in OSCEs (presentation)
• Marking video stations (group work with discussion)
• Marking live stations (group work with SPs and manikins)
• Standard setting (presentation)
• Networking!

Workshop materials

• Tutor briefing notes (to enable others to run this workshop)
• PowerPoint folder of the presentations for this workshop
• Handouts of the presentations
• Reading list
• Samples of stations for live marking.

Feedback

This workshop was so popular we had to run it three times to accommodate the waiting list of people! The feedback has been uniformly excellent.

We have very much enjoyed running them and have learnt a great deal from the participants. One of the best innovations that we brought in was asking delegates beforehand if they could e-mail any specific questions they had that they wanted answering.

This has meant that we can make sure that we really cover what the participants want.

What next?

During all three workshops, there was an overwhelming demand for information and training on standard setting for OSCEs and other performance-based assessments.

Therefore we will be running a workshop in the autumn entitled ‘Standard setting for undergraduate examinations: a beginners’ practical guide’.

Look out for this on the Subject Centre website (www.medev.ac.uk/resources/meetings/workshops/).

Benefits for participants:

• Understanding of the factors that contribute to the reliability of OSCEs and how to enhance these
• Experience of examiner training for OSCEs
• Experience of using videos of OSCE to calibrate marking
• Experience of using live stations (using SPs and manikins) to calibrate marking
• Understanding of the role of examiners in performance based standard setting

Benefits for the disciplinary community:

• Dissemination of good practice in examiner training for OSCEs
• A framework for designing and conducting examiner training
• A common framework for meeting QAA and GMC requirements in relation to examiner training and maintaining standards in assessment procedures

Workshop report: Interprofessional assessment – challenges and opportunities, Marian Traynor, Queen’s University Belfast

Taking the example of IPE at Queen’s University Belfast, Marian Traynor and colleagues led a Subject Centre workshop looking at the challenges that interprofessional assessment sets.

The workshop had three main objectives:

• To provide an overview of the challenges for IPE assessment
• To provide an overview of IPE at Queen’s University, Belfast
• To discuss future developments.

Overview

Participants were introduced to the complex nature of assessment exploring the Why? Who? What? Where? and How? aspects of assessment. Emphasis was placed on the need for assessment processes and procedures to be congruent with the aims of the educational curriculum and the need for a full contribution from all those involved in the assessment process.

This then lead on to a discussion as to how to achieve a balance between objectivity subjectivity within the assessment process and more especially how this can be achieved within a competency model.

Participants were reminded that a competency model is dependent on the professional judgement of the assessor as to whether or not the student meets the competency.

Having discussed some of the issues surrounding assessment within a uniprofessional curriculum participants were asked to explore the issues surrounding the assessment of a curriculum with an IPE focus.

Participants were particularly asked to consider the following within an IPE framework:

• Curriculum and learning outcomes
• Professional requirements
• Teacher/facilitator preparation
• Design of the assessment tool.

IPE at Queen’s

In order to enhance the discussion of assessment within an IPE framework information was provided on two IPE projects run at Queen’s University Belfast. Both projects were independently funded (one by the Subject Centre) and each included two different approaches to assessment.

Both projects focused on an undergraduate nursing and medical curriculum. The projects involved third year nursing students (children’s branch) and fourth year medical students (paediatric placement). While one project used presentation and role play as a method of assessment, the other focused on presentations and the completion of a reflective commentary. Both presentations highlighted the complex nature of assessment within an IPE programme. In order to further alert participants to the complexities surrounding assessment in an IPE context they were asked to do group work using a case scenario.

The scenario was clinically focused and involved health care students from two or more professions working together to deliver patient care. Participants were asked to consider the case scenario and the assessment process under the following headings:

• What will you be measuring?
• How will you measure it?
• Who will you involve?
• How will you ensure reliability and validity of the method?
• Does this fit with the current assessment profile in specific undergraduate programmes?
• Will an interprofessional assessment have an impact on the professional body requirements?

The future

The IPE projects run at Queen’s have highlighted the need for appropriate funding to support the ongoing development of IPE and the need for co-operation between professional bodies and universities. The projects have also highlighted the need for students to be involved in planning, particularly with regard to the assessment process. Participants at the workshop endorsed these recommendations adding that this should then result in enhanced links between the university and the NHS teachers, improved patient care, a better student learning experience and ultimately a change in workplace culture.

With such recommendations in place we can look forward to a workplace that recognises problem solving based on teamwork as the norm.

CETL focus: Assessment and learning in practice settings (ALPS), Trudie Roberts, Professor of Medical Education and CETL Director, Leeds School of Medicine

This Centre for Excellence in Teaching and Learning is a collaboration between the Universities of Leeds, Bradford, Huddersfield, Leeds Metropolitan and York St John College. Here the Director tells us the background to the CETL and what the future holds.

For newly qualified health and social care (H≻) professionals, taking up their first post is often highly stressful. There are many publications reporting their feelings of lack of preparedness for the role they are required to fulfil.

We believe that the key to equipping these professionals for the real world is increased partnership between HEIs, students and stakeholder employers involving work-based learning, in particular the attainment of patient/clientcentred professional competence.

Many of the competences required at graduation are shared across all H≻ professionals and recipients of that care are dependent on the smooth integration of these skills. By looking for commonality of purpose across H≻ education and sharing scarce resources to assess common outcomes, we can provide a more robust framework for the assessment of clinical/professional competence and use this assessment to drive strategic learning.

Quality assessments

Currently, HEIs can find it difficult for some professional groups in H≻ to provide the required number and quality of teacher assessments for students in the work place. One reason is that the pool of academics available to be involved in these assessments has diminished.

In many instances the practitioners carrying out these assessments for students on placements have minimal training and understanding of the process of reliable testing and of the use of assessment to drive learning. As full time clinicians or social workers, they also suffer from the ever-burgeoning demands of the services they provide.

With better training and understanding of the processes needed to achieve professional competence, this CETL aims to strengthen assessment processes whilst relieving some of the current assessment burden.

New learning concepts

In the last two decades the concept of learning has been reformulated based on new insights. There has emerged a greater understanding of how people learn, and particularly the social context within which learning happens. Effective learning is conceived as occurring when a learner constructs his or her own knowledge base that can be used as a tool to interpret the world and solve complex problems.

This implies that learners should be autonomous and selfregulating, and that they need to be motivated to continually use and expand their knowledge and skill base.

In effect, to become lifelong learners with the skills of reflection and self-efficacy, learners need to develop strategic learning behaviour, meaning they must master effective learning strategies.

The role of effective feedback in improving learning is well established. Additionally learners need metacognitive skills to be able to use feedback to improve their learning and to be able to reflect on their own and others’ perspectives.

Improving standards

‘Clinical’ competence can be defined as: the degree to which an individual can use the knowledge, skills and judgement associated with their profession to perform effectively in the domain of possible encounters defining the scope of professional practice1.

The ultimate purpose of performance assessments must be to improve the standards of patient and client care. The assessment of this competence is vital to providing students and their teachers with the confidence that they will be able to perform effectively in the next stage of their careers.

Assessors are not only interested in how the individual performs in observed situations but also in generalising about the individual’s ability to perform a variety of other tasks in a range of similar work situations, that is, estimating overall true professional/clinical competence.

The published literature shows that for work based assessments to be reliable and valid, multiple judgements need to be made of individuals by different assessors in a large sample of diverse practice situations2.

Competence profile

Valid assessment methods thus aim to reflect actual professional practice. A student’s competence profile should be based on multiple measures from a range of individuals, including peers, self, teachers and recipients of this practice. Assessment should be integrated into the learning process and equip the learner to be more effective in judging their own learning – so called ‘sustainable assessment’.

Such assessments judge against standards whilst not compromising future learning needs, and establish a basis for students to undertake their own assessment activities in the future. Research has shown convincingly that using assessment as a tool for learning, including good and well timed feedback, promotes the capacity for life long learning3.

Core aims

Using the ideas of Boyer4 we intend to pursue the scholarship of teaching through the CETL. Our core aims include a means through which the status of teaching may be raised, a means through which teachers may come to teach more knowledgeably and to provide a means through which the quality of teaching may be assessed. We propose to build on the ideas of Trigwell and Shale5 to develop staff involved in assessment.

The cultural divide

The cultural divide (previously described as the ‘theory practice gap’) is narrowing, but there is still an element of division between the theory provided by many HEIs and the practice required by employer stakeholders. The CETL’s academic development of a body of practitioners will achieve mutual confidence in the assessment of professional competence regardless of who assesses and who is being assessed. Involvement of patients and carers will provide the final element for complete professional assessment.

Award of this CETL funding for five years provides a tremendous opportunity for the HEIs to collaborate and spread good practice in work place assessment for the benefit not only to future students’ and practitoners’ professional practice but also to health and social care overall.

For further information contact t.e.roberts@leeds.ac.uk

References

1. Kane MT(1992) The assessment of professional competence. Evaluation and the health professions 15:163-82
2. Williams RG, Klamen DA,McGaghie WC (2003) Cognitive, social and environmental sources of bias in clinical performance rating.Teaching and Learning in Medicine 15(4):270-292
3. Boud D (2000) Sustainable Assessment: rethinking assessment for the learning society. Studies in Continuing Education 22:151-167
4. Boyer EL (1990) Scholarship reconsidered: priorities of the professional. Princeton, NJ, Carnegie Foundation for the Advancement of Teaching
5. Trigwell K & Shale S (2004) Student learning and the scholarship of university teaching. Studies in Higher Education, 29:523-536

CETL focus: Enquiry Based Learning (EBL), Professor Paul O’Neill, Associate Dean of Medical Undergraduate Studies and CETL Director, University of Manchester

The University of Manchester has been awarded a Centre for Excellence in Teaching and Learning (CETL) in Enquiry Based Learning (EBL).

HEFCE’s provision of £4.5M funding over five years will enable us to extend the range and scope of EBL activity throughout the University, disseminating and supporting related initiatives across the UK sector and in an international arena.

EBL describes an environment in which learning is driven by a process of enquiry owned by the student.

The tutor establishes the task and facilitates the process, but the students pursue their own lines of enquiry, draw on their existing knowledge and identify the consequent learning needs. They seek out relevant evidence and take responsibility for analysing and presenting this appropriately, either as part of a group or as an individual supported by others. They are thus engaged as partners in learning.

It is essential that our students are educated for knowledge creation, lifelong learning and leadership. They will take on leading roles in their future working environments; directing change, asking important questions, solving problems and developing new knowledge. Basing learning on a process of enquiry will develop the necessary abilities and attitudes, while still taking account of an increasingly diversified student population. EBL covers a spectrum of approaches.

Centre organisation

The Centre will bring clusters of excellence together as a critical mass that will influence education throughout our four Faculties and support the existing community of practice.

We will promote EBL using a ‘hub and spoke’ model — a central ‘hub’ supporting local developments through a spoke within each Faculty.

It will provide a home for expert staff (e.g. consultants in such areas as staff development and assessment) and student interns, as well as employing staff to direct and support the Centre. The Faculty spokes will comprise staff experienced in developing EBL within each Faculty’s disciplines.

These link staff will draw on Centre resources and our successful experience of change management to implement EBL in new areas. The capital spend will cover both a central resource of offices and training rooms, and local facilities for EBL within each Faculty.

Evaluation

We will conduct a programme of research and evaluation that builds on our existing track record. We will also provide infrastructure for dissemination, building on our links with bodies such as the Higher Education Academy and professional institutions, and expertise for generic topics such as student partnerships or assessment strategies.

Dissemination beyond the University will be based on our range of contacts, making use of the existing EBL network, the network on research-based learning to be hosted by the Academy and relevant Subject Centres.

We regard active engagement with the HE Academy as critical to wider dissemination.

Excellence

Our leadership of a regional EBL project and ongoing, expanding network will act as a prototype for the CETL in its approach to codevelopment and continuation through a network with regional, national and international links and in giving value for money.

Our outstanding track record of developing and producing gains in EBL will expand our own excellence and bring about the following impact:

• We will have ensured that around one third of our own students experience some form of EBL
• The CETL will have a strong influence on the University’s LTA strategy, so that a focus on enquiry increasingly characterizes the experience of our students
• We will have expanded our capacity for EBL both centrally and within Faculties, with many more staff possessing the ability to design and deliver EBL, and with suitable facilities in place
• Staff will have access to expert resources for the development of EBL practice
• Key staff will have been rewarded for their commitment to EBL, and the HR policy on recognition and eward will take account of staff expertise in and contribution to EBL
• We will have developed networks and partnerships with a range of institutions and organisations to promote EBL at local, regional, national and international levels.

We will have made a major contribution to pedagogic research in EBL and cognate areas as from its inception CETL activities will be underpinned by research into areas including the:

• Measurement of the impact of EBL, via research-based tracking method
• Identification of the characteristics of an ‘EBL friendly’ work environment
• Impact of EBL methods on student motivation
• Implications of a student's indigenous culture on their response to EBL
• Assessment of the existing theories underpinning EBL, and their application in differing contexts.

Reputation

Manchester will have a national and international reputation as ‘the’ centre for EBL.Our underpinning belief is that implementation of EBL should reflect its principles – a spirit of enquiry and collaborative learning. We recognise that both staff and students need to make transitions in adopting new approaches to learning and assessment, especially with the more open-ended approaches involved in EBL.

We will thus prioritise activity that helps staff and students to engage in action of their own.

For further information please contact: elizabeth.theaker@man.ac.uk or p_a_oneill@btopenworld.com

CETL focus: Healthcare professional education (CETL4HealthNE), Dr Pauline Pearson, CETL Deputy Director, School of Medical Education Development, University of Newcastle

Healthcare in the UK is constantly changing and developing. It is crucial for us to ensure that the students we educate will be fit for practice, and will really make a positive difference to individual patients’ experience of healthcare.To achieve this,we need to find effective ways to share our existing best practice, and to think creatively and collaboratively about how to respond to future needs.

CETL4HealthNE is a collaborative venture, led by University of Newcastle in partnership with colleagues at Durham, Northumbria, Sunderland and Teesside Universities, as well as some local NHS Trusts and our two Strategic Health Authorities. Each of our partners has particular strengths to bring to the collaboration, ranging from creative curriculum design, through innovative use of technology and success in widening participation to ground-breaking work with users and carers, and imaginative activities involving actors and creative writers.

Our programme of work is focused on six themes. The North East has some of the areas with the worst health experience in England1. It has suffered from massive economic and social decline in recent decades with the loss of major industries (such as coalmining, steel manufacture and ship building) contributing to further illhealth.

It is in this context that our CETL is based. Education is seen as a major driver for the sorts of change that will make a real, measurable difference to the health of individuals, families and communities in this region and beyond.

Good practice

The five Universities for the North East have been working together in a variety of areas for many years. However, the CETL gives us the opportunity, properly resourced, to share our good practice in health and social care education more systematically, and to build and develop conversations with NHS partners about the needs of healthcare in the future which can (and will) inform our educational programmes.

The themes we have highlighted represent the first fruits of such conversations. Health and social care professionals in the future will need to be able to work together effectively, within and across organisations, to provide high quality ‘seamless’ care for patients.

They will need to be able to think critically and creatively, to learn from each other, and to review each other’s practice without getting defensive.

One of the criticisms of some education for healthcare has been that it is too far removed from real life practice, and that newly qualified staff especially experience a ‘theorypractice gap’.

Work-based and experiential learning are important in seeking to bridge this gap.

In the future it will be important for professionals to use their skills and resources for greatest health impact, and to focus increasingly on promoting health and preventing illhealth in communities rather than solely on troubleshooting individual acute problems. Demographic pressures will lead to the scarcity or revision of some traditional roles and the establishment of new ones.

Technology changes very rapidly, and it is quite clear that the technologies we take for granted in healthcare today will be significantly different when this year’s entrants to medicine graduate, let alone in ten years time.It is no longer seen as acceptable for patients, carers and service users to be left out of the planning of healthcare – so it is equally important to involve them in planning education for health.

The six themes:

1. Interprofessional education
2. ‘People with Experience’ – user involvement
3. Peer group learning
4. Practice based approaches to learning
5. Health of the population
6. Preparation for modernised health care

CETL4 fellows

Our intention is to involve not only the ‘usual suspects’ of curriculum development and innovation – academic staff from all our partners – but also to include and pay serious attention to the contributions of students, patients (or people with experience of healthcare), health and social care practitioners, managers a nd others representing the organisational perspective and professional and statutory bodies.

Our 20 Fellows will be, in a sense, the tip of the iceberg, since we expect to involve a much wider range of associates (from UK and overseas) who we hope will bring broader expertise to the group. There are already well over 90 associates on our mailing list.

For each work theme, we have formed a group to create new ideas and plans for taking the theme forward, to steer and undertake piloting of particular innovations, and then to embed them into the mainstream of education and healthcare practice.

For example our Interprofessional Education (IPE) group will be building on the work of our existing Common Learning team and exploring the involvement of new professions, new locations and new areas of practice as well as interfaces with classroom based learning. Alongside the workgroups we are in the process of using capital funds to improve our infrastructure – for example enhancing network band width and connectivity to local NHS sites, increasing video-conferencing facilities and exploring the potential of hand held devices with wireless internet facilities for staff and students.

We have also begun to examine how to evaluate our work. CETL4HealthNE is a recent arrival on the higher education scene. However, we hope that our ideas and our alliances will grow and flourish. We would be interested to hear from others who are developing work in similar areas (whether or not they are CETL based). Our aim over the next five years (and beyond) is to make a real difference for all our partners, but above all, for patients.

For further information please contact p.h.pearson@ncl.ac.uk or visit www.cetl4healthne.ac.uk

Reference

1. Health and Deprivation: Inequality in the North,Townsend et al. 1988

CETL focus: Developing professionalism in medical students, Anne Garden, Professor of Medical Education and CETL Director, University of Liverpool

The Centre for excellence in developing professionalism in medical students is a CETL based at the University of Liverpool. Here the CETL Director, Professor Anne Garden gives a brief overview of what the CETL is all about.

The public look to the medical profession for help and guidance at many times in their lives. Often this is when people are feeling apprehensive, in pain or in distress.

They expect to be cared for by a competent, ethical and wise doctor who they can trust absolutely. This CETL will look at how this professionalism develops in medical students, how it can be nurtured and how it can be learned and assessed.

For these students, their development into professional practitioners involves many challenging and stressful encounters with doctors, nurses, patients and carers, not to mention teachers and examiners. These experiences form the learning resources through which they can develop their professionalism.

Reflective instrument

In the CETL we have already piloted a reflective instrument through which students may learn, plan and develop themselves into professional practitioners. Sharing these experiences, and their thoughts on them, may also help students to defuse stress and anxiety. During the project we will be working to develop and assess professional attitudes and behaviour in students as they progress through medical school.

This will be developed through to final year where their attainment of professional attributes is assessed in the work place. The intention is that this will lead to a seamless progression to a professional medical practitioner with the knowledge, skills and attitudes to continue their development to be a trusted, caring and wise doctor.

Project focus: A progress file learning system, Professor Liz Davenport, Barts and The London, Queen Mary’s School of Medicine and Dentistry

The Progress File Learning System (PFLS) has been developed by Barts and The London Queen Mary’s School of Medicine and Dentistry and other partners for use by dental undergraduates and their teachers.The PFLS comprises a Progress File / Logbook, Record of Achievement and Personal Development Planer (PDP) and Tutor Resource File and Record book.

We have encouraged the use of Progress Files (PF) as a means to promote reflection and self evaluative learning to underpin life long learning and continuing professional development. The PF was designed to encompass the expectations of the HE sector and Dearing1 and reduce the ‘front loading’ of training and learning. Pee and colleagues2 recognise the need to facilitate reflection, networking and communication and open discourse but at the same time acknowledged that systems such as these take time to become embedded.

If the PF is placed at the centre of student learning reflection, communication and open discourse results with the aim of promoting excellence in teaching and learning. Two hundred and fifty six clinical dental undergraduate students in years 2, 3, 4 and 5 were invited to complete a structured questionnaire as part of the general curriculum evaluation process.

The questionnaire was given out at the end of scheduled lectures in the autumn of 2003, twelve months after the implementation of the PF in the Institute of Dentistry. It was developed to investigate the use, usefulness and requirements of the PF, its design and format, the interaction between students with personal tutors, module and thread convenors and patients, their use of action plans and the impact of PF in learning in a clinical setting. A 50% response rate was achieved, where 127 responded from the four clinical years, and this was variable across the years mainly related to non attendance at lectures and failure to hand in questionnaires.

Approximately half the responders were female, and aged between 22 and 25 years of age. The undergraduates found the PF easy to use. The junior students (2nd and 3rd years) found it helped with their organisational skills. Despite this less than 50% used it to help them with identifying their strengths and weaknesses. However, the 3rd and 4th year students whilst finding their feet on the clinic were more positive but final year students felt it did not provide a means to identify areas for concern. Once the PF was established in the clinic, the students were better at recording all interactions with their patients. More than 60% of the students reported factual data, diagnosis and treatment plans for their patients as an ‘aide memoir’. The fourth years demonstrated the most consistent approach to recording data.

Feedback

The perceived impact on the management of the student’s patients is also shown in Table 1. Fourth year undergraduate students were better able to see what they were doing by using the PF, but on the whole the value was unclear. The junior students felt that the PF helped them to try harder whereas only a quarter of the 4th year was sure it helped.

Feedback was generally given at the end of a session and most appreciated when given in relation to a clinical procedure especially advice on how to improve as well as being praised on things well done. Typical feedback encompassed having errors pointed out.

Negative feedback was thought to influence their interaction with patients and was felt to be detrimental to their progress. It also made them nervous about the next appointment/encounter as did poor grades from the module/speciality tutors. In order for the undergraduate to get the most out of their PF they agreed time was needed to complete it and that it was useful in improving organisational skills. Time constraints have been a long standing complaint from staff and undergraduates alike.

Aid to reflection

Overall the PF has proved to be a useful tool as it provides a means to record interactions and progress whether in the clinic, seminar or clinical skills laboratory. It helps reflection about the treatment provided and achievements. It is believed to help to plan for the next clinical encounter. We believe that the undergraduates are better able to understand what they are doing and can develop coping strategies to fulfil their goals in the short, medium and long term. However, we need to continue to break down barriers to the PF’s use to facilitate innovative practice and promoting self assessment, life long learning and action planning as required for professional dental practice.

Next steps

Having evaluated the PF for two years now and in general have found it to be a successful tool for recording undergraduate practice experience and in promoting reflection we have moved on to develop an electronic Progress File (ePF) in partnership with six other dental schools and one Vocational Dental Practitioner Scheme.

Conclusions

• The PF therefore helps the majority of undergraduates to record and log their progress
• The PF encourages the undergraduate to think about what they are doing in order to manage their patients practice better
• Negative feedback and low grades many be detrimental to good patient undergraduate interaction

References

1. National Committee of Inquiry into Higher Education. Higher Education in the Learning Society. London: HSMO, 1997 www.leeds.ac.uk/educol/ncihe
2. Pee B,Woodman T, Davenport E S. Developing teaching and learning innovations in dental training: the example of the Progress File Learning System. British Dental Journal 2003; 195: 463-466

FDTL4 update: Web-based interprofessional learning network, Dr Giuseppe Cannavina, Project Director, University of Sheffield

The WILeN (Web-based Inter-professional Learning Network) project is one of the FDTL4 funded projects that received funding for three years, with additional funding support from the University of Sheffield, and in collaboration with the University of Derby. It aimed to develop a web-based inter-professional learning enironment, build a network of interest and disseminate the findings of the project.

The WILeN project was initiated to a ddress the need identified by the QAA and HEFCE for dentistry, nursing and other subjects allied to medicine to develop a more student centred approach to its teaching and promotion of inter-professional education through on-campus methods and remote modes of access such as virtual learning environments.

WILeN so far

The aims of the WILeN project were, therefore, to develop generic courses for health professionals that could be delivered as inter-professional education, further develop and promote the means for remote access to such learning environments, and establish a network across institutions to promote both elearning and inter-professional education.

WILeN was started at Sheffield in 2002 under the directorship of Dr Giuseppe Cannavina, and project managed by Dr Chris Stokes. The University of Derby, through Dr Kate Dann, is a partner of the network, and was involved in the project from its inception. The project had an official launch at the beginning of its second year.

Since the progress of WILeN was last reported in these pages in 2003, the project has been developing methodologies and software to support online inter-professional learning. Initial development for online learning was focussed on the iCT Portal and ‘Lega1’. The former is a generic topic portal that was used for both teaching and problem solving, the latter a developmental online assessment system.

New development - EPISTLE

After both the WILeN team’s and external evaluator’s comments on the functionality of the iCT Portal, and it’s shortcomings in being a dedicated platform to support interprofessional learning, the WILeN project team have developed a pedagogic and technical specification for an additional learning platform. This is called EPISTLE (Evidence & Problem-based Inter-professional Structured Learning Environment).

In addition to providing a teaching and learning platform for interprofessional education, it promotes evidence-based practice and group and collaborative learning. The EPISTLE system is a plug-in module to existing virtual learning environments (such as Blackboard, WebCT and Moodle) that facilitates the delivery of group-based problemsolving tasks. In the health professions, for example, medical cases can be presented and students are encouraged to discuss both the case and the evidence available to support a treatment plan. During the dissemination phase of development, the WILeN team have received many examples of how other disciplines would like to use this system.

As observed by the external evaluator, the EPISTLE system builds on a socio-constructivist

Expert comment learning methodology. This develops from the core idea that cognitive activity, goals and social interactions are intimately linked in particular ways and in particular contexts.

Knowledge and understandings emerge as a result of interaction and become distributed among the interacting group in a particular situation or context. To implement and facilitate this is very challenging, requiring substantial staff development, consideration of the role of the e-tutor, and developing student skills in computer-supported collaborative learning. Clarifying the different methodologies and how, when and where they are used will assist with the development of a coherent learning design for EPISTLE.

The [EPISTLE] system promises the possibility of integrating the different learning contexts and methodologies being addressed by the WILeN project and enhancing both generic learning (collaboration, communication, knowledge building and thinking) and specific knowledge, clinical and academic practices relating to case building and authoring, diagnosis and treatment. *(Sheena Banks – WILeN External Evaluator, 2004)*

This development is being undertaken as a priority project by the WILeN team, as it not only will meet the requirements of the WILeN project for e-learning and interprofessional learning, but it marks the fusing of the partnership between the University of Sheffield and the expertise at the University of Derby in e-learning and media development. The EPISTLE development has been identified as a transferable resource, and the University of Nottingham, Queen’s University Belfast and the University of Cambridge have all expressed an intention to develop cases and trial the system during the ‘Transferability’ funded phase of the WILeN project.

WILeN successes

An unexpected outcome of the WILeN project was Miss Sarah The WILeN project team. Pollington being awarded the LTSN- 01 / ADEE ‘Rewarding Excellence in Learning and Teaching’ 2004 prize for a written account of her involvement as a Clinical Consultant during development of the EPISTLE system.

This enabled the WILeN project to be disseminated to delegates at the Annual Meeting of the Association for Dental Education in Cardiff. This is the second year running that this prize has been won by a WILeN team member (Dr Chris Stokes won the dental category in 2003).

Getting in contact

The WILeN website (www.wilen.ac.uk) has been extensively rewritten during the second year to emphasize its role as dissemination for the project. The site has been changed to emphasise collaboration, incorporating the WILeN forum, and now features downloadable content and reports. If you wish to contact the WILeN team to discuss any aspect of the project, or to express an interest in the conference planned for 2006, please email us at c.w.stokes@shef.ac.uk or g.cannavina@shef.ac.uk

The learning styles of medical school entrants at Southampton University, Vikki Haley and Claire Smith, University of Southampton

The phenomena of a cohort of 219 students all with different experiences and learning preferences inspired the researchers to investigate the student cohort further.

Entrants to the 2003 five-year Bachelor of Medicine (BM) course volunteered to help the researchers understand the ‘student and their learning style and success’.The Southampton curriculum is a mixed mode, systems based course with longitudinal strands.

Substantial research has shown that students learn in a variety of ways1. Our research first identifies student learning style types, on entrance to medical school. The second part looks at the relationship between students preferential learning style and the type of institution they experience before embarking on the University of Southampton BM course.

Due to the mixed nature of the curriculum we were interested to see if students change their learning style preference after exposure to the curriculum for an academic year. Hence the third part of the project. Finally, in order to ensure teaching methods employed in the School of Medicine result in the same degree of learning between the different learning style groups, the first year primary BM exam results were compared. With informed consent students completed Honey and Mumford’s2 learning style inventory at the beginning and end of the year. This gave us a learning style preference: Activist, Theorist, Reflector and Pragmatist.

The students also provided us with demographic details and their BM primary results. 109 students out of the cohort of 219 (49.8%) completed the first inventory, 165 (75.3%) completed the second.

Project evaluation

This project identified the following key points:

• On entrance to medical school, the learning style of ‘reflector’ (40.37%) was dominant
• There is a correlation between previous institution and learning style, with sixth form students having a higher percentage of ‘activist’ learners
• Over 50% of students changed their learning style over the year, with activists becoming the dominant group, possibly reflecting the mixed mode nature of the curriculum
• Theorists achieved higher results in the end of year exam.

Whilst the debate continues over the usefulness of learning styles, this research has provided at local level an insight into a cohort of students attending the School of Medicine at the University of Southampton.

Not only has this work highlighted areas to be considered by the curriculum development working party, it has also raised questions about student admissions procedures.Students have also reported that our research has provided them with an insight into their own learning styles and given them an additional chance to think about how they learn.

Dissemination

The final Subject Centre report is available on line at the Subject Centre website (www.medev.ac.uk). The researchers are preparing to present this project to School of Medicine Staff. This will also be displayed on the Centre for Learning Anatomical Sciences website and notice board. The researchers plan to further analyse the results and submit findings to an appropriate journal.

For further details contact c.f.smith@soton.ac.uk or v.l.haley@soton.ac.uk

References

1. Gibbs and Habeshaw. Preparing to Teach.An Introduction to Effective Teaching in Higher Education. 2nd ed. 1998
2. Honey P. and Mumford A. Learning Styles Inventory (1992) www.peterhoney.com Undergraduate students at study.

Mini project update: The effect of social class on psychometric assessments, Professor Mary Ann Lumsden, Associate Dean for Admissions to Medicine, Chair of the Working in Health Action Group, University of Glasgow

Admission of school-leavers to medical schools in the United Kingdom usually requires high academic achievement in school-leaving exams and may discriminate against those from disadvantaged backgrounds since high thresholds discourage applicants.

A system of selecting medical students that places more emphasis on equal opportunities, with the aim of identifying potential, is highly desirable. Many of the characteristics important in the best practice of medicine can be identified with the aid of psychometric assessment tools that have been shown to be reliable predictors of medical school performance as well as being more objective than currently used methods of assessment.

The social, cultural and ethnic backgrounds of medical graduates should reflect the diversity of the patient population. Medical schools are now actively seeking entrants from a greater variety of backgrounds. Even with the appropriate ability, many of those living in deprived circumstances find admissions requirements difficult to attain because they often lack opportunity or encouragement to maximise their potential.

Inclusion of psychometric assessments, whose performance is relatively unaffected by social disadvantage, but rather reflects innate qualities both cognitive and non-cognitive, could provide a more equal opportunity if incorporated as part of the admissions procedure.

The Subject Centre provided funding to contribute to the cost of data analysis of the projects that are described in this article. The specific area of interest was the influence of social class on psychometric instruments developed to assess both cognitive and non-cognitive skills.

The subjects

The Admissions Project (Applicant Cohorts)

Scottish residents applying to the Scottish medical schools for admission to medicine in 2002 (Applicant Cohort 1) and 2003 (Applicant Cohort 2) took part in the project. Participation was voluntary and the results of the psychometric tests played no part in the decisions of the admission committees. Over 80% of the school-leaver applicants participated, with 510 in Cohort 1 and 507 in Cohort 2.

The Widening Access Project (Pupil Cohort)

Over 2500 pupils attending schools throughout Scotland, with below average participation rates in Higher Education took part. These pupils were in Year 10 of school (First Year of Standard Grades, the Scottish equivalent of GCSEs) and all had a realistic chance of getting five good passes with one science.

Tests used

The participants completed the following tests:

• An assessment of personality: narcissism, aloofness, self-confidence and empathy (known as the NACE) were assessed in both the applicant cohorts and the school pupil cohort,
• An assessment of cognitive ability: this was assessed using the following instruments, 1 and 2 each covering the same areas as 3 and 4 but in a single test;
  1. Logical Reasoning and Problem Solving (LRPS) (AC1)
  2. Mental Agility Test (MAT) (AC2)
  3. Raven’s Matrices (Pupil Cohort)
  4. Millhill Vocabulary (Pupil Cohort)
• An assessment of moral orientation: this was sat by both applicant cohorts but the results will not be discussed further in this article.

LRPS, MAT and NACE were developed by Professor David Powis, Dr Miles Bore and Dr Don Monro based at the University of Newcastle, NSW, Australia.

Social class

This was determined using the postcode of residence for each participant. This can then be used to assess deprivation either with Deprivation Category or the ACORN Category, which provide similar profiles. In both instances Category 1 are the most affluent and comfortably off and deprivation increases thereafter.

ACORN Category takes into account housing, total income, occupation etc. Figure 1 (below right) confirms that applicants to medicine tend to come from wealthy backgrounds.

Results

Profile of applicants

Eighty eight percent of the applicant cohorts were aged between 16 and 18 years and 89% were caucasian. The applicant cohort were principally from affluent backgrounds (see Figure 1) whereas the pupils from schools with low participation rates in Higher Education showed a more even distribution (Figure 2).

However, in both cohorts, there were few representatives from the most deprived sections of society.

Cognitive skills

Although in both the applicant and pupil cohorts, social class affected cognitive ability (Figures 3 and 4) to a minor degree, a group of individuals from deprived areas with above average ability were identified. There were also different levels of cognitive ability in those achieving high grades in school leaving exams (Figure 5). In addition there was no significant difference in any of the parameters assessed between those attending independent and state funded schools in the applicant cohorts.

Personality assessment

The personality traits assessed were not affected by social class in either the applicant, or pupil cohorts.

Discussion

If instruments such as those used are to identify aptitude and potential rather than learned skills they need to be less dependent on social background or schooling than school leaving exams. Although some influence of background was found in relation to cognitive ability, this project has demonstrated that within the groups derived from the areas of greatest deprivation, there are those with above average cognitive ability.

It is not unreasonable to suggest that these assessments might be used in conjunction with school leaving exams to give more information regarding the potential of that individual. For those still at school, additional encouragement and exposure to a future not previously considered, could raise expectation. Activities aimed at raising awareness of careers in medicine and healthcare in general are being organised for the pupil cohort. It is hoped that by determining the profile of those continuing within this programme and applying to University, it will be possibly to use these assessment tools to identify those with the potential to undertake a healthcare course in Higher Education.

Article written on behalf of the Admissions Departments and Widening Access Units of Aberdeen, Dundee, Edinburgh, Glasgow and St Andrews Universities together with those involved in the Working in Health Access Programme at the above and the University of Stirling.

Workshop report: Supporting students who struggle academically, Dr Jo Brown, Lecturer in Communication Skills & Senior Tutor for Academic Support and Dr Dason Evans, Lecturer in Medical Education and Sexual Health, Barts and The London, St Mary’s School of Medicine and Dentistry

In the current educational climate, abandoning students who are struggling academically is culturally, financially and ethically unacceptable.The Academic Support Programme (ASP) at Barts and the London School of Medicine and Dentistry began five years ago to address educational problems with students.

The programme is an educationally sound intervention which provides tailored academic help for medical students who are struggling or failing. It has a student centred philosophy that supports students to take control of their own learning by using existing, or developing new, effective learning strategies. The programme focuses on students in their 3rd, 4th and 5th years, when problems with their progress are likely to be identified in clinically based as well as knowledge based assessments. Evidence from the p rogramme shows that causes of academic failure are widespread and sometimes not academic in nature.

The ASP aims to offer an holistic service to students that addresses all issues affecting their learning and progress.

The interview

At the forefront of the programme is the educational assessment interview that is offered to students. This 45 minute interview with two experienced members of faculty, using a semi structured pro-forma, allows an in depth look at the student’s life since beginning medical school. It finds out how students study for various parts of the course, how they learn skills, what books they use, how much time they devote to regular study, if they are still motivated to be a doctor, how they get on with patients in a clinical situation, whether they study alone or in groups, how things are in the rest of their life and, most importantly, why they think things have gone wrong. At the end of the interview a structured pattern of support and a formal learning contract is negotiated with the student.

A range of interventions may be offered to students either singly or in groups and may include:

• How to learn clinical skills
• Reflective communication skills
• Exam technique
• Anxiety management
• Revision/notetaking techniques
• Time management skills.

Examples of students who have received academic support

Student 1:

An excellent student academically who fails her first clinical skills OSCE. She uses a deep learning style for theoretical knowledge, but an inappropriate and superficial style for learning clinical skills. She was supported with sessions on ‘how to learn clinical skills’.

Student 2:

Fails his 3rd year theory exams with a very poor score. He had excellent ‘A’ level exam results, good year one exam results and reasonable year two results. He has a good short term memory but a superficial learning style.The volume of work is finally too much to cram and remember and he fails. He was supported with sessions that encouraged a regular habit of self directed learning and explored deep learning styles.

Student 3:

An international student fails his clinical skills OSCE, saying he ‘didn’t know what to expect of the exam’. He is isolated and lives with other international students from other courses. He rarely speaks English outside of college and never mixes socially with other medical students. He never learns with his peers and feels like an outsider. He is miserable. We arranged a place in central halls of residence, negotiated financial help with his embassy and offered an academic support package, which revolved around working in a group.

Students may also be referred elsewhere for specialist help with mental or physical health, relationship or financial problems, or to a Learning Support Unit for help with specific learning difficulties. A full report of the interview is also sent to their Senior Tutor. See above for some examples of students who have received academic support.

Exam outcomes

Although exam success is only one outcome of academic support, it is nevertheless an important indicator. Here are the retake exam results for students who had undertaken a period of academic support following failure of their 3rd year OSCE : 2002: 16 students re-sat, 15 students passed with 13 receiving grade B or above. 2003: 17 students re-sat, 17 students passed with 14 receiving B or above 2004: 29 students re-sat, 28 students passed with 21 receiving grade B or above.

External evaluation

The ASP was evaluated by a consultant Educational Psychologist who concluded that the ‘outcomes and processes of the ASP educational assessment and subsequent support are highly effective’. He recommended the programme should be disseminated throughout the main curriculum by providing staff skills development workshops to facilitate the widest support to students.

The future

ASP staff ran a Subject Centre workshop in March 2005 and hope to repeat this. What was clear from the workshop was that medical, dental, nursing and veterinary schools throughout the country had identified academic support needs amongst their student groups.

Student problems were multifactorial and similar in nature between schools. An Academic Support Network was set up via the Subject Centre to share ideas and practice between institutions.

For further information contact j.maclean-brown@qmul.ac.uk or d.e.evans@qmul.ac.uk

Student evaluation

‘I felt that it (educational assessment interview) didn’t put me down, didn’t make me feel thick...’

‘I was really pleased they took such a broad interest and in depth... came out really happy that they were interested in me and how it was going for me...’

‘They (ASP staff) went through all the exams we had done and really allowed us to explore how we do them...’

Workshop report: Formal appraisal of undergraduates – worth the effort? Deborah Murdoch-Eaton, Director of Student Support and Professor of Medical Education, Medical Education Unit, University of Leeds

With assessment structures that aim to detect those students of an unsatisfactory standard, what incentive is there for students to aim high? Deborah Murdoch-Eaton reports on her Subject Centre workshop looking at how good formal appraisal can encourage students to achieve their potential.

Progressing successfully through their undergraduate career in medical school for the majority of students is dependent solely on achieving satisfactory course outcomes. However, a big question is the definition of ‘successful progress’ – have the students just done just enough to pass, or been encouraged and facilitated to achieve their potential, to do the ‘very best’ they are capable of? Current educational culture in most medical schools following implementation of the changes included in ‘Tomorrows Doctors’ have clearly defined outcome driven curricula, with assessment structures that aim to detect those students of an unsatisfactory standard, i.e. weeding out the bad apples. What incentive is there for students to aim high?

Feedback

Students receive feedback on their performance in a number of ways, most commonly by their assessment results (which may only consist of grades). This often means that students may only get feedback on those issues that are easily measurable. Outcomes measures may not take account of process i.e. how the student got there. Medical school is not just about reaching a required standard, it should also be about achieving ones’ potential.

Schools have a responsibility to not only inform students about their progress but also facilitate them to achieve the best that they can. A healthy, well balanced approach to study involves not only academic achievement, but an approach to study that develops efficient adult learners with mature life-long learning skills, and a holistic approach to their own development in all parameters of their lives.

Self evaluation

Personal development plans and progress files should incorporate some self-evaluation by reflection on performance. Reflective practice skills take time to develop, and may illustrate discrepancies between the students’ and teachers’ assessment of progress.

The value and motivation that students’ place on these is enhanced by a teachers’ interest in their development – and this is where formal appraisal has a role to play. A formal appraisal system has been running in Leeds whereby all 1st and 2nd year undergraduate students, and selected/self-nominated 3rd year students have a formal one-to-one appraisal with trained senior members of faculty. Students prepare by reflection on their progress over the year at medical school, including evaluation of their academic results in the light of their approach to study and extra-curricular activities.

Appraisal outcomes are formally documented with development of individual goals, and these are reviewed at subsequent appraisals.

Self esteem

The system is highly acclaimed by the students, and analysis of the outcomes clearly demonstrates not only an enhanced view of faculty and enhanced self-esteem, but a maturation in reflective practice with achievement of goals1.

Recent analysis of outcomes demonstrate that most common goals set were related to an aspect of study skills, ranging from general approach and preparation before tutorials through to quite specific revision skills. Nearly a third of 1st year students discussed changes required in their work-leisure balance. Some 10% had a documented goal to carry on the way they were, and were reassured they were doing fine! Success in implementing changes in approach to study is related to a healthy work-social life balance, and also addressing time management and organizational skills. Around 48% of 2nd and 3rd year students had made changes based on their previous year’s appraisal.

1) DG Murdoch-Eaton and MI Levene. Formal appraisal of undergraduate medical students – is it worth the effort? Medical Teacher 2004 26: p28-32

Personal tutors

Our experience in providing a successful appraisal system is that it also enhances the personal tutor system (which is often problematic across all medical schools). All medical schools are battling with large numbers of students to whom it is difficult to give individual and personalized feedback to optimize their development. Personal tutors are sent copies of the appraisal record forms documenting goals set by their students, and students are encouraged to see their personal tutors for followup.

The tangible nature of having something to look at together seems to help facilitate establishing the tutor – tutee relationship. Introducing appraisal early in undergraduate career implements early not only reflective practice, but is also underpinned on the premise that structured advice and support early in a student’s undergraduate career establishes efficient and appropriate attitudes to holistic learning and a medical career. Perhaps most importantly, it encourages students to achieve their potential, not just ‘good enough’. Another measure of success must be that the appraisers perceive the value, get considerable personal satisfaction out of the process and are willing to give up the time annually to do this, ‘reminds me why I enjoy teaching, a worthwhile activity’, and ‘reminder of what a delight medical students are’.

1. DG Murdoch-Eaton and MI Levene. Formal appraisal of undergraduate medical students – is it worth the effort? Medical Teacher 2004 26: p28-32

Mini project update: The effect of reflection on undergraduate learning, Dr Andy Grant, Senior Lecturer, Cardiff University

This study evaluated the effect of an optional reflective learning intervention on the learning of third year medical students.The intervention consisted of keeping a learning journal and taking part in fortnightly reflective tutorial groups. Participants found that they were better able to identify what was important to learn and better able to relate their learning in biosciences to the patients that they saw. Reflection gave students an opportunity to deal with emotional situations that they encountered.

Background

We had collected a significant amount of data from interviews with students who had been invited to participate in our study. We had interviewed those who had and those who had not chosen to participate. Both groups were able to tell us a great deal about the course and their learning.

Many had a style of learning involving taking down everything that was said in lectures and learning as much as possible of this for the exams. Some said that this style of learning had served them well so far and that they didn’t want to change it. There was a code of behaviour among the students which said that they should not appear keen in front of their peers.

Volunteering for our study, which involved attending tutorial groups at lunchtimes, looked too much like keenness for many students.

What we did not have at this stage was a clear picture of the effect of taking part in the study on students’ learning. Our grant from the Subject Centre made this possible.

Feedback

‘It felt nice to know med school is genuinely interested in making sure you are alright...’

‘This clarified that my efforts have been rewarded in my results – and how I could do even better...’

‘Allows reflection and reminders of previous failures and successes to encourage further results to be good – made me think...’

Collecting more data

With the insight from the interviews we had already done with participants we revised our interview guide. We wanted to know; what differences did taking part in the reflective learning study have on your learning? Why did it have that effect? We also wanted to know if students would continue to keep a learning journal after the study ended.

Reflective learning

Some of the students said that they had been using reflective learning techniques before they joined the study. The difference the study made for them was that it encouraged them to use these techniques that they already found useful for their learning more regularly.

Although taking part in the study was voluntary, students’ found that having a tutorial group in two days time was a potent stimulus to write an entry in their diary if they hadn’t already done so.

Benefits

The main benefit to students’ learning was metacognitive. They developed a greater sense of what they knew and what they needed to know. When students sat down to begin their revision rather than trying to memorise the contents of all their lecture notes they first took time to think what was most important to learn. Students were more self-directed in their learning in that they took time to integrate learning on a single topic from multiple sources rather than learning one lecture or one course at a time. Students reported a greater awareness of what they knew, finding this gave them a sense of confidence about their knowledge.

By reflecting on their learning styles students were able to be more confident in the learning style that they found suited them even if it appeared to be different to the style adopted by their peers. Both the learning journal and the tutorial groups were helpful as a source of reflection and discussion on emotionally distressing topics such as breaking bad news or death.

Learning techniques

Ours was one of relatively few studies of reflective learning in undergraduate medical students. Our study can inform further research in two ways.

Firstly, the effect that reflection can have on medical students’ learning. The benefits we reported are important in that they are associated with deeper learning and offered help with emotionally difficult material.

However, many of the students who did not sign up to the study thought that reflective learning was useful but that it did not relate to the kind of learning they perceived they had to do in the short term to ensure that they kept their place at medical school. One student even said it was dangerous to change one’s method of learning in the middle of your studies.

The other question that we have to address is, what would happen if we introduced reflective learning as part of the curriculum?

Our participants came from a self-selected group who had chosen to take part in something that suited their way of learning and wasn’t assessed in any way. How can we introduce reflective learning for all students without diluting the benefits?

What next?

Further research is needed to evaluate more general use of reflective learning. This study tells us that students need to see reflection as useful to their current learning and so any adjustment either to the reflective learning intervention or to the curriculum that bring them into alignment will help.

If reflective work is compulsory any assessment should be carried out in a way that does not tempt the student to go for best marks rather than write reflectively.

For example, if a learning journal is assessed the student should submit a summary rather than the journal itself so that the authenticity of the entries is not undermined.

For further information please contact grantaj@cf.ac.uk

Dental students and nurses: Investigating gender differences in communication and working styles

Ruth Freeman, Professor of Dental Public Health and Christine McWilliams, Research Assistant, Queen’s University Belfast

Male and female dentists have different experiences when working with female nurses. Women dentists fear being assertive and feel they receive less nursing assistance whereas male graduates flirt and believe they get more nursing support1,2. Do these patterns of communication occur in dental students? With the increased number of women dental students there is a need to investigate the role of gender in inter-professional working. With Subject Centre support Queen’s University, Belfast, investigated Belfast and Leeds clinical dental students’ interactions with female dental nurses.

Methodology

The investigation was in two parts. The first part invited all clinical dental students studying at Belfast and Leeds to complete the communication and working style questionnaire. It examined the students’ knowledge and attitudes towards dental nurses and their duties. The second part was qualitative and used semi-structured interviews to discover if gender influenced the students’ interactions with dental nurses.

Preliminary findings

In total 114 Belfast and 129 Leeds students took part. The results suggested that the dental students, irrespective of gender, considered that the dental nurse was essential for effective patient care. Despite this consensus, women students felt that they always had to ask for nursing assistance. The male students believed that the dental nurse was a friend who should be encouraged to ventilate her work problems which were solvable with ‘humour’ and ‘playfulness’. Male students were of the opinion that the best working combination was a male dentist with a female dental nurse.

Working styles

Two dental student working strategies emerged from the qualitative data. The first was a ‘friendly-working style’ and the second a ‘professionalworking style’. The essence of both styles was a synergistic interaction, the quality of which changed as the students advanced through their clinical years. The friendly-working and professional-working styles represented the two extremes of a professionalisation continuum (see Figure 1, facing page).

‘Friendly-working styles’ were characteristic of the novitiate student and represented a synergism based on clinical inexperience. The dental nurse was perceived as an ally and a ‘safe’ source of clinical knowledge and advice. S/he offered comfort and reassurance. The experienced dental nurse was able to defuse the students’ anxieties and through the knowledge they provided advice on when and how to use certain materials or techniques. The following is illustrative: ‘I was very, very slow and nervous about treating patients and the dental nurses were very, very good at putting the patient and me at ease. They can start conversations and I found that very helpful… I could think about the practical side of it. Even now when I forget to do something, or if I’m not too sure how to use a particular material, I ask them and they’re able to tell me.’ Fourth year male student, Leeds ‘Professional working styles’ were characteristic of more senior students. Professional-working styles reflected a shift from dependency to a type of cooperative working based upon the students’ veneer of clinical experience. Within the professional-working style the dental nurse was perceived as ahelper who quietly and efficiently facilitated the students’ clinical practice. S/he had, however, been demoted from her position as ‘allyteacher’ to ‘lackey-helper’. They were there to get the patient ‘in and out’ and to ‘get the job done’. The wish to be ‘boss’ was evident in senior male dental students who admitted to searching out younger student nurses because: ‘I can have authority over them but – with the older nurses – knowing they know so much more than me – they can still over-ride me.’Final year male student, Belfast

Despite this apparent desire for authority, it crumbled away when final year students were faced with the stress of clinical examinations. At these times they reverted back to their reliance upon the experienced nurse as ‘ally-teacher’: ‘When the nurse is there you gain confidence and she gives you reassurance – its subtle but during the exam – its invaluable – like you know the lining is going to be mixed right – so one less thing to worry about.’ Final year female student, Belfast

Gender distinctions

Clinical experience and gender did influence the dental students’ ability to obtain nursing assistance. Younger female students felt disempowered: ‘I did ask – well there were two, one was working with another student, and I asked the other if she was free to aspirate and she was cleaning a unit and she said: ‘Well you can wait a minute’.’ Third year female student, Leeds

Male dental students believed they were at an advantage and confessed to ‘being chatty’, ‘being friendly’ or ‘flirting’ with the nurses to gain chair-side assistance: ‘Well, I suppose the nurses would rather work with the male students.’ Fourth year male student, Belfast

‘It’s always a struggle to get a nurse, but somehow I always seem to get one and the popular boys seem to get more help...’ Final year male student, Leeds

Conclusions

The results suggest that differences in communication and working styles exist between male and female dental students. The women compared with the male students felt at a disadvantage when requesting and obtaining nursing assistance. Although professional and friendly working styles appeared not to be influenced by gender, since women students found it harder to request nursing support, this suggested that gender acted indirectly as a factor in gaining chair-side advice.

The finding that dental students used the experienced dental nurse as a chair-side advisor has implications for clinical teaching. Formalising the dental nurses’ teaching role would ensure fidelity of the education provided and promote appropriate interactions between student and nurse.

References

1. Freeman R. Gorter R. Braam A. Dentists Interacting and Working with Woman Dental Nurses: A Qualitative Investigation of Gender Differences in Primary Dental Care. British Dental Journal 2004; 196:161-165
2. Gorter R, Freeman R. Dentist- Assistant Communication Style: Perceived Gender Differences in The Netherlands and Northern Ireland. Community Dent and Oral Epidemiol 2005; 33:131-140