Abstract
Objective
This article aims to provide an overview of the current procedures in place by the General Medical Council to assess the fitness to practise of poorly performing doctors.
Method
It outlines the legal background, the structure and the rationale behind the type of assessments used.
Results
Specialty specific Tests of Competence and a workplace-based assessment are carried out by a specially trained and appointed team of assessors. All assessments are designed to the highest standards by expert teams, including trained stakeholders.
Conclusions
Assessments are continually being updated and validated to ensure high standards in this high stakes process.
Introduction
The General Medical Council (GMC) is the independent regulator of doctors in the UK. Its purpose is to protect, promote and maintain the health and safety of the public by ensuring proper standards in the practice of medicine. Since 1997 it has been able to use performance assessment as part of its Fitness to Practise procedures.
Referrals and legal background
Most complaints about doctors come from one of two sources: members of the public or a ‘person acting in public capacity’, on behalf of a public organization such as an NHS trust. In the last five years, the number of complaints received by the GMC has increased by 24%, with a total of 5168 complaints received in 2007. Around 39% of complaints are closed at initial assessment, within one month of referral. A further 34% are closed following contact with the employer. The remaining 27% are subject to a full investigation. 1
The Medical Act 1983 gives the GMC the power to place restrictions on a doctor's practice if his or her fitness to practise is impaired to a degree that requires action on registration. Cases closed at an early stage are deemed to fall below this threshold. Under the Medical Act 1983 as amended, impairment may only be as a result of misconduct, deficient professional performance, a criminal conviction or caution, physical or mental ill health, or a determination by a regulatory body.
Where a full investigation takes place, the information collected is reviewed by two decision-makers called case examiners. One will be medically qualified, the other a lay person. Case examiners can close a case, issue advice or issue a warning where they find a significant departure from the GMC's core guidance, Good Medical Practice, which does not reach the threshold of impairment. They can also invite the doctor to agree to restrict his or her practice (undertakings). If the allegations appear more serious and there is a realistic prospect of proving this, the case may be referred to a Fitness to Practise Panel. The Panel has the same options open to it as the case examiners but can also impose conditions on registration, suspend the doctor or remove the doctor's name from the medical register (erasure). Erasure is not an option if impairment is by reason of ill health alone. 2
The majority of complaints relate to allegations of misconduct. However, where poor performance may be the underlying issue, the doctor may be directed to undergo a performance assessment.
Structure of performance assessments
The performance assessment is made up of Tests of Competence and a workplace-based assessment with peer review. The doctor's performance is assessed by a specially appointed and trained assessment team which produces a report for the GMC.
Tests of Competence
Tests of Competence are assessments of knowledge and clinical skills. They comprise a knowledge test and an objective structured clinical examination (OSCE). Each Test is based in an individual specialty but is tailored to the practice of the doctor under investigation.
Knowledge test
The knowledge test conforms to guidance issued by the Postgraduate Medical Education Board (PMETB) on best practice in written examinations. 3 It is composed of Single Best Answer (SBA) questions and Extended Matching Questions (EMQ). Material has been reviewed and generated by cross-specialty writing groups consisting of practising clinicians, Royal College representatives, and doctors working in medical education in undergraduate and postgraduate settings. Existing material is edited by the group to ensure it is kept up to date and in line with changes in clinical practice. New questions are created by writers working in small groups and all material is blueprinted to Good Medical Practice. At each item writing day, training is given by clinicians and educators experienced in the field of item writing and there is opportunity for peer review and feedback of material. Box 1 shows an example of an SBA.
Example of a Single Best Answer (SBA) question
An 83-year-old woman with a chest infection becomes confused with poor concentration. She is restless and frightened. She is verbally abusive and has perceptual abnormalities. There is no significant previous psychiatric history.
What is the single most likely diagnosis?
A. acute confusional stateB. drug-induced psychosisC. Lewy body dementiaD. multi-infarct dementiaE. psychotic depression
Each individual Test of Competence is designed to reflect the doctor's practice and tailored to their grade and specialty, including areas of super specialization. The written test is composed of 200 items to be completed in three hours. The aim is for approximately 80% of the paper to contain core specialty specific questions and 20% to contain questions relevant to individual experience and practice. For instance, a general practitioner who works several sessions a week in a men's prison will have around 20% of their test items reflecting knowledge required for this setting.
OSCE
An OSCE is a circuit composed of several individual stations that require the doctor to demonstrate their clinical skills in front of an assessor. These skills might include examining a patient, taking a history or discussing management. They are well described as being valid and reliable in both undergraduate and postgraduate assessment. 4 The OSCE is constructed in a similar way to the knowledge test. Trained OSCE writing groups have reviewed existing material and new stations have been created to reflect changes in practice and make the most of new technologies, such as manikin technology and simulation. All the skills a doctor may use are tested: practical skills; communication and interpersonal skills; and clinical method. A similar approach is used to tailor the test to the individual doctor's practice.
Standard setting
Standard setting methodology is widely described in medical assessment. 5 It is a means of objectively setting a cut-off for a test to discriminate between those who perform well and those who don't. All written knowledge test questions are independently standard set using the Angoff method. A group of 8–12 doctors familiar with standard setting are recruited as judges. They are asked to conceptualize a group of doctors at the minimum standard to remain on the medical register. They are then asked to estimate the proportion of this group who would answer each question correctly. Individual judgements are discussed in the group and judges are given the opportunity to revise their judgement based on the discussion. Final proportions are averaged for each question to achieve a minimum acceptable score.
Validation of material
In order to provide a reference group for the scores attained in both the knowledge test and the OSCE, i.e. to validate the material, 47 specialty specific validation days have been held. Volunteer doctors are recruited from all clinical grades by widespread advertisement in the medical press and on websites used by doctors. Any doctor that has worked in the specialty for at least four months within the last year, who is fully registered with the GMC and not under active investigation, is eligible to become part of the reference group. Volunteers are told on recruitment that evidence of poor performance in all areas may result in referral to the GMC. Most recent validation days include general practice, general medicine, general surgery, paediatrics, obstetrics and gynaecology, general psychiatry, emergency medicine and anaesthetics.
Volunteer doctors take a knowledge test and OSCE under comparable examination conditions. Assessors for OSCE stations are trained and experienced assessors within either medical schools, Royal Colleges or the GMC. Volunteers, assessors and role-players are encouraged to give feedback on all material. Performance of the questions and OSCE stations is reviewed after the validation events to assist editing and to ensure the high quality of material used in the live Tests of Competence. If feedback on a question is poor or item analysis shows poor discrimination, this suggests that either there is a problem with the candidates' understanding of what is required, the question may be too difficult or too easy, or be ambiguous. This material is either revised and re-piloted or rejected. All material is piloted on a large number of doctors from a range of grades (FY2 – consultant) in order to obtain an appropriate peer group for comparison to the doctor referred for assessment.
Validity and reliability
The number of questions and format of the knowledge test is chosen to allow wide sampling by both content and context and, thus, improve validity. 6 Similarly, the number of OSCE stations is also chosen to allow thorough and valid assessment. 7 To ensure reliability, high standards are employed in the generation of test material, including the use of trained writers from a variety of stakeholder backgrounds, and in the assessment of the validation groups.
Workplace-based assessment
While the Tests of Competence test that a doctor can demonstrate their competence outside of the workplace, the most realistic assessment of a doctor's performance should aim to look at what the doctor actually does in observed practice. 8 This is the goal of the workplace-based assessment component. It comprises a site visit, review of case-notes, case-based discussion, third-party interviews and, if possible, a direct observation of the doctor's practice. It is carried out in addition to the Tests of Competence to provide added qualitative information about performance.
Reporting findings
The final written report is compiled to a template by the assessment team. It describes the Tests of Competence and workplace-based assessment events in detail and compares the doctor's performance in the Tests of Competence both to that of a group of peers and to the independent standard set score derived by the Angoff method. Data is presented in this way to the GMC, which takes all the available evidence into account to make a judgement on whether the doctor's fitness to practise is impaired.
Conclusions
The assessment of poorly performing doctors in the UK is guided by the Fitness to Practise procedures of the GMC. The Tests of Competence have been developed based on sound educational theory and continue to be reviewed to ensure they remain up to date, fair, valid and reliable.
Footnotes
Acknowledgements
This project is funded by the General Medical Council.