Abstract
Our previous three articles reviewed the applied knowledge test (AKT) offering insight, advice and guidance from an examiner’s and trainee’s perspective. We now move on to the clinical skills assessment (CSA) and intend to follow similar themes with advice from both the Clinical Lead for the CSA and also a trainee who has recently undertaken the assessment. In this first article, Dr Nicki Williams provides an insider’s overview of the exam. Next month we will look at the application process, how to prepare for the CSA and examiner top tips.
The MRCGP Clinical Skills Assessment (CSA)
Delivery of the CSA exam
The CSA takes place only in the RCGP in London, Euston. Morning and afternoon sessions of approximately 3 hours allow a capacity of 78 candidates per day. Exam days are grouped into ‘diets’ of varying length to reflect demand. Diets generally occur monthly between October and May to allow maximum candidate choice. Constraints to exam timings include the need to book the necessary rooms a year in advance without details of demand, but we do our best! Exam results are sent via ePortfolios about a week after the final exam day.
Question format
Each case in the CSA represents a timed 10-minute consultation, in which the integrated clinical and communication skills of the candidate are tested. Candidates sit in their designated consulting room. At the start of the consultation, the ‘patient’ knocks and enters, and the consultation begins. The examiner enters the room at the same time, and sits silently, out of direct eye contact, to observe the consultation within the room. After 10 minutes, a bell or buzzer sounds and the patient and examiner exit. After an interval of approximately 2 minutes, the process is repeated. Half way through the three-hour exam there is a comfort break with drinks and a snack offered. For telephone cases, candidates are asked to call the patient from the phone on their desk. For house visits you will be escorted to and from the visit room. Candidates with disabilities needing reasonable adjustments should notify us in advance through the system described on the website.
Examiners
Examiners are working GPs, often trainers, from around the UK. They offer a minimum of 10 days examining per year. In most cases that time includes annual leave and weekend working. Examiners are selected by a competitive process and trained repeatedly. Their performance, and that of the role players, is regularly quality assured. Ours is one of the most demanding programmes of any Medical Royal College and we constantly strive to drive up standards and ensure both reliability and fairness.
How are the cases produced?
CSA cases are written by a sub-group of examiners, trained to both write and scrutinise cases. We have over 700 cases blueprinted across all MRCGP curriculum areas. New cases are created to fill emerging themes or gaps in the blueprint.
Cases are written in four sections:
Instructions for the role player, to standardise performance Instructions to the examiner, including a marking scheme and notes to aid ‘calibration’ Case notes for the candidates, which are the only notes to appear on the iPad, so the only section that you will view; typically these are kept to an essential minimum (as well as the name and age of the patient, you may be given details of current medications, relevant past history, recent test results (with normal values) and consultants letters) Supporting evidence: A summary of current guidelines, e.g. National Institute for Health and Care Excellence (NICE)/Scottish Intercollegiate Guidelines Network (SIGN); any four-nation difference is highlighted in this section
Cases are scrutinised before every case (re)appearance, hence all information remains up to date and appropriate. This quality assurance process also takes into consideration any examiner feedback from the last performance.
Construction of a CSA ‘palette’
Each of you will experience a combination (palette) of 13 cases at CSA. That day’s palette will appear on all three circuits and for both morning and afternoon sessions. On different days there are different palettes, and much effort is spent in ensuring their balance.
Cases are classified in many ways, e.g. curriculum area, age group of the patient, social class. These ‘tags’ are then used to ensure that each palette samples widely across the curriculum. Thus, for instance:
No curriculum area is covered more than once in the same palette Every palette is likely to have at least one case involving a child, and one involving an older adult: there is a spread of ages and social class At least one case of the 13 will significantly involve a diversity issue: age, disability, gender reassignment, race, religion or belief, sex, sexual orientation, marriage and civil partnership, or pregnancy/maternity Every palette will have a minimum number of cases where a physical examination is expected There will be a maximum of one telephone call or house visit in each palette
… and so on. There are many tags! Even then, a palette may not be perceived as fair. A small team of even more specialised case writers scrutinise the combination that has been created to fit the blueprint, consider the candidate viewpoint and suggest changes. Perhaps one case might represent migraine (neurology) and another sinusitis (ear, nose and throat): but to the candidate, they are both ‘headache’ cases. One is substituted out. We often find that the suggested substitution creates a clash with another case, so an alternative needs to be found. And so on. The process continues until consecutive scrutinisers consider that ‘fairness’ has been achieved.
Calibration
The day’s palette appears simultaneously in three circuits, so for every case there are three examiners paired with three role players (‘patients’). For the first 90 minutes of every exam day, these examiners and role players meet together in a calibration session and role play the case, ensuring that whatever circuit a candidate is placed on, the challenge will be the same.
Marking
Every examiner will see a single case played out 26 times in a day and grades each candidate’s behaviour using the agreed descriptors as practised in the calibration meeting. Marks are awarded in the separate, equally weighted, domains of Data Gathering, Clinical Management and Interpersonal Skills. Each domain has 0–3 marks available, so the maximum number of marks per case is nine, and the total possible for the exam is 13 × 9 = 117.
Standard setting
Standard setting in the CSA relies on the internationally accepted Borderline Group method, which allows the pass mark for the day to alter allowing for any variation in the difficulty of palettes. Having marked, the examiner considers whether the candidate has, overall, demonstrated the level of competence for independent practice. Some performances are classed as ‘borderline’. After allowing for a standard error of measurement, marks for those borderline performances are used to create the pass mark for that day. Note that you do not ‘pass’ every case. Your total score is what matters.
Feedback
Although the CSA is designed as a summative exam, some feedback on performance is offered to both passing and failing candidates. Examiners have a choice of 16 standard feedback statements, grouped either by marking domain or classified as ‘global’. From autumn 2016, feedback to candidates includes the domain scores for every case and also a notification of feedback statements that the examiner marked as being appropriate for that case.
Cost
To sit the CSA exam currently costs £1663. All tax paid on examination fees can be reclaimed from HMRC via normal tax returns. The exam itself, like the AKT, is cost neutral. Fixed costs include room hire (Searcy’s hold the franchise for this) and fees, and accommodation for both role players and examiners. Examiners are working GPs, and, unlike our secondary care examiner colleagues, need backfill fees towards locum cover for their absence.
Following requests from the National AiT committee, we introduced a system in July 2013 to enable new trainees to build a fund over time to contribute toward the cost of their CSA and AKT examination fee payments. This benefit has allowed trainees to make six or seven instalments of £220. The fund built can be used against AKT and CSA fee payments and will be made available as part of the examination booking process for those who opted into this scheme.
The MRCGP is always evolving, so for the most up-to-date information please consult the RCGP website. If you would like to contribute to Crammer’s Corner please contact us at editorialoffice@innovaitjournal.co.uk.