Abstract
UK sexual health services are shifting from hospital-based clinics into primary care, creating a need for high quality clinical sexual health training for non-specialists. Here we describe development, evaluation and costing of a new competency-based training programme, the Sexually Transmitted Infection (STI) Foundation Competency (STIFCompetency) programme, based on the Department of Health's toolkit for delivering more specialized sexual health in primary care. We used an action research paradigm with two iterative cycles. Evaluation was to Kirkpatrick's third level with triangulation of results between trainers and trainees, and different methods, including portfolio evaluation, nominal group technique process, semi-structured interviews, Likert questionnaires and chlamydia testing rates. All 13 primary care clinicians completed the training successfully (median 20 hours) and rated STIFCompetency highly. Trainers needed to reduce their clinical workload to accommodate the training. Average cost per trainee was £1125, reflecting the need for direct observation of competence across a wide range of clinical skills.
INTRODUCTION
Basic elements of sexual health care should be offered in every English general practice. 1,2 The British Association for Sexual Health and HIV (BASHH) previously developed a standardized two-day Sexually Transmitted Infection (STI) Foundation Competency (STIFCompetency) theory course 3 to equip primary care professionals with the relevant knowledge to deliver these basic elements of sexual health care. To date, genitourinary (GU) medicine consultants have delivered around 600 STIF courses to over 23,000 delegates.
More recently, the UK Department of Health (DH) recognized that some primary care providers are being commissioned to provide more complex sexual health services. They defined nine sexual health competencies as essential for a more specialized service: 4 raising the issue of sexual health and/or offering chlamydia screening to men and women aged under 25 years; sexual history taking and management of at-risk groups; genital examination; diagnosis of genital tract infections; HIV testing; use of referral pathways; epidemiology of STIs; partner notification; safer sex advice; and published a ‘Toolkit’ 4 to assist learning, which was recommended to take place within specialist GU medicine services.
However, GU medicine clinicians reported practical difficulties in using the DH toolkit assessment tools, were concerned about their capacity to deliver an educationally onerous programme within very stretched clinical units and were unsure regarding how to appropriately calculate costs. To address this, BASHH, in collaboration with Tower Hamlets Primary Care Trust (PCT), commissioned Barts & The London NHS Trust to develop, pilot, evaluate and cost a new sexual health clinical training programme for primary care clinicians. This would focus on the nine essential competencies required to deliver a more specialized service 4 and adapt the DH assessment tools to better suit patterns of working and patient flow within GUM clinics. Here we report on the development and evaluation of this educational intervention: STIFCompetency.
METHODS
Development and implementation of the training programme
Trainees
We developed and adapted course materials as appropriate and provided each trainee with the following learning materials: DH toolkit; 4 task-based mini-Clinical Evaluation Exercise (mini-CEX) forms based on those in the DH toolkit; teaching time log; learning needs assessment (developed for the course to determine trainees’ baseline self-assessed knowledge, skills and attitudes so that trainers could target training appropriately [available from authors on request]); latest STIF course delegate manual 3 and a digital video disc demonstrating genital examination.
Each trainee was assigned a trainer to act as their overall training supervisor. Trainees were assessed principally by direct observation using the mini-CEX forms, augmented where appropriate with case-based discussion.
Trainers
We appointed STIFCompetency trainers, based on previous teaching experience and level of interest, consisting of medical consultants, associate specialists, specialist registrars, nurse consultants and nurse practitioners from our unit. We then familiarized the trainers with the programme and a medical educationalist provided specific training on the use of mini-CEX forms.
Planning the training
Trainers and trainees planned five initial training and assessment sessions using a session template. Further sessions could be added, according to the trainee's particular needs, until all nine essential competencies were assessed as satisfactory or above. Figure 1 shows the structure of a typical training and assessment session.
Structure of a typical training session
Evaluation of the training programme
The project was divided into two cycles and followed an action research paradigm 5 in which the first cycle was completed by an evaluation that informed the design of the next.
Summary of parameters evaluated, methods used and Kirkpatrick level 6 assessed
*PBR = Payment-by-results (
Trainee semi-structured interviews
We conducted 20-minute tape-recorded interviews with trainees one to two months after each trainee had completed the programme to explore trainees’ perceptions of elements of the STIFCompetency programme and the impact of the programme on their practice in order to evaluate learner reaction and learning gained. 6 Interviews were transcribed verbatim.
Two researchers coded and summarized transcripts. Recurring themes and concepts were identified to make up a thematic framework, which was then applied systematically to the transcripts. Trustworthiness was enhanced by double coding the transcripts, comparison and consensus building.
Costs of training
We subtracted the average number of patients seen by trainers and trainees during a training clinic from our clinic's average number of patients seen per clinician during a routine non-training clinic and then estimated the income lost for the corresponding number of patients.
Change in clinical practice
In order to evaluate change in behaviour after learning 6 we compared the average number of chlamydia tests undertaken in the trainees’ practices (trained practices) in the quarter before the training programme began with the average number of chlamydia tests undertaken in the quarter following the end of the training period. As the training period ran for a total of 24 months, we also determined the corresponding values from all other practices in the PCT (untrained practices) to assess any background change in chlamydia testing over time.
RESULTS
Thirteen trainees (9 general practitioners, 2 nurse practitioners and 2 practice nurses; 10 women and 3 men) from 13 practices were trained in two groups between October 2007 and September 2009. The more experienced were trained first, as determined by the learning needs assessment. All had previously attended an STIF theory course and were already providing some sexual health services. Data are available from 12, as one trainee's paperwork was unavailable for analysis.
Trainees: acceptability and feasibility of the training
Trainee semi-structured interviews
Seven trainees participated in recorded telephone interviews; six transcripts were available for analysis (1 was unintelligible).
Three main themes emerged:
Views on structure and content of the STIFCompetency programme: Trainees were highly satisfied with the programme overall and felt that the competencies were appropriate for general practice. Most trainees liked the continuity of one or two trainers and spoke positively of the one-to-one nature of the training. Some trainees felt they would like more sessions and they all felt that less experienced health-care professionals would need more training time. Case-based discussions were perceived as valuable to explore issues that had not arisen with patients in clinic. A few recommended that future trainees should be encouraged to bring cases for discussion to ensure that the training related to primary care.
Acceptability and feasibility of programme: All trainees found the STIFCompetency programme acceptable and feasible within their current practice and would recommend it to colleagues. Flexibility to arrange sessions when trainees were free rather than be offered set session times was particularly appealing. Trainees felt that the main hurdle to completing training was finding time to attend clinics and coordinating the sessions with their normal work.
Impact on practice: Most of the trainees felt ready to transfer their learning to their clinical setting and most reported that they had fed back their learning to colleagues, either formally or informally. All trainees reported that their clinical practice had changed as a result of STIFCompetency training. Most trainees now see more patients with sexual health problems, are more likely to raise the issue of sexual health with patients, are more likely to examine patients with symptoms of an STI and half of them reported taking more STI tests.
Trainers: acceptability and feasibility of the training
Nominal group technique process 7
Five of the eight trainers attended the nominal group technique (NGT) process. We asked the following two questions:
What were the main hurdles in implementing this pilot?
What changes should be made for the next round of training?
The main hurdles identified were:
Coordinating sessions between trainers and trainees; First version of task-based mini-CEX forms did not work well; Conflict between training and service provision. Trainers to provide a timetable of available training sessions; Re-design mini-CEX forms to follow patient pathway more closely; Make trainers supernumerary.
Their suggested and subsequently implemented solutions were:
Questionnaire surveys
After completion of the first group of trainees, five of the eight trainers completed an anonymous questionnaire (available from authors on request). Their responses supported the findings of the NGT process. Positive outcomes for trainers included ‘greater connectedness with local primary care sexual health’, a better understanding of primary care issues, a sense of achievement, identifying shortcomings in their own knowledge and experience of a one-to-one style of teaching. Trainers preferred not to ‘share’ trainees as this was felt to be confusing for both trainers and trainees, leading to an inefficient use of clinic time.
After completion of the second group of trainees, responses from the trainers’ questionnaire (completed by four out of six trainers) showed increased satisfaction in all areas. All trainers felt that by being supernumerary, the teaching load became feasible within their current job plan.
Duration of training
Median time to achieve competence was 20 hours (range 10–28) equating to five clinic sessions (four hours per session). This was spread over a median of 10.5 weeks (3–43). The trainee who took 43 weeks had 19.5 contact hours, which reflected her practical difficulties in organizing the training. The duration of training appeared to correlate with self-assessed competence at the start of STIFCompetency training. Less-competent trainees needed longer to train (data not shown).
Number of trainers needed
The number of trainers per trainee ranged from one to six, with most trainees having one or two trainers.
Costs of training
Trainers saw fewer patients per clinic when they were training (3–4 patients) than during a standard clinic (7–8 patients). This equates to a loss of income equivalent to four patients (2 new; 2 follow-up) which is £450 per training clinic (2009–2010 payment-by-results [PBR] tariff; excludes market forces factor [MFF]). Approximately 50% of the PBR tariff is attributable to direct patient costs (e.g. swabs or medication). An estimated cost of training for five sessions was therefore £1125 (excluding MFF).
Change in clinical practice
The mean numbers of chlamydia tests taken by trained practices in the quarters before and after the training period were 36 and 53, respectively, (data not available from 2 practices). The mean number of chlamydia tests taken by untrained practices in the same quarters was 18 and 28, respectively (data not available from one practice).
CONCLUSIONS
The STIFCompetency training package provides an intensive, validated and costed method for training and assessing primary care clinicians to provide more specialized sexual health care to DH standards. The methods demonstrate the utility of an action research approach.
Most competency assessments were carried out by only one trainer, which has implications for their reliability. However, the trainees generally reported preferring one or two trainers to several different ones. Trainers saw fewer patients per clinic than usual when they were training and this had financial implications for the GU medicine service. Some trainees found arranging sessions around their work commitments rather difficult and this required trainers to be flexible in their approach. Overall, the programme was rated highly by trainees and clinical practice changed subjectively as a result of the training.
The evaluation of educational interventions is notoriously difficult. 8 We used a wide range of evaluation techniques, triangulating between methods and sources to capture as broad a range of parameters as possible. Although we included a measure of clinical practice change, this was not the primary outcome of the study and we were only able to measure rates of chlamydia testing by practice rather than by a clinician. However, trainees reported feeding back elements of their learning to their colleagues and it may be that this has influenced both individual and practice-level testing behaviour. Over the period of the study, although the number of chlamydia tests performed by trained practices rose, a similar increase also occurred in untrained practices. More refined monitoring of STI testing rates are needed as well as the development of quality outcome measures for sexual health care to assess the impact of this type of training on delivery of sexual health care.
Competency training and assessment for the sexual health skills recommended by DH has costly implications as it demands direct observation of competence across a wide range of clinical skills. Robust programmes of training and assessment such as this are essential for ensuring high-quality safe patient care, especially as it will be delivered in settings with little or no direct specialist supervision. Skilling up the non-specialist workforce will require significant financial investment and this must be considered ahead of service development. PBR income lost is one way of costing such training and assessment programmes but there is clearly potential for more sophisticated costing methods.
The STIFCompetency training package forms a natural extension to the STIF theoretical course and, following an extended pilot in other services, is now available in an expanded form through the BASHH website as STIFIntermediateCompetency (
Footnotes
ACKNOWLEDGEMENTS
We thank Dr Jyoti Dhar, Dr Gail Crowe, Dr Tom Shackleton and Krissie Lucas for their help with this project. We are grateful to all the trainers and trainees who participated in the pilot.