Knowledge of Clostridium difficile infection among UK health-care workers: development of a knowledge assessment tool

Abstract

Doctors’ knowledge provides the basis to support good practice in infection prevention and control. However, there exists a paucity of validated knowledge assessment tools that can be reliably employed to identify poor knowledge levels of Clostridium difficile infection (CDI) within populations of doctors, preventing the effective identification of knowledge deficiencies and focused targeting of educational interventions. Here, we describe a development process to validate a novel CDI knowledge assessment tool for doctors. Two previously published CDI knowledge questionnaires were amalgamated to produce a combined questionnaire. Content was further evaluated by a panel of CDI experts, producing the ‘Lothian’ questionnaire. These questionnaires were tested in control populations comprising either infection control nurse (ICN) specialists or non-clinically trained individuals, and a cohort of medical staff. We compared the efficacy of the ‘Lothian’ questionnaire against that of previous questionnaire reports. We found that all of the questionnaires studied significantly discriminated between non-clinical and clinical populations (ICNs and medical staff) (P < 0.001) and had similar levels of sensitivity and specificity in discrimination between these targeted populations. This study describes the development of a robust CDI knowledge assessment tool that can be used to evaluate knowledge levels among doctors, compare populations and assist the targeting of educational interventions and plot trends following such interventions.

Keywords

Clostridium difficile knowledge staff questionnaire infection

Introduction

Clostridium difficile infection (CDI) is the most common infectious cause of health-care-associated diarrhoea.¹ The consequences of CDI are of great importance to both the individual and the health-care institution, as it entails a higher risk of additional infections, longer length of stay and potential ward closures in order to control outbreaks.^2–5 Symptomatic CDI can increase patient mortality rates by 10 times or more⁵ and surgical management, although required in only a minority of cases, has a grim prognosis with mortality rates ranging from 46% to 67%.^6,7 Estimates suggest that CDI costs an average-sized district general hospital £400,000 and consumes more than 2000 lost bed days annually.⁸ Recent studies have also demonstrated that the excess treatment cost for CDI patients over the four-year study period was £20,000,000 (€22,500,000).⁹

A wide range of infection control measures have been introduced to address levels of CDI. While few have been assessed by randomized controlled trials, those with the strongest evidence base for CDI prevention include antimicrobial stewardship, glove use and hand hygiene (utilizing soap and water) (reviewed in Hsu et al.¹). Such diverse infection control interventions require a high level of health-care worker (HCW) knowledge to allow early recognition of risk factors and clinical features, appropriate investigations to be performed and subsequent management instituted in accordance with CDI guidelines and best practice.

The contemporary assessment of English National Health Service (NHS) CDI rates¹⁰ suggests that, while reductions in incidence have occurred, the above measures are only partially effective. Effective infection control programmes have been shown to reduce infection trends,¹¹ but such measures may be more successful if supported by appropriate educational programmes targeting doctors who possess poor knowledge levels.¹²

A robust CDI knowledge assessment tool could have many uses, including better identification and assessment of an individual's need for education, monitoring of the effectiveness of interventions and identifying trends and, finally, allowing comparisons of knowledge levels in different HCW populations, both locally and nationally.¹³

Recent reports confirm a lack of knowledge of CDI among HCWs,^14,15 but such studies may be undermined by the limited translational relevance they possess; the assessment tool employed within each study had been utilized in discrete HCW populations without repetition or having undergone a validation/development process. Furthermore, neither specify an appropriate level of knowledge or ‘pass’ grade. In addition, the authors of these studies have recently identified inaccuracies within the content of each other's assessment tools in the scientific literature.¹⁶ All of these factors compound the difficulties of researchers and/or infection control specialists who may aim to use a published assessment tool to assess doctors’ knowledge of CDI for their own aims.

Given the previous controversies surrounding such assessment tools and in order to better assist in the provision of a robust assessment tool for use in the above aims, we developed a novel knowledge assessment tool for CDI knowledge among doctors through design of a validation process to confirm acceptability, accuracy and translational ability.

Methods

Previously published questionnaires by Aroori et al.¹⁴ and Tsagkaraki et al.¹⁵ were used to produce our assessment tools. All content was initially included; however, evaluation of the combined questionnaire identified statements that were duplicated or highly similar in both original questionnaires. For the purposes of design, these statements were amalgamated into one statement in order to produce the combined assessment tool. Additionally, statements were reformatted from their original standing order in source questionnaires and grouped into overall subject area sections in the combined questionnaire.

Content was then evaluated by a panel of five CDI experts (see Acknowledgements). Panel members were blinded to the origin of individual content statements and answers. They evaluated both the accuracy of each questionnaire statement and whether the statements represented an evaluation of reasonable expectations of basic CDI knowledge in doctors. Following this process, statements that were not felt to agree with the above provisions by a majority of panel experts were removed from the combined questionnaire to form a ‘Lothian’ questionnaire assessment tool (Table 1).

Table 1

The combined assessment tool for CDI knowledge

No.	Question	Section	Correct response [Annals], (JHI)	Source	Panel
1	What type of organism is C. difficile?	Organism		An	Y
	Anaerobic bacillus		[X]		Y
	Gram positive coccus		–		Y
	Gram negative bacillus		–		Y
	Fungus		–		Y
2	What percentage of normal healthy adults carry toxin-producing C. difficile in their gut flora?	Organism		An	Y
	None		–		Y
	5%		[X]		Y
	15–70%		–		Y
	100%		–		Y
3	What is the difference between the C. difficile responsible for the outbreaks of more severe C. difficile disease that have occurred in North America and the UK and the strain responsible for the outbreaks in the 1980s and 1990s?	Organism		An	N
	a It produces a greater amount of cytotoxins A and B		–		Y
	b It carries additional clostridial toxin		–		Y
	c It has increased sporulation capacity		–		N
	d Fluoroquine resistance		–		Y
	e All of the above		[X]		Y
4	Risk factors for developing C. difficile–associated disease (CDAD) (i.e. colitis, diarrhoea) include:	Organism			Y
	a Gastrointestinal surgery		True	JHI	Y
	b Prolonged hospital stay		True	An	Y
	c Clindamycin		True	JHI	Y
	d Foreign travel		False	JHI	Y
	e Clinically significant co-morbidities (i.e. pregnancy, diabetes, obesity)		True	JHI	Y
	f Recent chemotherapy/immunosuppressants		True	JHI/An	Y
	g Proton pump inhibitors		True	JHI	Y
	h Usage of antacids and stool softeners		True	An	Y
	i Ampicillin		True	JHI	Y
	j Age > 65/advanced age		True	JHI/An	Y
	k β-Blockers		False	JHI	Y
5	Reducing the quantity of broad spectrum antibiotics prescribed is unlikely to reduce the incidence of CDAD because of ongoing transmission of C. difficile spores	Organism	False	JHI	Y
6	Transmission of C. difficile is by the faecal-oral route	Organism	True	JHI	Y
7	Only those who come into contact with the patient's stools are able to transfer the microorganism	Organism	False	JHI	Y
8	By contrast to inhospital CDAD, community acquired CDI is not a frequent issue	Organism	True	JHI	Y
9	Chlorine-containing detergents are superior to non-chlorine-containing ones in reducing the incidence of CDAD	Organism	True	JHI	Y
10	Deep cleaning has not been shown to alter the number of spores remaining on environmental surfaces	Organism	False	JHI	Y
11	The length of hospital stay is not relevant to the percentage of patients colonized with C. difficile	Organism	False	JHI	Y
12	Which of the following antibiotics are most frequently cited causes of CDI?	Organism		An	Y
	a Cephalosporins		–		Y
	b Aminopenicillins		–		Y
	c Fluoroquinolones		–		Y
	d Clindamycin		–		Y
	e Cephalosporins aminopenicillins and clindamycin		–		Y
	f All of the above		[X]		Y
13	What is the incubation period of CDAC?	Symptoms			N
	<2 days		–	An	N
	2 days–8 weeks		[X], (7 days)	JHI/An	N
	8–10 weeks		–	An	N
	10 weeks		–	An	N
14	Antibiotic-related diarrhoea that occurs in hospital is due to C. difficile	Symptoms	False	JHI	Y
15	Bloody diarrhoea suggests a diagnosis of CDAD	Symptoms	False	JHI	Y
16	CDAD is usually accompanied by a fever	Symptoms	True	JHI	Y
17	CDAD can be fatal	Symptoms	True	JHI	Y
18	CDAD confers lifelong immunity	Symptoms	False	JHI	Y
19	Which of the following conditions may be due to CDI	Symptoms		An	Y
	a Pseudomembranous colitis		– (True)	JHI/An	Y
	b Antibiotic-associated colitis		–	An	Y
	c Antibiotic-associated cases of diarrhoea		–	An	Y
	d Toxic megacolon		–	An	Y
	e All of the above		[X]	An	Y
20	What is the gold standard test for the identification of pathogenic CDI?	Diagnosis		An	Y
	a Cytotoxin assay for detecting cytotoxin B		[X]	An	Y
	b ELISA		–	An	Y
	c Latex agglutination test		–	An	Y
	d Stool culture		– (False)	JHI/An	Y
	e All of the above		–	An	Y
21	What is the single most successful measure in reducing the number of cases of symptomatic CDI?	Treatment		An	Y
	a Antibiotic restriction		[X]	An	Y
	b Using gloves when examining patients		–	An	Y
	c Washing hands		–	An	Y
	d Prophylactic use of metronidazole		–	An	Y
	e All of the above		–	An	Y
22	What is the most effective way of preventing transmission of C. difficile?	Treatment		An	Y
	a Hand washing with soap and water		[X]	An	Y
	b Rubbing hands with alcohol-based products		– (False)	JHI/An	Y
	c Wearing gloves		–	An	Y
	d All of the above		–	An	Y
23	What is the first line antibiotic choice for the treatment of mild CDAD	Treatment		An	Y
	a Oral vancomycin		– (False)	JHI/An	Y
	b Oral metronidazole		[X]	An	Y
	c IV vancomycin		–	An	Y
	d IV metronidazole		–	An	Y
	e Any of the above		–	An	Y
24	The treatment choice in patients who failed to respond to oral metronidazole in CDAD is:	Treatment		An	Y
	a Addition of oral vancomycin		–	An	Y
	b Switch to oral vancomycin		[X]	An	Y
	c Switch to IV metronidazole		–	An	Y
	d Addition of IV vancomycin		–	An	Y
	e Switch to IV metronidazole and oral vancomycin		–	An	Y
25	Recurrence of CDAD after effective treatment is infrequent	Treatment	False	JHI	Y
26	After the diarrhoea has ceased the patient can be discharged	Treatment	True	JHI	N
27	It is necessary to treat all cases of CDAD	Treatment	False	An	Y

To provide control populations with assumed low and high levels of CDI knowledge, respectively, and to compare previous questionnaires, the combined questionnaire was administered to a population of non-clinical scientific and administrative staff at the Medical Research Council Human Genetics Unit (MRC HGU), Edinburgh, and to a population of Infection Control Nurse (ICN) specialists. The non-clinical population was approached randomly at their place of work on the day of sampling, without prior notification, and asked to complete the questionnaire immediately online. ICN specialists working within NHS in Scotland were contacted directly by email through the Scottish branch network of the Infection Prevention Society (IPS). Subjects were asked to complete an online version of the combined questionnaire tool. All of the non-clinical subjects who were approached (n = 39) participated in the study; out of the 107 Scottish members of the IPS, 24 (22.4%) responded to the email request.

The questionnaire was trialed in a small cohort of Medicine of the Elderly doctors attending the April 2010 spring meeting of the British Geriatric Society (BGS) at the Edinburgh International Conference Centre, Edinburgh. Questionnaires were printed and included in the delegates’ welcome packs, and verbal announcements and reminders by the conference chair encouraged the attendees to complete the questionnaires individually and anonymously and return them to a marked collection point. Responses were received from 36 (7.8%) of the 461 delegates who attended.

For each respondent, we calculated the percentage of correct answers for each questionnaire. A missing response was considered an incorrect answer. In order to determine if there was a difference in the mean percentage of correct answers in the three groups, an analysis of variance (ANOVA) has been performed. As an ANOVA reveals whether there are differences between groups but not where those differences are, Tukey's test has been used to test pair-wise differences.

In order to identify whether there is an optimal cut-off in the percentage of correct answers within a questionnaire for distinguishing between groups, a receiver operator characteristic (ROC) curve has been produced and is presented along with area under the curve (AUC) and 95% confidence intervals (CIs) for the AUC. Statistical significance was accepted at P = 0.05.

This study was approved by the Lothian Regional Ethics Committee. The permissions for the various arms of this study were obtained from the relevant organizations (see Acknowledgements).

Results

Population demographics

Control populations were made up of 39 non-clinical scientific and administrative staff, 24 ICNs and 36 doctors attending the BGS conference in April 2010. Of the delegates attending BGS, 5.6% (n = 2) did not list a grade. Of the 34 who did, 44% (n = 15) were specificity registrars (SpRs), 41% (n = 14) were consultants, 9% (n = 3) were associate specialists and the remaining 6% (n = 2) were general practitioners (GPs).

Evaluation of statements by an expert panel

The combined questionnaire comprised 37 individual statements. Nine statements were sourced from the Aroori questionnaire and 21 from the Tsagkaraki questionnaire. In addition, seven statements on the combined questionnaire were generated as an amalgamation of two similar statements from both source questionnaires.

Three statements were deemed inappropriate by a majority of the expert panel: statement 3c (describing sporulation capacity of C. difficile; panel members did not feel this was a basic CDI knowledge requirement for doctors), statement 26 (describing discharge of a patient after diarrhoea had ceased; panel felt that more information was required before this could be answered with certainty, with a presumed assumption in this case that CDI had been the main cause of admission) and statement 13 (regarding C. difficile incubation; panel did not think that the answer to this was as clear cut within the current scientific literature).

In agreement with published criticism, the panel had difficulties with an Aroori question (question 19),¹⁶ but thought that the question was appropriate if the answer was changed to ‘all of the above’, rather than as formatted in the source questionnaire. Similarly, while the panel felt that question 20 regarding laboratory diagnostics was appropriate, they recognized previous concerns¹⁶ that this may be dependent on local practices and may require modification depending on where the questionnaire was being used. The panel also advised that future questionnaires should have slight modifications, including questions 8 and 16, which a majority of the expert panel felt would be improved by alternate wording that would clarify the statements.

Questions 3, 13 and 26 were therefore removed from the combined questionnaire to form a ‘Lothian’ questionnaire. All other statements were maintained in their original format in order to allow a representative comparison of source questionnaires through analysis of population responses of the combined questionnaire.

Evaluation of content in control populations

The percentage of correct answers gained per population for individual statements in the combined questionnaire is illustrated in Figure 1. As demonstrated, the trend for a higher percentage of correct answers achieved by ICNs and doctors is maintained throughout most statements; however, there are a number of areas of interest. In four statements (statements 8, 16, 20 and 21) less than 50% of ICNs got the answer correct. In addition, despite an absence of clinical training, 80% of non-clinical staff were able to answer the following statements correctly (statements 4a, 4b, 4f, 4k, 7, 17 and 18). These questions may therefore be appropriate to include in a questionnaire but may be non-discriminatory between populations.

Figure 1

Percentage of correct answers per population for each statement within the combined questionnaire

While there was no overall statistical difference between the percentage of correct answers scored by doctors and ICNs, doctors tended to answer more questions correctly in the diagnostics and treatment sections than ICNs. The greatest difference between non-clinical and clinical/ICN population CDI knowledge levels was also to be found in the diagnostics and treatment sections (Figure 2).

Figure 2

Comparison of population knowledge levels per questionnaire section (± standard error of the mean)

The mean percentage of correct answers achieved by doctors were 79.0% for consultants, 75.9% for registrars/trainees and 71.8% for others. Owing to the small size of this pilot population, these differences were not significant.

Comparison of questionnaires

The distribution of the percentage of correct responses from each population per questionnaire is illustrated in Figure 3. There is a significant difference in means across the groups (P < 0.001) for all questionnaires. In all questionnaires, assessments could statistically discriminate between non-clinical staff and doctors (P < 0.05) and non-clinical staff and ICNs (P < 0.05). No tested questionnaire could discriminate between ICN and doctor populations (P > 0.05).

Figure 3

Boxplot of overall questionnaire percentage marks achieved per population

In order to establish whether there is an appropriate ‘pass mark’ to discriminate between non-clinical staff and ICN/doctor populations, an ROC curve was generated (Figure 4). There was no statistically significant difference between the Lothian and combined questionnaire AUCs (0.95; 95% CI, 0.91–0.99 and 0.96; 95% CI, 0.93–0.99, respectively) compared with the Aroori and Tsagkaraki questionnaires (0.93; 95% CI, 0.88–0.98 and 0.95; 95% CI, 0.91–0.99, respectively). A Lothian questionnaire pass mark of 63.2% (21 correct answers out of 34) allows the ability to distinguish between non-clinical staff and ICN/doctor populations with a sensitivity of 0.95 and specificity of 0.795.

Figure 4

Receiver operator characteristic curve to determine cut-off point between non-clinical and ICN/doctors for each questionnaire

Discussion

This study aimed to develop and/or determine the most robust assessment tool to identify doctors who possess low levels of CDI knowledge.

To date, those who used a questionnaire to assess CDI knowledge had difficulty sourcing a validated assessment tool. This was due to the absence of any control populations, formal validation processes, repetition or acceptability testing of some of the previously published questionnaires. In addition, critics, including authors of such papers, have identified weaknesses in the content of previous questionnaire reports.

Here, we confirm that previously published questionnaires, irrespective of the above problems, will statistically discriminate between non-clinical and clinical/specialist populations. The process that we have undertaken has also provided a novel questionnaire assessment tool that has been evaluated by a panel of experts and undergone evaluation in control populations to confirm that the content is both accurate and acceptable in the evaluation of a population of doctors.

The statements were initially reviewed by an expert panel. Through this review process a number of statements were revealed to have factual difficulties, to have local practice variation or to require rewording to improve the clarity of the statements. These concerns were borne out in testing within the control populations when some statements generated answer patterns that were unexpected in the positive control population. The questions that caused these issues included questions 8 and 16, both of which the experts had recommended would require rewording, and questions 20, which was felt to be dependent on local practices. Future questionnaires may benefit from modifications to the statement content, with particular attention to local circumstances. The science and epidemiology of CDI are constantly changing; therefore, variation in policy or data by region or in future knowledge gain requires future questionnaires to be modified to reflect current and local knowledge. The difficulties of accounting for local circumstances or variations in practice have also been a feature of previous infection control knowledge questionnaire studies.¹²

The small proportion of responses received from those at the geriatric conference was disappointing and is reflective of a response rate in voluntary questionnaire studies. While drawn from a wide national base of attendees, results could also reflect a bias within a geriatrician population towards those who attend conferences. In addition, there may well be a bias towards those who have greater knowledge of CDI and thus preferentially returned the questionnaire. This could explain why no questionnaire was able to discriminate statistically between ICNs and geriatricians. Therefore, no general conclusions in relation to knowledge levels of geriatricians as a population could be drawn without repeat testing in a larger population.

There was a small difference in methodology utilized between the different populations tested. Non-clinical and ICN populations completed an electronic form of the questionnaire, while the geriatricians completed a paper hard-copy. While all questionnaires were of identical content, we are unable to describe whether a different form of questionnaire could have any influence on a participant's score.

Future work, testing an updated questionnaire in a large cohort of hospital doctors of varying grades, would help further validate this CDI knowledge assessment tool.

Conclusions

Here, we describe the development of a robust CDI knowledge assessment tool containing previously peer-reviewed content, evaluated through testing in control populations and reviewed by a panel of experts. This questionnaire significantly discriminates between populations with assumed high and low levels of knowledge of CDI with quantification of sensitivity and specificity across a pass-mark range. This has relevance to those aiming to assess individual levels of CDI knowledge, compare levels within populations, identify risk factors for poor knowledge levels or evaluate trends over time and/or following interventions.

Footnotes

Acknowledgements

The authors thank the organizing committee of the BGS 2010 Edinburgh Spring Meeting, the staff and management of the MRC HGU, Edinburgh and Rona Broom and the Scottish Members of the Infection Prevention Society. In addition, the authors thank the panel of expert reviewers, which also included Dr Pota Kalima, Professor John Starr and Dr Karen McSween. This research was presented as a poster at the Federation of Infection Societies meeting in Edinburgh in November 2010.

References

Hsu

, Abad

, Dinh

, Safdar

Prevention of endemic healthcare-associated Clostridium difficile infection: reviewing the evidence.

Am J Gastroenterol 2010; 105: 2327–39.

Pepin

, Valiquette

, Alary

ME.

Clostridium difficile-associated diarrhea in a region of Quebec from 1991 to 2003: a change in pattern of disease severity.

CMAJ 2004; 171: 466–72.

McFarland

LV.

Diarrhea acquired in the hospital.

Gastroenterol Clin North Am 1993; 22: 563–77.

Eriksson

, Aronsson

. Medical implications of nosocomial infection with Clostridium difficile. Scand J Infect Dis 1989; 21: 733–4.

Rodrigues

, Brady

, Rodrigues

, Graham

, Gibb

AP.

Clostridium difficile infection in general surgery patients; identification of high-risk populations.

Int J Surg 2010; 8: 368–72.

Hermsen

, Dobrescu

, Kudsk

KA.

Clostridium difficile infection: a surgical disease in evolution.

J Gastrointest Surg 2008; 12: 1512–7.

Chan

, Kelly

, Helme

, Gossage

, Modarai

, Forshaw

Outcomes following colectomy for Clostridium difficile colitis.

Int J Surg 2009; 7: 78–81.

Spencer

RC.

Clinical impact and associated costs of Clostridium difficile-associated disease.

J Antimicrob Chemother 1998; 41 (Suppl. C): 5–12.

Reddy

, Taori

, Poxton

IR.

Changes in laboratory and clinical workload for Clostridium difficile infection from 2003 to 2007 in hospitals in Edinburgh.

Clin Microbiol Infect 2010; 16: 340–6.

10.

Health Protection Agency. Quarterly Analyses: Mandatory MRSA Bacteraemia and CDI in England (up to January-March 2010). London: Health Protection Agency, 2010

11.

Weiss

, Boisvert

, Chagnon

Multipronged intervention strategy to control an outbreak of Clostridium difficile infection (CDI) and its impact on the rates of CDI from 2002 to 2007.

Infect Control Hosp Epidemiol 2009; 30: 156–62.

12.

Brady

, McDermott

, Cameron

, Graham

, Gibb

AP.

UK healthcare workers’ knowledge of meticillin-resistant Staphylococcus aureus practice guidelines; a questionnaire study.

J Hosp Infect 2009; 73: 264–70.

13.

Brady

, Rodrigues

, Gibb

AP.

Clostridium difficile knowledge in healthcare workers; conclusions in the absence of broader evaluation.

Ann R Coll Surg Engl 2010; 92: 630

14.

Aroori

, Blencowe

, Pye

, West

Clostridium difficile: how much do hospital staff know about it?

Ann R Coll Surg Engl 2009; 91: 464–9.

15.

Tsagkaraki

, Sampaziotis

, Cooke

, Gkrania-Klotsas

Assessing staff knowledge about Clostridium difficile diarrhoea.

J Hosp Infect 2009; 72: 81–3.

16.

Cooke

, Gkrania-Klotsas

Clostridium difficile infection: more known than is realised.

Ann R Coll Surg Engl 2010; 92: 89–90.