To what extent does medical device maintenance pose clinical risk?

Abstract

Healthcare is a risky business and patient safety is afforded a high profile. Although great lengths are taken to ensure medical devices are intrinsically safe, risks can occur when they are introduced into the clinical setting. A risk assessment tool, Medical Device Risk Assessment (MeDRa) was used to collect quantitative data relating to the contributory factors and control measures associated with using medical devices in the clinical setting. The tool utilizes the responses from healthcare professionals to a set of questions about risk as a basis for the risk assessment. The raw data was exported to SPSS for statistical analysis and mathematical modelling. Pearson's Chi-square tests for association and relative risk calculations formed the basis for the analysis. This paper reports on one aspect of the risk assessment relating to the maintenance of medical devices. There are two categories of maintenance: technical and user maintenance. The analysis showed that although there was a high level of conformity in technical maintenance, user maintenance was highlighted as a potential problem. MeDRa identifies potential risk factors that might impact on patient care.

Introduction

We live in an era where patient safety and the quality of healthcare provision are constantly on the minds of both the public and healthcare professionals not least because of the rising number of litigation cases. It is important that this concept is realized in the context of medical device maintenance.

Routine maintenance schedules, wherever undertaken, are usually based on manufacturers' recommendations, but are they realistic? Manufacturers' routine inspection and preventative maintenance schedules are founded on the theoretical failure rates of the constituent components. The decision to follow these schedules is normally justified from a legal perspective; that is, not to follow them would expose the individual and organization to increased liability in any subsequent litigation.¹ However, it is arguable that routine maintenance activity can actually be detrimental.

This paper explores why we rely on manufacturers' schedules and what we could be doing to develop more realistic and effective preventative maintenance work.

Are devices intrinsically safe?

There are three European Directives concerning medical devices:

Active Implantable Medical Device Directive (90/385/EEC);

Medical Devices Directive (93/42/EEC);

In vitro Diagnostic Medical Devices Directive (98/79/EEC).

These directives have been implemented into UK legislation by the Medical Device Regulations,² a statutory instrument which consolidates all existing medical device regulations into a single piece of information, and came into force on 13 June 2002. The majority of medical devices also need to be CE marked, meaning that the manufacturer is confident that the medical device conforms to the essential requirements in the directives and is fit for its intended purpose. There is also a raft of standards relating to medical devices, the principal one being BS EN 60601-1-1 supported by specific standards for particular types of equipment.

The safety of medical devices is overseen by the Medicines and Healthcare products Regulatory Agency (MHRA) whose remit is the regulation of medicines and medical devices and equipment used in healthcare and the investigation of harmful incidents.³

With all these safeguards concerning the design and manufacture of devices, safety is clearly paramount.

Devices in clinical practice

Starting with the hypothesis that devices are intrinsically safe, it would be logical to deduce that any failures are caused by the clinical context in which the device is being used. Risks are situationally and culturally specific in that the skills mix, environment and many other factors impact upon them. In 2006, 7975 incidents involving medical devices were reported to the MHRA, though anecdotally it is well known that there is considerable under-reporting of such incidents.⁴

Nurses are the primary users of medical devices in the direct care of patients. The nursing literature is replete with articles that emphasize the importance of nurses being knowledgeable about and proficient in the use of the medical devices they use in patient care.⁵ While knowing how to set up and operate a medical device is important, often routine maintenance is also required.

There are two categories of maintenance: technical and user maintenance. Routine technical maintenance is often called Planned Preventative Maintenance, but is better described as inspection and functional checking.⁶ Examples of user maintenance include cleaning filters, battery conditioning and calibration, and the procedures for undertaking these activities are normally laid out in the operator's manual.

Risk assessment

To explore the risks associated with medical devices in the clinical context, we devised a self-administered questionnaire comprising 38 questions, each requiring a dichotomous response. Examples include:

CM₁₁: Are MHRA and manufacturer recommendations adhered to for calibration and maintenance?

CM₁₉: Is necessary user maintenance, including calibration, carried out by clinical staff?

The questionnaire, entitled Medical Device Risk Assessment (MeDRa), was computerized and hosted on the hospital intranet site. Ward managers and heads of departments completed the questionnaire for each category of medical device used in the clinical area under their direct supervision. The themes explored in this question set were based on the protocol for the investigation and analysis of clinical incidents but were tailored to medical devices.⁷ The design of this tool has been widely reported.^8–10

There were seven questions relating to device maintenance: three questions about contributory factors that would impact on the risk consequences (CF₁–CF₃), and four questions about control measures which would mitigate risk (CM₉–CM₁₁ and CM₁₉). The raw quantitative data was exported to a statistics package (SPSS^®) for analysis.

It must be recognized that risk assessment responses will be tempered by the individual's assumptions, preconceptions and comprehension. In effect, the responses will be biased by individual professional judgements, the social and organizational culture and both the political and economic climates within the NHS.

Results

The data relate to 885 risk assessments conducted across the 13 directorates of the acute hospital trust, within which are 56 departments. The directorates are distributed across the three hospitals sites, and some directorates have departments in more than one hospital. Two of the hospitals take emergency admissions, medical and surgical, and have A & E provisions. There were no missing data fields as MeDRa prohibited this. Cross-tabulation techniques were employed to explore the relationships and provide a commentary between the variables. This kind of cross-tabulation is also called a contingency table or cross-classification.¹¹

What follows is a detailed description of the findings from the statistical analysis of the questions relating to device maintenance issues.

User calibration and maintenance of medical devices was only carried out in 17.3% (n = 153) instances. However technical maintenance and calibration fared much better, at 94.9% (n = 840).

Table 1 shows that there is no significant relationship between directorates and whether or not they have device maintenance issues (χ² = 17.005, 12df, P = 0.149). However, directorate five of itself is particularly notable, as 40% of respondents indicated one or more risk factors across the suite of questions about Device maintenance.

Table 1

Cross-tabulation of Hospital Directorates against Devmaint01

Part A

			Devmaint01		Total
			0.00	1.00	Total
Directorate	1	Count	137	20	157
		% within Description	87.3%	12.7%	100.0%
		% within Devmaint01	17.9%	16.8%	17.7%
	2	Count	32	4	36
		% within Description	88.9%	11.1%	100.0%
		% within Devmaint01	4.2%	3.4%	4.1%
	3	Count	33	8	41
		% within Description	80.5%	19.5%	100.0%
		% within Devmaint01	4.3%	6.7%	4.6%
	4	Count	107	21	128
		% within Description	83.6%	16.4%	100.0%
		% within Devmaint01	14.0%	17.6%	14.5%
	5	Count	9	6	15
		% within Description	60.0%	40.0%	100.0%
		% within Devmaint01	1.2%	5.0%	1.7%
	6	Count	60	6	66
		% within Description	90.9%	9.1%	100.0%
		% within Devmaint01	7.8%	5.0%	7.5%
	7	Count	43	3	46
		% within Description	93.5%	6.5%	100.0%
		% within Devmaint01	5.6%	2.5%	5.2%
	8	Count	48	8	56
		% within Description	85.7%	14.3%	100.0%
		% within Devmaint01	6.3%	6.7%	6.3%
	9	Count	8	1	9
		% within Description	88.9%	11.1%	100.0%
		% within Devmaint01	1.0%	.8%	1.0%
	10	Count	52	9	61
		% within Description	85.2%	14.8%	100.0%
		% within Devmaint01	6.8%	7.6%	6.9%
	11	Count	159	18	177
		% within Description	89.8%	10.2%	100.0%
		% within Devmaint01	20.8%	15.1%	20.0%
	12	Count	34	6	40
		% within Description	85.0%	15.0%	100.0%
		% within Devmaint01	4.4%	5.0%	4.5%
	13	Count	44	9	53
		% within Description	83.0%	17.0%	100.0%
		% within Devmaint01	5.7%	7.6%	6.0%
Total		Count	766	119	885
		% within Description	86.6%	13.4%	100.0%
		% within Devmaint01	100.0%	100.0%	100.0%

Part B
χ ²-tests	Value	df	Asymp. Sig. (2-sided)
Pearson χ ²	17.005*	12	0.149
Likelihood ratio	14.815	12	0.252
Linear-by-linear association	0.280	1	0.597
Valid cases (n)	885

Three cells (11.5%) have expected count less than 5. The minimum expected count is 1.21

The five workgroups indicated to varying degrees (χ² = 8.357, 4df, P = 0.080) that there were risk issues relating to device maintenance, although there is no significant statistical association between workgroups and maintenance. However, nursing support workers were twice as likely (23.9%, RR = 1.90, 95%CI 1.04–3.46) to indicate risk issues relating to device maintenance questions than other work groups (Table 2). Nursing support workers tend to use ‘low-tech’ devices that are often neglected for maintenance in favour of the more complex devices. It could be argued that the greater proliferation of such devices, for example electric profiling bed frames, offer a higher risk than limited numbers of complex devices because of their frequency of use and the lack of maintenance. Furthermore, this could be construed as a training issue or clinical pressures on an already over-burdened workforce. Further audits being undertaken at present may disentangle some of these issues.

Table 2

Cross-tabulation of workgroup against Devmaint01

Part A

			Devmaint01		Total
			0.00	1.00	Total
Workgroup	Consultant/surgeon	Count	46	4	50
		% within used1	92.0%	8.0%	100.0%
		% within Devmaint01	6.0%	3.4%	5.6%
	Medic	Count	69	9	78
		% within used1	88.5%	11.5%	100.0%
		% within Devmaint01	9.0%	7.6%	8.8%
	Qualified nurse/midwife	Count	532	82	614
		% within used1	86.6%	13.4%	100.0%
		% within Devmaint01	69.5%	68.9%	69.4%
	Nursing support workers	Count	51	16	67
		% within used1	76.1%	23.9%	100.0%
		% within Devmaint01	6.7%	13.4%	7.6%
	Allied professions	Count	68	8	76
		% within used1	89.5%	10.5%	100.0%
		% within Devmaint01	8.9%	6.7%	8.6%
	Total	Count	766	119	885
		% within used1	86.6%	13.4%	100.0%
		% within Devmaint01	100.0%	100.0%	100.0%

Part B
χ ²- Tests	Value	df	Asymp. Sig. (2-sided)
Pearson χ²	8.347*	4	0.080
Likelihood ratio	7.591	4	0.108
Linear-by-linear association	1.194	1	0.275
Valid cases (n)	885

0 cells (.0%) have expected count less than 5. The minimum expected count is 6.72

The purpose for which a device is used and device maintenance were not significantly associated (χ² = 0.850, 1df, P = 0.357). Device maintenance issues were not specific to the purpose of the device (Table 3).

Table 3

Cross-tabulation of purpose against Devmaint01

Part A

			Devmaint01		Total
			0.00	1.00	Total
Purpose	Diagnosis	Count	179	33	212
		% within purpose	84.4%	15.6%	100.0%
		% within Devmaint01	23.4%	27.7%	24.0%
	Treatment/therapy	Count	587	86	673
		% within purpose	87.2%	12.8%	100.0%
		% within Devmaint01	76.6%	72.3%	76.0%
Total		Count	766	119	885
		% within purpose	86.6%	13.4%	100.0%
		% within Devmaint01	100.0%	100.0%	100.0%

Part B
χ²-Tests	Value	df	Asymp. sig. (2-sided)	Exact sig. (2-sided)	Exact sig. (1-sided)
Pearson χ²	1.076*	1	0.300
Continuity correction^†	.850	1	0.357
Likelihood ratio	1.046	1	0.306
Fisher's exact test				0.300	0.178
Linear-by-linear association	1.075	1	0.300
Valid cases (n)	885

0 cells (.0%) have expected count less than 5. The minimum expected count is 28.51

^†

Computed only for a 2 × 2 table

Conclusion

Patterns of risk or, when identified, specific risks, were used to inform the corporate risk register, for example training issues or staffing levels.¹² This risk assessment system has subsequently been web-hosted under the name Medarae™ and is currently undergoing an extensive regional evaluation. This research has identified that maintenance is a causal factor in organizational risk. However, to establish how this impacts on clinical risk will require further work to identify if there is a correlation between maintenance shortcomings and the prevalence of clinical incidents. A particular issue has been identified in user maintenance, which appears to be linked in some degree to nursing support workers rather than qualified staff. What must be established now, perhaps through specific audits, is why user maintenance is not being undertaken and whether it affects device reliability. If the lack of user maintenance is not affecting patient safety then can we also reduce technical maintenance, which as already established, is for the most part of inspection and functional checking rather than true preventative maintenance? Looking at this holistically, the cost savings, by reducing the time spent performing maintenance, could be re-directed to improving patient care. Perhaps we should cast aside the manufacturers' guidance and develop our own strategies out of real experience – after all, it worked for the aviation industry which has an enviable safety pedigree.

References

Medicines and Healthcare Products Regulatory Agency. Managing Medical Devices DB2006(05). London: MHRA, 2006

Medical Device Regulations. Statutory Instrument No. 618. London: The Stationery Office, 2002

Medicines and Healthcare Products Regulatory Agency. Medicines and Medical Device Regulation: what you need to know. London: MHRA, 2007

Medicines and Healthcare Products Regulatory Agency. Adverse Incident Reports 2006 DB2007(02). London: MHRA, 2007

McConnell

. How and what staff nurses learn about the medical devices they use in direct patient care. Res Nurs Health 1995; 18: 165–72

Cook

. Preventative maintenance on infusion devices may lack effectiveness. SCOPE 2006; 15: 4–7

Vincent

, Taylor-Adams

, Chapman

, A Protocol for the Investigation and Analysis of Clinical Incidents. London: Clinical Risk Unit, University College London and ALARM, 1999

Brown

. Prevention is better than cure in medical device risk. J Eng Tech 2003; 6: 26–7

Brown

. Finding the hidden risks with medical devices: a risk profile tool. Qual Primary Care 2004; 12: 135–8

10.

Brown

. Identifying risks using a new assessment tool: the missing piece of the jigsaw in medical device risk assessment. Clinical Risk 2007; 13: 56–9

11.

Bland

. An Introduction to Medical Statistics. 3rd edn. Oxford: Oxford University Press, 2000

12.

Brown

. Learning out of a Piranha-infested pool: a corporate risk register. Clinical Risk 2006; 12: 226–8