Abstract
Introduction
Medical work environments are known to have a high level of complexity because of such factors as sophisticated technologies and procedures, massive amounts of information, and the need for teamwork (Kohn, Corrigan, & Donaldson, 1999; Weinger & Slagle, 2002). It is thus not surprising that medical teams commit mistakes affecting patient safety (Leape, 2004). Indeed, medical errors, many preventable, are widespread in hospitals (Cao, Stetson, & Hripcsak, 2003; Leape et al., 1991). A crucial standing issue is how relevant information on the sources and causes of medical errors can be best collected and analyzed to guide activities to improve patient safety.
Following the Institute of Medicine (Kohn, Corrigan, & Donaldson, 1999). and the American Hospital Association (1999) publications on medical errors, substantial government funds have been directed to the study of errors and patient safety, with a special emphasis on the study and collection of adverse events, near misses, and errors (Leape & Berwick, 2005). Topics identified as problematic included medication administration, physical layout, data recording and management, work procedures, and medical device design.
The prime source of information for identifying problems and guiding corrective efforts to improve patient safety in hospitals has been accident investigations, which has been complemented by the study and development of reporting systems of adverse events and incidents (Altman, Clancy, & Blendon, 2004; Johnson, 2007). There has been a growing awareness of the problem, increased investment, and emphasis on the study of errors. Nonetheless, there has not been a real change in the overall situation or observable witnessed reduction in the overall number and frequency of errors (e.g., Cullen et al., 1995; Leape & Berwick, 2005).
Gopher (2004) has argued that error and incident reporting systems have four main disadvantages as an information source to unveil the major causes and contributors to errors: (a) low frequency of reporting and representation bias, (b) memory and report biases, (c) absence of comparative activity base levels, and (d) hindsight bias. Because of accountability and legal implications, the overall number and hence the sample size of reports is small and may not represent the nature or actual frequency of such events. For example, Cullen et al. (1995) claim that the majority of medical errors, including 90% of those that actually cause harm, are not reported. Cook, Woods, and Miller (1998) estimate that no more than 5% of all errors are ever reported. This low frequency of reports may also be limited in its content categories and thus may not constitute a representative sample (see also Jha et al., 1998). Interestingly, the efficacy of error reporting does not seem to be much better in the civil aviation community, despite its long tradition and continuing effort to encourage reports, including the use of voluntary nonpunitive systems (Aviation Safety Reporting System, n.d.).
A second factor influencing the quality of error reports are memory biases and possible reporting diversions potentially associated with moral, legal, and financial implications. A third problem is that error and incident reports usually lack proper base rate levels for reference and comparison. Error and event evaluation are most commonly not linked with normative or rate information on the activities within which they occurred; that is, one usually does not know the frequency distribution of nonerroneous activities of the same type in routine care (e.g., medication administrations, filling command forms). When base rates are absent as a reference, it is difficult to evaluate the relative or overall impact of an event on the activity in question, understand its causes, and develop appropriate preventive steps. Using the population of the total number of detected or reported errors as the only comparative base for evaluating the relative frequency, type, and seriousness of errors may lead to serious interpretation biases and distortions (see Donchin et al., 1995).
Fourth, a serious drawback in using insights gained from the analysis of errors to direct future improvement efforts is that they are based on the “wisdom of hindsight.” As such, they have the value of a post hoc explanation but have a very limited power in providing valid prospective predictions or recommendations (Fischhoff, 1975; Gopher, 2004). In summary, the use of incidents and adverse event reports and the investigation of errors have only limited value as a systematic and credible approach to identifying the major causes and contributors to reduced work safety in the care environment.
Recognizing these limitations has led several research groups to propose and examine alternative and supplementary information sources and data collection methodologies. In anesthesiology, Weinger and colleagues (Weinger & Slagle, 2002; Weinger, Slagle, Jain, & Ordonez, 2003) have conducted a line of studies and argued for the value of nonroutine events (NREs) in clinical operations as indicators of error- and problem-prone areas. In their words,
We recently proposed a new construct, “non-routine events” (NRE), as a mechanism to more efficiently capture dysfunctional clinical system attributes or potentially dangerous conditions. NRE is defined as any event that is perceived by care providers or skilled observers to be unusual, out-of-the-ordinary, or atypical. (Weinger & Slagle, 2002, p. S59)
They collected NREs by interviews and retrospective questionnaires and demonstrated their relevance and value. They have further developed this approach and constructed the Comprehensive Open Ended Non-routine Event Survey (CONES), which was given to anesthesiologists immediately after they had handed off their patients to postanesthesia care (Oken, Rasmussen, Slagle, Kuykendall, & Ordonez, 2007). A very similar approach was adopted by Einav et al. (2010), who studied the influence of team presurgery briefing on the conduct of gynecological and orthopedic surgeries. Observers were recording NREs during the regular conduct of surgeries. The frequency of NREs per surgery was reduced by 25% when presurgery briefings were held.
Tracking and documenting the occurrence of NREs is an important step in building up relevant data sets of potential predictors or factors that may increase the probability of errors and influence the quality of care. NREs are proactive, precautionary indicators that are simpler to observe and record, appear in higher frequency than errors or adverse events, and can be readily classified in relation to the conducted activities. They are also less likely to be influenced by the biases associated with error reports.
However, the NRE approach may have limitations. They document only single deviant events, which may include but are not necessarily focused on system irregularities related to human factors (HF). What may not be captured are continuing and repeating difficulties in carrying out tasks and daily chores because of poor HF. In such cases, impaired efficiency and safety may become “routinized” and may not show up as NREs (e.g., Weiss & Ilgen, 1985). NREs may be more easily defined and captured during the conduct of well-defined and time-constrained activities but may be much more difficult to define or identify in less structured daily care activities, such as occurs on hospital wards. Ongoing collection of NREs on a large scale is challenging, as they require the involvement of interviewers or observers. By comparison, incident and adverse event reporting systems can be filled by medical staff members without close professional involvement across units and organizations.
What is hence needed and appears to be missing is a complementary information system that can capture, collect, and describe from an HF viewpoint enduring, repeating, and ongoing difficulties, problems, and obstacles in the conduct of daily tasks in the care unit that stem from poor HF. Such deficiencies may influence the conduct of time- and task-structured, as well as less structured, care procedures. The growing complexity of health care environments and procedures has been well recognized. However, there is also an increased awareness of the user-unfriendly and hostile characteristics of medical work environments and their negative effect on work efficiency and patient safety (e.g., Carayon, 2007; Gopher & Donchin, 2011; Leape, 2004). Absence of appropriate HF and ergonomic design has been identified as a major contributor to errors and reduced work safety. To get a better understanding of the problems involved and to detect focal indicators of their significance and range, we need an information collection tool that addresses these aspects and can be applied widely.
The present article describes the development of such an approach and its evaluation on four wards of two tertiary care hospitals in Israel. The motivation of this effort was to obtain an HF overview of the daily difficulties, problems, and hazards that are experienced by clinical staff and that influence the efficiency, quality, and safety of their work. An ideal, in which HF professionals can be assigned to accompany the treatment process of each clinician in each unit, is not practical. Instead, a more modest and focused use of HF professionals in a hospital is required. Consequently, a two step-approach was developed.
In the first step, we developed a reporting system in which clinicians could voluntarily identify and document problems and difficulties encountered in their work routine. In the second step, these reports were reviewed, categorized, and interpreted by a team of HF professionals to identify HF-related focal problems that may affect daily work, care quality, and patient safety. The study was conducted in three stages, which are described separately as follows: (a) development of report formats, (b) administration and collection of reports in wards, and (c) validation studies of the focal problem categories emerging from filled reports.
Stage 1: Development Of Reporting Forms
The development of reports and their administration was first conducted at the Rambam Hospital (government hospital) and then at the Hadassah Hospital (private). The two differ in their organizational structure and management style. Prior to the beginning of the study, the approach was presented and discussed in each hospital with its director and senior management. Meeting objectives were twofold: One was to present the logic of the approach and obtain the support for the study plan of work. The second, equally important, objective was to obtain the hospitals’ commitment to adopt the recommended changes of the HF team that would be based on the analysis of the collected reports. Both hospitals supported the study, and there was a general agreement that immediate steps would be taken if costs were reasonable. In addition, a senior advisory committee was appointed in each hospital to accompany the research team. Because the study did not involve medical procedures or contact with patients, the study was formally considered exempt by the hospitals’ Helsinki Committee in Rambam and Hadassah.
Stage 1 consisted of two steps:
A systematic analysis of daily work routines on the ward to map major HF aspects associated with the clinical staff daily work.
Development of forms to report relevant problems, difficulties, and hazards.
Method
Ward analysis
The Technion HF research team was composed of one faculty member and three graduate students, all of whom are HF professionals with previous medical systems experience. The team worked closely with the medical director and head nurse of one internal medicine ward and the orthopedic ward at the Rambam Medical Center. The team conducted personal interviews with key medical staff at each ward as well as conducted fifteen 3- to 4-hr observations in each of the wards across a period of 2 weeks, designed to study the physical environment and facilities, basic and key work routines (i.e., shift change, doctors’ rounds and medication delivery), ward conventions, and care tasks. The observations and their analysis applied the general protocols and task analysis approach developed by Gopher and Weil (1989) for the study of HF aspects in industrial work settings.
Taken together, the interview and observation data showed that the two wards could be characterized along five main HF work-related problem areas: (a) work procedures and structure, (b) physical space and layout, (c) medication administration, (d) equipment and instruments, and (e) reporting and recording (see Figure 1). Although consistent with the existing health care HF literature (e.g., Carayon 2007; Food and Drug Administration, 1999), to the best of our knowledge, this approach of inclusive classification of HF work-related problems in hospital wards has not been formalized in this manner.
The five human factors problem areas and their contributing factors.
Forms Development
Structure and content
The forms were designed to enable both informal and formal coverage of the reported problem as well as facilitate reporter-generated possible solutions. Each of the five forms was divided into three segments (see Figure 2). In the upper segment, respondents were first asked to describe the problem or difficulty in their own words. In the middle segment, they were asked to select one or more contributing factors from a list of features and factors associated with this problem. The respondents could add factors that were not listed. In the third segment, respondents were encouraged to suggest problem solutions and add any other comments. Respondents were then asked to indicate to which group they belonged, physicians or nurses, and if willing, to identify themselves (for further questioning if necessary). It was explained that respondents could remain anonymous if they chose and that only researchers would have access to their identity. Various drafts of the forms were discussed with ward leadership prior to determining the final form. Figure 2 presents an example of a blank work procedures and structure report form.
An example of a work procedures and structure report form.
General Considerations
Two key concerns shaped form development. First, were the forms successful in specifying problem areas that are recognized as relevant and that would motivate responses from ward staff members? Second, did the format and mode of reporting facilitate completion by busy staff members?
Notably, although the internal and the orthopedic wards differ appreciably in their types of patients and care procedures, the forms appeared to adequately cover the work activities under way at both wards. Moreover, when piloting the forms for use at the two gynecological wards in Hadassah, we reexamined their relevance and no change or restructure was required. Hence, the five HF-related categories covered by the forms were sufficiently inclusive and useful across wards that are widely diversified in setting, patient type, and care procedures.
The final version consisted of a single page for each HF problem area and required a few minutes to complete. We constructed the middle segment of the reports to include an aforementioned list of factors and features associated with this problem area to help reporters to identify, observe, and recognize contributing HF-related factors for each of the reported problems. We hoped that this design would increase the relevance and information value of the reports. In addition, we thought it may foster the clinician’s understanding of the underlying issues to link HF problems in the workplace and those problems’ etiology. Theoretically, this approach might prompt the respondent to consider or identify other problems that might result from similar factors and hence be more inclined to propose practical solutions to the described problem. Additional conceptual support and evidence for our approach can be found in Zohar (1999) and Zohar and Luria (2003), who conducted similar efforts in military and industrial work environments.
Stage 2: Collection Of Reports In The Wards
Method
Sample
The orthopedic ward and one internal medicine ward of the Rambam Medical Center in Haifa (900 beds) and two gynecology wards, at Ein-Karem and Mount Scopus hospitals of Hadassah Medical Center in Jerusalem (1,050 beds), participated in the study. The Rambam orthopedic ward had 30 patient beds, 20 nurses, and 15 physicians. The internal medicine ward had 35 beds, 26 nurses, and 10 physicians. The Ein-Karem gynecology ward had 25 beds, 18 nurses, and 14 physicians, and the Mount Scopus ward had 25 beds, 17 nurses, and 11 physicians. To recapitulate, we tested the reporting system at two medical centers to examine its generalizability to different types of wards and organizational cultures.
Procedure
Reports from each of the four wards were collected during a 12-week period. An introductory 90-min training session was held with the medical staff of each ward (physicians and nurses in addition to a senior representative from hospital management project advisory committee). The purpose of the study and the reporting system were presented. Staff members were requested to use the forms ad lib to report difficulties and problems that they confront in their daily work. Blank forms were placed at the forms storage cabinets in each hospital ward. Completed reports were deposited in a locked box and collected by the Technion team once a week.
Reports evaluation
Each member of the three-member HF research team reviewed all collected reports from each ward and categorized them. A joint meeting of the team reviewed the individual reports, discussed mismatches, and arrived at an agreed-on grouping. This process yielded three groups of problem categories: (a) general problem topics, (b) specific (local) problems, and (c) problems not related to HF. General problem topics were problems that emerged in several reports, addressed aspects of the same or similar type difficulty within a specific problem area, and were considered as possibly having a significant impact on care quality and patient safety. Specific topics were usually isolated reports addressing a specific, local problem (e.g., an old and slow computer in the residents’ office). Unrelated problems were reports that dealt with problems that had no direct relation to HF or safety considerations (e.g., insufficient cleaning of windows).
Results
A total of 359 problem reports were received: 118 reports from Rambam and 241 from Hadassah (during the same period, no incident reports were received from the obligatory reporting system in either hospital). For comparison, during the 5-year period prior to and including the period of the present study, a total of only 149 incident reports were received from the two wards at Rambam and 51 incident reports from the two Hadassah wards via their obligatory reporting systems (these data were drawn from system databases). Table 1 presents the main topics and frequency of the problem reports received as well as those obtained with the incident-reporting system. Both the relative number and the nature of the reported problems obtained with the two systems are very different. Whereas falls were the most frequently reported event in incident reports and medication-related events second, falls were absent and medication events were relatively rare in the problem-reporting system. Instead, instruments and equipment, work procedures, and structure were the most frequent complaints across the four wards.
Frequency and Topics of Collected Reports With the Two Reporting Systems for Each of the Four Wards
In addition, incident reports were filled exclusively by the nursing staff, whereas doctors in both hospitals did not submit even a single report (see Figure 3). In the problem-reporting system, doctors at both centers contributed about 30% of the total number of reports. Of the doctors’ reports, 21% were filled out anonymously. The relative frequency of doctors’ reports compared with nursing staff problem reports corresponds to their relative time and share in the ward activities.
Frequency and reporting source with the existing incident- and new problem–reporting systems at each medical center.
Table 2 presents the major problem categories identified by the HF team following the review and grouping of the collected reports. The four wards differed in the nature of their most frequently reported problems. Problem topics in bold were selected by the HF team, the ward director, and the head nurse for subsequent validation studies and possible intervention.
Topics of Reports in Each of the Four Wards
Note. Problem topics in bold were selected by the HF team, the ward director, and head nurse for subsequent validation studies and possible intervention.
Discussion
The results obtained with the new reporting system were encouraging. The system and format of problem reports were well accepted in all wards, with both doctors and nurses contributing to filling out reports. Another encouraging outcome is that the emergent dominant category and types of problem reported in each ward were unique to that ward. For example, the major problem category reported in the Rambam internal care ward was monitoring patient status. This ward provides care mainly to cardiovascular patients, and thus the problem is directly related to its clinical focus. In the orthopedic ward, the main reports were concerned with support, mobilization, and rehabilitation equipment, which again closely correspond with its clinical responsibilities. The focus of reports in the two gynecological wards was completely different. Hence, although comprehensive and inclusive, the composition of reports shows that the system was also sensitive to differences between wards and was properly employed by the clinical staff to reflect those differences.
A significant outcome of the problem-reporting systems is the relatively large number of reports generated within a short period (see Figure 3 and Table 1). Also noticeable is the contribution of both doctors and nurses to filling reports. The general problem of limited rates of incident reporting is already well recognized. Its reasons and causes have been discussed in the Introduction section. What needs to be considered is the large number of and willingness to submit problem reports by both physicians and nurses. One possibility may be the attention, emphasis, and support given to the study and the reporting system by the ward director and the hospital management. In this context, it should be recognized that the importance of reporting incidents and adverse events is equally emphasized by hospital management and does not differentiate between physicians and nurses in the call for submitting reports.
Another important distinction is that problem reports are generalized and submitted anonymously, minimizing fears of accountability and blame. A third possible factor may be the benefits of the report to the reporter. Unlike incident and error reporting, in which the value and prospective benefit are generic and abstract, problem reporting may lead to direct corrective interventions that will influence the reporter’s personal work environment, particularly since there is explicit management commitment to apply recommendations. A fourth explanatory factor may be the emphasis on reporting ongoing difficulties, nuisances, and problems in the conduct of daily work. These may be more easily detected and reported compared with single incidents or adverse events. The four contributing factors are not mutually exclusive. The last three contribute to the fact that physicians were more willing to contribute problem reports.
As evident from Tables 1 and 2, the topics included in the incident- and problem-reporting systems differed. Incident and adverse events reports are accounts of negative consequences that should be avoided. Problem reports describe obstacles and performance difficulties that increase interference and workload and decrease efficiency and safety. Similar to NREs (Weinger et al., 2003), problem reports describe causes, processes, and mechanisms that increase the likelihood of failures and adverse events. Hence, in the assessment of the value of problem reports, the major consideration is not the similarity of their topics to those of incident reports but their merit as valid pointers and guides for corrective interventions to improve care quality and reduce errors.
Validation and evaluation of the primary topics identified by the HF team from the review of problem reports were the goals of the third stage of the present study. This stage assessed the power and validity of the collected problem reports, as well as their classification and grouping by the HF research team, in identifying major problem(s) in each of the four wards that influence the quality of care and that may need intervention.
Stage 3: Validation Studies Of Reported Problems
Stage 3 focused on the most frequent problem topic raised in each ward (see Table 2) and involved an in-depth study in the ward to examine the approach viability. Studies were conducted in three of the four wards. The study in the orthopedic ward of Rambam was discontinued when the ward moved to a new space in which the environment and daily working conditions changed. Because the main topic of complaint varied in each ward, a different method and appropriate measurement approach had to be adopted to enable the evaluation of the corresponding topic.
Rambam Internal Medicine Ward
The most frequent problem raised in this ward was the quality of ongoing monitoring of patients’ medical status (38% of the reports). The main body of patients in this ward comprised cardiovascular cases. Thus, the ability to provide close and high-quality monitoring is crucial for the quality of care and patient well-being.
Method
To examine monitoring processes, we conducted a simulation study with a random sample of 100 medical files of patients who had been hospitalized and discharged from the ward during the study period. Patient identification details were removed from all files, and patients’ medical information at admittance were then reviewed by a senior internist from other internal wards in Rambam.
Three senior internists defined a standard list of nine indices for monitoring patients’ medical status. The required frequency at which the five ward-administered indices (heart rate, blood pressure, oxygen saturation, temperature, and transcutaneous echocardiography) should be monitored across patients could vary from several times a day to continuous monitoring, depending on the type of illness and severity. Accordingly, a 0-to-9 range was established wherein 0 was no monitoring required and 9 was continuous. The other four indices (electrocardiograph, serum glucose, glucose and protein in urine [urine test], and weight) were provided by external support units and were rated on a binary scale (present or absent) because they were mostly obtained once a day.
The 100 files were randomly divided into two groups of 50 and included only the admission records (medical history, initial findings, and prior medical diagnosis) of the patient. Each group of 50 files was evaluated by a different panel of three senior internists who were not members of the ward from which the reports were obtained. Each of the panels scored the recommended daily monitoring profile for the nine indices for each of the 50 patients. The recommended profiles could then be compared with the actual monitoring frequency as was recorded in the patient’s medical record during hospitalization (not seen by the reviewing panel). Then, two additional independent panels of three senior internists were provided with the complete medical records of 50 patients (100 records total) and were asked to evaluate the quality of the actual monitoring given each patient using a 5-point scale (5 indicating very good). The conducted simulation enabled us to obtain two measures of the quality of monitoring: One is a comparison between recommended and actual monitoring profile, and the second is an overall subjective evaluation of the quality of care given that is based on the complete medical record of the patient.
Results
Table 3 presents the averages of daily recommended versus actual monitoring frequencies of within-ward obtained indices. Table 4 compares the two measures obtained from outside units. As Table 3 shows, the actual monitoring of the five within-ward monitored indices was significantly lower than the recommended frequencies. In contrast, no such difference was found between the four indices provided by units external to the ward (Table 4). The second evaluation of monitoring quality, which was based on the full medical records of patients, yielded an overall average score of 3.3 (SD = .875) on a 1-to-5 scale, representing a medium-level rating. Hence, the problem reports indicating monitoring difficulties within the ward that affect daily care capabilities was manifested in both evaluation measures.
Average Daily Recommended Versus Actual Monitoring Frequency Within the Internal Ward
Note. ECO = transcutaneous echocardiography.
Average Recommended Versus Actual Daily Frequency of Measurements Provided to the Internal Ward by External Units
Note. EKG = electrocardiograph.
Hadassah, Ein-Karem Gynecology Ward
The most frequent problems reported in this ward were identified as ambiguities in work structure and the division of responsibilities between physicians and nurses (26% of the reports). If existing, this type of problem is highly conducive to failures of coordination, handover, and communication problems in teamwork.
Method
We decided to investigate topics of ambiguity in work structure and division of responsibilities by extracting and comparing the mental models of physicians and nurses for their most prevalent work procedures. Mental models are simplified representations of complex physical or social systems and processes, developed and used to interact with complex systems and tasks (Moray, 1999). Sinreich, Gopher, Ben-Barak, Marmor, and Lahat (2005) have argued that in low-structure work situations, team members’ mismatched mental models of the work process are good indicators of coordination failures and difficulties. They adopted the flowchart methodology from industrial engineering, which describes the flow of work and the use of resources in production, and expanded it to describe mental models of clinical staff work in medical care environments. Studying medical emergency units, Sinreich et al. asked physicians and nurses to describe their perception (mental model) of the flow of care procedures and the use of resources (physicians, nurses, laboratories, imaging facilities, etc). Mismatches and gaps in the flowcharts of work procedures increased the frequency of problems and errors. This mismatch has been interpreted to reflect divergent mental models about the structure of clinical work.
In the presently studied gynecology ward, we applied the same approach and methodology. We identified the ward’s main care and surgical missions and constructed a flowchart with each staff member to reflect his or her perception of this work procedure, its structure, and use of resources. Figure 4 depicts the chief nurse’s perception of the procedure of terminating pregnancies by curettage. Using this methodology, we were able to extract and evaluate the degree of similarity between and within medical and nursing groups (eight nurses vs. seven physicians) for the type and sequence of procedural elements, probabilities for transition between elements, and the use of supporting resources.
An example of the head nurse’s perceived flowchart of the procedure of terminating pregnancies by curettage. Percentage indicates flow ration for each route.
Another aspect of the methodology elicited the perceived duration and significance of each procedural element. All participants were asked to evaluate the average and maximum durations (in minutes) as well as the contribution of each element to the overall success of the procedure on a 1-to-5 scale (5 being very important).
As ambiguity of work structure and of division of responsibilities was the most frequent complaint in Hadassah Ein-Karem problem reports but very rare in Mount Scopus, we conducted in parallel the same work at Mount Scopus as a control cognition to evaluate whether flowchart mismatches indeed distinguish the Ein-Karem ward from Mount Scopus gynecology ward.
Results
When comparing the perceptions of physicians and nurses for the three main procedures at the Ein-Karem ward (short surgery, long surgery, and hospitalization for observation only), the greatest perception of inconsistencies and mismatches occurred in short surgery, which is also the most frequent procedure (57%) performed in the ward (the terms short and long surgery refer to the seriousness of the procedure and the duration of postsurgery hospitalization.)
Table 5 presents the comparative perception of the short surgery process by physicians and nurses. Generally, when meeting a patient at different steps, both physicians and nurses assume the patient has undergone more medical cross-checks than he or she really did. For example, physicians underestimate the level of involvement of the administrative staff, who are estimated by nurses to handle 90% of the postsurgery admissions. In other words, physicians assume much more nursing monitoring and less administrative involvement. Nurses assume that a much higher percentage of patients undergo medical examination before nursing admission. There is also divergence in the estimated sequence of an order of medical admissions and supplementary medical tests. These types of disagreements not only disrupt the flow of work activities but also create responsibility diffusion, communication gaps, and wrong assumptions in the conduct of medical work, which could increase incidents and errors and impair the quality of care.
Physicians and Nurses’ Perception of “Short Surgery,” Gynecology Ward, Ein-Karem, Hadassah Medical Center
p < 0.001. *** p < 0.1.
A comparison of how physicians and nurses perceived the average and maximum duration of the composites of the medical procedures also revealed significant differences. Each group estimated longer durations and higher workload for the task segment it performs. For example, physicians estimated that the nurses’ admission procedure takes 19.3 ± 3.4 min, whereas the nurses estimated 30 ± 5.1 min (p < .004, Mann-Whitney U test). Nurses, on the other hand, felt that it took physicians only 5 ± 1.9 min to review a patient’s chart, whereas physicians estimated the task at 9.4 ± 2.6 min (p < .002, Mann-Whitney U test). Thus, each group estimated its tasks to be more demanding and loading than those of the other group, suggesting insufficient knowledge of each other duties, a further contributor to difficulties in team work.
The comparative study at the Mount Scopus ward did not reveal any significant differences in the estimates given by the two groups (p levels were greater than 5% for all comparisons). This finding supports the singularity of the main complaint for the ward in which it emerged.
Hadassah, Mount Scopus Gynecology Ward
The most frequent problem topic emerging from the problem reports at this ward was communication and information transfer between physicians and nurses (19% of the reports). This is a long-recognized problem associated with increased error rates and reduced care quality (e.g., Cook, Render, & Woods, 2000; Donchin et al., 1995). An objective measurement study of this problem within the ward is hard to conceive, plan, and conduct.
Method
How can the numerous formal and informal aural and written communication instances between clinical team members in their daily work be observed, collected, and evaluated? We had to compromise on a more modest but achievable approach. A six-question anonymous questionnaire was constructed and given to all staff members, who were asked to evaluate on a 5-point scale the frequency and severity of communication and information transfer problems possibly affecting their performance efficiency and patient safety. Questions were asked for both general evaluation and specific ranking of the quality of exchange within and between nursing and the medical staff members.
Although drawn from the collection of specific reports that were written by the ward team members, the questionnaire employed concepts and language that were much more general and inclusive than any single problem report. For example, one physician report claimed that “nurses sometimes fail to carry out a written command because of unclear writing or lack of understanding.” Another physician complained that “nurses perform commands but do not fully understand their meaning and hence are not aware and able to detect unreasonable or erroneous commands.” The language of the respective question in the questionnaire was much more general: “Q3, Are there sometimes difficulties in passing information related to patient treatment?” The questions, their phrasing, and the required estimates were generalized from the specific problem reports.
As a validation control condition, the same questionnaire was also given to members of the Hadassah Ein-Karem genecology ward, where this type of problem was not highlighted.
Results and discussion
The results of the questionnaires in the two wards were submitted to a two-way analysis of variance in which the two factors were role (physician or nurse) and ward (Mount Scopus or Ein-Karem). A separate analysis was conducted for each of the six questions. In both hospitals, physicians evaluated a higher level of information transfer problems than did nurses (average rating = 2.96 vs. 2.5, respectively), F(1, 24) = 4.24, p = .05. Physicians also rated greater difficulties arising from discrepancies between aural and written commands (average rating = 2.69 vs. 1.94), F(1, 24) = 6.77, p = .02. Most important for the present investigation were the interaction effects found between hospitals. Mount Scopus nurses rated significantly higher than their Ein-Karem colleagues difficulties in communicating essential information regarding patients’ care protocol (average rating = 2.91 vs. 2.28), F(1, 24) = 4.93, p = .04. A similar difference appeared when physicians in both hospitals rated their difficulties in communicating and transferring information to nurses: Mount Scopus physicians’ average was 3.20, whereas Ein-Karem average was 2.32, F(1, 24) = 7.78, p = .01.
Note also that on a 5-point scale on which 5 indicates frequent and serious, the average ratings of both nurses and physicians in Mount Scopus were higher than the midpoint (2.5), whereas those of Ein-Karem were lower. Taken together, the questionnaire results reconfirm the validity of the main problem topics emerging from the problem reports. They also reaffirm the sensitivity of the developed forms to the specific locus of problem in each of the different wards.
General Discussion and Conclusions
To recapitulate, the driving motives of the present work have been twofold: One is the urgent call for HF involvement and intervention in health care work environments to improve care and patient safety; the second is the absence of an adequate approach to data collection that can be applied at wide range and employed to generate an adequate database to guide HF professionals and focus their efforts. The most common information source has been the investigation and reporting systems of incidents and adverse events. More recently, this source of information has been augmented and complemented by the collection and analysis of NREs in the conduct of medical procedures (Einav et al., 2010; Weinger & Slagle, 2002). The value and constraints of each approach and information source have already been discussed in the Introduction section.
The problem-reporting system that has been developed and experimented in the present work attempts to widen the scope of the HF sources of information and explore the plausibility of a new approach to a wide range of collection and coverage. The work was conducted in three stages (development of forms, data collection in four wards, validation studies), all of which were completed successfully. In this final discussion, we would like to reiterate the main signifiers of the proposed approach and discuss its potential implications and prospective applications.
A major distinctive feature of the approach is that clinical team members produce direct reports on repeating difficulties in their daily work. Although these are subjective and nonprofessional accounts (and hence are further reviewed by a HF team), there are two additional features that should be noted. One is that these reports do not represent an account or recount of a single incident or event. Rather, they are complaints on continuing, ongoing, and repeating problems that interfere with the conduct of work. As such, they may constitute a more balanced account of the daily work at the ward. In addition, reports may cover within the unit a variety of work topics, which are more or less structured, synchronized or unsynchronized, physical, procedural, and varying task segments. Work environments with multiple tasks and reduced structure are common in health care and make capturing incident reports or recording NREs difficult. Direct problem reports are therefore changing the emphasis and widening the scope of reported problems.
In this context, it is instructive to reiterate the fact that the five topics of forms we developed and their specific descriptive items were inclusive and general enough to be applied and comfortably used by all clinical staff members in hospital wards that differ greatly by type of patient, care procedures, and equipment. Nonetheless, the forms also show their differential sensitivity to reflect the typical problems in each ward.
The results of the report collection and validation stage show that a two-step approach, in which reports are collected and then are collectively reviewed by an HF team, may provide the required caution and cushion to enable robust parallel data collection across the multiple units of an organization. This two-step approach also allows the economical and efficient use of a limited number of HF professionals to review, focus, and prioritize efforts. The prospects of this approach and its power are further strengthened by the large number and wide range of topics covered by the reports as well as by the significant participation of physicians. These combined factors increase the likelihood that the problem report database is able to provide a more representative sample of the HF-related problems in a care unit and their severity.
An important advantage of the problem report system is that it avoids blame and accountability issues since no specific accidents, incidents, or adverse events are discussed. Moreover, reporting may have more direct benefits if problems are identified and addressed. An indirect, second-order benefit may also be the increased awareness on the part of ward and hospital management of the difficulties and problems associated with bad HF design. Such awareness may lead to the introduction of improvements before complaints emerge. Similarly, being acquainted with the topics and scope of HF concerns, clinical staff members may seek and initiate their own preventive acts and solutions to perceived problems. These may all be very positive by-products.
Follow-Up and Future Directions
The present work represents a positive first-step study of the problem report approach and its potential applications. There are several important topics and challenges for future continuation and testing of this approach. The first is the ability to apply the problem report system not only for a limited period of 12 weeks but as a continuing reporting system. Questions associated with this topic are the overall rate of responding, survivability, and identity distribution of reporters. Technical challenges are the incorporation, parallel collection, and interpretation of reports across the whole hospital. A related challenge is structuring, formalizing, and evaluating the process of reports review and interpretation by an HF team(s). An important challenge is the mode of interaction with hospital management on the outcome of the reports and on adopting recommended changes. Last but not least are the actual introduction of changes into units and the follow-up of the effects at the unit and hospital levels.
At this time, all of the aforementioned topics are directions for future efforts. However, we can informally report some follow-up consequences related to the conducted work. At the Rambam hospital, the orthopedic ward was moved to a new and larger space, where many of the problems raised in the old unit were resolved. In the internal ward, the existing and problematic monitoring equipment was replaced. In Hadassah Ein-Karem, presentation of the results and discussions with the division management and ward directors led to clarification and modifications of procedures. At Mount Scopus ward, presentation of the results and their discussion with the ward physicians and nurses team led to insights, clarification of protocols, and establishment of improved information transfer and communication procedures. At this stage, we did not have the funds and ability to conduct follow-up studies. At present and following a pilot investigation, a 3-year study of the approach is being conducted at Changi General Hospital in Singapore.
Key points
The current prime source of information for identifying problems and guiding corrective efforts to improve patient safety in hospitals focuses on adverse events, accidents, and incidents reports.
Those reports have only limited scientific value as an information source for developing a systematic and well-balanced remedial program to increase patient safety and reduce errors.
This study involved the testing of a problem-reporting system in which clinical staff members are asked to report human factors–related problems of their daily work.
The new reporting system was able to pinpoint major problems topics in each of the four studied hospital wards
The overall number of reports was much larger than the number obtained with the existing obligatory incident-reporting system. In addition, physicians were active contributors to problem reports, whereas in the obligatory incident-reporting system, reports were submitted exclusively by the nursing staff.