Electronic Prescribing Usability: Reduction of Mental Workload and Prescribing Errors Among Community Physicians

Abstract

Background: Medical errors are common in hospitals, and research is always needed to find ways of reducing these. This study attempts to address three gaps in this field. First, the factors leading to the reduction of mental workload and its relationship with the reduction of prescribing errors by improving electronic prescribing (e-prescribing) usability have not been empirically examined before. Second, the past research in the field of e-prescribing usability lacks robust theoretical models. Third, there are no existing studies to examine the direct influences of user interface consistency and error prevention with the reduction of mental workload and prescribing errors. Materials and Methods: A quantitative survey method was used to collect data from 188 community physicians. The partial least squares path modeling technique was applied to analyze the data. Results: Prescribing errors were reduced by improving the information quality, user interface consistency, system ease of use, and mental workload reduction. Mental workload is reduced by ease of use, error prevention, and consistency. No significant relationships between prescribing error reduction with error prevention and also between information quality with mental workload reduction were found. Conclusions: The designers of e-prescribing should improve the error prevention and consistency of the system and make it easy to use if they wish for the system to reduce users' mental workload. They should also improve the system information quality, ease of use, and consistency if they claim that their system reduces physicians' prescribing errors. The system should also reduce users' mental workload to meet this objective.

Introduction

Past studies have shown that medical errors are quite common in health centers,¹ leading to much mortality^2
–4 and imposing huge costs on the health sector.^5,6 To reduce medical errors, health centers have switched to using e-health systems (EHS) as one of the solutions that can reduce such errors.^7
–9 However, some scholars have found that EHS might even increase medical errors, poor care consequences, and mortality.¹⁰ Thus, deploying EHS is not merely a source of improvement in reducing medical errors as there are some contradictions in the findings of the past research with regard to the impacts of such systems on the reduction of medical errors.^8

–11 Some researchers have highlighted the unexpectedly high likelihood of patients' safety threats that may arise from EHS, including electronic prescribing (e-prescribing) technology.^12,13 For instance, it has been reported that e-prescribing systems can increase medication error likelihoods.¹⁴ Similarly, it was found that using e-prescribing systems has brought about a 44% increase in medical error rates compared with usiing handwritten prescriptions.¹⁵ Other research reported that mortality rates rose in the intensive care unit after the implementation and use of an e-prescribing system.¹¹ Bosman¹ also reported an increase in the number of medication errors and mortalities after using e-prescribing systems in health centers. Hence, there may be some factors in EHS that can help users reduce medical errors. Therefore, it seems necessary to study the factors that result in sustainability of EHS reduction of medical errors through EHS,^12,16 including e-prescribing systems.

Two factors proposed to reduce medical errors are system usability and users' mental workload. However, there are some gaps in this regard. First, although it has been stated that there is a relationship between workload and medical errors,¹⁷ no empirical evidence has been reported in this regard. Moreover, past studies on health professionals' workload have focused on measuring their mental workload working under certain circumstances,^14

–20 studied the consequences of workload,²¹ or studied the impacts of EHS functionalities on users' workload.¹ However, we found no studies testing the impact of EHS usability on community physicians' workload, which calls for more research. Furthermore, the conceptualization of “medical errors” is usually general, and most studies do not conceptualize different kinds of medical errors.^12,22 Thus, it seems essential to explore the impacts of mental workload reduction on prescribing errors instead of merely general “medical errors.”

Another issue concerns the research method used in the past research. Some scholars have used different qualitative techniques to study the likelihood of medical errors in EHS. For instance, John et al.²³ applied a scenario-based study for a decision support system for intensive care units, Horsky et al.²⁴ used semistructured interviews with physicians and interface usability inspection of a computerized physician order entry (CPOE), Koppel et al.¹⁴ applied focused group and interview methods to study the impact of CPOE systems on different health professional groups, including nurses, physicians, pharmacists, etc., and Kushniruk et al.²⁵ used the users' think-aloud method while working with the CPOE system. In other research, Chan et al.²⁶ asked staff physicians, residents, and medical students to complete four simulated orders with three different order formats in CPOE, whereas Cho et al.²⁷ reviewed error charts derived from CPOE use in two different hospitals over a 4-month period.

Despite the wide attention of researchers to use qualitative methods in their research, the use of robust quantitative methods has been neglected or ignored in this field. As Oztekin et al.²⁸ observed, past studies have either used qualitative research approaches or have not used a valid/reliable quantitative scale to evaluate usability. Although the qualitative research approach is a scientific process, it suffers from the subjectivity of the analysis and the lack of generalizability of findings.²⁹ On the other hand, quantitative research can help overcome these limitations.²⁹

However, most of the few available empirical studies suffer from inaccuracy as recently Weir et al.³⁰ reviewed the quantitative studies about the impacts of EHS and concluded that “none of the reviews provided usable quantitative data.”^p.224 Although a few recent attempts have been made to empirically develop and test quantitative scales,^12,13 more studies are needed to develop and validate robust quantitative scales to examine system usability in this field.¹² Furthermore, most of the quantitative research studies concerning the efficiency of EHS have an insufficient sample size.^20,24,31 Another limitation is that the past research in this field has hardly developed and tested a theoretical usability model.^13,32 Therefore, the use of models to empirically examine the influence of e-prescribing on users' outcomes is desirable. Indeed, we neither aim to claim that this research provides superior data compared with all the past qualitative or quantitative research, nor do we intend to differentiate between this research and the past research on issues such as the types and varieties of systems studied or variations in the features of them. We, however, believe that our research, methodologically, is an attempt to overcome the mentioned shortcomings and a step forward in this field.

This study used a quantitative research method to investigate the impact of e-prescribing systems' usability on the reduction of doctors' prescribing errors and mental workload. With regards to modeling the usability antecedents of prescribing errors in e-prescribing systems, only one research study, carried out by Peikari et al.,¹² was found. However, that research has a limited approach compared with this research. First, their model lacks the potential role of mental workload in reducing prescribing errors. Furthermore, they have neglected measuring the direct influence of system consistency and error prevention features on reducing prescribing errors, which calls for new research. Another difference is that the current research collected data from community physicians, which is different with the scope of the research by Peikari et al.¹² Community health professionals face more challenge in using EHS compared with those professionals working in hospitals,¹³ and therefore more research is needed to study the impacts of such systems on community health professionals.

Moreover, because medical errors are known as an index for patient safety and care quality,^33,34 recognizing the effects of e-prescribing usability on medical errors can improve our knowledge about the relationship between e-prescribing systems usability with patient safety and care quality. Doctors are the second victims of medical errors because of the sentimental, legal, and professional issues faced.³⁵ The results of the present study show the extent to which doctors believe that e-prescribing systems have assisted them to reduce their mental workload and also provide patients with error-free health services.

Theoretical Foundation

According to the input-process-output model of Cooper,³⁶ the input quality and the process quality determine the quality of output. The input quality deals with the system's design characteristics, including error prevention, consistency, and ease of use, whereas the process quality refers to the quality of the information.¹² Another model suggests that the performance of users is highly influenced by system attributes.³⁷ Likewise, some scholars have found that the system features can predict users' medical performance with the system.^{12,13,38
–40} Some recent studies have shown that users' performance with a system is also influenced by the quality of information.^10,12,41,42 Furthermore, according to the standards of human–computer interaction, in any information system, users' outcomes and performance with the system can be improved with better usability. Thus, to improve the error-free performance of physicians, user interface quality should be enhanced.

As stated earlier, most of the past research on the relationships between system usability and users' outcomes lack theoretical models.^13,32 We used the model of Peikari et al.^12,13 of system usability (including ease of use, consistency, error prevention, and information quality) for our research. Following the above discussions, it is hypothesized that:

H1. The ease of use of e-prescribing systems reduces doctors' prescribing errors.

H2. The quality of the information generated by e-prescribing systems predicts a reduction of doctors' prescribing errors.

H3. The error prevention of e-prescribing systems predicts prescribing error reduction.

H4. E-prescribing systems' user interface consistency results in prescribing error reduction.

H5. The ease of use of e-prescribing systems reduces doctors' mental workload.

H6. The quality of information produced by e-prescribing systems reduces doctors' mental workload.

H7. The error prevention of e-prescribing systems leads to doctors' mental workload reduction.

H8. The consistency of e-prescribing systems' user interface leads to the reduction of users' mental workload.

Multiple psychological and physiological factors, including increased mental and physical workload, make errors more likely to happen in clinics.¹⁹ Work overload is the key factor to individuals' fatigue and reduction of the service quality provided by those individuals.¹⁹ Hence, it is hypothesized that:

H9. Reduction of mental workload positively predicts the reduction of prescribing errors.

The model of this research is illustrated in Figure 1.

Fig. 1.

Research model.

Materials and Methods

Research Method

This study used a quantitative survey method, and cross-sectional data were collected. As mentioned earlier, some scholars have highlighted the lack of robust quantitative research in this field. Using a survey is an efficient and effective method to collect users' perceptions and attitudes about the outcomes and impacts of EHS.^43,44 Many researchers have recommended using surveys to evaluate EHS usability in the postimplementation phase of the system life cycle.^24,45

Self-Report Scale

One of the methods widely recognized and used by researchers to assess the characteristics of information systems and their impacts and outcomes is users' self-report scales.³⁷ Evaluating a system by using self-report scales can provide a reliable indicator of impacts and outcomes of an information system.^46
–48 This is especially true in the case of medical errors, whereby many health centers lack an efficient system to report them, and secondary data in this regard are unavailable, inaccurate, or biased.¹² Using self-measured rating for mental workload by asking individuals about their perceptions of their workload,^49
–51 reduction of medical errors,^12,13 or the characteristics of an information system^{10,28,33,52

–61} has been well recognized in the academic world. Therefore, using self-report measurement in this research can provide reliable and valid data.

Target Population

Only one group of stakeholders should be considered as the respondents to evaluate the requirements and impacts of EHS because different groups of users have different objectives and expectations.^8,62 Therefore, the target population for this research was community physicians who had worked with e-prescribing systems for a minimum of 3 months. The diversity of community physicians can make the results more generalizable compared with research that has collected data from only one hospital.

Questionnaire Development

All questions, except the demographic section, were designed based on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). As illustrated in Table 1, the questionnaire was developed based on previous published studies. After the development of the scale, it was evaluated and improved by four university lecturers who were specialists in health informatics and three physicians. Following this, the scale went through three stages of pilot testing. Each stage included five respondents, who answered the questions independently, and the questions that required revision were modified. For the main round of data collection, a purposive sampling approach was used. We followed the method of Krejcie and Morgan,⁶³ and because the identified population of community doctors was 778, a total number of 256 questionnaires was distributed among them. In a 3-month period, the sampled respondents were contacted three times to prompt a response to the questionnaire, and after 3 months, 188 usable questionnaires were received.

Table 1.

Scale Source and Reliability

VARIABLE	REFERENCE	CRONBACH'S ALPHA	CR	AVE	SKEWNESS	KURTOSIS
Information quality	Peikari et al.¹²	0.73	0.84	0.65	−1.94	0.04
Ease of use	Peikari et al.¹²	0.81	0.88	0.72	−1.15	0.67
Consistency	Peikari et al.¹²	0.78	0.85	0.60	−1.06	1.62
Error prevention	Peikari et al.¹²	0.83	0.88	0.66	−2.26	−1.05
Prescribing error reduction	Peikari et al.¹²	0.83	0.88	0.66	−4.38	0.31
Mental workload reduction	Prichard et al.⁶⁴	0.790	0.87	0.70	−0.49	0.56

AVE, average variance extracted; CR, composite reliability.

Results

The details of our samples' profiles have been illustrated in Table 2. As shown, a majority of respondents (36.2%) had more than 9 years of experience in using computers, 44.1% of respondents had more than 1 year of experience of using e-prescribing systems, and 49.5% of them were medical doctors for more than 5 years.

Table 2.

Respondents' Demographics (n=188)

CHARACTERISTIC	FREQUENCY	PERCENTAGE
Gender
Male	69	36.7
Female	119	63.3
Age (years)
20–30	88	46.8
31–40	49	41.5
41–50	34	8.5
>50	68	3.7
Computer experience
<4 years	37	19.7
4–6 years	49	26.1
7–9 years	34	18.0
>9 years	68	36.2
System experience
3–6 months	54	28.7
6 months–1 year	51	27.1
>1 year	83	44.1
Medical experience
<1 year	40	21.3
1–5 years	55	29.3
>5 years	93	49.5

We tested the scale for multivariate normality, and as is illustrated in Table 1, the assumption of normality is violated for the variable “error reduction” because its kurtosis exceeded 2.58.⁶⁵ If the normality is violated, the partial least squares (PLS) approach, as the second generation of the quantitative analyses, should be used to analyze the data.⁶⁶ SmartPLS version 2.0.M3 (SmartPLS GmbH, Boenningstedt, Germany) was used for the data analysis. PLS is not sample size intensive,⁶⁶ and the number of respondents in this research meets the minimum required sample size for PLS.

The scale reliability was tested by Cronbach's alpha and composite reliability, and as shown in Table 1, all the values for the variables exceeded 0.7, indicating a perfect scale reliability. Scale validity was investigated by confirmatory factor analysis. As shown in Supplementary Table S1 (Supplementary Data are available online at www.liebertpub.com/tmj), the loading range for all the question items was from 0.70 to 0.88 (p<0.001, 6.78<t<52.97). Furthermore, the rule of Fornell and Larcker⁶⁷ was observed in the present study. It was found that item loadings on their corresponding factors are higher than their cross loadings. In addition, as shown in Table 1, all the average variance extracted values for all the variables exceeded 0.5. These imply the convergent and discriminate validities of the scale.

As shown in Figure 2, the outcomes of the bootstrapping test showed that the relationships between the interface consistency of e-prescribing systems with the reduction of prescribing errors and mental workload were positive and significant (p<0.01). Ease of use of e-prescribing systems was significantly predictive of the reduction of users' mental workload (p<0.01) and prescribing errors (p<0.05). The results showed that although information quality significantly predicted physicians' reduction of prescribing errors (p<0.01), it did not have any significant influence on the reduction of their mental workload. There was also a significant relationship between system error prevention with reduction of mental workload (p<0.01), whereas no significant relationship was found between error prevention and prescribing error. The results illustrated a significant relationship between the reduction of mental workload and prescribing errors among the physicians (p<0.01). Thus, as summarized in Table 3, all the hypotheses except H3 and H6 were empirically supported.

Fig. 2.

Bootstrapping results.

Table 3.

Results of the Hypotheses

HYPOTHESIS	P VALUE	RESULT
H1. Ease of use→reduction of prescribing errors	<0.05	Supported
H2. Information quality→reduction of prescribing errors	<0.01	Supported
H3. Error prevention→reduction of prescribing errors	>0.05	Not supported
H4. Interface consistency→reduction of prescribing errors	<0.01	Supported
H5. Ease of use→reduction of mental workload	<0.01	Supported
H6. Information quality→reduction of mental workload	>0.05	Not supported
H7. Error prevention→reduction of mental workload	<0.01	Supported
H8. Interface consistency→reduction of mental workload	<0.05	Supported
H9. Reduction of mental workload→reduction of prescribing errors	<0.001	Supported

Discussion

Although EHS, including e-prescribing systems, are expected to have positive impacts on the quality of care, including the reduction of errors, the results of the past research have shown that such systems are not always a source of improvement in the quality of care and reduction of errors. Two factors were suggested as the key players in this context: system usability and users' workload. However, some gaps were identified in this regard. The first gap was in relation to the empirical relationship between users' workload and medical errors: none of the past research has examined the relationship that may exist between these two variables. The second gap deals with the research methods and data analysis techniques used in the past research. Despite the advantages of the past research in developing this field, most of them have either used qualitative methods—which suffer from the researchers' subjectivity and also limits the generalization of the findings²⁹—or have applied quantitative techniques without observing the necessities and high standards of quantitative methods and data analyses, such as collecting data from sufficient sample size, testing the reliability or validity of the scale, etc. Consequently, the findings of much of the past quantitative research do not provide precise quantitative information and results.³⁰ Moreover, the past research has hardly developed and validated a theoretical model for usability.^13,32

The above shortcomings motivated us to use a quantitative research method and investigate the impacts of e-prescribing systems' usability on the reduction of community physicians' mental workload and prescribing errors. To do this, nine hypotheses were proposed, a self-report survey instrument was adopted and adapted from the past published sources, and the questionnaires were distributed among 256 community physicians who had working experience of at least 3 months with e-prescribing systems. Because the normality of the data was not met for the variable “error reduction,” the PLS technique, which is robust against the data normality violations,⁶⁶ was applied to test the hypotheses. The results illustrated that apart from the impact of information quality on the reduction of mental workload and the impact of error prevention features of e-prescribing system on the reduction of prescribing errors, all the other hypotheses were empirically supported. The following paragraphs deal with the discussion of our findings.

Some scholars have proposed interface consistency in terms of menus, words, and buttons as one of the most significant features in the design of EHS.^9,42,68 User interface consistency enables physicians to better concentrate on various aspects of their work. They can focus more on the precision of diagnoses and accurate prescribing rather than concentrating on remembering and identifying numerous icons and terminologies while working with the system. Consequently, this can decrease their mental workload and result in fewer errors. The results suggest that the user interface in e-prescribing systems should be designed in a way that shows consistency in the terminologies, menus, and words used across various screens of the system. Furthermore, the system terminologies need to be similar to those used in the real world. On the other hand, if the consistency standards are not observed in the design and development of EHS, there is the possibility of increasing users' mental workload and prescribing errors.

Another variable examined was the information quality generated by e-prescribing systems. A user's understanding about the output value of an information system is regarded as information quality.⁶⁹ Decision making in medical centers is information intensive,⁷⁰ and quality information is a critical factor in clinics. The results of this research reveal that obtaining quality information is a significant predictor for the reduction of prescribing errors.

The outcomes also propose that information details generated by e-prescribing systems should provide the information required by the doctors. Information needs to be precise, appropriate, and accessible to reduce physicians' prescribing errors. An e-prescribing system can reduce errors by providing useful information on different aspects of a patient's medication process such as ordered medicine details, laboratory tests, etc.

Information quality enables doctors to effectively get the required information about the patient's health and medical histories and to diagnose patients' problems accordingly. This can make e-prescribing systems a valuable source of assistance for physicians in analyzing critical events, incidents, and special cases in a patient's health and medication process. Information quality also enables doctors to prescribe the most appropriate medicine for their patients. Therefore, provision of quality information can assist doctors in preventing medical errors. On the other hand, if e-prescribing systems do not produce good quality information for doctors, this can jeopardize patient safety. This is consistent with Holden,⁴² who observed that two-fifths of doctors who used e-prescribing systems stated that the system jeopardizes patients' safety because of doctors' dependence on inaccurate information of the system.

The results, however, found no significant empirical relationship between information quality and reduction of mental workload. A possible explanation for this finding may be that despite the implementation of an e-prescribing system with good usability, physicians still do not fully trust the information generated by the system, and they double check the data using their own knowledge and experience. It is because doctors are aware of the consequences of wrong diagnoses and prescribing based on the wrong information. Hence, they do not only rely on the information provided by the system and analyze the situation themselves. Therefore, quality information does not reduce their mental workload.

This study has also examined the impact of ease of use, which refers to the complexity of the system in a reverse manner.⁷¹ We observed that system ease of use is the most important factor in the reduction of mental workload. Our findings show that, generally, any e-prescribing features that improve the ease of use of the system can result in the reduction of a user's mental workload. We also found that the ease of use of e-prescribing systems leads to the reduction of prescribing errors. This finding is consistent with the past research.^12,39,40 The results imply that e-prescribing systems should be designed in a way that makes the user interface more friendly to reduce physicians' mental workload and prescribing errors. For example, a system should enable doctors to easily select and prescribe the medicine from a list of medicines or enable them to prescribe the medicine in the system in a few simple steps. As a result, these steps will decrease the mental workload of the doctors who use the system. Moreover, using an easy-to-use system can help doctors concentrate on different aspects of the delivery of care services rather than concentrating and struggling with the system menus and features, which can help them in reducing prescribing errors.

Error prevention was another variable that was examined in this study with regard to its influences on the reduction of mental workload and prescribing. Error prevention refers to the feature of an information system that enables the users to simply prevent or recover from the system errors.⁷² The results showed that the error prevention features of e-prescribing systems have a significant influence on the reduction of mental workload, although the analysis found no significant relationships between error prevention and the reduction of prescribing errors. One possible explanation is that error prevention features of e-prescribing systems have no direct impacts on doctors' quality and accuracy of diagnoses or prescribing processes. Although such features help users prevent or recover from system bugs and errors, they are not associated with improving the quality of diagnoses or prescribing activities of the doctors, and consequently they do not have any impact on the reduction of prescribing errors. However, because such features help doctors prevent or recover from system errors, they reduce doctors' mental workload.

Another finding of this research was regarding the impact of mental workload reduction with the reduction of prescribing errors. Although it is believed that increased mental and physical workloads make errors more likely to happen in clinics,¹⁹ there were no empirical data in this regard. Our analyses, however, were a step forward in this field. This finding illustrates that when physicians' mental workload is reduced—while working with an e-prescribing system—they can better focus on the care delivery process, including diagnosis and prescribing, and hence there is less likelihood of prescribing errors occurring. The results imply that if the developers of an e-prescribing system intend to develop their system in a way that reduces physicians' prescribing errors, they need to initially design it in a way that it reduces users' mental workload.

The results of the present study show different novelties. As discussed earlier, the previous usability studies in this field have rarely used theoretical models to study the impacts of system usability.^13,32 In order to address this gap, we used and validated the usability model of Peikari et al.¹² to examine the direct influence of usability on the reduction of physicians' mental workload and prescribing errors. Also, there were no quantitative models illustrating the relationships between e-prescribing system usability with the reduction of mental workload, and this is a significant novelty of this study. We collected data from a methodologically adequate sample size and used appropriate, advanced analyses techniques. Thus, results are much more precise and generalizable compared with those of the previous studies. Furthermore, this is the first study to use PLS to empirically examine the relationships between e-prescribing systems usability and the reduction of mental workload. Also, the mental workload reduction scale, validated in this study, can be used by scholars in their future research.

There are, however, some limitations in the present research that should be considered for future research. First, prescribing errors and mental workload are not reduced only through e-prescribing usability, but some other managerial and organizational factors and solutions, which were not included in this study, can also play important roles in this regard. Moreover, the reduction of prescribing errors and mental workload by the system usability can be predicted by many other usability factors that were not used in our research model. This model can be extended in future research.

We also suggest researchers consider users' characteristics such as their experience with the system, optimism toward the technology, and computer knowledge as the moderators between system usability and users' outcomes such as reduction of errors and mental workload. We also suggest that future attempts use comparative approaches and collect data on system complexity from the respondents who have tried to do complex tasks with the system.

Another limitation of this research is that the data for this research were collected from community physicians who had worked with e-prescribing systems for a minimum of 3 months. The respondents were categorized into three subgroups in that regard: those with 3–6 months, 6–12 months, and over 1 year of experience of working with the system. It would be interesting to also to have a subgroup for those with over 2 or even 3 years of experience in future research and know whether the mental workload and prescribing errors with the system correlate with the users' experience with the system. Another suggestion for future research is that researchers study if the health professionals' reliance on the system correlates with their experience with the system.

It is also suggested that future research differentiates between system impacts on the errors that occur during the diagnoses process, as well as those that occur in the prescribing stage. Also, future research should consider more outcome variables such as communication, decision making quality, or employee empowerment and examine the relationships between the system usability and these outcomes with the reduction of errors. It is also suggested that factors that have an impact on a physician's trust in the system and its information should be studied.

Footnotes

Disclosure Statement

No competing financial interests exist.

References

Bosman

. Impact of computerized information systems on workload in operating room and intensive care unit. Best Pract Res Clin Anaesthesiol, 2009; 23:15–26.

Blaser

, Schnabel

, Biber

, et al. Improving pathway compliance and clinician performance by using information technology. Int J Med Inform, 2007; 76:151–156.

Bates

, Cohen

, Leape

, et al. Reducing the frequency of errors in medicine using information technology. J Am Inform Assoc, 2001; 8:299–308.

Eslami

, de Keizer

, Abu-Hanna

. The impact of computerized physician medication order entry in hospitalized patients—A systematic review. Int J Med Inform, 2008; 77:365–376.

Lee

, Lee

, Schneiderjans

. Medical error reduction: The effect of employee satisfaction with organizational support. Serv Indust J, 2011; 31:1311–1325.

Effken

, Loeb

, Kang

, et al. Clinical information displays to improve ICU outcomes. Int J Med Inform, 2008; 77:765–777.

Pai

F-Y

, Huang

K-I

. Applying the Technology Acceptance Model to the introduction of healthcare information systems. Technol Forecast Soc Change, 2011; 78:650–660.

Khajouei

, Wierengac

, Hasmana

, et al. Clinicians satisfaction with CPOE ease of use and effect on clinicians' workflow, efficiency and medication safety. Int J Med Inform, 2011; 80:297–309.

Khajouei

, Jaspers

MWM

. The impact of CPOE medication systems' design aspects on usability, workflow and medication orders: A systematic review. Methods Inf Med, 2010; 49:3–19.

10.

Ammenwerth

, Rauchegger

, Ehlers

, et al. Effect of a nursing information system on the quality of information processing in nursing: An evaluation study using the HIS-monitor instrument. Int J Med Inform, 2011; 80:25–38.

11.

Han

, Carcillo

, Venkataraman

, Clark

RSB

, Watson

, Nguyen

, Bayir

, Orr

. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics, 2005; 116:1506–1512.

12.

Peikari

, Zakaria

, Norjaya

, et al. Role of CPOE usability in the reduction of prescribing errors. Healthc Inform Res, 2013; 19:93–101.

13.

Peikari

, Shah

, Zakaria

, et al. The impacts of second generation e-prescribing usability on community pharmacists outcomes. Res Social Adm Pharm, 2014; 11:339–351.

14.

Koppel

, Metlay

, Cohen

, et al. Role of computerized physician order entry system in facilitating medication errors. JAMA, 2005; 293:1197–1203.

15.

Abdel-Qader

, Harper

, Cantrill

, et al. Pharmacists' interventions in prescribing errors at hospital discharge: An observational study in the context of an electronic prescribing system in a UK teaching hospital. Drug Saf, 2010; 33:1027–1044.

16.

Mäenpää

, Suominen

, Asikainen

, et al. The outcomes of regional healthcare information systems in health care. A review of the research literature. Int J Med Inform, 2009; 78:757–771.

17.

Ridley

. Relating nursing workload to quality of care in child and adolescent mental health inpatient services. Int J Health Care Qual Assur, 2007; 20:429–440.

18.

Lindberg

, Rosenqvist

. Implementing TQM in the health care service: A four-year following-up of production, organisational climate and staff wellbeing. Int J Health Care Qual Assur, 2005; 18:370–384.

19.

Padilha

, Sousa

RMC

, Kimura

, et al. Nursing workload in intensive care units: A study using the Therapeutic Intervention Scoring System-28 (TISS-28). Intensive Crit Care Nurs, 2007; 23:162–169.

20.

Young

, Hooper

. Assessment of workload using NASA Task Load Index in perianesthesia nursing. J Perianesth Nurs, 2008; 23:102–110.

21.

Holden

, Patel

, Scanlon

, et al. Effects of mental demands during dispensing on perceived medication safety and employee well-being: A study of workload in pediatric hospital pharmacies. Res Social Adm Pharm, 2010; 6:293–306.

22.

Reckmann

, Westbrook

, Koh

, Lo

, Day

. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc, 2009; 16:613–623.

23.

John

, Peter

, Chacko

, et al. A computer-assisted recording, diagnosis and management of the medically ill system for use in the intensive care unit: A preliminary report. Indian J Crit Care Med, 2009; 13:136–142.

24.

Horsky

, Kuperman

, Patel

. Comprehensive analysis of a medication dosing error related to CPOE. J Am Med Inform Assoc, 2005; 12:377–382.

25.

Kushniruk

, Triola

, Borycki

, et al. Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application. Int J Med Inform, 2005; 74:519–526.

26.

Chan

, Shojania

, Easty

, Etchells

. Does user-centred design affect the efficiency, usability and safety of CPOE order sets?. J Am Med Inform Assoc, 2011; 18:276–281.

27.

Cho

, Park

, Choi

, et al. Understanding the nature of medication errors in an ICU with a computerized physician order entry system. PLoS One, 2014; 9:e114243.

28.

Oztekin

, Kong

, Uysal

. UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. Int J Indust Ergonom, 2010; 40:455–469.

29.

González

, Masip

, Granollers

, et al. Quantitative analysis in a heuristic evaluation experiment. Adv Eng Software, 2009; 40:1271–1278.

30.

Weir

, Staggers

, Laukert

. Reviewing the impact of computerized provider order entry on clinical outcomes: The quality of systematic reviews. Int J Med Inform, 2012; 81:219–231.

31.

Corrao

, Robinson

, Swiernik

, et al. Importance of testing for usability when selecting and implementing an electronic health or medical record system. J Oncol Pract, 2010; 6:120–124.

32.

Yen

P-Y

, Bakken

. Review of health information technology usability study methodologies. J Am Med Inform Assoc, 2012; 19:413–422.

33.

K-W

, Liu

C-L

. A mobile nursing information system based on human-computer interaction design for improving quality of nursing. J Med Syst, 2012; 36:1139–1153.

34.

Babaei

, Ganjalikhani

. Medical informatics: On an ongoing road to the final destination. Sri Lanka J Biomed Inform, 2010; 1:186–190.

35.

Edrees

, Paine

, Feroli

, et al. Health care workers as second victims of medical errors. Pol Arch Med Wewnj, 2011; 121:101–107.

36.

Cooper

. Decision production: A step toward a theory of deCISion-makerial information requirements. Int Conf Inf Syst, 1983; 251–268.

37.

Etezadi-Amoli

, Farhoomand

. A structural model of end user computing satisfaction and user performance. Inf Manage, 1996; 30:65–73.

38.

Daugherty

, Richey

, Genchev

, et al. Reverse logistics: Superior performance through focused resource commitments to information technology. Transport Res E, 2005; 41:77–92.

39.

Bates

, Teich

, Lee

, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc, 1999; 6:313–321.

40.

Peute

LWP

, Jaspers

MWM

. The significance of a usability evaluation of an emerging laboratory order entry system. Int J Med Inform, 2007; 76:157–168.

41.

Kositanurit

, Osei-Bryson

K-M

, Ngwenyama

. Re-examining information systems user performance: Using data mining to identify properties of IS that lead to highest levels of user performance. Expert Syst Applic, 2011; 38:7041–7050.

42.

Holden

. Physicians' beliefs about using EMR and CPOE: In pursuit of a contextualized understanding of health IT use behavior. Int J Med Inform, 2010; 79:71–80.

43.

Wilkin

, Castleman

. Development of an instrument to evaluate the quality of delivered information systems. In: Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS38). Piscataway, NJ: IEEE, 2003. doi: 10.1109/HICSS.2003.1174627.

44.

Lee

T-T

, Mills

, Bausell

, et al. Two-stage evaluation of the impact of a nursing information system in Taiwan. Int J Med Inform, 2008; 77:698–707.

45.

Bastien

JMC

. Usability testing: A review of some methodological and technical aspects of the method. Int J Med Inform, 2010; 79:e18–e23.

46.

Devaraj

, Kohli

. Performance impacts of information technology: Is actual usage the missing link?. Manage Sci, 2003; 49:273–289.

47.

Conway

, Lance

. What reviewers should expect from authors regarding common method bias in organizational research. J Business Psychol, 2010; 25:325–334.

48.

Fleenor

, Smither

, Atwater

, et al. Self–other rating agreement in leadership: A review. Leadersh Q, 2010; 21:1005–1034.

49.

Fraenkel

, Cowie

, Daley

. Quality benefits of an intensive care clinical information system. Crit Care Med, 2003; 31:120–125.

50.

Saleem

, Patterson

, Militello

, et al. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J Am Med Inform Assoc, 2007; 14:632–640.

51.

Yang

C-W

, Yenn

T-C

, Lin

. Assessing team workload under automation based on a subjective performance measure. Saf Sci, 2010; 48:914–920.

52.

Ahn

, Ryu

, Han

. The impact of Web quality and playfulness on user acceptance of online retailing. Inf Manage, 2007; 44:263–275.

53.

Goetzinger

, Park

, Lee

, et al. Value-driven consumer e-health information search behavior. Int J Pharm Healthc Market, 2007; 1:128-142.

54.

Joshi

, Sarda

, Tripathi

. Measuring effectiveness of HCI integration in software development processes. J Syst Software, 2010; 83:2045–2058.

55.

Mwachofi

, Walston

, Al-Omar

. Factors affecting nurses' perceptions of patient safety. Int J Health Care Qual Assur, 2011; 24:274–283.

56.

Oroviogoicoechea

, Watson

. A quantitative analysis of the impact of a computerised information system on nurses' clinical practice using a realistic evaluation framework. Int J Med Inform, 2009; 78:839–849.

57.

Oztekin

, Nikov

, Zaim

. UWIS: An assessment methodology for usability of web-based information systems. J Syst Software, 2009; 82:2038–2050.

58.

Tully

, Buchan

. Prescribing errors during hospital inpatient care: Factors influencing identification by pharmaHISts. Pharm World Sci, 2009; 31:682–688.

59.

Tung

F-C

, Chang

S-C

, Chou

C-M

. An extension of trust and TAM model with IDT in the adoption of the electronic logistics information system in HIS in the medical industry. Int J Med Inform, 2008; 77:324–335.

60.

Oroviogoicoechea

, Watson

, Beortegui

, et al. Nurses' perception of the use of computerised information systems in practice: Questionnaire development. J Clin Nurs, 2010; 19:240–248.

61.

Melas

, Zampetakis

, Dimopoulou

, et al. Modeling the acceptance of clinical information systems among hospital medical staff: An extended TAM model. J Biomed Inform, 2011; 44:553–564.

62.

Nevo

, Wade

. How to avoid disappointment by design. Commun ACM, 2007; 50:43–48.

63.

Krejcie

, Morgan

. Determining sample size for research activities. Educ Psychol Measur, 1970; 30:607–610.

64.

Prichard

, Bizo

, Stratford

. Evaluating the effects of team-skills training on subjective workload. Learn Instruct, 2011; 21:429–440.

65.

Hair

, Black

, Babin

, Anderson

. Multivariate data analysis, 7th ed. Upper Saddle River, NJ: Prentice Hall, 2010.

66.

Hair

, Ringle

, Sarstedt

. PLS-SEM: Indeed a silver bullet. J Market Theory Pract, 2011; 19:139–151.

67.

Fornell

, Larcker

. Evaluating structural equation models with unobservable variables and measurement error. J Market Res, 1981; 18:39–50.

68.

Zhang

, Johnson

, Patel

, et al. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform, 2003; 36:23–30.

69.

Chen

C-W

. Impact of quality antecedents on taxpayer satisfaction with online tax-filing systems—An empirical study. Inf Manage, 2010; 47:308–315.

70.

Aggelidis

, Chatzoglou

. Hospital information systems: Measuring end user computing satisfaction (EUCS). J Biomed Inform, 2012; 45:566–579.

71.

Hernandez

, Jimenez

, Martin

. The impact of self-efficacy, ease of use and usefulness on e-purchasing: An analysis of experienced e-shoppers. Interact Comput, 2009; 21:146–156.

72.

Daniels

, Fels

, Kushniruk

, et al. A framework for evaluating usability of clinical monitoring technology. J Clin Monit Comput, 2007; 21:323–330.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.02 MB

0.00 MB