Healthcare staff digital literacy levels and their attitudes towards information systems

Abstract

This study aimed to assess the digital literacy levels and attitudes towards information systems of staff in a health service that will be implementing an electronic health record so that barriers towards implementation could be addressed. A survey measuring staff confidence levels and their attitudes towards information systems was developed. Data were collected over a five-week period, with data analysed using frequency analysis and a chi-square analysis. There were 407 respondents to the survey. The majority (70-80%) of which reported high digital literacy levels, expressing confidence in using technology. Respondents also reported positive attitudes towards information systems. However, one-fifth reported anxiety using information systems. Given poor staff engagement with information systems adversely affects the safety and quality of patient care, health services should provide targeted education and training to address staff with low digital literacy levels and/or confidence with using information systems prior to implementation of an electronic health record system.

Keywords

attitudes digital literacy electronic health records healthcare professional training healthcare staff information systems

Introduction

Research has shown that digital information can transform the quality and sustainability of health and care. In a 2015 study, 62 per cent of articles reviewed (n = 154) found that health information technology was associated with improvement in one or more aspects of care.¹ In this study, it was identified that health information technology had a positive impact on access to care, preventive care, care process, patient satisfaction, effectiveness of care and efficiency of care.¹ This has been recognised globally with health services in various countries implementing electronic health records (EHRs).^2,3

In Australia, state and territory governments have been investing in information systems (IS) and technology, and implementing state-wide EHRs.⁴ Federal and state governments have also developed digital health strategies, with the implementation of digital clinical systems a cornerstone of these strategies.^5–7 Hence, it is expected that the adoption of EHRs and IS across Australian healthcare services in the coming years will continue.

Digital literacy is defined as ‘being able to make use of technologies to participate in and contribute to modern social, cultural, political and economic life’ (para 2).⁸ This includes things such as the ability to utilise the IS as well as the supporting infrastructure. For example, skills in typing and entering information via a keyboard and the ability to use a mouse. Previous studies have identified that digital literacy levels impact staff engagement with IS, with poor computer skills and low computer experience identified to influence attitudes towards IS and information and communication technology (ICT), and to be one of the main barriers for adopting EHRs.^9–11 It has also been identified that for EHRs to have a positive impact on patient safety, clinicians must be able to use these systems effectively after they are made available.¹⁰ Thus, given the increasing use of technology in healthcare, it is important for healthcare staff to be digitally literate.^12,13 Furthermore, poor attitudes towards IS have been found to negatively impact staff engagement levels with IS, with poor staff engagement with IS adversely affecting the safety and quality of patient care.^14,15

There has been minimal research into healthcare staff digital literacy levels, with the findings of the studies that have been conducted generally demonstrating that staff have low levels of digital literacy.^16–19 In a recent systematic review that looked at English language studies relating to pharmacy staff digital literacy levels and computer skills training, Australian, Canadian and American hospital and community pharmacy staff were identified to lack digital literacy knowledge.¹⁸ In addition, a 2012–2013 study of Scottish pharmacy staff found that staff perceived their digital literacy to be at a basic level, citing a lack of confidence in using ICT.^17,19 Similar findings were reflected in a cross-sectional study conducted with 500 healthcare professionals, where 81.1 per cent of the surveyed population demonstrated poor computer knowledge.²⁰ Results specifically within the nursing profession concur with these findings. In Australia, a survey of over 4000 nurses showed that nurses’ experience and confidence in using ICT is only limited to basic computer applications, with nurses expressing low confidence in the use of computer-based applications.¹⁶ Similarly, a study assessing computer hardware and software usage identified that American nurses working at a community hospital had low computer literacy levels, with nurses stating that they had little to no experience with seven out of the 15 hardware and software items surveyed.²¹

In contrast, there have been mixed results in previous studies of healthcare staff attitudes towards IS. Previous studies have identified that nurses, physicians, social workers, dietitians, unit clerks and patient attendants hold positive attitudes towards IS.^22–30 In these studies, healthcare professionals stated that IS assisted them to do their job, improve the safety and quality of patient care, avoid duplication, increase ease of access, assist with quick decision-making and increase efficiency.^22–30 However, healthcare professionals in other studies have identified frustrations with IS, with healthcare staff disagreeing that IS improve patient care and increase efficiency,^31–33 and healthcare staff citing IS to be technically cumbersome and time-consuming.^27,34,35

Although studies around healthcare staff digital literacy levels and attitudes towards IS have been conducted, there is a lack of Australian literature with only two of the afore mentioned studies conducted within the Australian healthcare setting. Thus, the primary aims of this project were to assess Australian healthcare staff digital literacy levels and their attitudes towards IS.

We also aimed to determine whether there were any differences in digital literacy and/or attitudes towards IS between clinical and non-clinical staff and between older (>50 years) and younger (⩽50 years) staff.

Methods

Study site

The study was performed in a 997-bed tertiary health service, providing acute, sub-acute, mental health, aged care and community services located in regional Victoria, which has a total of 6775 employees.

The health service utilises a mix of paper and IS to facilitate core business processes. The core health record is a digital medical record (hand-written paper notes scanned at the completion of the episode of care), patient administration functions (e.g. management of demographic information) are supported by an IS, and some specialist services have implemented clinical IS to support their workflows (e.g. emergency department and intensive care). From a corporate perspective, core finance and payroll functions are supported by an IS; however, there are gaps in areas such as rostering and basic workflow support (e.g. authorisation of invoice payment and staff leave requests).

The health service has developed a digital strategy with a view to becoming a paper-lite organisation, including implementation of an EHR over the next five years.

Survey design

A review of the literature identified a number of survey instruments that had previously been used to asses healthcare staff digital literacy levels. However, these instruments were found to be

Too long and not validated;³⁶

Validated and reliable but created more than 20 years ago;³⁷

Validated but only in a specific healthcare discipline such as nursing.³⁸

A review of the broader literature in non-health settings found digital literacy to be most commonly assessed by (1) measuring users’ frequency of use and (2) measuring users’ confidence levels, with these studies using a Likert-type scale.^39–41 Self-efficacy, also known as the belief in one’s capabilities to organise and execute courses of actions required to accomplish a task, was also found to be an important factor that separates digitally literate users from novices, with prior experience identified to be the strongest predictor of self-efficacy.³⁹ Hence, 13 survey questions related to users’ confidence levels and frequency of use for commonly used hardware/software devices were developed, with a five-point Likert-type scale used to measure digital literacy in this study (Appendix 1).

Questions from the technology acceptance model (TAM) and the unified theory of acceptance and use of technology (UTAUT) model were used to assess healthcare staff attitudes to IS (Appendix 1).

The TAM is a measurement scale for predicting user acceptance of computers and has two dimensions: perceived usefulness and perceived ease of use.¹⁴ Each dimension comprises six items rated on a seven-point Likert-type scale, with responses ranging from one to seven, representing ‘extremely unlikely’, ‘quite unlikely’, ‘slightly unlikely’, ‘neither’, ‘slightly likely’, ‘quite likely’ and ‘extremely unlikely’.^14,42,43

The UTAUT is a TAM based on the theory of planned behaviour to explain and predict user acceptance and use of information technology.^44,45 It has 31 questions, with questions scored using a seven-point Likert-type scale with responses ranging from one to seven, representing ‘strongly disagree’, ‘moderately disagree’, ‘somewhat disagree’, ‘neither agree nor disagree’, ‘somewhat agree’, ‘moderately agree’ and ‘strongly agree’.^44,46,47 The model has eight domains and these domains were found to account for 70 per cent of the variance in usage intention.⁴⁴ However, for this survey, only questions from four domains (attitudes towards technology, social influence, facilitating conditions and anxiety) were used, as two of the domains (perceived usefulness and perceived ease of use) were from the TAM and the other two domains were only applicable if the system was already being used in the organisations.

There have been a number of previous studies that have utilised the TAM, the UTAUT model or an integrated model of the TAM and the UTAUT.^{23,25,32,40,41,45,48–51} These instruments have been used across an extensive range of health professions such as nurses, physicians, occupational therapists and physiotherapists, as well as across a wide range of ethnic groups.^{23,25,32,47–51}

The TAM and the UTAUT are robust and have high reliability and validity.^14,44 These instruments have also been found to predict a substantial proportion of the use or acceptance of ICT and a significant proportion of staff intention to use clinical IS.^{14,44,47,52,53} The most basic version of the TAM model has been found to account for between 30 and 40 per cent of information technology acceptance,⁵³ and the UTAUT accounting for 70 per cent of the variance in usage intention.⁴⁴

Data collection

Ethical approval was obtained from the health service’s research ethics committee (ethics reference no. 16/230) and La Trobe University and data were collected over a Five-week period in 2017.

Given the health service had 6775 staff at the time of the survey, it was calculated that based on a five-per cent margin of error and a 95-per cent confidence interval, at least 364 individuals needed to participate in this study to achieve statistical significance.

A link to the online survey was distributed to staff via a variety of methods (e.g. newsletters, email and intranet). Paper copies of the questionnaire were also distributed to 433 staff without access to a computer.

Data analysis

Data collected were aggregated and analysed using frequency analysis for survey respondents’ frequency of use and confidence. A chi-square test was used to compare survey respondents’ frequency of use, with survey responses dichotomised into frequent (Very often or Often) and infrequent users (Never, Rarely, or Sometimes).⁵⁴

Similarly, a chi-square test was used to compare survey respondents’ confidence levels, with responses dichotomised into Confident (Moderately confident or Completely confident) and Not confident (Not at all confident, Slightly confident or Somewhat confident).⁵⁵

Descriptive data (mean (SD)) were reported for responses on the TAM and UTAUT as both the TAM and the UTAUT scored responses to the questions from one to seven and previous studies using the TAM and UTAUT have reported their results descriptively.^43,47 T-tests were then utilised to analyse for differences between groups. A significance level of p ⩽ 0.05 was set for all analysis.

Results

Demographics

Of the 407 responses received (6% response rate), 366 (90%) were received electronically and 41 (10%) paper hard-copies were returned. The rate of missing data was less than five per cent of the sample size for majority of the questions and less than 10 per cent of the sample size for the UTAUT questions.

In general, the demographic details of the participant group were representative of all staff with respect to age and length of tenure at the health services. However, proportionally, when compared with clinical staff, there was a greater proportion of non-clinical staff responding to the survey (Table 1). Among clinical survey respondents, nurses comprised majority of the group (48.3%) with physicians accounting for 21.1 per cent.

Table 1.

Demographic details of survey respondents.

		Survey participant demographics (%)n = 407	All staff demographics (%)n = 6775
Age range (years)	⩽50	60.4	66.8
	>50	39.1	33.0
	Unknown	0.7	0.10
Length of tenure (years)	0–1	11.8	15.4
	1–3	12.3	16.8
	3–5	13.8	13.0
	5–10	16.2	20.8
	>10	45.9	34.0
Clinical/non-clinical staff	Clinical	51.4	71.3
	Non-clinical	48.4	28.7
	Unknown	0.2	0.0

Digital literacy levels

Survey respondents were frequent users of computers, Microsoft Office applications, email, smartphones and the Internet with more than 80 per cent using these devices/software ‘Often’ or ‘Very often’ (Figure 1). However, they were less frequent users of tablets and social media (Figure 1).

Figure 1.

Survey respondents’ frequency of use.

Based on the dichotomised survey responses, when compared with older staff (>50 years), younger staff (⩽50 years) were identified to be more frequent users of computers, mobile devices, email, the Internet and social media, with these differences found to be significantly different (Table 2).

Table 2.

Age differences in frequency of use.

Variable	Dichotomised groups	Frequent user (%)	Infrequent user (%)	χ² value^a	p-value
Computers	⩽50 (n = 245)	95.5	4.5	12.602	0.000
	>50 (n = 158)	85.4	14.6	12.602	0.000
	Clinical (n = 208)	97.1	2.9	19.035	0.000
	Non-clinical (n = 197)	84.8	8.9	19.035	0.000
Microsoft applications	⩽50 (n = 244)	83.6	16.4	2.377	0.123
	>50 (n = 155)	77.4	22.6	2.377	0.123
	Clinical (n = 208)	79.8	20.2	0.273	0.601
	Non-clinical (n = 193)	81.9	18.1	0.273	0.601
Smartphones	⩽50 (n = 245)	89.4	10.6	10.331	0.001
	>50 (n = 156)	77.6	22.4	10.331	0.001
	Clinical (n = 208)	84.6	15.4	0.021	0.886
	Non-clinical (n = 195)	85.1	14.9	0.021	0.886
Tablets	⩽50 (n = 243)	53.9	46.1	0.788	0.375
	>50 (n = 156)	49.4	50.6	0.788	0.375
	Clinical (n = 208)	49.5	50.5	0.957	0.328
	Non-clinical (n = 193)	54.4	45.6	0.957	0.328
Email	⩽50 (n = 245)	95.5	4.5	9.149	0.002
	>50 (n = 157)	87.3	12.7	9.149	0.002
	Clinical (n = 208)	94.7	5.3	4.136	0.042
	Non-clinical (n = 195)	89.2	10.8	4.136	0.042
The Internet	⩽50 (n = 245)	94.3	5.7	11.703	0.001
	>50 (n = 155)	83.9	16.1	11.703	0.001
	Clinical (n = 208)	92.3	7.7	1.986	0.159
	Non-clinical (n = 194)	88.1	11.9	1.986	0.159
Social media	⩽50 (n = 244)	67.6	32.4	31.494	0.000
	>50 (n = 156)	39.1	60.0	31.494	0.000
	Clinical (n = 208)	58.7	41.3	0.501	0.479
	Non-clinical (n = 194)	57.0	43.9	0.501	0.479

χ² tests were conducted based on 1 degree of frequency (df = 1).

Clinical and non-clinical staff reported similar levels of use of information technology for all categories except computers and email where clinical staff reported using these modalities significantly more frequently (Table 2).

Participants expressed high confidence levels with computers, Microsoft Word, smartphones, email and the Internet with either close to or more than 80 per cent stating that they were ‘moderately confident’ or ‘completely confident’ using these devices/software (Figure 2). However, they expressed lower levels of confidence when asked about Microsoft Excel and Microsoft Access (Figure 2), with approximately 50 and 20 per cent stating that they were ‘moderately confident’ or ‘completely confident’, respectively. Furthermore, only 51 per cent stated that they were ‘moderately confident’ or ‘completely confident’ when asked about non-clinical IS (Figure 2).

Figure 2.

Survey respondents’ confidence levels.

Analysis of dichotomised results revealed that, compared to staff >50 years of age, staff ⩽50 years reported significantly higher confidence levels across all domains except for Microsoft Access (Table 3).

Table 3.

Age and clinical/non-clinician staff differences in confidence levels.

Variable	Dichotomised groups	Confident (%)	Not confident (%)	χ² value^a	p-value
Computers	⩽50 (n = 244)	91.0	9.0	25.105	0.000
	>50 (n = 157)	72.0	28.0	25.105	0.000
	Clinical (n = 207)	82.6	17.4	0.081	0.775
	Non-clinical (n = 196)	83.7	16.3	0.081	0.775
Microsoft Word	⩽50 (n = 243)	85.2	14.8	13.789	0.000
	>50 (n = 155)	69.7	30.3	13.789	0.000
	Clinical (n = 207)	77.8	22.2	0.242	0.623
	Non-clinical (n = 193)	79.8	20.2	0.242	0.623
Microsoft PowerPoint	⩽50 (n = 242)	72.7	27.3	29.748	0.000
	>50 (n = 154)	45.5	54.5	29.748	0.000
	Clinical (n = 207)	60.9	39.1	0.086	0.769
	Non-clinical (n = 191)	62.3	37.7	0.086	0.769
Microsoft Excel	⩽50 (n = 243)	58.8	41.2	17.977	0.000
	>50 (n = 154)	37.0	63.0	17.977	0.000
	Clinical (n = 207)	41.5	58.5	12.665	0.000
	Non-clinical (n = 192)	59.4	40.6	12.665	0.000
Microsoft Access	⩽50 (n = 241)	25.3	74.7	3.283	0.070
	>50 (n = 154)	17.5	82.5	3.283	0.070
	Clinical (n = 207)	16.4	83.6	7.616	0.006
	Non-clinical (n = 190)	27.9	72.1	7.616	0.006
Smartphones	⩽50 (n = 242)	86.4	13.6	24.441	0.000
	>50 (n = 156)	65.4	34.6	24.441	0.000
	Clinical (n = 207)	79.2	20.8	0.134	0.714
	Non-clinical (n = 193)	77.7	22.3	0.134	0.714
Tablets	⩽50 (n = 246)	76.0	24.0	15.212	0.000
	>50 (n = 158)	57.6	42.4	15.212	0.000
	Clinical (n = 209)	71.3	28.7	1.089	0.297
	Non-clinical (n = 197)	66.5	33.5	1.089	0.297
Email	⩽50 (n = 243)	91.8	8.2	5.173	0.023
	>50 (n = 154)	84.4	15.6	5.173	0.023
	Clinical (n = 207)	88.4	11.6	0.141	0.708
	Non-clinical (n = 192)	89.6	10.4	0.141	0.708
The Internet	⩽50 (n = 243)	93.8	6.2	13.887	0.000
	>50 (n = 155)	81.9	18.1	13.887	0.000
	Clinical (n = 207)	90.8	9.2	1.104	0.293
	Non-clinical (n = 193)	87.6	12.4	1.104	0.293
Social media	⩽50 (n = 242)	74.4	25.6	43.676	0.000
	>50 (n = 155)	41.3	58.7	43.676	0.000
	Clinical (n = 207)	62.3	37.7	0.031	0.860
	Non-clinical (n = 192)	61.5	38.5	0.031	0.860
Clinical information systems	⩽50 (n = 220)	72.7	27.3	2.404	0.121
	>50 (n = 134)	64.9	35.1	2.404	0.121
	Clinical (n = 246)	72.3	27.7	1.498	0.221
	Non-clinical (n = 107)	66.2	33.8	1.498	0.221
Non-clinical information systems	⩽50 (n = 227)	52.0	48.0	0.192	0.661
	>50 (n = 145)	49.7	50.3	0.192	0.661
	Clinical (n = 190)	46.0	57.3	4.719	0.030
	Non-clinical (n = 181)	54.0	42.7	4.719	0.030

χ² tests were conducted based on 1 degree of frequency (df = 1).

When comparing clinical and non-clinical staff, there were minimal differences between the confidence of the two groups except for Microsoft Excel and Microsoft Access. With these applications, the non-clinical staff group reported having statistically significant higher confidence levels (Table 3).

Attitudes towards IS

Overall, the perceived usefulness questions scored the highest indicating that survey respondents perceived IS to be useful in their job. Furthermore, although perceived ease-of-use questions scored highly, these questions scored slightly lower than the perceived usefulness questions, with flexibility having the lowest score. Table 4 portrays the survey respondents’ answers to the TAM questions revealing their attitudes towards IS.

Table 4.

Survey responses to the TAM questions.

	Mean (SD)
Perceived usefulness
Using information systems would enable me to accomplish tasks more quickly (n = 391)	5.92 (1.33)
Using information systems in my job would improve my job performance (n = 391)	5.84 (1.33)
Using information systems in my job would increase my productivity (n = 390)	5.80 (1.34)
Using information systems would enhance my effectiveness on the job (n = 390)	5.87 (1.29)
Using information systems would make it easier to do my job (n = 390)	5.83 (1.37)
I would find information systems useful in my job (n = 390)	5.96 (1.25)
Perceived ease of use
Learning to operate information systems would be easy for me (n = 391)	5.66 (1.37)
I would find it easy to get information systems to do what I want them to do (n = 389)	5.13 (1.50)
My interaction with information systems would be clear and understandable (n = 390)	5.31 (1.40)
I would find information systems to be flexible to interact with (n = 391)	4.96 (1.57)
It would be easy for me to become skillful at using information systems (n = 390)	5.53 (1.38)
I would find information systems easy to use (n = 390)	5.36 (1.43)

TAM: technology acceptance model; SD: standard deviation.

1 = extremely unlikely; 2 = quite unlikely; 3 = slightly unlikely; 4 = neither unlikely nor likely; 5 = slightly likely; 6 = quite likely; 7 = extremely likely.

Across questions that evaluated perceived usefulness and ease of use in the TAM questionnaire, the average score for those participants aged ⩽50 years was higher (indicating a higher level of perceived usefulness) than those aged >50 years (Table 5). These differences were statistically significant for five out of the six perceived usefulness and all six ease-of-use questions (Table 5).

Table 5.

Age and clinical/non-clinical staff differences in TAM questions.

	Dichotomised groups	Mean (SD)	T	df	p-value
Perceived usefulness
Using information systems would enable me to accomplish tasks more quickly	⩽50 (n = 238)	6.08 (1.16)	2.820	387	0.005
	>50 (n = 151)	5.69 (1.54)	2.820	387	0.005
	Clinical (n = 203)	5.93 (1.23)	0.086	388	0.931
	Non-clinical (n = 187)	5.91 (1.44)	0.086	388	0.931
Using information systems in my job would improve my job performance	⩽50 (n = 238)	5.97 (1.18)	2.410	387	0.016
	>50 (n = 151)	5.64 (1.51)	2.410	387	0.016
	Clinical (n = 203)	5.83 (1.22)	−0.089	388	0.929
	Non-clinical (n = 187)	5.84 (1.43)	−0.089	388	0.929
Using information systems in my job would increase my productivity	⩽50 (n = 238)	5.95 (1.19)	2.855	386	0.050
	>50 (n = 151)	5.56 (1.51)	2.855	386	0.050
	Clinical (n = 203)	5.78 (1.29)	−0.286	387	0.775
	Non-clinical (n = 186)	5.82 (1.39)	−0.286	387	0.775
Using information systems would enhance my effectiveness on the job	⩽50 (n = 238)	6.02 (1.14)	2.956	386	0.003
	>50 (n = 150)	5.63 (1.48)	2.956	386	0.003
	Clinical (n = 203)	5.83 (1.21)	−0.497	387	0.620
	Non-clinical (n = 186)	5.90 (1.38)	−0.497	387	0.620
Using information systems would make it easier to do my job	⩽50 (n = 238)	5.97 (1.23)	2.616	386	0.009
	>50 (n = 150)	5.60 (1.54)	2.616	386	0.009
	Clinical (n = 203)	5.75 (1.36)	−1.111	387	0.267
	Non-clinical (n = 186)	5.90 (1.38)	−1.111	387	0.267
I would find information systems useful in my job	⩽50 (n = 238)	6.09 (1.10)	2.636	386	0.009
	>50 (n = 150)	5.75 (1.43)	2.636	386	0.009
	Clinical (n = 203)	5.96 (1.13)	−0.010	387	0.992
	Non-clinical (n = 186)	5.96 (1.36)	−0.010	387	0.992
Perceived ease of use
Learning to operate information systems would be easy for me	⩽50 (n = 238)	5.90 (1.14)	4.630	386	0.000
	>50 (n = 150)	5.26 (1.59)	4.630	386	0.000
	Clinical (n = 203)	5.60 (1.26)	−0.795	388	0.427
	Non-clinical (n = 187)	5.71 (1.45)	−0.795	388	0.427
I would find it easy to get information systems to do what I want them to do	⩽50 (n = 237)	5.28 (1.43)	2.633	385	0.009
	>50 (n = 150)	4.87 (1.58)	2.633	385	0.009
	Clinical (n = 202)	4.99 (1.53)	−1.971	386	0.049
	Non-clinical (n = 186)	5.28 (1.46)	−1.971	386	0.049
My interaction with information systems would be clear and understandable	⩽50 (n = 238)	5.45 (1.29)	2.677	385	0.008
	>50 (n = 148)	5.06 (1.53)	2.677	385	0.008
	Clinical (n = 203)	5.20 (1.40)	−1.639	387	0.102
	Non-clinical (n = 186)	5.43 (1.41)	−1.639	387	0.102
I would find information systems to be flexible to interact with	⩽50 (n = 238)	5.08 (1.51)	2.088	386	0.037
	>50 (n = 150)	4.74 (1.63)	2.088	386	0.037
	Clinical (n = 203)	4.71 (1.61)	−3.312	388	0.001
	Non-clinical (n = 187)	5.23 (1.48)	−3.312	388	0.001
It would be easy for me to become skillful at using information systems	⩽50 (n = 238)	5.77 (1.22)	4.609	385	0.000
	>50 (n = 149)	5.13 (1.52)	4.609	385	0.000
	Clinical (n = 203)	5.40 (1.33)	−2.001	387	0.046
	Non-clinical (n = 186)	5.68 (1.41)	−2.001	387	0.046
I would find information systems easy to use	⩽50 (n = 238)	5.62 (1.28)	4.587	386	0.000
	>50 (n = 150)	4.95 (1.55)	4.587	386	0.000
	Clinical (n = 203)	5.24 (1.37)	−1.865	387	0.063
	Non-clinical (n = 186)	5.51 (1.47)	−1.865	387	0.063

TAM: technology acceptance model; SD: standard deviation.

1 = extremely unlikely; 2 = quite unlikely; 3 = slightly unlikely; 4 = neither unlikely nor likely; 5 = slightly likely; 6 = quite likely; 7 = extremely likely.

In contrast, there were no significant differences in perceived usefulness identified when comparing clinical to non-clinical staff (Table 5). However, non-clinical staff generally reported higher perceived ease-of-use scores than clinical staff, with these differences being statistically significant in three of the six questions (Table 5).

Table 6 details the results for individual questions within the UTAUT. Participants rated the attitudes towards technology questions most highly, stating that using IS is a good idea. However, they scored lower on the social influence and facilitating conditions questions, which suggests there is a need to address the organisation’s culture to enable successful implementation. Furthermore, although survey respondents had low anxiety levels around using IS, questions in this domain had the lowest scores indicating that staff still had some apprehension and issues with trusting the system.

Table 6.

Survey responses to the UTAUT questions.

	Mean (SD)
Attitude towards technology
Using information systems is a good idea (n = 377)	6.27 (1.13)
Working with information systems is fun (n = 376)	4.94 (1.47)
I like working with information systems (n = 375)	5.41 (1.44)
Social influence
People who influence my behaviour think that I should use information systems (n = 375)	5.03 (1.51)
People who are important to me think that I should use information systems (n = 374)	5.04 (1.44)
The senior management of the organisation have been helpful in the use of information systems (n = 375)	4.60 (1.66)
In general, the organisation has supported the use of information systems (n = 375)	4.84 (1.65)
Facilitating conditions
I have the necessary resources to use information systems (n = 376)	4.59 (1.77)
I have the knowledge necessary to use information systems (n = 375)	5.10 (1.62)
Information systems are not compatible with other systems I use (n = 376)	4.13 (1.67)
A specific person (or group) is available for assistance with information system difficulties (n = 376)	4.60 (1.68)
Anxiety
I feel apprehensive about using information systems (n = 376)	2.95 (1.70)
It scares me to think that I could lose a lot of information using information systems by hitting the wrong key (n = 374)	3.14 (1.84)
I hesitate to use information systems for fear of making mistakes I cannot correct (n = 374)	2.68 (1.68)
Information systems are somewhat intimidating to me (n = 375)	2.71 (1.74)

UTAUT: unified theory of acceptance and use of technology; SD: standard deviation.

1 = strongly disagree; 2 = disagree; 3 = slightly disagree; 4 = neither disagree nor agree; 5 = slightly agree; 6 = agree; 7 = strongly agree.

Across all questions that evaluated attitudes towards technology, social influence and facilitating conditions in the UTAUT questionnaire, compared to staff aged >50 years, the average score for those participants aged ⩽50 years was higher, indicating more positive attitudes towards technology, an increased belief that people they respect want them to use technology and an increased belief that they had the conditions to facilitate the use of technology (Table 7). Furthermore, across all the questions that evaluated anxiety, the average score for participants aged ⩽50 years was lower, indicating less anxiety about using technology than staff aged >50 years (Table 7). These differences were statistically significant for two out of the three attitudes towards technology questions, two out of the four social influence questions, one out of the four facilitating conditions questions and all the anxiety questions (Table 7).

Table 7.

Age and clinical/non-clinical staff differences in UTAUT questions.

	Dichotomised groups	Mean (SD)	T	df	p-value
Attitude towards technology
Using information systems is a good idea	⩽50 (n = 224)	6.39 (0.96)	2.309	372	0.021
	>50 (n = 150)	6.13 (1.27)	2.309	372	0.021
	Clinical (n = 194)	6.29 (0.93)	0.214	374	0.830
	Non-clinical (n = 182)	6.26 (1.31)	0.214	374	0.830
Working with information systems is fun	⩽50 (n = 224)	5.05 (1.36)	1.692	371	0.092
	>50 (n = 149)	4.79 (1.63)	1.692	371	0.092
	Clinical (n = 194)	4.74 (1.44)	−2.838	373	0.005
	Non-clinical (n = 181)	5.17 (1.49)	−2.838	373	0.005
I like working with information systems	⩽50 (n = 224)	5.61 (1.33)	3.310	370	0.001
	>50 (n = 148)	5.11 (1.55)	3.310	370	0.001
	Clinical (n = 193)	5.25 (1.31)	−2.159	372	0.032
	Non-clinical (n = 181)	5.57 (1.56)	−2.159	372	0.032
Social influence
People who influence my behaviour think that I should use information systems	⩽50 (n = 224)	5.10 (1.47)	1.153	371	0.249
	>50 (n = 149)	4.92 (1.54)	1.153	371	0.249
	Clinical (n = 194)	5.03 (1.27)	−0.048	372	0.961
	Non-clinical (n = 180)	5.03 (1.73)	−0.048	372	0.961
People who are important to me think that I should use information systems	⩽50 (n = 224)	5.13 (1.43)	1.392	370	0.165
	>50 (n = 148)	4.92 (1.42)	1.392	370	0.165
	Clinical (n = 194)	5.00 (1.32)	−0.636	371	0.525
	Non-clinical (n = 179)	5.09 (1.56)	−0.636	371	0.525
The senior management of the organisation has been helpful in the use of information systems	⩽50 (n = 224)	4.77 (1.62)	2.353	371	0.019
	>50 (n = 149)	4.36 (1.69)	2.353	371	0.019
	Clinical (n = 194)	4.34 (1.60)	−3.225	372	0.001
	Non-clinical (n = 180)	4.89 (1.69)	−3.225	372	0.001
In general, the organisation has supported the use of information systems	⩽50 (n = 224)	5.00 (1.59)	2.343	371	0.020
	>50 (n = 149)	4.60 (1.73)	2.343	371	0.020
	Clinical (n = 194)	4.69 (1.58)	−1.949	372	0.052
	Non-clinical (n = 180)	5.02 (1.71)	−1.949	372	0.052
Facilitating conditions
I have the necessary resources to use information systems	⩽50 (n = 224)	4.71 (1.74)	1.720	371	0.086
	>50 (n = 149)	4.39 (1.80)	1.720	371	0.086
	Clinical (n = 194)	4.33 (1.73)	−3.105	373	0.002
	Non-clinical (n = 181)	4.89 (1.76)	−3.105	373	0.002
I have the knowledge necessary to use information systems	⩽50 (n = 224)	5.33 (1.51)	3.546	370	0.000
	>50 (n = 149)	4.74 (1.72)	3.546	370	0.000
	Clinical (n = 194)	4.88 (1.58)	−2.814	372	0.005
	Non-clinical (n = 180)	5.34 (1.64)	−2.814	372	0.005
Information systems are not compatible with other systems I use	⩽50 (n = 224)	4.19 (1.69)	0.874	371	0.383
	>50 (n = 149)	4.03 (1.63)	0.874	371	0.383
	Clinical (n = 194)	4.23 (1.60)	1.224	373	0.222
	Non-clinical (n = 181)	4.02 (1.73)	1.224	373	0.222
A specific person (or group) is available for assistance with information system difficulties	⩽50 (n = 224)	4.66 (1.59)	1.050	371	0.295
	>50 (n = 149)	4.47 (1.80)	1.050	371	0.295
	Clinical (n = 194)	4.40 (1.62)	−2.512	373	0.012
	Non-clinical (n = 181)	4.83 (1.71)	−2.512	373	0.012
Anxiety
I feel apprehensive about using information systems	⩽50 (n = 224)	2.71 (1.62)	−3.449	371	0.001
	>50 (n = 149)	3.32 (1.76)	−3.449	371	0.001
	Clinical (n = 194)	3.13 (1.60)	2.196	373	0.029
	Non-clinical (n = 181)	2.75 (1.78)	2.196	373	0.029
It scares me to think that I could lose a lot of information using information systems by hitting the wrong key	⩽50 (n = 223)	2.95 (1.79)	−2.492	369	0.013
	>50 (n = 148)	3.43 (1.91)	−2.492	369	0.013
	Clinical (n = 194)	3.44 (1.77)	3.447	371	0.001
	Non-clinical (n = 179)	2.79 (1.87)	3.447	371	0.001
I hesitate to use information systems for fear of making mistakes I cannot correct	⩽50 (n = 223)	2.49 (1.57)	−2.714	369	0.007
	>50 (n = 148)	2.97 (1.80)	−2.714	369	0.007
	Clinical (n = 194)	2.87 (1.61)	2.264	371	0.024
	Non-clinical (n = 179)	2.47 (1.73)	2.264	371	0.024
Information systems are somewhat intimidating to me	⩽50 (n = 223)	2.44 (1.60)	−3.686	370	0.000
	>50 (n = 149)	3.11 (1.88)	−3.686	370	0.000
	Clinical (n = 194)	2.93 (1.72)	2.603	372	0.010
	Non-clinical (n = 180)	2.47 (1.75)	2.603	372	0.010

UTAUT: unified theory of acceptance and use of technology; SD: standard deviation.

1 = strongly disagree; 2 = disagree; 3 = slightly disagree; 4 = neither disagree nor agree; 5 = slightly agree; 6 = agree; 7 = strongly agree.

Compared to clinical staff, non-clinical staff expressed more positive attitudes towards technology and had an increased belief that people they respect want them to use technology and that they had the conditions to facilitate the use of technology (Table 7). However, this was only statistically significant for two out of three of the attitudes towards technology questions, one out of four of the social influence questions and two out of four of the facilitating conditions questions (Table 7). Non-clinical staff also expressed less anxiety about using technology compared to clinical staff and this was statistically significant across all questions in this domain (Table 7).

Discussion

Globally, the implementation of IS in both clinical and non-clinical settings is increasing.^2,3,56 There have been numerous studies that have attempted to identify critical success factors for implementation of IS in the health setting.^57,58 Critical factors identified to date include the need to invest in training, assess the skill level of the user and provide targeted training, and provide different modes of training to meet user requirement. Building staff competence and confidence in ICT has also been shown to improve staff self-efficacy and subsequent engagement with the IS.⁵⁸ Thus, it is critical to understand staff digital literacy levels and attitudes towards IS so that targeted training can be provided as a component implementation of a new IS to facilitate a successful outcome.

Staff at this health service were shown to have high digital literacy levels citing high frequency of use and expressing confidence when asked about their interactions with various devices/systems. This finding is in contrast with previous studies which have reported that healthcare staff have low digital literacy levels.^16–21 One possible explanation for this difference could be the fact that this study was conducted at a health service that already utilises IS to support some clinical, administrative and corporate workflows.

Users >50 years of age were identified to be less frequent users of various hardware devices and software systems, with this user group also expressing lower confidence levels using these devices. This is concerning as digital literacy levels have been shown to affect staff engagement with IS, with poor computer skills and low computer experience identified to one of the main barriers for adopting EHRs.^9–11 This suggests that when planning the implementation of an IS, health services should provide additional and more frequent training to older users, in order to increase the uptake of new technology in this user group.

When asked about their attitudes towards IS, majority of staff reported positive attitudes. This aligns with previous studies that have found healthcare staff report that IS help them to do their job and increase their efficiency.^{16,22,24–30} However, some survey participants stated that IS would not be easy to use, with staff >50 years were more likely to report issues with ease of use of IS than those 50 years or younger. Clinical staff were also more likely to report issues with ease of use than non-clinical staff.

Perceived ease of use has been shown to be linked to self-efficacy, with self-efficacy found to impact behavioural intention to use EHRs.^59,60 This suggests that when implementing a new IS, to increase behavioural intention to use EHRs, health services should provide staff with frequent opportunities to become familiar with the system, before the system goes live, with a focus on staff aged >50 years and clinical staff. Indeed, studies have identified that investing in training and offering multiple types of training (i.e. one-on-one, classroom and online training) are critical factors for a successful EHR implementation in a health setting.⁵⁸ Other methods to improve ease of use include having clear written instructions on how to perform an EHR function and delivering training using scenarios that mimic the end-user’s workflow.^57,61

Although survey respondents scored highly on the UTAUT domains of ‘attitudes towards technology’ and ‘social influence’, they did not score highly in the ‘facilitating conditions’ domain. This is concerning as having organisational support to invest in training and having empowerment to learn have been identified as critical factors to ensure the successful implementation of an EHR in health settings.^57,58 Hence, when implementing a new EHR, organisations should specifically set aside time for staff to attend training and increase support in the period immediately after EHR implementation.⁵⁸ Optimisation, whereby the system is refined to meet the needs and the goals of the organisation post-implementation, should also be encouraged.⁵⁸ Scores in the anxiety domain were generally low (indicating a low level of anxiety). This could be related to the respondents’ previous experience with using digital technology, as evidenced through the survey response mode (90% of survey respondents used an electronic device to answer the survey questions). Further analysis of the anxiety domain revealed that clinical staff and staff >50 years of age were more likely to express anxiety about using IS, compared to non-clinical staff and staff ⩽50 years of age. One possible contributing factor to this anxiety could be lower digital literacy skills in these groups. A previous cross-sectional study conducted across six health services which identified that potential EHR users at some sites did not possess basic computer literacy skills which contributed to staff anxiety when using the new system.⁵⁸ It is also possible that the large number of IS in use and sub-optimal systems may be contributing to this increased anxiety.

Anxiety has been found to contribute to healthcare staff resistance to use of technology. Thus, when planning the implementation of an IS, health services should address the underlying anxieties of the intended users and consider differences in requirements based on role (clinical vs non-clinical) and age. In the aforementioned cross-sectional study, individualised computer skills assessments were undertaken pre-implementation of the EHR, to help better understand staff knowledge and confidence using ICT.⁵⁸ These individualised computer skills assessments then enabled the trainers to know who required computer training and provide computer training to these individuals, prior to providing them with specific EHR training, and this was demonstrated to reduce staff anxiety.⁵⁸

Study limitations

This study is not without limitations. The study had a response rate of just six per cent (n = 407), this is lower than a review of 31 studies showing a mean response rate of 36.8 per cent.⁶² Questionnaire length may have been a contributing factor to the response rate. Another contributing factor may have been survey fatigue, in the weeks preceding the distribution of this survey, an organisation-wide employee opinion survey had just concluded. However, it must be noted that the study’s response rate and still acceptable as a minimum sample size of 364 was calculated to achieve a five-per cent margin of error and a 95-per cent confidence interval, to achieve statistical significance with a five-per cent margin of error and a 95-per cent confidence interval based on the number of staff at the study site. Most survey respondents also used an electronic device to answer the survey questions which might indicate that the survey is capturing the opinions of individuals who are already digital literate. Hence, this may have led to a bias in results as survey respondents may have had an interest in IS and therefore they might be more frequent and confident users of ICT, and their responses might have been more positive. However, we did try to control for this variable through distribution of a paper-based survey for those staff who do not have regular computer access. Another limitation was that, although survey respondent demographics were generally representative of the staff population at this health service, in this study, there were more non-clinical than clinical participants, which is not representative of the staff population. Hence, although non-clinical staff expressed less anxiety about using IS compared to clinical staff in this study, this finding should be interpreted with some caution. The final study limitation is that the study was conducted at a single site, where there is a mix of IS and paper records. Hence, caution should be exercised when extrapolating these results to health services without any digital IS.

Conclusion and recommendations

This study has demonstrated that while staff are generally confident in using ICT and hold positive attitudes towards IS, there remains a portion of the workforce that is not confident in using ICT and have negative attitudes towards IS. Previous studies have demonstrated that poor computer skills and poor attitudes towards IS are barriers towards EHR adoption. Thus, when implementing IS/EHRs, health services should actively address the aforementioned factors that contribute to the successful implementation of an EHR and provide more intensive training to staff >50 years of age and staff transiting from paper to EHRs. This may increase staff digital literacy levels and improve their attitudes towards IS, thereby increasing staff engagement with IS and resulting in safer and more high-quality care being provided.

Footnotes

Appendix 1 Acknowledgements

The authors thank Mr Adamm Ferrier, Dr Deborah Glesson and Ms Anne-Marie Becket for their feedback and support.

Author’s note

Author Angeline Kuek is now affiliated with Health, Ageing and Human Services, KPMG, Australia.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Angeline Kuek

References

Buntin

Burke

Hoaglin

, et al. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff 2011; 30(3): 464–471.

Healthcare Information and Management Systems Society (HIMSS). Electronic health records: a global perspective, part I. 2nd ed. Chicago, IL: HIMSS, 2010.

Healthcare Information and Management Systems Society (HIMSS). Electronic health records: a global perspective, part II. 2nd ed. Chicago, IL: HIMSS, 2010.

Australian Digital Health Agency. Safe, seamless and secure: evolving health and care to meet the needs of modern Australia. Australia’s National Digital Health Strategy, Australian Government, 2017, https://conversation.digitalhealth.gov.au/sites/default/files/adha-strategy-doc-2ndaug_0_1.pdf

Australian Digital Health Agency. Health ministers approve Australia’s National Digital Health strategy. Australian Government, 2017, https://www.digitalhealth.gov.au/news-and-events/news/health-ministers-approve-australia-s-national-digital-health-strategy

State of Victoria, Department of Health and Human Services. Statewide health ICT strategic framework. Melbourne, VIC, Australia: Victorian Government, 2014.

State of Victoria, Department of Health and Human Services. Digitising health: how information and communications technology will enable person-centred health and wellbeing within Victoria. Melbourne, VIC, Australia: Victorian Government, 2016.

British Computer Society. BCS digital literacy for life programme, 2017, http://www.bcs.org/category/17853

Ajami

Bagheri-Tadi

Barriers for adopting electronic health records (EHRs) by physicians. Acta Inform Med 2013; 21(2): 129–134.

10.

Huryk

LA.

Factors influencing nurses’ attitudes towards healthcare information technology. J Nurs Manag 2010; 18(5): 606–612.

11.

Ochieng

Hosoi

Factors influencing diffusion of electronic medical records: a case study in three healthcare institutions in Japan. Health Inf Manag 2006; 34(4): 120–129.

12.

Health Care Business and Technology. Medical technology, 2016, www.healthcarebusinesstech.com/medical-technology/

13.

Jones

Furukawa

MF.

Adoption and use of electronic health records among federally qualified health centers grew substantially during 2010-12. Health Aff 2014; 33(7): 1254–1261.

14.

Davis

FD.

Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart 1989; 13(3): 319–340.

15.

Hillestad

Bigelow

Bower

, et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff 2005; 24(5): 1103–1117.

16.

Hegney

Buikstra

Eley

, et al. Nurses and information technology: final report. Commonwealth of Australia, 2007, https://anmf.org.au/documents/reports/IT_Project.pdf

17.

MacLure

Stewart

Self-reported digital literacy of the pharmacy workforce in North East Scotland. Pharmacy 2015; 3(4): 182–196.

18.

MacLure

Stewart

Digital literacy knowledge and needs of pharmacy staff: a systematic review. J Innov Health Inform 2016; 23(3): 840.

19.

MacLure

Stewart

A qualitative case study of e-health and digital literacy experiences of pharmacy staff. Res Social Adm Pharm 2018; 14: 555–563.

20.

Alwan

Awoke

Tilahun

Knowledge and utilization of computers among health professionals in a developing country: a cross-sectional study. JMIR Hum Factors 2015; 2(1): e4.

21.

Campbell

McDowell

DE.

Computer literacy of nurses in a community hospital: where are we today?

J Contin Educ Nurs 2011; 42(8): 365–370.

22.

Ayatollahi

Bath

Goodacre

Paper-based versus computer-based records in the emergency department: staff preferences, expectations, and concerns. Health Informatics J 2009; 15(3): 199–211.

23.

Dharmarajan

Gangadharan

Applying technology acceptance model (TAM) to determine the acceptance of nursing information system (NIS) for computer generated nursing care plan among nurses. Int J Comput Trends Technol 2013; 4(8): 2625–2629.

24.

Gurdas Topkaya

Kaya

. Nurses’ computer literacy and attitudes towards the use of computers in health care. Int J Nurs Pract 2015; 21(Suppl. 2): 141–149.

25.

Kim

Lee

Hwang

, et al. Analysis of the factors influencing healthcare professionals’ adoption of mobile electronic medical record (EMR) using the unified theory of acceptance and use of technology (UTAUT) in a tertiary hospital. BMC Med Inform Decis Mak 2016; 16: 12.

26.

Kirkendall

Goldenhar

Simon

, et al. Transitioning from a computerized provider order entry and paper documentation system to an electronic health record: expectations and experiences of hospital staff. Int J Med Inform 2013; 82(11): 1037–1045.

27.

Kossman

Scheidenhelm

SL.

Nurses’ perceptions of the impact of electronic health records on work and patient outcomes. Comput Inform Nurs 2008; 26(2): 69–77.

28.

Kuo

Liu

CC.

An investigation of the effect of nurses’ technology readiness on the acceptance of mobile electronic medical record systems. BMC Med Inform Decis Mak 2013; 13: 88.

29.

Sheikhtaheri

Kimiafar

Sarbaz

Evaluation of system quality of hospital information system: a case study on nurses’ experiences. Stud Health Technol Inform 2014; 205: 960–964.

30.

Wang

Chen

, et al. Attitudes toward inter-hospital electronic patient record exchange: discrepancies among physicians, medical record staff, and patients. BMC Health Serv Res 2015; 15: 264.

31.

Chow

Chin

Lee

, et al. Nurses’ perceptions and attitudes towards computerisation in a private hospital. J Clin Nurs 2012; 21(11–12): 1685–1696.

32.

Schenk

Mayer

Ward-Barney

, et al. RN perceptions of a newly adopted electronic health record. J Nurs Adm 2016; 46(3): 139–145.

33.

Zadvinskis

Chipps

Yen

PY.

Exploring nurses’ confirmed expectations regarding health IT: a phenomenological study. Int J Med Inform 2014; 83(2): 89–98.

34.

Holden

Asan

Wozniak

, et al. Nurses’ perceptions, acceptance, and use of a novel in-room pediatric ICU technology: testing an expanded technology acceptance model. BMC Med Inform Decis Mak 2016; 16(1): 145.

35.

Ward

Stevens

Brentnall

, et al. The attitudes of health care staff to information technology: a comprehensive review of the research literature. Health Info Libr J 2008; 25(2): 81–97.

36.

Balen

Jewesson

PJ.

Pharmacist computer skills and needs assessment survey. J Med Internet Res 2004; 6(1): e11.

37.

Kay

RH.

The relation between locus of control and computer literacy. J Res Comput Educ 1990; 22(4): 464–474.

38.

Gassert

McDowell

. Evaluating graduate and undergraduate nursing students’ computer skills to determine the need to continue teaching computer literacy. In: Medinfo’95: proceedings of the eighth world congress on medical informatics (eds Greenes

Paterson

Protti

), Vancouver, BC, Canada, 23–27 July 1995. Vancouver, BC, Canada: World Congress on Medical Informatics.

39.

Eastin

LaRose

Internet self-efficacy and the psychology of the digital divide. J Comput-Mediat Comm 2000; 6(1): JCMC611.

40.

Litt

Measuring users’ internet skills: a review of past assessments and a look toward the future. New Media Soc 2013; 15(4): 612–630.

41.

Nasah

DaCosta

Kinsell

, et al. The digital literacy debate: an investigation of digital propensity and information and communication technology. Educ Techn Res 2010; 58(5): 531–555.

42.

Godoe

Johansen

TS.

Understanding adoption of new technologies: technology readiness and technology acceptance as an integrated concept. J Eur Psychol Student 2015; 3(1): 38–52.

43.

Chen

Yang

K-F

Tang

F-I

, et al. Applying the technology acceptance model to explore public health nurses’ intentions towards web-based learning: a cross-sectional questionnaire survey. Int J Nurs Stud 2008; 45(6): 869–878.

44.

Venkatesh

Morris

Davis

, et al. User acceptance of information technology: toward a unified view. MIS Quart 2003; 27(3): 425–478.

45.

Venkatesh

Thong

JYL

Unified theory of acceptance and use of technology: a synthesis and the road ahead. J Assoc Inf Syst 2016; 17(5): 328–376.

46.

Liu

Miguel Cruz

Rios Rincon

, et al. What factors determine therapists’ acceptance of new technologies for rehabilitation – a study using the unified theory of acceptance and use of technology (UTAUT). Disabil Rehabil 2015; 37(5): 447–455.

47.

Schaper

Pervan

GP.

ICT and OTs: a model of information and communication technology acceptance and utilisation by occupational therapists. Int J Med Inform 2007; 76(Suppl. 1): S212–S221.

48.

Ifinedo

Technology acceptance by health professionals in Canada: an analysis with a modified UTAUT model. In: Proceedings of the 45th Hawaii international conference on system sciences, Maui, HI, 4–7 January 2012.

49.

Lin

Hsu

, et al. The usage behavior and intention stability of nurses: an empirical study of a nursing information system. J Nurs Res 2016; 24(1): 48–57.

50.

Sezgin

Yıldırım

SÖ.

A literature review on attitudes of health professionals towards health information systems: from e-health to m-health. Proc Tech 2014; 16: 1317–1326.

51.

Sharifian

Askarian

Nematolahi

, et al. Factors influencing nurses’ acceptance of hospital information systems in Iran: application of the unified theory of acceptance and use of technology. Health Inf Manag 2014; 43(3): 23–28.

52.

Aggelidis

Chatzoglou

PD.

Using a modified technology acceptance model in hospitals. Int J Med Inform 2009; 78(2): 115–126.

53.

Holden

Karsh

BT.

The technology acceptance model: its past and its future in health care. J Biomed Inform 2010; 43(1): 159–172.

54.

Whiteside

SPH

Sattler

Hathaway

, et al. Use of evidence-based assessment for childhood anxiety disorders in community practice. J Anxiety Disord 2016; 39: 65–70.

55.

Jackson

Coultas

Ashmore

, et al. Domain-specific self-efficacy is associated with measures of functional capacity and quality of life among patients with moderate to severe chronic obstructive pulmonary disease. Ann Am Thorac Soc 2014; 11(3): 310–315.

56.

Van der Aalst

WMP

Stahl

. Modeling business processes: a Petri net-oriented approach. Cambridge, MA: The MIT Press, 2011.

57.

Laramee

Bosek

Kasprisin

, et al. Learning from within to ensure a successful implementation of an electronic health record. Comput Inform Nurs 2011; 29(8): 468–477; quiz 478–479.

58.

McAlearney

Robbins

Kowalczyk

, et al. The role of cognitive and learning theories in supporting successful EHR system implementation training: a qualitative study. Med Care Res Rev 2012; 69(3): 294–315.

59.

Chau

PY.

Influence of computer attitude and self-efficacy on IT usage behavior. J Organ End User Com 2001; 13(1): 26–33.

60.

. The role of Internet self-efficacy in the acceptance of web-based electronic medical records. In: Mahmood

(ed.) Contemporary issues in end user computing. Hershey, PA: Idea Group Publishing, 2007.

61.

Pantaleoni

Stevens

Mailes

, et al. Successful physician training program for large scale EMR implementation. Appl Clin Inform 2015; 6(1): 80–95.

62.

Sheehan

KB.

E-mail survey response rates: a review. J Comput-Mediat Comm 2001; 6(2): JCMC621.