Evaluation of the awareness and effectiveness of IT security programs in a large publicly funded health care system

Abstract

Background:

Electronic health records are becoming increasingly common in the health care industry. Although information technology (IT) poses many benefits to improving health care and ease of access to information, there are also security and privacy risks. Educating health care providers is necessary to ensure proper use of health information systems and IT and reduce undesirable outcomes.

Objective:

This study evaluated employees’ awareness and perceptions of the effectiveness of two IT educational training modules within a large publicly funded health care system in Canada.

Method:

Semi-structured interviews and focus groups included a variety of professional roles within the organisation. Participants also completed a brief demographic data sheet. With the consent of participants, all interviews and focus groups were audio recorded. Thematic analysis and descriptive statistics were used to evaluate the effectiveness of the IT security training modules.

Results:

Five main themes emerged: (i) awareness of the IT training modules, (ii) the content of modules, (iii) staff perceptions about differences between IT security and privacy issues, (iv) common breaches of IT security and privacy, and (v) challenges and barriers to completing the training program. Overall, nonclinical staff were more likely to be aware of the training modules than were clinical staff. We found e-learning was a feasible way to educate a large number of employees. However, health care providers required a module on IT security and privacy that was relatable and applicable to their specific roles.

Conclusion:

Strategies to improve staff education and mitigate against IT security and privacy risks are discussed. Future research should focus on integrating health IT competencies into the educational programs for health care professionals.

Keywords

clinical informatics health information technology computer-assisted instruction evaluation patient data privacy

Introduction

Electronic health information systems offer benefits to both the health care of individuals and the improvement of the health care system (Cavoukian and Alvarez, 2012) but health information stored in an electronic system poses unique risks to privacy and security. Personal health information from multiple sources can be amassed and quickly accessed by a variety of users (Cavoukian and Alvarez, 2012) but it contains sensitive corporate information and intimate details of a patient’s life and the theft, loss, or unauthorised use and disclosure of personal health information can have dire consequences. Improperly accessed, such information may result in discrimination, stigmatisation, and psychological or economic harm to the individual (Cavoukian and Alvarez, 2012; Win, 2005). In addition, if patients do not have confidence that their information will be kept secure, they may refrain from disclosing critical information or avoid seeking treatment altogether (Appari and Johnson, 2010).

Previous studies have focused on errors related to health technology (Ash et al., 2004), threats to patient information (Rindfleisch, 1997), barriers to professional academic curriculum in the area of health technology (Borycki et al., 2011), and potential for e-learning (Haigh, 2004). However, evaluation of health care technology through the lens of in-house, organisation-wide online education and training programs and staff awareness of such programs has been limited. Health care providers need appropriate education and training in health information systems and information technology (IT) to ensure that information is used only to provide patient care or for work-related purposes relevant to specific occupations (Rindfleisch, 1997) otherwise information should be held in confidence outside of work parameters. Research has demonstrated that proper education can be effective in reinforcing appropriate behavior and use of technologies to help protect health care information (Rindfleisch, 1997) but that current educational approaches to training have not advanced at the same rate as technological advancements in health care settings (Borycki et al., 2011). Poor understanding of health information systems among health care professionals can also result in negative interactions between health care professionals, patients, and technology, which may ultimately result in medical errors (Borycki and Kushniruk, 2008; Koppel et al., 2005, Kushniruk et al., 2005; Patel et al., 2001). Effective approaches to education and training are needed to ensure that health care providers know how to handle electronic health information securely (Ash et al., 2004; Borycki et al., 2011).

Context for the current study

The site for this study was a large fully integrated health system with five geographical zones in Canada. This organisation had over 108,000 employees, including around 99,900 direct employees. Programs and services were offered at more than 650 facilities throughout the province, including hospitals, clinics, continuing care facilities, cancer centres, mental health facilities, and community health sites. As with other health care systems, there is a duty to protect the security, privacy, and confidentiality of information in its custody and control. For 6 years, this organisation had focused on educating staff about IT security and privacy by increasing the number of awareness campaigns and by providing staff with relevant educational training tools and supports. Education and awareness training modules had also been developed that targeted staff understanding of security and privacy and relevant legislative policies and acts. In 2015, it became mandatory for all staff and physicians to complete IT security and privacy training and to sign a confidentiality and user agreement at least once every 3 years. These requirements were linked with the formal staff performance evaluation process. Although, all staff were supposed to complete the online training annually, staff compliance with this policy remained an issue.

It was unclear whether the educational training and awareness material had resonated more positively with staff who occupied specific roles within the organisation than with other staff and whether the information provided in the training and awareness programs was sufficiently relevant and adequate to enable staff to do their jobs. If staff did not have the basic knowledge and understanding of security and privacy concepts, then patients and the organisation would be at risk for inappropriate disclosure of information (Rindfleisch, 1997). Therefore, an evaluation of two specific training modules (plus some additional material) was conducted to assist in clarifying if the material was easy to understand and applicable to various occupations. This study is the result of that evaluation. The study focused on phase 1 of the project to (i) evaluate the awareness of two IT security educational training modules and supporting material (e.g. policies) for staff and (ii) identify gaps or improvements, or both, to the IT security educational material.

Method

Research design and sample selection

The study targeted a total of 15 sites: 12 acute care sites (e.g. hospitals), 2 professional buildings, and 1 urgent care facility. Quota sampling was used to select sites and ensure representation from all regions. According to the proportion of the staff working in each geographic zone, the aim was to include the higher number of participants from urban zones. Three rural sites were included as well to identify any difference between rural and urban staff. Thus, the sampling strategy was designed to achieve diversity in terms of occupational groups. The quota matrix (Whitley, 2002) was developed to include an appropriate proportion of health care professionals in focus groups, based on their proportional representation in the organisation and geographical zone.

Participants

A total of 191 participants were included in the study: 180 from preselected occupational groups (clinical and nonclinical occupations) through 27 focus groups and 18 one-on-one interviews; and 11 from executive management and professional groups not included in the original sample (e.g. physicians), through telephone interviews (9) and face-to-face interviews (2).

Data collection

Information sheets

Each participant completed a short demographic information sheet that included questions about the participants’ level of awareness of IT security and privacy.

Interview schedule

The researchers developed a semi-structured interview guide for use in the interviews and the focus groups. The same guide was used across all interviews and with focus group participants, in order to provide consistency across all sites while also allowing for local variation.

Focus groups

The number of people attending focus groups varied, as did the composition of the groups at the different sites. Each focus group included between 5 and 10 participants and lasted for approximately 1 h. All focus groups were conducted on a face-to-face basis. Although focus group participants included both clinical and nonclinical staff (e.g. nurses, technicians, clerical staff), separate focus groups were conducted with clinical and nonclinical staff so any differences in their perspectives on the IT security educational material could be compared.

Ethical considerations

The IT Security Awareness Project was a quality improvement project, and all processes for data collection of participants met organisational ethical standards and approval. Informed consent was obtained from all participants after they had read the study information and completed the demographic information sheet. Their verbal consent for audio recording of focus groups and interviews was also obtained when they were informed about the study.

Modules

Two specific training modules were evaluated during data collection to capture awareness levels of participants and their perceptions of the educational training material on IT security and privacy (Alberta Health Services, 2011):

Annual Continuing Education (ACE) Secure – Collect IT, Protect IT: This course fulfilled requirements for Information Privacy and IT Security training for all employees. It was a short online course that provided an overview of the privacy legislation, the responsibilities of workers to protect the privacy of individuals, confidentiality of information, and the security of IT resources.

Information Privacy and IT Security Awareness: This was a 60-min training module that provided an overview of privacy legislation. It outlined staff responsibility to protect the privacy of individuals, confidentiality of information, and security of IT resources. Completion of the module was required within the first 3 months of employment or as designated by the employees’ program.

Data analysis

Demographic data were analyzed using SPSS version 19. Descriptive statistics used included means and frequencies.

Researchers analyzed qualitative data to identify the key emerging themes in data using thematic analysis. The research team separately identified themes and then met several times before converging on the final themes. With permission, the focus group discussions and interviews were recorded, so accuracy could be checked during the write-up and analysis of data.

Results

Demographic characteristics

Figure 1 shows actual proportion of each occupational group in the organization and the proportion of each occupational group in the survey. Around 40% participants were clinical staff, 54% nonclinical, and 6% executives of the organisation. Fifty-one percent of participants worked full-time and 60% did not have direct reporting staff. Nonclinical staff were those whose occupation included general support services, health information management, management, and laboratory/diagnostic imaging staff. All other occupational groups were clinical.

Figure 1.

Proportion of staff in the organization versus proportion of evaluation participants by occupation. RN: registered nurse; GSS/Aux: general support services/auxiliary nursing; HIM: health information manager; LPN: licensed practical nurse; HCA: health care aide; Lab/Diagnostics: laboratory or diagnostic imaging; MD: medical doctor; EMS: emergency medical services/ambulatory; P/T: professional/technical.

Themes

Overall, five themes were found from the qualitative data. First, participants talked about their level of awareness with regard to the IT security educational and training programs offered within the organisation and commented on their usefulness. Second, participants discussed differences between IT security and IT privacy, as well as where these two constructs overlapped. A third theme revolved around breaches of security and IT privacy that participants had experienced on the job. The fourth and fifth themes centered on the challenges participants faced in completing the IT security programs and the opportunities for improving program content and compliance.

Awareness

Table 1 shows the differences in both levels of awareness of training modules and completion rates between the nonclinical and clinical groups. Nonclinical staff were more aware of training modules than were clinical staff. Nonclinical staff also completed the ACE module in a higher proportion.

Table 1.

Comparing the awareness level of IT security and privacy modules between clinical and nonclinical participants.^a

	Clinical numbers (%)	Nonclinical numbers (%)
Participant groups	76 (40)	104 (54)
Aware of ACE Secure – Collect It Protect It module^b
Yes No	51 (77.3) 15 (22.7)	87 (85.3) 15 (14.7)
Have completed ACE Secure – Collect It Protect It module^b
Yes No Do not remember	32 (54.2) 17 (28.8) 10 (16.9)	70 (72.9) 14 (14.6) 12 (12.5)
Aware of Information Privacy and IT Security Awareness module^b
Yes No	28 (42.4) 38 (57.6)	64 (62.7) 38 (37.3)

^aExecutives were excluded from this analysis.

^bTotal is less due to missing data.

Most participants (82.1%) were aware of the ACE Secure – Collect It Protect It module and the majority (65.8%) had also completed it. Just over half of participants were aware of the Information Privacy and IT Security Awareness module (54.8%). The lowest level of awareness among occupational groups for both modules (ACE and Information Privacy and IT Security Awareness modules) was among physicians.

Participants said they had been informed about these training modules through management staff, administrative assistants, educators, or during orientation. Compliance for completing the ACE Secure – Collect It Protect It module was believed to be high and most staff were aware that the module was mandatory and that a signed confidentiality agreement was required upon completion. As one nonclinical staff member said, “It is a learning requirement for all staff and goes with performance appraisals. They have to do them before the performance appraisal.”

Participants also mentioned accessing other resources such as the internal website and the IT help desk for questions. However, the level of awareness about additional resources varied among participants. Most staff were unaware of IT security awareness month, where cyber security tips and information were shared throughout the month. An example was provided by a clinical staff member, “Don’t know about policies, phishing campaign, or security awareness month.”

Module content

A majority of participants believed that the content of ACE Secure – Collect It Protect It module was appropriate as it provided basic information on IT security and privacy breaches. The module was a good reminder every year and made participants more aware of which procedures to follow. As one clinical staff member said, “For frontline staff, for sure module [ACE] resonate with staff; it explains policies/legislature and scenarios are relevant.”

In contrast, some participants reported that the content was not sufficient and that staff needed finer details, more realistic scenarios that were applicable to their jobs, and more updates on changes in the policies; that going through the e-learning module once a year was not sufficient given that only basic information was provided in the ACE Secure – Collect It Protect It module. One nonclinical staff member said, “There is so much focus on working with the patients. It does not apply to me, so you need separate learning for the nonclinical staff.”

Module content was also less relevant for students and volunteers, as a clinical staff member said, “The content is inapplicable to students/volunteers or people coming to job shadow; we need something more relevant.”

Differences between IT security and privacy

Several participants were able to clearly differentiate between IT security and privacy, as defined within the organisation. IT security was understood as security around their computer systems, including having credentials and appropriate passwords for login, locking computer stations, and computer firewalls. Privacy (which encompasses privacy and confidentiality) was understood as protecting personal or health information and appropriately accessing information based on role. Many participants noted that these concepts were related such that privacy is more likely to be compromised when the system is not secure. However, some participants were unable to distinguish between IT security and privacy issues. As one clinical staff member said, “I am not 100% sure what’s the difference. We talk about them as the same so do not know if there is any difference.”

Many participants were unfamiliar about where to report incidents. If faced with a security breach, they would report the incident to one or more of the following: (a) their manager, (b) their colleague, (c) the privacy commissioner, (d) security department, (e) IT help desk, or (f) search the internal website for information. Participants also found it confusing to have different incident reporting procedures for each type of issue and suggested a common pathway of reporting IT security and privacy incidents. Of help would be to have someone who redirected telephone calls to the appropriate reporting body. As an executive said, “I would not know where to report a privacy or security incident; I hope that privacy and security departments share the information and can re-direct.”

Common breaches

Sharing usernames and passwords, not encrypting emails, not opening spam emails, and leaving computers without logging off were the most common security breaches reported across zones. Password sharing occurred mostly because of delays in getting IT accounts operational for new staff. Participants noted that some units had generic accounts for all staff to access. Participants understood that this practice was inappropriate as it might not be possible to identify the responsible person if a breach occurred. As one nonclinical staff member pointed out:

The hire process for managers needs to be improved. You shouldn’t allow people to start the job without having login details. It is a conflicting message to new hires if we are breaching security on the first day by giving them someone’s password.

Spam emails and telephone calls were commonly identified as security breaches that participants were aware of or had experienced. Additionally, participants reported receiving confidential information in the form of a misdirected fax or email. Sending emails to individuals outside of the organisation without using encryption methods was another common security breach. Concerns were mentioned about potential breaches occurring when physicians requested patient information through personal cellphones and when physicians used personal email addresses instead of organisational email addresses. One clinician talked about the use of personal devices, “Texting about patients, physicians do this all the time, it’s huge and getting more common, doctors say to staff that the only way they will communicate is through their personal phones.”

Screen visibility was another common privacy issue for clinical staff, as they noted it is often easy for patients or other people walking by their workstations to see patient information on monitors. Staff also logged into electronic health records and then walked away without logging out.

Challenges

Most departments had complied with the guideline to encourage staff to complete the ACE Secure – Collect It Protect It module; however, some noted challenges for timely completion of ACE modules. Although the ACE module on IT security was short, staff were also required to complete several other modules as part of their annual continuing education, and completion of all modules was time-consuming (e.g. nurses had 17 h of mandatory education to complete every year). Some departments provided designated hours for staff to complete the learning modules while others did not provide extra time or compensation. Many participants had to complete modules during their shifts, which interfered with learning as there were several distractions. There was support among participants for designated hours for module completion. As one nonclinical staff member suggested, “The only thing that would improve compliance [for completing the modules] would be given designated hours because we don’t get time to do it. If you are given time then you can absorb it better.”

Participants from rural sites or those who worked a casual shift discussed unique challenges. Compliance with completing the module was particularly challenging for casual staff, as they often worked when units were short-staffed. Access to computers and ability to access the system for some providers could also become barriers to module completion. For example, health care aides had less secure passwords to remember because they accessed the system infrequently. As one clinician said, “Our Health Care Aides do not have to access computer for any patient need. You order lab work on paper, and many other things are still on paper. There is no equity; urban has advantage in terms of IT.”

Discussion

Summary of findings

The focus groups revealed that a large majority of staff were aware of IT security education programs and had completed the mandatory modules. Additionally, participating staff were aware of the fundamental differences between IT security and IT privacy but acknowledged their overlap. Staff identified several breaches occurring within the hospital (e.g. spam emails, stolen computers, unencrypted emails) and identified challenges and ways to improve module content and compliance.

Awareness

Participants were mostly aware of the annual IT module they were required to take, and the majority had completed it. Tying completion of the modules to staff performance appraisals might have been one of the primary driving forces behind staff compliance. Moreover, many clinical participants noted how, not only was compliance with the module tied to their performance appraisals, but they would also be unable to get their yearly certification without completing the module. On the other hand, although many of the nonclinical staff spoke about module compliance being tied to their performance appraisal, they often noted a lack of follow through from their managers or supervisors. A few even mentioned how the employees themselves were tasked with locating the modules required of them. This could have indicated to staff that the organisation security staff or senior management, or both, were not overly concerned with or committed to IT security awareness and training. In a discussion on barriers to IT security awareness, Tsohou et al., (2008) noted how the organisational culture can indeed affect staff attitudes and perceptions on the importance of IT security awareness.

Although staff indicated generally good awareness about IT security, their awareness levels might have been affected by the communications sent out during the time this evaluation was being conducted. Specifically, during this evaluation, the IT security department circulated a few emails on IT security and privacy awareness which might have increased participants’ level of awareness. However, this might have potentially positive implications for future organisational strategies. Saunders (2012) discussed how organisations would be more successful at educating their employees on IT security and increasing their awareness levels if they used an automated system to deliver messages on an ongoing basis. This particular point was reinforced by focus group participants in this study, who said how beneficial and helpful it would be if the organisation sent out monthly or quarterly reports and updates on the number of IT security breaches that had occurred, as well as information about the aftermath of these incidents.

Module content

A major concern with the IT modules that staff were required to complete was how relevant the content was to their own jobs. This was particularly concerning among nonclinical staff who felt that module content was not applicable to them. Most of the examples illustrated how IT security was relevant to patient care and how security breaches could compromise patient privacy. These examples were not considered effective for nonclinical staff who did not interact with patients or those removed from patient care. Content needs to be tailored according to different professions as nonclinical staff have different needs from clinical staff. One possibility would be having separate modules for different professional groups, which outline how security concerns and impacts upon the work they do. Different levels of education could also be made available with the respective stages of career progression (Mantas et al., 2011). Another possibility would be moving away from online learning and having staff attend in-person sessions where an educator would provide the necessary information and answer specific questions. Educators should be super-users or experts in IT security and provide examples and exercises related to staff and health care providers’ work situations (Thomson and von Solms, 1998). However, moving away from online learning would involve substantial resources and costs for large organisations.

Many participants also urged for updated module content, as they felt that the content shown in the training was out of date and did not reflect the rapid changes in technology. Although staff found the annual IT module to be a good refresher, compared to staff who had been on the job for only a year or two, more tenured staff (i.e. 5 or more years) wanted updates on new information and policies rather than always having to relearn the basics. As for new graduates, students, and volunteers, the information found in the modules was insufficient for their needs. Orientation and training new staff about privacy, security, computer basics, and policies may have been an oversight in the past, as a new employee orientation checklist was developed after phase I of this study. This is especially true considering that many new graduates are entering the workplace with generally insufficient knowledge of health informatics (Canadian Association of Schools of Nursing (CASN), 2012). One solution is to include courses in the undergraduate curriculum on the role of health care professionals as IT users (Mantas et al., 2011). Moreover, professional colleges may need to consider establishing professional standards of practice when managing information and electronic records (Robillard and Tolfree, 2015). Although the CASN circulated a report highlighting a list of nursing informatics competencies that registered nurses should possess upon graduation (CASN, 2012), to date, these competencies have not yet been incorporated into nursing curricula or have made modest progress (Canadian Nursing Informatics Association, 2016).

IT security and privacy

Participants overall had some understanding of IT security and privacy and understood that information access was based on their role. Understanding the difference between privacy and security is important, especially when it concerns incident reporting. Indeed, many participants were unsure of where to report an incident. Part of the confusion also stemmed from having two different incident reporting processes for IT security and IT privacy. Not properly reporting a privacy or security breach has the potential to exacerbate the incident. Without proper education programs and material, the risk of breach is increased. As such, health care organisations need to ensure that their employees understand the importance of safeguarding and securing information and that they are provided with guidelines on how to do it. All clinical and nonclinical staff should also be made aware of when, where, and how to report incidents.

Common breaches

Common breaches found were in relation to emails, screen visibility, and unlocked workstations. Improper access to information can negatively impact patients’ perceptions of and confidence in the organisation. This might then discourage patients from disclosing critical information or seeking treatment (Appari and Johnson, 2010). Many of the breaches reported by participants in this study were within staff control to prevent. This was reassuring as previous research has documented possible interventions to promote the proper safeguarding and securing of information. Possible interventions to address commonly discussed breaches may involve sending alerts and reminders that prompt staff to use IT security best practices (Rindfleisch, 1997). Another option could be setting up audit trails, as this may impact on staff motivation to abide by IT security best practices (Siponen, 2000).

Nonetheless, some breaches reported by participants in this study occurred due to issues outside of the control of the staff. For example, many participants reported that they shared usernames and passwords because new employees had not yet been given access to systems and programs that they needed to do their jobs. As per organisation policy, staff were required to protect their user ID and password and never to allow anyone who did not have proper credentials to access their computers. Thus, the organisation should ensure that new employees are given access immediately when they start their job to avoid such breaches. Interestingly, many health care staff also expressed concern with respect to physicians using their own personal devices for communicating patient information. While the use of Smartphones and health-related apps among health care professionals has become more common and has the potential to benefit health care, there has been a lack of education and focus in regard to the IT security of patient information (Boulos et al., 2011; Luxton et al., 2011; Martinez-Perez and Torre-Diez, 2015). One way to referee this would be to implement a secure messaging platform such as the one recently implemented by the Alberta Medical Association (2016).

Challenges

Participants spoke of challenges with completing the mandatory security and privacy training. Most of the barriers included time, pay, staff shortages, and equipment. However, some spoke of more personal barriers, such as finding child care to complete training at set times or the inability to access the learning modules from home. These challenges are not uncommon; previous literature had identified several challenges and concerns e-learners experience. These include, but are not limited to, the time-consuming nature of e-learning, managing time appropriately, staff shortages, and inappropriate facilities (Childs et al., 2005). Lack of compliance with completing IT security training modules is concerning because inadequate proper education and knowledge to safeguard and secure personal information compromises a health care organisation’s ability to deliver on their privacy duty and commitment.

Results of this study showed that casual staff and staff working in rural areas were more challenged with limited resources, such as locating computers and staff workload and coverage. Paid leave and improved staff coverage are two possible strategies identified by the study to improve completion of education and training, especially in rural areas (Curran et al., 2006). Technical problems at work or at home provided another set of challenges identified with training module compliance. For instance, computer, printer, and application compatibility or Internet access and access speed can all interfere with module completion (Childs et al., 2005). Making these challenges less burdensome might involve ensuring staff have dedicated time to complete the modules either by introducing in-person orientation days or providing staff with paid time to complete the training. Organisations should also ensure that staff have access to computers. As many study participants noted, searching and registering for the required courses was burdensome. Thus, organisation could also consider an automated registration process.

Limitations

Although a quota matrix was used to select proportionate numbers of staff from various professions and zones, it was not possible to use random sampling to recruit participants into the study. Lack of time and resources prevented a random sampling of the population, which limits the generalisability of the study’s findings. However, this study offers perspectives from clinical, nonclinical, and executive employees about the relevance of the material presented through organisation-wide e-learning to their respective jobs. Participants also suggested strategies to overcome challenges or gaps with the privacy and security training.

Conclusion

Generally, the two IT educational modules offered useful information to those who completed the training. The compliance with completing the training modules improved after making module completion mandatory and tied to performance appraisals. However, further strategic development could focus on the role of management staff in improving staff compliance since lack of staff knowledge about IT security and privacy poses organisational risks.

This study also found gaps with the IT awareness and educational material available to staff. Of importance is that health care providers need modules with privacy and security content that are relevant to providers’ specific roles. This is not surprising given the more recent focus on the development of IT competencies with health care providers. Further research is needed to integrate health IT competencies into educational programs in universities, colleges, and health care organisations and to train more health care providers in the role of IT specialists to deliver the education. Basic knowledge of IT privacy and security concepts could continue to be delivered through e-learning, as this could be a feasible option for large organisations. Modules could be updated to include gaps found in this study, such as where to report IT security and privacy incidents, sharing login details, and unencrypted transfer of patient information. However, since many experienced clinicians have a solid understanding of IT privacy and security concepts, further education should focus on new technology introduced into practice, updates on policy changes and how that impacts practice, and case studies of incidents that occurred in the organisation with the resulting consequences. This education may need to be delivered face-to-face for practical reasons and for clinicians to have the opportunity to ask further in-depth questions.

Footnotes

Acknowledgements

The authors would like to thank Colleen Thomas (Director, Governance, Risk and Compliance, Information Risk Management, Alberta Health Service) for supporting this work.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Alberta Health Services (2011) Information and privacy: education and training. Available at: http://www.albertahealthservices.ca/info/page3962.aspx (accessed 19 April 2017).

Alberta Medical Association (2016) About DR2D secure messaging. Available at: https://www.albertadoctors.org/leaders-partners/ama-dr2dr-secure-messaging/about-dr2dr (accessed 19 January 2017).

Appari

Johnson

(2010) Information security and privacy in healthcare: current state of research. International Journal of Internet and Enterprise Management 6(4): 279–314.

Ash

Berg

Coiera

(2004) Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. Journal of American Medical Information Association 11(2): 104–112.

Borycki

Kushniruk

(2008) Where do technology-induced errors come from? Towards a model for conceptualizing and diagnosing errors caused by technology. In: Kushniruk

Borycki

(eds) Human Social and Organizational Aspects of Health Information Systems. Hershey: Idea Group, pp. 148–166.

Borycki

Joe

Armstrong

. (2011) Educating health professionals about the electronic health record (EHR): removing the barriers to adoption. Knowledge Management and E-Learning: an International Journal 3(1): 51–62.

Boulos

Wheeler

Tavares

. (2011) How smartphones are changing the face of mobile and participatory healthcare: an overview with example from eCAALYX. BioMedical Engineering Online 10(1): 24.

Canadian Association of Schools of Nursing (2012) Nursing informatics entry-to-practice competencies for registered nurses. Available at: http://www.casn.ca/wp-content/uploads/2014/12/Nursing-Informatics-Entry-to-Practice-Competencies-for-RNs_updated-June-4-2015.pdf (accessed 19 January 2017).

Canadian Nursing Informatics Association (2016) IMIA-NI Representative Annual Report Canada – 2016. Available at: https://cnia.ca/imia-ni-representative-annual-report-canada-2016/ (accessed 19 January 2017).

10.

Cavoukian

Alvarez

(2012) Embedding privacy into the design of EHRs to enable multiple functionalities – Win/Win. Available at: https://www.infoway-inforoute.ca/en/component/edocman/resources/reports/338-embedding-privacy-into-the-design-of-ehrs (accessed 19 January 2017).

11.

Childs

Blenkinsopp

Hall

. (2005) Effective e-learning for health professionals and students – barriers and their solutions. A systematic review of the literature – findings from the HeXL project. Health Information and Libraries Journal 22(suppl 2): 20–32.

12.

Curran

Fleet

Kirby

(2006) Factors influencing rural health care professionals’ access to continuing professional education. Australian Journal of Rural Health 14(2): 51–55.

13.

Haigh

(2004) Information technology in health professional education: why it matters. Nurse Education Today 24(7): 547–552.

14.

Koppel

Metlay

Cohen

. (2005) Role of computerized physician order entry systems in facilitating medication errors. JAMA 293(10): 1197–203.

15.

Kushniruk

Borycki

Kuwata

. (2005) Predicting changes in workflow resulting from healthcare information systems: ensuring the safety of healthcare. Healthcare Quarterly 9(spec): 114–118.

16.

Luxton

McCann

Bush

. (2011) mHealth for mental health: integrating smartphone technology in behavioral healthcare. Professional Psychology: Research and Practice 42(6): 505–512.

17.

Mantas

Ammenwerth

Demiris

. (2011) Recommendations of the international medical informatics association (IMIA) on education in biomedical and health informatics: first revision. Methods of Information in Medicine 49(2): 105–120.

18.

Martinez-Perez

Torre-Diez

(2015) Privacy and security in mobile health apps: a review and recommendations. Journal of Medical Systems 39(1): 181–189.

19.

Patel

Kushniruk

Yang

. (2001) Impact of a computer-based patient record system on data collection, knowledge organizations and reasoning. Journal of the American Medical Informatics Association 7(6): 569–585.

20.

Rindfleisch

(1997) Confidentiality, information technology, and health care. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.18.2296&rep=rep1&type=pdf (accessed 19 January 2017).

21.

Robillard

Tolfree

(2015) An overview of Alberta’s electronic health record information system. Alberta Health Services, Canada. Available at: http://www.albertanetcare.ca/documents/An_Overview_of_Albertas_ERHIS.pdf (accessed 19 January 2017).

22.

Saunders

(2012) Security awareness – once is never enough. EDPACS 46(4): 15–17.

23.

Siponen

(2000) A conceptual foundation for organizational information security awareness. Information Management and Computer Security 8(1): 31–41.

24.

Thomson

von Solms

(1998) Information security awareness: educating your users effectively. Information Management and Computer Security, 6(4): 167–173.

25.

Tsohou

Kokolakis

Karyda

. (2008) Investigating information security awareness: research and practice gaps. Information Security Journal: A global Perspective 17(5–6): 207–227.

26.

Whitley

(2002) Principles of Research in Behavioral Science. Toronto: McGraw Hill.

27.

Win

(2005) A review of security of electronic health records. Health Information Management 34(1): 1–13.