Nurturing the digital baby: Open innovation for development and optimization

Abstract

The primary aim around developing and optimizing an electronic health record is to improve patient care and population health. The objective of this study is to design and evaluate an action research approach for the optimization of the design of a summary page artefact within an electronic health record for newborn healthcare. An action research approach was chosen for its participatory democratic process for developing practical knowledge and solutions. Collaborative workshops lead by an independent graphic facilitator with a ‘bottom up’ approach, involving self-selected motivated members from multidisciplinary healthcare teams, were designed and conducted. To evaluate this approach, insights were drawn from behavioural and design science paradigms to demonstrate that knowledge and understanding of the design problem and its solution were acquired in building the optimized summary page artefact. Information system development for healthcare requires consideration not just of what we do but how and why we do things. Our analysis demonstrates that action design research represents an agile and lean approach for successful optimization and implementation of information system development in healthcare.

Keywords

action research electronic health records information service development innovation optimization

Introduction

Inspired by the success of participatory action design research in other industries, the objective of this study was to design and evaluate an action research approach for the optimization of a neonate summary page within a national electronic health record (EHR) for mothers and newborns in Ireland, the Maternity and Newborn Clinical Management System (MN CMS). Harnessing the power of information and communication technology (ICT) to improve healthcare is critical but extremely challenging. Much of the literature around the implementation and optimization of ICT within healthcare has focused on ‘wiring’ the healthcare system, addressing functionality issues and technical specifications rather than considering leadership, cultural change, communication, and strategies to secure adoption by clinicians.¹ Physicians, traditionally the team leaders in hierarchical multidisciplinary healthcare teams, have been trained to accept evidence-based quantitative methods such as randomized controlled trials as the gold standard when it comes to testing the effect of an intervention or treatment. They can, at times, view qualitative research with distrust, finding some difficulty in accepting research methodologies where the generation of hypotheses often replaces the testing of hypotheses, explanation replaces measurement, and understanding replaces generalizability.² Physicians can be less familiar with qualitative explorations of beliefs and understandings that explore why the results of research are often not implemented in clinical practice.^3,4 It is well recognized that it can take a long time and be difficult to translate quantitative biomedical evidence–based research from randomized controlled clinical trials research into improved clinical practice.⁵ Recognized barriers to an ICT intervention, such as EHR implementation, include its lack of relative advantage, it not being perceived as being better than paper, its high complexity and its low compatibility with clinician needs, and their past experiences. These barriers are often compounded by a lack of organizational slack particularly with respect to staffing to meet the needs of ‘business as usual’ in a publicly funded healthcare system and the associated lack of additional resources to devote to adapting the EHR to the healthcare system and the healthcare system to the EHR.⁶ Some physicians may even see implementation of an EHR as a ‘perceived threat’ to their control over their work and their autonomy as a clinician. In general, people do not ‘resist new technology, but rather they resist the effect that the new technology may have on their own lives’.⁷

The MN CMS is a national project implementing a complete EHR for mothers and newborns into 19 hospitals (4 tertiary, 4 regional, and 11 peripheral centres) across six maternity networks in the Irish Healthcare System (see Figure 1). For the initial design and build of the MN CMS in a digitally naive Irish healthcare environment, a linear project management ‘top down approach’ was used, where workstream members were selected by the organization and sent away from their workplace for several days to industry-led design gateway sessions to deliver a safe and acceptable system on time and on budget (see Figure 2). But delivery of neonatal intensive care is complex, providing care to ill newborns in a critical care environment involving a multidisciplinary team working together across many dimensions of care in an often quickly changing and nonlinear fashion – with at times competing and conflicting demands upon different members of the healthcare delivery team.

Figure 1.

A full EHR for mothers and babies.

Figure 2.

Vendor-led project management approach to initial design process for build of MN-CMS EHR.

Initial interviews carried out by this researcher (B.P.M.) with representatives across all professions within the multidisciplinary neonatal healthcare team confirmed the significant challenges in ways of working that the introduction of MN CMS introduced into modern neonatal intensive care in Ireland. These interviews which helped inform adoption of an action research approach for this optimization project confirmed that the degree of acceptance of MN CMS relates to its complexity, its relative advantage, its perceived ease of use and its perceived usefulness to meet the shared goals and beliefs of the team.^8,9 Prior to implementation of the EHR, information was stored as fragmented paper records held by different disciplines within their own self-controlled silos of information. Following implementation of the EHR, feedback from interviews emphasized the need for greater multidisciplinary involvement in a non-hierarchical manner in any optimization process of this new shared digital EHR. All interviewees wished to ensure that optimization of the shared digital EHR solution met the needs of all team members contributing to the complex integrated care provided to a newborn infant in today’s critical care neonatal intensive care unit (NICU) environment.^10,11

Methods

Action research

Action research is based on theory and concepts that combine behavioural and design science paradigms.¹² It utilizes collaborative workshops ‘where knowledge and understanding of a problem domain and its solution are achieved in the building and application of the designed artefact’.¹³ This approach captures tacit knowledge from end users across the multidisciplinary healthcare team in a participatory democratic manner and validates the emerging system prototype within the organizational context of the real world of healthcare. By harvesting shared learning between end users, subject experts and industry consultants within this creative process knowledge is built.¹⁴ Action research comprises the following four steps:

Problem formulation

Building, intervention, and evaluation

Reflection and learning

Formalization of learning.

By its participatory and democratic nature, action research is ideally suited to identifying problems and potential solutions for improving clinical practice across the multidisciplinary team, bridging the ‘theory-practice gap’, contributing to both social science and social change.¹⁵ We did not have the resources for a completely new design and build of the entire MN CMS EHR so to address the key aims of improving patient care and communication across the healthcare team the neonate summary page was chosen for redesign as an improved workflow-based summary page. The basis for the study was that the participatory and democratic nature of action research made it suited to identifying problems and potential solutions for the summary page to improve clinical practice and communication across the multidisciplinary neonatal healthcare team.

Workshop design

For this study, we designed and conducted collaborative workshops lead by an independent graphic facilitator and attended by the multidisciplinary neonatal healthcare team end users, subject experts, and industry design consultants. One workshop was held in each of the four hospitals (see Figure 3). The focus of each workshop was to capture feedback and ideas to improve the neonate summary page within MN CMS for its use across the multidisciplinary neonatal healthcare team. From the outset, we aimed to adopt an action research approach tailored towards maximizing engagement, participation, democratic process, and production of social knowledge and social change. We acknowledged absent organizational slack and a workforce within maternity services that at times felt under siege to maintain current levels of service let alone enter change cycles, optimization, and transformation. This action research approach required collaboration across multiple stakeholders–multidisciplinary neonatal healthcare team end users, the research team and industry consultants to advise on technical aspects of the MN CMS, all within the constraints of the real world of healthcare delivery in a publicly funded healthcare system.

Figure 3.

Action research for MN-CMS optimization.

Neonatology services in Ireland have a long-standing history of collaboration, benchmarking, and quality improvement initiatives.¹⁶ It is against this collaborative background that we designed this action research approach for optimization of the neonate summary page within the recently implemented EHR. We utilized four short iterative cycles, one at each of four hospital locations once each week with the four cycles taking place over the course of a month. Each cycle was a workshop held onsite at the hospital location. These workshops were scheduled to facilitate and maximize participation by as many key stakeholders as possible across the multidisciplinary healthcare team in each location. By having the workshops onsite, these stakeholders both knew each other and the local organizational context in which the optimized solution was required to operate.

The approach to each successive workshop was informed by reflections and learnings from the previous workshops.¹⁷ Rather than being industry led or directed by the researcher, the workshops were managed by an Independent Graphic Facilitator. The role of the Independent Facilitator was to foster open honest feedback and enthusiasm from all the participants and graphically harvest the learning outputs. The use of an independent graphic facilitator was chosen as an innovative means to capture the maximum feedback and ideas from the healthcare team in the limited time available at each workshop (see Figure 4).

Figure 4.

Visual illustration of workshop goal for participants.

As a healthcare organization moves from initial implementation and stabilization of an information system towards its optimization, relationships and behaviours shift from the information system driving operations more towards the end users operations driving the information system design.¹⁸ Thus, in contrast to the ‘top down approach’ used for the initial design/build of the system where workstream members were selected by the organization and sent away from their workplace for several days to industry-led design gateway sessions, for these action research optimization workshops, a ‘bottom up’ democratic approach was adopted.¹⁹ An open invitation was issued across the multidisciplinary healthcare team in each hospital using both formal and informal communication channels. End users were invited to attend the workshop in their own hospital location for as much or as little time as they could manage, to contribute towards optimizing the neonate summary page for the shared goal of improved newborn healthcare delivery. The workshops were scheduled in collaboration with senior healthcare team members in the afternoon, on site at each hospital in training rooms that could be accessed by staff who might need to come and go as required for ongoing clinical service commitments. No protected time or cross-cover was sought from senior hospital management to facilitate attendance. Inclusion criteria for participation included all trained registered users of MN-CMS across the multidisciplinary neonatal healthcare team in each of the four hospital sites. The only exclusion criteria were any staff who were not trained registered users of MN-CMS.

Following introductions and an overview of the purpose of each workshop, the participants were divided into smaller groups, ensuring disciplines and grades were represented across all the groups to promote a multidisciplinary focus on finding solutions that might apply to all. Key aspects that arose were visually recorded with both a ‘quick fix’ sheet and a ‘parking lot’ sheet to capture joint output. All notes, sticky notes and flipcharts were gathered/collected as well as digital images of ‘parking lot’ and flow diagrams by the independent graphic facilitator and this researcher (B.P.M.). Industry consultants attended the workshops in an observer capacity to assist in capturing output and to answer any technical questions that arose from participants in relation to the evolving artefact.

Each workshop used visually facilitated familiar real-world scenarios, for example, daily ward rounds to engage all participants and to assist them in problem formulation, taking the current neonate summary page as the initial prototype for optimization and upgrade. All stakeholders were encouraged to contribute input towards requirements and needs in terms of the developing artefact features. Reciprocal understanding based on each other’s knowledge and roles helped identify early prototype features that addressed these identified needs and requirements. This approach provided a safe reciprocal space for reflection, learning, and action for the participants. Participants were given an opportunity to make clear the perceived strengths, weaknesses, and opportunities around the use of the current neonate summary page. Through visual facilitation, workshops were designed to engage participants from the very outset, igniting their creativity using metaphors and images. Participants were asked to become active by visualizing and clustering ideas and workflows themselves.

As the first problem formulation exercise, a bus metaphor was utilized as a short and humorous icebreaker to gain trust among the workshop participants. This bus metaphor exercise allowed participants in interdisciplinary groups of 3–4 to reveal their experience with the initial neonate summary page within the EHR implemented in their workplace metaphorically. By asking participants to compare visually their experience with the current EHR to a bus journey between two locations, many nuances emerged that otherwise might not have been revealed verbally (See Figure 5). From this ice-breaking problem formulation overview, each workshop then moved on to building, intervention, and evaluation. Participants were asked to indicate which aspects of the current neonate summary page worked well and were easy to use and which ones were more complex and less intuitive. The results from all four workshops were compiled into one diagram (see Figure 6).

Figure 5.

Examples of bus metaphors.

Figure 6.

Evaluation of existing neonate summary page.

Participants were then asked to construct their ideal step-by-step screen view for specific workflows/scenarios that occur daily in the hospital ward. As an agile and easily adaptable visual methodology, participants were asked to map out their ideal workflows by noting individual steps on single ‘post-it’ pieces of paper and then arranging these in the best order possible on large sheets of paper. In order to compare the transcribed workflows from different groups, similar categories were grouped into a composite visual representation using coloured boxes connected to each other with solid lines (see Figure 7). Relatively flat connecting lines indicate similar ordering and priorities of categories between groups, while steeper lines indicate larger differences in the ordering of similar themes between groups.

Figure 7.

Ward round workflow scenarios composite.

Results

Participation and engagement

Understanding and recognizing the real-world constraints around operating in a busy clinical healthcare environment where clinician time is limited, aiming for maximum engagement with minimum impact upon clinical service delivery, four workshops were held in May and June 2018. One workshop in situ in each of the four hospital sites. Across the four workshops, 10 consultant neonatologists/paediatricians participated. Fourteen doctors in training in paediatrics (11 specialist registrars/registrars and 3 senior house officers) contributed to the workshops. Eighteen NICU nurses across all levels of experience from staff nurses to nurse managers and advanced neonatal nurse practitioners attended across the four sessions. Six pharmacists with expertise in NICU prescribing practice contributed as did two neonatal dietitians and the only neonatal speech and language therapist and neonatal occupational therapist in Ireland. One administration officer working within a neonatal unit attended one of the workshops. Overall, 53 multidisciplinary neonatal healthcare team members across four hospitals contributed to the workshops. They did this within their normal busy working schedules, based on local informal internal arrangements with colleagues to ensure there was no significant interruption to ongoing ‘business as usual’ clinical service provision.

The overall shared goal of optimization of the information system to improve user experience and newborn healthcare delivery was demonstrated by the attendance of multiple healthcare team members across all four sites, with no formal cross-cover required and indeed even some team members coming in on days off in order to participate and contribute to the process. Working together participants graphically defined the differences between goal state and current state, each challenging and informing the other to reduce these differences in keeping with the technology acceptance model around perceived ease of use and perceived usefulness of the technology to promote acceptance of the developing prototype across all of the hospitals.²⁰

Sociotechnical change

The innovative visually assisted analyses, allowed the participants to identify a common theme to act as a foundation for the Workflow MPage prototype artefact. They recognized that the initial EHR design and build had in many ways transposed ways of working within traditional paper-based fragmented silos of information into new digital silos with a persisting emphasis on recording information rather than communicating and sharing it. This followed a long-standing process embedded into medical hierarchical culture where information is reviewed, and an assessment and plan are recorded separately by different disciplines each within their own area within the patient record. Communication across persons, disciplines, organizations, and time for clinical governance, audit, research, and business information continued to be time consuming, laborious, and inefficient.

Through the participatory and democratic nature of this action research approach, the neonatal multidisciplinary healthcare team collaborated with the EHR industry designers to re-evaluate this traditional way of working together. They developed a revised workflow, with a new process embedded into the developing artefact based around real-time review of information and communication from the shared digital platform and only then proceeding to documenting a medical note.

This action research approach helped all the workshop participants envision how the shared digital platform of the developing artefact could best be utilized for better communication and clinical care (see Figure 8). Within this action research approach participants examined not just what we do but how and why we do things, with an evolving artefact prototype being built around a newer agreed workflow. In comparison to the initial design and build of MN CMS, this action research approach had greater end-user participation and engagement with their greater emphasis on workflow for improved clinical care and communication prior to technical design and build, designing a new sociotechnical system rather than simply a technical system to fully exploit the technical capabilities of the ICT (see Figure 9). Even as this artefact is currently being built into the production domain for implementation across our national EHR, this agreed and adopted revised workflow is already leading to a more collaborative flattened team structure approach to clinical care at the cotside.

Figure 8.

Evolution of sociotechnical artefact – workflow MPage.

Figure 9.

Design/build versus optimization of HER.

Qualitative feedback

Participants who attended the workshops were invited at the outset to provide feedback by email to the researcher (B.P.M.) on the workshop process. This feedback from across the sites and multidisciplinary team was informative and insightful. The workshops were felt to be a

good way to involve end users on a post go-live design: users had the chance to be heard, to highlight what was working well and what was not working so well and also had the chance to suggest the changes that needed to happen to make the system more user-friendly. Having an external facilitator helped to create a ‘safe environment’ where users could express their thoughts openly. This also created a more collaborative environment across the team that was responsible for following up on the required next steps. (Industry Design Consultant)

While the independent graphic facilitator led the workshops, at times, the complex specialist nature of workflows did require interjections from the NICU work stream lead as a subject expert, who had endeavoured to remain an observer rather than a participant in producing the outputs. This is a well-recognized and reported conflict of interest for external researchers in participatory action research.²¹

Overall, I found the workshops interesting. I think having an independent facilitator greatly assisted the end users in providing honest feedback rather than feeding back directly to the design team or vendor. There were however times when I felt I had to interject in the feedback process to almost ‘translate’ to the independent facilitator in terms of medical wording or workflow steps- this was difficult at times as I tried to let the facilitator lead the sessions. (NICU Workstream Lead)

In the visually assisted creative process utilized in these iterative workshops, participants outlined their satisfaction with the visual approach used, which facilitated an interactive and varied creative design process.

The ‘bus’ metaphor facilitated a creative illustration of a very complex story – the journey to where we are now. Pictures paint a thousand words and using the graphic of the ‘bus’ certainly gave free rein to participants to express themselves and their experience of the journey to date. The use of ‘post-its’ and different colours to capture thoughts on workflows, and comments on the existing Summary page was very effective in capturing a lot of information in a short time period. (NICU Pharmacist)

I particularly enjoyed the ‘Bus Metaphor’ section of the session! I think this allows the end user to creatively tell their story/journey while highlighting the frustrations they have been feeling in a safe manner. I would warmly recommend this for future sites as it gives end users a voice in terms of their experience which has been different at each workshop.

(NICU Workstream Lead)

The predominant theme to come from participant feedback emphasized the participatory democratic aspect of the action research approach adopted in the workshops

I found the workshop in my hospital very interesting and beneficial especially as there was multi-disciplinary participation. It was very well organized, and I think it would be great if there was a similar process for the maternity side of MN-CMS in order to plan optimization of the maternity section. (Informatics Pharmacist)

User involvement in implementation and optimization processes has been shown to foster an increased sense of user ownership.²² It was felt that the independent graphic facilitator did

a great job of putting this all together. Really interesting to see what the other hospitals views are and how similar the comments are. I thought it was a really useful workshop – as much as I love the electronic record it would be great to fine tune it further. (Consultant Neonatologist 1)

Physicians attitude to EHR use and their perceptions around ease of use have been shown to be positively influenced by their perceptions of involvement.²³ Ongoing evaluation and optimization based on clinician feedback, in particular from physicians addressing immediate needs around clinical problems and workflows as has taken place in these workshops rather than emphasizing future potential benefits to the overall organization from a business management perspective has been shown to be critical for achieving EHR acceptance.²⁴

By bringing small groups of self-selected, enthusiastic, motivated end users together in situ with industry design consultants, these graphically facilitated iterative collaborative workshops produced a creative process – empowering smaller expert teams that know each other rather than a large diverse newly formed team of experts:

The strengths of the workshop I felt were the fact that all disciplines met together, and each group was represented when broken into smaller groups. The workshop helped us to identify where the system is failing and how it could be improved with better communication and recording making it more efficient to access information and therefore making it a safer patient record. I found by being broken into the smaller group which represented Consultant Neonatologist, NCHD, SLT, OT and myself ((Nurse) that I got an overall view of how collectively we all had similar issues in that we were struggling to access information in a concise and efficient manner and therefore sometimes wasting time or in some cases missing information about the patient. As a group we got to prioritize what information was required in order of importance, summarizing it in a concise and relevant way. (NICU Nurse)

Design is inherently an iterative process – very few things are completely novel but rather build on previous knowledge, in an evolving search for satisfactory solutions to work in the real-world environment. Through the repeated cycles of problem formulation, graphical build, and reflection and evaluation, on site in four hospitals within 4 weeks, the industry designers in collaboration with the healthcare team brought together an artefact prototype for build in a test domain.

Discussion

Delivery of healthcare is complex, with multiple stakeholders operating in an ever-changing resource-constrained environment. The primary aim around introducing and optimizing an EHR is to improve patient care and ultimately population health but achieving this goal and measuring the contribution of ICT implementation towards this goal is extremely challenging with many ‘unknown unknowns’ along the journey. Deployment of ICT within complex industries, such as the space industry, aviation, and nuclear power, has been accomplished safely. Healthcare ICT development and optimization can learn much from these other industries about development and implementation of ‘safety critical computing’.²⁵ Pilots did not improve aviation safety nor did nuclear power operators improve nuclear safety by themselves. They worked closely with experts in cognitive, social, and physical performance and safety to improve safety.²⁶

As a complex sociotechnical system, many issues in healthcare are ‘wicked’ and are worked through in a messy and nonlinear fashion by multidisciplinary teams. This can prove to be a challenge to ICT systems designers who seek to apply simple linear rational workflow approaches. The Neonate Summary Workflow MPage design artefact under construction in these workshops is a novel instantiation. Neonatal intensive care is an emerging specialist niche under development within this vendor’s overall strategic marketplace. This evolving artefact solves a previously unsolved problem around safely capturing and documenting care in a complex multidisciplinary critical healthcare environment. These workshops demonstrate the feasibility of the action research process for effective problem representation and development of a prototype artefact (Neonate Summary Workflow MPage). This is an example of an emerging knowledge process integrating expert knowledge with local knowledge sharing, accommodating evolving knowledge bases and knowledge translation through iterations with functional prototypes.²⁷ This in situ collaborative small group approach defines the ideas and practices through which the optimization, implementation and use of the information system can be effectively and efficiently accomplished.²⁸

This action research represents a lean and agile approach to design for faster optimization and implementation across multidisciplinary teams within an organization and between organizations that are endeavouring to manage change on a national scale while still maintaining clinical services with limited resources especially in terms of healthcare staffing. In the collaborative process outlined in this action research, the artefact development is seen to be a function of the persons, harnessing their pre-existing relational connections to each other in the multidisciplinary neonatal healthcare team for collaboration within the innovation context, the alignment of the innovation within the wider healthcare environment, and the technological capability of the innovation.²⁹

Health information systems research such as this action research approach to the optimization needs to be disseminated not just to technology-oriented design science audiences for evaluation of the artefact developed. Wider dissemination should include healthcare providers and most especially healthcare management-oriented audiences so as to bring the design science and behavioural science components together to consider the artefact development process described as an agile innovation process for wider organizational gain. Although iterative, collaborative, and agile, the action research approach applied in this project has its limitations. Teams of well-intentioned clinicians and software engineers from industry may believe that understanding clinical processes combined with clever programming may solve the challenges faced in healthcare ICT design, but framing problems too narrowly, failing to recognize the ‘unknown unknowns’ can lead to a failure of the system to ‘fit’, owing to a lack of understanding of goals, roles and tasks and how team members feel about their work. For these solutions to be truly transformative we require extensive analysis not just of what we do, but how and why we do things, ideally with contributions from behavioural scientists, design scientists, and healthcare team members towards a better understanding of the required ‘team cognition’ for better communication and coordination of care and how an EHR can enhance this goal.²⁶ Optimal design and evaluation should include social, contextual, and organizational issues (communication, care, control, and context) that brief workshops such as in this project cannot fully encapsulate, even with innovative graphical visual facilitation to capture some of the non-verbal nuances that may be lost in translation.³⁰

Acknowledging the limitations of a small study evaluating optimization of a niche area within a recently implemented EHR, bringing healthcare teams and industry designers together in situ to communicate shared empathic understanding of roles and goals as demonstrated in this action research optimization project is seen to be only the beginning of a longer, wider, deeper collaboration that is required between healthcare organizations, clinicians, industry, and academic partners. It represents the commencement of a more open agile approach to an ongoing design process aiming to improve usability, efficiency, safety, and user satisfaction for information systems innovation in healthcare. By investing in the necessary personnel, bringing these multidisciplinary viewpoints across these four partnerships together in an environment that promotes team working, collaboration and sharing of ideas, networks, connectivity, and value are created that benefits all participants in what becomes a rich source of learning and future research, an innovation ecosystem to truly meet the complex challenges faced in healthcare.³¹

Information system development (ISD) for healthcare requires consideration not just of what we do but how and why we do things. Our analysis demonstrates that action design research represents an agile and lean approach for successful optimization and implementation of ISD in healthcare.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

Approval was obtained from the Social Research Ethics Committee of University College Cork for this study.

ORCID iDs

Brendan Paul Murphy

Michelle Carr

References

DeVore

Figlioli

. Lessons premier hospitals learned about implementing electronic health records. Health Aff 2010; 29(4): 664–667.

Poses

Isen

. Qualitative research in medicine and health care: questions and controversy. J Gen Intern Med 1998; 13(1): 32–38.

Haines

Jones

. Implementing findings of research. BMJ 1994; 308(6942): 1488–1492.

Jones

. Why do qualitative research? BMJ 1995; 311(6996): 2.

Sackett

Rosenberg

Gray

, et al. Evidence based medicine: what it is and what it isn’t. BMJ 1996; 312: 71–72.

Greiver

Barnsley

Glazier

, et al. Implementation of electronic medical records: theory-informed qualitative study. Can Fam Physician 2011; 57(10): e390–e397.

Samhan

Joshi

(eds). Resistance of healthcare information technologies; literature review, analysis, and gaps. In: 48th Hawaii international conference on system sciences, Kauai, HI, 5–8 January 2015.

Carayon

Karsh

B-T

. Sociotechnical issues in the implementation of imaging technology. Behav Inform Technol 2000; 19(4): 247–262.

Moore

Benbasat

. Integrating diffusion of innovations and theory of reasoned action models to predict utilization of information technology by end-users. In: Kautz

Pries-Heje

(eds) Diffusion and adoption of information technology. Boston, MA: Springer, 1996, pp. 132–146.

10.

Barbosa

. Teamwork in the neonatal intensive care unit. Phys Occup Ther Pedi 2013; 33(1): 5–26.

11.

Brodsky

Gupta

Quinn

, et al. Building collaborative teams in neonatal intensive care. BMJ Qual Saf 2013; 22(5): 374–382.

12.

Sein

Henfridsson

Purao

, et al. Action design research. MIS Quart 2011; 35: 37–56.

13.

Hevner

. Design science in information systems research. MIS Quart 2004; 28(1): 75–105.

14.

Coenen

Donche

Ballon

, et al. LL-ADR: action design research in living labs. In: 48th Hawaii international conference on system sciences, Kauai, HI, 5–8 January 2015.

15.

Meyer

. Using qualitative methods in health related action research. BMJ 2000; 320(7228): 178–181.

16.

Murphy

Armstrong

Ryan

, et al. Benchmarking care for very low birthweight infants in Ireland and Northern Ireland. Arch Dis Child Fetal Neonatal Ed 2010; 95(1): F30–F35.

17.

Bergvall-Kareborn

Hoist

Stahlbrost

, et al. Concept design with a living lab approach. In: 42nd Hawaii international conference on system sciences, Big Island, HI, 5–8 January 2009.

18.

Akerson

. EHR implementation missed the mark. How to hit the bullseye with optimization projects? 2016, https://www2.deloitte.com/content/dam/Deloitte/us/Documents/life-sciences-health-care/us-lshc-ehr-implementation-missed-the-mark-v1.pdf

19.

Robertson

Cresswell

Takian

, et al. Implementation and adoption of nationwide electronic health records in secondary care in England: qualitative analysis of interim results from a prospective national evaluation. BMJ 2010; 341: c4564.

20.

Venkatesh

. Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. Inform Syst Res 2000; 11(4): 342–365.

21.

Dickson

Green

. The external researcher in participatory action research. Educ Action Res 2001; 9(2): 243–260.

22.

Lorenzi

Riley

. Managing change: an overview. J Am Med Inform Assoc 2000; 7(2): 116–124.

23.

Morton

Wiedenbeck

. A framework for predicting EHR adoption attitudes: a physician survey. Perspect Health Inf Manag 2009; 6(Fall): 1a.

24.

Doolan

Bates

James

. The use of computers for clinical care: a case series of advanced US sites. J Am Med Inform Assn 2003; 10(1): 94–107.

25.

Leveson

Diaz-Herrera

. SafeWare: system safety and computers. Reading, MA: Addison-Wesley Reading, 1995.

26.

Karsh

B-T

Weinger

Abbott

, et al. Health information technology: fallacies and sober realities. J Am Med Inform Assn 2010; 17(6): 617–623.

27.

Markus

Majchrzak

Gasser

. A design theory for systems that support emergent knowledge processes. MIS Quart 2002; 26: 179–212.

28.

Denning

. A new social contract for research. Commun Acm 1997; 40(2): 132–134.

29.

Costello

Donnellan

Curley

. A theoretical framework to develop a research agenda for information systems innovation. CAIS 2013; 33: 26.

30.

Kaplan

. Evaluating informatics applications: clinical decision support systems literature review. Int J Med Inform 2001; 64(1): 15–37.

31.

Edwards

. The quadruple helix model of open innovation (Innocentive innovation blog), 2018.