Abstract
This study identified behavioral and organizational barriers and facilitators related to the implementation of a clinic-based pediatric injury prevention program. Safe N’ Sound (SNS), an evidence-based tailored injury prevention program designed for pediatric primary care, was implemented in five pediatric clinics in North Carolina. Office managers participated in structured interviews; health care providers participated in focus groups. Waiting room observations were conducted in participating clinics. Qualitative data captured perceptions of program implementation, including experience in integrating the program into clinical practice, usage by parents and providers, and recommendations for improving implementation. Reported facilitators of program use included usefulness and likeability of customized materials by parents and physicians and alignment with clinic priorities for injury prevention. Barriers included perceived staff burden despite the program’s low staff requirements. Consequently, practices experienced difficulty integrating the program into the waiting room environment and within existing staff roles. Recommendations included formalizing staff roles in implementation. Waiting room observations supported greater technology maintenance and staff involvement. Findings suggest a dynamic relationship between program implementation and the adopting organization. In addition to considering characteristics of the intervention, environment, and personnel in intervention development, implementation may require customization to the organization’s capacity.
Keywords
Introduction
Disconnects in the processes encompassing the development of a program, its evaluation, and its implementation in real-world settings limit the public health impact of health behavior research (Brownson & Simoes, 1999; Ferlie & Shortell, 2001; Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). Factors affecting the translation of research to practice span intervention characteristics, intended target settings, and research design. Barriers may reflect limited resources, time, organizational support, prevailing practices that work against innovation, and competing demands. Research designs may not optimally support translation, with insufficient evaluation of cost, reach, setting adoption, implementation, maintenance, and sustainability (Glasgow & Emmons, 2007).
Challenges in the dissemination of evidence-based health promotion programs suggest a need for qualitative investigation of the translation of such programs into practice to better understand factors that contribute to implementation success or failure. We investigate these factors in the context of a computer-based pediatric injury prevention intervention. Previous research on uptake of computer-based programs in health care and community settings has focused largely on measuring their reach or effectiveness (Bergman, Beck, & Rahm, 2009; Glasgow, Nelson, Strycker, & King, 2006; Kreuter, Alcaraz, Pfeiffer, & Christopher, 2008; Thompson, Lozano, & Christakis, 2007; Trinks, Festin, Bendtsen, & Nilsen, 2010; Walton et al., 2010; Williams, Boles, & Johnson, 1998). There are insufficient data on organizational and behavioral factors related to implementation and maintenance of evidence-based programs. Barriers and facilitators are commonly omitted or are reported in the context of “anticipated” items that may aid in long-term program maintenance (Glasgow et al., 2006).
Unintentional injuries are a leading cause of childhood morbidity and mortality (National Center for Injury Prevention and Control, 2013). Anticipatory guidance during pediatric health care is a recommended and efficacious strategy (DiGuiseppi & Roberts, 2000), which parents value (Schuster, Duan, Regalado, & Klein, 2000). However, it is frequently not provided (Chen, Kresnow, Simon, & Dellinger, 2007) and typically consists of brief recommendations for specific safety devices (e.g., car seat) or behavioral adjustments (e.g., storage of dangerous substances); duration of counseling averages 1 minute (Chen et al., 2007). Time constraints and competing demands are well-documented reasons for the gap between recommendations and practice (Woods, 2006).
Safe N’ Sound (SNS) is a computer-based injury prevention program that facilitates targeted communication in pediatric primary care by providing individually tailored information to both parents and providers on the child’s injury risks and specific behavioral recommendations. Parents complete an assessment using a touch screen computer in the waiting room; the program prints a booklet for the parents tailored to the child’s age, risk factors, and parent perceptions and a corresponding summary for pediatricians. SNS can be used at each well-child visit through age four, allowing parents to receive information consistent with the child’s age and changing injury risks. SNS has been evaluated in multiple settings (Nansel et al., 2002; Nansel, Weaver, Jacobsen, Glasheen, & Kreuter, 2008), has been adapted for community-based clinics (Weaver et al., 2008), and is available in Spanish and English. SNS was favorably received by parents and physicians and demonstrated effectiveness in promoting injury prevention behaviors, especially among low income families (Nansel et al., 2008).
We implemented SNS in five pediatric primary care practices without on-site research support to study implementation in routine care. The overall reach of SNS—the percentage of eligible patients who completed the program—was 7.6% (Weaver et al., 2011). Although this represents a large number of parents who received information with demonstrated efficacy, it is nonetheless lower than optimal. Concurrent to program implementation, qualitative methods were used to explore factors related to implementation, maintenance, and reach. The purpose of this manuscript is to describe results from the qualitative study, including facilitators, barriers, and recommendations to enhance program implementation and maintenance.
Method
Five pediatric clinics within a North Carolina health care consortium were provided the computer, program, supplies, and training to implement SNS. The consortium’s injury center invited member clinics who had previously requested information on injury prevention programming, as well as additional clinics within the consortium to increase the diversity of practice types. Initial communication with the medical director and office manager determined a consensus of interest within clinic leadership to adopt the program. Participating clinics included one rural, two urban, and two suburban clinics. Size ranged from 2 to 10 providers. Detailed study methods and demographic characteristics of the participating clinics are described elsewhere (Weaver et al., 2011). The study, which took place from July 2006 to April 2008, was approved by the institutional review board of the Carolinas Medical Center; informed consent was obtained from participants.
During an introductory meeting with the office manager and lead physician, the program was presented as a way to enhance routine provision of injury prevention anticipatory guidance; to improve the quality of that guidance by facilitating targeted communication specific to each family’s needs; and to streamline counseling time by prioritizing among potential injury prevention topics. Providers were given information on the program’s efficacy in prompting adoption of injury prevention behaviors. The study coordinator, a member of the consortium’s injury center, met individually with office managers to explain program implementation and discuss logistical issues, including type of device (tablet vs. desktop), placement in the office, and assignment of implementation tasks. The office manager and clinic staff then decided on specific aspects of the program implementation to best meet their clinic’s needs, with minimal coaching from the study coordinator. Brochures, banners, and flyers in the waiting room, and an invitation computer screensaver encouraged families to participate. Clinic staff members were asked to direct parents to use the program when checking in for their appointment and to routinely check the device in order to ensure that it was working. The study coordinator provided additional coaching as needed to address difficulties encountered; for example, suggesting that front desk staff highlight scheduled appointments of patients within the eligible age range to prompt them to use the program.
The program began at the smallest clinic for 3 months to determine unforeseen logistical issues in implementing the program and then was implemented in the remaining four sites for a 3-month implementation phase and a 6-month continuation phase. During implementation, clinics could contact a research staff member for general project support. In the continuation phase, we withdrew program support to assess program usage in routine clinical practice.
Measures
Provider focus groups
One focus group was conducted with all providers at each clinic. Questions addressed implementation and use of SNS, including issues that facilitated or hindered usage, perspectives regarding the program materials, implementation logistical issues, and recommendations. Twenty-one providers (16 pediatricians, 4 family practice physicians, and 1 nurse practitioner) participated in 5 focus groups during the implementation phase.
Office manager interviews
Individual interviews with each clinic’s office manager were conducted during the implementation phase to understand the barriers and facilitators of program use. Interviews were conducted during the continuation phase to determine long-term use, gauge the level of investment, and collect suggestions for improvement. All clinic managers participated.
Waiting room observations
Twelve clinic observation sessions were conducted—one at the initial site and two or three at each of the remaining four sites, with at least one observation in the implementation and one in the continuation phase. Different days and clinic periods were observed. In total, 33 hours of waiting room observations were conducted, during which staff observed 119 potential users.
Analysis
Focus groups and interviews were audiotaped and transcribed verbatim. Broad categories derived from the focus group and interview guides were used to structure the coding scheme. Three team members (JT, NLW, MB-H) reviewed each transcript and independently identified text units that addressed the coding scheme. The team collaboratively incorporated additional codes emerging from the data into the coding scheme. Text describing the codes was then analyzed together to identify themes for each code. Themes represent ideas that were widespread in the discussion. Content covered by the focus group guide is shown in Table 1.
Content Areas of Focus Groups and Interviews
Results
Findings From Interviews and Focus Groups
Facilitators of SNS implementation and maintenance
Facilitators of SNS implementation and use primarily pertained to the appeal and utility of the materials. Staff and providers reported that parents liked the personalized materials and saw them as visually appealing and engaging (Table 2, Item a). Participants indicated that providing materials that parents could review at home was beneficial (Table 2, Item b). Parents’ comfort with the computer device and ease of program use were also mentioned as facilitators.
Common Facilitators and Barriers of Safe N’ Sound (SNS) Implementation and Maintenance With Example Quotes
Participants indicated that SNS aligned with existing clinic priorities for injury prevention (Table 2, Item c). Providers preferred SNS to their previously used injury prevention handouts because customized take-away messages focused their discussion of injury prevention. Providers voiced agreement with the SNS content (Table 2, Item d).
Barriers to SNS implementation and maintenance
Although program materials were seen as appealing, participants indicated that the kiosk did not entice parents to use the program, which was a frequently mentioned barrier. Intrinsic interest in SNS was not enough to prioritize kiosk usage; many staff and providers described a need for the kiosk to be more visually appealing (Table 2, Item e).
Staff and providers noted the burden required by staff to implement and maintain the program. One office manager said that it was “easy to implement the program but hard to sustain it.” Concerns were voiced that staff have other duties and may forget to ask parents to complete the kiosk assessment or to check the device to ensure it is working (Table 2, Item f). Lack of organizational leadership to engage staff and low level of involvement from providers were also cited (Table 2, Item g).
The waiting room environment was perceived as being a difficult place for parents to complete SNS. Although the program was designed to fit into the flow of the clinic, there were concerns that parents could get called for their appointment before finishing the program. Competing parent demands included attending to multiple children or completing paper forms (Table 2, Item h). Participants also noted that patients and clinic staff do not typically encounter kiosks in the waiting room, complicating integration into the clinical setting (Table 2, Item i).
Management of the provider summary sheets posed another barrier. These were printed along with the parent booklet but were often not given to the provider. Additionally, because use of SNS did not qualify for insurance reimbursement, staff and providers reported it was assigned a lower priority than reimbursable programs.
Differences in perspectives across time
Between the implementation and continuation phases, for some office managers certain facilitators evolved into barriers. During the implementation phase, office managers at three clinics stated that SNS did not pose substantial staff burden. However, when interviewed later, they all indicated a burden. At one clinic, the office manager noted that having an existing venue for injury prevention was helpful for efforts to implement SNS. Later, the manager indicated that since the clinic already addressed injuries well, SNS was perceived as providing insufficient additional benefits (Table 2, Item j).
Clinic recommendations
Office managers and providers recommended improving the integration of SNS into clinical practice, improving program use by parents, and increasing program endorsement by office staff and providers, particularly in terms of directing parents to use the kiosk (Table 3). Participants also indicated that program support should be formalized into the clinical job so that SNS use becomes a part of the clinical routine. There was strong support for a program that required neither staff nor provider involvement. Providers thought the material could stand alone.
Recommendations for Safe N’ Sound (SNS) Implementation From Interviews and Focus Groups
Most recommendations involved ways to improve program use by parents. These included offering incentives to front desk staff for the number of parents who completed SNS and improving program marketing to encourage parent use. All the provider focus groups raised the need to have a feedback loop that would promote implementation of SNS such as demand from parents and support from colleagues.
Findings From Waiting Room Observations
Waiting room observations supported barriers identified in the interviews, including technology maintenance, staff participation, and attention from parents to the program. Of the 119 eligible families arriving during observation periods, eight were invited by staff, three of whom elected to participate. Additionally, four parents used the program without any invitation. Of the seven users, three completed the program.
Of the 119 families, 110 were in the waiting room with sufficient time to complete SNS prior to their visit. Parents typically used waiting room time to talk on the phone, make appointments, feed their children, or use the restroom. The observer informally noted that for many families, more than one parent was present. None of the parents who used the program experienced any difficulties.
Potential barriers noted included the placement of the device and problems with the printer. At one clinic, there was little space between the device and seating area, offering those completing the program little privacy. At another clinic, placement of the device near the receptionist’s desk created crowding of the receptionist area. In two clinics, the device was placed away from the receptionist and thus its presence could not remind staff to invite parents to use it. Program signs were removed by some staff. At one clinic, the wireless Internet was frequently down; information technology staff subsequently trained front desk staff to fix this problem.
Discussion
In this implementation study, we disseminated a program with demonstrated efficacy and of value to providers and parents. The program was designed to fill a known gap between recommendations for injury prevention anticipatory guidance and the realities of limited time and competing priorities during well-child visits. SNS was distributed in a process similar to the way programs are often distributed: We provided the program, materials, resources, and minimal program support largely without an implementation strategy, per se. By mimicking the distribution process of programs offered at conferences, by websites, or evidence-based clearinghouses, we were able to study the resulting uptake that might be expected from such passive dissemination approaches and the factors that reportedly influenced uptake.
Findings from the implementation of SNS show dynamic relationships between implementer, user, setting, and intervention that inform dissemination efforts. These relationships were recently synthesized and summarized as a Consolidated Framework for Implementation Research (Damschroder et al., 2009). Conceptual domains addressed include intervention characteristics, inner and outer settings, characteristics of the individuals involved, and the process of implementation, and provide a useful structure for interpreting findings regarding SNS program implementation.
Characteristics of the intervention served to facilitate implementation. Providers liked the materials and valued the individualized messages that parents perceived as highly relevant. However, findings were mixed as to whether the program offered a significant advantage over current practice, suggesting that participants’ experiences with implementation barriers may have reduced the program’s perceived value. Providers’ reports that SNS did not have sufficient intrinsic interest for parents suggest that they attributed the low rate of program use to characteristics of the program or parents rather than reflecting a need to address existing implementation barriers. While clinics initially preferred desktop to tablet devices for security reasons, their subsequent experience in implementation caused some to develop a preference for portable devices. In future applications, barriers associated with particular technologies may change. As clinics move toward expanded use of electronic medical records, there may be greater opportunities for integrating health promotion programs into existing computer systems. Specifically, electronic medical records may include prompts for program usage, and program feedback could be directly exported to the patient’s medical record for physician usage and resolve communication barriers between staff and patient.
Several challenges encountered in SNS implementation are understood within the context of the inner (organization’s culture, climate, and environmental readiness) and outer (economic, political, and social context) cultures of the setting. Although there was a consensus among leadership in program adoption, this did not translate into leadership in the actual day-to-day use of the program. The lack of organizational leadership in promoting SNS may partly reflect the structural organization of primary care practices, in which the roles and responsibility of providers and staff tend to function in parallel, making it difficult for any one person to facilitate organizational change that affect personnel across multiple roles. Program maintenance may require changes in organizational systems to adequately support program integration. Lack of SNS integration into clinic procedures, including staff directing parents to the kiosk, represented a substantial barrier. Providers were reluctant to make requests of clinic staff that would increase time demands. Methods to achieve such organizational changes likely need to be generated from the practice itself. Certainly, successful implementation would require greater communication and coordination between providers, nurses, office manager, and front desk staff to integrate program-related tasks into existing responsibilities. For example, in this study, office managers determined staff roles in implementation, and most sites did not initially involve nurses. When it became apparent that the program might benefit by greater nurse involvement, office staff elected to handle communication with nursing staff rather than have research staff provide additional training. Importantly, there were few external contingencies to enhance SNS implementation. While the program was seen as aligning with injury prevention priorities, providers indicated that they would be more supportive if they heard positive feedback from medical conferences, pediatric peers, and patient satisfaction surveys. Importantly, SNS was not a billable service, an essential driver in health care.
Individual characteristics influencing implementation included staff perceptions regarding the burden of program implementation and time for parents to complete the program. Individual and setting characteristics may explain the observed shift in perception regarding program burden. Initial enthusiasm for SNS was likely influenced by the low staffing requirement previously agreed on by the office managers. However, due to difficulties integrating the program tasks into existing roles, these tasks, while minimal, were perceived as more burdensome than initially anticipated. Consequently, practices recommended that the program be less staff driven. Additionally, because office managers were the primary point of communication across roles (providers, nurses, and office staff), most of the effort involved in program maintenance fell to them. The lack of program integration may likewise account for perceptions regarding program use and feasibility. Notably, while staff reported lack of time for parents to complete the program in the waiting room as a barrier, waiting room observations showed ample time for SNS use for the majority of parents. However, the program required users to engage in a novel waiting room behavior, as parents are accustomed to waiting passively for staff direction. In addition to a lack of prompting from staff, distractions and other priorities appeared to have prevented parents from taking the initiative to use the kiosk.
Conclusions
Findings from this implementation study may inform components of future program implementation strategies. In practice, programs are often disseminated with “toolkits,” complete with all necessary program materials; however, these are seldom accompanied by an implementation plan. Findings suggest that new programs might be better sustained if implementation strategies can be customized to the capacity of the adopting organization relative to the demands of the innovation. When the program requires little change from existing procedures or when organizational capacity to implement needed changes is high, a typical, minimal implementation strategy may well be sufficient. However, it is more likely that there will be unanticipated organizational factors that need to be addressed for successful program implementation and maintenance. For instance, in clinics that experience frequent staff changes, it may be important that the implementation plan includes changing job descriptions to account for activities related to the innovation and creating feedback loops to establish the program in the flow of care. In a smaller clinic, it may be important to partner directly with the health care provider, whereas in larger clinics, it may be necessary to partner with the nurse manager.
Customized implementation strategies might be empirically investigated in future research. Such research should focus on the need to account for specific program functions in job descriptions and to accurately estimate required staff time. Importantly, these shifts in system supports may require substantial time in larger hospital systems that may not be as nimble as smaller clinics. Even for programs that are easy to implement and require few resources, it is important to have defined roles to support the implementation processes. Our findings are consistent with other translation research indicating the importance of establishing this infrastructure early in the adoption process (Bradley et al., 2004). Findings also suggest the need for innovative approaches to deliver training or coaching across personnel involved in program delivery, as opposed to targeting education of the project to clinic leadership, especially sites with multiple layers of authority. Health promotion researchers may also find insight by exploring how commercial organizations integrate new products and services into existing workflow.
Findings from this study demonstrate the complex relations of intervention characteristics with organizational and personnel factors and suggest the utility of customizing implementation to the capacity of the adopting organization. Findings can inform the design, evaluation, and implementation strategies of future interventions across different areas of health and health care settings, allowing for greater translation of research into practice.
Footnotes
This research was supported by the Intramural Research Program of the Eunice Kennedy Shriver National Institute of Child Health and Human Development.
The authors have no conflicts of interest to disclose.