Formative Evaluation of a Workforce Investment Organization to Provide Scaled Training for Home Health Aides Serving Managed Long-Term Care Plan Clients in New York State

Background and Significance

The Home Health Aide Workforce and Workforce Development Initiatives

There is a critical need to improve the quality, supply, and retention of aides amid increasing demand (Bureau of Labor Statistics, 2021; Marquand & Chapman, 2014; Nisbet & Morgan, 2019; Spetz et al., 2019). Experts have called for competency-based education requirements and support for training programs that recognize the value and contribution that aides bring to team-based home care (Stone & Bryant, 2019).

Despite the need for better training, limited evidence exists to inform the development and adoption of training programs within value-based payment models. Previous research evaluating training programs for aides and other direct care workers suggests that workforce development initiatives can help improve job performance, engagement, and retention (Danilovich et al., 2020; Feldman et al., 2019; Guerrero et al., 2020). Training for in-home supportive services dementia caregivers was found to increase their self-reported ability to provide care (Guerrero et al., 2020). Aides participating in a health interviewing program expressed that the training gave additional meaning to their work and facilitated conversations with clients (Danilovich et al., 2020). An initiative to improve skills, job satisfaction, and retention among aides found that an expanded training curriculum designed around adult-learners and peer-mentoring increased retention (Feldman et al., 2019). Workforce development initiatives implemented among direct care workers in nursing homes suggest mixed results. Training programs for certified nursing assistants that went beyond licensing and credentialing requirements were associated with greater job satisfaction (Han et al., 2014). Another large randomized study of a quality improvement program orienting nursing home staff to tools and strategies for identifying, assessing, communicating, and documenting resident status changes found no effects on hospitalization and emergent care utilization (Kane et al., 2017). The results of this study may be attributable to barriers surrounding program implementation, including the complexity of culture change, leadership instability, competing demands, scarce resources, and technical problems (Tappen et al., 2017). These findings highlight the importance of understanding how large-scale initiatives like the WIO are implemented in practice, particularly when those initiatives are intended to support systemwide transformation through value-based payment models.

Present Study and Guiding Research Questions

The success of complex interventions like WIO depend largely on their structural or system-level contexts (DeCorby-Watson et al., 2018; LaFond et al., 2002; McCreight et al., 2019; Smith et al., 2006). This study is part of a larger evaluation effort that attempts to understand the process, policy context, development, implementation, outcomes, and lessons learned from WIO. Qualitative interviews were conducted with healthcare organizations and program stakeholders to address the following research questions: (RQ1) What was the social and political context surrounding WIO implementation, and what problems were WIO training programs designed to address? (RQ2) How do stakeholders describe their experiences with WIO implementation, including their concerns, barriers, and facilitators? (RQ3) What were the outcomes, intended or otherwise, of WIO programs for collaboration and quality improvement efforts among MLTC plans and agencies?

Setting and Program Description

This study focused on the development and implementation of care quality training and skill-building programs for aides whose employer agencies participated in one WIO collaboratively formed by the 1199 Service Employees International Union Training & Employment Funds, Continuing Care Leadership Coalition, and Visiting Nurse Service of New York (Ladders to Value Workforce Investment Organization, 2020). Under risk-sharing agreements, the training programs we studied were prioritized by respective agencies implementing the training among their workforce. The programs oriented aides to improve client performance on value-based payment measures with strategies for recognizing and reporting changes in their client’s health status, and working within an interdisciplinary care team (Appendix A). These training programs were distinct from those offered by other WIOs as the implementation involved substantial intra- and inter-organizational coordination. The selected programs were delivered through classroom lectures, with incorporated field internships in one program involving advanced training (i.e., Health Coaching). Agencies instituted mandatory completion requirements for aides serving clients of MLTC health plans affiliated with the WIO. Figure 1 displays a Logic Model detailing program processes and intended outcomes (Centers for Disease Control and Prevention, 2018). Briefly, this model proposes that investments into long-term care workforce development programs facilitated the delivery of training programs for aides. Training programs are intended to (1) enhance knowledge, skills, and team-based engagement among aides (short-term outcome); (2) strengthen communication and collaboration among MLTC plans and contracted agencies (intermediate outcome); and (3) improve retention, recruitment, and recognition of aides (long-term outcome). Contextual factors surrounding the programs include the transformation to value-based payment, challenges and barriers to the recruitment and retention of aides, and calls for increased support for training programs that recognize the value and contribution of aides. Between April 2018 and February 2020, a total of 11,163 aides across the four participating agencies completed one or more of these training programs. The WIO reimbursed agencies for the time that aides devoted to training programs in lieu of client services. Training recipients were mostly female (96.7%) with a mean age of 52 years (standard deviation = 11.9 years). These characteristics are comparable to U.S. home care workers (PHI., 2019).OPEN IN VIEWER

Data Collection

18 individual or group interviews were conducted with 26 organizational stakeholders involved in the development and implementation of the WIO. Interview participants were recruited by email and phone. We aimed to interview leaders from two MLTC plans and five agencies participating in the WIO. Stakeholders from both targeted MLTC plans and three of the five agencies took part in interviews individually or as a group (see Table 1). Interviews were conducted in-person and via phone between December 2019 and March 2020. Written (for in-person interviews) or verbal informed consent (for phone interviews) was obtained from each participant prior to their participation. Semi-structured interview guides were developed for each stakeholder group. Questions encompassed: (1) perspectives on challenges inherent to the recruitment and retention of aides within long-term care, (2) implementation of training programs, (3) development and application of new skills, and (4) organizational changes observed following program implementation. Interview sessions lasted between 30 and 60 minutes, were audio recorded, and transcribed verbatim. Identifying information was removed from all transcripts. Each transcript was assigned a unique identification number and classified by stakeholder group. The Institutional Review Board at the Visiting Nurse Service of New York approved all study procedures.

OPEN IN VIEWER

Table 1.

Stakeholder Group Participation and Format of Qualitative Interviews.

Stakeholder Group	Number of Study Participants	Interview Format
Licensed home care services agency (agency) administrators	n = 9	Group and individual interviews
Managed long-term care (MLTC) plan administrators	n = 6	Group and individual interviews
Training and education staff employed by participating agencies and MLTC plans	n = 4 (agency); n = 1 (MLTC plan)	Group and individual interviews
Home health aides (aides) who completed the WIO training programs	n = 6	Individual interviews

Data Analysis

Thematic analysis was used to draw insights from and summarize key features of the qualitative data by examining the perspectives of different WIO program stakeholders and highlighting their similarities and differences (Nowell et al., 2017). Transcripts of the interviews were imported into NVivo software to aid with creating a codebook and data analysis. The lead author (DR) familiarized themself with the data by reading transcripts line-by-line and documenting initial thoughts about codes and themes. Next, the lead author developed a coding framework by reviewing five transcripts for a priori topic areas (e.g., implementation challenges, aide skills, interdisciplinary teamwork), and emerging concepts (e.g., value of aide services, collaboration between MLTC plans and agencies). This framework was revised and refined through discussions with the second author (MF), incorporating feedback on content and parsimony. The lead author then applied this final framework by coding all transcripts. Three authors (DR, MF, EF) independently reviewed coded excerpts related to each of the research questions. Then, over the course of several meetings, these authors met to discuss themes by identifying patterns, connections, and distinctions in the data. Stakeholder groups were analyzed separately and then compared and contrasted to highlight emergent themes and divergent perspectives. Findings and analytic decisions were discussed in regular team meetings. Detailed notes and memos documented personal assumptions to limit biases and strengthen objective interpretation. We returned to this audit trail throughout the analytic phase to confirm our interpretations and ensure that no relevant data was inadvertently or systematically excluded (Lincoln & Guba, 1986).

Implementation Context: Value-Based Payment Re-Oriented Organizational Workforce and Service Delivery Priorities for Plans and Agencies

Managed Long-Term Care plan and agency stakeholders described the New York State long-term care system as a diffused industry with complex plan-agency affiliations, a transient home care workforce, and a lack of centralized management over organizations comprising the State’s long-term care system. The transition to value-based payment motivated MLTC plans and agencies to focus on “quality,” manifested concretely through standardized metrics intended at operationalizing abstract notions of “value” and “member choice.” Financial risk-sharing introduced through the value-based payment model placed pressures on plans and agencies to get the “monetary reward” and motivate their staff to be more “proactive” in working toward improving metric performance, including through training and education programs as well as collaborative quality improvement initiatives. As a response to the State’s value-based payment requirements, MLTC plan administrators described their efforts to “fine tune” their agency networks to enable development of risk-sharing contracts and closer collaboration with agencies serving larger shares of their members. Workforce Investment Organizations funding enabled agencies to develop and implement training programs for aides that focused on selected quality metrics incentivized by the payment system. Consequently, some training program curricula were directly aligned with risk-sharing value-based payment contracts between MLTC plans and agencies.

Implementation Context: Home Health Aide Recruitment and Retention is Undermined by their Undervalued, Precarious Role in Long-Term Care

The undervalued and precarious role of aides within the growing long-term care industry was seen by agency administrators as a central barrier to workforce recruitment and retention. While aides themselves expressed motivations for pursuing a job in home care, including flexible scheduling and a desire to serve in a caring profession, both agency administrators and aides pointed to issues of aides working across multiple employer agencies, receiving minimal compensation and recognition for physically and emotionally challenging labor, and lacking a team-based connection to other healthcare workers. Agency administrators also noted competition with other low-wage positions, minimal licensing and training requirements, a lack of credentialing programs to distinguish between aides with greater or lesser training and experience, and limited career advancement opportunities.

Implementation Experiences: WIO Flexibility and Resource Sharing were Valued by Stakeholders but Generated Uneasiness

Workforce Investment Organizations policies granted agencies considerable flexibility in the design and implementation of training programs. This flexibility was valued by both agency and MLTC administrators because it encouraged sharing of training materials between agencies and allowed training programs to focus on shared quality goals defined by performance on value-based payment measures. Workforce Investment Organizations’s flexibility also enabled smaller agencies with limited staffing resources to subcontract with larger agencies to deliver training programs to their aides. While MLTC administrators favored flexibility, agency administrators expressed hesitation about collaborating with competing agencies, fearing they may lose aides and patients to competing agencies with preferred training/service offerings.

Implementation Experiences: Training Program Instruction was Viewed Favorably, Yet Delivery and Implementation Challenges Existed

Agency administrators and aides rated the quality of instruction favorably. Managed Long-Term Care plan administrators felt they were not close enough to training programs to judge them. Instructors described taking a practical, hands-on approach in which they used multiple, interactive modalities to foster learning and engagement, while reducing repetition. Aides valued the time devoted to training programs, the clarity of instruction, and the applicability of course concepts to client care (e.g., understanding how health promotion efforts can be viewed through the transtheoretical model “stages” of health behavior change). Instructor experience, training, and versatility were viewed by agency administrators and instructional staff as beneficial to the program’s quality. Instructors also noted the complexity of their roles in working with such a diverse workforce, reporting that they served as counselors for aides navigating their healthcare careers and needed to balance their pedagogy between aides with different literacy levels. While the content and delivery of the programs were viewed positively by aides and instructional staff, agency administrators described challenges with scheduling aides for training programs, balancing training requests with client care schedules, overtime rules, family responsibilities, and aides’ work hours across multiple agencies.

Program Outcomes: WIO Further Facilitated Inter-Organizational Communication and Provided a Structure for Collaboration

Managed Long-Term Care plan and agency administrators both viewed WIO as facilitating cross-organization communication and providing a “structure” for collaborations already underway with the shift to value-based payment. Managed Long-Term Care plans and agencies introduced new workflows, interventions, initiatives, and technologies alongside WIO training programs to monitor high-risk clients and improve communication within and across organizations. One example included clinical rounds to discuss client health and healthcare utilization, including recent falls and hospitalizations. Steps were taken to integrate aides into interdisciplinary healthcare teams by encouraging them to notify MLTC plans of medical problems observed in the home that might otherwise go unreported, such as medication reactions, wound deterioration, infections or falls. Aides described how training programs improved their observation and communication skills, by encouraging them to voice their concerns and interact with clinical staff, leading them to feel more empowered and engaged. However, agency and MLTC administrators noted several barriers to communication and collaboration, including multiple aides working for the same client, gaps in data and information sharing between plans and agencies that allow aides to share clinical events and alerts with care management teams, and the exclusive focus of WIO training on aides as opposed to other interdisciplinary team members.

Program Outcomes: Stakeholders viewed WIO Training Programs as Beneficial but Questioned Measurable Impacts

Agency and MLTC administrators praised WIO training programs for enabling a large number of aides to learn new concepts and skills, and promoting confidence, empowerment, and engagement within a value-based payment environment. However, in the absence of a formal outcome evaluation of WIO, MLTC administrators expressed uncertainty about the measurable accomplishments of WIO beyond simple training numbers, and difficulty in discerning which aspects of training programs worked and how. According to MLTC administrators, difficulties in discerning program accomplishments may be attributed to a lack of standardized data collection efforts, as well as performance incentives emphasizing client health outcomes rather than aides’ application of preventative strategies taught in training programs. Managed Long-Term Care administrators questioned whether the WIO could meet the dual goals of achieving value and facilitating retention of aides. Agencies hypothesized that training programs improved care quality and facilitated more trusting client-aide relationships, which could facilitate aide retention and improve client performance on value-based payment measures. However, agencies lacked data to support their hypothesis. Finally, WIO investments were time-limited and stakeholders questioned how, and in what form, to continue the programs. Aides and agency administrators also expressed frustration that skills acquired through training programs were not directly tied to career advancement opportunities.