Abstract
This article describes lessons learned about implementing evaluations in hospital settings. In order to overcome the methodological dilemmas inherent in this environment, we used a practical participatory evaluation (P-PE) strategy to engage as many stakeholders as possible in the process of evaluating a clinical demonstration project. Demonstration projects, in this context, push the envelope about what is known about effectiveness in novel settings, and turnover of staff and patient populations can present challenges to gathering optimal data. By using P-PE, we built capacity in the environment while expanding possibilities for data collection. Suggestions are made based on our experience.
There are inherent methodological dilemmas in the evaluation of clinical demonstration projects in hospital settings. This article discusses a case using practical participatory evaluation (P-PE) to assess the implementation of a single program. We present a case study and lessons learned, along with suggestions for evaluators who encounter similar situations.
Clinical demonstration projects are small pilot programs designed to test the feasibility and effectiveness of novel interventions in clinical practice settings, such as a hospital. These projects typically involve a single unit or small group of units, and are driven by a “project champion” in the clinical unit and a research coordinator who belongs to a research team and may be either a staff evaluator or a research partner, may or may not work in the same hospital, and generally is not a part of the clinical unit. Clinical demonstration projects are used when there is reason to believe that an intervention may be appropriate or effective, but there are no data related to feasibility or implementation. Projects of this type are often implemented in locations where difficult populations or unique conditions prevent more traditional types of evidence-based paradigms; varying lengths of exposure to interventions, confounds regarding outcome measurement, and staff turnover in the hospital units are just a few of the issues that may arise in evaluation of these programs. The goal of clinical demonstration projects is to determine if a model of intervention can be translated into real-life clinical application feasibly with minimal burden added to staff workloads, and whether patient outcomes indicate a positive impact of the intervention. Evaluations of clinical demonstration projects may be the deciding factor in whether or not a promising intervention is disseminated more widely or discontinued. These evaluations are generally commissioned by hospital administration, whose primary interest lies in feasibility and patient benefit, usually measured according to benchmarks for a set range of health outcomes.
Hospital settings, while highly structured, may provide unique challenges to evaluation of novel interventions (Ovretveit, 1997; Smith, Preston, Buchanan, & Jordan, 1997). While guidelines for quality evaluations abound (see e.g., Agency for Healthcare Research and Quality [AHRQ], 2000; Robert Wood Johnson Foundation Evaluation [RWJF], 2009), defining the parameters of what information in the environment is useful and what is not can prove to be a moving target. Issues like staff turnover, shifting priorities, control of programming, and “floating audiences” (Kiernan, 2002; Smith et al., 1997; Themessl-Humer & Grutsch, 2003) may be viewed either as substantial obstacles or as opportunities to develop creative and flexible evaluation plans in partnership with stakeholders.
Staff turnover in a busy hospital occurs at three levels: normal expected turnover of staff, rotations of staff such as resident physicians and nursing students who stay for short, defined periods of time, and float pool staff supplementation. While a program may be instituted with a particular group of staff on a unit, often only a few individuals will still be in place beyond a few weeks or months. This rotation of people—normal in this setting—makes continuity a challenge. Embedding a new program and building capacity for a culture of evaluation become ongoing and continuously changing opportunities. For this reason, the identification of champions who can facilitate orientation and engagement of new staff in an ongoing manner is critical. In our case study program, we had a core team of dedicated and experienced practice leaders, but experienced an influx and loss of staff support at all three of these levels over the course of only a few short months.
Related to the staffing issue, shifting priorities within a unit can be problematic to evaluation partnerships. In our case, for example, staffing levels on the unit had to remain at certain proscribed levels; when team members were out, or additional staff were needed for a patient crisis, priorities shifted, with patient care and standard practice naturally rising to the top, and activities like team meetings or group activities pushed to the bottom. While no one would argue with such a reprioritization, any busy week or high priority patient could derail an evaluation plan and bring an intervention to a halt.
Our evaluation quickly found that no matter how highly valued or excellent the patient outcomes, group activities were always prioritized at the bottom of the unit functioning needs. When well-staffed and lacking high-need patients, unit staff were enthusiastic about activities that aimed to improve patient and staff well-being and teach skills for long-term success; however, when understaffed or overwhelmed with required staff tasks, it was difficult for leadership to accommodate such activities in the workflow. Without such accommodations, extra activities (e.g., those things added on top of normal workload) got quickly dropped. We found that although unit staff may have seen value in engaging in planned activities, as a larger part of their patient-focused care plans, control of programming was in the hands of unit leaders, who were tasked with focusing primarily on the safe functioning of the unit—a goal that always superseded all other activities.
In addition to challenges created by staff turnover, patient turnover creates a floating audience dilemma. Kiernan (2002) describes floating audiences as a population that has “markedly erratic” or widely variant attendance, leading to inconsistent exposure to materials or experiences within a program. Within a hospital setting, confounds such as inpatient length of stay and varying discharge expectations mean that any program intended to impact such a population will, by nature, face the issue of a floating audience. While standardized measures may be used to document participant outcomes, such as measures of anxiety, physical performance, or attendance and attrition, varying dosage or program exposure rates confound standard assessments. Faced with this type of dilemma, the evaluator is faced with several options: (1) seek to break the program into components or segments of a population, where more even exposure can be observed or (2) seek alternate meaningful observations, such as implementation feasibility, opportunities, and barriers to data collection and process evaluation.
The general process of evaluation, long established in the literature, is summed up by Kundin (2011): Establish criteria of merit, construct standards on how to measure performance, measure performance and compare this with standards, and synthesize and integrate data into a judgment of merit or worth (p. 352). However, while this might seem like a straightforward endeavor in a medical setting, clinical demonstration projects push the envelope of established experience and knowledge, leading to situations where there may be limited standardized information against which to compare new data. Additionally, with floating audiences and other methodological obstacles, the practice of designing an evaluation for such programs becomes what Kundin describes as “an integrated act of perceiving and valuing” (p. 352). The determination of what should be valued and by whom lies beyond the decision of the evaluator alone, and tension between accountability pressure and development support guides the process (Ovretveit, 1997; Themessl-Humer & Grutsch, 2003, p. 96).
Practical Participatory Evaluation and Engagement of the Environment
There are dozens of models and theories of evaluation, and equally as many points of view as to which one is best. There is not a single path to the answers that our stakeholders need or want, but rather a range of options, leaving us to choose the model that is “most right” for our methodological and contextual needs. The use of a participatory approach to evaluation in a hospital setting is one such choice, and provides an opportunity to explore a range of meaningful observations that are alternatives to those outcomes standardly defined in hospital settings. Cousins and Whitmore (1998, p. 88) describe a range of participatory evaluation styles, including P-PE. They explain that P-PE contains “the implicit assumption that evaluation is geared toward program, policy, or organizational decision making … that stakeholder participation in evaluation will enhance evaluation relevance, ownership, and thus utilization” (Zukoski & Luluqisen, 2011). This model of evaluation has the practical goal of programmatic decision making, group problem solving, and balanced partnership between the evaluator and stakeholders (Cousins & Whitmore, 1998; Smits & Champagne, 2008; Zukoski & Luluqisen, 2011). Growing out of the work of Stufflebeam and Wholey, P-PE is oriented toward information needed for decision making and the people who will make use of it in that context. While some approaches to evaluation focus on methods or on valuing, P-PE is a “use” oriented model, grounded in accountability and control, but focused on the need for actionable information and the context in which it is meant to be used (Alkin, 2004, figure 2.1, p. 13; King, 2007; Smits & Champagne, 2008). P-PE recognizes that engagement with groups of stakeholders is embedded within a larger organizational context, and that for continued possibility of utilization, evaluation should fit within organizational development. This means that there must be an explicit collaboration between professional evaluators and practice-based staff. Alkin (2004, p. 50) asserts that “if we care about utilization, then the way to get utilization is to get buy-in. The way to get buy-in is to have program personnel participating in the evaluation”. Cousins (2004) further explains that it is “the conditions and factors associated with organizational context and cultures that is key to explaining the extent to which (a program) is effective in leading to desirable practical consequences” (p. 327). Put simply, to capture the essence of the context and cultural factors that impact on the formation of an intervention, a partnership with program staff in all stages of evaluation is the best approach for both active engagement in the activities of the program and rich opportunities for meaningful exploration of the experience. A key feature of this approach is the training of organizational (practice) staff in the technical skills and considerations of evaluation, so that capacity is built for future/ongoing evaluation of the program. In this model, “process is as important as the final products and outcomes” (Zukoski & Luluqisen, 2011, p. 2).
Case Study
According to the Encyclopedia of Research Design (Salkind, 2010), a case study is an in-depth inquiry into a bounded program, for the purpose of exploring a phenomena in its context, with the opportunity to “shed light on the particularity of a phenomenon or process while opening up avenues of understanding about the entity or entities involved in the process” (Putney, 2010). We approached the process of implementation and evaluation of an inpatient Tai Chi exercise clinical demonstration program as a descriptive case study, in order to provide a context for the evaluation challenges presented in this article, and to illustrate some of the experiences and lessons learned.
Study Site and Context
This case study was based in a locked, inpatient mental health unit (MHU) in a large tertiary referral hospital. This unit provides a therapeutic, safe milieu for mental health patients who have exhibited substantial mental distress or psychosis, or bizarre or dangerous behavior to themselves or others. The majority of residents on the 20-bed unit are male, physically ambulatory, and range in age from early 20s to over 80 years. The average length of stay is 10 days, although this may range from a 48 hr hold to several months, in extreme cases. Patients are discharged when they are deemed to have improved coping skills, decreased agitation, and decreased dangerousness to themselves and others. The unit is staffed by nurses and nursing assistants around the clock, with social workers, physicians, and other professional staff usually present during normal business hours and available as needed on call.
In a larger contextual sense, the MHU is one unit of a hub hospital, connected to a network of sites from which patients are referred for treatment. As with most hospitals, there are both micro (unit specific) and macro (hospital or system wide) target goals and concerns that are monitored in an ongoing fashion. Although clinical demonstration projects are pilots of innovative ideas, there is a constant pressure for rapid dissemination of programs or processes that may hold potential for success in meeting patient well-being targets.
Method
Team members included the authors, the Tai Chi instructor, the rehabilitation nurse for the unit (who served as both the program and the evaluation champion), two staff nurses, two nursing assistants, and two social workers. The data for this descriptive case study were collected primarily through interviews with each of these individuals and observations, along with process notes related to the team experience. The authors served as key members of the intervention team, and as such, have insight into the unfolding of events. Other team members were approached about participation in this article, but declined.
All members of the team except the first author were interviewed by either the first author or the nurse champion. The first author kept process and bracketing notes that were shared with the nurse champion. We were not allowed to record interviews, for the protection of both patients and staff, so interview data and observations were in note form. As the interviews occurred in the middle of the implementation phase, and were focused on the process of both evaluation and implementation of the program, there was ongoing responsiveness to themes that emerged. Notes from team meetings and program data were reviewed and supplemented by additional discussions with practice-based staff. Meeting notes were taken by the first author. Program data were captured by the nurse champion. For evaluation purposes, the authors and nurse champion all reviewed available information separately, looking for themes and lessons learned. These were discussed and agreed upon. Members of the team met every other week to discuss and problem solve issues.
Process Narrative
Unit staff of the MHU approached program implementation and evaluation specialists in the hospital for assistance implementing a Tai Chi exercise program for their patients. Tai Chi, used in this context, is a gentle balance and stretching exercise that may promote better physiological and psychological well-being. There is strong evidence in the scientific literature supporting the link between exercise and stress reduction, and there is a growing body of evidence that Tai Chi may be one specific practice that is linked to improved physiological outcomes. It does not require any equipment, is low impact and nonaerobic, and can be practiced at any time, anywhere. Unit staff wanted to demonstrate not only improved outcomes for patients—specifically for anxiety reduction and sleep improvement, but also show hospital administration that the program could be maintained long term at little cost, benefiting stakeholders at individual, group, and organizational levels. A core group of unit staff agreed to partner with the evaluation staff. Initial meetings of the group centered around creating a partnership and establishing respect for an egalitarian sharing of expertise on the team. As described previously, these individuals included a number of nurses, several nursing assistants and patient care staff, unit social workers, the Tai Chi instructor, a program evaluation specialist, and a well-known physician researcher.
After some initial discussion, it was clear that this would be a formative evaluation, with a focus on feasibility of the Tai Chi program within the organizational context of the unit. For environments like MHUs, interventions become part of the therapeutic milieu. The theoretical construct of the milieu incorporates a range of interventions that are intended to work together to support behavioral change and mental health improvements in the patient (including psychopharmacological treatment, individual and group therapy, social activities, and structured expectations). It is therefore impossible to be able to establish direct causality of any single intervention within that environment, as the number of confounding variables is beyond any statistical control. However, it was hypothesized that the addition of a promising intervention that is safe, cost-effective (requiring no equipment or supplies), and safely replicable by patients in their home setting after release, would be a reasonable and viable option for the hospital, if a sustainable model could be implemented. There was also a need to remove stigma and build capacity for evaluation among the unit staff. With these goals in mind, a P-PE approach was deemed best, as that would empower the unit staff to share their rich expertise of the contextual experience, as well as work together with the broadest range of expertise to problem solve along the process of implementation. By using P-PE, there would be access to both practical consequences in real time and technical knowledge to examine conditions and cultural context.
Two primary goals were agreed upon by all members of the team: (1) determination as to what level of data could be collected about the program without imposing undue burden on the nursing care staff and (2) whether staff could be effectively trained to conduct the Tai Chi sessions without the trainer present, in order to promote sustainability of the program. These goals focused on assessing the feasibility of both sustainability of the program itself and capacity for capturing meaningful data. The team observed that there were some patterns to data collection in the unit already, and that assessment of the new program might be facilitated by using familiar routines, for both staff and patients. For example, one staff nurse explained the regular and consistent use of scaled questions (“On a scale of 1–10, where is your anxiety right now?”), and showed the first author how short, easily understood questionnaires or notations in medical records could capture this information with minimal burden to staff. There was group discussion about how such existing routines fit with identified evaluation needs and goals, as well as various options for data collection. The team identified a number of outcomes that were important, and worked together to create both a conceptual model of program impact (see Figure 1), and a logic model for the program (see Figure 2).
Conceptual model of individual factors to be considered for intervening with mental health inpatient.
Logic model of Tai Chi program in inpatient mental health unit.
Having identified the primary goals, the design of the conceptual model provided an opportunity to graphically illustrate both the normal route of patient trajectory, in the experience of the unit, and to identify the point in this process where the team attempted to intervene. Precipitating factors for inpatient stays and how they progressed were explicitly mapped (see Figure 1). This visual helped the team identify that, while the hospital benchmarks were important, the intervention would be best measured by examining behavioral change related specifically to the Tai Chi classes, since our intervention point was located earlier in the model than the long-term outcomes would capture, given the milieu environment. It also clarified that use of long-term outcomes would involve a range of confounding factors unrelated to the intervention, and might preclude a clear measurement of effectiveness and feasibility.
Once a conceptual model was drafted, the team was able to work on a logic model—identifying the specific inputs, throughputs, proximal, and distal outcomes for this evaluation (see Figure 2). Our nurse champion indicated that the logic model “not only ensured that all were ‘on the same page,’ but helped tie activities directly to outcomes.” The logic model also delineated specifically what we needed to measure, in terms of outcomes.
Results
This evaluation marked the first time that unit staff had been engaged as equal partners in an evaluation process, according to those interviewed. That we were able to complete this evaluation is a success unto itself. We learned a great deal about the feasibility of the program, as well as about the challenges of conducting evaluations in this setting. Several issues related to feasibility were raised in interviews. One chronic issue that was raised by all staff members interviewed was whether data from participants who had only attended once before being discharged could be meaningfully combined with data from participants who had attended 4–5 times over the course of a longer stay in the hospital. This concern reflected the ongoing reality of patient turnover and inconsistent dose/exposure—one of our “moving targets.”
Another challenge that was identified in an interview with the charge nurse concerned data protection, specifically issues of privacy for participating patients and how to collect longitudinal, repeat measures without revealing patient identities outside of the medical records. This was consistent with the authors’ concerns on the same point. This related both to feasibility issues and to tracking of distal outcomes. These challenges were met by a collaborative effort by the team, sharing patient care expertise and evaluation technical knowledge to find the best possible, minimally burdensome solutions. We were able to come up with reliable and feasible ways of extracting and deidentifying participating patient data to address the latter concern, while the former “moving target” issue remained an ongoing challenge. The team learned the importance of documenting barriers, such as staff turnover (another “moving target”), intervention orientation for new staff, and task scheduling within the unit; by examining the documentation together, the team was able to notice patterns and investigate options for overcoming these barriers. One example of this was a pattern related to running the class with assistance from an external person—in this case the Tai Chi instructor, who was not a regular staff member of that unit. Despite being able to continue in his absence (one of the feasibility items that we established through evaluation), the class was pushed down the priority list when the external person—the Tai Chi instructor—was not present, a process that seemed to indicate that the program was being perceived as an “activity” and not as a “therapeutic intervention,” making it even less of a priority when the unit became busy than it might have been had it been perceived as “therapeutic” by all unit staff. This insight helped clarify an important barrier to sustainability, and allowed the team to problem solve how this could be framed as an ongoing opportunity in this environment.
By the end of one 2-month evaluation period, 36 patients and 17 unit staff had participated in the Tai Chi program and engaged with evaluation measures. Using a 10-point Likert-type scale of anxiety, rated immediately pre and post session, 82% of patient participants rated positive change after their first class (N = 33; M 1 = 4.6, M 2 = 2.9; t = 3.09, p = .01). Data assessing the sleep patterns, PRN (as needed) medication use, and fall rates of participants on days with the classes were identified as too burdensome to the staff to reliably collect. Staff were enthusiastic about the program and expressed positive feedback about the perceived value for patients, both in verbal interviews and in follow-up written surveys. We learned, however, that enactment of either reliable program implementation or evaluation in a closed hospital unit does not appear to be correlated to perceived value or interest. Adaptation and support for workflow seems to be the key aspect in this issue. While the perceived value of both the program as a part of the therapeutic milieu and the evaluation of its feasibility was high among staff, with no systematic adaptation of workload, the Tai Chi sessions and associated evaluation tools were added to existing workload, creating a constant tension of how to get “work” done and still meet additional expectations, such as running the group and collecting needed data.
Discussion
While evaluation of clinical demonstration projects in a hospital setting presents a myriad of challenges, their expansion and dissemination rely on evaluation of some sort. By using a participatory approach to evaluation in hospital settings, evaluators engage with a range of stakeholders to identify target goals, important key events and stories, patterns in the data, and barriers and opportunities related to sustainability of programs that may be alternate meaningful observations to outcomes-related data from standardized clinical measures. Although such partnerships require a sharing of control over the process, something that can be difficult for professional evaluators and researchers, it is worth the effort (Cousins & Whitmore, 1998). The team participation of stakeholders throughout the process of evaluation opens doors to rich data that might have been missed in a more traditional model of evaluation.
In our experience, this model of evaluation also provides alternate means of assessment that can help gather meaningful and useful data, even in the face of “moving target” obstacles, such as floating audiences, staff turnover, and shifting priorities. Accepting that patient data captured through instrumentation is sometimes impossible to reliably obtain, and standards against which to compare demonstration project outcomes may not yet exist, evaluation teams may acceptably turn to other types of information that will help gain some insight into the system, such as implementation feasibility, barriers to data collection, and opportunities for partnership.
In our evaluation, we found that the trust built among team members led to opportunities to explore the perceived burden and value of different types of data to practice-based staff. The relationships helped to mitigate anxiety about progress when team meetings were missed, Tai Chi groups were cancelled, or data collection was inadvertently skipped in the rush to complete tasks on the unit. Instead of viewing this as an obstacle to evaluation, we viewed these occurrences as data that informed our understanding of the context and culture—specifically what and when burden became substantial for unit staff. By opening dialogue about perceptions, challenges, and needs of both staff and evaluation requirements, we were able to identify ways in which evaluation data were useful to unit staff, and remove the mystery about data collection and analysis. Involvement in the process of problem identification, program design, evaluation planning, and data collection removed previously associated evaluation anxiety on the part of the practice-based staff. Despite the previously mentioned barriers, unit staff who participated in this evaluation began to incorporate an evaluation culture into their workflow—asking more outcomes-based and process-oriented questions, and looking at their data in new ways; one of the nurse leaders even initiated a deeper study into a related topic. This change not only opened opportunities for unit staff to be more involved in outcomes measurement for the larger organization (i.e., the hospital), but laid the groundwork for future evaluation partnerships on the unit. With a solid partnership in place, the likelihood of reliable and valid data collection using more traditional instrumentation methodologies is greater, as evaluators and stakeholders have invested time and energy into understanding the perceptions and values related to the work and needs for evaluation of outcomes for various audiences.
Summary
Feasibility is one of the most fundamental aspects of evaluation of clinical demonstration projects in hospital settings. Whether or not a program should be expanded, rolled out across multiple sites or throughout a facility, adapted for further pilot testing, or completely discontinued hinges on the ability to gather some amount of meaningful data upon which to base such a decision. Issues of effectiveness become secondary issues to the core problem of whether or not a program can even be done in the context of the environment: whether outcomes or participation data can reliably be collected at all in the context of a “moving target” enrollment of hospital discharges, whether staff can be assigned and trained in the context of a “moving target” turnover rate on any given unit, whether patients will participate, whether supervisors will endorse the program as a part of the workflow, whether additional burden is deemed acceptable, and so on.
We used a P-PE approach to partner with stakeholders on a locked inpatient MHU to explore the feasibility of a Tai Chi intervention clinical demonstration program. During the course of that experience, we faced a range of challenges that we believe to be unique to hospital inpatient settings. As a result of these experiences, we suggest the following lessons learned: Identify at least one champion. Participatory models depend on the equal ownership of the evaluation by both evaluators and participants. In hospital settings, it can be expected that even the most enthusiastic program staff can drift away from both the program and its evaluation in the course of maintaining their normal frame of duties. As previously described, reprioritization of unit needs or rotation of unit staff can bring even the best evaluation plan to a halt. Having a champion in the unit will help maintain a focus both on the program and the evaluation, bolster ongoing enthusiasm, and make sure that any halts are temporary. Additionally, this person serves as a gatekeeper to staff who may not have been as involved in the process, opening a door to communication with other staff that they work with for feedback, as well as engaging and orienting new staff as they rotate onto the unit. Get buy-in to the process. While this may seem to be an automatic assumption when a team endeavors to use a P-PE model, it is anything but a straightforward endeavor. While a champion may certainly bolster enthusiasm and encourage engagement among staff for a particular process, there are a range of things that need the attention of staff members on an inpatient floor of a hospital, a fact that holds true for all staff, whether nurses, aides, social workers, or other types of professionals. It can be a challenge to convince some members of the staff of the value of a program evaluation, for reasons that may be as simple as workload or as complicated as resistance to change. If workload accommodation for the evaluation is not incorporated into the workings of the unit, there is little hope for sustainability, and staff can be expected to react to added work as anyone might expect—with resistance. Even in the face of great enthusiasm for a program and clear understanding of the need for evaluation, when there is added burden to the workload of any staff members, this will disintegrate the buy-in of the staff. Keep open lines of communication. P-PE involves a team approach to reflective, ongoing discussion about the process of evaluation. It is easy, however, for busy hospital-based teams to be willing to turn over evaluation to evaluation personnel once a program is in the stages of implementation. This does not, however, fit the P-PE model, and a discontinuation of that process not only closes off the possibility of developing partnership, but also removes the access to the rich data that ongoing feedback, reflection, and discussion may bring. Evaluators must remember that the staff of the hospital unit are the experts on their particular situation and on the strengths, weaknesses, opportunities, and threats to meaningful implementation of demonstration projects. By keeping open lines of communication among as many staff as possible, whether actively involved in the evaluation team or not, valuable insight into the context can be gleaned, and provide material for further team reflection and consideration. Use the conceptual model and logic model development to think outside of the box for meaningful outcomes. Most practice-based staff, whatever their professional orientation, will not have much experience with these types of models. It has been our experience, however, that once an attempt is initiated to create a visual representation of the program process, the creation of these kinds of aids can prompt creative thinking about opportunities, goals, intended outcomes, and possible unintended or long-term outcomes, as well. Additionally, by considering how pieces of the larger puzzle are connected to one another and ordered temporally, opportunities for previously unnoticed data collection may arise. This exercise can also give the participating staff a more macro view of their program and how it is integrated into the larger picture of the hospital goals, something that is easily overlooked in the day-to-day operation of an inpatient unit. By sharing this perspective with practice-based staff partners, evaluators can bolster the macro understanding of need for evaluation and encourage the continuing development of an evaluation culture. Plant the seeds for capacity building for future evaluation. Staff teams that work in inpatient hospital settings are not often used to evaluation personnel engaging them as equal partners in consideration of programmatic changes. Hospital requirements usually necessitate a top–down approach to program evaluation, involving chart reviews, data collection, and observations by outsiders to the unit. Opportunities for meaningful team building to reflect on capacity for change and the process of implementation tend to be rarer in this environment than in other types of environments. Such opportunities can provide an invaluable chance to plant the seeds for future collaborations—empowering staff and broadening their understanding of the value of evaluation for their jobs and their patients.
Footnotes
Acknowledgments
The opinions and assertions contained in this article are the views of the authors and are not to be construed as official or as reflecting the views of the Veterans Administration. The authors gratefully acknowledge Karen Durniat-Suehrstadt for her contribution to this paper.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) declared the following support for the research, authorship, and/or publication of this article: The authors would like to acknowledge the support from VISN 11 for a clinical demonstration project, the VA Ann Arbor Health Care System Geriatric Research, Education and Clinical Center (GRECC), and the Office of Research and Development and Rehabilitation Research and Development Service of the Department of Veterans Affairs. In addition, the authors acknowledge support from National Institute on Aging (NIA) grant AG024824 (University of Michigan Claude D. Pepper Older Americans Independence Center) and the Dorothy and Herman Miller Fund for Mobility Research in Older Adults. Dr. Alexander is also a recipient of the K24 Mid-Career Investigator Award in Patient-Oriented Research AG109675 from NIA.