Contextualizing the Positive Effects of the Well Elderly 2 Trial: A Response to Schelly and Ohl (2019)

Abstract

Importance: A recent reanalysis of data from the Well Elderly (WE) 2 study purportedly indicated that the intervention did not achieve clinically meaningful or statistically significant effects; this article addresses these criticisms.

Objective: To contextualize the WE 2 study as targeting a nonclinical population and demonstrate that the intervention produced substantively important, statistically significant effects.

Design: Secondary analysis of WE 2 intervention-based pre–post change scores.

Setting: The original trial occurred primarily in senior centers and senior housing facilities in greater Los Angeles.

Participants: Independent-living older adults (N = 324) who were assessed before and after intervention.

Intervention: The WE intervention, a version of the Lifestyle Redesign® (LR) approach, was administered by occupational therapists over 6 mo by means of group and individual sessions.

Outcomes and Measures: The 36-item Short Form Health Survey, the Life Satisfaction Index–Z, and the Center for Epidemiologic Studies Depression Scale.

Results: The WE intervention was associated with statistically significant improvement on 10 of 12 outcome variables that were examined.

Conclusions and Relevance: Because the WE intervention was hypothesized to reduce age-related decline and followed a population-oriented approach, the expectation that average results would be clinically meaningful was inappropriate. The intervention produced positive effects across a wide array of outcome domains. In settings in which clinical meaningfulness is an appropriate index of intervention outcomes, evidence suggests that LR produces effects that are clinically meaningful. As an evidence-based intervention, LR should be considered useful both in population-oriented contexts and in addressing discrete health conditions.

What This Article Adds: Valid analyses demonstrate that the positive experimental effects of the WE 2 study are, in fact, genuine and cost-effective, and LR in clinically oriented contexts has produced statistically significant, clinically meaningful results. Clearly and accurately representing the evidence base of occupational therapy in prevention and chronic care is of critical importance to advance the field as a whole.

As indicated eloquently in a recent article by Schelly and Ohl (2019) in the American Journal of Occupational Therapy, the assessment of clinical meaningfulness (CM) is an important consideration in evaluating the outcomes of randomized controlled trials of occupational therapy interventions. Moving beyond tests of statistical significance, which can yield positive results for even trivial intervention-based change given a sufficiently large sample size, measures of CM document whether an intervention produces changes that meet or exceed “the smallest difference in a score that is considered to be worthwhile or important” (Hays & Woolley, 2000, p. 419) to clinicians and patients. Thus, CM facilitates clinical decision making by clarifying whether an intervention is likely to produce a perceptible change in targeted outcomes. Schelly and Ohl’s reiteration of the need to consider CM is timely because the proliferation of clinical trial research in occupational therapy necessitates adherence to currently accepted standards of rigor.

In making their case for the importance of assessing CM, Schelly and Ohl (2019) reexamined data from the University of Southern California (USC) Well Elderly (WE) 2 study, a randomized, controlled effectiveness trial (Clark et al., 2012). In this trial, 460 community-dwelling older adults (ages 60–95 yr) were randomized to a 6-mo prevention-based Lifestyle Redesign® (LR) occupational therapy condition or to a control condition. Results indicated that, relative to control participants, intervention recipients had more favorable preintervention-to-postintervention change scores on an array of outcomes (pain reduction, vitality, social function, mental health, mental component summary, life satisfaction, and depression; all ps ≤ .04, mean Cohen’s d = 0.20). In reanalyzing the data, Schelly and Ohl applied various criteria for assessing CM, which uniformly indicated a lack of clinically meaningful effects. Moreover, they suggested that experimental artifacts, especially regression to the mean in conjunction with between-groups differences in baseline scores, created the false impression of a treatment effect and that in actuality no evidence for a genuine effect was obtained.

Although Schelly and Ohl’s (2019) emphasis on CM is meritorious, it is unfortunate that they chose the WE 2 trial to illustrate their point. In this article, we demonstrate the following points:

Schelly and Ohl’s application of CM criteria was inappropriate, given the context, goals, and hypotheses of the trial.

Their negative conclusions about the effectiveness of the WE intervention were largely fueled by questionable methodological decisions and procedures.

Their contention of an illusory intervention effect is refuted by new analyses that use a wider range of available information pertaining to the intervention’s outcomes.

Apart from the WE intervention, other LR interventions, when administered to a variety of different treatment populations in clinically relevant contexts, have repeatedly produced effects that are clinically meaningful.

We then discuss the role of the WE 2 trial in the context of the ongoing LR project and conclude by outlining implications for occupational therapy research and practice.

Inappropriateness of Applying Clinical Meaningfulness Indicators to the Well Elderly 2 Study

Several authors have cautioned that it is necessary to consider the full treatment context when using CM measures (e.g., Bernstein & Mauger, 2016) and that in some cases, small differences in outcomes can be substantively important (Hays & Woolley, 2000). In line with this caveat, we view the routine application of CM techniques to the WE 2 intervention as problematic for multiple interconnected reasons.

Population-Oriented Versus Targeted Interventions

First, Schelly and Ohl (2019) overlooked that the underlying rationale for the WE 2 intervention centers on the importance of achieving population-based aggregated effects. In contrast to most occupational therapy interventions, which explicitly target people with a specific health condition, population-oriented interventions are potentially applied to all people irrespective of their health status, and they can be extremely effective even when the group-based mean level of change is small (Bellinger, 2007; Jayadevappa et al., 2017). This scenario is related to what has been termed the prevention paradox, in which an intervention offers great benefit to a population despite providing only a small amount of benefit to any given individual (Rose, 1985). Accordingly, widely applied interventions that enhance health-related quality of life to a nonclinically significant degree are deemed worthwhile if they are inexpensive enough (Hays & Woolley, 2000).

As a hypothetical example, an intervention that increased shoulder range of motion (ROM) by 3° but cost only $1.00 and could be administered in only 1 min would likely be beneficial at the population level despite producing an effect that would not meet the criteria for CM. The intervention would apply to a range of people, including those with full ROM, for whom it might forestall the development of limitations; those with modest ROM deficits, for whom it might help to maintain function; and those with significant ROM deficits, for whom it might provide functional benefits. Although the benefits would be minimal at an individual level, the aggregate improvement across the population would render it cost-effective. In this regard, the WE intervention is arguably viable, despite its lack of clinically meaningful effect, because it has produced cost-effective, statistically reliable benefits for treated people in both the WE 1 and WE 2 trials (Clark et al., 2012; Hay et al., 2002).

Relevance of Clinical Meaningfulness in Preventive Interventions

In a related matter, in contrast to the vast majority of occupational therapy interventions, the WE 2 intervention is fundamentally nonclinical. Clinical interventions pertain to the treatment of disease, involve patients, or take place in a clinic (“Clinical,” 2012) and are exemplified by standard areas of occupational therapy practice such as stroke rehabilitation, traumatic brain injury, and outpatient mental health. In such therapeutic situations, people who have an explicit health-relevant problem or limitation receive services specifically designed to ameliorate the resultant performance deficits (rehabilitation) or develop skills or compensatory strategies to address limitations imposed by the condition (habilitation) to promote their occupational engagement and participation. In such contexts, the assessment of CM is typically pertinent and important.

In contrast, the essence of the WE 2 intervention is preventive and not rehabilitative or habilitative. Although preventive interventions such as WE fall squarely within the scope of occupational therapy, they are rarely implemented in contemporary practice, given constraints and barriers in reimbursement models, which typically require demonstration of the medical necessity of services (e.g., that a diagnosis exists and there is a current occupational performance deficit that can be affected by skilled services) to be reimbursed. Because the WE intervention is delivered to well, community-living older adults, its chief goal is to prevent decline. Given this relatively robust population, there is no general expectation of large or clinically meaningful health-related improvements because the treatment recipients are typically functioning at an already high level and have no specifically targeted health problem as a chief focus of treatment. Although some people may experience large gains, especially those who have below-average health at baseline, this is not a necessary condition for intervention success. Thus, Schelly and Ohl’s (2019) various applications of CM techniques were misguided because they did not align with the basic intent of the WE intervention, which follows a nonclinical, preventive model for largely healthy older adults in a community setting.

Magnitude of Anticipated Intervention Effects

A final problem with Schelly and Ohl’s (2019) analysis is their assumption that a beneficial intervention effect must consist of a clinically important level of improvement. However, this assumption is at odds with the model of change hypothesized in the WE 2 study. Given the WE intervention’s preventive thrust, we did not posit that sizable improvements were necessary to demonstrate success. Rather, the study hypothesis was that the intervention, to at least some degree, would diminish health-related declines relative to untreated older adults (Clark et al., 2012). Accordingly, various patterns of change, such as minimal positive change, no change, or even decline, could all indicate intervention success, provided that group-based average change scores are reliably more favorable for treated versus untreated older adults. For example, in an initial trial of the WE intervention (the WE 1 study), the majority of supportive outcomes involved intervention group declines that were smaller than those in the control group (Clark et al., 1997).

Schelly and Ohl’s (2019) assumption regarding the magnitude of change was also flawed, given that (as we have outlined) the hypothesized preventive effect in WE 2 does not pertain to changes that are clinically meaningful for a given patient but instead pertains to meaningfulness from a societal vantage point as a result of the robust observation of health-related decline in the rapidly expanding older adult population. Illustrating this downward trend, in a large cross-sectional survey of nearly 10,000 participants ages 65–104 yr, scores on the various 36-item Short-Form Health Survey (SF–36) scales deteriorated by approximately 1 point per year on average, with reductions accelerating with advancing age (Walters et al., 2001). Given this trend, as well as high health care treatment costs associated with older adulthood, reduced group-based decline is a viable outcome to target.

Although it is possible to follow participants over a prolonged longitudinal interval to investigate whether between-groups differences gradually achieve CM as health-related declines accelerate faster in untreated versus treated people, this was not the intent of the WE 2 study, which was limited to a 6-mo window for detecting experimentally based change. Although Schelly and Ohl (2019) attempted to discount the value of a limited change model on the basis of a lack of knowledge about how much decline to expect in the study period, the observation of highly consistent decline in old age among the population as a whole (e.g., Wilson et al., 2013), when coupled with the trial’s statistically significant experimental result, implies that the intervention did in fact lessen (or reverse, at least temporarily) health-related declines over the 6-mo study interval. This conclusion holds even if the naturally occurring rate of decline is not precisely known.

Questionable Methodological Procedures

We appreciate Schelly and Ohl’s (2019) assiduousness in applying to the WE 2 data a wide gamut of statistical techniques, including analyses involving effect size, standard deviation (SD), standard error of measurement (SEM), minimal difference, fragility of effect, and poor scores at baseline. However, many of these indices overlap considerably with each other, and the consistently negative result that they produce is merely a restatement of small obtained effect size, an issue discussed in Clark et al. (2012, pp. 778–788). Therefore, we view the lack of CM as largely expected. At the same time, however, Schelly and Ohl’s analyses are marred by a number of questionable procedures, which led them to overstate their case. Three examples follow.

Use of Incorrect Standard Deviation Values

One of the criteria for CM is whether intervention recipients, considered at the individual level, improved by at least 0.5 SD. However, the one-half SD values reported by Schelly and Ohl (2019, Table 2, p. 7) in their table of CM indicators are considerably higher than the one-half SD values that were observed in the WE 2 data. Although they did not indicate how they obtained their reported one-half SD values (as was also the case with their values for SEM, which were used in other CM analyses), distribution-based CM measures are best derived from the immediate sample that is being studied (Guyatt et al., 2002; Rai et al., 2015). WE 2 trial participants were generally healthy older adults, which likely led to smaller SDs than have typically been reported for older adult samples that include both healthy and unhealthy people (e.g., Walters et al., 2001). Although we are not suggesting that most WE 2 participants achieved clinically meaningful effects, Schelly and Ohl’s overestimation of SD led to CM benchmarks that were inappropriately stringent even at the participant level, thereby minimizing the number of intervention recipients classified as “better” when using the one-half SD criterion.

Omission of Extreme Improvers

To bolster their argument for the lack of an intervention effect, in selected analyses Schelly and Ohl (2019) omitted “extreme improvers”—people with high improvement on at least six of the seven analyzed outcome scales. No reference was cited for this apparently unprecedented procedure, which was guaranteed to reduce the experimental effect.

In this context, it is noteworthy that the potential for some participants to achieve across-the-board improvements is a theoretically anticipated strength of the WE intervention, because of its holistic thrust and emphasis on the power of healthy occupation to simultaneously improve multiple health-related outcomes. Although the extreme improvers tended to have low scores at baseline with an increased possibility of regression to the mean (Schelly & Ohl, 2019, pp. 8–9), this does not imply that their change scores were invalid and should be eliminated. In analyzing the WE 2 data, Clark et al. (2012) previously applied analysis of covariance to adjust for baseline scores and obtained results comparable to those Schelly and Ohl reported using an independent-samples t test. By retaining extreme improvers in the analyses, Clark et al. accounted for the theoretical expectation that those lowest at baseline have the greatest potential to genuinely benefit from the intervention, apart from any artifactual gains associated with regression to the mean. This expectation is supported by the fact that among the bottom 10% of baseline scorers samplewide, the treatment group improved more than the control group on 11 of 12 outcomes, with the between-groups difference achieving statistical significance for the SF–36 Vitality scale and Mental Component Summary (p < .05) despite a very small sample size (data not shown).

We should also note that, to achieve a fair test using Schelly and Ohl’s (2019) elimination procedure, a corresponding number of the poorest intervention responders should also have been considered for removal, and appropriate robust statistical measures should have been applied (Wilcox, 2011), because the scores of intervention recipients with abnormally high baseline values are expected to regress to the mean in a downward direction. In analyzing trial data, the selective elimination of improvers from an experimental or control condition is likely to induce significant bias, thereby invalidating any conclusions regarding intervention effects.

Misapplication of Fragility Analysis to Continuous Outcome Data

The fragility index is defined as the minimum number of positive intervention responders that, if transferred to the group of nonresponders, would overturn a statistically significant result on a binary (dichotomous) outcome (Walsh et al., 2014). As stressed repeatedly in the literature, this index does not apply to continuous outcomes (e.g., Lobo, 2019) such as the measures included in the WE 2 trial. When the fragility index is used in its proper context, the successful responders who are transferred reflect average successful responders on a dichotomous outcome (i.e., those categorized as improved). In contrast, eliminating the very highest intervention responders on a continuous measure may lead to artificially high estimates of fragility because such responders make up an extremely select subset who scored far above the average of other successful responders. Schelly and Ohl’s (2019) fragility estimates are therefore difficult to interpret because they are based on a nonvalidated extension of the procedure to continuous outcomes.

Finally, although Schelly and Ohl (2019) were critical of the reliance on significance testing, it is noteworthy that the widely accepted, dichotomous version of the fragility index is strongly correlated with p values or transformed p values (e.g., r = −.94 in Carter et al., 2017).

New Analyses That Demonstrate a Positive Intervention Effect for Lifestyle Redesign

In critiquing the WE intervention, Schelly and Ohl (2019) ignored various strands of favorable evidence that support this treatment approach. What is especially puzzling is that Schelly and Ohl did not address Clark et al.’s (2012) reported pre–post change score analysis for control participants who crossed over to receive the intervention after the trial’s experimental phase. This distinct group of intervention recipients exhibited a degree of benefit similar to what was found for intervention recipients in the study’s experimental phase. This mirroring in outcome strongly supports the intervention’s effectiveness.

Table 1 presents the pre–post change score outcomes for the full set of WE 2 participants who received the intervention (i.e., initial experimental treatment recipients plus crossed-over control participants). For each SF–36 outcome, as well for the Life Satisfaction Index–Z (LSI–Z; Wood et al., 1969) and Center for Epidemiological Studies Depression Scale (CES–D; Radloff, 1977), values include the sample size; pre, post, and change-based mean and SD or SEM; effect size; and two-tailed p value. We note that, although one-tailed statistical tests were used in the original study and were justified on the basis of the known direction of effect based on prior research, with no evidence of adverse outcomes, reduced potential for Type 2 error, and existing theory supporting a unidirectional research hypothesis, we present two-tailed p values for the sake of comparability with the analyses presented by Schelly and Ohl (2019).

Table 1.

Analysis of Outcome Change Scores for All Participants (Intervention Group Plus Crossed-Over Control Group)

Measure	N	Pre, M (SD)	Post, M (SD)	Change		2-tailed p a
Measure	N	Pre, M (SD)	Post, M (SD)	M (SEM)	Cohen’s d	2-tailed p a
SF–36
Physical Functioning	324	38.73 (12.05)	39.94 (12.41)	1.21 (0.48)	0.10	.01
Role Physical	324	40.56 (11.14)	41.43 (10.75)	0.87 (0.57)	0.08	.13
Bodily Pain	324	43.45 (11.72)	45.14 (11.14)	1.69 (0.52)	0.14	.001
General Health	324	45.17 (10.09)	46.12 (9.58)	0.95 (0.45)	0.09	.03
Vitality	323	50.09 (10.50)	51.46 (9.49)	1.37 (0.49)	0.13	.01
Social Functioning	324	44.81 (11.61)	45.71 (10.70)	0.89 (0.55)	0.08	.11
Role Emotional	324	39.96 (13.68)	41.25 (12.35)	1.30 (0.65)	0.09	.05
Mental Health	323	47.57 (11.88)	49.28 (10.36)	1.70 (0.56)	0.14	.002
Physical Component Summary	323	41.09 (10.13)	42.08 (10.29)	0.99 (0.40)	0.10	.01
Mental Component Summary	323	47.55 (11.98)	48.94 (10.60)	1.39 (0.55)	0.12	.01
LSI–Z	323	16.98 (5.61)	17.76 (5.29)	0.78 (0.23)	0.14	.001
CES–D	322	13.66 (10.91)	12.18 (9.47)	−1.48 (0.48)	−0.14	.002

Note. CES–D = Center for Epidemiologic Studies Depression Scale; LSI–Z = Life Satisfaction Index–Z; M = mean; SD = standard deviation; SEM = standard error of the mean; SF–36 = 36-Item Short Form Health Survey.

Paired-samples t test.

In the combined sample of intervention recipients, statistically significant improvements from baseline were observed on 10 of the 12 outcome variables, using a two-tailed test. Eight of the p values were significant at or beyond the .01 level, underscoring the breadth and robustness of the WE 2 intervention’s beneficial effects. The effect size values in Table 1 are slightly lower than those reported for the between-groups experimental comparison in Clark et al. (2012). These lower values reflect the absence of a comparison group evidencing age-related decline when assessing pre–post change in a single overall intervention group. However, in the context of presumed population decline, small but statistically reliable pre–post improvements over a 6-mo period, such as those reported in Table 1, are consistent with a posited prevention effect.

If Schelly and Ohl’s (2019) version of the fragility index is applied (despite our misgivings about using it in the current context; see the “Misapplication of Fragility Analysis to Continuous Outcome Data” section), the resulting median value across the seven previously assessed outcomes (LSI–Z, CES–D, SF–36 Bodily Pain, Vitality, Social Functioning, Mental Health, and Mental Component Summary) is equal to 8. This value is noticeably larger than the corresponding median of 1 obtained by Schelly and Ohl, and it surpasses the cutoff score (n < 5) that they posit would exemplify a fragile trial result (p. 4). Although there is no precedent for applying Schelly and Ohl’s fragility index (or indeed any version of the fragility index) to continuous variables, our obtained values indicate that Schelly and Ohl, according to their own criteria, are likely to have underestimated the robustness of the WE 2 intervention’s positive outcomes.

Additional evidence stemming from the WE 2 dataset also points to intervention effectiveness (Juang et al., 2018; Pisca, 2015; Wilcox et al., 2013). For example, Juang et al. (2018) documented theoretically predicted pathways linking the intervention to reduced depression via increased activity, a crucial intervention target (effect sizes = 0.44 and 0.23 for activity frequency and perception of activity significance, respectively).

Schelly and Ohl (2019) speculated that the WE intervention may have had a placebo effect on older adults with low baseline scores. Although use of self-report outcomes was a study limitation, four considerations argue against major or undesirable placebo effects:

The SF–36 and other study outcome measures have a high degree of validation and are routinely applied successfully in trials involving repeated measures.

In the WE 1 study, a social activity control group (also susceptible to a placebo effect) evidenced poorer outcomes than the intervention group, as well as outcomes that were either comparable to or worse than those of the study’s untreated control group (Clark et al., 1997).

Other varieties of LR intervention, when applied to ameliorate health-relevant symptoms in clinical populations, have produced beneficial changes in physiological parameters, which are less susceptible to placebo effects (Pyatak et al., 2018; Schepens Niemiec et al., 2018).

Consistent with the practical intent of the WE 2 study, placebo effects may be inherent to an intervention and are deemed to be desirable insofar as they serve as an added mechanism to heighten positive change in real-world implementations (Bystad et al., 2015).

In addition, Schelly and Ohl (2019) proposed that individualized sessions may have generated positive effects via participants’ engagement in traditional, nonpreventive occupational therapy procedures. However, the purpose of individual sessions in the WE intervention was to apply principles of LR on a personalized basis to improve health-related outcomes, and other forms of occupational therapy were used only in highly limited circumstances; fidelity to these procedures was monitored throughout the implementation of WE 2 via weekly debriefing sessions (Mandel et al., 1999).

Clinically Meaningful Effects of Lifestyle Redesign

It is critical to differentiate between the WE intervention and LR. Although Schelly and Ohl (2019, p. 2) seem to equate the WE intervention with LR, the LR framework is much broader than WE, is highly adaptable to different treatment populations, and has been successfully applied in a variety of clinical contexts. Key defining characteristics of LR include administration or oversight by an occupational therapist; strategic focus on daily activities, habits, and routines; education in occupational self-analysis to promote awareness of the relationship between one’s daily activities and health; and use of therapist–client collaboration to support client autonomy and intrinsic motivation. In many applications of the LR intervention framework, CM measures constitute an appropriate consideration in evaluating treatment effects insofar as, unlike the WE 2 study, the primary treatment goal is to remedy adverse consequences of a discrete health problem in a clinical population. Table 2 summarizes four recent studies of clinically oriented LR interventions, each of which produced clinically significant effects in treating people who had a specified health-related problem.

Table 2.

Evidence of Clinically Meaningful Effects for Nonpreventive Lifestyle Redesign Interventions

Study	Population	Design	Sample Size	Outcome Variables: CM-Relevant Result
Pyatak et al. (2018)	Young adults with uncontrolled diabetes	RCT	Tx, n = 41; Ctl, n = 40	• HbA1c change: ES = 0.50• Mean between-groups difference in HbA1c = 0.9%, >0.5% (external anchor)
Schepens Niemiec et al. (2018)	Late midlife Latino primary care patients with bothersome symptoms	Pre–post change	N = 37	• Symptom reduction—MYMOP2: ESs = 1.04 and 0.92 for symptoms 1 and 2, respectively• For symptoms 1 and 2, based on 0.5 ES: improved = 69% and 68%; same = 19% and 20%; worse = 11% and 12%, respectively
Uyeshiro Simon and Collins (2017)	Clinic patients with chronic pain	Pre–post change	N = 45	• COPM: ESs = 1.46 (performance) and 1.70 (satisfaction); M increase = 2.02 (performance) and 2.78 (satisfaction), both >2.0 (external anchor)
Pyatak et al. (2019)	Adult primary care patients with uncontrolled diabetes	Pre–post change	Tx, n = 37	• HbA1c change: ES = 0.50• Mean decline in HbA1c = 0.9%, >0.5% (external anchor)

Note. COPM = Canadian Occupational Therapy Performance Measure; Ctl = control group; ES = effect size; HbA1c = hemoglobin A1c; M = mean; MYMOP2 = Measure Yourself Medical Outcome Profile; RCT = randomized controlled trial; Tx = treatment group.

Role of the Well Elderly 2 Trial in the Wider Lifestyle Redesign Research Program

It is important to situate the WE 2 study within the significantly more extensive LR project that has been ongoing for more than two decades. As an effectiveness trial, the WE 2 study was performed in real-life settings that were fraught with multiple sources of variability such as a highly heterogeneous sample, significant variation in the type of study site and degree of on-site agency buy-in, inclusion of people who did not attend any intervention sessions, and use of a relatively short intervention period (Clark et al., 2012). Although such considerations dampen a tested intervention’s overall magnitude of effect by producing experimental error (Gitlin et al., 2003), the WE 2 study provides fertile ground for identifying useful enhancements in lifestyle-based occupational therapy.

To this end, the WE 2 dataset enables subgroup analyses that help identify the conditions—such as participant characteristics, therapist variables, or aspects of intervention delivery—under which the intervention has the most impact. For example, analyses of intervention response in the WE 2 trial have indicated that participation in five or more individual sessions was strongly associated with improvement on a number of outcomes, with Cohen’s d in one instance equal to 0.59 (Wilcox et al., 2013). This result has been used as justification for increased emphasis on individualized client–therapist contacts, which have been a hallmark of several recent successful LR interventions (e.g., Pyatak et al., 2018).

Moreover, when juxtaposed with other investigations of LR, the WE 2 study contributes important information on topics such as the relative cost-effectiveness of using different types of interveners (e.g., occupational therapists vs. supervised lay providers), optimal treatment dosage, cultural tailoring, therapist–client ethnic matching, and solutions to common implementation problems. This cumulative knowledge base has informed the development of LR extensions that have produced clinically meaningful results in nonpreventive treatment contexts (Table 2). Further exemplifying the value of this knowledge, Zilbershlag et al. (2019) developed a culturally adapted LR program to delay age-related decline and promote healthy day-to-day living among well older adults in Israel. Preliminary results for pre–post change among 52 participants reveal statistically significant positive effects for multiple quality-of-life indicators.

To date, a clear majority of trials have supported the efficacy or effectiveness of LR or interventions that substantially overlap with LR. However, primarily null results have been obtained in studies of interventions for well older adults in the United Kingdom (Mountain et al., 2017) and for low-income, ethnically diverse adults with spinal cord injury (Carlson et al., 2019). In these trials, interpretation of the results was clouded by methodological problems such as a significant lack of power, use of a relatively short intervention interval, or an undesirable delay in postintervention testing. Nonetheless, these trials have contributed to the evolving LR knowledge base by providing information about the potential limits of LR’s effectiveness in selected treatment contexts.

Implications for Occupational Therapy Practice

The practice implications of this study are as follows:

Occupational therapy LR interventions achieve positive, targeted effects in both preventive and clinical contexts.

Currently accepted techniques for assessing CM should be considered as part of a comprehensive evaluation of the results of randomized controlled trials of clinically oriented interventions that address discrete health-relevant symptoms.

Interpretive caution is warranted in drawing conclusions about the meaning of CM-related outcomes, especially in intervention contexts that are nonclinical or preventive, involve complex effectiveness trials, or are characterized by a weak overall effect in conjunction with strong subgroup effects.

Conclusion

The USC WE 2 intervention produced statistically significant and cost-effective beneficial results, despite typically having small effects on individual recipients. This positive result contributes to the growing evidence base demonstrating LR’s effectiveness in harnessing the power of daily occupation to have a positive impact on health-related outcomes in both preventive and therapeutic contexts. Recent trends in health care, including a growing need for wellness-related services and increased emphasis on team-based care, present significant opportunities for occupational therapists to contribute to prevention, population health, and chronic care management (Leland et al., 2017). Thus, accurately characterizing the effectiveness of lifestyle-based interventions such as LR is of crucial importance for the profession, and it is imperative that occupational therapy researchers report the findings of these interventions using well-validated analytic approaches that adhere to currently accepted standards of rigor, including, when appropriate, measures of CM.

Yet, it is equally important to understand instances in which the routine application of CM measures may not coincide with the purpose of a given intervention. This lack of fit occurred with the WE 2 study, which, instead of a clinical model, used a preventive, population-oriented approach to forestall health-related declines in independent-living older adults. Representing the evidence base of occupational therapy in prevention and chronic care clearly and accurately, both within the profession and to stakeholders in the wider health care community, is critically important to advance the field as a whole.

Footnotes

Acknowledgments

We gratefully acknowledge Florence Clark, for her valuable input into the conceptualization of this article; Katie Jordan, for her thoughtful review of portions of the manuscript; and Khatira Mehdiyeva and Ray Hernandez, for their assistance with the literature review. Finally, this project would not have been possible without the research participants and the entire Well Elderly 2 study team and data and safety monitoring board. This research was supported by the National Institute on Aging (Grant R01 AG021108). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health. This study is registered with (NCT00786344).

References

Bellinger

D. C.

(2007). Interpretation of small effect sizes in occupational and environmental neurotoxicology: Individual versus population risk. Neurotoxicology, 28, 245–251. https://doi.org/10.1016/j.neuro.2006.05.009

Bernstein

J. A.

, & Mauger

D. T.

(2016). The minimally clinically important difference (MCID): What difference does it make? Journal of Allergy and Clinical Immunology, 4, 689–690. https://doi.org/10.1016/j.jaip.2016.04.022

Bystad

, & Wynn

(2015). How can placebo effects best be applied in clinical practice? A narrative review. Psychology Research and Behavior Management, 8, 41–45. https://doi.org/10.2147/PRBM.S75670

Carlson

Vigen

C. L. P.

Rubayi

Blanche

E. I.

Blanchard

Atkins

, . . . Clark

(2019). Lifestyle intervention for adults with spinal cord injury: Results of the USC-RLANRC Pressure Ulcer Prevention Study. Journal of Spinal Cord Medicine, 42, 2–19. https://doi.org/10.1080/10790268.2017.1313931

Carter

R. E.

McKie

P. M.

, & Storlie

C. B.

(2017). The fragility index: A P-value in sheep’s clothing? European Heart Journal, 38, 346–348. https://doi.org/10.1093/eurheartj/ehw495

Clark

Azen

S. P.

Zemke

Jackson

Carlson

Mandel

, . . . Lipson

(1997). Occupational therapy for independent-living older adults: A randomized controlled trial. JAMA, 278, 1321–1326. https://doi.org/10.1001/jama.1997.03550160041036

Clark

Jackson

Carlson

Chou

C. P.

Cherry

B. J.

Jordan-Marsh

, . . . Azen

S. P.

(2012). Effectiveness of a lifestyle intervention in promoting the well-being of independently living older people: Results of the Well Elderly 2 randomised controlled trial. Journal of Epidemiology and Community Health, 66, 782–790. https://doi.org/10.1136/jech.2009.099754

Clinical. (2012). In Farlex Partner medical dictionary. Retrieved from https://medical-dictionary.thefreedictionary.com/clinical

Gitlin

L. N.

Belle

S. H.

Burgio

L. D.

Czaja

S. J.

Mahoney

Gallagher-Thompson

, . . . Ory

M. G.

; REACH Investigators. (2003). Effect of multicomponent interventions on caregiver burden and depression: The REACH multisite initiative at 6-month follow-up. Psychology and Aging, 18, 361–374. https://doi.org/10.1037/0882-7974.18.3.361

10.

Guyatt

Osoba

Wyrwich

, & Norman

(2002). Methods for explaining the clinical significance of health status measures. Clinical Therapeutics, 24, 5. https://doi.org/10.1016/S0149-2918(02)85088-3

11.

Hay

LaBree

Luo

Clark

Carlson

Mandel

, . . . Azen

S. P.

(2002). Cost-effectiveness of preventive occupational therapy for independent-living older adults. Journal of the American Geriatrics Society, 50, 1381–1388. https://doi.org/10.1046/j.1532-5415.2002.50359.x

12.

Hays

R. D.

, & Woolley

J. M.

(2000). The concept of clinically meaningful difference in health-related quality-of-life research: How meaningful is it? PharmacoEconomics, 18, 419–423. https://doi.org/10.2165/00019053-200018050-00001

13.

Jayadevappa

Cook

, & Chhatre

(2017). Minimal important difference to infer changes in health-related quality of life—A systematic review. Journal of Clinical Epidemiology, 89, 188–198. https://doi.org/10.1016/j.jclinepi.2017.06.009

14.

Juang

Knight

B. G.

Carlson

Schepens Niemiec

S. L.

Vigen

, & Clark

(2018). Understanding the mechanisms of change in a lifestyle intervention for older adults. Gerontologist, 58, 353–361. https://doi.org/10.1093/geront/gnw152

15.

Leland

N. E.

Fogelberg

D. J.

Halle

A. D.

, & Mroz

T. M.

(2017). Occupational therapy and management of multiple chronic conditions in the context of health care reform. American Journal of Occupational Therapy, 71, 7101090010. https://doi.org/10.5014/ajot.2017.711001

16.

Lobo

D. N.

(2019). Fragility, spin, and interpretation of randomized clinical trials. Critical Care Medicine, 47, 486–488. https://doi.org/10.1097/CCM.0000000000003604

17.

Mandel

Jackson

Zemke

Nelson

, & Clark

(1999). Lifestyle Redesign: Implementing the Well Elderly Program. Bethesda, MD: AOTA Press.

18.

Mountain

Windle

Hind

Walters

Keertharuth

Chatters

, . . . Roberts

(2017). A preventative lifestyle intervention for older adults (Lifestyle Matters): A randomised controlled trial. Age and Ageing, 46, 627–634. https://doi.org/10.1093/ageing/afx021

19.

Pisca

N. E.

(2015). Examination of the long-term psychosocial and functioning effects of a healthy living, life management behavior intervention for older adults (Doctoral dissertation). Retrieved from http://digitallibrary.usc.edu/cdm/ref/collection/p15799coll40/id/182514

20.

Pyatak

E. A.

Carandang

Vigen

C. L. P.

Blanchard

Diaz

Concha-Chavez

, . . . Peters

A. L.

(2018). Occupational therapy intervention improves glycemic control and quality of life among young adults with diabetes: The Resilient, Empowered, Active Living with Diabetes (REAL Diabetes) randomized controlled trial. Diabetes Care, 41, 696–704. https://doi.org/10.2337/dc17-1634

21.

Pyatak

King

Vigen

C. L. P.

Salazar

Diaz

Schepens Niemiec

S. L.

, . . . Shukla

(2019). Addressing diabetes in primary care: Hybrid effectiveness–implementation study of Lifestyle Redesign® occupational therapy. American Journal of Occupational Therapy, 73, 7305185020. https://doi.org/10.5014/ajot.2019.037317

22.

Radloff

L. S

. (1977). The CES–D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401. https://doi.org/10.1177/014662167700100306

23.

Rai

S. K.

Yazdany

Fortin

P. R.

, & Aviña-Zubieta

J. A.

(2015). Approaches for estimating minimal clinically important differences in systemic lupus erythematosus. Arthritis Research and Therapy, 17, 143. https://doi.org/10.1186/s13075-015-0658-6

24.

Rose

(1985). Sick individuals and sick populations. International Journal of Epidemiology, 14, 32–38. https://doi.org/10.1093/ije/14.1.32

25.

Schelly

, & Ohl

(2019). Examining clinical meaningfulness in randomized controlled trials: Revisiting the Well Elderly II. American Journal of Occupational Therapy, 73, 7301205120. https://doi.org/10.5014/ajot.2019.030874

26.

Schepens Niemiec

S. L.

Blanchard

Vigen

C. L. P.

Martínez

Guzmán

Concha

, . . . Carlson

(2018). Evaluation of ¡Vivir Mi Vida! to improve health and wellness of rural-dwelling, late middle-aged Latino adults: Results of a feasibility and pilot study of a lifestyle intervention. Primary Health Care Research and Development, 19, 448–463. https://doi.org/10.1017/S1463423617000901

27.

Uyeshiro Simon

, & Collins

C. E. R.

(2017). Lifestyle Redesign® for chronic pain management: A retrospective clinical efficacy study. American Journal of Occupational Therapy, 71, 7104190040. https://doi.org/10.5014/ajot.2017.025502

28.

Walsh

Srinathan

S. K.

McAuley

D. F.

Mrkobrada

Levine

Ribic

, . . . Devereaux

P. J.

(2014). The statistical significance of randomized controlled trial results is frequently fragile: A case for a Fragility Index. Journal of Clinical Epidemiology, 67, 622–628. https://doi.org/10.1016/j.jclinepi.2013.10.019

29.

Walters

S. J.

Munro

J. F.

, & Brazier

J. E.

(2001). Using the SF-36 with older adults: A cross-sectional community-based survey. Age and Ageing, 30, 337–343. https://doi.org/10.1093/ageing/30.4.337

30.

Wilcox

(2011). Introduction to robust estimation and hypothesis testing (3rd ed.). San Diego, CA: Academic Press.

31.

Wilcox

Carlson

Azen

, & Clark

(2013). Avoid lost discoveries, because of violations of standard assumptions, by using modern robust statistical methods. Journal of Clinical Epidemiology, 66, 319–329. https://doi.org/10.1016/j.jclinepi.2012.09.003

32.

Wilson

R. S.

Boyle

P. A.

Segawa

Begeny

C. T.

Anagnos

S. E.

, & Bennett

D. A.

(2013). The influence of cognitive decline on well-being in old age. Psychology and Aging, 28, 304–313. https://doi.org/10.1037/a0031196

33.

Wood

Wylie

M. L.

, & Sheafor

(1969). An analysis of a short self-report measure of life satisfaction: Correlation with rater judgments. Journal of Gerontology, 24, 465–469. https://doi.org/10.1093/geronj/24.4.465

34.

Zilbershlag

Maeir

Gilboa

Yosef

A. B.

, & Katz

(2019). Adaptation, development, and implementation of the Lifestyle Redesign® (LR) program for healthy elderly in Israel. Poster presented at the 2019 AOTA Annual Conference & Expo, New Orleans, LA.