Implementation of a Guided,Digital Cognitive Behavioral Program for Anxiety in Primary Care: Preliminary Findings of Engagement and Effectiveness

Abstract

Background:

Implementation of digital behavioral health programs in primary care (PC) can improve access to care for patients in need.

Introduction:

This study provides preliminary data on user engagement and anxiety symptom change among patients referred by their PC provider to a guided, mobile cognitive behavioral program, Lantern.

Materials and Methods:

Adults aged 20–65 years with at least mild anxiety (GAD-7 ≥ 5) during routine clinical screening in two PC practices were offered Lantern. The primary outcome was self-reported anxiety collected at baseline and 2 months. Linear mixed effects modeling was used to examine anxiety symptom reduction from baseline to 2 months. Post hoc analyses evaluated how number of units completed, number of techniques practiced, and days of usage impacted symptom change.

Results:

Sixty-three participants signed up for Lantern and had both baseline and 2- month GAD-7. A mixed effects model adjusted for age, gender, medical complexity score, and physical health found a significant effect of time on GAD-7 (β = −2.08, standard error = 0.77, t(62) = −2.71, p = 0.009). Post hoc analyses indicated that mean number of units, techniques, and usage days did not significantly impact GAD-7 change over 2 months. However, there was significantly greater improvement in anxiety in participants who completed at least three techniques.

Discussion:

Results benchmark to previous studies that have found statistically significant symptom change among participants after 4–9 weeks of face-to-face or Internet-based cognitive behavioral therapy (CBT).

Conclusions:

This study suggests that use of Lantern is associated with anxiety reduction and provides proof-of-concept for the dissemination and implementation of guided, CBT-based mobile behavioral health interventions in PC settings.

Introduction

Anxiety disorders are common and costly mental health problems in the United States, affecting 18% of the population and carrying an economic burden of at least $42 billion dollars.^1,2 With more mental health visits occurring in primary care (PC) than in psychiatric settings,³ PC physicians (PCPs) often bear the burden of managing patients with anxiety. While PCPs may be able to provide initial assessment and some treatment (e.g., psychiatric medication), they are less equipped to facilitate referrals for ongoing evidence-based behavioral healthcare, often due to a shortage of behavioral health specialists.³ Increasingly, behavioral healthcare has been integrated into PC settings to provide a direct access pipeline to needed services and be proactive and preventive in approaching behavioral health problems.

Cognitive behavioral therapy (CBT) is well established as “the gold standard,” evidence-based psychological intervention for patients with anxiety.^4,5 CBT is an active, collaborative, problem-focused, and skill-based approach traditionally delivered in the form of face-to-face (FTF) psychotherapy. CBT focuses on modifying individuals' dysfunctional patterns of behavior and thinking to improve mood and coping.⁶ Treatment typically occurs over the course of 10–20 weekly, 50-min sessions.^7,8

Numerous meta-analyses support the efficacy of CBT for adult anxiety.^5,9,10 The average post-treatment response rate is 49.5% across all anxiety disorders,¹¹ with an uncontrolled pre- to-post-test effect size of Hedge's g = 0.92 for generalized anxiety disorder (GAD).¹² CBT outcome research has shown durability of up to 14 years in symptom improvement,^5,13
–15 with less interim mental health treatment at follow-up for CBT recipients compared to other forms of psychotherapy, medication, or placebo.¹⁵

Despite CBT's status as the standard of care for multiple psychological disorders, not all patients who may benefit from CBT are able to access it. This is due to cost, time, stigma, provider shortage, and practical problems (e.g., lack of transportation).^16,17 In pace with today's digital revolution, there is a push to provide CBT through interactive, Internet- and mobile-based platforms that may overcome the access burdens of traditional FTF psychotherapy. Digital CBT can be provided with or without guidance from a therapist or trained coach (e.g., as guided or pure self-help), when and where it is most convenient for the individual. Guided digital CBT programs have been found to be as effective as FTF psychotherapy in reducing anxious and depressive symptoms.^18,19 In addition, digital CBT programs have demonstrated effectiveness in reducing anxiety and depression in PC.^20

–23

Lantern is one such guided, digital cognitive behavioral program that combines evidence-based anxiety management skills with motivational coaching. Providing Lantern in PC settings can improve patient access, decrease the burden on PCPs and behavioral health providers, and increase efficiency of stepped-care approaches to behavioral health treatment. A study underway at the University of Pittsburgh Medical Center (UPMC) is evaluating the acceptability, feasibility, and effectiveness of Lantern over 12 months, compared to treatment as usual (TAU), among PC patients with elevated anxiety. Of interest is Lantern users' early response to the program, as early response to psychotherapy has been found to predict favorable treatment outcomes.^24
–26 Previous studies have reported that 60–80% of symptom improvement in depression occurs within the first 4 weeks of CBT and such early response predicts later treatment outcome.^27

–30 Literature on early response in GAD is limited; however, one study of older adults with GAD found that a 6.5-point reduction in anxious symptomatology after 1 month in CBT predicted treatment responder status at post-treatment.³¹ Additional studies of technology-delivered CBT programs have demonstrated effects after relatively short-term treatment (e.g., 8–9 weeks).^32,33 These findings suggest that digital CBT programs demonstrate symptom improvement within a relatively short period of time, and short-term improvement may help predict long-term improvement.

In this study, we examined preliminary data, from the ongoing study previously introduced, on anxiety symptom change over 2 months among PC patients who accessed Lantern to benchmark our results to the existing literature and provide proof-of-concept for Lantern as an evidence-based behavioral intervention for adults with GAD in PC settings.

Materials and Methods

Design

This study is a substudy of a larger and ongoing prospective, pragmatic, two-arm parallel comparison trial of Lantern to TAU at 4 PC clinics.³⁴ In the parent study, two clinics serve as active sites, offering Lantern to eligible study participants, and two serve as control sites, providing enhanced TAU. The active sites were chosen based upon demographic similarities indicating that participants would be drawn from similar populations. Specifically, the two clinics are located in demographically similar regions, and examination of electronic health record data for all patients in each clinic between 2015 and 2016 found their patient populations to be comparable with regard to patient age, ethnicity, and mean number of chronic conditions. Primary outcome measures for the parent study are improvements in participants' anxiety symptoms and quality of life (QoL), and acceptability and feasibility of Lantern's mobile app. Anxiety outcomes are collected at baseline, 2, 6, and 12 months and QoL outcomes are collected at baseline, 6, and 12 months. At this time, study enrollment has closed, although 6- and 12-month data collection is ongoing. The parent study is approved by the UPMC Institutional Review Board. In this study, we focus on the early, 2-month change in anxiety symptoms for only those participants who accessed Lantern (Clinical Trials Registry: NCT03035019).

Participants

At all sites, the GAD-7 item questionnaire (GAD-7)³⁵ and 12-item Short Form Survey (SF-12)³⁶ are provided to patients as part of routine clinical screening. Patients with GAD-7 ≥ 5, indicating at least mild anxiety, are invited to participate in the study if they are aged 20–65 years and have access to a smartphone or tablet. Patients are excluded from the study if they are non-English speaking (Lantern is only available in English); pregnant; reporting current suicidal ideation, psychosis, or post traumatic stress disorder; or participating in a concurrent study on acute back pain.

Intervention (Lantern)

Lantern is a mobile phone application (“app”) that delivers a guided, cognitive behavioral program based on empirically supported CBT protocols for GAD.^37
–39 Lantern was developed in partnership with academic settings to ensure that it provides theoretically sound content and meets clinical fidelity. Its interface was designed by individuals experienced in technology product development and user design to promote engagement on the mobile platform. See Table 1 for a summary of the six core components of Lantern's anxiety program. These components are delivered in a total of 40 brief 10–15 min units that introduce a total of 26 techniques.

Table 1.

Lantern's Anxiety Program

COMPONENT	SUMMARY
Education	Education about anxiety and the inter-relation between thoughts, emotions, and behaviors.
Relaxation	Empirically supported relaxation exercises, such as diaphragmatic breathing and guided imagery.
Cognitive restructuring	Guidance and techniques to challenge maladaptive thoughts in service of developing more adaptive thoughts.
Exposure	Education about the relationship between anxiety and avoidance and how to use the principle of systematic exposure to anxiety-provoking situations to overcome anxiety in the long term.
Mindfulness	Mindfulness exercises for observing thoughts and feelings without judgment.
Habit formation and skills generalization	Guidance on how to generalize Lantern skills to user's life and form healthy habits.

Lantern users are paired with a human coach whose role is to help users remain engaged by using motivational techniques, answering questions, and pacing their progress. Lantern coaches communicate with users through asynchronous, in-app text messaging (see Fig. 1a and b for screen shots of the Lantern app and an example of coach-user messaging). Each Lantern coach follows a user through his or her duration in the program and has access to an internal coaching dashboard that provides information on what content the user has completed and all coach-user messages. Lantern primarily employs Master's-level coaches with backgrounds in health and wellness coaching or mental health treatment. Coaches are supervised by doctoral-level clinical psychologists who provide ongoing training and supervision in CBT, digital mental health, coaching methods, and risk. Coaches review user messages daily for signs of high risk symptoms; provide crisis (e.g., county hotline, 911) or therapy referrals as appropriate; and notify Lantern staff and the study team of any risk issues and referrals.

Fig. 1.

(a) Screen shot of Lantern home page. (b) Example of coach-user messaging.

Measures

Demographics

Age, gender, ethnicity, race, and chronic medical conditions were extracted from participants' electronic medical records.

Medical complexity score

Medical complexity score was derived from the total number of ICD-10 chronic medical conditions coded across all of a participant's PC visits.

Anxiety

The GAD-7 is a 7-item self-report questionnaire assessing symptoms of GAD according to Diagnostic and Statistical Manual of Mental Disorders, 4th Edition criteria. Individuals are asked whether, over the past 2 weeks, they have experienced symptoms such as “feeling nervous, anxious or on edge,” and “not being able to stop or control worrying.” Items are rated on a scale of 0–3 and summed. Total scores of 0–4 indicate no or minimal anxiety; 5–9 indicate mild anxiety; 10–14 indicate moderate anxiety; and 15–21 indicate severe anxiety.³⁵ The GAD-7 is one of the most widely used anxiety measures in PC and mental health settings.⁴⁰ It has good internal consistency (Cronbach's α = 0.92), test-retest reliability (intraclass correlation coefficient = 0.83), and validity in the general population.³⁵

Quality of life

The SF-12 is a 12-item, validated self-report measure that examines eight different aspects of functioning and QoL.³⁶ It produces summary scores for physical and mental health. Baseline physical health QoL score was used in this study as a covariate and proxy for physical disease load.

Engagement

Engagement was assessed by usage data obtained within the Lantern app for each participant. Data included the number of units completed, the number of times a technique was practiced, and the number of days the participant logged into the app. An “engaged” user was defined as a participant who completed ≥3 techniques.

Procedure

Patients were screened for study eligibility by PC clinic staff, using tablets hosting the GAD-7 and SF-12, in the waiting room as part of routine care. If a patient's GAD-7 ≥ 5, a best practice alert suggesting a psychosocial intervention was automatically generated for the patient's PCP within the electronic health record. Eligible patients then met with clinic staff to learn about the study and provide informed consent. If a patient consented, his or her PCP ordered Lantern through the electronic health record and the patient was able to sign up for Lantern by downloading the mobile app. Each participant was provided access to Lantern for 2 years.

Two months after enrollment, participants were sent the GAD-7 through secure e-mail. Participants who did not complete the GAD-7 after receiving the e-mail were contacted by phone by research assistants blinded to their Lantern progress to complete the GAD-7.

Data Analyses

We used descriptive statistics to summarize demographic characteristics, medical complexity, and baseline GAD-7 and SF-12 for all consented participants offered Lantern. We tested for baseline differences among these variables between Lantern participants who were and were not included in the 2-month cohort for the current analyses, as well as between Lantern participants from Clinic A versus Clinic B.

A mixed-effects linear model was used to examine change in GAD-7 from baseline to 2 months. To account for within-subject correlation and Lantern being provided at two clinics, we included a random subject effect nested within clinic. Time was entered as a fixed effect. We first tested for the effect of time (2 months versus baseline) on symptom reduction without covariates. We then tested for the effect of time on symptom reduction with age, medical complexity score, gender, and baseline SF-12 physical health score as covariates.

To determine whether individual changes in baseline to 2-month GAD-7 scores were statistically reliable, described as “beyond the scope of what could reasonably be attributed to chance or measurement error,”⁴¹ we calculated the reliable change index (RCI) for each participant. An RCI >1.96 indicated that a participant's decrease in GAD-7 was statistically reliable.

Two post hoc analyses were conducted. The first examined baseline to 2-month change in GAD-7 among only participants with GAD7 ≥ 8 at baseline. This cutoff was used given recent evidence indicating that it offers the highest sensitivity and specificity balance for GAD,⁴² and its use by the National Institute of Health and Clinical Excellence⁴³ and in studies of anxiety and comorbid medical conditions.⁴⁴ The second examined whether 2-month change in GAD-7 differed depending on number of units completed, number of times techniques practiced, or days of usage. This was operationalized through the test of an interaction term of each of these variables.

Results

Participant Characteristics

Five hundred ninety-three individuals consented to participate in the study and were referred to Lantern. Three hundred ten (52%) participants installed Lantern, with 63 also completing both baseline and 2-month GAD-7. See Figure 2 for flow chart of study enrollment. The majority of these participants were female (77.8%) and white (92.1%) with a mean age of 40.4 years (standard deviation [SD] = 13.4, range = 20.4–64.0). On average, participants had 2.40 (SD = 2.00, range = 0–9) chronic conditions (i.e., medical complexity score). Mean SF-12 physical health score was 43.9 (SD = 11.9, range = 18.7–58.5). No statistically significant differences in baseline demographic and symptom severity variables were observed between the 63 participants included in the 2-month cohort and the 530 participants who were not (Table 2).

Fig. 2.

Flowchart of study enrollment, Lantern sign-up, and assessment completion. GAD-7, generalized anxiety disorder 7-item questionnaire.

Table 2.

Baseline Demographic and Symptom Severity Characteristics for Lantern Participants With and Without 2-Month Generalized Anxiety Disorder-7 Item Questionnaire Scores^a

	WITH 2-MONTH GAD-7 ( n = 63)	WITHOUT 2-MONTH GAD-7 ( n = 530)
Age: M (SD)	40.4 (13.4)	41.6 (13.0)
Gender (male): n, %	14, 22.2	153, 28.9
Race (white): n, %	58, 92.1	479, 90.4
Medical complexity score: M (SD)	2.40 (2.00)	2.17 (1.99)
SF-12 physical health: M (SD)	43.4 (11.9)	42.5 (11.6)
BL GAD-7 score: M (SD)	11.5 (4.82)	11.2 (4.92)
Hyperlipidemia: n, %	13, 20.6	80, 15.1
Gastrointestinal distress: n, %	16, 25.4	115, 21.7
Chronic pain: n, %	8, 12.7	75, 14.2
Sleep medication: n, %	15, 23.8	92, 17.4
Opioid medication: n, %	27, 42.9	248, 46.8
Antidepressant medication: n, %	52, 82.5	380, 71.7
Benzodiazepines: n, %	20, 31.8	195, 36.8

Differences between groups were not statistically significant.

BL, baseline; GAD-7, generalized anxiety disorder 7-item questionnaire; M, mean; SD, standard deviation; SF-12, 12-item Short Form Survey.

Of the 63 participants, 29 were recruited from Clinic A and 34 from Clinic B. Statistically significant differences in baseline demographic characteristics did emerge between clinic sites with regard to SF-12 Physical Health score and sleep medication use (Table 3).

Table 3.

Baseline Demographic and Symptom Severity Characteristics Comparing the 63 Participants Who Consented to the Study, Accessed Lantern, and Had Both Baseline and 2-Month Generalized Anxiety Disorder-7 Item Questionnaire Scores Recruited from Clinic A Versus Clinic B

	CLINIC A ( n = 29)	CLINIC B ( n = 34)
Age: M (SD)	40.5 (12.3)	40.3 (14.4)
Gender (male): n, %	3, 10.3	11, 32.4
Race (white): n, %	25, 86.2	33, 97.1
Medical complexity score: M (SD)	2.31 (1.89)	2.47 (2.11)
SF-12 physical health: M (SD)^*	40.4 (12.1)	46.8 (11.2)
Baseline GAD-7 score: M (SD)	12.6 (4.81)	10.5 (4.68)
Hyperlipidemia: n, %	5, 17.2	8, 23.5
Gastrointestinal distress: n, %	7, 24.1	9, 26.5
Chronic pain: n, %	2, 6.9	6, 17.7
Sleep medication: n, %^*	11, 37.9	4, 11.8
Opioid medication: n, %	15, 51.7	12, 35.3
Antidepressant medication: n, %	23, 79.3	29, 85.3
Benzodiazepines: n, %	8, 27.6	12, 35.3

p < 0.05.

Within this 2-month period of the study, among all participants using Lantern, coaches sent a total of 37 texts suggesting referrals to users. Nine were referrals to seek medical advice; 7 to suggest FTF psychotherapy; and 21 to contact area crisis resources if the user believed himself or herself to be in immediate danger. There were no adverse outcomes.

Change In GAD-7

Across the 63 participants, mean baseline GAD-7 score was 11.5 (SD = 4.82) and mean 2-month GAD-7 score was 9.40 (SD = 4.94). Our unadjusted mixed-effects model found that the effect of time was significant on anxious symptomatology, indicating that GAD-7 significantly decreased from baseline to 2 months (β = −2.08, standard error [SE] = 0.77, t(62) = −2.71, p = 0.009) with a small to moderate effect size (d = 0.34, 95% confidence interval [CI] [−0.01 to 0.70]). When our model was adjusted for age, gender, medical complexity, and SF-12 physical health QoL score, there continued to be a significant effect of time on GAD-7 (β = −2.15, SE = 0.78, t(61) = −2.76, p = 0.008). One participant was removed from the adjusted analysis due to missing covariate data (Table 4).

Table 4.

Results of Mixed-Effects Models

	β	SE	df	t	p
Unadjusted	−2.08	0.77	62	−2.71	0.009
Adjusted^a	−2.15	0.78	61	−2.76	0.008

Model adjusted for age, medical complexity score, gender, and baseline SF-12 physical health score.

SE, standard error.

Statistically reliable improvement on GAD-7 was found for 19 of the 63 participants (30.1%).

Post Hoc Analyses

On average over 2 months, participants used Lantern for 18.4 days (SD = 15.8, range = 0–61), completed 11.2 units (SD = 14.3, range = 0–67), and practiced 16.1 techniques (SD = 20.3, range = 0–105). Users are able to repeat units and techniques as often as they wish, thus accounting for some participants having completed more than 40 units and 26 techniques. Two participants completed all units by 2 months. Longitudinal analyses indicated that number of units or techniques completed and days of app use did not significantly impact GAD-7 change over 2 months (Table 5). However, group analyses did find differential effects for participants completing ≥3 techniques (Table 6). Additional analyses comparing baseline demographic variables between those who completed at least three techniques to those who completed less found statistically significant differences in terms of age hyperlipidemia and physical health QoL (Table 7).

Table 5.

Effect of Number of Units, Techniques Completed, and Usage Days on Change in Anxiety Symptoms from Baseline to 2 Months

	β	SE	df	t	p
No. of units	0.03	0.05	61	0.52	0.61
No. of techniques	0.02	0.04	61	0.58	0.56
Days used app	0.01	0.05	61	0.06	0.95

Table 6.

Effect of Completing ≥3 or <3 Techniques on Anxiety Symptoms

	β	SE	df	t	p	d	95% CI
≥3 techniques (n = 43)	−2.19	0.92	42	−2.36	0.02	0.36	−0.07 to 0.79
≤2 techniques (n = 20)	−1.65	1.14	19	−1.31	0.21	0.29	−0.35 to 0.94
0 techniques (n = 11)	0.09	1.79	10	0.54	0.96	−0.02	−0.91 to 0.87

CI, confidence interval.

Table 7.

Baseline Demographic and Symptom Severity Characteristics Comparing Lantern Participants Who Completed <3 Techniques or ≥3 Techniques

	<3 TECHNIQUES ( n = 20)	≥3 TECHNIQUES ( n = 43)
Age: M (SD)^*	46.2 (14.7)	37.6 (11.9)
Gender (male): n, %	6, 30.0	8, 18.6
Race (white): n, %	18, 90.0	40, 93.0
Medical complexity score: M (SD)	2.65 (1.93)	2.28 (2.04)
SF-12 physical health: M (SD)^*	39.0 (12.7)	46.2 (11.0)
Baseline GAD-7 score: M (SD)	11.8 (5.11)	11.3 (4.74)
Hyperlipidemia: n, %^*	5, 25.0	8, 18.6
Gastrointestinal distress: n, %	4, 20.0	12, 27.9
Chronic pain: n, %	3, 15.0	5, 11.6
Sleep medication: n, %	5, 25.0	10, 23.3
Opioid medication: n, %	9, 45.0	18, 41.9
Antidepressant medication: n, %	17, 85.0	35, 81.4
Benzodiazepines: n, %	5, 25.0	15, 34.9

p < 0.05.

Forty-six (n = 46) participants had a baseline GAD-7 ≥ 8. For this subset of participants, mean baseline GAD-7 score was 13.6 (SD = 3.90) and mean 2-month GAD-7 score was 10.0 (SD = 5.37). The effect of time on GAD-7 among these participants was significant (β = −3.54, SE = 0.93, t(45) = −3.80, p < 0.001) with a medium effect size (d = 0.56, 95% CI [0.14 to 0.98]). The mean 3.6-point reduction on the GAD-7 approached clinically meaningful improvement, defined as a 4-point reduction on GAD-7.

Discussion

This study examined the early impact of a guided, digital cognitive behavioral program, Lantern, on anxiety symptoms among adults in PC. The study is novel in that anxiety screening and triggering of a physician order for Lantern are done automatically with the electronic health record. In a pragmatic study design with minimal research infrastructure (e.g., clinical staff, not research assistants, recruited participants, and no compensation was provided for study participation), our results show that it is feasible for patients in “real-world settings” to engage with a digital behavioral health program with no adverse events reported to date. Results indicate that participants' anxiety symptoms significantly decreased over 2 months. These results benchmark to previous studies that have found statistically significant symptom reduction within 4–9 weeks of participation in FTF or Internet-based CBT for anxiety and depression.^27,31
–33

At 2 months, we found a small to medium effect size (d = 0.34) for Lantern, with users completing an average of 11 units and 16 techniques over 15 days. For those engaged users completing at least three techniques, the effect size was ∼25% greater than for those completing less than three techniques. Although at least one study has found that extent of usage and modules completed are not significantly related to symptom outcomes,⁴⁵ it makes sense that the extent to which individuals engage with a digital program may have differential effects on outcomes. Among those with GAD-7 scores suggestive of “clinical caseness,” we found a medium effect size (d = 0.56).

Although studies of FTF and Internet-based CBT for GAD have found large effect sizes (e.g., d = 0.92 for FTF CBT¹² and d = 1.67 for Internet-based CBT³²), it is important to note a major difference between those studies and this one. Those studies recruited participants who met diagnostic criteria for GAD; ours recruited patients with elevated, but not necessarily diagnostic, symptoms. Furthermore, participants in studies of Internet-based CBT have typically completed 5 to 6 treatment modules within 8–9 weeks. We examined symptom change at 2 months. Patients who download Lantern are able to use it flexibly; although their coaches are available to enhance motivation and answer questions, patients ultimately determine the pace and intensity with which they use Lantern. Our sample completed an average of eleven 15-min units by the 2-month study assessment time point, equating to ∼165 min total or 21 min weekly engagement in app activities. This is far from being exposed to all core components, yet a significant and reliable anxiety symptom change was observed. This holds promise for our primary study outcomes at 6 months.

Furthermore, previous research indicates that effect sizes are lower in more clinically representative samples.^12,46 In our study, patients were screened for eligibility in the context of routine medical care. In addition to endorsing elevated anxiety symptoms, these patients present with many medical and psychiatric comorbidities, including diabetes, obesity, hyperlipidemia, asthma, chronic pain, depression, and substance use. In addition, although the fact that our study found only 30% of participants demonstrated reliable reductions in GAD-7 score between baseline and 2 months, it is important to note that relatively few studies of CBT for anxiety disorders have reported reliable change. Those that have indicate a wide range of reliable change, from 17% to 55% of participants receiving Internet or FTF CBT for anxiety.^47

–50 Again, these studies have largely included only participants meeting DSM-IV criteria for GAD. Thus, we do believe that our findings provide proof-of-concept that a CBT-based mobile phone program has the potential to reduce patients' anxious symptomatology.

This study has several limitations and strengths. Limitations include the small number of clinics used as participant recruitment sites, lack of randomization, and potentially low completer rates due to the absence of research-related facilitators of study recruitment. Of the 593 individuals who provided consent to participate in the study and were given an access code to Lantern, 52.3% downloaded the app. The majority of those who downloaded the app went on to use Lantern—82.5% completed at least one technique. This is remarkable engagement given that there were no study incentives or regular interaction with study or clinical staff after patients were referred to Lantern during their PC visit. This is also higher engagement than in other similarly designed studies in PC that relied upon e-mail and other reminders to complete program components. For example, Newby et al.⁵¹ found that 59% of their PC sample completed at least one session even with e-mail reminders to encourage patients to complete sessions within a specific timeframe.

Still, nearly half of the individuals who consented did not download Lantern. There are several operational implementation issues that may have contributed to this. Patients were provided with an access code to download Lantern during their clinic visits; if situational factors prevented them from downloading Lantern at that time (e.g., time constraints, poor Internet connectivity in the medical clinic, and lack of storage space on their phones), they were probably less likely to remember to do so in the future. The initial implementation of a product like Lantern into PC is typically the most difficult. As we gain the trust of PCPs and clinic staff by working with them to improve implementation, we are likely to optimize uptake of Lantern by both PCPs and their patients.

Engaging patients in behavioral healthcare is an enormous undertaking in PC. Qualitative feedback from the study PCPs indicates that they welcome the availability of an evidence-based CBT program that their patients can access immediately. This has helped the PCPs to be more effective at engaging their patients in behavioral healthcare by mitigating wait times and other logistic barriers to patients accessing FTF psychotherapy.

This study is the first to evaluate a coach-guided cognitive behavioral program delivered by mobile app in PC. It is a strength that both screening and referral to Lantern are being integrated into the electronic health record, thus providing both an ecologically valid assessment of the program and highlighting its implementation success. In addition, minimal research infrastructure within patient recruitment sites allows for a more “real-world” evaluation of the acceptability, feasibility, and effectiveness of digital behavioral health interventions. Future research will benefit from longer term follow-up and analyses of how users actually engage with such programs. In the current study, we found that lower age and higher SF-12 physical health score (indicating less physical impairment) may have been related to engagement; however, there may be other factors that affect how patients use digital health programs. Such research will inevitably inform the design, dissemination, and implementation of these digital interventions.

Footnotes

Acknowledgments

The authors acknowledge Katheryn McAuliff, Meredith Strassburger, Katie Scott, Jean Miewald, and Dina Perry for their assistance and support with this study. This research was funded by University of Pittsburgh Medical Center (UPMC) Enterprises and Thrive Network, Inc. d/b/a Lantern. UPMC Enterprises had no role in the study design; data collection, analysis, or interpretation; or writing or submitting of the article. Thrive Network, Inc. had a collaborative role in the study design; data collection; and writing and submitting of the article. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Disclosure Statement

J.S.Y. and M.O. are employees of Thrive Network, Inc. d/b/a Lantern. E.S. is an employee of the University of Pittsburgh Physicians, which is an affiliate of University of Pittsburgh Medical Center (UPMC). UPMC has a financial interest in Thrive Network, Inc., which develops and commercializes Lantern's products and services. Dr. Szigethy receives compensation from her employer for consulting work in connection with Lantern's products and services. Dr. Szigethy is also a consultant to AbbVie and an editor of the book, Cognitive-Behavior Therapy for Children and Adolescents, for which she receives royalties. She is a past consultant to iHope Network. Meredith Wallace and Francis Solano have no competing financial interests.

References

DuPont

, Rice

, Miller

, Shiraki

, Rowland

, Harwood

. Economic costs of anxiety disorders. Anxiety, 1996; 2:167–172.

Hoffman

, Dukes

, Wittchen

. Human and economic burden of generalized anxiety disorder. Depress Anxiety, 2008; 25:72–90.

Kroenke

, Unutzer

. Closing the false divide: Sustainable approaches to integrating mental health services into primary care. J Gen Intern Med, 2017; 32:404–410.

Cuijpers

, Sijbrandij

, Koole

, Huibers

, Berking

, Andersson

. Psychological treatment of generalized anxiety disorder: A meta-analysis. Clin Psychol Rev, 2014; 34:130–140.

Hans

, Hiller

. A meta-analysis of nonrandomized effectiveness studies on outpatient cognitive behavioral therapy for adult anxiety disorder. Clin Psychol Rev, 2013; 33:954–964.

Erickson

, Newman

. Cognitive behavioral psychotherapy for generalized anxiety disorder: A primer. Expert Rev Neurother, 2005; 5:247–257.

Zinbarg

, Craske

, Barlow

. Mastery of your anxiety and worry. New York, NY: Oxford University Press, 2006.

Rygh

, Sanderson

. Treating generalized anxiety disorder: Evidence-based strategies, tools, and techniques. New York, NY: Guildford Press, 2004.

Deacon

, Abramowitz

. Cognitive and behavioral treatments for anxiety disorders: A review of meta-analytic findings. J Clin Psychol, 2004; 60:429–441.

10.

Olantunji

, Cisler

, Deacon

. Efficacy of cognitive behavioral therapy for anxiety disorders: A review of meta-analytic findings. Psychiatr Clin North Am, 2010; 33:557–577.

11.

Loerinc

, Meuret

, Twohig

, Rosenfield

, Bluett

, Craske

. Response rates for CBT for anxiety disorders: Need for standardized criteria. Clin Psychol Rev, 2015; 42:72–82.

12.

Stewart

, Chambless

. Cognitive-behavioral therapy for adult anxiety disorders in clinical practice: A meta-analysis of effectiveness studies. J Consult Clin Psychol, 2009; 77:595–606.

13.

Stanley

, Wilson

, Novy

, Rhoades

, Wagener

, Greisinger

, Cully

, Kunik

. Cognitive behavior therapy for generalized anxiety disorder among older adults in primary care: A randomized clinical trial. JAMA, 2009; 301:1460–1467.

14.

DiMauro

, Domingues

, Fernandez

, Tolin

. Long-term effectiveness of CBT for anxiety disorders in an adult outpatient clinic sample. Behav Res Ther, 2013; 51:82–86.

15.

Durham

, Chambers

, MacDonald

, Power

, Major

. Does cognitive-behavioural therapy influence the long-term outcome of generalized anxiety disorder: An 8–14 year follow-up of two clinical trials. Psychol Med, 2003; 33:499–509.

16.

Harvey

, Gumport

. Evidence-based psychological treatments for mental disorders: Modifiable barriers to access and possible solutions. Behav Res Ther, 2015; 68:1–12.

17.

Kazdin

, Blase

. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect Psychol Sci, 2011; 6:21–37.

18.

Andersson

, Cuijpers

, Carlbring

, Riper

, Hedman

. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: A systematic review and meta-analysis. World Psychiatry, 2014; 13:288–295.

19.

Cuijpers

, van Straten

, Andersson

, van Oppen

. Psychotherapy for depression in adults: A meta-analysis of comparative outcome studies. J Consult Clin Psychol, 2008; 76:909–922.

20.

Andrews

, Williams

. Up-scaling clinician assisted internet cognitive behavioural therapy (iCBT) for depression: A model for dissemination into primary care. Clin Psychol Rev, 2015; 41:40–48.

21.

Reger

, Gahm

. A meta-analysis of the effects of internet- and computer-based cognitive-behavioral treatments for anxiety. J Clin Psychol, 2008; 65:53–75.

22.

Vallury

, Jones

, Osterbroek

. Computerized cognitive behavior therapy for anxiety and depression in rural areas: A systematic review. J Med Internet Res, 2015; 17:e139.

23.

Kaltenthaler

, Parry

, Beverley

, Ferriter

. Computerised cognitive-behavioural therapy for depression: Systematic review. Br J Psychiatry, 2008; 193:181–184.

24.

Fennell

MJV

, Teasdale

. Cognitive therapy for depression: Individual differences and the process of change. Cognit Ther Res, 1987; 11:253–271.

25.

Lutz

, Stulz

, Kock

. Patterns of early change and their relationship to outcome and follow-up among patients with major depressive disorders. J Affect Disord, 2009; 118:60–68.

26.

Masheb

, Grilo

. Rapid response predicts treatment outcomes in binge eating disorder: Implications for stepped care. J Consult Clin Psychol, 2007; 75:639–644.

27.

Ilardi

, Craighead

. The role of nonspecific factors in cognitive-behavior therapy for depression. Clin Psychol (New York), 1994; 1:138–155.

28.

Lewis

, Simons

, Kim

. The role of early symptom trajectories and pretreatment variables in predicting treatment response to cognitive behavioral therapy. J Consult Clin Psychol, 2012; 80:525–534.

29.

Hilbert A

, ldebrandt

, Agras

, Wilfley

, Wilson

. Rapid response in psychological treatments for binge eating disorder. J Consult Clin Psychol, 2015; 83:649–654.

30.

Thompson-Brenner

, Shingleton

, Sauer-Zavala

, Richards

, Pratt

. Multiple measures of rapid response as predictors of remission in cognitive behavior therapy for bulimia nervosa. Behav Res Ther, 2015; 64:9–14.

31.

Bradford

, Cully

, Rhoades

, Kunik

, Kraus-Schuman

, Wilson

, Stanley

. Early response to psychotherapy and long-term change in worry symptoms in older adults with generalized anxiety disorder. Am J Geriatr Psychiatry, 2011; 19:347–356.

32.

Titov

, Andrews

, Robinson

, Schwencke

, Johnston

, Solley

, Choi

. Clinician-assisted Internet-based treatment is effective for generalized anxiety disorder: Randomized controlled trial. Aust N Z J Psychiatry, 2009; 43:905–912.

33.

Paxling

, Almlov

, Dahlin

, Carlbring

, Breitholtz

, Eriksson

, Andersson

. Guided internet-delivered cognitive behaviour therapy for generalized anxiety disorder: A randomized controlled trial. Cogn Beh Ther, 2011; 40:159–173.

34.

Szigethy

, Solano

, Wallace

, Perry

, Morrell

, Scott

, Bell

, Oser

. A study protocol for a non-randomized comparison trial evaluating the feasibility and effectiveness of a mobile cognitive behavioral program with integrated coaching for anxious adults in primary care. BMJ Open, 2018; 8:e019108.

35.

Spitzer

, Kroenke

, Williams

, Lowe

. A brief measure for assessing generalized anxiety disorder: The GAD-7. Arch Intern Med, 2006; 166:1092–1097.

36.

Ware

Jr , Kosinski

, Keller

. A 12-item short-form health survey: Construction of scales and preliminary tests of reliability and validity. Med Care, 1996; 34:220–233.

37.

Newman

, Borkovec

. Cognitive-behavioral treatment of generalized anxiety disorder. Clin Psychol, 1995; 48:5–7.

38.

Newman

. Recommendations for a cost-offset model of psychotherapy allocation using generalized anxiety disorder as an example. J Consult Clin Psychol, 2000; 68:549–555.

39.

Newman

, Przeworksi

, Consoli

, Taylor

. A randomized controlled trial of ecological momentary intervention plus brief group therapy for generalized anxiety disorder. Psychotherapy (Chic), 2014; 51:198–206.

40.

Martinson

, Cancer

, Sweeney

. Assessment of anxiety in primary care. In: Maruish

, ed. Handbook of psychological assessment in primary care settings, 2nd ed, New York: Routledge, 2017; 319–320.

41.

Jacobson

, Truax

. Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol, 1991; 59:12–19.

42.

Plummer

, Manea

, Trepel

, McMillan

. Screening for anxiety disorders with the GAD-7 and GAD-2: A systematic review and diagnostic metaanalysis. Gen Hosp Psychiatry, 2016; 39:24–31.

43.

National Collaborating Centre for Mental Health. Common mental health disorders: The NICE guideline on identification and pathways to care. London: British Psychological Society and The Royal College of Psychiatrists, 2011.

44.

Terrill

, Hartoonian

, Beier

, Salem

, Alschuler

. The 7-item generalized anxiety disorder scale as a tool for measuring generalized anxiety in multiple sclerosis. Int J MS Care, 2015; 17:49–56.

45.

Berger

, Urech

, Krieger

, et al. Effects of a transdiagnostic unguided Internet intervention (‘velibra’) for anxiety disorders in primary care: Results of a randomized controlled trial. Psychol Med, 2017; 47:67–80.

46.

Weisz

, Donenberg

, Han

, Weiss

. Bridging the gap between laboratory and clinic in child and adolescent psychotherapy. J Consult Clin Psychol, 1995; 63:688–701.

47.

Arch

, Eifert

, Davies

, Plumb Vilardaga

, Rose

, Craske

. Randomized clinical trial of cognitive behavioral therapy (CBT) versus acceptance and commitment therapy (ACT) for mixed anxiety disorders. J Consult Clin Psychol, 2012; 80:750–765.

48.

Hedman

, Andersson

, Ljotsson

, Andersson

, Ruck

, Mortberg

, Lindefors

. Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: A randomized controlled non-inferiority trial. PLoS One, 2011; 6:e18001.

49.

Stiles

, Barkham

, Mellor-Clark

, Connell

. Effectiveness of cognitive-behavioural, person-centred, and psychodynamic therapies in UK primary-care routine practice: Replication in a larger sample. Psychol Med, 2007; 38:677–688.

50.

Robinson

, Titov

, Andrews

, McIntyre

, Schwencke

, Solley

. Internet treatment for generalized anxiety disorder: A randomized controlled trial comparing clinician vs. technician assistance. PLoS One, 2010; 5:e10942.

51.

Newby

, Mackenzie

, Williams

, McIntyre

, Watts

, Wong

, Andrews

. Internet cognitive behavioural therapy for mixed anxiety and depression: A randomized controlled trial and evidence of effectiveness in primary care. Psychol Med, 2013; 43:2635–2648.