A randomized controlled pilot study of CBT-I Coach: Feasibility,acceptability,and potential impact of a mobile phone application for patients in cognitive behavioral therapy for insomnia

Abstract

There has been growing interest in utilizing mobile phone applications (apps) to enhance traditional psychotherapy. Previous research has suggested that apps may facilitate patients’ completion of cognitive behavioral therapy for insomnia (CBT-I) tasks and potentially increase adherence. This randomized clinical trial pilot study (n = 18) sought to examine the feasibility, acceptability, and potential impact on adherence and sleep outcomes related to CBT-I Coach use. All participants were engaged in CBT-I, with one group receiving the app as a supplement and one non-app group. We found that patients consistently used the app as intended, particularly the sleep diary and reminder functions. They reported that it was highly acceptable to use. Importantly, the app did not compromise or undermine benefits of cognitive behavioral therapy for insomnia and patients in both groups had significantly improved sleep outcomes following treatment.

Keywords

cognitive behavior therapy computer applications insomnia mobile devices psychotherapy

Chronic insomnia affects more than 1 in 10 people.¹ Rates are even higher among veterans using Veterans Health Administration (VHA) services.² Cognitive behavioral therapy for insomnia (CBT-I) is an effective treatment for insomnia^3–7 with superior long-term efficacy⁸ and fewer risks compared to hypnotic medications.⁹ As such it is widely recommended as a first-line treatment for insomnia.¹

The VHA has an ongoing national dissemination initiative to train its licensed mental health care providers (i.e. non-sleep specialists) to deliver CBT-I.¹⁰ Patient outcomes from this training program are comparable to those of published research trials.^11,12 However, patient non-adherence to treatment recommendations (e.g. going to bed only when sleepy and limiting naps) may be undermining the benefit for some patients.

Mobile applications (apps) utilized with smartphones have the potential to improve traditional psychotherapy by enhancing access to psychoeducation and psychotherapy skills (e.g. relaxation techniques), facilitating monitoring of symptoms and outcomes, and assisting with relapse prevention.^13,14 Emerging literature is beginning to accumulate showing the promise of apps for depression, stress, psychosis, eating disorders, and substance use^15–19 and integrating mental health apps into treatment within a VA setting shows initial promise.²⁰ Although there is increasing availability of consumer sleep technologies for sleep improvement, many of the existing apps focus on monitoring sleep and providing sleep education, rather than assisting with active therapy.^21–23

The VHA National Center for Post-Traumatic Stress Disorder (PTSD), in partnership with Stanford University School of Medicine and the Department of Defense’s National Center for Telehealth and Technology, built CBT-I Coach, a patient app designed to be used as an adjunct to face-to-face CBT-I. It was released for both iOS²⁴ and Android²⁵ mobile devices in 2013. It was specifically designed to help facilitate patients’ completion of CBT-I tasks, potentially increasing patient adherence to the protocol.²⁶ The design and content of the app has been described in detail in previous publications.^26,27 Briefly, the app provides education about sleep processes, developing positive sleep routines, and improving sleep environments. Key features of the app include a sleep diary to record daily sleep variables (see Figure 1), ability to update a sleep prescription (recommended bedtime and wake time) in consultation with CBT-I providers, tools and guided exercises for quieting the mind (see Figure 1), education about sleep and sleep-health behaviors, reminder functions and alarms to help change sleep habits (e.g. reminders for when to stop caffeine intake, start wind-down time, and alarms for prescribed bedtime and wake time). See Figure 1 for a screenshot of the home page of the app.

Figure 1.

Example screenshots from CBT-I Coach.

In a recent survey, VA CBT-I clinicians indicated that they believe this app may improve care and increase adherence,²⁶ and an initial feasibility study investigating the use of the app among patients engaged in a cannabis cessation attempt indicated daily engagement with the app.²⁷ However, it is uncertain if integrating CBT-I Coach into the existing VA CBT-I protocol is feasible in terms of whether patients will consistently use it as intended and if they will find it acceptable to use while engaged in this therapy. It is also unknown if CBT-I Coach will actually improve patient adherence to the therapy and, importantly, whether using the app will compromise or undermine benefits of CBT-I, a well-established evidence-based treatment, on sleep-related outcomes. Therefore, the current study sought to examine the feasibility and acceptability of CBT-I Coach, as well as explore the potential impact of this device on adherence and sleep outcomes. The first-author, in conjunction with the developers of the app (second, sixth, and seventh authors) utilized the existing app in a sample of patients receiving CBT-I in a clinical setting. We hypothesized that participants randomized to CBT-I with the app and without the app would report significant improvements in sleep, but that the app group would have higher adherence than the non-app group.

Methods

Participants

This study was approved by the Veterans Affairs Institutional Review Board. Forty-one consecutive referrals for CBT-I at a Midwestern VA Medical Center were pre-screened following CBT-I intake assessments by project clinicians, with further screening via phone by the project coordinator. Inclusion criteria included commitment to begin CBT-I, ownership of a smartphone, and willingness to use CBT-I Coach. Exclusion criteria included moderate or greater suicidal or homicidal ideation, significant alcohol or drug use, and active psychotic symptoms. Patients not meeting these criteria where referred elsewhere or underwent CBT-I outside of the study.

Figure 2 presents the flow of participants through the study from March 2014 to November 2014. Twenty-three potential participants were excluded, most commonly for not owning a smartphone (n = 9) and not imminently planning to begin CBT-I (n = 8). Two participants declined to participate due to lack of interest. After obtaining informed consent, 18 participants were randomized equally to either CBT-I plus CBT-I Coach (app group) or CBT-I without the app (non-app group) by the project coordinator, utilizing a computerized random number generator. Table 1 presents baseline characteristics. The average age was 48.50 years (standard deviation (SD) = 14.93), and participants were primarily White males with Android smartphones. There were no significant differences between the two conditions on baseline insomnia symptom levels or demographics, with the exception of marital status (p < .05), where those in the app group were more likely to be married compared to the non-app group (55.56% vs 11.11%). On average, participants reported clinical levels of insomnia. Three participants dropped out of the non-app group (two lost interest in treatment and dropped out after three sessions, one moved and dropped out after two sessions); none dropped out of the app group. There were no significant differences between retained participants and those who dropped out on baseline insomnia symptom levels or demographics.

Figure 2.

Flow of participants through the study.

Table 1.

Baseline characteristics.

	Coach	Non-coach
	(N = 9)	(N = 9)
Demographics
Age, mean (SD)	50.11 (15.74)	46.89 (14.85)
Men (%)	6 (66.67)	5 (55.56)
White race (%)	8 (88.89)	9 (100.00)
Married (%)	5 (55.56)	1 (11.11)*
Education > High School (%)	8 (88.89)	7 (77.78)
Sleep
Insomnia Severity Index (SD)^a	19.22 (3.70)	20.38 (5.55)
Phone
Phone type (%)
iPhone	2 (22.22)	3 (33.33)
Android	7 (77.78)	6 (66.67)

SD: standard deviation.

Based on n = 8 in non-coach due to missing data.

Significant χ² difference at p ⩽ .05. Insomnia Severity Index ⩾ 15 indicates clinical levels of insomnia.

Procedure

Participants completed semi-structured interviews at baseline and following treatment, as well as self-report measures prior to each CBT-I treatment session. Participants received a US$25 gift card for completing the baseline assessment and a US$50 gift card for the final assessment. Pre-treatment interviews were identical for both app and non-app groups and were used to obtain demographic information and baseline experience with apps, including questions about phone usage and comfort, and app usage. Following the initial interview, participants assigned to the app group were shown how to download and use the app. The app participants used the app in conjunction with the standard CBT-I procedures, including using it to complete sleep diaries. Non-app group participants completed CBT-I according to standard procedures without CBT-I Coach.

In order to explore the impact of the app on adherence to treatment recommendations and sleep outcomes, participants in both groups reported the number of days they completed homework, amount of time spent on homework each week, and completed an insomnia questionnaire before each therapy session. Therapists completed a measure of patient adherence to treatment recommendations at the end of each session for all participants. At post-treatment, app group participants completed semi-structured interviews focused on their use and engagement with the app, including their impressions of CBT-I Coach, which elements they used, barriers to app usage, perceived value of the app, and potential enhancements of the app. The non-app group participated in a semi-structured interview during which they were shown the app and asked for their thoughts on integrating the app into CBT-I and suggestions for enhancements.

Treatment and therapists

Participants in both conditions completed CBT-I with one of two VA clinical psychologists who completed the VHA CBT-I training initiative.^10,11 Treatment consisted of weekly 1-h individual therapy sessions based on the CBT-I manual developed by VA.¹⁰ The protocol consists of five treatment sessions, with patients attending fewer or more sessions if clinically indicated. The average number of treatment sessions completed in this study were four (66.67% of patients), with almost a third of patients completing five sessions (27.78% of patients). The basic components of this protocol include: (1) sleep restriction, which involves limiting time in bed to consolidate sleep; (2) stimulus control, which involves restricting the bed/bedroom to sleep; and (3) cognitive restructuring, which addresses maladaptive thoughts and beliefs about sleep.

Measures

Insomnia Severity Index

The Insomnia Severity Index (ISI)²⁸ is a 7-item self-report measure designed to provide a global measure of difficulties sleeping at night and daytime impairment and was completed by participants during each session. Respondents rate statements using 4-point scales from 0 to 4 (response options differ by item) and items are summed to provide a total score. Scores of 0–7 indicate no clinical insomnia, 8–14 indicate subthreshold insomnia, 15–21 indicate moderate insomnia, and 22–28 indicate severe insomnia. This instrument has adequate psychometric properties, including internal reliability (coefficient alphas ranging from .76 to .78) and concurrent validity with sleep diaries and polysomnography.²⁸

Adherence scale

Adherence to CBT-I recommendations was measured using the Patient Adherence Form that was created for the VA CBT-I Training Program.¹⁰ Starting at the end of session two, therapists rated the extent to which participants followed six specific recommendations (e.g. adhering to recommended bedtime and wake time, limiting naps, and scheduling worry time) on a scale from 1 (no adherence) to 6 (complete adherence) or not applicable (NA) if the recommendations had not yet been introduced during the therapy. Scores were averaged to create a total adherence score that has shown good psychometric properties.¹²

Statistical analyses

Descriptive statistics were calculated for variables related to feasibility and acceptability from the semi-structured interviews. Independent samples t-tests were calculated to examine variables potentially related to treatment impact, including patient adherence and homework completion. Treatment outcomes analyses were conducted using intent-to-treat for ISI scores. These analyses included all available data from 17 participants; one participant was excluded due to errors in coding outcome data. Hierarchical Linear Modeling (HLM) was conducted using SAS (PROC mixed) to determine mean values at each time point and test the effect of treatment and condition on ISI scores during five treatment sessions. Fixed effects were specified for time and condition, whereas random effects accounted for the nested nature of the data with repeated measures over time within individuals. Effect sizes were calculated using Cohen’s d, which represents the standardized differences between means.

Results

Feasibility and acceptability of integrating CBT-I Coach into therapy

Table 2 presents descriptive statistics of smartphone and mobile app variables for all study participants. The majority of participants indicated having their phone with them all of the time (n = 14, 77.8%) and owning a smartphone for at least 5 years (n = 13, 72.2%). Most participants indicated using their smartphone for calls (n = 16, 88.9%) and apps (n = 15, 83.33%) at least daily and most indicated being very comfortable using their smartphone generally (n = 16, 88. 9%) and apps specifically (n = 14, 77.8%). Very few participants had used apps for mental health or sleep (n = 2, 11.1%) and none had used CBT-I Coach previously. Demographic variables, including age, gender, marital status, ethnicity, and education, were not significantly related to degree of comfort with apps or frequency of app use.

Table 2.

Pre-treatment interview: feasibility and acceptability of mobile applications (n = 18).

	N (%)
How often do you have your phone with you?
Most of the time	4 (22.22)
All of the time	14 (77.78)
Length of smartphone ownership
⩽ 4 years	5 (27.78)
5+ years	13 (72.22)
Frequency of smartphone use
Calls
At least daily	16 (88.89)
At least weekly	2 (11.11)
Applications
At least daily	15 (83.33)
At least monthly	3 (16.67)
Current comfort
Smartphone
Very comfortable	16 (88.89)
Mostly comfortable	2 (11.11)
Applications
Very comfortable	14 (77.78)
Mostly comfortable	1 (5.56)
Neutral	3 (16.67)
Use of smartphone applications
Health
Yes	9 (50.00)
No	9 (50.00)
Mental health
Yes	2 (11.11)
No	16 (88.89)
Sleep
Yes	2 (11.11)
No	16 (88.89)

Table 3 summarizes findings from the post-treatment semi-structured interview with the app group participants. The most commonly used element of the app was the sleep diary, followed by the educational materials, relaxation exercises, and reminders. All participants reported that the sleep diary was a helpful component (n = 9, 100%; e.g. “the sleep efficiency numbers made me feel better about my sleep and decreased my anxiety about sleep. I liked being able to push enter and get the results”), followed by reminders (n = 2, 22.2%; e.g. “Reminders kept me on track”). All participants indicated that they would recommend the app to family or friends. In general, feedback from participants in the app group was positive and focused on the personalized feedback provided by the app, particularly as it related to the sleep diary information (e.g. “I like the app because it gives you info on what you are doing compared to what you think you are doing”). The non-app participants also provided feedback about CBT-I Coach. One participant indicated that it would increase compliance: “I think it would be very helpful because no one loses their phone, but I lost my sleep log.” Another stated, “I think it’s a good idea, it’s a lot easier to do homework because you always have your phone with you.”

Table 3.

Post-treatment interview: feasibility and acceptability of CBT-I Coach with participants in the app condition.

	N = 9 (%)
Which elements do you use?
Sleep diary	9 (100.0)
Learn: habits and sleep	6 (66.67)
Learn: sleep 101	5 (55.56)
Tools: quiet your mind	5 (55.56)
Reminders	5 (55.56)
I need more sleep	4 (44.44)
Tools: prevent insomnia in the future	4 (44.44)
Learn: CBT-I glossary	2 (22.22)
What part(s) of the app were most helpful?
Sleep diary	9 (100.00)
Reminders	2 (22.22)
Relaxation	1 (11.11)
Information	1 (11.11)

CBT-I: cognitive behavioral therapy for insomnia.

Impact of CBT-I Coach on CBT-I adherence and outcomes

There were no statistically significant differences between the app and non-app group on average time spent on homework (d = .66 in favor of non-app group), number of days completing homework (d = .11 in favor of non-app group), and days completing sleep diaries (d = .36 in favor of app group). The effect size for the group difference on adherence scores was large (d = .76), favoring the app group, but it was not statistically significant (t(16) = 1.53, p = .15).

HLM intent to treat analysis estimates for time was significant for ISI scores, F(4, 42) = 8.84, p < .001, d = 1.84 (see Figure 3). The main effect for treatment condition (d = .21) and the interaction of treatment time by condition (d = .73) were statistically non-significant. Among participants who completed treatment, four (26.7%) had no insomnia, eight (53.3%) reported subthreshold insomnia, one (6.7%) reported moderate insomnia, and two (13.3%) reported severe insomnia according to ISI scores.

Figure 3.

Hierarchical linear model predicted Insomnia Severity Index (ISI) scores at each therapy session (n = 17). Error bars represent standard error above and below predicted values.

Discussion

This is the first study to report the feasibility and acceptability of integrating an app with CBT-I. Previous research suggests that providers see the app as potentially improving care and increasing adherence²⁶ and this study suggests that integrating CBT-I Coach with individual CBT-I is highly feasible and acceptable to patients. Overall, the app was favorably received by all participants in the app group and participants in the non-app group responded favorably when they were introduced to the app at the end of therapy. The qualitative data suggest that patients were using the app as it was intended (particularly the sleep log and reminder functions) and the app improved accessibility to therapy materials.

This study is also the first to report on the impact of CBT-I Coach on process variables, including homework completion, adherence, and accessibility to therapy components. As hypothesized, CBT-I remains an effective treatment after integrating the app. Use of the app did not appear to erode or dilute the basic elements of the therapy. There was also some indication that app use related to better adherence to therapeutic recommendations. Although this finding was non-significant, the large effect size produced suggests that members of the app group are perceived as being more adherent by therapists. The study was clearly underpowered, but this potential effect represents a promising area for future research.

Although this study represents an important first step in investigating the role of mobile interventions in the treatment of insomnia, there are several limitations that should be noted. The trial was non-blinded, which may have biased adherence ratings by therapists (e.g. higher adherence scores for patients in app group). The research context may have increased homework compliance and sleep diary completion by patients in both groups (e.g. higher homework completion across all patients due to weekly monitoring, resulting in non-significant group differences). In addition, the app did not provide time-date stamped data, and so we could not confirm if participants completed the sleep diary within an hour or two of awakening as is recommended. Moreover, we were not able to collect objective information on the amount of time spent using the app, as this capacity is not built into the app.

Integrating mobile health (mHealth) into behavioral sleep treatments represents a promising area for future research, particularly within the context of stepped care models. Demand for sleep treatment often exceeds the availability of trained providers, which validates and necessitates the use of stepped care models. These models are often conceptualized in the shape of a pyramid, with the base representing low intensity, low resource treatment modalities (e.g. self-help) as an entry point, with treatments growing progressively more resource heavy and intensive as one moves to higher levels.^29,30 Apps based on CBT-I principles may represent an entry level step for some patients and pilot trials are needed with stand-alone CBT-I apps to determine treatment efficacy. In addition to potentially serving as a stand-alone treatment method, behavioral sleep treatment apps provide an efficient way to collect information (e.g. baseline sleep diary data) and deliver basic sleep education to patients while they are waiting for care. Finally, the app could potentially be utilized as a screening tool to help providers determine what level of care is necessary for an individual patient, in accordance with baseline symptom levels and initial improvement in sleep following independent app utilization; additional studies with apps are needed to determine how they can contribute to personalized treatment by matching patients with appropriate levels of care.

Footnotes

Acknowledgements

The views expressed in this article are those of the authors and do not reflect the official policy or position of the US Department of Veterans Affairs.

Funding

This material is the result of work supported with resources and the use of facilities at the Minneapolis VA Health Care System and the VA Palo Alto Health Care System.

References

National Institutes of Health. Manifestations and management of chronic insomnia in adults. Sleep 2005; 28: 1049–1057.

Mustafa

Erokwu

Ebose

et al . Sleep problems and the risk for sleep disorders in an outpatient veteran population. Sleep Breath 2005; 9: 57–63.

Morin

Culbert

Schwartz

. Nonpharmacological interventions for insomnia: a meta-analysis of treatment efficacy. Am J Psychiatry 1994; 151: 1172–1180.

Murtagh

Greenwood

. Identifying effective psychological treatments for insomnia: a meta-analysis. J Consult Clin Psychol 1995; 63: 79–89.

Smith

Perlis

Park

et al . Comparative meta-analysis of pharmacotherapy and behavior therapy for persistent insomnia. Am J Psychiatry 2002; 159: 5–11.

Wang

Tsai

. Cognitive behavioural therapy for primary insomnia: a systematic review. J Adv Nurs 2005; 50: 553–564.

Irwin

Cole

Nicassio

. Comparative meta-analysis of behavioral interventions for insomnia and their efficacy in middle-aged adults and in older adults 55+ years of age. Health Psychol 2006; 25: 3–14.

Jacobs

Pace-Schott

Stickgold

et al . Cognitive behavior therapy and pharmacotherapy for insomnia: a randomized controlled trial and direct comparison. Arch Intern Med 2004; 164: 1888–1896.

Manber

Edinger

Gress

et al . Cognitive behavioral therapy for insomnia enhances depression outcome in patients with comorbid major depressive disorder and insomnia. Sleep 2008; 31: 489–495.

10.

Manber

Carney

Edinger

et al . Dissemination of CBTI to the non-sleep specialist: protocol development and training issues. J Clin Sleep Med 2012; 8: 209–218.

11.

Karlin

Trockel

Taylor

et al . National dissemination of cognitive behavioral therapy for insomnia in veterans: therapist and patient-level outcomes. J Consult Clin Psychol 2013; 81: 912–917.

12.

Trockel

Karlin

Taylor

et al . Cognitive behavioral therapy for insomnia with veterans: evaluation of effectiveness and correlates of treatment outcomes. Behav Res Ther 2014; 53: 41–46.

13.

Clough

Casey

. Technological adjuncts to enhance current psychotherapy practices: a review. Clin Psychol Rev 2011; 31: 279–292.

14.

Luxton

McCann

Bush

et al . mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof Psychol Res Pr 2011; 42: 505–512.

15.

Garrison

Pal

Rojiani

et al . A randomized controlled trial of smartphone-based mindfulness training for smoking cessation: a study protocol. BMC Psychiatry. Epub ahead of print 14 April 2015. DOI: 10.1186/s12888-015-0468-z.

16.

Donker

Petrie

Proudfoot

et al . Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013; 15: e247.

17.

Bucci

Barrowclough

Ainsworth

et al . Using mobile technology to deliver a cognitive behaviour therapy-informed intervention in early psychosis (Actissist): study protocol for a randomised controlled trial. Trials. Epub ahead of print 10 September 2015. DOI: 10.1186/s13063-015-0943-3.

18.

Monney

Penzenstadler

Dupraz

et al . mHealth app for cannabis users: satisfaction and perceived usefulness. Front Psychiatry. Epub ahead of print 27 August 2015. DOI: 10.3389/fpsyt.2015.00120.

19.

Tregarthen

Lock

Darcy

. Development of a smartphone application for eating disorder self-monitoring. Int J Eat Disord 2015; 48: 972–982.

20.

Erbes

Stinson

Kuhn

et al . Access, utilization, and interest in mHealth applications among veterans receiving outpatient care for PTSD. Mil Med 2014; 179: 1218–1222.

21.

Behar

Roebuck

Domingos

et al . A review of current sleep screening applications for smartphones. Physiol Meas 2013; 34: R29–R46.

22.

Bhat

Ferraris

Gupta

et al . Is there a clinical role for smartphone sleep apps? Comparison of sleep cycle detection by a smartphone application to polysomnography. J Clin Sleep Med 2015; 11: 709–715.

23.

Kientz

Choe

et al . Consumer sleep technologies: a review of the landscape. J Clin Sleep Med. Epub ahead of print 22 Jun 2015. DOI: 10.5664/jcsm.5288.

24.

Hoffman

Taylor

Manber

et al . CBT-I Coach (Version 1.0) (Mobile application software), 2013, http://itunes.apple.com

25.

Hoffman

Taylor

Manber

et al . CBT-I Coach (Version 1.0) (Mobile application software), 2013, https://play.google.com/store

26.

Kuhn

Weiss

Taylor

et al . CBT-I Coach: a description and clinician perceptions of a mobile app for cognitive behavioral therapy for insomnia. J Clin Sleep Med 2016; 12: 597–606.

27.

Babson

Ramo

Baldini

et al . Mobile app-delivered cognitive behavioral therapy for insomnia: feasibility and initial efficacy among veterans with cannabis use disorders. JMIR Res Protoc. Epub ahead of print 17 July 2015. DOI: 10.2196/resprot.3852.

28.

Bastien

Vallieres

Morin

. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med 2001; 2: 297–307.

29.

Espie

. “Stepped care”: a health technology solution for delivering cognitive behavioral therapy as a first line insomnia treatment. Sleep 2009; 32: 1549–1558.

30.

Edinger

. Is it time to step up to stepped care with our cognitive-behavioral insomnia therapies? Sleep 2009; 32: 1539–1541.