Randomized controlled equivalence trial comparing videoconference and in person delivery of cognitive processing therapy for PTSD

Abstract

Introduction

In an effort to improve access to and utilization of health care, the Veterans Health Administration (VHA) continues to investigate the effectiveness of video-teleconferencing (VTC) technologies for service delivery. While previous research focused on the efficacy of VTC treatment for post-traumatic stress disorder (PTSD) in Vietnam era veterans, few studies have evaluated the efficacy of this modality and treatment for the Iraq/Afghanistan era veterans. The aim of this randomized clinical trial was to evaluate equivalence between in person and VTC psychotherapy for PTSD in this newer cohort.

Methods

Veterans of the Iraq/Afghanistan conflict from two VHA hospitals in the United States were recruited and randomized to receive cognitive processing therapy (CPT) for PTSD either in person (IP) or over VTC. Clinician-administered and self-report measures were collected before, during, and after treatment.

Results

A trend was observed which suggested that CPT over VTC may be equivalent to the treatment delivered in person, as suggested by previous studies. Regardless of treatment, veterans who received the intervention in both conditions reported significant decreases on post-treatment measures.

Discussion

This study highlighted research and clinical challenges in providing services to the newest veteran generation in general as well as unique challenges with VTC. One complicating factor to the statistical power of this study was a treatment dropout rate twice the original estimate. Factors that could have influenced this high dropout rate are explored.

Keywords

Telehealth Telecare Telepsychiatry

Introduction

Post-traumatic stress disorder (PTSD) is the most common mental health diagnosis affecting service men and women who have served in Operation Enduring Freedom, Operation Iraq Freedom and Operation New Dawn (OEF/OIF/OND). A meta-analysis of prevalence estimates for PTSD among these service members report prevalence rates typically ranging from 5% to 20%.¹ OEF/OIF/OND veterans diagnosed with PTSD have shown higher rates of comorbid pain-related diagnoses, substance use disorders and sleep issues.²

Veterans of OEF/OIF/OND represent the newest group of veterans utilizing the Department of Veterans Affairs (VA) services.³ Interestingly, a review of VA data suggested that only 9.5% of OEF/OIF veterans received adequate specialty care for mental health.⁴ One study found that OEF/OIF veterans diagnosed with PTSD, when compared to other veterans, were less likely to receive minimally adequate specialty care in the year following their diagnosis.⁵ Veterans of OEF/OIF are twice as likely to drop out of trauma-focused treatment after initiation, relative to Vietnam veterans.⁶ OEF/OIF veterans who complete treatment have shown greater improvement in trauma-related symptoms than a Vietnam veteran sample, suggesting the importance and potential impact of being able to engage and treat this newer era of veterans.⁷

Telemental health (TMH) refers to the provision of mental health treatment over video teleconference (VTC). A review of studies has shown that care, across a variety of disciplines, delivered over TMH systems can be as effective as in-person care (IP).⁸ Studies that examined the cost of TMH compared with IP treatment have generally found that TMH is similar in cost or less expensive than IP treatment^9,10 relative to having on-site or travelling specialty care providers. TMH is generally less costly and less time-consuming for the patient due to decreased travel expense and time,¹⁰ and thus offers a cost-effective means of increasing patient access to specialty care. TMH is an increasingly common service-delivery method for treatment to veterans with PTSD and other mental health problems, and yet there are relatively few randomized trials of TMH provision of an evidence based psychotherapy (EBP) for PTSD compared with in person treatment.

Reviews of the TMH literature^11–13 have generally demonstrated that TMH is an effective and satisfactory means of providing services; however, the studies to date are limited by methodological flaws including non-standardized treatments and poor study design. Specifically, most studies to date have mistakenly utilized traditional significance testing to demonstrate that the experimental treatment (i.e. TMH) is as good as the standard treatment.¹⁴ The most appropriate and rigorous analysis would follow either noninferiority or equivalence model design. The consensus from experts in the research and clinical application of EBPs via TMH indicates that this is a growing area of research with various site considerations and treatment protocol adjustments that may be necessary when implementing treatment via TMH.

There are a small number of randomized clinical trials that may address some of these various methodological and practical issues, while also demonstrating the equivalence of TMH and IP treatment of PTSD utilizing evidence-based psychotherapies. Specifically, studies have evaluated prolonged exposure therapy provided via TMH in clinic¹⁵ and home¹⁶ settings as well as cognitive processing therapy (CPT) and group cognitive only CPT.^18–20 Most of these studies did not focus solely on veterans of the OEF/OIF/OND era.

The primary aim of the current study was to evaluate the equivalence of individual CPT delivered via TMH or IP to OEF/OIF/OND veterans who were diagnosed with PTSD resulting from a military-related stressor.

Method

Participants and procedures

The study was conducted from 2008 to 2013, with approval from local institutional review boards of participating sites. Three-hundred and ten OEF/OIF/OND veterans (93% male) receiving mental health services at two Midwestern VA hospitals were screened for eligibility. Inclusion criteria were: qualification as an OEF/OIF/OND veteran, English-speaking, a current diagnosis of PTSD on the clinician-administered PTSD scale (CAPS),²¹ a military-related traumatic event (i.e. combat, sexual assault, non-combat physical assault), on a stable psychotropic medication regimen for at least one month prior to baseline assessment, and willing to maintain that regime over the course of the study (see Figure 1). Exclusion criteria were: already completed a trial of CPT; active diagnosis of psychotic, bipolar, or substance-dependence disorders; acute suicidal or homicidal ideation; or significant cognitive impairment.

Figure 1.

CONSORT flow diagram.

After completing the baseline assessment, eligible veterans were randomized to the TMH or IP treatment arm with an allocation ratio of 1:1 (based on permuted blocks of size 4, e.g. AABB, ABAB, ABBA, stratified by gender). Veterans who were randomized to TMH were treated by a therapist located at the VA hospital distant from where they were recruited. Due to problems with therapist availability at one site, 10 veterans in the TMH condition were treated by a therapist at their recruitment site to prevent them from waiting more than 1–2 weeks to begin treatment. These dyads still met over TMH. Of the 90 participants (N = 45 VTC and N = 45 IP), 51 completed at least 10 sessions and were included in the completer analysis.

Measures

Participants were assessed with the CAPS²¹ and the structured clinical interview for DSM-IV-TR axis I disorders (SCID-I)¹⁷ to assess their study eligibility and diagnostic comorbidities at baseline, and immediately post-treatment and 12-weeks post-treatment. Participants were required to have a score of 45 or greater (using the 1/2 scoring rule) on the CAPS and meet PTSD DSM-IV criteria on the SCID-I. CAPS PTSD severity was the primary clinical outcome. Ten percent of the videotaped CAPS (average kappa 0.82) and SCID-I (average kappa 0.86) interviews were randomly selected for rating by a second assessor.

Participants were also assessed at baseline, mid-treatment (sessions 5 and 9), immediately post-treatment, and 12-weeks post-treatment with various self-report measures. These included: PTSD checklist (PCL) and Beck depression inventory-II (BDI-II), and working alliance inventory.^22–24

Treatment

Cognitive processing therapy²⁵ is an evidence based treatment for PTSD. Individual sessions occurred once or twice a week and lasted approximately 50 minutes. Eight CPT-trained doctoral-level psychologists and one masters-level social worker served as study clinicians. All sessions were videotaped and 10% of them (N = 66) were sent to an expert CPT adherence and competence rater. For adherence, the element was scored based on whether or not it occurred; for competence, a rating was made on a seven-point scale ranging from 1 (poor) to 7 (excellent). Study clinicians demonstrated adherence to 96% of the unique and essential session elements. The average competence rating for all reviewed sessions was 6.53.

Statistical analysis

This study was designed as a randomized, multisite equivalence trial comparing TMH to IP. The primary efficacy endpoint was the total CAPS score. The TMH and IP interventions were deemed equivalent if the two-sided 95% confidence interval for the difference in total CAPS score between arms at the post-treatment assessment was between the equivalence margin, which was defined as +/−10 points of the total CAPS score. The rationale for choosing an equivalence margin of δ = 10 for the total CAPS scores was based on the results reported by Monson et al.²⁶ Equivalence between the TMH and IP intervention with respect to secondary endpoints was defined using the ratio between TMH and IP at the post-treatment assessment. Specifically, if the two-sided 95% confidence interval for the ratio between TMH and IP for a secondary outcome measure was between 0.8 and 1.2, then the TMH and IP intervention was considered as equivalent with respect to this secondary outcome measure.

A sample size of 42 patients per study arm was planned for this trial. This sample size would have provided 80% power to declare equivalence at the two-sided 0.05 significance level between the two interventions with respect to the total CAPS score at the post-treatment assessment, assuming an equivalence margin of δ = 10 and an overall standard deviation of 14 points in the total CAPS score.

The intent-to-treat (ITT) population was used as the primary analysis population. Changes within arms were evaluated using a paired t-test and reported in terms of Cohen’s effect size δ. A linear mixed effects model with unstructured correlation structure was used to evaluate longitudinal changes from the baseline to the week 5, week 9, post-test and week 12 post-test assessments within each arm and for both arms combined. Baseline characteristics were compared between study arms using a chi-square test or two-sample t-test. All p-values are two-sided and p < 0.05 was used to define statistical significance. Statistical analysis was conducted using SAS software (SAS Institute Inc., Cary NC), version 9.4.

Results

Ninety eligible subjects were randomized. Veterans were primarily male (93.3%) and had either never been married (34.4%) or were married/remarried (38.9%) (the remainder were either divorced or separated) with a mean age of 30.93 (SD = 6.05). Most had served on regular duty (58.9%; 21.1% Reserves, and 20.0% National Guard) in Iraq (75.6%; 11.1% in Afghanistan, or 13.3% in both). There were no significant differences in primary baseline characteristics between arms. Due to greater than predicted rates of dropout (43.3% across the study), the sample size required to declare equivalence was not achieved. However, a trend was observed of equivalence between treatment arms on the CAPS (δ = −0.5, 95% CI: −12.4–11.4, p = 0.094). At the post-treatment assessment, veterans across both treatment conditions were equivalent at the 95% confidence interval in terms of the secondary outcome of therapeutic alliance (ratio = 1.03, 95% CI: 0.98–1.08, p < 0.001), and a trend was observed of equivalence on the PCL (ratio = 0.92, 95% CI: 0.78–1.09, p = .079).

Significant reductions of symptoms of PTSD and depression were observed at primary post-test assessment for the combined sample and treatment arms (see Table 1) and across time (see Table 2).

Table 1.

Means +/− SD at baseline and post-test within each arm and both arms combined.

		Baseline		Post-test
		N	Mean ± SD	N	Mean ± SD	d (effect size)	t-statistic
TMH	CAPS	45	81.5 ± 15.7	25	51.2 ± 28.3	−1.4	6.85*
	PCL	45	60.7 ± 10.5	25	47.1 ± 16.2	−0.8	4.02*
	BDI–II	45	30.3 ± 14.2	24	19.2 ± 13.2	−0.6	3.01*
IP	CAPS	45	78.6 ± 15.1	26	50.7 ± 22.1	−0.9	4.56*
	PCL	45	60.1 ± 10.2	26	43.2 ± 13.3	−1.0	5.18*
	BDI-II	45	28.2 ± 11.0	26	15.5 ± 10.6	−0.9	4.45*
Combined	CAPS	90	80.0 ± 15.4	51	51.0 ± 25.1	−1.1	7.89*
	PCL	90	60.4 ± 10.3	51	45.1 ± 14.8	−0.9	6.47*
	BDI-II	90	29.2 ± 12.7	50	17.3 ± 11.9	−0.7	5.12*

TMH: telemental health; IP: In person; CAPS: clinician-administered PTSD scale total severity; PCL: PTSD checklist; BDI-II: Beck depression inventory-II

Indicates p < 0.05

Table 2.

Adjusted means and standard errors (SE) at baseline, week 5, week 9 and week 12 post within each arm and for both arms combined.

		Baseline		Week 5		Week 9		Post		Week 12 Post
		Mean	SE	Mean	SE	Mean	SE	Mean	SE	Mean	SE
TMH	CAPS	81.5	3.2					51.39*	4.0	50.02*	4.4
	PCL	60.7	1.9	56.2	2.4	47.90*	2.6	48.07*	2.3	45.13*	2.5
	BDI-II	30.3	2.0	40.33*	2.6	35.6	2.8	19.26*	2.6	20.99*	2.7
	WAI			155.1	3.5			159.2	3.4	159.8	3.5
IP
	CAPS	78.6	3.1					51.87*	4.1	48.24*	4.2
	PCL	60.1	1.8	56.1	2.0	49.46*	2.1	46.17*	2.2	45.94*	2.3
	BDI-II	28.2	1.8	25.4	2.0	22.13*	2.1	17.08*	2.2	17.29*	2.3
	WAI			159.7	2.6			161.0	2.8	159.2	2.9
Combined
	CAPS	80.0	2.2					52.03*	2.9	49.15*	3.1
	PCL	60.4	1.3	56.17*	1.5	48.89*	1.6	47.12*	1.6	45.58*	1.7
	BDI-II	29.2	1.4	31.5	1.7	27.6	1.8	17.99*	1.7	18.91*	1.8
	WAI			158.0	2.1			160.0	2.1	159.5	2.2

TMH: telemental health; IP: In person; CAPS: clinician-administered PTSD scale total severity; PCL: PTSD checklist; BDI-II: Beck depression inventory-II; WAI: working alliance inventory.

Indicates p < 0.05

Discussion

The study supports the utilization of EBPs for PTSD delivered via TMH in OEF/OIF/OND veterans. Due to the higher than expected dropout rate in both treatment conditions, the study lacked sufficient statistical power for the original analyses, with a 95% confidence interval between CAPS scores. Significance of the primary analysis test was found at a 90% confidence interval trend level. This trend suggests that those receiving CPT may experience similar benefit regardless of the manner in which the treatment is delivered. These results are consistent with previous research,²⁰ which utilized a noninferiority design with a 95% confidence interval with CPT-C groups. Of additional note, the working alliance between veteran and provider was deemed statistically equivalent between treatment conditions.

There were several limitations with this study, the most significant of which was the dropout rate. Research shows that veterans completing a course of CPT reported significant reductions in PTSD symptoms. This study dropout rate (43.3%) greatly exceeded the 20% estimate we utilized based on Monson et al.’s²⁶ study of CPT with Vietnam-era veterans. Our higher dropout rate for OEF/OIF/OND veterans is similar to a rate more recently reported for this cohort (68% in cognitive-behavioural therapies for PTSD).²⁷ Though we attempted to collect data on veterans who terminated CPT early, the influencing factors for dropout remain unclear due to lack of response from these veterans. There was no significant difference in dropout rates between treatment conditions (42% vs. 44%), suggesting the modality of treatment was not a contributing factor. Change in disability status may have impacted retention; however, Garcia et al.²⁷ found no differences between treatment dropouts and completers on disability filing status, PTSD disability, or overall disability ratings.

This study also highlighted another limitation related to recruitment and engagement of the OEF/OIF/OND cohort. The original study protocol was amended from twice-weekly CPT sessions to offering veterans weekly sessions as an alternative. This change addressed initial concerns presented by veterans who declined study participation due to the time constraints from school, family or work. While this change may have positively impacted recruitment, ongoing issues relating to managing additional life factors could have had a negative impact on retention. The question that warrants continued research is not necessarily CPT or TMH effectiveness, but rather how best to promote treatment engagement and retention in this veteran cohort.

Another unique challenge in providing TMH in the VA system is the scheduling of appropriate space and equipment at both the provider and the veteran sites of care. As previously noted, 10 veterans assigned to the TMH arm met with providers that were located within the same facility, resulting in a greater need of resources. Expansion of home-based TMH (provider treats veteran at his or her residence) may alleviate some of these additional challenges since scheduling of space and equipment is less cumbersome (only needed at the provider site) and the elimination of patient travel time reduces important logistical barriers to care.

Conclusion

This study suggests that TMH CPT for PTSD may be as good as IP CPT, as demonstrated by the trend level. Those veterans who received a full course of CPT reported significant reductions in PTSD and depression symptoms. Treatment dropout was nearly double the original estimate. Examination of recent literature suggests this is attributable to the specific veteran population rather than the methods or intervention utilized¹¹ since studies of other veteran populations or TMH care have lower dropout than that reported here. Future studies should focus on enhancing treatment engagement and retention in the OEF/OIF/OND veteran population.

Footnotes

Acknowledgements

This research was supported by the administration of Edward Hines Jr and William S Middleton VA Hospitals.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by U.S. Army Medical Research and Materiel Command: Award Number: W81XWH-08-2-0130.

References

Ramchand

Schell

Karney

. Disparate prevalence estimates of PTSD among service members who served in Iraq and Afghanistan: possible explanations. J Trauma Stress 2010; 23(1): 59–68.

Stecker

Fortney

Owen

. Co-occurring medical, psychiatric, and alcohol-related disorders among veterans returning from Iraq and Afghanistan. Psychosomatics 2010; 51(6): 503–507.

Hermes

Rosenheck

Desai

. Recent trends in the treatment of posttraumatic stress disorder and other mental disorders in the VHA. Psychiatr Serv 2012; 63(5): 471–476.

Seal

Maugen

Cohen

. VA mental health service utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. J Trauma Stress 2010; 23(1): 5–16.

Duckart

O’Malley

. Correlates of utilization of PTSD specialty treatment among recently diagnosed veterans at the VA. Psychiatr Serv 2011; 62(8): 943–949.

Erbes

Curry

Leskela

. Treatment presentation and adherence of Iraq/Afghanistan era veterans in outpatient care for posttraumatic stress disorder. Psychol Serv 2009; 6(3): 175–183.

Chard

Schumm

Owens

. A comparison of OEF and OIF veterans and Vietnam veterans receiving cognitive processing therapy. J Trauma Stress 2010; 23(1): 25–32.

Hailey

Roine

Ohinmaa

. The effectiveness of telemental health applications: A review. Can J Psychiatry 2008; 53(11): 769–778.

Frueh

Deitsch

Santos

. Procedural and methodological issues in telepsychiatry research and program development. Psychiatr Serv 2000; 51: 1522–1527.

10.

Trott

Blignault

. Cost-evaluation of a telepsychiatry service in Northern Queensland. J Telemed Telecare 1998; 4: 66–68.

11.

Gros

Morland

Greene

. Delivery of evidence-based psychotherapy via video telehealth. J Psychopathol Behav Assess 2013; 35(4): 506–521.

12.

Sloan

Gallagher

Feinstein

. Efficacy of telehealth treatments for posttraumatic stress-related symptoms: a meta-analysis. Cognit Behav Ther 2011; 40(2): 111–125.

13.

Richardson

Frueh

Grubaugh

. Current directions in videoconferencing tele-mental health research. Clin Psychol Sci Pract 2009; 16(3): 323–338.

14.

Greene

Morland

Durkalski

. Noninferiority and equivalence designs: issues and implications for mental health research. J Trauma Stress 2008; 21: 433–439.

15.

Thorp

Fidler

Moreno

. Lessons learned from studies of psychotherapy for posttraumatic stress disorder via video teleconferencing. Psychol Serv 2012; 9(2): 197–199.

16.

Strachan

Gros

Yuen

. Home-based telehealth to deliver evidence-based psychotherapy in veterans with PTSD. Contemp Clin Trials 2012; 33(2): 402–409.

17.

First

Spitzer

Gibbon

. Structured clinical interview for DSM-IV-TR axis I disorders, research version, patient edition with psychotic screen (SCID-I/P W/ PSY SCREEN), New York: Biometrics Research, New York State Psychiatric Institute, November 2002.

18.

Morland

Mackintosh

Greene

. Cognitive processing therapy for posttraumatic stress disorder delivered to rural veterans via telemental health: a randomized noninferiority clinical trial. J Clin Psychiatry 2014; 75(5): 470–476.

19.

Morland

Hynes

Mackintosh

. Group cognitive processing therapy delivered to veterans via telehealth: a pilot cohort. J Trauma Stress 2011; 24(4): 465–469.

20.

Morland

Greene

Rosen

. Issues in the design of a randomized noninferiority clinical trial of telemental health psychotherapy for rural combat veterans with PTSD. Contemp Clin Trials 2009; 30: 513–522.

21.

Blake

Weathers

Nagy

. The development of a clinician-administered PTSD scale. J Trauma Stress 1995; 8: 75–90.

22.

Weathers FW and Ford J. Psychometric review of the PTSD checklist. In: Stamm BH (ed.) Measurement of stress, trauma and adaptation. Baltimore: Sidran Press, 1996, pp. 250–251.

23.

Beck

Steer

Ball

. Comparison of Beck depression inventories -IA and -II in psychiatric outpatients. J Pers Assess 1996; 67(3): 588–597.

24.

Horvath

Greenberg

. Development and validation of the working alliance inventory. J Couns Psychol 1989; 36: 223–233.

25.

Resick

Monson

Chard

. Cognitive processing therapy: veteran/military version, Boston: National Center for PTSD, 2007.

26.

Monson

Schnurr

Resick

. Cognitive processing therapy for veterans with military-related posttraumatic stress disorder. J Consult Clin Psychol 2006; 74(5): 898–907.

27.

Garcia

Kelley

Rentz

. Pretreatment predictors of dropout from cognitive behavioral therapy for PTSD in Iraq and Afghanistan war veterans. Psychol Serv 2011; 8(1): 1–11.