A Pilot Trial of Telephone-Based Collaborative Care Management for PTSD Among Iraq/Afghanistan War Veterans

Abstract

Background: Collaborative care and care management are cornerstones of Primary Care-Mental Health Integration (PC-MHI) and have been shown to reduce depressive symptoms. Historically, the standard of Veterans Affairs (VA) collaborative care was referring patients with posttraumatic stress disorder (PTSD) to specialty care. Although referral to evidence-based specialty care is ideal, many veterans with PTSD do not receive such care. To address this issue and reduce barriers to care, VA currently recommends veterans with PTSD be offered treatment within PC-MHI as an alternative. The current project outlines a pilot implementation of an established telephone-based collaborative care model—Translating Initiatives for Depression into Effective Solutions (TIDES)—adapted for Iraq/Afghanistan War veterans with PTSD symptoms (TIDES/PTSD) seen in a postdeployment primary care clinic. Materials and Methods : Structured medical record extraction and qualitative data collection procedures were used to evaluate acceptability, feasibility, and outcomes. Results: Most participants (n=17) were male (94.1%) and white (70.6%). Average age was 31.2 (standard deviation=6.4) years. TIDES/PTSD was successfully implemented within PC-MHI and was acceptable to patients and staff. Additionally, the total number of care manager calls was positively correlated with number of psychiatry visits (r=0.63, p<0.05) and amount of reduction in PTSD symptoms (r=0.66, p<0.05). Overall, participants in the pilot reported a significant reduction in PTSD symptoms over the course of the treatment (t=2.87, p=0.01). Conclusions: TIDES can be successfully adapted and implemented for use among Iraq/Afghanistan veterans with PTSD. Further work is needed to test the effectiveness and implementation of this model in other sites and among veterans of other eras.

Introduction

Approximately one-fifth of Veterans Affairs (VA)-enrolled Iraq/Afghanistan veterans have been diagnosed with posttraumatic stress disorder (PTSD), which makes PTSD the most commonly diagnosed mental disorder among Iraq/Afghanistan veterans seeking VA care.^1,2 Despite need, fewer than 10% of Iraq/Afghanistan veterans with PTSD attend the nine or more sessions (in ≤15 weeks) recommended for treating PTSD, and only one-third receive care in a VA clinic that specializes in empirically supported treatments for PTSD,³ likely because of barriers (e.g., stigma, transportation and time, trust in mental health professionals).^3,4 Given these issues with specialty care engagement, primary care-based treatment may be needed.

To help address these issues, VA has implemented Primary Care-Mental Health Integration (PC-MHI) teams within primary care clinics.^5
–7 A cornerstone of PC-MHI is the co-located collaborative care model, including a care management program.^6,8 Collaborative care has been shown to reduce depressive symptoms among VA primary care patients.⁹ Collaborative care explicitly enhances communication, coordination, and the referral process between primary care and mental healthcare providers through the use of care managers (CMs). CMs, typically nurses, are supervised by mental health specialists and assist primary care providers (PCPs) in assessing and treating mental health symptoms. CMs track symptoms, educate patients about symptoms and related treatments, and provide follow-up contact to maintain connection between patient and provider. CMs typically conduct these activities over the phone, reducing travel burdens for patients and conserving hospital space.

Despite the promise of collaborative care, VA's directive to offer comprehensive mental health services within PC-MHI,⁶ and the fact that the primary VA collaborative care models assess for PTSD symptoms as part of routine care,^8,10 the use of these models among veterans with PTSD has been limited. Likely because of factors such as providers lacking confidence to treat PTSD within primary care,¹¹ collaborative care algorithms have typically recommended referral of patients with PTSD to specialty clinics. Thus, although integration of mental health services into primary care has improved PTSD detection,⁷ PTSD symptoms are often undertreated in this setting.¹² Given inadequate specialty mental health treatment engagement for Iraq/Afghanistan veterans with PTSD,³ care coordination models would likely optimize treatment for those whose symptoms can be managed in primary care and encourage and facilitate referral to specialty mental healthcare for those with more severe symptoms.¹⁰

Primary care-based anxiety treatment is acceptable to and effective for civilians.¹³ Additionally, adapting collaborative care models to include PTSD-specific tools appears to be feasible and associated with clinical improvements among military service members immediately postdeployment. One study found that a model of collaborative care (RESPECT-Mil) was positively viewed by patients and providers and that a majority of patients who participated in treatment for 6 weeks or longer experienced significant improvements in PTSD and depressive symptoms.¹⁴ A randomized controlled trial of a newly implemented collaborative care model adapted for veterans with PTSD (RESPECT-PTSD) was completed by the National Center for PTSD investigators; participants were more likely to fill antidepressant prescriptions and to have a mental health visit but did not experience mental health symptom improvements.¹¹ The present study builds on prior work to determine how to best deliver collaborative care within VA for veterans with PTSD.

To complement VA and U.S. Department of Defense efforts, this pilot study evaluated the adaptation of an existing collaborative care model for management of depression in primary care to be applied among Iraq/Afghanistan veterans with PTSD: Translating Initiatives for Depression into Effective Solutions (TIDES).⁸

The primary objective of this study was to assess the acceptability and feasibility of TIDES/PTSD. Acceptability was measured by assessing provider and patient satisfaction with various elements of the program (e.g., telephone delivery, content of calls, coordination with providers). Feasibility was examined based on whether we were able to successfully implement TIDES/PTSD (i.e., establish CM practices, update the CM scripts and templates, and enroll and follow veterans) and providers' ratings of program components' feasibility following its completion. As a secondary aim, we examined pre–post changes in symptoms and associations with mental health engagement; it was hypothesized that the degree of participation in TIDES/PTSD would be associated with significantly reduced PTSD symptoms from baseline to follow-up and with mental healthcare utilization.

Materials and Methods

Tides/Ptsd Adaptation

Within VA, there are two CM-based collaborative care programs recommended for PC-MHI teams: Behavioral Health Lab¹⁰ and TIDES. The latter is an evidence-based, telephone-delivered CM program designed to support PCPs caring for veterans with depressive disorders within primary care.⁸ The TIDES protocol involves monthly telephone calls that are carried out by nurse CMs. TIDES calls are guided by a medical record-based template and script that (1) assesses current psychiatric symptoms and medication adherence and side effects, (2) provides patient education and self-management and psychosocial support, and (3) facilitates treatment referrals when indicated. TIDES CMs form an important therapeutic bridge among the patient, the PCP, and a supervising psychiatrist, who aids in making treatment recommendations to the PCPs.⁸ CMs also guide patients in brief, self-directed interventions such as behavioral activation. Below is a portion of the script provided to CMs for use during follow-up telephone calls:

Last time we talked about ways you can help yourself to feel better, such as planning more pleasurable activities into your life. Do you remember the plan you made at the time? How is that going? Have you been doing the activities you set for yourself? If not, what do you think interfered with your intentions? Maybe we can come up with a new plan today that feels better to you.

With the PC-MHI initiative resulting in increased presence of co-located mental health providers (e.g., psychologists) in primary care, CMs may also work directly with these providers.

In the current study, TIDES protocols, tools, and electronic medical record templates were modified to include content relevant to PTSD medication treatment (e.g., prazosin), psychosocial information, and symptom management. For example, information regarding typical PTSD symptoms such as hyperarousal and emotional numbing were assessed and discussed. To supplement CM telephone calls, all participants were provided with a printed copy of Strategies for Managing Stress After War: Veteran's Workbook and Guide to Wellness, a therapist-guided bibliotherapy tool designed for patients who are unwilling or unable to attend mental health services for PTSD.¹⁵ This tool was selected because it provides education regarding the common ways that posttraumatic problems present (such as via behavioral difficulties with families and friends), promotes risk-reduction, and encourages follow-through with referrals consistent with best-practice prevention guidelines for posttraumatic stress.¹⁶ Based on symptom presentation, CMs directed veterans to read specific chapters that provided behavioral and cognitive modification coping techniques (e.g., progressive relaxation, cognitive restructuring).

Participants

Twenty participants enrolled (response rate, 62.5%), and 17 completed at least one CM call. Of these, average age was 31.2 (standard devation [SD]=6.4) years, and 94.1% were male. Most had served in the Army (62.5%) and were white (70.6%). The remaining participants were black (11.8%), Asian (5.9%), Native Hawaiian or Pacific Islander (5.9%), and biracial (5.9%). The current relationship status of participants was as follows: 37.5% married, 12.5% living with a significant other, 18.8% separated or divorced, and 31.3% never married.

Procedure

Feasibility and acceptability of TIDES/PTSD were tested in a VA postdeployment health clinic providing integrated primary care and mental health services to postcombat veterans. Participants were referred to the pilot study by their PCPs and/or mental health providers if the patient expressed interest and the provider believed the patient would benefit from TIDES/PTSD, at which point eligibility was ascertained, and written informed consent was obtained. To be enrolled, veterans had to have deployed to Iraq and/or Afghanistan and report at least subthreshold PTSD symptoms (≥35 on the PTSD Checklist–Military Version [PCL-M]^17,18). Exclusion criteria included prominent suicidal or homicidal ideation, psychotic symptoms, and active substance abuse. Participants enrolled in the study between August 2008 and May 2009. CMs followed participants for an average of 4.23 months (range, 0.5–10 months). This study was approved by the local Institutional Review Board.

Measures

CMs entered data directly into participants' electronic medical records using TIDES/PTSD templates, which are medical record notes that guide the CM through a series of predetermined questions, measures (e.g., the PCL-M), and basic interventions (e.g., encouraging medication compliance), with the latter being based on the participants' responses to questions. Once the template is completed, it is stored in the veteran's electronic medical record as a note that can be accessed by VA providers. Templated notes allow for standardization of procedures and ease data extraction. All quantitative measures were extracted from the veterans' medical records using standardized procedures. Process of care measures included psychotherapy and psychiatry utilization, no-shows (scheduled appointments that the patient did not attend), and CM contact (i.e., number of TIDES/PTSD telephone sessions in which the veteran participated). The primary clinical outcome was PTSD symptom severity, measured with the PCL-M. Depression was assessed with the Patient Health Questionnaire-9 (PHQ-9).¹⁹ Initial and last clinical measures obtained during the study were used for pre–post comparison. Analyses were performed in SPSS version 19 software (2010 release; IBM Corp., Armonk, NY).

Qualitative data collection procedures to evaluate feasibility and acceptability of the adapted model included a mixture of open-ended questions administered in a semistructured interview using an interview guide and 5-point Likert-scale ratings; all qualitative materials and methods were developed by the research team. A team of two trained evaluators, one serving as the interviewer and one taking notes, conducted approximately 30-min semistructured interviews with a purposive sample of clinical staff (n=7) and patients (n=9). Clinical staff included CMs and primary care and mental health personnel. Patients were identified to maximize the range of demographics and the extent of follow-up received (i.e., both high-involvement and low-involvement participants). Patient interviews and rating scales focused on satisfaction with the program, as did staff interviews, which also asked about issues pertaining to feasibility of implementation. The rating scales were prompted with “How practical is [model component] to put into place?” to assess feasibility and “How desirable or useful is [model component]?” to assess acceptability. Rating scales were summarized using averages. After each interview, interview notes were reviewed by both members of the team for accuracy and completeness.

Results were analyzed using a simple grid in Microsoft (Redmond, WA) Excel^® so that common themes (e.g., compatibility with PC-MHI, refinement recommendations) across respondents could be identified and summarized. Interviewers addressed discrepancies in thematic interpretations through discussion and review.

Results

Table 1 contains data regarding associations among mental healthcare utilization, CM contact, and clinical outcomes. Total CM contact was strongly associated with number of psychiatry visits. Neither psychotherapy utilization nor no-shows was significantly correlated with CM contact. CM contact was strongly associated with reduction in PTSD symptoms. The average initial PCL-M rating was 53.65 (SD=10.71), and the average follow-up rating was 46.76 (SD=15.71), a significant reduction [t=2.87 (df=16), p=0.011]. Results were comparable when those with at least three CM contacts were selected [mean score difference=9.21 (SD=9.24); t=3.73 (df=13), p=0.003]. Average CM contact was not statistically associated with reduction in depression symptoms among those with comorbid depression at baseline (PHQ-9≥10; n=13). However, their depression decreased. The average initial PHQ-9 score was 14.69 (SD=3.47), and average follow-up PHQ-9 was 11.38 (SD=3.91), a significant reduction [t=2.20 (df=12), p=0.048]. Difference in PHQ-9 score was not statistically different when those who engaged in at least three CM contacts were selected [mean score difference=3.09 (SD=5.87); t=1.75 (df=10), p=0.111).

Table 1.

Associations Among Mental Healthcare Utilization, Care Manager Calls, and Posttraumatic Stress Disorder and Depression Symptom Rating Reduction

		CORRELATION (R)^a WITH
VISIT TYPE	FULL SAMPLE [MEAN (SD)]	CM CALLS	PCL-M PRE–POST SCORE CHANGE	PHQ-9 PRE–POST SCORE CHANGE^b
CM calls	4.53 (2.24)	—	0.660^c	−0.436
Psychiatry visits	1.47 (1.33)	0.625^c	0.318	−0.500
Psychotherapy visits	3.71 (6.56)	−0.010	−0.038	0.072
No shows	1.24 (1.79)	−0.049	0.355	0.197

Correlations calculated with Pearson's r.

Analyses run among those with depression at baseline (≥10 on the Patient Health Questionnaire-9 [depression] [PHQ-9]; n=13).

p<0.01.

CM, care manager; PCL-M, Posttraumatic Stress Disorder Checklist–Military Version; SD, standard deviation.

Qualitative Findings

Table 2 presents clinical staff members' ratings of feasibility and acceptability regarding TIDES/PTSD program components. Staff ratings were consistently high; the lowest ranked component was the feasibility of using the patient workbook. In semistructured interviews, all staff interviewed reported that the model integrated well with existing clinic processes and that it was a useful augmentation tool for promoting patient engagement and improving access. Several also noted that the program enhanced the PC-MHI mission. Several providers recommended increased interface between CMs and clinicians and increased flexibility of veteran contact hours. A summary of participant ratings and semistructured interviews is presented in Table 3. Participants were generally satisfied with TIDES/PTSD. The lowest ranked component was health education. All veterans said they would recommend TIDES/PTSD to others, and two already had. All veterans recommended that the model of care be continued. Four recommended that the program refine protocols to reduce redundancy, modify assessment scales, and increase individualization.

Table 2.

Component Ratings by Clinical Staff Following Translating Initiatives for Depression into Effective Solutions/Posttraumatic Stress Disorder Adaptation (n=7)

PROGRAM COMPONENT	FAMILIAR WITH COMPONENT (N)	AVERAGE FEASIBILITY RATING	AVERAGE SATISFACTION RATING
Medication follow-up	5	4.4	5.0
Patient workbook	7	3.7	4.6
Patient self-management support	4	4.8	5.0
Psychosocial support	4	5.0	5.0
Other patient education and support	4	4.3	4.8
Screening for depression	6	5.0	5.0
Monitoring patient progress	5	4.4	5.0
Facilitating communication among team	5	4.8	4.8

Average feasibility and desirability ratings include only respondents who indicated familiarity with the component. Staff were asked to rate the components on a scale of 1–5, where 1=poor, 2=fair, 3=moderate, 4=good, and 5=excellent.

Table 3.

Summary of Participant Satisfaction with Translating Initiatives for Depression into Effective Solutions/Posttraumatic Stress Disorder Adaptation (n=9)

PROGRAM COMPONENT (RATING SCALE)	AVERAGE RATING
Medication support	4.7
Health education	3.8
Health self-management	4.3
General concern and interest	4.3
Assistance in communication with providers	4.6

SEMISTRUCTURED INTERVIEW FEEDBACK	% ENDORSED
Comfortable with being called	100%
Felt comfortable with frequency of calls	89%
Program should continue	100%
Would recommend to others	100%

Patients were asked to rate the components on a scale of 1–5, where 1=poor, 2=fair, 3=moderate, 4=good, and 5=excellent.

Discussion

Findings indicate that a telephone-based collaborative care model can be adapted and implemented for use among Iraq/Afghanistan veterans with PTSD, in line with other studies of collaborative care treatment for anxiety disorders.^13,14 The TIDES/PTSD telephone-based collaborative care model was acceptable to patients and staff, receiving consistently high satisfaction ratings. Moreover, the program was viewed as feasible by providers across a wide variety of program components, templates were successfully adapted, and the program was successfully integrated into VA PC-MHI care. This smooth integration into existing treatment protocols is notable given the mixed findings regarding implementation of collaborative care for PTSD in military and VA primary care settings. Engel et al.¹⁴ reported successful implementation of a telephone-based collaborative care program for PTSD among military service members, but Schnurr et al.¹¹ noted significant difficulties in implementation, accompanied by poor effectiveness outcomes. They hypothesized that the poor outcomes were related to the observed higher referral rate to specialty mental healthcare and poorer overall fidelity to the treatment protocol compared with studies assessing similar treatments for depression. They reported that PCPs' confidence in addressing PTSD symptoms within primary care was lower than for depression, likely contributing to the poor implementation. The authors in both studies recommended further evaluation on implementation and efficacy of similar protocols. The current study represents a promising pilot of an alternative method that may lend itself to effective integration into existing systems of care.

Care management is a cornerstone of collaborative care. Psychiatry visits and total CM contact were highly correlated, suggesting that collaborative care may promote psychiatry engagement. Otherwise, no significant associations with utilization were identified. TIDES/PTSD may need to be modified to maximize its ability to coordinate and promote engagement with mental healthcare. Findings also may reflect that veterans already engaged in behavioral therapy are receiving adequate care management from other sources (e.g., providers of evidence-based in-person psychotherapy), but that collaborative care provides an important safety net for those otherwise not receiving psychotherapeutic interventions.

Telephone-based collaborative care may also serve as a safety net for veterans, such as those residing in rural areas, who find it difficult to engage in mental health services because of distance.^3,20,21 In our sample those living further from VA facilities engaged in more TIDES/PTSD calls, whereas those living closest to VA facilities engaged in more in-person psychotherapy (data not shown). Thus, telephone-based collaborative care may prove useful in promoting care for Iraq/Afghanistan veterans with PTSD facing barriers due to the time and costs of travel. Identifying treatments that address such geographic barriers is essential, given that living further from VA facilities interferes with receiving needed mental healthcare services for Iraq/Afghanistan veterans.³ As such, TIDES/PTSD may prove a useful model for improving care for those living further from VA facilities because travel is not required for telephone-based care.

Clinical data suggest TIDES/PTSD was associated with improvement in PTSD. Combat-related PTSD tends to be chronic without treatment,^22,23 and, as such, it is unlikely this improvement was entirely due to spontaneous recovery. A randomized controlled trial will be necessary to draw firm conclusions regarding the efficacy of the treatment. Although overall depression ratings decreased at follow-up among those with baseline depression, symptom reduction was not associated with CM contact.

Limitations

Several limitations warrant mention; study findings should be interpreted with caution. The sample consisted of Iraq/Afghanistan veterans seen in a VA postdeployment clinic, which may limit ability to generalize findings to other VA clinics and/or other veteran subpopulations. For this small pilot study we were unable to include a comparison group of usual care; without one, it is unclear whether improvements in PTSD can be attributed to TIDES/PTSD. Symptom measurement may be subject to bias; because PCL-M ratings were assessed during CM calls, symptom change for those participating in few calls may be underestimated. Several variables that would have provided rich information (e.g., prior mental health treatment, the conflict in which they served [i.e., Iraq, Afghanistan, or both]) were not assessed in this study. Because patients were not randomly assigned to a variable number of sessions, examining outcomes based on number of sessions attended is biased, as patient characteristics that predict both treatment adherence and outcomes cannot be teased apart. Only one woman participated in this study, despite substantial efforts to recruit women; future studies of TIDES/PTSD should require recruitment of women. Finally, sample size was small, thereby limiting ability to test for statistical significance, which may have led to spurious statistical results.

Conclusions

This study adds to prior evidence that telephone-based care is beneficial for symptom management and the treatment of depressive and anxiety disorders^{13,14,24
–26} and that it can be delivered with good adherence to treatment protocols.²⁷ Within VA, telemental health has an increasingly prominent role.²⁸ Telephone-based care is poised to address veteran geographic access to care barriers in a cost-effective manner by reducing the need to travel for care.²⁹ In an effort to provide an empirical basis for this large expansion of services, this study provides preliminary findings supporting the feasibility, acceptability, and effectiveness of an adapted telephone-based collaborative care model for Iraq/Afghanistan veterans with PTSD. The following modifications to TIDES/PTSD may be needed to be responsive to staff and patient suggestions and to improve care coordination: (1) clearer processes for CMs making recommendations to providers, (2) a referral management protocol to identify patients appropriate for care management and those needing specialty care, and (3) enhanced and tailored self-management strategies (e.g., exercise assessment and recommendations) and use of Web-based and phone-based materials. Further work is needed to rigorously test effectiveness and implementation of TIDES/PTSD in general VA primary care settings, among veterans from all eras. Such work should be conducted with larger samples and use appropriate comparison groups to more clearly estimate the impact of TIDES/PTSD. This model is poised to improve functioning of veterans with PTSD and may hold promise for non-VA setting implementation.

Footnotes

Acknowledgments

This material is based on work supported by the Office of Research and Development, U.S. Department of Veterans Affairs (grant SHP08-184).

Disclosure Statement

No competing financial interests exist.

References

Cohen

, Gima

, Bertenthal

, Kim

, Marmar

, Seal

. Mental health diagnoses and utilization of VA non-mental health medical services among returning Iraq and Afghanistan veterans. J Gen Intern Med, 2010; 25:18–24.

Seal

, Metzler

, Gima

, Bertenthal

, Maguen

, Marmar

. Trends and risk factors for mental health diagnoses among Iraq and Afghanistan veterans using Department of Veterans Affairs health care, 2002–2008. Am J Public Health, 2009; 99:1651–1658.

Seal

, Maguen

, Cohen

, et al. VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. J Trauma Stress, 2010; 23:5–16.

Vogt

. Mental health-related beliefs as a barrier to service use for military personnel and veterans: A review. Psychiatr Serv, 2011; 62:135–142.

Post

, Metzger

, Dumas

, Lehmann

. Integrating mental health into primary care within the Veterans Health Administration. Fam Syst Health, 2010; 28:83–90.

U.S. Department of Veterans Affairs. Uniform mental health services in VA Medical Centers and Clinics. 2008. Available at www.mirecc.va.gov/VISN16/docs/UMHS_Handbook_1160.pdf (last accessed October 12, 2013 ).

Zivin

, Pfeiffer

, Szymanski

, et al. Initiation of Primary Care-Mental Health Integration programs in the VA Health System: Associations with psychiatric diagnoses in primary care. Med Care, 2010; 48:843–851.

Rubenstein

, Chaney

, Ober

, et al. Using evidence-based quality improvement methods for translating depression collaborative care research into practice. Fam Syst Health, 2010; 28:91–113.

Hedrick

, Chaney

, Felker

, et al. Effectiveness of collaborative care depression treatment in Veterans' Affairs primary care. J Gen Intern Med, 2003; 18:9–16.

10.

Tew

, Klaus

, Oslin

. The Behavioral Health Laboratory: Building a stronger foundation for the patient-centered medical home. Fam Syst Health, 2010; 28:130–145.

11.

Schnurr

, Friedman

, Oxman

, et al. RESPECT-PTSD: Re-engineering systems for the primary care treatment of PTSD, a randomized controlled trial. J Gen Intern Med, 2013; 28:32–40.

12.

Grubaugh

, Magruder

, Waldrop

, Elhai

, Knapp

, Frueh

. Subthreshold PTSD in primary care: Prevalence, psychiatric disorders, healthcare use, and functional status. J Nerv Ment Dis, 2005; 193:658–664.

13.

Roy-Byrne

, Craske

, Sullivan

, et al. Delivery of evidence-based treatment for multiple anxiety disorders in primary care: A randomized controlled trial. JAMA, 2010; 303:1921–1928.

14.

Engel

, Oxman

, Yamamoto

, et al. RESPECT-Mil: Feasibility of a systems-level collaborative care approach to depression and post-traumatic stress disorder in military primary care. Mil Med, 2008; 173:935–940.

15.

Whealin

, DeCarvalho

, Vega

. Strategies for managing stress after war: Veteran's workbook and guide to wellness. Hoboken, NJ: John Wiley & Sons, Inc., 2008.

16.

VA/DoD Clinical Practice Guideline Working Group. Management of post-traumatic stress. 2010. Available at www.healthquality.va.gov/Post_Traumatic_Stress_Disorder_PTSD.asp (last accessed July 5, 2011 ).

17.

National Center for PTSD. PTSD Checklist (PCL). 2010. Available at www.ptsd.va.gov/professional/pages/assessments/assessment-pdf/pcl-handout.pdf (last accessed October 10, 2013 ).

18.

Weathers

, Litz

, Herman

, Huska

, Keane

. The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility. Presentation at the Annual Convention of the International Society for Traumatic Stress Studies, San Antonio, TX, 1993.

19.

Kroenke

, Spitzer

, Williams

. The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med, 2001; 16:606–613.

20.

Cully

, Jameson

, Phillips

, Kunik

, Fortney

. Use of psychotherapy by rural and urban veterans. J Rural Health, 2010; 26:225–233.

21.

Mohamed

, Neale

, Rosenheck

. VA intensive mental health case management in urban and rural areas: Veteran characteristics and service delivery. Psychiatr Serv, 2009; 60:914–921.

22.

Orcutt

, Erickson

, Wolfe

. The course of PTSD symptoms among Gulf War veterans: A growth mixture modeling approach. J Trauma Stress, 2004; 17:195–202.

23.

Zlotnick

, Warshaw

, Shea

, Allsworth

, Pearlstein

, Keller

. Chronicity in posttraumatic stress disorder (PTSD) and predictors of course of comorbid PTSD in patients with anxiety disorders. J Trauma Stress, 1999; 12:89–100.

24.

DuHamel

, Mosher

, Winkel

, et al. Randomized clinical trial of telephone-administered cognitive-behavioral therapy to reduce post-traumatic stress disorder and distress symptoms after hematopoietic stem-cell transplantation. J Clin Oncol, 2010; 28:3754–3761.

25.

Rollman

, Belnap

, LeMenager

, et al. Telephone-delivered collaborative care for treating post-CABG depression: A randomized controlled trial. JAMA, 2009; 302:2095–2103.

26.

Williams

Jr , Gerrity

, Holsinger

, Dobscha

, Gaynes

, Dietrich

. Systematic review of multifaceted interventions to improve depression care. Gen Hosp Psychiatry, 2007; 29:91–116.

27.

McKay

, Van Horn

, Oslin

, et al. A randomized trial of extended telephone-based continuing care for alcohol dependence: Within-treatment substance use outcomes. J Consult Clin Psychol, 2010; 78:912–923.

28.

Godleski

, Darkins

, Peters

. Outcomes of 98,609 U.S. Department of Veterans Affairs patients enrolled in telemental health services, 2006–2010. Psychiatr Serv, 2012; 63:383–385.

29.

Bee

, Bower

, Lovell

, et al. Psychotherapy mediated by remote communication technologies: A meta-analytic review. BMC Psychiatry, 2008; 8:60.