Technology Use and Interest in Computerized Psychotherapy: A Survey of Veterans in Treatment for Substance Use Disorders

Abstract

Introduction: This study examined interest in computerized psychotherapies (CPTs) and its relation to use of information technology among individuals receiving Veterans Health Administration (VHA) outpatient treatment. Materials and Methods: Veterans receiving treatment in a VHA substance use disorder outpatient clinic completed a self-report questionnaire. The survey addressed recent experience using information technology and potential interest in using CPTs for symptoms/functional problems associated with substance use and mental health disorders. Demographic, diagnostic, and information technology use data were compared between those expressing interest in CPT and those not expressing an interest, as well as with nationally representative veteran data from the 2010 National Survey of Veterans (NSV). Results: Of 151 respondents, 82% were interested in CPT for at least one problem, and 60% were interested for more than one. The most commonly selected CPTs were for substance use (46%), depression (45%), problem solving (43%), and insomnia (42%). None of the 23 measures of information technology use was associated with interest in CPTs. Compared with respondents not interested in any CPTs, those interested in CPT were older (t ₁₅₀=2.1, p=0.042) and more likely to be African American [χ²(1)=8.8, p=0.032], to have reported a drug use disorder [χ²(1)=4.2, p=0.041], and to have reported more than one substance use or psychiatric disorder [χ²(1)=8.5, p=0.014]. The majority of respondents reported use of Internet and e-mail (65% and 64%, respectively), proportions comparable to respondents to the NSV. Conclusions: Among veterans receiving outpatient substance use treatment, interest in CPT is high and unrelated to information technology use. Efforts to implement CPTs may interest this population.

Introduction

In its 2001 report, Crossing the Quality Chasm, the Institute of Medicine specifically emphasized that the Internet and other information technologies should be used to augment and foster more continuous and timely health service delivery.¹ Health information technology (HIT) includes a broad range of applications used to store, retrieve, and share health information for communication and decision-making.² The expansion of such services offers a potentially unique opportunity to increase access to health services among traditionally underserved populations, such as individuals with substance use disorders.³ The Veterans Health Administration (VHA), the largest integrated healthcare system in the United States, has become a leader in many aspects of HIT with the use of a comprehensive electronic medical record, an online health information portal, My HealtheVet (MHV) (www.myhealth.va.gov), and videoconferencing-based healthcare.^4,5

Internet- or computer-based therapies for mental health and substance use disorders are one of the fastest growing areas of HIT. Computerized psychotherapies (CPTs) are interactive, often based on cognitive-behavioral therapeutic principles, frequently delivered over the Internet, and focused on fostering behavior change for problems such as substance use, insomnia, depression, and anxiety.^6,7 They present standardized yet individualized material in rich formats and provide a high degree of fidelity to evidence-based models at the pace and in a setting chosen by the user.⁸ In addition, such therapies may reduce (1) costs associated with therapist time, (2) avoidance of treatment due to stigma, and (3) impaired access due to lack of trained therapists in some locations, scheduling conflicts, and transportation barriers.^9,10 Although the subjects and formats vary among CPTs, evidence of their efficacy has been demonstrated in several systematic reviews.^11

–14

However, the magnitude of such benefits may be blunted by obstacles related to an individual's interest in CPT or access to and use of the technology used to engage in such programs, such as the hardware and Internet infrastructure, as well as the skills needed for HIT use.¹⁵ In the general adult population, individuals who are elderly, male, less educated, or members of racial or ethnic minorities or who have low socioeconomic status have been found to be less likely to use such technology than their counterparts.^16

–19

Given that VHA and other safety-net healthcare systems serve relatively large proportions of individuals who have substance use problems as well as other long-term health and functional problems such as chronic medical disorders, severe mental illnesses, or homelessness,^20,21 such barriers may be especially relevant to the implementation of HIT services such as CPTs in these contexts. Although similar sociodemographic disparities in HIT use found in the general population are also found among VHA service users, research indicates that there is no difference in HIT use among veterans who use VHA services, veterans who do not, and their nonveteran counterparts.^22
–24 In addition, there is some evidence that veterans who are in mental health treatment are no less likely to use Internet or MHV than those not in mental health treatment.²⁴ However, there is little information on the use of or interest in CPT among veterans with substance use disorders or on whether their current level of technology use affects this interest.

As VHA continues to develop CPT programs in preparation for their implementation and to expand its use of HIT in general,²⁵ there is a need to evaluate interest in such services and barriers among potential users. The current study surveyed interest in CPT and its relation to the use of information technology in a sample of VHA patients in treatment for substance use disorders. In addition, the rate of information technology use in this sample was compared with that in a national sample of veterans to determine the comparability of information technology use between veterans in substance use treatment and that of the general veteran population. Hypotheses were that interest in CPT would be low and associated with lower information technology use and selected barriers to use among veterans with substance use disorders compared with a national sample of veterans.

Materials and Methods

Sample

Participants were a convenience sample of 151 veterans participating in outpatient substance use treatment in the VA Connecticut Healthcare System (West Haven, CT) between November 1, 2012 and November 1, 2013 out of 1,431 veterans who had contact with the clinic during the period of recruitment. Participants obtaining outpatient methadone treatment for opioid dependence were excluded because this treatment takes place at a separate location. Research staff approached individual prospective participants in the clinic waiting room. Procedures for the anonymous, written, self-administered questionnaire were explained, and voluntary participation was elicited. Subjects received a description of the survey content and a definition of computer-based programs on a survey cover sheet. Of 191 individuals approached, 151 completed the survey (79.1% response rate). The institutional review boards of the VA Connecticut Healthcare System and the Yale University School of Medicine (New Haven, CT) approved a waiver of written informed consent for this study.

Corresponding data from participants in the substance use clinic were compared with technology use data from the 2010 National Survey of Veterans (NSV), a series of self-administered questionnaires completed by a nationally representative sample of noninstitutionalized veterans. The survey was mailed using address-based sampling to a total of 13,058 veterans, 8,710 of whom completed surveys, for a response rate of 66.7%. Sampling and weighting procedures have been described elsewhere and were applied to analyses including NSV data.²⁶

Measures

The clinic survey included multiple-choice questions regarding demographics as well as substance use and psychiatric diagnoses. Information technology use questions addressed current Internet and e-mail use for general as well as health-related purposes, as well as equipment used to access the Internet. Current Internet and/or e-mail use was measured by two dichotomous questions in addition to multiple-choice questions addressing the frequency, location, and purpose of current Internet/e-mail use. The survey also contained questions addressing use of the Internet to access health-related information and specifically the use of the MHV Web site. Veterans were also queried as to the type of equipment (e.g., computer, smartphone) used to access the Internet and perceived barriers (e.g., computer literacy, lack of finances, privacy concerns) to Internet use. A separate set of questions addressed interest in engaging in CPT for various problems (“Would you be interested in computer programs to help you with any of the following?” followed by a list of eight problems that could potentially be addressed by available CPTs).

Analysis

Responses were tabulated and analyzed by frequency and measures of central tendency. Differences in aggregate responses between veterans who expressed interest in using computerized therapy and those who did not were compared and evaluated using chi-squared and t tests. Responses to seven questions on information technology use from substance use clinic patients were compared with those of the NSV sample and tested using chi-squared and t tests.

Results

Table 1 shows that a majority of veterans in the clinical sample were male, were white, had a high school education, and had a mean age of 52 years (standard deviation=10.9 years). Compared with the NSV sample, the clinical sample was on average 9 years younger (t _8,643=6.8, p<0.001) and included a 19% lower proportion of individuals with a college or advanced degree [χ²(4)=40.0, p<0.001], as well as a 26% higher proportion of African Americans [χ²(1)=96.5, p<0.001], a lower proportion of whites, and similar proportions of Hispanics and those of another race or ethnicity. In the clinical sample, individuals reporting only an alcohol or drug use disorder were equal in number (n=40, 27%), whereas approximately one-third reported both disorders. Two-thirds reported a comorbid psychiatric disorder.

Table 1.

Demographic and Diagnostic Characteristics of a Sample of Veterans in Outpatient Treatment for Substance Use Disorders Compared with a National Sample of Veterans Responding to the 2010 National Survey of Veterans

VARIABLE	SUBSTANCE USE TREATMENT (N=151)	NSV (N=8,710)	TEST STATISTIC^a	DF	P VALUE
Age [weighted mean (weighted SD)]	52.0 (10.9)	61.1 (15.8)	6.8	8,643	<0.001
Gender (male) [n (weighted %)]	139 (94.6)	7,279 (91.9)	1.4	1	0.244
Race/ethnicity [n (weighted %)]
White	81 (53.6)	7,672 (85.3)	137.5	1	<0.001
Black/African American	57 (37.8)	603 (11.5)	96.5	1	<0.001
Hispanic	10 (6.6)	389 (5.3)	0.6	1	0.456
Other^b	3 (2.0)	332 (3.8)	1.4	1	0.236
Education [n (weighted %)]			40.0	4	<0.001
Partial high school	9 (6.0)	510 (5.4)
High school graduate/GED	68 (45.0)	2,202 (26.0)
Partial college/no degree	49 (32.5)	2,464 (30.0)
College graduate	20 (13.3)	2,289 (26.9)
Master's/doctoral/professional degree	5 (3.3)	1,075 (11.6)
Substance use disorder diagnosis [n (%)]
Alcohol use disorder only	40 (26.5)
Drug use disorder only	40 (26.5)
Both alcohol and drug use disorder	48 (31.8)
Neither disorder	23 (15.2)
Psychiatric disorder diagnosis [n (%)]
Major depression	61 (40.4)
Posttraumatic stress disorder	47 (31.1)
Other anxiety disorder	41 (27.2)
Bipolar disorder	32 (21.2)
Psychotic disorder	11 (7.3)
Other psychiatric disorder	3 (2.0)
None	51 (33.8)

The t test was used for comparison of means of continuous variables (age); the chi-squared test was used for comparison of categorical variables.

Category of “other” contains Asian, American Indian, Alaskan Native and “other,” which were all less than 3% of the respective samples.

df, degrees of freedom; NSV, National Survey of Veterans; SD, standard deviation.

When presented with a list of potential problems to be addressed by CPT, 124 respondents (82%) selected at least one problem for which they expressed interest in CPT treatment, whereas 91 (60%) selected more than one (Table 2). Respondents most frequently expressed interest in computerized therapy for drug or alcohol use (n=70, 47%), followed closely by therapies for sleep problems/insomnia, depression, and general problem solving, all selected by between 42% and 45% of veterans surveyed. Among those selecting multiple therapies, the most common additional therapy selected was for depression.

Table 2.

Interest in Computerized Therapy in a Sample of Veterans in Outpatient Treatment for Substance Use Disorders (n=151)

VARIABLE	NUMBER (%) IN SUBSTANCE USE TREATMENT
“Would you be interested in computer programs to help you with any of the following?”
Selecting any therapy	124 (82.1)
Selecting more than one therapy	91 (60.3)
Specific therapies
Alcohol or drug use	70 (46.4)
Depression	68 (45.0)
General problem solving	65 (43.1)
Sleep problems/insomnia	64 (42.4)
Social problem/isolation	56 (37.1)
Managing money more efficiently	45 (29.8)
Posttraumatic stress disorder	44 (29.1)
Remembering to take my medication	17 (11.3)

Over half of the clinic sample described using the Internet and/or e-mail at least occasionally (n=98 [65%] and n=97 [64%], respectively), proportions that were not statistically significantly different from responses to the same questions by the national sample of respondents to the NSV (Table 3). However, among Internet users in the clinical sample, only 76% (n=74) stated they used the Internet at least once a day or once a week, compared with 92% in the NSV sample [χ²(1)=34.0, p<0.001]. In addition, the location of Internet and e-mail use differed between the groups, with relatively less use at home and more use in other locations among the substance use clinic sample (Table 3). A relatively large number of clinic patients (n=40, 29%) indicated they had used the MHV Web site, a much larger proportion than those reporting MHV use in the NSV sample (3.2%) [χ²(1)=260.6, p<0.001], which included a large portion of veterans not using VHA services. Regarding the respondent's willingness to use the Internet for specific purposes, the proportion of clinic patients selecting “very willing” or “somewhat willing” was greatest for “obtaining information about VHA benefits” and “applying for VHA benefits” (n=98 [74%] and n=95 [71%], respectively). Regarding owning equipment with which to access the Internet/e-mail, only 70 clinic veterans (46%) indicated that they owned a computer or smartphone. When the sample was queried about health communication over the Internet or e-mail, the most commonly reported activity was “communicating with friends or family about health or healthcare.” When asked about barriers to Internet use, the most common barrier selected by the clinic sample was “don't know how to use a computer and/or type” (n=32, 21%), followed by “financial barriers” (n=24, 16%).

Table 3.

Health Information Technology Use Characteristics of a Sample of Veterans in Outpatient Treatment for Substance Use Disorders Compared with a National Sample of Veterans Responding to the 2010 National Survey of Veterans

VARIABLE	SUBSTANCE TREATMENT (N=151)	NSV (N=8,710)	χ²	DF	P VALUE
“Do you use the Internet, at least occasionally” [n (wt %)]	98 (65.3)	5,898 (72.3)	3.4	1	0.058
“How often do you access the Internet or World Wide Web?” [n (wt %)]^a			34.0	1	<0.001
“At least once a day” or “once a week but not every day”	74 (75.5)	4,997 (91.9)
“At least once a month…,” “once a year…,” “Less than once a year”	24 (24.5)	471 (8.6)
“Where do you go on-line to use the Internet?” [n (wt %)]^b
Home	54 (55.1)	5,225 (93.4)	219.8	1	<0.001
Some place other than home	43 (43.9)	309 (6.6)	203.9	1	<0.001
“Do you send or receive emails, at least occasionally?” [n (wt %)]	97 (69.8)	5,573 (70.6)	0.1	1	0.832
“Where do you go on-line to send or receive e-mails?” [n (wt %)]^c
Home	52 (47.7)	5,320 (93.7)	340.7	1	<0.001
Some place other than home	44 (45.8)	323 (6.3)	228.1	1	<0.001
“Have you ever used the MyHealtheVet Web site…” [n (wt %)]	40 (29.4)	297 (3.2)	260.6	1	<0.001
“How willing are you to use the Internet for these activities?” [n (wt %)]
Obtain information about VA benefits (“very or somewhat willing”)	98 (74.2)	5,109 (68.8)	1.8	1	0.180
Apply for VA benefits (“very or somewhat willing”)	95 (71.4)	4,797 (65.6)	2.0	1	0.161
Obtaining news and information (“very or somewhat willing”)	91 (68.9)	5,566 (74.1)	1.8	1	0.182
Researching goods and services (“very or somewhat willing”)	87 (65.9)	5,430 (73.6)	3.9	1	0.048
Purchasing goods and services (“very or somewhat willing”)	66 (50.0)	4,595 (62.7)	9.0	1	0.003
Respond to polls (“very or somewhat willing”)	44 (33.3)	3,180 (44.4)	6.4	1	0.011
Use of equipment to access the Internet
Computer or laptop	55 (36.4)
Smartphone	37 (24.5)
“In the past year, how often did you…” (once a week or more than once a week)^c
Look on the Internet for information or advice about healthcare?	20 (18.5)
Use e-mail/Internet to communicate with
A doctor or other healthcare provider?	7 (6.6)
Family/friend about health/healthcare?	26 (24.1)
Others with similar health concerns	9 (8.4)
“If you do not use the Internet, what are the barriers/reasons?”
Don't know how to use a computer and/or type	32 (21.2)
Financial barriers	24 (15.9)
Not helpful/not needed	13 (8.6)
Privacy concerns	10 (6.6)
Transportation	2 (1.3)
“Would you like to learn more about using a computer?” (yes)	102 (67.6)

Proportion of those stating they use the Internet.

Proportion of those stating they use the Internet, and more than one selection could be made.

Proportion of those stating they use the Internet or send and receive e-mail.

df, degrees of freedom; NSV, National Survey of Veterans; VA, Veterans Administration; wt %, weighted percentage.

When clinic respondents who expressed interest in CPT were compared with those who did not, statistically significant differences emerged for some demographic and diagnostic variables but not for any of the variables addressing technology use, Internet use, or perceived barriers to use. The group interested in CPT were on average 5 years older (t ₁₅₀=2.1, p=0.042), more likely to be African American [χ²(1)=8.8, p=0.032], and more likely to report a diagnosis of drug use disorder or more than one substance use or psychiatric disorder [χ²(1)=4.2, p=0.041 and χ²(1)=6.6, p=0.011, respectively]. When factors associated with of interest in the four most commonly selected CPT targets (substance use, depression, insomnia, and problem solving) were examined separately, statistically significant differences primarily reflected corresponding diagnostic considerations and not measures of computer/Internet use.

Discussion

In this survey of veterans with substance use disorders using VHA outpatient treatment services, there was frequently expressed interest in CPTs, as demonstrated by the fact that 82% of the sample indicated they were interested in at least one psychotherapy for a specific psychiatric and/or functional problem, most commonly substance use, depression, general problem solving, or insomnia, whereas 60% selected more than one. It is surprising that not one of the 23 measures describing current technology, Internet, or e-mail use or measures of barriers to information technology use was significantly associated with interest in participating in CPT. These findings support the potential for implementing CPTs as a treatment option among VHA patients in substance use treatment.

Few prior studies have directly evaluated interest in computerized therapies, and none specifically among individuals in substance use treatment. Such interest is likely a key factor in engagement and an eventual determinant of successful implementation of CPTs.²⁷ For comparison, a recent survey of older veterans revealed that over half expressed interest in using the Internet to track goals as well as manage anxiety and depression symptoms.²⁵ Similar surveys among individuals complaining of anxiety in the United Kingdom found similar results: 90% reported interest in computerized anxiety treatment in one study, and in another, one-half of individuals over the age of 65 years with depression and/or anxiety disorders indicated interest in CPT.^28,29 In addition, the latter studies also support the current finding that individuals interested in CPT were on average 5 years older than those who were not interested in CPT. Prior studies have also shown that disadvantaged populations for which there may be barriers to HIT use are just as willing to participate in computerized therapies as those who have more access.^30,31 These findings have also been bolstered by other studies suggesting that CPTs in general are as acceptable as face-to-face therapy and that the implementation of such therapies in clinical settings can be effective.^32,33

The second main finding in this study is that measures of current technology use and perceived barriers to HIT use were not associated with interest in CPT among veterans in substance use treatment. This finding encourages making CPTs available to clinic populations with substance use, as systematic barriers such as current use of and access to HIT do not appear to diminish interest, which is likely a key factor in the successful implementation of such programs. In addition, interest in CPT was significantly higher among individuals with a dual diagnosis, both an alcohol or drug use disorder, as well as older individuals and African Americans, all groups that many have thought would experience barriers that would discourage interest in using HIT.³ These findings suggest that interest in this form of service delivery is not limited by complex mental health problems.

Few other studies have queried individuals directly about perceived barriers to the use of computerized interventions or HIT. This study found that not knowing how to use a computer or to type was the most commonly selected barrier to technology use, similar to a study of individuals with serious mental illness where expense, lack of computer skills, difficulty with typing, and difficulty with reading all emerged as relevant barriers.³ Similarly, a Web-based survey of student perceptions of barriers to online learning revealed that lack of technical skills was a commonly selected barrier.³⁴ However, this finding still encourages the implementation of CPT in this group as selection of this barrier was not associated with reduced interest in participating in CPT, and it may be mitigated with proper support and training. Healthcare organizations such as VHA may be able to provide training programs that can fill gaps in technical training for people with little HIT experience.

However, trends in HIT use for the general population have been documented that may represent barriers to the widespread implementation of CPT. Technology ownership, access to an Internet connection, and Internet use are commonly but not consistently found to be lower in older age groups, minorities, and residents of rural locations, as well as those with lower levels of education, limited literacy, and low socioeconomic status.^{15,16,18,23,35
–37} For individuals with mental illness, evidence of such barriers are mixed, with evidence of significant barriers found in some studies,³ whereas others have found that individuals with chronic disease, mental health problems, and other stigmatizing conditions are actually more likely than others to use the Internet for health information.^9,38,39

For VHA service users, sociodemographic correlates of general and health-related Internet use are similar to those found in the general population. The NSV found that over 65% of veterans using VHA services also used the Internet, a proportion similar to that found among veterans who did not use VHA services and the nonveteran population, a finding replicated in this survey of veterans in VHA substance use treatment.^22,24 When use of the VHA's e-health portal, through which veterans can access their health record, manage prescriptions, and contact providers, was evaluated, 20% of NSV veterans in VHA care reported using the site,²⁴ a proportion that did not differ from the sample of veterans with substance use and mental health disorders examined in this study. These results indicate that similar systematic barriers to technology use likely exist for the general population, those with mental health and substance use disorders, and veterans. Although these barriers may affect eventual participation in CPT, they do not appear to affect interest in CPT among VHA service users.

Several limitations of this study deserve comment. As this is the only such survey of which the authors are aware and it occurred in a relatively small convenience sample at a single site, the generalizability of the results is unknown. In addition, the question that addressed the principal variable of interest was “Would you be interested in computer programs to help you with the following problems?” Respondents were left to define for themselves the meaning of “interested” and “computer programs.” However, we feel that this question, the associated list of specific functional and psychiatric problems, and the description of computer-based programs we provided supplied respondents with clear direction in this context. Although five of the most important potential barriers to information technology use were queried in this survey, this list may not have contained all potential personal and institutional barriers. Furthermore, interest in CPT may be related to prior of use of and provider attitudes toward such programs, and future surveys should include questions regarding these issues. In addition, future research that goes beyond self-reported interest and investigates the association between interest and uptake and engagement in such therapies are needed. Moreover, we were unable to identify individuals who completed the NSV and were also in treatment for substance use disorders for a more direct comparison. However, the proportion of current information technology users in this survey was not dissimilar from that of the 2010 NSV sample, although the treatment sample consisted of younger individuals, a higher proportion of African Americans, and those with less education.

CPTs are fast growing and wide-ranging and offer opportunities, in concert with face-to-face therapies, for lowering the cost of disease management among individuals with substance use disorders who use health service within and beyond VHA. Data on the efficacy of such programs are also promising.⁷ However, CPTs have yet to be systematically implemented in VHA and similar safety-net healthcare systems, especially among those with substance use disorders. This survey of veterans in treatment for substance use disorders found that 82% of individuals expressed interest in CPT for a specific psychiatric and/or functional problem, and the findings support making CPTs available as a potentially feasible option for enhancing the care of veterans and others with substance use disorders.

Footnotes

Acknowledgments

We would like to Kevin Payne, BS, for his support in the distribution and collections of survey materials. The research outlined here was supported by the Department of Veterans Affairs, Veterans Health Affairs, VISN 1 Career Development Award V1CDA2012-17 to E.D.A.H. and by the VA New England Mental Illness Research, Education, and Clinical Center, West Haven, CT.

Disclosure Statement

E.D.A.H. and J.T. report no competing financial interests. R.R. has received research support from Janssen Pharmaceutica Products and Wyeth Pharmaceuticals within the last year, in addition to AstraZeneca Pharmaceuticals LP, Bristol-Myers Squibb, and Eli Lilly and Co. in the past. He has received consulting fees from Bristol-Myers Squibb, Eli Lilly and Co., Roche Pharmaceuticals, and Janssen Pharmaceutica Products. He is a testifying expert in Jones ex rel. the State of Texas v. Janssen Pharmaceutica Products.

References

Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academies Press, 2001 .

Thompson

, Brailer

. The decade of health information technology: Delivering consumer-centric and information-rich health care. Washington, DC: U.S. Department of Health and Human Services, 2004 .

Borzekowski

, Leith

, Medoff

, et al. Use of the Internet and other media for health information among clinic outpatients with serious mental illness. Psychiatr Serv, 2009; 60:1265–1268.

Chumbler

, Haggstrom

, Saleem

. Implementation of health information technology in Veterans Health Administration to support transformational change: Telehealth and personal health records. Med Care, 2011; 49(Suppl):S36–S42.

Deen

, Godleski

, Fortney

. A description of telemental health services provided by the Veterans Health Administration in 2006–2010. Psychiatr Serv, 2012; 63:1131–1133.

Barak

, Proudfoot

. Defining internet-supported therapeutic interventions. Ann Behav Med, 2009; 38:4–17.

Carroll

, Ball

, Martino

, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: A randomized trial of CBT4CBT. Am J Psychiatry, 2008; 165:881–888.

Newman

, Consoli

, Taylor

. Computers in assessment and cognitive behavioral treatment of clinical disorders: Anxiety as a case point. Behav Ther, 1997; 28:211–235.

Berger

, Wagner

, Baker

. Internet use and stigmatized illness. Soc Sci Med, 2005; 61:1821–1827.

10.

Kenwright

, Liness

, Marks

. Reducing demands on clinicians by offering computer-aided self-help for phobia/panic feasibility study. Br J Psychiatry, 2001; 179:456–459.

11.

Wieland

, Falzon

, Sciamanna

, et al. Interactive computer-based interventions for weight loss or weight maintenance in overweight or obese people. Cochrane Database Syst Rev, 2012; 8:CD007675.

12.

Kiluk

, Sugarman

, Nich

, et al. A methodological analysis of randomized clinical trials of computer-assisted therapies for psychiatric disorders: Toward improved standards for an emerging field. Am J Psychiatry, 2011; 168:790–799.

13.

Kaltenthaler

, Parry

, Beverley

, Ferriter

. Computerised cognitive-behavioural therapy for depression: Systematic review. Br J Psychiatry, 2008; 193:181–184.

14.

Kaltenthaler

, Brazier

, De Nigris

, et al. Computerised cognitive behaviour therapy for depression and anxiety update: A systematic review and economic evaluation. Health Technol Assess, 2006; 10:1–186.

15.

Eng

, Maxfield

, Patrick

, Deering

, Ratzan

, Gustafson

. Access to health information and support. JAMA, 1998; 280:1371–1375.

16.

Baker

, Wagner

, Singer

, Bundorf

. Use of the internet and e-mail for health care information. JAMA, 2003; 289:2400–2406.

17.

Chang

, Bakken

, Brown

, et al. Bridging the digital divide: Reaching vulnerable populations. J Am Med Inform Assoc, 2004; 11:448–457.

18.

Hsu

, Huang

, Kinsman

, et al. Use of e-health services between 1999 and 2002: A growing digital divide. J Am Med Inform Assoc, 2005; 12:164–171.

19.

Laz

, Berenson

. Racial and ethnic disparities in Internet use for seeking health information among young women. J Health Commun, 2013; 18:250–260.

20.

Gamache

, Rosenheck

, Tessler

. Overrepresentation of women veterans among homeless women. Am J Public Health, 2003; 93:1132–1136.

21.

Gamache

, Rosenheck

, Tessler

. The proportion of veterans among homeless men: A decade later. Soc Psychiatry Psychiatr Epidemiol, 2001; 36:481–485.

22.

Houston

, Volkman

, Feng

, Nazi

, Shimada

, Fox

. Veteran Internet use and engagement with health information online. Mil Med, 2013; 178:394–400.

23.

McInnes

, Gifford

, Kazis

, Wagner

. Disparities in health-related internet use by US veterans: Results from a national survey. Inform Prim Care, 2010; 18:59–68.

24.

Tsai

, Rosenheck

. Use of the internet and an online personal health record system by US veterans: Comparison of Veterans Affairs mental health service users and other veterans nationally. J Am Med Inform Assoc, 2012; 19:1089–1094.

25.

Crabb

, Rafie

, Weingardt

. Health-related internet use in older primary care patients. Gerontology, 2011; 58:164–170.

26.

Department of Veterans Affairs. National survey of veterans, active duty service members, demobilized National Guard and Reserve members, family members, and surviving spouses. Rockville, MD: Westat, 2010 .

27.

Kaltenthaler

, Sutcliffe

, Parry

, Beverley

, Rees

, Ferriter

. The acceptability to patients of computerized cognitive behaviour therapy for depression: A systematic review. Psychol Med, 2008; 38:15210–15213.

28.

Elsegood

, Powell

. Computerised cognitive-behaviour therapy (cCBT) and older people: A pilot study to determine factors that influence willingness to engage with cCBT. Counsel Psychother Res, 2008; 8:189–192.

29.

Graham

, Franses

, Kenwright

, Marks

. Psychotherapy by computer. A postal survey of responders to a teletext article. Psychiatr Bull, 2000; 24:331–332.

30.

Gustafson

, Hawkins

, Boberg

, et al. CHESS: 10 years of research and development in consumer health informatics for broad populations, including the underserved. Int J Med Inform, 2002; 65:169–177.

31.

Tsai

, Klee

, Rosenheck

, Harkness

. Internet use among veterans with severe mental illness. Psychiatr Serv, 2014; 65:564.

32.

Kaltenthaler

, Parry

, Beverley

. Computerized cognitive behaviour therapy: A systematic review. Behav Cogn Psychother, 2004; 32:31–55.

33.

Marks

, Mataix-Cols

, Kenwright

, Cameron

, Hirsch

, Gega

. Pragmatic evaluation of computer-aided self-help for anxiety and depression. Br J Psychiatry, 2003; 183:57–65.

34.

Tyler-Smith

. Early attrition among first time elearners: A review of factors that contribute to drop-out, withdrawal and non-completion rates of adult learners undertaking elearning programmes. J Online Learn Teach, 2006; 2:73–85.

35.

Atkinson

, Saperstein

, Pleis

. Using the internet for health-related activities: Findings from a national probability sample. J Med Internet Res, 2009; 11:4.

36.

Choi

. Relationship between health service use and health information technology use among older adults: Analysis of the US National Health Interview Survey. J Med Internet Res, 2011; 13:e33.

37.

Jensen

, King

, Davis

, Guntzviller

. Utilization of internet technology by low-income adults: The role of health literacy, health numeracy, and computer assistance. J Aging Health, 2010; 22:804–826.

38.

Haviland

, Pincus

, Dial

. Datapoints: Type of illness and use of the internet for health information. Psychiatr Serv, 2003; 54:1198–1198.

39.

Powell

, Clarke

. Internet information-seeking in mental health: Population survey. Br J Psychiatry, 2006; 189:273–277.