First,Do No Harm: Referring Primary Care Patients with Depression to an Internet Support Group

Abstract

Background:

Internet Support Groups (ISGs) offer people easy access to information regarding depression as well as support from others who are either currently suffering from depression or have previously suffered from depression. The safety and efficacy of ISGs for people with depression have not been thoroughly studied.

Introduction:

The safety and helpfulness of a depression ISG were assessed by analyzing pre- and postintervention depressive symptoms, other psychological outcomes, and participant ratings of helpfulness.

Materials and Methods:

Participants were recruited through self-referral from six primary care offices. Participants were given access to a depression ISG and participated in an ISG for 6 weeks.

Results:

Thirty-four (n = 34) participants enrolled in the study (mean age = 32.53, standard deviation [SD] = 16.10). Depressive symptoms approached significance for decreasing over time and self-efficacy increased over time. No self-harm occurred over the course of the study, but two participants developed self-harm ideation. Ratings of ISG helpfulness were mixed.

Discussion:

Primary care patients participating in depression ISGs reported few adverse experiences directly related to the ISG. Depressive symptoms and self-efficacy have beneficial findings while ratings of helpfulness were mixed.

Conclusions:

Primary care patients can benefit from the use of an ISG. This could be particularly pertinent to people in rural settings where mental health resources are not as available. An ISG offers a low-cost and easily accessible resource for primary care patients with depression.

Introduction

Depressive disorders are a substantial problem for many people and are a primary cause of disability burden in developed countries.¹ It is estimated that less than half of those with depression seek help from a healthcare professional and even fewer receive appropriate treatment.^2,3 Both antidepressant medications and psychotherapy treat depression. However, the efficacy of antidepressants varies widely,⁴ the availability of effective psychotherapy can be limited, treatment can have high costs, and treatment attrition rates for psychotherapy are high.^5,6

The Internet has great potential for depression treatment. Increasing numbers of people use the Internet as a resource for mental health information and treatment.⁷ Benefits of Internet treatment include decreased stigmatization due to anonymity,⁸ cost-effectiveness,⁹ wide dissemination, easy access, and no requirement for highly trained mental health professionals.¹⁰ Furthermore, Internet interventions demonstrated efficacy in decreasing anxiety and depression symptoms.^11

–14

Internet Support Groups (ISGs) have demonstrated efficacy in improving patient outcomes for Alzheimer's caregivers,^15,16 AIDS patients,¹⁷ and cancer patients.¹⁸ Depression ISGs are the most common ISGs^19,20 and despite their availability, there is a paucity of studies investigating depression ISGs.^19
–21 The efficacy of depression ISGs is mixed with some studies showing improvement in depression outcome scores, while other studies do not show any improvement in depression outcome scores.^20,22,23 Depression ISGs are a special type of ISG that differs from ISGs for Alzheimer's caregivers, AIDS patients, and cancer patients as depression ISGs predominately focus on the mental health topic of depression and its associated symptoms and not on symptoms of a physical chronic illness. In addition, the popularity of depression ISGs combined with the limited research in this area necessitates investigation into depression ISGs. Furthermore, to our knowledge, there is no published literature on whether depression ISGs could support primary care treatment of depression and if depression ISGs could be safely used for this purpose.

We previously conducted the Psycho-Babble study, a randomized trial of the feasibility of referring primary care patients with depression to a depression ISG. We found that a patient-oriented brochure was more useful than a physician recommendation letter for increasing ISG participation.²⁴ The majority of depression ISG users have a depression history or are currently depressed.^20,21,24,25 A depression ISG goal is to provide a safe environment to access depression information and obtain peer support. Possible ISG use risks include (1) hurtful comments could be posted,²⁶ which could worsen the depression symptoms of those who read them or could encourage negative behaviors such as suicide attempts, (2) people could post inaccurate information, and (3) anecdotal personal stories about treatment may discourage or delay others from seeking treatment.²⁵ The popularity of ISGs combined with the relatively small number of studies investigating ISGs demands further investigation into the experiences of people with depression utilizing ISGs. The patient-to-patient model that is implicit in the ISG approach could provide a low-cost method of supporting recovery from depression. However, there is a need to know more about ISG potential harms.

This study examines the helpfulness and safety of the Psycho-Babble study described above.²⁴ We evaluate participant responses with regard to depressed mood, self-efficacy, self-harm ideation, upsetting postings, and perceived helpfulness. Our primary hypothesis is that participant depression scores would not worsen over time. Additional hypotheses over time measured that self-efficacy would increase, loneliness would decrease and social support would increase, self-harm ideation would not increase, hopelessness would not increase, participants would not report viewing upsetting or concerning content, and that the participants would find the ISG helpful.

Materials and Methods

Participants were recruited from six primary care offices in both urban and suburban areas. Participant (n = 34) recruitment and study details are more thoroughly described in the original Psycho-Babble study.²⁴ In brief, the Internet portal and the ISG were assessed using automated activity tracking of visits, minutes logged in, viewing of posts, and content posted. An e-mail reminder was sent to participants who had not visited the portal within 2 weeks of enrollment. We collected baseline and 6-week follow-up measures. Informed consent and IRB approval were obtained for this study.

Inclusion and Exclusion Criteria

Inclusion criteria were that the patient (1) has a Patient Health Questionnaire-9 (PHQ-9) score of 8 or above with depressed mood or anhedonia and is considering treatment for depression, (2) is of age 18 or older, (3) has visited a primary care clinic in the previous 6 months, and (4) has Internet access for the next 4 weeks, has been on the Internet at least three times, and has used e-mail by himself or herself. Exclusion criteria were that the patient (1) rejects all treatment for depression, (2) has viewed or posted on any ISG more than once in the previous month, (3) has symptoms of mania or hypomania or has been diagnosed with bipolar disorder, (4) has a history of psychiatric hospitalization, (5) has a history of attempting suicide, or (6) has a PHQ-9 suicidal ideation item score of 1 or above (thoughts that he or she would be better off dead or of hurting himself or herself at least several days over the last 2 weeks).

Safety Measures for Participants at Risk for Self-Harm

The existing Psycho-Babble adverse event reporting mechanism was available to participants. This was a link that allowed participants to complete an online form detailing the adverse event they experienced. Participants were asked to describe the event, to indicate any serious consequences of the event, and whether the adverse event was caused by their Psycho-Babble participation.²⁶ Additionally, each participant received a phone call 4–6 weeks after enrollment where the participant was asked if there were any concerns related to participating in the ISG.^24,26 Risk of self-harm was assessed using a protocol adapted from the assessment and treatment of suicidal behavior.²⁷ If that assessment indicated the participant was at risk of self-harm, the participant was referred to treatment in collaboration with their primary care provider.

Internet Portal Intervention

A portal (www.psycho-babble-study.org) was constructed to provide primary care patients with fact-based information on mental health as well as access to a well-established ISG. The fact-based information included content from the National Institute of Mental Health (NIMH)²⁸ and the MoodGYM²⁹ online depression intervention. The Internet portal called Psycho-Babble is described in detail in a prior publication.²⁴ The portal linked to preselected message boards on the ISG that two focus groups deemed most relevant to primary care patients.²⁴ After reading messages, participants could then choose to register at the ISG to post their own messages.

Psycho-Babble is a mental health peer support group started in 1998. Online mental health groups can be classified as autonomous self-help groups or support groups led by mental health professionals. Psycho-Babble is a hybrid group that combines the empowerment of self-help with the supportiveness of a milieu maintained by a mental health professional. The asynchronous online message board format makes the group more accessible and, in some ways, safer than groups that meet face-to-face.²⁶ The role of peer support in Psycho-Babble is that participants have the advantage of being able to receive advice and support from peers through the portal while maintaining anonymity. This occurs in a setting where interactions are monitored for civility.

Measurements

Depression

The PHQ-9 assessed current prevalence of depression (likely major depression, minor depression, or dysthymia) and core symptoms of depression in the last 2 weeks (0 = nearly every day, 1 = more than half the days, 2 = several days, or 3 = none at all).²⁷ In addition, the Center for Epidemiologic Studies Depression Scale 10-item measure (CES-D-10) was used. The CES-D-10 asked how often during the past week the participant had felt or behaved in certain ways (<1, 1–2, 3–4, or 5–7 days). The CES-D-10 yielded scores between 0 and 30.³⁰ For both measures, higher scores indicate greater depressive symptoms.

Self-efficacy

Self-efficacy was assessed using the Mastery Scale. This seven-item scale measures the extent to which participants see themselves as being in control of forces that affect their lives. Participants indicated how much they agreed with each statement using a Likert-type scale (1 = strongly disagree to 4 = strongly agree).³¹

Loneliness

Loneliness was measured using the loneliness item of CES-D-10. This item asks about the frequency of feeling lonely in the past week on a scale from 0 to 3 (0 = <1 day to 3 = most of the days).³⁰

Perceived social support

Perceived social support was measured using the Perceived Social Support from Friends scale (PSS-fr). The PSS-fr is a six-item scale that measures the extent to which an individual perceives that his/her needs for social support are being fulfilled by friends. Whether “friends” includes “online friends” was unspecified. The scale is Likert type: 1 = hardly ever, 2 = some of the time, or 3 = often.³² A higher score indicates more perceived social support.

Hopelessness

Hopelessness was measured using a single item from the PHQ-9, which asks, “In the last two weeks, how often have you felt hopeless about the future?” The participants rated their hopelessness using a 4-point Likert-type scale (0 = not at all to 3 = nearly every day).³³

Self-harm ideation

Self-harm was assessed using a single item from the PHQ-9, which asks, “In the last two weeks, how often have you had thoughts that you would be better off dead or of hurting yourself and/or others.” Participants reported self-harm ideation using a 4-point Likert-type scale (0 = not at all to 3 = nearly every day).³³ We created a dichotomous variable of either not at all or had any self-harm ideation.

Concerning content

Participants were asked to indicate if they found any of the content on the ISG concerning by indicating how much they agreed with the item “the Babble site brought up sad or angry feelings for me” using a 5-point Likert-type scale (1 = strongly disagree to 5 = strongly agree).

Program ratings

These ratings assessed four domains: (1) helpfulness of the ISG, (2) Psycho-Babble Web site specific, (3) emotional support, and (4) reciprocal learning. The helpfulness of the ISG refers to the usefulness, practicality, and benefits of using the ISG. Psycho-Babble Web site specific refers to how helpful the Web site is as well as how the Web site triggered emotional feelings. Emotional support refers to how much support participants felt they received and provided on the ISG. Reciprocal learning examines interactions between participants using the ISG. Participants indicated how much they agreed with statements for each domain using a scale ranging from 1 = strongly disagree to 5 = strongly agree and answered no/yes to reciprocal learning topics.

Demographics

Self-reported demographic information included age, gender, race/ethnicity, education level, marital status, income, whether participants discussed depression with their primary care physician, prior treatment for depression, whether anyone in the participant's family had been treated for depression, and whether the participant ever received counseling.

Statistical Analysis

Descriptive statistics were used to report study demographics and participant ratings. Inferential statistics of paired t tests were used to compare continuous variables. Missing data were analyzed using the independent samples t test for the continuous variable of age and the Pearson chi-square test for the categorical variable of gender.

Results

Demographics

Of the 49 agreeing to participate, 34 entered the trial and completed baseline questionnaires and 31 interacted with the depression ISG. There were 32 who completed at least one postenrollment measure, while there were 17 dropouts who did not complete even one postenrollment measure. Table 1 shows demographic information for the sample at baseline. Participants had a mean age of 32.53 years (±16.10), and 79.4% (27/34) were female. Just over half were nonwhite, over three-quarters had completed some college courses or were college graduates, and over three-quarters were never married. With regard to treatment history, less than half had previously talked to a primary care physician about depression. The sample participants who completed the postassessment measures were younger compared with those missing at follow-up (completed: M = 32.5 ± 12.9, missing: M = 45.9 ± 19.01, p = 0.007) and a greater percentage of females compared with those missing at follow-up, (completed: 79.4%, n = 27, missing: 37.5%, n = 6, p = 0.013).

Table 1.

Demographics of 34 Participants at Baseline

ITEMS	MEAN/%	SD/N
Age (years), (mean, SD)	32.53	6.10
Gender
Male	20.60	7
Female	79.41	27
Race/ethnicity
White	47.06	16
Asian	5.92	2
Black	38.24	13
Hispanic	2.94	1
Other	5.92	2
Education
Some high school	11.74	4
High school graduate	5.88	2
Some college	44.11	15
College graduate	38.23	13
Marital Status
Married	8.82	3
Divorced/separated/widowed	8.82	3
Never married	79.41	27
Missing	2.94	1
Talk to PCP
Yes	50.00	17
No	44.11	15
Missing	5.88	2
Treatment
Yes	44.11	15
No	32.35	11
Missing	23.53	8
Family treatment
Yes	21.41	10
No	67.65	23
Missing	2.94	1
Counseling
Yes	58.82	20
No	38.23	13
Missing	2.94	1
Income^a (USD) (n = 22), (mean, SD)	47,325.86	62,382.83

Twelve participants did not report income.

PCP, primary care physician; SD, standard deviation.

Depression and Depression-Related Measures

Depression and depression-related measures were collected at baseline and 6 weeks after enrollment. Table 2 shows that mean CES-D-10 scores showed improvement from pre- (M = 11.86 ± 0.92) to postassessment (M = 9.64 ± 1.42) with lower, but not statistically significant, mean depressive symptoms (p = 0.09). A statistically significant difference was observed between pre-and postassessment self-efficacy scale scores. Mean postintervention scores were higher (M = 17.17 ± 0.60) than at baseline (M = 15.83 ± 0.56) (p = 0.050). The other variables did not differ.

Table 2.

Depression and Depression-Related Outcomes

OUTCOME	ALPHA (PRE/POST)	N	MEAN (N) (PRE)	SD (%)	MEAN (N) (POST)	SD (%)	p
CES-D	0.87/0.91	14	11.86	0.92	9.64	1.42	0.09
PHQ-9	0.77/0.90	27	9.63	1.11	8.67	1.33	0.47
Self-Harm	n/a	30	(0)	(0)	(2)	(6.67)	n/a
Loneliness	n/a	15	1.20	0.22	1.00	0.24	0.51
Self-Efficacy	0.81/0.61	12	15.83	0.56	17.17	0.60	0. 050
Social Support	0.82/0.75	10	9.50	1.04	9.60	0.78	0.91
Hopelessness	n/a	34	1.68	0.18	1.47	0.16	0.36

Lower mean CES-D scores at post suggest depression symptom improvement. Higher mean self-efficacy scores at post indicate self-efficacy improvement.

Values in bold signify a statistically significant difference between pre- and post-assessment scores.

CES-D-10, Center for Epidemiologic Studies Depression Scale 10-item; n/a, not applicable; PHQ-9, Patient Health Questionnaire-9.

Self-Harm Ideation

Two participants developed self-harm ideation with PHQ-9 scores rising from 13 to 17, a, and from 14 to 25, b. Participant a reported becoming increasingly depressed as the result of the closure of her psychiatric clinic and being unable to find a new location of care. She was referred to social services for an immediate well-being check and if necessary to receive follow-up psychiatric care. Participant b reported self-harm thoughts related to family problems. The primary care physician for b was notified and referral services were offered. Both a and b went to the Web site, viewed at least one post, and spent 11 and 22 min, respectively.

At baseline, three individuals (c, d, and e) reported passive thoughts of death. At follow-up, two did not report self-harm ideation (c and d). However, neither demonstrated a decline in depressed mood, both starting and completing with PHQ-9 of 8/8 for c and 12/12 for d. Participant d reported concerns about employment loss as the main cause and that feelings of life not worth living were long-standing and mild with no specific current action intended. Participant e had PHQ-9 = 10 at baseline and was not available for follow-up. All three participants (c, d, and e) visited the Web site, but did not view posts and spent 2, 1, and 1 min, respectively, on the Web site.

Participant ISG Ratings

Table 3 shows that for all helpfulness items, the mean scores were in between disagree and neutral. The Psycho-Babble Web site-specific item that the Web site brought up, sad or angry feelings, had a mean score of disagree. The mean scores for the emotional support items approached neutral.

Table 3.

Ratings of Program

STATEMENTS	STRONGLY DISAGREE (%)	N	DISAGREE (%)	N	NEITHER (%)	N	AGREE (%)	N	STRONGLY AGREE (%)	N	MEAN (SD)	N
Helpfulness
Received practical advice about taking depression medications	11.11	2	22.22	4	44.44	8	11.11	2	11.11	2	2.89 (1.13)	18
Felt more uncertain about my future	11.11	2	38.89	7	38.89	7	5.56	1	5.56	1	2.56 (0.98)	18
Could not find anyone to communicate with who was similar	16.67	3	11.11	2	44.44	8	22.22	4	5.56	1	2.89 (1.13)	18
Became more confused about my depression treatment choices	11.11	2	50	9	27.78	5	5.56	1	5.56	1	2.44 (0.98)	18
Could not communicate with someone when I needed them	5.88	1	41.18	7	35.29	6	11.76	2	5.88	1	2.71 (0.99)	17
Psycho-Babble Web site Specific
The babble site brought up sad or angry feelings for me	31.82	7	50	11	9.09	2	4.55	1	4.55	1	2.00 (1.02)	22
I found another Internet site listed on the Babble study site helpful	36.84	7	36.84	7	21.05	4	5.26	1	n/a	n/a	1.95 (0.91)	19
Emotional Support
Internet support group helps me deal with my problems	8.33	2	29.17	7	29.17	7	29.17	7	4.17	1	2.92 (1.06)	24
Other members of the group care about me	8.33	2	12.5	3	62.5	15	16.67	4	n/a	n/a	2.88 (0.80)	24

Table 4 shows that the yes responses for reciprocal learning ranged from 13.04% to 39.13%. The three statements that received the highest yes responses were offered advice to anyone about problems, ISG member gave you help in solving a problem, and ISG member felt in a similar situation.

Table 4.

Reciprocal Learning

RECIPROCAL LEARNING	YES (%)	N
Internet support group member gave you information	16.67	24
Internet support group member felt in a similar situation	20.83	24
Offered advice to anyone about problems	39.13	23
Encouraged another Internet depression support group member	13.04	23
Information provided to another participant about something related to depression	13.04	23
Internet support group member helped you with another approach	17.39	23
Internet support group member gave you help in solving a problem	31.82	22

Discussion

CES-D depressive symptom scores approached significance for decreasing over time. PHQ-9 depressive symptom scores had nonsignificant decreases over time. Self-efficacy scores increased over time. Loneliness and social support did not change over time. Self-harm and hopelessness did not increase over time. Participants did not indicate becoming upset and on average disagreed with the statement, “The babble site brought up sad or angry feelings for me.” Postintervention ratings of the site's helpfulness were mixed with scores ranging from disagree to neutral.

A statistically significant difference was observed between pre-and poststudy self-efficacy scores, with participants indicating higher levels of self-efficacy at poststudy. No statistically significant difference was observed between pre-and poststudy CES-D scores; however, a downward trend in CES-D scores was observed, indicating small improvements in depression symptoms. This downward trend supports our hypothesis that depressed mood would not worsen over time. An important part of the investigation was trying to determine if a depression ISG for primary care patients was a safe way to help alleviate feelings of depression. The downward trend in CES-D scores over time indicates an improvement in symptoms. Although this was not statistically significant, this lends some support to the safety of a depression ISG as depression symptoms did not worsen over time. This is consistent with the results of previous studies on ISGs.^20,34 It should be noted that the ISGs used in those two studies^20,34 differed in their content and from the monitoring system used in our study. Other investigators have found no significant difference in pre- and postintervention scores for people utilizing an ISG for depression.^22,23 Additionally, there were statistically significant differences observed for age and gender between the participants who provided poststudy outcome scores and those who did not. This could have resulted in more favorable outcomes being reported with younger and more technology-savvy individuals completing all assessments. Similarly, the skew toward females may limit our understanding of how men responded.

Our hypothesis that self-harm ideation would not increase was supported by our data. At baseline, 3/34 participants reported self-harm ideation. Two did not report continuing self-harm ideation at follow-up, while the third could not be contacted at follow-up. The finding that self-harm ideation did not continue for two of the three participants who initially reported self-harm ideation offers support with regard to the safety of an ISG. Furthermore, hopelessness did not increase. Our study suggests that a depression ISG can be safely conducted when an appropriate safety protocol and procedures are put in place and these safety protocols and procedures are utilized to address safety issues as they arise. Our finding may not be representative of other ISGs as our study had active monitoring for adverse events and self-harm, as well as an established protocol for when participants had increases in self-harm ideation. It would be useful to compare the rates and outcomes between depression ISGs that have active monitoring and a protocol for increases in self-harm ideation with those depression ISGs that do not. Such research would allow for the eventual goal of establishing a consistent and effective safety protocol across depression ISGs. The monitoring system used in the ISG for this study may represent a key function for making the depression ISG safe to use. These findings are encouraging and are in line with other ISG studies that found improvements with reduced CES-D depression symptoms.^20,22

Loneliness and social support did not change. This is surprising as ISGs offer easy access to a supportive community. One previous study found that emotional support was the most popular reason for using an ISG.²⁰ It is possible that not all individuals find increased social support and decreased loneliness through a depression ISG.

Participants did not report viewing concerning or upsetting content. On average, participants disagreed with the statement, “the Babble site brought up sad or angry feelings,” and only 1/22 strongly agreed with that statement. Previous research with an ISG found that participants felt their interactions with the group improved their symptoms²⁰ and participants also indicated that they felt they could discuss subjects that they could not discuss elsewhere.²⁶ Our study does not offer much support for ISG helpfulness as participants either disagreed or were neutral in their ISG helpfulness ratings. This differs from a previous study in which half felt they could discuss subjects they were uncomfortable discussing elsewhere.²⁵

There are several study limitations. First, the study had a small sample size. Second, we could not determine if the more actively engaged participants were as depressed as those participants who were not posting comments. Thus, it is difficult to conclude if ISGs are effective for a wide range of depression severity or if they are better suited for those with milder depression symptomology. Third, the lack of a control group makes it impossible to determine if participants using the ISG were at higher risk for self-harm ideation than people not undergoing any intervention. Fourth, there is no way to tell if the safety protocol used in this study increases the safety of using the ISG compared with ISGs with less stringent protocols and monitoring systems. Fifth, the assessments occurred only 6 weeks after starting the ISG, which is a shorter time duration than many individuals choose to interact with an ISG in nonresearch settings. The strengths of this study include a diverse sample population and the use of a wide range of assessments, which has not been simultaneously studied in any previous ISG study.

Conclusions

In conclusion, depression scores did not worsen over time, indicating that ISGs are potentially safe to use. Participants had statistically significant increases in self-efficacy and a downward trend was observed in participant CES-D scores. Taken as a whole, the ISG appears to be safe to use, but only had limited impact on depression outcome scores. Although no self-harm behavior occurred, two participants did report developing self-harm ideation. Participants did not find the content of the ISG concerning, and responses were mixed regarding the helpfulness of the ISG. Although the sample size for this study is small, the results warrant further investigation with a larger sample. Future research should investigate how to optimize utilization and participation in an ISG, what population is best served by an ISG (e.g., age, gender, depression severity, and education background), how to more effectively improve depression symptoms, and whether the incidence of suicide ideation of participants in an ISG is comparable with those undergoing other forms of treatment. Furthermore, future studies should investigate the efficacy of ISGs in the long term and how such results compare with traditional treatments for depression. The monitoring systems of other ISGs should be examined as well as their safety protocols to determine if they play an important role in preventing adverse events from occurring and, if so, how to standardize safety protocols across the existing depression ISGs. Finally, future research should examine who is at highest risk for self-harm ideation while using an ISG and how, if at all, material on the ISG may increase the risk.

Footnotes

Acknowledgments

This study was supported by NIMH R-34 R34 MH 073742-01 and NIMH K-08 MH 072918-01A2 (Dr. Van Voorhees' salary support during study).

Disclosure Statement

Dr. Van Voorhees has served as a consultant for Prevail Health Solutions, Inc., Mevident, Inc., Verimed, Inc., and Social Kinetics, Inc. The University of Chicago has granted a no-cost license to Mevident, Inc., to adapt the CATCH-IT intervention. Dr. Van Voorhees has agreed to support the company working 5.5 days as consultant at $1,000/day. Dr. Hsiung is the single member of Dr. Bob, LLC, which owns and operates the Psycho-Babble ISG. The net revenue of Dr. Bob, LLC, is less than $1000/year. All other authors have no competing financial interests.

References

Andrews

, Titov

. Depression is very disabling. Lancet, 2007; 370:808–809.

Andrews

, Issakidis

, Sanderson

, Corry

, Lapsley

. Utilising survey data to inform public policy: Comparison of the cost-effectiveness of treatment of ten mental disorders. Br J Psychiatry, 2003; 184:526–533.

Andrews

, Issakidis

, Carter

. Shortfall in mental health service utilisation. Br J Psychiatry, 2001; 179:417–425.

Fournier

, DeRubeis

, Hollon

, et al. Antidepressant drug effects and depression severity: A patient-level meta-analysis. JAMA, 2010; 303:47–53.

Weissman

, Verdeli

, Gameroff

, et al. National survey of psychotherapy training in psychiatry, psychology, and social work. Arch Gen Psychiatry, 2006; 63:925–934.

Haby

, Donnelly

, Corry

, Vos

. Cognitive behavioural therapy for depression, panic disorder and generalized anxiety disorder: A meta-regression of factors that may predict outcome. Aust N Z J Psychiatry, 2006; 40:9–19.

Taylor

, Luce

. Computer- and Internet-based psychotherapy interventions. Curr Dir Psychol Sci, 2003; 12:18–22.

Cuijpers

. Bibliotherapy in unipolar depression: A meta-analysis. J Behav Ther Exp Psychiatry, 1997; 28:139–147.

Christensen

, Griffiths

. The prevention of depression using the Internet. Med J Aust, 2002; 177:122–125.

10.

Hoek

, Schuurmans

, Koot

, Cuijpers

. Prevention of depression and anxiety in adolescents: A randomized controlled trial testing the efficacy and mechanisms of Internet-based self-help problem-solving therapy. Trials, 2009; 10:1–13.

11.

Andersson

, Bergstrom

, Carlbring

, Lindefors

. The use of the Internet in the treatment of anxiety disorders. Curr Opin Psychiatry, 2004; 18:1–5.

12.

Spek

, Cuijpers

, Nykicek

, Riper

, Keyzer

, Pop

. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychol Med, 2007; 37:319–328.

13.

Spek

, Nykicek

, Smits

, et al. Internet-based cognitive behaviour therapy for subthreshold depression in people over 50 years old: A randomized controlled clinical trial. Psychol Med, 2007; 37:1797–1806.

14.

Zetterqvist

, Maanmies

, Strom

, Andersson

. Randomized controlled trial of Internet-based stress management. Cogn Behav Ther, 2003; 32:151–160.

15.

Gallienne

, Moore

, Smyth

. Alzheimer's caregivers: Psychosocial support via computer networks. J Gerontol Nurs, 1993; 19:15–22.

16.

Brennan

, Moore

, Smyth

. The effects of a special computer network on caregivers of persons with Alzheimer's disease. Nurs Res, 1995; 44:166–172.

17.

Brennan

, Ripich

, Moore

. The use of home-based computers to support persons living with AIDS/ARC. J Community Health Nurs, 1991; 8:3–14.

18.

Gustafson

, Hawkins

, Pingree

, et al. Effect of computer support on younger women with breast cancer. J Gen Intern Med, 2001; 16:435–445.

19.

Davidson

, Pennebaker

, Dickerson

. Who talks? The social psychology of illness support groups. Am Psychol, 2000; 55:205–217.

20.

Houston

, Cooper

, Ford

. Internet support groups for depression: A 1-year prospective cohort study. Am J Psychiatry, 2002; 159:2062–2068.

21.

Griffiths

, Calear

, Banfield

. Systematic review on Internet support groups (ISGs) and depression (1): Do ISGs reduce depressive symptoms?. J Med Internet Res, 2009; 11:e40.

22.

Andersson

, Bergstrom

, Hollandare

, Carlbring

, Kaldo

, Ekselius

. Internet-based self-help for depression: Randomised controlled trial. Br J Psychiatry, 2005; 187:456–461.

23.

Ellis

, Campbell

, Sethi

, O'Dea

. Comparative randomized trial of an online cognitive-behavioral therapy program and an online support group for depression and anxiety. J Cyber Ther Rehabil, 2011; 4:461–467.

24.

Van Voorhees

, Hsiung

, Marko-Holguin

, et al. Internal versus external motivation in referral of primary care patients with depression to an Internet Support Group: Randomized controlled trial. J Med Internet Res, 2013; 15:e42.

25.

Powell

, McCarthy

, Eysenbach

. Cross-sectional survey of users of Internet depression communities. BMC Psychiatry, 2003; 3:19.

26.

Hsiung

. The best of both worlds: An online self-help group hosted by a mental health professional. Cyberpsychol Behav, 2000; 3:935–950.

27.

Linehan

. Assessment and treatment of suicidal behavior manual. Copy from Dr. Linehan at University of Washington, Seattle, 1998:98–105.

28.

Depression. National Institute of Mental Health website. Available at www.nimh.nih.gov/health/publications/depression/index.shtml (last accessed March 20, 2015 ).

29.

Christensen

, Grittiths

, Korten

. Web-based cognitive behavior therapy: Analysis of site usage and changes in depression and anxiety scores. J Med Internet Res, 2002; 4:e3.

30.

Radloff

. The CES-D scale: A self report depression scale for research in the general population. Appl Psychol Meas, 1977; 1:385–401.

31.

Pearlin

, Lieberman

, Menaghan

, Mullan

. The stress process. J Health Soc Behav, 1981; 22:37–356.

32.

Procidano

, Heller

. Measures of perceived social support from friends and from family: Three validation studies. Am J Community Psychol, 1983; 11:1–24.

33.

Kroenke

, Spitzer

, Williams

. The PHQ-9: Validity of a brief depression severity measure. J Intern Med, 2001; 16:606–613.

34.

Griffiths

, Mackinnon

, Crisp

, Christensen

, Bennett

, Farrer

. The effectiveness of an online support group for members of the community with depression: A randomised controlled trial. Plos One, 2012; 7:1–9.