Use of a smartphone application to screen for bipolar spectrum disorder in a community sample

Abstract

This study describes the use of a smartphone application based on the Korean version of the Mood Disorder Questionnaire in screening for bipolar spectrum disorders in a large general population. All data were collected between May 2011 and July 2011. A total of 27,159 individuals participated in the survey, using a smartphone application. The prevalence of positive screening results for bipolar spectrum disorders among 27,159 participants using the smartphone Korean Mood Disorder Questionnaire application was 8.2 percent. These results are similar to traditional paper-based results. The Korean Mood Disorder Questionnaire positive group exhibited more frequent occurrences of previous psychiatric treatment than the group with negative results. In a logistic regression analysis involving subjects with past psychiatric history, age group significantly predicted Korean Mood Disorder Questionnaire results. The smartphone application may be a useful screening tool for bipolar spectrum disorders. This study included only individuals who actively participated, and thus, the possibility of a selection bias should be considered.

Keywords

bipolar spectrum disorder Korean Mood Disorder Questionnaire mobile health screening smartphone

Introduction

Bipolar disorder is a serious and relatively common psychiatric illness that is clinically associated with significant morbidity and mortality rates. The prevalence of typical bipolar I disorder is approximately 1 percent of the general population. However, recent research has suggested that bipolar spectrum disorders (BSD), including bipolar I disorder, bipolar II disorder, cyclothymic disorder, and bipolar disorder, not otherwise specified, are considerably more prevalent (3.0%–6.5%).^1,2

The problems of misdiagnosis and underdiagnosis of BSD have been highlighted previously.^1,3 Studies of depressed patients have found evidence that BSD patients are often misdiagnosed with other psychiatric disorders, particularly major depressive disorder; in addition, patients may be misdiagnosed with personality disorders, psychotic disorders, and anxiety disorders.^4,5 Even among patients who are accurately diagnosed with BSD, the latency between the first experiences of symptoms and the correct diagnosis is often greater than 10 years,^6,7 and the long duration of the untreated BSD may result in subsequent morbidity.⁸ Hence, there is substantial need for screening tools for BSD that have a high degree of sensitivity and specificity and that can be easily implemented.

Several self-report screening questionnaires have been developed to detect BSD.^9–12 The Mood Disorder Questionnaire (MDQ) is the most widely studied because it can be quickly and easily used to recognize BSD in a clinical setting with relatively good sensitivity and very good specificity.^13,14 Comparison of the three most widely used self-report screening instruments—the MDQ, the bipolar spectrum diagnostic scale (BSDS),¹⁵ and the hypomanic checklist for BSD (HCL-32)¹¹—within a primary care or general population has shown that the highest degree of sensitivity was obtained with the BSDS, with the MDQ exhibiting the highest degree of specificity.^16,17 A recent systematic review revealed that screening tests for bipolar disorder may include a high number of false positives;¹⁸ therefore, applying the MDQ, which exhibits the highest degree of specificity, may be desirable in screening for BSD.

Several studies have used the MDQ to estimate the prevalence of BSD.^2,19–21 However, previous studies within the general population or in a primary care setting have used traditional paper-based collection through mail or face-to-face communication. Using mail may be logistically easier than face-to-face evaluation; however, neither method is anonymous, leading to the possibility that subjects respond differently because they are aware that their identities are not concealed.²² Indeed, an anonymous survey resulted in a much higher prevalence of BSD in mental health screening.²³

The rapid diffusion and accessibility of smartphones provides opportunities to enhance the screening of psychiatric illness within the general population with a greater degree of anonymity. According to a survey by the Korean Communications Commission,²⁴ approximately 27 percent of adults in South Korea own and use a smartphone. Moreover, in one recent study, 50 percent of psychiatric patients indicated an interest in using an app on a daily basis to monitor their mental health,²⁵ and another study has explored the role of mobile phone apps in screening for depression.²⁶ Given the accessibility of smartphones, further investigation of the role of smartphone apps in screening and monitoring psychiatric illness is likely to be useful.

In this study, therefore, our purpose was to enhance the detection of BSD by developing and distributing a free Korean MDQ (K-MDQ) smartphone application that can be completed anonymously. We relied on a Korean translation of the MDQ whose validity has been established.²⁷ The purpose of our study was to evaluate the feasibility of the MDQ application and to determine correlates for positive results on the MDQ with the aim of demonstrating that mental health screening can be done easily and reliably on a smartphone in a smartphone-using population.

Methods

The K-MDQ includes three criteria. In Criterion 1, the K-MDQ screens for a lifetime history of manic or hypomanic symptoms, using 13 yes/no items. Criterion 2 queries whether several manic or hypomanic symptoms or behaviors have been experienced during the same period, and Criterion 3 assesses the functional impairment resulting from the illness on a 4-point scale (from “no” to “severe” problems).²⁷ In this study, respondents were considered as testing positive for BSD when they provided “yes” responses to at least seven of the symptoms in Criterion 1, in addition to confirming that multiple symptoms clustered in the same time period (Criterion 2) and indicating that the symptoms caused “moderate” or “serious” problems (Criterion 3). We used the K-MDQ as the basis for a smartphone application that was developed in both Android and iOS versions. The iOS version was registered with the iPhone App Store on 12 May 2011, and the Android version was registered on the Android market on 5 May 2011. Data collection was performed on 11 July 2011. On the first page of the application, participants were informed that the information they submitted could be used for the purpose of academic or scientific research. Each item of the K-MDQ was presented sequentially one item at a time on the smartphone screen such that the next item did not appear if a score was not entered for the displayed item. Demographic data including age, sex, and past psychiatric treatment history were collected. The result of the screening test was made available to respondents.

The internal consistency of the K-MDQ was determined, using Cronbach’s α. We performed a factor analysis to determine construct validity, extracting principal components using varimax rotation. To define the factors that were associated with a positive K-MDQ result, we compared the frequency of the K-MDQ positive rate between age groups and gender groups using a chi-squared test. In addition, we performed a logistic regression using gender, age group, and past psychiatric history as predictors and K-MDQ positive results as the dependent variable. All statistical tests were two-tailed with a significance level set at 0.05. Our Institutional Review Board reviewed and approved the protocol, and the study was conducted in accordance with sound clinical practices and the Helsinki Declaration. Our Institutional Review Board waived patient-specific informed consent for this confidential survey and the anonymous reporting of aggregate data.

Results

Demographics characteristics

Data from 27,159 participants were collected during the 2 months from 5 May 2011 to 11 July 2011. There were 17,985 downloads of the iPhone application and 3816 of the Android application. The majority of the subjects were female (74.6%) and between the ages of 10 and 29 years (75.0%). The demographic characteristics are presented in Table 1.

Table 1.

Demographic characteristics of respondents.

	Total (N = 27,159), N (%)
Sex
Male	6894 (25.4)
Female	20,265 (74.6)
Age (years)
10–20	20,362 (75.0)
30–40	6196 (22.8)
⩾50	601 (2.2)

Clinical characteristics

The clinical characteristics of participants are presented in Table 2. There were 2864 (10.5%) participants with a previous history of psychiatric treatment. The proportion of patients with a past psychiatric history differed significantly across age groups: 9.7 percent in the 10–29 group, 11.1 percent in the 30–49 group, and 34.6 percent in the 50 or older group (χ² = 387.395, df = 2, p < 0.001).

Table 2.

Clinical characteristics of respondents.

	Past psychiatric history		Significance
	Presence (n = 2864)	Absence (n = 24,295)	Significance
Sex
Male	722 (10.5%)	6172 (89.5)	0.821
Female	2142 (10.6%)	18,123 (89.4)	0.821
Age group (years)
10–20	1969 (9.7%)	18,393 (90.3%)	<0.001
30–40	687 (11.1%)	5509 (88.9%)
⩾50	208 (34.6%)	393 (65.4%)

Reliability and factor analysis of the K-MDQ application

The internal consistency of the K-MDQ was good, as indicated by a Cronbach’s α of 0.80. Correlation coefficients between the individual items on the questionnaire and the total K-MDQ score ranged from 0.43 (“spending money”) to 0.62 (“more active,” more social, or outgoing) and all were statistically significant (p < 0.01). Two factors with eigenvalues greater than 1 were extracted from the factor analysis; these accounted for 40.1 percent of the total variance (Table 3). Factor 1, with an eigenvalue of 3.9, corresponded to a group of five items (Items 3, 5, 8, 9, and 10) and accounted for 30.1 percent of the total variance. Factor 2 with an eigenvalue of 1.29 contributed 9.9 percent toward the variance and corresponded to a group of eight items (Items 1, 2, 4, 6, 7, 11, 12, and 13). The frequency of endorsement of individual K-MDQ items ranged from 31.2 to 75.8 percent. The most frequently endorsed items were “racing thoughts” (74.7%) and “easily distracted” (75.8%); the least frequently endorsed items were “excessive, foolish, or risky” (48.4%) and “spending money” (31.2%). The K-MDQ positive group exhibited significantly higher levels of endorsement for all of the items relative to the K-MDQ negative group, with the magnitude of the difference between the groups being the highest for the item “sending money” (Figure 1).

Table 3.

Factor loadings of K-MDQ application from principal-component analysis with varimax rotation in all participants.

Item	Factor 1	Factor 2
1. Feel so good or hyper		0.46
2. So irritable		0.58
3. More self-confident	0.69
4. Less sleep		0.39
5. More talkative	0.48
6. Thought racing		0.53
7. Easily distracted		0.52
8. More energy	0.83
9. More active	0.80
10. More social or outgoing	0.55
11. More interested in sex		0.41
12. Excessive, foolish, or risky		0.64
13. Spending money		0.59
Eigenvalue	3.92	1.29
Total variance explained	30.12%	9.93%

K-MDQ: Korean version of Mood Disorder Questionnaire.

Figure 1.

Comparison of K-MDQ application item frequencies between groups.

Correlates for K-MDQ positive results

The number of participants who were scored as K-MDQ positive (i.e. who met all three criteria as described in the “Methods” section) was 2224 (8.2%). The correlates for MDQ positive results as revealed by a univariate analysis (chi-square test) are presented in Table 4. The proportion of K-MDQ positive results was higher among participants with a past psychiatric history (12.3%) than among participants without a past psychiatric history (7.7%, χ² = 70.429, df = 1, p < 0.001). In the logistic regression analysis (Table 4), gender and age group were not predictors of K-MDQ positive results; the presence of a past psychiatric history (odds ratio = 1.690, 95% confidence interval (CI) = 1.495–1.909, p < 0.001) was found to be the only predictor of K-MDQ positive results.

Table 4.

Correlates for K-MDQ positive.

	K-MDQ positive N = 2224, N (%)	K-MDQ negative N = 24,935, N (%)	Univariate (chi-square test) significance	Multivariate (Logistic regression analysis)
	K-MDQ positive N = 2224, N (%)	K-MDQ negative N = 24,935, N (%)	Univariate (chi-square test) significance	β	χ ²	OR (95% CI)	significance
Female	1646 (74.0)	18,619 (74.7)	0.494	−0.051	0.966	0.951 (0.859–1.052)	0.326
Age (years)
10–20	1686 (75.8)	18,676 (74.9)	0.621
30–40	489 (22.0)	5707 (22.9)		−0.071	1.682	0.932 (0.837–1.037)	0.195
⩾50	49 (2.2)	552 (2.2)		−0.174	1.294	0.840 (0.622–1.134)	0.255
Presence of past history of psychiatric illness	351 (12.3)	2513 (7.7)	<0.001	0.525	70.738	1.690 (1.495–1.909)	<0.001

K-MDQ: Korean version of Mood Disorder Questionnaire, K-MDQ positive: Meet all the three criteria of Mood Disorder Questionnaire, OR: odds ratio, CI: confidence interval.

Among participants without a past psychiatric history, the K-MDQ positive rate for males (7.6%) did not differ significantly from that of females (8.0%; χ² = 0.703, df = 1, p = 0.402), nor were there any significant differences in the K-MDQ positive rate across age groups: 7.8 percent in the 10–29 years group, 7.2 percent in 30–49 years group, and 8.7 percent in the 50 years or older group (χ² = 2.890, df = 2, p = 0.236). The multivariate analysis revealed that there were no significant predictors of the K-MDQ positive rate among subjects without a past psychiatric history (data not shown).

Among participants with a past psychiatric history, the K-MDQ positive rate did not differ by gender (12.0% in males, 12.3% in females; χ² = 0.038, df = 1, p = 0.845), or by age group (12.4% in the 10–29 years group, 13.4% in the 30–49 years group, and 7.2% in the 50 years or older group; χ² = 5.780, df = 2, p = 0.056). However, the logistic regression analysis indicated that age group significantly predicted K-MDQ positive results among subjects with a past psychiatric history. The odds ratio of the 10–29 years group compared with the 50 years or older group was 1.8 (95% CI = 1.1–3.1, p = 0.032); and that of the 30–49 years group compared with the 50 years or older group was 2.0 (95% CI = 1.1–3.5, p = 0.018).

Discussion

Efficiency of the smartphone application as a mental-health-screening tool

We conducted screening for BSD in the general community population using smartphone applications that incorporated the K-MDQ. Only 2 months were required to collect K-MDQ data from 27,159 participants. The number of participants in this study is considerably larger than in previous community-based studies using the MDQ.^19,28–30 Moreover, in comparison with a study that involved a similar sample size,³¹ in which 16,803 participants were tested over a period of 6 months using the 27-item Mood Swing Survey Questionnaire, this study was conducted over a relatively short period of time.

There are some benefits to screening for psychiatric illness over the Internet or on a mobile device.²² First, the response is easily accessible and relatively anonymous—although it is possible to link survey results with the respondent’s Internet protocol (IP) address, a court order is required to identify the person; therefore, anonymity is fairly well preserved.²² Second, the data can be automatically stored in a database for later data mining and analysis, significantly reducing costs relative to a printed survey. It is clear that there is tremendous potential for mobile communication to transform healthcare and clinical intervention in the community.³² Several previous studies have evaluated the use of mobile phones in supporting healthcare and public health interventions, particularly with regard to the collection and collation of data for healthcare research,³³ in medical and healthcare education, and in clinical practice within the community.³⁴ Moreover, there may be substantial benefits to the user due to possible improvements in the early detection of BSD and subsequent evaluation and intervention involving psychiatric services. It has been suggested that individuals with mental health difficulties often hesitate to contact psychiatric services even when they wish to be evaluated for their symptoms, because they do not want to be labeled as “mental patients” and prefer to avoid the negative associations and stigma that accompany such a label.³⁵ These subjects could benefit from easily administrable self-screening instruments such as the application used in this study.

Feasibility of the smartphone application as a mental-health-screening tool

Screening of 27,159 participants using the K-MDQ smartphone application in this study indicated that 8.2 percent were scored as positive for BSD. This value is similar to that reported by Rouillon et al.³⁶ among patients consulting general practitioners in France (8.3%), but higher than those previously reported for the general population in the United States (3.7%)² and Australia (2.5%).³⁷ Moreover, the mean K-MDQ score for all subjects in our sample was 7.7 ± 3.4. This score is quite high, given that the cutoff score for Criterion 1 alone is 7.²⁷

Our findings may be due to several factors. First, the age and culture of the participants should be considered. In a previous study investigating the prevalence of BSD among Korean college students using a paper version of the K-MDQ,³⁸ the prevalence of BSD was 18.6 percent, a value that is considerably higher than is generally reported in international studies. The high K-MDQ positive rate and the high mean scores obtained in this study could be due to the age of the participants; 75 percent of our participants were between 10 and 29 years of age. In previous studies, positive screening results for BSD have been associated with younger age.^10,36 Second, this study included only active participants due to the fact that it was conducted using K-MDQ smartphone applications, which needed to be downloaded by the participants. Subjects in this study may have been motivated to participate because of their diagnosed/undiagnosed psychiatric problems. This may have contributed to the high K-MDQ positive rate in our study. Indeed, in the logistic regression analysis, the presence of a past psychiatric history was found to be the only predictor of K-MDQ positive results. Third, the difference may be due to differences in evaluation methods across studies. Several instruments used in psychiatric research related to depression^39,40 and anxiety disorder⁴¹ have been validated for administration via the Internet. However, Internet-based administration of these instruments yields different results compared with paper-and-pencil assessments.⁴⁰ Therefore, it is questionable whether offline cutoff scores can be readily applied to scores obtained from participants over the Internet.^42,43

Limitations of this study

A limitation of this study is that our sample was not a random sample and was not selected as being representative of the general population. Because this study included only active participants who voluntarily downloaded the K-MDQ smartphone applications, we were unable to obtain any data from people who chose not to participate. Hence, we were unable to calculate a refusal rate or to present the number/characteristics of those who were included in the study and those who were excluded. As a result, we cannot exclude the possibility that the findings may be biased in favor of positive results. The possibility of a single person submitting more than one set of responses is another limitation of this study. Although the total number of application downloads was 21,801, our total number of response sets was 27,159, which indicates that the application executed an average of 1.25 times on each device; this suggests that individual participants may have responded more than once, or that more than one participant may have submitted responses using a single device. With regard to the age distribution evident in this study, the youngest age group represented the largest proportion of participants. According to a survey by the Korean Communications Commission,²⁴ the probability of owning and using a smartphone in South Korea decreases steadily with age: 46 percent between 10 and 29 years, 34 percent between 30 and 49 years, and 4 percent among those 50 years or older. In light of these results, it is clear that the higher proportion of participants between 10 and 29 years (75.0%) and the lower proportion of participants between 30 and 49 years (22.8%) of age reflects the statistics for ownership and use of smartphones and cannot be taken as representative of the general population. Hence, self-selection and age group may have resulted in a biased population. Of relevance to this study are findings from a previous study investigating correlates of non-response in a web-based survey of alcohol involvement among college students.⁴⁴ Non-response was found to be related to lower levels of experience with the web and lower rates of heavy episodic drinking. The results from this study, which included younger subjects and reported high K-MDQ scores, may have been affected by similar selection biases. However, because screening in the community is especially successful with high-risk samples,⁴⁵ the use of smartphone applications by younger members of the population may in fact prove to be clinically effective in screening for bipolar disorder.

Our results suggest that BSD screening tools can be delivered via smartphone applications and that their use can make the screening process faster and easier. It is possible that their convenience and anonymity contributed to the large number of individuals who voluntarily used our applications. The findings and limitations of this study suggest that age and diverse levels of smartphone experience should be considered in using smartphone applications for mental health screening.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Akiskal

Bourgeois

Angst

. Re-evaluating the prevalence of and diagnostic composition within the broad clinical spectrum of bipolar disorders. J Affect Disord 2000; 59(Suppl. 1): S5–S30.

Hirschfeld

Calabrese

Weissman

. Screening for bipolar disorder in the community.J Clin Psychiatry 2003; 64: 53–59.

Hirschfeld

. Bipolar spectrum disorder: improving its recognition and diagnosis. J Clin Psychiatry 2001; 62(Suppl. 14): 5–9.

Ghaemi

Boiman

Goodwin

. Diagnosing bipolar disorder and the effect of antidepressants: a naturalistic study. J Clin Psychiatry 2000; 61: 804–808.

Ghaemi

Goodwin

. “Cade’s disease” and beyond: misdiagnosis, antidepressant use, and a proposed definition for bipolar spectrum disorder. Can J Psychiatry 2002; 47: 125–134.

Hirschfeld

Lewis

Vornik

. Perceptions and impact of bipolar disorder: how far have we really come? Results of the national depressive and manic-depressive association 2000 survey of individuals with bipolar disorder. J Clin Psychiatry 2003; 64: 161–174.

Lish

Dime-Meenan

Whybrow

. The National Depressive and Manic-depressive Association (DMDA) survey of bipolar members. J Affect Disord 1994; 31: 281–294.

Baldessarini

Tondo

Hennen

. Treatment-latency and previous episodes: relationships to pretreatment morbidity and response to maintenance treatment in bipolar I and II disorders. Bipolar Disord 2003; 5: 169–179.

Depue

Krauss

Spoont

. General behavior inventory identification of unipolar and bipolar affective conditions in a nonclinical university population. J Abnorm Psychol 1989; 98: 117–126.

10.

Hirschfeld

Williams

Spitzer

. Development and validation of a screening instrument for bipolar spectrum disorder: the Mood Disorder Questionnaire. Am J Psychiatry 2000; 157: 1873–1875.

11.

Angst

Adolfsson

Benazzi

. The HCL-32: towards a self-assessment tool for hypomanic symptoms in outpatients. J Affect Disord 2005; 88: 217–233.

12.

Bowden

Singh

Thompson

. Development of the bipolar inventory of symptoms scale. Acta Psychiatr Scand 2007; 116: 189–194.

13.

Hirschfeld

Holzer

Calabrese

. Validity of the mood disorder questionnaire: a general population study. Am J Psychiatry 2003; 160: 178–180.

14.

Miller

Klugman

Berv

. Sensitivity and specificity of the Mood Disorder Questionnaire for detecting bipolar disorder. J Affect Disord 2004; 81: 167–171.

15.

Ghaemi

Miller

Berv

. Sensitivity and specificity of a new bipolar spectrum diagnostic scale. J Affect Disord 2005; 84: 273–277.

16.

Carvalho

Takwoingi

Sales

. Screening for bipolar spectrum disorders: a comprehensive meta-analysis of accuracy studies. J Affect Disord 2014; 172C: 337–346.

17.

Smith

Griffiths

Kelly

. Unrecognised bipolar disorder in primary care patients with depression. Br J Psychiatry 2011; 199: 49–56.

18.

Cerimele

Chwastiak

Dodson

. The prevalence of bipolar disorder in general primary care samples: a systematic review. Gen Hosp Psychiatry 2014; 36: 19–25.

19.

Chung

Tso

Chung

. Validation of the Mood Disorder Questionnaire in the general population in Hong Kong. Compr Psychiatry 2009; 50: 471–476.

20.

Twiss

Jones

Anderson

. Validation of the Mood Disorder Questionnaire for screening for bipolar disorder in a UK sample. J Affect Disord 2008; 110: 180–184.

21.

Weber Rouget

Gervasoni

Dubuis

. Screening for bipolar disorders using a French version of the Mood Disorder Questionnaire (MDQ). J Affect Disord 2005; 88: 103–108.

22.

Gordon

McNew

. Developing the online survey. Nurs Clin North Am 2008; 43: 605–619.

23.

Warner

Appenzeller

Grieger

. Importance of anonymity to encourage honest reporting in mental health screening after combat deployment. Arch Gen Psychiatry 2011; 68: 1065–1071.

24.

Korea Communications Commission. Annual report 2011. Gwacheon, South Korea: Korea Communications Commission, 2012.

25.

Torous

Friedman

Keshavan

. Smartphone ownership and interest in mobile applications to monitor symptoms of mental health conditions. JMIR Mhealth Uhealth 2014; 2: e2.

26.

BinDhim

Shaman

Trevena

. Depression screening via a smartphone app: cross-country user characteristics and feasibility. J Am Med Inform Assoc 2015; 22: 29–34.

27.

Jon

Hong

Yoon

. Validity and reliability of the Korean version of the Mood Disorder Questionnaire. Compr Psychiatry 2009; 50: 286–291.

28.

Dodd

Williams

Jacka

. Reliability of the Mood Disorder Questionnaire: comparison with the Structured Clinical Interview for the DSM-IV-TR in a population sample. Aust N Z J Psychiatry 2009; 43: 526–530.

29.

Fisher

Goldney

Dal Grande

. Bipolar disorders in Australia. A population-based study of excess costs. Soc Psychiatry Psychiatr Epidemiol 2007; 42: 105–109.

30.

Mangelli

Benazzi

Fava

. Assessing the community prevalence of bipolar spectrum symptoms by the mood disorder questionnaire. Psychother Psychosom 2005; 74: 120–122.

31.

Parker

Blanch

Fletcher

. Evaluating the utility of a bipolar self-test screening measure. Australas Psychiatry 2011; 19: 317–320.

32.

Boulos

Wheeler

Tavares

. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online 2011; 10: 24.

33.

Blaya

Fraser

Holt

. E-health technologies show promise in developing countries. Health Aff 2010; 29: 244–251.

34.

Lindquist

Johansson

Petersson

. The use of the Personal Digital Assistant (PDA) among personnel and students in health care: a review. J Med Internet Res 2008; 10: e31.

35.

Corrigan

. How stigma interferes with mental health care. Am Psychol 2004; 59: 614–625.

36.

Rouillon

Gasquet

Garay

. Screening for bipolar disorder in patients consulting general practitioners in France. J Affect Disord 2011; 130: 492–495.

37.

Goldney

Fisher

Grande

. Bipolar I and II disorders in a random and representative Australian population. Aust N Z J Psychiatry 2005; 39: 726–729.

38.

Bae

Kim

Lee

. Prevalence of bipolar spectrum disorder in Korean college students according to the K-MDQ. Neuropsychiatr Dis Treat 2013; 9: 869–874.

39.

Spek

Nyklícek

Cuijpers

. Internet administration of the Edinburgh Depression Scale. J Affect Disord 2008; 106: 301–305.

40.

Donker

van Straten

Marks

. Brief self-rated screening for depression on the Internet. J Affect Disord 2010; 122: 253–259.

41.

Coles

Cook

Blake

. Assessing obsessive compulsive symptoms and cognitions on the Internet: evidence for the comparability of paper and Internet administration. Behav Res Ther 2007; 45: 2232–2240.

42.

Buchanan

. Internet-based questionnaire assessment: appropriate use in clinical contexts. Cogn Behav Ther 2003; 32: 100–109.

43.

Van Gelder

Bretveld

Roeleveld

. Web-based questionnaires: the future in epidemiology? Am J Epidemiol 2010; 172: 1292–1298.

44.

Cranford

McCabe

Boyd

. Reasons for nonresponse in a web-based survey of alcohol involvement among first-year college students. Addict Behav 2008; 33: 206–210.

45.

Mitchell

. Clinical utility of screening for clinical depression and bipolar disorder. Curr Opin Psychiatry 2012; 25: 24–31.