Alcohol Abuse Management in Primary Care: An e-Learning Course

Abstract

Background: The mental health knowledge gap challenges public health. The Alcohol Abuse Management in Primary Care (AAMPC) is an e-learning course designed to cover alcohol-related problems from the primary care perspective. The goal of this study was to verify if the AAMPC was able to enhance healthcare professionals' alcohol-related problems knowledge. Materials and Methods: One hundred subscriptions for the AAMPC were offered through the federal telehealth program. The course was instructor-led and had nine weekly classes, delivered synchronously or asynchronously, at the students' convenience, using a varied array of learning tools. At the beginning, students took a test that provided a positive score, related to critical knowledge for clinical management, and a negative score, related to misconceptions about alcohol-related problems. The test was repeated 2 months after course completion. Results: Thirty-three students completed the course. The positive score improved significantly (p<0.001), but not the negative score. Students with previous experience with e-courses presented greater improvement on the positive score (p<0.036). Eighty-percent of the students thought the course excelled in meeting its objectives. Web conferences and video and audio recordings were the most appreciated learning tools. Course satisfaction was negatively related to frequency of Internet access (Spearman's rho=−0.455, p=0.022). Conclusions: E-learning was highly appreciated as a learning tool, especially by students with the least frequency of Internet use. Nonetheless, it worked better for those previously familiar with e-courses. The AAMPC e-course provided effective knowledge transmission and retention. Complementary strategies to reduce misconceptions about alcohol-related problems must be developed for the training of primary care staff.

Introduction

The use of e-learning for continuing education of health professionals is on the rise.¹ E-learning presents several advantages: it saves money related to transportation costs, published material, and physical space. If combined with recorded material, the learning experience can be reproduced several times, while securing procedural uniformity,^2,3 resulting in an appealing time-flexible cost-effective strategy.⁴ Such accessibility makes it particularly interesting for the continuing education of either hard-to-reach or large health professional groups, such as primary care health teams in charge of a whole region.^5,6 The gap between the high incidence of mental disorders and access to treatment is a well-established fact, reinforcing the World Health Organization declaration that there can be no health without mental health.⁷ Regarding the three most prevalent mental health conditions, the treatment gap for mood and anxiety disorders ranges between 50% and 60% (i.e., more than half of individuals bearing such conditions were not in treatment). The gap for alcohol abuse and dependence, the highest, was estimated at nearly 80%. This gap is probably underestimated because of lack of data from low- and middle-income countries.⁸

Such a gap is in contrast with the burden that alcohol represents to society. It is estimated that alcohol-related problems are responsible for 3% of the disease burden around the world and that it is the first cause of days lost to disability or early death for individuals 10–24 years of age, accounting for 7% of the disease burden in this age bracket.⁹

Alcohol abuse and dependence is widespread and a burden upon society, and the primary level should be the preferred instance to deal with it within the healthcare system.¹⁰ Therefore, the World Health Organization has proposed that strengthening the mental health component by means of training health personnel and investing in telemedicine are two of a 25-item list of top challenges for global mental health.¹¹

Several countries, such as Brazil, Canada, Australia, and Germany, among others, are currently exploring initiatives in continuing education for health professionals through e-learning that includes mental health.^{1,5,12

–15} A recent meta-analysis showed large effect sizes for Internet-based learning for health professionals. Knowledge and test performance improvement were associated with interactivity, practice exercises, repetition, and feedback.¹⁶ Specific initiatives for training in mental health are sparse, although encouraging. Efficacy of computer-based learning was reported for training of nursing homes' staff on depression and dementia¹⁷ and training of community mental health professionals on Dialectical Behavior Therapy.¹⁸ Conversely, Beidas et al.¹⁹ reported limited gains in therapist adherence, skill, and knowledge on the treatment of anxiety among youth. This apparent conflict points to the need of investigating variations on suitability of e-learning according to professional profile and specific themes. We found only two reports on computer-based learning for addiction: one that described the demographic profile of physicians using an Internet course on Alcoholics Anonymous²⁰ and another describing a computer-based program to train addiction counselors to help patients cope with cravings.²¹ The authors of the latter found significant but modest improvement in the counselors' test performance.²¹ Generally, the reports sound unanimous regarding acceptance of computer-based distance learning for mental health, but they are not specific to the factors that modulate this appreciation (e.g., previous experience with e-learning), or what are the preferred tools and methods.

Brazil's public health system has been undergoing major restructuring with the progressive implementation of a new program at the primary care level based on family doctors, nurses, and community health agents—the so-called Family Health Program.²² Alcohol abuse and addiction is the third most common mental disorder among users of primary care centers, being reported by over 6% of this population,²³ and it is also a frequent cause of requests for mental health specialist's supervision from the primary care staff. Hence, we have devised and piloted an Alcohol Abuse Management in Primary Care (AAMPC) e-course, designed for (but not exclusive to) primary care staff. The content of the AAMPC covers key aspects of alcohol-related problems from the perspective of family medicine such as basic knowledge about alcohol abuse, management of patients and family members, and available community resources. Its structure is short and simple, so it can be administered at a distance, in keeping with the intrinsic qualities of e-learning: flexibility and accessibility.

The primary goal of this study was to determine if the AAMPC was able to enhance healthcare professionals' general knowledge about alcohol abuse. As secondary goals, we wanted to determine, among the topics of AAMPC, which ones presented greater knowledge improvement, to investigate if students' characteristics (demographics, professional trade, previous education, and experience with e-learning) would modulate knowledge acquisition and course appreciation and, finally, which of the learning tools used in the course were most valued by the students.

Materials and Methods

Recruitment and Profiling of Participants/Students

The AAMPC e-course was announced in the Internet networks of the Federal Tele-health Program. The announcement underscored its content directed for primary care staff; however, participation of health professionals from other levels within the public healthcare system was welcome, if they had been previously registered as primary care providers. Participation was free of charge, and the course was certified by the Commission for Culture and Extension of the Medical School of the University of São Paulo, São Paulo, Brazil. Following directions from the institutional ethics board, anonymity was secured, and students provided electronic consent for the use of their data. The enrollment period lasted from August 2010 until October 2010. The course was delivered from October to December 2010.

Of the 100 initial subscriptions, 67 students initiated the course, and 33 effectively concluded it. Figure 1 charts the pathways leading to the final sample.

Fig. 1.

Prevention of alcohol abuse distance learning course. CCEX, Commission for Culture and Extension.

To control for course attrition, completers and noncompleters were compared regarding their demographic profiles, and no differences were found. The mean age of course completers was 34.0 years old (standard deviation=10.3 years); 81.8% were female; 57.6% were single, divorced, or widowed; 57.6% had a university degree; and 64.7% were health professionals currently involved with administration and/or clinical care at primary centers, whereas the remaining 35.3% were health professionals not currently involved with primary care, but with a record of having previously worked in primary care centers. Seventy-three percent of the students had daily access to Internet, 87.8% had access to high-quality Internet connection, and 66% declared they had had previous experience with e-courses.

Course Structure

The course had nine classes lasting 1 h each; all were delivered by the Internet, except for classes #1, #5, and #9, which lasted 1.5 h and were transmitted from the Web conference room. Because of space limitation, only four spots were available for students to follow the sessions face to face from the conference room. This was previously scheduled, and each student was allowed to follow face to face just one conference—only eight students applied. Different sets of tools were used throughout the course, including web conferences, video exhibitions, text reading followed by multiple-choice questions (MCQs), e-chats and audio chats (in which students sent written questions and received oral answers from the speaker), and Internet forums for requests and doubts. The AAMPC was an instructor-led (synchronous learning) e-course; however, if the student missed one activity, he or she could access an audio or video recording of it (asynchronous learning). Additionally, the Web conferences and parts of the audio chats with the most frequently asked questions were edited and made available at the course site for further consultation, named, respectively, video and audio tips. Table 1 presents a description of the classes' content and materials applied.

Table 1.

Course Objectives and Content

LESSON	DURATION	TITLE	OBJECTIVE	METHODOLOGY
Class 1	1.5 h	The effects of alcohol upon the individual	Introduce the Web-based learning tools and dynamics	Face-to-face class or Web meeting^a
			Appreciate alcohol-related harm	Video exhibition
Class 2	1 h	The alcohol-related burden upon society	Provide information about alcohol consumption in Brazil and worldwide, highlighting key risk and protection factors for alcohol abuseIntroduce alcohol law	Text reading followed by MCQ completion and chat with the teacher
Class 3	1 h	Risk and protective factors for alcohol abuse	Identify risk and protective factors	Text reading followed by MCQ completion
			How to minimize risk factors and to strengthen protective factors	Web meeting
Class 4	1 h	Truths and myths about alcohol issues	Provide information about alcohol metabolism, alcohol abuse consequences. and alcohol association with other drugs	Text reading followed by MCQ completion and chat with the teacher
Class 5	1.5 h	Alcohol and family: how to proceedIntroduction of Community Reinforcement and Family Training (CRAFT) and the alcohol screening questionnaire (CAGE)	How to identify people with alcohol abuse symptoms	Text reading followed by MCQ completion
			Facilitate regular alcohol screening and management of family-related issues	Face-to-face class or Web meeting^a
Class 6	1 h	How investigate alcohol problems	Provide guidelines on how to deal with alcohol abusers	Text reading followed by MCQ completion, video exhibition, and audio chat with teacher
			Provide first aid techniques in case of alcohol intoxication
Class 7	1 h	Alcohol and motivation to change	The Stages of Change model	Web meeting
Class 8	1 h	Alcohol and motivation to change	The Stages of Change model	Text reading followed by MCQ completion and audio chat with the teacher
Class 9	1.5 h	Seeking solutions to alcohol-related problems within community resources	Debate and reflect upon real cases followed by the students and introduce 12-step–based programs and the most common resources available in the community	Face-to-face class or Web meeting^a

Those students who were not able to attend the face-to-face class, the Web conference, or chat could watch the recorded video made available 4 days after the presentation.

MCQ, multiple-choice questions (correct answers were provided right after completion).

Alcohol Knowledge Assessment

Before beginning the course, the students answered an Alcohol Knowledge Questionnaire (AKQ) covering all the content of the course, designed by the authors, to gauge the students' previous knowledge about clinical management of alcohol abuse. The test contained 10 questions, covering the course content, with a variable number of statements that the reader had to rate as either true or false, except items #3 (approaching alcohol problems during home visiting), #6 (first aids for alcohol intoxication), and #9 (approaching alcohol problems with the patient), which were MCQs. Two months after the AAMPC e-course completion, the students repeated the AKQ to assess retention of knowledge. This time frame was adopted in order to provide enough distance from the end of the course and yet not too much so we would not risk losing contact with the students. The AKQ is available upon request from the corresponding author.

Course Appreciation Assessment

At course completion, the students also filled out another questionnaire, designed by the authors, to assess the course appreciation, divided into two parts: satisfaction and relevance of the e-tools used for learning. Questions addressing satisfaction asked about how well the course met its objectives, its applicability, study load, course length, site navigation, and quality of texts, Internet forums, video and audio materials. The rates were organized in an increasing fashion, using a scale of poor (1), regular (2), good (3) and excellent (4). A list of the e-tools used during the course was presented, and course completers rated their perceived relevance for learning from 0 (not important at all) to 10 (most important). The course appreciation questionnaire is available upon request from the corresponding author.

STATISTICAL ANALYSIS

Two measures were defined for each true-or-false item from the AKQ to assess general knowledge about alcohol: a positive score and a negative score. The positive score was based on the proportion of true assertions endorsed by the student and represented knowledge considered critical for optimal clinical management of alcohol-related problems (e.g., “Inquiring about general health and stress related symptoms is a good ‘ice-breaker.’ A too confronting style should be avoided.”). The negative score was based on the proportion of false assertions endorsed by the student and represented common misconceptions and biases in approaching patients with alcohol problems (i.e., “The best thing to do is to wait for a health problem to show up, so you can suggest a visit to the doctor.”). A final score was calculated subtracting the negative score from the positive score for each item, except for items #3, #6, and #9, which were scored either 1 or 0, because they had a MCQ format and were analyzed separately. The positive and negative item scores were summed up to tally, respectively, a total positive score and a total negative score; then a final total score was calculated by subtracting the total negative score from the total positive score and adjusting the raw outcome to vary between 0 and 10 (standardized values). Total positive, total negative, and final total scores were calculated for pre- and post-course assessments.

Association measures between performance on the AKQ pre- and post-course and sample demographics were performed using correlation analysis (either Pearson's or Spearman's when appropriate) and mean comparison (t test) for, respectively, continuous and categorical variables. The distribution of pre- and post-course scores was checked with the Kolmogorov–Smirnov test; then an analysis of variance for repeated measures was performed. Any variable reaching significance at the association analysis was introduced as either a cofactor (if categorical) or a covariable (if continuous).

Performance variation for each item from the AKQ between pre- and post-course was analyzed using Wilcoxon's test because the majority of the item scores had distributions other than normal. Two performance rankings for the AKQ items at pre- and post-course were built. Then, a third ranking for knowledge improvement was built based on the Z statistics from the Wilcoxon's test (a lower Z value indicates a higher knowledge improvement from pre- to post-course). McNemar's test was used for questions #3, #6, and #9 because they were in MCQ format.

The scoring of the satisfaction items were summed up and then divided by the number of valid items for each individual in order to build a mean global score for course satisfaction. The same procedure was adopted for the relevance items in order to build a mean global score for relevance attributed to the learning tools used during the course. Satisfaction and Relevance mean scores were compared with demographics and the final total score of the AKQ, using mean comparison (analysis of variance) and correlation analysis (either Pearson's or Spearman's when appropriate) for, respectively, categorical and continuous variables. Exploratory analyses were performed for each item of the satisfaction and relevance assessments searching for associations with demographics and performance on the AKQ. Relevance scores attributed for each e-tool were compared between themselves using Wilcoxon's test to verify the most important ones for learning according to the students. A significance level (p) equal to 0.05 was adopted for all statistical testing.

Results

The investigation of potential relationships between sample characteristics and course performance did not reveal significant associations, except for previous experiences with e-courses. No difference was found for the total final score between students with and without previous experiences with an e-course (t ₃₁=0.689, p=0.496); however, students without previous experience performed better at the pretest in trend levels (t ₃₁=−1.973, p=0.057). A 2×2 analysis of variance for repeated measures, having time as within-subjects factor and previous experience with e-course as between-subjects factor, revealed a significant improvement in the AKQ score from pre- to post-course (F _1,31=15.243, p<0.001) and a significant interaction between time and group (i.e., students with previous experience with e-courses had a greater variation on the AKQ score from pre- to post-course) (F _1,31=4.809, p=0.036) (Fig. 2).

Fig. 2.

Performance on the Alcohol Knowledge Questionnaire and previous e-course experience.

Questions #3, #6, and #9, analyzed separately because of their MCQ format, did not show significant variation at McNemar's test; however, they already had a high frequency of correct answers at the pre-course assessment: 60.6% (p=0.109), 78.9% (p=0.754), and 90.9% (p=1.00), respectively, suggesting that a probable ceiling effect precluded knowledge gain appreciation in this case. Conversely, the remaining items from the test, which were scored on a continuous fashion, displayed a significant variation between pre- and post-course final scores, except for questions #5 and #10 and for question #8, which approached significance. As can be seen from Table 2, the final scores that varied from pre- to post-course did so because of the variation on the corresponding positive score, as no negative score was significantly modified.

Table 2.

Pre- and Post-test Scores Comparison, by Item (n=33)

	MEAN (SD) SCORE
THEME	PRE-TEST	POST-TEST	Z	P
Question #1: Patient approach (range, 0–1)
Positive score	0.51 (0.30)	0.80 (0.21)	−3.919	<0.001
Negative score	0.06 (0.13)	0.15 (0.44)	−0.897	0.429
Final score^a	0.45 (0.36)	0.76 (0.24)	−3.832	<0.001
Question #2: Family guidance (range, 0–1)
Positive score	0.55 (0.26)	0.83 (0.20)	−4.422	<0.001
Negative score	0.05 (0.10)	0.04 (0.09)	−0.707	0.480
Final score^a	0.50 (0.28)	0.80 (0.22)	−4.398	<0.001
Question #4: 12-steps program (range, 0–1)
Positive score	0.51 (0.32)	0.74 (0.29)	−3.194	0.001
Negative score	0.14 (0.14)	0.14 (0.13)	−0.302	0.763
Final score^a	0.36 (0.36)	0.61 (0.35)	−3.264	0.001
Question #5: Alcohol metabolism (range, 0–1)
Positive score	0.58 (0.24)	0.58 (0.24)	−0.077	0.938
Negative score	0.13 (0.11)	0.12 (0.12)	−1.117	0.264
Final score^a	0.46 (0.22)	0.46 (0.23)	−0.204	0.838
Question #7: Risk and protection (range, 0–1)
Positive score	0.29 (0.28)	0.47 (0.35)	−2.449	0.014
Negative score	0.55 (0.28)	0.53 (0.21)	−0.333	0.739
Final score^a	−0.26 (0.33)	−0.06 (0.34)	−2.229	0.026
Question #8: Addiction consequences (range, 0–1)
Positive score	0.61 (0.26)	0.73 (0.27)	−2.016	0.044
Negative score	0.02 (0.73)	0.05 (0.10)	−1.342	0.180
Final score^a	0.59 (0.26)	0.68 (0.28)	−1.799	0.072
Question #10: Alcohol problems signs (range, 0–1)
Positive score	0.76 (0.21)	0.83 (0.16)	−1.454	0.146
Negative score	0.03 (0.10)	0.05 (0.18)	−0.378	0.705
Final score^a	0.73 (0.22)	0.78 (0.22)	−1.051	0.293
Total score (range, 0–10)^b
Positive score	5.45	7.11	−4.038	<0.001
Negative score	1.40	1.54	−0.617	0.537
Final total score^a	4.04	5.58	−3.672	<0.001

Comparisons were performed with Wilcoxon's test.

Final scores were calculated by subtracting the negative score from the corresponding positive score.

Standardized values.

SD, standard deviation.

The comparison of the pre- and post-course scores suggests that general knowledge about alcohol addiction (alcohol problem signs and addiction consequences, respectively, questions #10 and #8), which was already good before the course, remained good; thus they did not vary over time. Themes related to practical management of patients and their families (questions #1 and #2) presented the highest improvement. Knowledge related to risk and protection factors and 12-step program significantly varied, but their intermediate position in the knowledge improvement ranking (third column in Table 3) suggests that there could be further knowledge advancement. Finally, knowledge related to alcohol metabolism presented the smallest improvement between pre- and post-course. Table 3 shows the ranking of knowledge for the continuously scored questions for pre- and post-course and for knowledge improvement over the course.

Table 3.

Proficiency Ranking of Themes

PRE-COURSE RANKING	POST-COURSE RANKING	PRE- AND POST-COURSE IMPROVEMENT RANKING^a
Question #10: Alcohol problem signs	Question #2: Family guidance	Question #2: Family guidance
Question #8: Addiction consequences	Question #10: Alcohol problem signs	Question #1: Patient approach
Question #2: Family guidance	Question #1: Patient approach	Question #4: 12-steps program
Question #1: Patient approach	Question #8: Addiction consequences	Question #7: Risk and protection
Question #5: Alcohol metabolism	Question #4: 12-steps program	Question #8: Addiction consequences
Question #4: 12-steps program	Question #5: Alcohol metabolism	Question #10: Alcohol problem signs
Question #7: Risk and protection	Question #7: Risk and protection	Question #5: Alcohol metabolism

Based on the Z coefficient from the Wilcoxon test displayed on Table 2.

Regarding the course appreciation, participants had a very positive opinion about the course: 84% thought that the objectives were excellently achieved, and 80% thought that the course had an optimal applicability. The less appreciated aspects were course study load (24% rated “excellent” and 64% rated “good”) and course length (20% rated “excellent” and 68% rated “good”). About the perceived importance of didactic materials and interventions, participants attributed the highest relevance to Web conferences and recorded videos and audios and the least relevance to face-to-face classes. Table 4 summarizes the main findings.

Table 4.

Relevance Attributed to Tools Used in Distance Learning

	MEAN	SD	MINIMUM	MAXIMUM	Z	P
Web conference^a	9.76	0.663	7.0	10.0	—	—
Video and audio tips^a	9.76	0.597	8.0	10.0	0^b	1.00
Text reading with MCQ^a	9.72	0.613	8.0	10.0	−0.431^b	0.666
AKQ completion^a	9.44	1.003	6.0	10.0	−1.510^b	0.131
Audio chat	9.36	1.186	5.0	10.0	−1.930^b	0.054
Chat	9.20	1.384	5.0	10.0	−2.047^b	0.041
Forum	8.72	1.514	4.0	10.0	−3.093^b	0.002
Face-to-face class	8.04	2.950	0	10.0	−3.017^b	0.003

Scores ranged from 0 to 10 (n=25).

Most relevant learning tools.

Wilcoxon's test for related measures: compared with the highest score (Web conference).

AKQ, Alcohol Knowledge Questionnaire; MCQ, multiple-choice questions; SD, standard deviation.

No significant associations between global satisfaction and relevance mean scores and demographics and performance at the AKQ were found, except for a significant relationship between Satisfaction mean score and Frequency of Internet Access—students with the least frequency of Internet access were the most satisfied with the e-course (Spearman's rho=−0.455, p=0.022). At the exploratory analysis, relevance attributed to face-to-face classes was negatively correlated with the final score at the AKQ (Spearman's rho=−0.466, p=0.019). In the ranking of relevance for learning, new e-tools (Web conference, video and audio tips, and text reading followed by MCQ completion) performed better than traditional ones (chats and forums), with face-to-face encounters with teachers being the least relevant, according to the students.

Discussion

The data show that e-learning is an effective way of transmitting information about alcohol-related problems to primary care staff. Indeed, a significant increase in alcohol knowledge was found even 2 months after the course completion. Thus, e-learning secured both knowledge gain and retention. No variable, except previous experience with e-learning, was related to course performance, indicating that e-learning is an accessible mean for teaching mental health topics. Despite profiting less from the course, students without previous experience enjoyed e-learning the most, probably because of a novelty effect. Offering a practice pre-course and extending the tutorial time may help inexperienced students in future versions of the course.

However, only the positive subscores improved. In other words, the course was successful in providing new knowledge, but was not effective in dispelling misconceptions regarding alcohol-related problems. The field of addiction is plagued with stigma and prejudice.²⁴ Therefore, it is imperative that educational initiatives with a focus on changing misconceptions that hinder optimal clinical management be developed and tested in future studies. The identification of this blind spot was only possible because of the adoption of true-or-false type of questions for the assessment of the learning process. Indeed, this type of question proved more useful in providing a balanced evaluation about knowledge and biases and more sensitive to changes in the cognitive status of students than MCQs.

The comparison between pre- and post-course scores suggests that general knowledge about alcohol addiction did not improve over time. This lack of improvement was probably due to a ceiling effect because the pre-course scores in this case were already high. Despite presenting the third highest score improvement, the 12-step programs score was not well positioned in the post-course score ranking, meaning that there is still room and need for improvement in knowledge about 12-step programs and other community resources to support addiction patients. At the same time, the topics related to practical management of addiction patients and their families were optimally classified both at the post-course ranking and at the improvement ranking. Thus, the AAMPC may help accomplish a key directive of the Family Health Program, which is to enable primary care staff and services to deal with highly prevalent behavioral challenges, such as addictions. Nonetheless, it is up to future studies to establish if e-learning is capable of actually shaping addiction clinical management toward best evidence-based and cost-effective practices.

It is interesting that the students who attributed more relevance to face-to-face classes profited less from the course. This reinforces the perception that students who are less familiar with e-learning may benefit from blended models and then smoothly progressing to strictly e-learning as they become less dependent on face-to-face interaction. It is interesting that taking the knowledge test was among the four most appreciated activities by the students, possibly because it enables students to gauge by themselves how well the course worked for them.

The current study has important limitations, mainly its small sample size and the attrition rate, which, although high, is in keeping with previous reports of e-learning dropout.²⁵ Besides, the follow-up period was short, and no further assessment was conducted to ascertain that knowledge retention is secured for a longer period. Finally, the lack of a control group does not yield the comparison of e-learning performance against traditional in-person teaching, or simple text reading followed by test completion. Thus, the power and cost-effectiveness of e-learning to teach and engage students in mental health in comparison with other methods remain to be clarified. However, the current data allow the conclusion that e-learning is effective in increasing the knowledge of primary care professionals about the clinical management of alcohol-related problems.

Future studies should focus on strategies to foster long-term retention of knowledge, removal of misconceptions, actual change for better clinical practices, and cost-effectiveness of e-learning.

Footnotes

Disclosure Statement

No competing financial interests exist.

References

Curran

. Tele-education. J Telemed Telecare, 2006; 12:57–63.

Khanna

, Aschenbrand

, Kendall

. New frontiers: Computer technology in the treatment of anxious youth. Behav Ther, 2007; 30:22–25.

Abeles

, Verduyn

, Robinson

. Computerized CBT for adolescent depression (“Stress-busters”) and its initial evaluation through an extended case series. Behav Cogn Psychother, 2009; 37:151–165.

Calear

, Christensen

. Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Med J Aust, 2010; 192(11 Suppl):S12–S14.

Eikelboom

, Weber

, Atlas

, Dinh

, Mbao

, Gallop

. A tele-otology course for primary care providers. J Telemed Telecare, 2003; 9(Suppl 2):S19–S22.

Griffiths

, Lindenmeyer

, Powell

, Low

, Thorogood

. Why are health care interventions delivered over the Internet? A systematic review of the published literature. J Med Internet Res, 2006; 8:e12.

Prince

, Patel

, Saxena

, et al. No health without mental health. Lancet, 2007; 370:859–877.

Kohn

, Saxena

, Levav

, Saraceno

. The treatment gap in mental health care. Bull World Health Organ, 2004; 82:858–866.

Gore

, Bloem

, Patton

, et al. Global burden of disease in young people aged 10–24 years: A systematic analysis. Lancet, 2011; 377:2093–2102.

10.

Gual

, Sabadini

. Implementing alcohol disorders treatment throughout the community. Curr Opin Psychiatry, 2011; 24:203–207.

11.

Collins

, Patel

, Joestl

, et al. Grand challenges in global mental health. Nature, 2011; 475:27–30.

12.

dos Santos

AeF

, Haddad

, Alves

, Torres

, de Souza

, de Melo

MoC

. Evaluating the experience of training through videoconferences in primary care. Telemed J E Health, 2011; 17:722–726.

13.

Joshi

, Novaes

, Iyengar

, et al. Evaluation of a tele-education programme in Brazil. J Telemed Telecare, 2011; 17:341–345.

14.

Kind

. The Internet as an adjunct for pediatric primary care. Curr Opin Pediatr, 2009; 21:805–810.

15.

Paixão

, Miot

, Wen

. Tele-education on leprosy: Evaluation of an educational strategy. Telemed J E Health, 2009; 15:552–559.

16.

Cook

, Levinson

, Garside

, Dupras

, Erwin

, Montori

. Internet-based learning in the health professions: A meta-analysis. JAMA, 2008; 300:1181–1196.

17.

McPhaul

, Rosen

, Bobb

, et al. An exploratory study of mandated safety measures for home visiting case managers. Can J Nurs Res, 2007; 39:173–189.

18.

Dimeff

, Woodcock

, Harned

, Beadnell

. Can dialectical behavior therapy be learned in highly structured learning environments? Results from a randomized controlled dissemination trial. Behav Ther, 2011; 42:263–275.

19.

Beidas

, Birkett

, Newcomb

, Mustanski

. Do psychiatric disorders moderate the relationship between psychological distress and sexual risk-taking behaviors in young men who have sex with men? A longitudinal perspective. AIDS Patient Care STDS, 2012; 26:366–374.

20.

Sellers

, Galanter

, Dermatis

, Nachbar

. Enhancing physicians' use of Alcoholics Anonymous: Internet-based training. J Addict Dis, 2005; 24:77–86.

21.

Cucciare

, Weingardt

, Greene

, Hoffman

. Current trends in using Internet and mobile technology to support the treatment of substance use disorders. Curr Drug Abuse Rev, 2012; 5:172–177.

22.

Paim

, Travassos

, Almeida

, Bahia

, Macinko

. The Brazilian health system: History, advances, and challenges. Lancet, 2011; 377:1778–1797.

23.

Gonçalves

, Kapczinski

. [Prevalence of mental disorders at a referral center for the Family Health Program in Santa Cruz do Sul, Rio Grande do Sul State, Brazil]. Cad Saude Publica, 2008; 24:2043–2053.

24.

Williamson

. Destigmatizing alcohol dependence: The requirement for an ethical (not only medical) remedy. Am J Public Health, 2012; 102:e5–e8.

25.

Levy

. Comparing dropouts and persistence in e-learning courses. Comput Educ, 2007; 48:185–204.