Vietnam as a case example of school-based mental health services in low and middle income countries: Efficacy and effects of risk status

Abstract

The purposes of this study were to (a) assess the efficacy of a universal classroom-based mental health and social skills program for primary school students in Vietnam, and (b) given the universal nature of the intervention, assess outcomes as a function of risk status (high versus low). RECAP-VN is a semi-structured program that provides students with classroom social skills training, and teachers with in-classroom consultation on program implementation and classroom-wide behavior management. Project data were collected at three time-points across the academic year from 443 second grade students in regards to their social skills and mental health functioning, in the Vietnamese cities of Hanoi and Danang. Mental health functioning (emotional and behavioral mental health problems) was the ultimate outcome target (at Time 3), with social skills intermediate (at Time 2) outcomes targeted to improve mental health functioning. Significant treatment effects were found on both social skills and mental health functioning. However, although program effects on mental health functioning were significant for both low and high risk status groups, program effects on social skills were only significant for low risk status students, suggesting that different mechanisms may underlie program effects for high and low risk status students. Overall the results of this study, one of the first to assess directly the effects of a school-based program on mental health functioning in a low or middle income country, provide some support for the value of using school-based programs to address the substantial child mental health treatment gap found in low- and middle-income countries.

Keywords

child mental health global health LMIC low and middle income countries risk status school-based Vietnam

Child mental health problems are a significant challenge and burden not only in high income countries (HIC) such as the United States and in Europe and the UK but also in low and middle income countries (LMIC) (Patel, Kieling, Maulik, & Divan, 2013). Overall, the prevalence and characteristics of child and adolescent mental health problems (hereafter referred to as ‘child’ mental health problems) in LMIC are generally comparable to that encountered in HIC (Murray, Dorsey, & Lewandowski, 2014). For instance in Vietnam, an Asian LMIC of approximately 93 million people, a recent nationally representative epidemiology survey found that over 12% of the children (approximately 3 million children and adolescents) had mental health problems of sufficient severity to warrant treatment (Weiss et al., 2014). Such mental health problems, although of concern in their own right, are particularly important because of their close link to functional impairment, including impairment in school functioning. In this same Vietnamese sample, for example, mental health problems were the single largest risk factor for life functional impairment, with behavioral mental health problems associated with a 250% increase in school impairment (Dang, Weiss, & Trung, 2016). Despite this well documented need for mental health services, however, in Vietnam as in many other LMIC there is a significant lack of mental health treatment infrastructure including mental health policy, trained mental health professionals to provide treatment, and physical infrastructure (e.g., clinics) for children and adolescents (Malhotra & Padhy, 2015; Patel et al., 2013; Weiss et al., 2012; World Health Organization, 2007).

School-based mental health services

Given that most children spend a significant portion of their day in schools, particularly at the younger ages, schools represent a logical place for provision of child mental health services (Murray et al., 2014; Weist, 2003). This is true in HIC but particularly in LMIC given their general lack of child mental health infrastructure (Weiss et al., 2012). School-based programs can provide for: (a) early identification of children with mental health problems; (b) direct access to children (i.e., working with children in the schools is not dependent on parents bringing them to a clinic); (c) direct access to one of children’s most important environments, the school; and (d) reduced stigma (i.e., children do not need to go to a ‘mental’ health clinic). School-based mental health programs generally focus on (a) mental health promotion (i.e., enhancing students’ general psychological well-being and mental health), or (b) mental health intervention (i.e., prevention or reduction of students’ mental health symptoms such as anxiety or oppositional behavior). Mental health intervention programs can range from (a) universal prevention programs that target the entire school, with the goal of preventing development or exacerbation of mental health symptoms through use of a consistent, adaptive school-wide environment, to (b) indicated preventive interventions that target specific children manifesting risk factors for mental health problems but who do not meet criteria for specific mental health categories, (c) to selective interventions involving children with specific mental health disorders or problems (Fazel, Patel, Thomas, & Tol, 2014; Petersen, Bhana, Lund, & Herrman, 2014). This broad model conforms with the multi-tier response model for intervention and intergrated service delivery developed and approved by the US National Association of State Directors of Special Education as well as US National Association of School Psychology (Batsche et al., 2005; Brown-Chidsey & Steege, 2011). Although various mental health programs emphasize different techniques and strategies, in general most have a central focus on support for development of adaptive social skills, and on parent, teacher, etc. reinforcement of desired behavior and appropriate negative consequences for undesired behavior (e.g., de Boo & Prins, 2007).

Delivering child mental health service through schools has been a major focus for over 30 years in HIC (Paternate, 2005; Weare & Nind, 2011; Atkinson et al., 2014). In LMIC, the value of school-based mental health services also has been recognized. School-based mental health services have become a key strategy for addressing the child mental health gap, as they are a relatively efficient strategy for reducing barriers to child mental health treatment in LMIC (Patel & Kleinman, 2003). In 1995, the World Health Organization launched its Global School Health Initiative to strengthen health—including mental health—promotion and education for children around the world, with a particular focus on LMIC. However, although research in HIC shows fairly consistently that school-based mental health programs can be effective, the generalizability of these results to LMIC is unclear, because: (a) LMIC have significantly fewer resources in general as well as (b) significantly fewer human resources trained in mental health; and (c) cultural differences such as individualism versus collectivism that potentially could influence the perceived social appropriateness and effectiveness of programs (Eshun, 2009; Marsiglia & Booth, 2015).

Thus, in order to determine the actual utility of school-based mental health services in LMIC, research in these settings is necessary. To date, there have been few studies evaluating school-based mental health prevention or intervention in LMIC. In their review of mental health interventions for young people in LMIC, Barry and colleagues identified 22 evaluation studies, 14 of which involved school-based programs, half in turn of which (seven) used an experimental design for their evaluation (Barry, Clarke, Jenk, & Patel, 2013). Most of these seven school-based programs focused on improving social and emotional problem solving skills rather than on improving actual mental health functioning. Evaluations of these programs generally reported positive effects on students’ self-esteem, motivation, and self-efficacy, but these studies focused on outcomes that are important (e.g., social skills) but not the outcomes of ultimate interest (i.e., mental health functioning; life functioning).

Overall, there have been only a very small number of studies of school-based mental health treatment programs in LMIC. Fazel et al. (2014) reviewed mental health programs in LMIC and found that universal, whole-school mental health promotion programs were generally effective, whereas in regards to mental health treatment programs there was relatively little actual research, particular in non-conflict affected regions of the world. And although the importance of early intervention (e.g., at the primary school level) is well recognized (Nafpaktitis & Perlmutter, 1998), the large majority of LMIC school-based mental health work has focused on adolescents (Foley & Hochman, 2006). Our own literature review of school-based mental health interventions (i.e., studies assessing mental health outcomes such as anxiety, behavior problems, etc. as the outcome of interest) in Asia, the world’s most populous continent, identified five studies of mental health programs: Two in Japan (Matsumoto & Shimizu, 2016; Sato et al., 2009) which is a high income country, one in a politically violent area of Indonesia and thus generalizability of its results to non-violent areas is unclear (Tol et al., 2008), one in Taiwan (Tang et al., 2009) which is also a high income country, and one in India (Singhal, Manjula, & Sagar, 2014). All five studies targeted students with internalizing problems (depression, anxiety), and none addressed behavior problems, which tend to be more stigmatized and a significant problem in collectivistic Asian countries (Lopez & Guarnaccia, 2000; Weisz et al., 1993). A literature review of school psychology program in low- and middle-income countries more generally found that psychological and mental health counseling services implemented in schools in some Asian countries such as the People’s Republic of China, Taiwan, Singapore, Macau (D’Amto, van Schalkwyk, Zhao, & Hu, 2013; Ding, Kuo, & Van Dyke, 2008; Van Schalkwyk & Sit, 2013). However, in none of these studies did the scope of the service include a classroom-based progam with structured currriculum, which has been found to be a more effective and efficient approach to school-based services (Ager et al., 2011; Weist et al., 2008).

RECAP-Vietnam school-based mental health program

RECAP-VN is an adaptation of the RECAP (Reaching Educators, Children, and Parents) program (Han, Catron, Weiss, & Marciel, 2005; Weiss, Harris, Catron, & Han, 2003) that was developed and evaluated in the United States. As implemented in the US, RECAP is a school-based, multi-systemic (i.e., involving the school and home), cognitive-behavioral and social skills training program for elementary school children with emotional and behavioral problems. It involves (a) classroom groups with the entire class, (b) classroom teachers, (c) small group sessions with RECAP participants, (d) individual sessions with RECAP participants, and (e) parents. Several studies in the US (e.g., Han et al., 2005; Weiss et al., 2003) have indicated that it is effective with primary school children.

The RECAP-VN program was adapted for Vietnam as a universal intervention program, in order to provide the program to as many students as possible, given Vietnam’s LMIC context of relatively limited resources. RECAP-VN includes: (a) a student-focused social skills and adaptive problem-solving curriculum, implemented twice a week in participating classrooms as part of the school curriculum over one academic year. The curriculum consists of 32 lessons (See Supplemental Online Materials, Table 1), divided into seven modules focusing on the development of adaptive social skills (See Supplemental Online Materials, Figure 1); and (b) a behavior management system implemented by the teachers and RECAP classroom consultant that focuses on reinforcement (e.g., praise; use of a token system) for desired student behavior and appropriate punishment (e.g., time-out; loss of privileges such as recess) for undesired behavior. Teachers receive site-based training and monthly consultation on program implementation throughout the academic year. Because of limited resources in Vietnam, RECAP-VN’s current configuration does not include the small group and parent components that are part of the original RECAP. RECAP-VN focuses on both social skills and mental health outcomes, with the social skills viewed as intermediate outcomes, and mental health functioning as the ultimate outcomes (Nezu & Nezu, 2010). That is, the ultimate purpose of the RECAP-VN program is to improve the mental health and life functioning of its students, and social skills are seen as one pathway towards achieving this goal (Horner, Sugai, & Todd, 2005).

Table 1.

Outcome variables at baseline.

Variable	Tx Group Mean (SD)	Cntl Group Mean (SD)
SSRS Assertive	2.24 (0.57)	2.22 (0.59)
SSRS Cooperative	2.57 (0.48)	2.56 (0.52)
SSRS Empathy	2.51 (0.56)	2.45 (0.60)
SSRS Self-control	2.01 (0.68)	2.04 (0.68)
SBQ Externalizing	1.48 (0.56)	1.50 (0.63)
SBQ Internalizing	1.75 (0.79)	1.79 (0.77)

Notes: SSRS scale runs from 1 (Never) to 3 (Very often). SBQ scale runs from 1 (Not true) to 4 (Very true).

Figure 1.

RECAP lesson content: Developmental Progression of RECAP skills over a school year.

Although the goal of universal mental health programs (i.e, programs that target the entire school or entire classroom) is to support and improve the mental health functioning of all students in the setting, analysis of outcomes as a function of risk status (i.e., whether the student is high or low risk based on mental health functioning) is important for universal mental health program development (Stallard, Simpson, Anderson, & Goddard, 2008). Analyses of the extent to which program outcomes vary as a function of risk status (i.e., the statistical interaction between risk status and treatment group) can indicate whether the program truly is universal (i.e., works with all students), or whether and how the program might be modified to increase its effects for all students targeted. Despite the importance of analysis of risk status, few studies have assessed intervention outcomes as function of risk status, and to the best of our knowledge no studies in LMIC have assessed the effects of risk status on treatment outcomes. Most studies that have considered risk status have assessed different groups separately without statistical comparison (e.g., Singhal, Manjula, & Sagar, 2014).

Purpose of the current study

The primary aims of this study were to assess: (a) the effects of the RECAP-VN program on a cohort of Vietnamese elementary school children, across one academic year, and (b) assess the extent to which these effects varied as a function of high versus low risk status. We made three hypotheses: (a) at the final outcome assessment (T3, in late spring near the end of the academic year) students assigned to the RECAP-VN condition would show significantly lower levels of mental health problems than students assigned to the control condition (services as usual); and (b) at the midpoint outcome assessment (T2, in the middle of the winter and of the academic year) students assigned to the RECAP-VN condition would show significantly higher levels of social skills as compared to students assigned to the control condition. We made these hypotheses specific to these time-points because as intermediate outcomes, students’ social skills were expected to change prior to their mental health functioning, which were the ultimate outcomes. Finally, based on Weiss and colleagues’ (2015) discussion regarding facilitator versus proximal process moderator models, we hypothesized (c) that the effects of the RECAP-VN program would be stronger for high risk students (i.e., those with elevated levels of mental health problems) because the program targets social skills deficits expected to be more proximal and higher among students with mental health problems (Weiss, Han, Tran, Gallop, & Ngo, 2015).

Method

Setting and participants

In order to broaden the generalizability of the study results, elementary schools were selected from two cities in Vietnam, two schools from Hanoi (the national capital, and the second largest city in the country) and one school in Danang (a major secondary city in Vietnam). The schools were selected to be as representative of their cities as possible (e.g., in regards to socio-economic status of the students’ families), although with the small number of schools in the study it was not possible to actually be fully representative. Following the recommendations of the three school principals in regards to the optimal grades upon which the program should focus, the project was implemented in second grade classrooms. In Vietnam, there is a national education curriculum, and the second grade overall has relatively less intense academic demands than other elementary school grades, thus providing classes with more time to focus on non-academic topics such as mental health. At each school, prior to the beginning of the academic year, an introductory meeting was held with all second grade teachers, who were invited to participate in the project; all teachers choose to participate.

In order to obtain a representative sample of students from the schools, all students within each of the second grade classrooms were eligible and invited to participate in the study. At the beginning of the academic year in Vietnam, schools have a group parent conference to introduce families to the new grade and teachers, etc. with whom their child will be placed. At project schools, teachers and RECAP-VN consultants spent 15 minutes introducing the families to the RECAP-VN program. Parents were given a summary letter with a consent form to take home, and parents who were interested in having their child participate in the project returned the signed consent form. In each school, half of the second grade classrooms were randomly assigned to the treatment (RECAP-VN) group (N = 8), and half to the control group (N = 8). Of the 515 families that were given the consent letters, consent was obtained for 443 students (86% participation rate), all of whom completed the T1 assessment. Data were obtained from 379 students at T2 and from 404 at T3, for follow-up participation rates of 86% and 91% respectively. At baseline the mean age of the participating students was 8.71 years (SD = 0.45), with 51% male; all students were ethnically Vietnamese

Control and treatment groups

Control group

Children in the control group were assessed on the same schedule as the treatment group but their classrooms and teachers received no mental health intervention or support from the project.

Treatment group

Treatment group participants received the RECAP-VN classroom programs, which has the goals of: (a) helping students learn a set of skills for functioning adaptively; (b) developing prosocial classroom norms and expectations for children’s interactions with others; and (c) providing training and support for teachers’ use of adaptive classroom management techniques (e.g., appropriate positive reinforcement and negative consequences). The program provides training for students in: (a) social skills (e.g., making friends, avoiding involvement with negative peer behavior); (b) reattribution (for hostile attributions of others’ intentions as well as unrealistic self-appraisals); (c) communication skills; (d) enhancing self-monitoring and self-control’ (e) affect recognition and expression; and (f) relaxation. The original RECAP curriculum (Han et al., 2005; Weiss et al., 2003) was translated and adapted into Vietnamese by a group of four Vietnamese and US psychologists (including the author of the US RECAP program), and five Vietnamese elementary teachers.

For the first two months of the academic year, 45 minute classroom lessons are taught by the RECAP-VN consultants with the teacher twice per week, and then decreased to once per week for the rest of the academic year. Skills taught in the lessons are reinforced daily by the teachers and consultants (when in classroom) using modeling, explicit discussion of behavioral and affective consequences of behavior choices, and reinforcement via tokens. Through the academic year, beyond helping to provide the lessons, the consultants spent two hours per week in each classroom for observation of students’ behavior, and support for teachers’ program implementation. The teacher component focuses on increasing teachers’ mental health literacy (e.g., understanding the problem behavior is in large part a response to home and school environments, rather than the student being an inherently ‘bad’ child) and classroom management skills. Classroom management skills include development of appropriate rules and discipline and their implementation, and use of reinforcement to support the students in their use of RECAP-VN skills and to foster a positive classroom culture, wherein students’ adaptive behavior is positively reinforced and supported. Classroom behavior management strategies applied in the RECAP-VN program focus on the use of relatively high rates of positive reinforcement (e.g., by using tokens to concretely support teachers and students focusing on positive behavior) to increase the frequency of desired student behavior, and the judicious use of ignoring, redirection, and appropriate negative consequences to reduce the rate of undesired student behavior.

Teacher training on the RECAP-VN focuses on the administration of the program lessons and the use of positive and effective classroom behavior management strategies. Teachers received initial training on the RECAP-VN program during an initial one-day workshop at the beginning of the school year. Topics included: (a) symptoms of some common mental problems in children; (b) understanding reasons for children’s behavior (i.e., what factors are reinforcing the behavior); (c) establishing effective classroom expectations and structure; (d) importance of and techniques for reinforcement of positive student behavior; (e) use of consistent and effective discipline to reduce negative behavior; (f) adaptive communication skills; and (g) modeling adaptive problem-solving in naturally occurring situations. These training objectives were achieved via: (a) discussion of the principles and techniques of RECAP-VN, and their empirical and theoretical bases; (b) review of program lessons and key objectives; (c) role-play and discussion of implementation techniques and strategies, including ways to integrate RECAP-VN into classroom academic instruction; and (d) discussion of what forms of flexibility are acceptable within the model (e.g., different forms of positive reinforcement may be used, as long as the positive reinforcement is administered appropriately).

Throughout the school year, program consultants provided in-classroom consultation to teachers as needed to support implementation of the intervention program. While in the classroom, the consultant helped to reinforce and model the program’s principles and techniques, and provided teachers and their teaching assistants with supportive and corrective feedback regarding their implementation of program strategies and techniques (e.g., helping teachers to customize the program for the particular needs of their classroom) and tailoring the behavioral management system to fit the needs of the classroom.

Clinical training, supervision and maintenance of intervention integrity

Three Vietnamese masters-level child psychologists served as the RECAP-VN consultants for the eight classrooms, one psychologist in each school. Training and supervision in the RECAP-VN program was provided by two Vietnamese child psychologists and three US child psychologists, including the developers of the US RECAP program. RECAP-VN consultants participated in a three-day training that provided the conceptual and clinical background of the program, manuals and related materials, forms of flexibility acceptable within the model and how to handle clinical issues within the framework of the model (e.g., teacher resistance to providing high rates of positive reinforcement). The three consultants received three hours of group supervision per week focused on resolving the issues in class regarding the lessons and students’ problems. In addition, a clinical supervisor (CM) periodically visited the classrooms to observe the consultants.

Measures

Measures in this study were translated, culturally adapted, and back translated by a highly experienced (e.g., they have officially adapted and translated the WISC-IV for Vietnam, as well as many other assessment instruments; e.g., Dang, Weiss, Pollack, & Nguyen, 2011) bilingual team of psychologists and educators in Vietnam and the US using standard procedures to maintain the semantic, content, technical, and conceptual content of the measure (Hambleton, 2005). In this process, we followed the recommendations of Van Widenfelt, Treffers, De Beurs, Siebelink, and Koudijs (2005) and others who argue for the use a consensus approach to translation rather than strict translation-back translation. In strict translation-back translations, translators often make literal translations of items that back translate well to the original wording but may fail to capture critical nuanced meanings in both translations. This failure may not be identified in the back translation, since the translation and back-translation are similar literal translations. The validity of the translation was checked through independent back-translations. Measures were then reviewed by teams of teachers from participating schools, with translations adjusted based on their feedback, and measures re-evaluated, etc.

Student Behavior Questionnaire

The SBQ (Weiss et al., 2003) is a brief problem behavior checklist for students that produces two broad-band factors, emotional internalizing mental health problems (e.g., ‘I am sad and unhappy’) and behavioral externalizing problems (e.g., ‘I talk back and argue with people’). Items are rated on a 1 (‘not true’) to 4 (‘very true’) scale. The SBQ subscales have an average correlation of 0.83 with comparable scales on the Youth Self-Report Form (Achenbach, 2009).

Social Skills Rating System

The SSRS (Gresham & Elliott, 1990) is a widely used, standardized measure of children’s social behaviors. In the present study, the child version was used, which includes Cooperation (e.g., ‘I listen to adults when they are talking to me’), Assertion (e.g., ‘I tell others when I am upset with them’), Self-Control (e.g., ‘I control my temper when people are angry with me’), and Empathy (e.g., ‘I listen to my friends when they talk about problems they are having’) subscales. Items are rate on a three-point Likert scale ranging from 1 (‘Never’) to 3 (‘Very often’). The SSRS has been found to have good reliability and validity with, for instance, mean correlations of approximately 0.55 with the Social Behavior Assessment (Diperna & Volpe, 2005).

Procedure

Data were collected from students at three time-points: Baseline (T1: September), mid-year (T2: early January), and end of the academic year (T3: May), with the study implemented from September 2013 to May 2014. Students completed their questionnaires in school during a free period, with assistance from research assistants if needed. Students received a small gift (e.g., a pencil, notebook) equal to approximately US $2.00 for each assessment. The study was reviewed and approved by the Vietnam National University School of Education US FWA IRB (00018223).

Statistical analysis

As noted above, our evaluation focused on two sets of variables, social skills as intermediate outcomes, and mental health as ultimate outcomes. The social skills variables (the four SSRS subscales) were analysed in an analysis of covariance (ANCOVA) with treatment group as a categorical fixed effect, T1 SSRS scores as the covariate, and T2 (reflecting their status as intermediate outcomes) SSRS scores as the dependent variable. The mental health variables (the two SBQ subscales) were analysed similarly, but with T1 SBQ scores as the covariate and T3 (reflecting their status as ultimate outcomes) SBQ scores as the dependent variable. In addition, we assessed whether the effects of treatment differed as a function of mental health risk status, with higher and lower risk status defined as being above or below the mean (respectively) on the combined SBQ Internalizing and Externalizing subscales. In these analyses, the tests of interest were the interactions between risk status and treatment group, which assessed whether the effects of treatment group on the six dependent variables differed as a function of risk status. All analyses were conducted using SAS 9.4 Proc GLM.

Propensity score-based weighting were used in all analyses reported below to adjust for potential baseline group differences due to random sampling error. Propensity score weightings provide a statistical basis for equating groups on a large number of covariates measured at baseline, enhancing the internal validity of evaluation studies (Austin, 2011; Guo & Fraser, 2010). Propensity score weights were calculated using SAS 9.4 Proc Logistic, using baseline demographic, and social skills and mental health scores. The propensity score model produced AUC = 0.80 indicating a good propensity model (Austin, 2011; Guo & Fraser, 2010).

Results

We first tested whether the treatment and control groups differed at baseline on mean levels of the various outcome measures. All differences were non-significant (all p > 0.40; see Table 1 for means and standard deviations). We next assessed the effects of treatment on the social skills variables at Time 2. Using the ANCOVA as described above, we found that two of the four SSRS variables showed significant treatment effects, favoring the RECAP-VN group (see Table 2). The effect of treatment on Assertive Behavior at Time 2 was significant with a semi-partial η² of 0.02, and the effect of treatment on Self-Control at Time 2 was significant with a semi-partial η² of 0.03. Finally, we assessed the effects of treatment on the two SBQ mental health subscales at Time 3. Both were significant, favoring the treatment group (see Table 2), with a semi-partial η² of 0.01 for both the SBQ Internalizing and the SBQ Externalizing subscales.

Table 2.

Primary analysis results.

Dependent variable	F, p	Semi- partial η²	Post-Tx Means
T2 SSRS Assertive	F(1,399) = 8.16, p < 0.005	0.02	Tx = 2.39, Cntl = 2.27
T2 SSRS Cooperative	F(1,392) = 0.25, ns	0.00	Tx = 2.62, Cntl = 2.61
T2 SSRS Empathy	F(1,394) = 0.66, ns	0.00	Tx = 2.57, Cntl = 2.53
T2 SSRS Self-control	F(1,403) = 10.51, p < 0.005	0.03	Tx = 2.25, Cntl = 2.10
T3 SBQ Externalizing	F(1,405) = 4.73, p < 0.05	0.01	Tx = 1.45, Cntl = 1.55
T3 SBQ Internalizing	F(1,405) = 5.56, p < 0.05	0.01	Tx = 1.73, Cntl = 1.89

In our final set of analyses, we tested whether the effects of treatment differed as a function of mental health risk status, including the main effect of risk status and its interaction with treatment group in the above analyses, across the six dependent variables. Three of the six interactions were significant, all for SSRS subscales (see Table 3). For all three of these significant interactions, the effect of treatment on the SSRS subscale was significant for the low risk group (i.e., students below the mean of the combined SBQ Int and SBQ Ext subscales) but not the high risk group, with the low risk treatment group participants having higher levels of reported social skills than low risk control participants. However, the effects of the program on the ultimate outcomes, SBQ internalizing and externalizing problems did not differ as a function of risk status.

Table 3.

Effects of risk status as a moderator of treatment effects.

Dependent variable	Risk x Tx Group interaction	Post-Tx Means for Low Risk Group¹
T2 SSRS Assertive	F(1,397) = 7.22, p < 0.01	Tx = 2.46, Cntl = 2.23**
T2 SSRS Cooperative	F(1,390) = 1.66, ns
T2 SSRS Empathy	F(1,392) = 7.42, p < 0.01	Tx = 2.59, Cntl = 2.46*
T2 SSRS Self-control	F(1,401) = 7.20, p < 0.01	Tx = 2.31, Cntl = 2.05***
T3 SBQ Externalizing	F(1,403) = 1.09, ns
T3 SBQ Internalizing	F(1,403) = 2.46, ns

Notes: 1 = means for Low Risk group differ at * = p<.05, ** = p<.001, *** = p<.0001.

Discussion

Although there have been other school-based studies focused on intermediate outcomes such as social skills, the present study is one of the first in an LMIC to directly assess effects on specific mental health symptoms and functioning of a school-based program for primary school students. We found significant effects on several of our intermediate targets (social skills) as well as on our ultimate outcome (mental health). In regards to social skills, there were significant treatment effects on the SSRS Assertive and Self-control scales (with semi-partial η2 = 0.02 and 0.03, respectively) but not the Cooperation or Empathy scales. It is interesting to note that the two SSRS scales showing significant effects appear to reflect more individualistic, self-focused skills (assertiveness, and self-control) whereas the two SSRS scales not showing significant effects were more group or other-focused (cooperation, and empathy). At the baseline, in both groups levels of the other-focused social skills were higher than levels of the self-focused skills. The treatment effect on the self-focused social skills involved increases in the skills in the treatment group but not in the control group (see Tables 2 and 3). Like many countries in Asia and more generally in LMIC including Africa (Wyer, Chiu & Hong, 2009), Vietnam is collectivistic society where there is a preference for a relatively tightly-knit social framework in which individuals are highly and closely connected to members of their in-group (Nguyen, Le, & Boles, 2010). In order to maintain this tightly connected social structure, members of collectivistic societies tend to favor behaviors that support group harmony (e.g., cooperation) and de-emphasize behaviors that do not directly support harmony (e.g., individual assertiveness) (Triandis & Gelfand, 2012). It is not surprising, then, that at Baseline higher level of other-directed social skills were found in both groups. Our outcome results suggest that programs such as the RECAP-VN program may be particularly useful for helping student s, at least in Vietnam and perhaps in other collectivistic cultures including those in Asia and in Africa, to develop social skills that are not emphasized in general society.

In regards to mental health outcomes, there were significant treatment effects on both internalizing (emotional) and externalizing (behavioral) mental health problems. However, treatment effect sizes for mental health functioning were relatively small, and lower than those for social skills (semi-partial η2 = 0.01, verus 0.02 and 0.03, respectively). In general other research has found that mental health functioning is more difficult to improve than social skills, perhaps because the former’s determinants are more complex (Prince-Embury & Saklofske, 2014).

For the Assertive, Empathy, and Self-control SRS scales, there was a significant risk status effect with the program, but contrary to our hypothesis program effects on these SSRS scales were significant among the low risk students but not among the high risk students. Given that there was no overall treatment effect on the Cooperative SSRS scale, this means that the program only had an effect on the SSRS scales among the low risk students. The overall effect of treatment on the two SBQ scales was significant and did not differ as a function of risk status, which means that the program was effective in regards to mental health functioning with both high and low risk students. Taken together, these findings suggest that the program may have different mechanisms for its effects on mental health functioning (i.e., the SBQ) for the low versus high risk groups. That is, the program enhanced social skills among the low risk group which suggests that that may be at least one mechanism for its effects among this subgroup. In regards to the high risk group, given that the program did not impact on social skills, it may be that the other major component of the program (working with teachers to increase their reinforcement of appropriate student behavior, and negative consequences for undesired behavior) was responsible for treatment effects. Teachers were trained and encouraged to use positive discipline such as praise and other strategies to reinforce and maintain positive behaviors, and appropriate strategies to discourage unwanted behavior (e.g., talking in class), which may have been responsible for treatment effects among the high risk students (Freiberg, Stein, & Huang, 1993; McNeely, Nonnemaker, & Blum, 2002).

Overall, the results of this study suggest the RECAP-VN program may have value as a universal intervention for countries similar to Vietnam (e.g., collectivistic LMIC with rapidly developing and changing societies). In fact, the purpose of the present study was not simply to evaluate a program for a specific country (i.e., Vietnam) but rather to provide data regarding its and similar programs’ utility more generally. The program does have several advantages for use in LMIC more generally. First, the universal approach can increase accessibility to services and reduce stigmatization towards mental health problems, which are particular barriers for mental health provision in LMIC (Patel, Chowdhary, Rahman, & Verdeli, 2011). The classroom-based structure means that it can be beneficial for a relatively large number of students, which is particularly important in low resource LMIC. Because the classroom component is implemented by the teachers after initial program training, with on-going consultation from the clinicians, the task-shifting approach can integrate the mental health activities directly into the educational setting, further helping to reduce stigma and increase efficiency. Finally, the classroom-based social skills component may be viewed favorably by school administration, teachers as well as parents in Vietnam (as evidenced by our high program acceptance rates) – and hopefully other similar LMIC countries – because it is similar in appearance to ‘life skills’ training programs. Life skills programs have been developed as a non-academic curriculum designed to directly improve psychosocial competence and mental health promotion, and indirectly improve academic performance, making them viewed positively by schools and families in LMIC (Barry et al., 2013; Liu, Liu, Yan, Lee, & Mayes, 2015).

However, the small effect sizes for the mental health outcomes indicates that additional development and enhancement of the program will be important to increase its practical utility. In the current version of RECAP-VN, the social skills training is not highly individualized, in that it is presented to the entire class which makes student-specific training difficult. In contrast, the teacher classroom intervention component is individualized, in that individual students are targeted for specific positive reinforcement or negative consequences. Taken together, this suggests that individualization of the program may be one approach to increasing its effect size.

Thus, one particular target for enhancement may be to individualize the social skills component in such a way that it will be effective with the high risk as well as low risk students. For example, the social skills component could be provided in the classroom to all the students but high risk students could also participate in a ‘practice’ group where they could receive more individualized support and feedback. A second potential target to increase the size of the treatment effect may be to increase teacher motivation, since teachers represent one half of the program targets. The present study did not assess teacher behavior and motivation, which would be an important area for future research, to determine the extent to which teachers’ influenced outcomes.

The primary limitation of the current study is that it relied on a single informant, the student him or herself. Teachers and parents might have provided a useful complementary perspective on students’ behaviors, but for funding reasons were not assessed in the present study. Observation would be a particularly useful assessment method. In addition, because of the relatively small number of schools (three) involved in the study, and the fact that only a single LMIC (Vietnam) was involved, our results in regards to the programs efficacy for LMIC in general should be considered tentative and the study a pilot. Larger scale replication in multiple countries and more cities will be necessary before the program can be considered an evidenced-based treatment for LMIC. But within this context, its findings are promising, and support further research into more comprehensive school-based mental health programs as one approach for reducing the child mental health treatment gap in LMIC. The findings also strongly highlight the importance of considering differential outcomes for high and low risk groups receiving the same program.

Footnotes

Acknowledgements

We gratefully acknowledge the students who participated in this study, and the support of schools. The psychologists working for RECAP-VN were Trinh Dinh, Ly Tran, and Nhung Nguyen.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the U.S. National Institutes of Health grants from the Fogarty International Center D43-TW009089 and R21 TW008435; and by the Vietnamese National Foundation for Sciences and Technology Development (NAFOSTED) grant VII.2-2011.11.

Author biographies

Hoang-Minh Dang, is an Associate Professor in Child and Adolescent Clinical Psychology, and a Director of Center for Research, Information and Services in Psychology (CRISP), College of Education, Vietnam National University, Vietnam. Dr. Hoang-Minh s research focuses on (a) adapting Western evidence-based intervention for emotional/behavioral problems in the context of Vietnam; (b) developing the school-based mental health services and working with schools to set up and coordinate services; (c) impact of urbanization and industrialization factors on the development child psychopathology; (d) parent behavior on child psychpathology psychopathology.

Bahr Weiss, is an Associate Professor in the Clinical Sciences (Clinical Psychology) program, Peabody College of Education and Human Development, Vanderbilt University, USA; a Visiting Foreign Professor at Vietnam National University, Hanoi; and a licensed clinical psychologist in Tennessee, USA. He has been active in school-based mental health since the early 1990 s, and has been working in Vietnam since 2001, funded by the US National Institutes of Health. His current professional interests include global mental health, conduct problems in children and adolescents, and cultural influences on the development and treatment of child and adolescent mental health problems.

Cao Minh Nguyen, is a Researcher at the Center of Information, Research and Service in Psychology (CRISP), College of Education, Vietnam National University. His research focuses on child mental health, adapting and applying school based interventions in school settings in Vietnam. He currently is pursuing his PhD at the Central Queensland University, Australia.

Nam Tran, is a Faculty member in the Child and Adolescent Clinical Psychology program at the College of Education, Vietnam National University, Vietnam. He received his PhD in Psychology from Vanderbilt University, USA. His research interests include child psychopathology, psychological intervention, prevention, and cross cultural issues. His clinical interests include working with children and adolescents with mood and behavior disorders. His current research focuses on parenting behaviors and children’s coping mechanisms in different cultures.

Amie Pollack, is a Senior Research Associate at Peabody College of Education and Human Development, Vanderbilt University, USA and a Visiting Professor at Vietnam National University, Vietnam. Her earlier work has focused on individual and school-based interventions for trauma and dissemination of evidenced-based treatments for children. Her work in Vietnam has included development of the graduate program in Clinical Psychology at College of Education, Vietnam National University, study of risk and resiliency in communities affected by frequent typhoons in Vietnam, assessment of barriers to global health development, and providing technical assistance to a variety of programs focused on mental health development in Vietnam.

References

Achenbach

T. M.

(2009) The Achenbach system of empirically based assessemnt (ASEBA): Development, findings, theory, and applications, Burlington, VT: University of Vermont Research Center for Children, Youth, & Families.

Ager

Akesson

Stark

Flouri

Okot

McCollister

Boothby

(2011) The impact of the school-based Psychosocial Structured Activities (PSSA) program on conflict-affected children in northern Uganda. Journal of Child Psychology and Psychiatry 52(11): 1124–1133. doi: 10.1111/j.1469-7610.2011.02407.x.

Austin

P. C.

(2011) A tutorial and case study in propensity score analysis: An application to estimating the effect of in-hospital smoking cessation counseling on mortality. Multivariate Behavioral Research 46: 119–151. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1080/00273171.2011.540480.

Barry

M. M.

Clarke

A. M.

Jenk

Patel

(2013) A systematic review of the effectiveness of mental health promotion interventions for young people in low and middle income countries. BMC Public Health 13(1): 1. doi: 10.1186/1471-2458-13-835.

Batsche

Elliott

Graden

J. L.

Grimes

Kovaleski

J. F.

Prasse

(2005) Response to intervention: Policy considerations and implementation, Alexandria, VA: National Association of State Directors of Special Education.

Brown-Chidsey

Steege

M. W.

(2011) Response to intervention: Principles and strategies for effective practice, New York, NY: Guilford Press.

Dang

H. M.

Weiss

Pollack

Nguyen

C. M.

(2011) Adaptation of the Wechsler Intelligence Scale for Children-IV (WISC-IV) for Vietnam. Psychological Studies in Population/Family Planning 56(4): 387–392. doi: 10.1007/s12646-011-0099-5.

Dang

H. M.

Weiss

Trung

L. T.

(2016) Mental health functioning and functional impairment among a nationally representative sample of Vietnamese children. Social Psychiatry and Psychiatric Epidemiology 51(1): 39–47. doi: 10.1007/s00127-015-1114-2.

D’Amato

R. C.

van Schalkwyk

G. J.

Zhao

B. Y.

(2013) Understanding the development of school psychology in Mainland China. School Psychology International 34(2): 131–144. doi: 10.1177/0143034312453392.

10.

de Boo

G. M.

Prins

P. J.

(2007) Social incompetence in children with ADHD: Possible moderators and mediators in social-skills training. Clinical Psychology Review 27(1): 78–97. doi: 10.1016/j.cpr.2006.03.006.

11.

Ding

Kuo

Y. L.

Van Dyke

D. C.

(2008) School psychology in China (PRC), Hong Kong and Taiwan: A cross-regional perspective. School Psychology International 29(5): 529–548. doi: 10.1177/0143034308099200.

12.

Diperna

J. C.

Volpe

R. J.

(2005) Self-report on the social skills rating system: Analysis of reliability and validity for an elementary sample. Psychology in the Schools 42(4): 345–354. doi: 10.1002/pits.20095.

13.

Eshun, S. (Ed.). (2009). Culture and mental health: Sociocultural influences, theory, and practice. Chichester: John Wiley & Sons.

14.

Fazel

Patel

Thomas

Tol

(2014) Mental health interventions in schools in low-income and middle-income countries. The Lancet Psychiatry 1(5): 388–398. doi: 10.1016/S2215-0366(14)70357-8.

15.

Foley

G. M.

Hochman

J. D.

(2006) Mental health in early intervention: Achieving unity in principles and practice, Baltimore, MD: Brookes Publishing.

16.

Freiberg

H. J.

Stein

T. A.

Huang

S. Y.

(1995) Effects of a classroom management intervention on student achievement in inner city elementary schools. Educational Research and Evaluation 1(1): 36–66. Retrieved from https://dx-doi-org.web.bisu.edu.cn/10.1080/1380361950010103.

17.

Gresham, F. M., & Elliott, S. N. (1990). Social skills rating system (SSRS): Manual. Circle Pines, MN: American Guidance Service.

18.

Guo

Fraser

M. W.

(2010) Propensity score analysis: Statistical methods and applications, Thousand Oaks, CA: Sage.

19.

Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting Psychological and Educational Tests for Cross-Cultural Assessment. NJ: Lawrence Erlbaum.

20.

Han

S. S.

Catron

Weiss

Marciel

K. K.

(2005) A teacher-consultation approach to social skills training for pre-kindergarten children: Treatment model and short-term outcome effects. Journal of Abnormal Child Psychology 33(6): 681–693. doi: 10.1007/s10802-005-7647-1.

21.

Horner

R. H.

Sugai

Todd

A. W.

(2005) School-wide positive behavior support. In: Bambara

Kern

(eds) Individualized supports for students with problem behaviors: Designing positive behavior plans, New York, NY: Guilford Press, pp. 359–390.

22.

Liu

Yan

Lee

Mayes

(2015) The impact of life skills training on behavior problems in left-behind children in rural China: A pilot study. School Psychology International 37(1): 73–84.

23.

Lopez

S. R.

Guarnaccia

P. J. J.

(2000) Cultural psychopathology: Uncovering the social world of mental illness. Annual Review of Psychology 51: 571–598. doi: 10.1146/annurev.psych.51.1.571.

24.

Malhotra

Padhy

S. K.

(2015) Challenges in providing child and adolescent psychiatric services in low resource countries. Child and Adolescent Psychiatric Clinics of North America 24(4): 777–797. doi: 10.1016/j.chc.2015.06.007.

25.

Marsiglia

F. F.

Booth

J. M.

(2015) Cultural adaptation of interventions in real practice settings. Research on Social Work Practice 25(4): 423–432. doi: 10.1177/1049731514535989.

26.

Matsumoto

Shimizu

(2016) The FRIENDS Cognitive Behavioral Program in Japanese schools: An examination of the treatment effects. School Psychology International 37(4): 397–409. doi: 10.1177/0143034316649639.

27.

McNeely

C. A.

Nonnemaker

J. M.

Blum

R. W.

(2002) Promoting school connectedness: Evidence from the national longitudinal study of adolescent health. Journal of School Health 72(4): 138–146.

28.

Murray

L. K.

Dorsey

Lewandowski

(2014) Global dissemination and implementation of child evidence-based practices in low resources countries. In: Beidas

R. S.

Kendall

P. C.

(eds) Dissemination and implementation of evidence-based practices in child and adolescent mental health, Oxford: Oxford University Press.

29.

Nafpaktitis

Perlmutter

B. F.

(1998) School-based early mental health intervention with at-risk students. School Psychology Review 27(3): 420.

30.

Nezu

A. M.

Nezu

C. M.

(2010) Problem-solving therapy for relapse prevention in depression. In: Richards

Perri

M. G.

(eds) Relapse prevention for depression, Washington, DC: American Psychological Association, pp. 99–130.

31.

Nguyen, H. H. D., Le, H., & Boles, T. (2010). Individualism–collectivism and cooperation: A cross-society and cross-level examination. Negotiation and Conflict Management Research, 3(3), 179–204. doi:10.1111/j.1750-4716.2010.00057.x.

32.

Patel

Chowdhary

Rahman

Verdeli

(2011) Improving access to psychological treatments: Lessons from developing countries. Behaviour Research and Therapy 49(9): 523–528.

33.

Patel

Kieling

Maulik

P. K.

Divan

(2013) Improving access to care for children with mental disorders: A global perspective. Archives of Disease in Childhood 98(5): 323–327. doi: 10.1016/j.brat.2011.06.012.

34.

Patel

Kleinman

(2003) Poverty and common mental disorders in developing countries. Bulletin of the World Health Organization 81(8): 609–615. doi: 10.1590/S0042-96862003000800011.

35.

Paternate

C. E.

(2005) School-based mental health programs and services: Overview and introduction to the special issue. Journal of Abnormal Child Psychology 33: 657–663. doi: 10.1007/s10802-005-7645-3.

36.

Petersen

Bhana

Lund

Herrman

(2014) Primary prevention of mental disorders. In: Byrne

Rosen

(eds) Early intervention in psychiatry: EI of nearly everything for better mental health, Chichester: Wiley-Blackwell, pp. 119–133.

37.

Prince-Embury

Saklofske

D. H.

(2014) Resilience interventions for youth in diverse populations, New York, NY: Springer.

38.

Sato

Imajo

Togasaki

Ishikawa

S. I.

Sato

(2009) School-based cognitive behavioral intervention for depressive symptoms in children. Japanese Journal of Educational Psychology 57(1): 111–123. doi: 10.5926/jjep.57.111.

39.

Singhal

Manjula

Sagar

K. J. V.

(2014) Development of a school-based program for adolescents at-risk for depression in India: Results from a pilot study. Asian Journal of Psychiatry 10: 56–61. doi: 10.1016/j.ajp.2014.03.011.

40.

Stallard

Simpson

Anderson

Goddard

(2008) The FRIENDS emotional health prevention programme. European Child and Adolescent Psychiatry 17(5): 283–289. doi: 10.1007/s00787-007-0665-5.

41.

Tang, T. C., Jou, S. H., Ko, C. H., Huang, S. Y., & Yen, C. F. (2009). Randomized study of school-based intensive interpersonal psychotherapy for depressed adolescents with suicidal risk and parasuicide behaviors. Psychiatry and Clinical Neurosciences, 63(4), 463–470. doi: 10.1111/j.1440-1819.2009.01991.x.

42.

Tol

W. A.

Komproe

I. H.

Susanty

Jordans

M. J.

Macy

R. D.

De Jong

J. T.

(2008) School-based mental health intervention for children affected by political violence in Indonesia: A cluster randomized trial. JAMA 300(6): 655–662. doi: 10.1001/jama.300.6.655.

43.

Triandis

H. C.

Gelfand

M. J.

(2012) A theory of individualism and collectivism. Handbook of Theories of Social Psychology 2: 498–520.

44.

Van Schalkwyk

G. J.

Sit

H. H.

(2013) Evaluating school-based psychological and counselling services in Macao using a qualitative approach. School Psychology International 34(2): 154–165. doi: 10.1177/0143034312453395.

45.

Van Widenfelt

B. M.

Treffers

P. D.

De Beurs

Siebelink

B. M.

Koudijs

(2005) Translation and cross-cultural adaptation of assessment instruments used in psychological research with children and families. Clinical Child and Family Psychology Review 8(2): 135–147. doi: 10.1007/s10567-005-4752-1.

46.

Weare

Nind

(2011) Mental health promotion and problem prevention in schools: What does the evidence say? Health Promotion International 26(suppl 1): i29–i69. doi: 10.1093/heapro/dar075.

47.

Weiss

Dang

H. M.

Lam

T. T.

Nguyen

M. C.

Nguyen

H. T.

Pollack

(2014) A nationally representative epidemiological and risk factor assessment of child mental health in Vietnam. International Perspectives in Psychology: Research, Practice, Consultation 3(3): 139–153. doi: 10.1037/ipp0000016.

48.

Weiss

Han

Tran

N. T.

Gallop

Ngo

V. K.

(2015) Test of ‘facilitation’ vs. ‘proximal process’ moderator models for the effects of multisystemic therapy on adolescents with severe conduct problems. Journal of Abnormal Child Psychology 43: 971–983. doi: 10.1007/ s10802-014-9952-z.

49.

Weiss

Harris

Catron

Han

S. S.

(2003) Efficacy of the RECAP intervention program for children with concurrent internalizing and externalizing problems. Journal of Consulting and Clinical Psychology and Psychotherapy 71: 364–374. doi: 10.1037/0022-006X.71.2.364.

50.

Weiss

Ngo

V. K.

Dang

H. M.

Pollack

Trung

L. T.

Tran

C. V.

(2012) A model for sustainable development of child mental health infrastructure in the LMIC world: Vietnam as a case example. International Perspectives in Psychology: Research, Practice, Consultation 1: 63–77. doi:10.1037/a0027316.

51.

Weist

M. D.

(2003) Challenges and opportunities in moving toward a public health approach in school mental health. Journal of School Psychology 41(1): 77–82. doi: 10.1016/S0022-4405(02)00146-2.

52.

Weist, M. D., Evans, S. W., & Lever, N. A. (Eds.). (2008). Handbook of school mental health: Advancing practice and research. New York, NY: Springer Science & Business Media.

53.

Weisz

J. R.

Somsong

Chaiyasit

Weiss

Achenbach

T. M.

Eastman

K. L.

(1993) Behavioral and emotional problems among Thai and American adolescents: Parent reports for ages 12–16. Journal of Abnormal Psychology 102(3): 395–402.

54.

World-Health-Organization (2007) Monitoring and evaluation of mental health policies and plans, Geneva: WHO.

55.

Wyer

R. S.

Chiu

Hong

(2009) Understanding culture: Theory, research, and application, New York, NY: Psychology Press.