Effects of Check and Connect on Attendance,Behavior,and Academics

Abstract

Objectives:

This study examined the effects of Check & Connect (C&C) on the attendance, behavior, and academic outcomes of at-risk youth in a field-based effectiveness trial.

Method:

A multisite randomized block design was used, wherein 260 primarily Hispanic (89%) and economically disadvantaged (74%) students were randomized to treatment or control conditions within 14 urban middle and high schools. The social service organization Communities In Schools implemented C&C in each of the schools, and the effects were compared to those of typical Communities In Schools services. Hierarchical linear modeling was used to account for the nested or random school-level effects when modeling student-level responses to the intervention.

Results:

Controlling for pretest performance and all relevant student- and school-level characteristics, C&C was significantly related to improvements in academic performance and reductions in disciplinary referrals. No significant effects were found for attendance.

Conclusions:

C&C is a promising intervention to improve outcomes for at-risk youth in school settings. Application to social work practice and research are discussed.

Keywords

Check & Connect dropout attendance randomized trial

Educational achievement and school completion are basic components of the healthy development of children and youth and of the success of young adults across their lives. Further, to be competitive in global markets, local, state, and national economies need an educated and skilled workforce. Despite the obvious benefits of academic success and school completion for both youth and society, too few students graduate from high school—particularly low-income students and students in racial and ethnic subgroups. For the class of 2007–2008, the proportion of public high school freshman who graduated with a regular diploma within 4 years of entering high school was 74.9%, with state averages ranging from 51.3% to 89.6% (Chapman, Laird, & KewalRamani, 2010). In 2008, the event drop-out rate—an estimate of students who left high school without earning a high school diploma or passing the General Educational Development tests—was 3.5%. Higher rates were found for Black (6.4%) and Hispanic (5.3%) students compared to White (2.3%) students. Students living in low-income families were also more likely to drop out of school. These students dropped out at a rate of 8.7% compared to 2.0% of students in high-income families (Chapman et al., 2010). In 2008, the status drop-out rate, which reflects the percentage of individuals in a given age range who are not in high school and have not earned a high school diploma or passed the General Educational Development tests, was reported at 8%, or approximately 3 million 16- to 24-year-olds. Status drop-out rates were higher for males than for females and higher for Black and Hispanic students than for White students (Chapman et al., 2010).

Although drop-out rates in the United States have been trending downward since 1972 (Chapman et al., 2010), current rates remain a serious social and economic issue. Dropout and poor academic achievement are related to numerous negative outcomes for individuals and society. High school dropouts typically have poorer physical and mental health (Vaughn, Salas-Wright, & Maynard, in press), are less likely to get a job, and earn significantly less relative to those who complete high school (Rouse, 2007). High school dropouts also are more likely to engage in criminal activity, be arrested, and be incarcerated (Bureau of Justice Statistics, 2004; Lochner & Moretti, 2004); less likely to engage in civic activity (Dee, 2004; Milligan, Moretti, & Oreopoulos, 2004); and less likely to report positive well-being (Oreopoulos & Salvanes, 2011; U.S. Department of Commerce, 2009) than those who graduate from high school. Moreover, it is estimated that high school dropouts cost society close to $240,000 over their lifetime in lower tax contributions, higher reliance on social welfare, and higher rates of criminal activity (Chapman et al., 2010; Levin & Belfield, 2007; Rouse, 2007). The high costs of dropout to individuals and societies; the rates at which U.S. high school students drop out; and the increased emphasis on academic achievement, test scores, and graduation through federal initiatives and the No Child Left Behind Act have highlighted the need to further understand predictors of dropping out as well as develop interventions to reduce dropout.

Although the causes of dropout are complex and several factors have been implicated, four of the most salient malleable student-level factors linked to dropout are academic achievement, attendance, engagement, and behavior (Rumberger, 2011). Poor grades and course failures are highly predictive of dropout. In a study of students in the Chicago Public School system, Allensworth and Easton (2007) found that grade point average and course failures were the most accurate predictors of nongraduates. Other studies have produced similar findings (see Balfanz, Herzog, & Mac Iver, 2007; Kurlaender, Reardon, & Jackson, 2008; Silver, Suanders, & Zarate, 2008). Rumberger and Lim’s (2008) review of the literature found that grades were a more consistent predictor of dropout than test scores, concluding that “grades are a more robust measure of academic achievement than test scores” (p. 19), because grades reflect both effort and ability throughout the school year, whereas test scores reflect ability measured on one or two days.

Attendance, engagement, and school behavior are additional factors consistently linked to dropout. For example, students who are regularly absent from school are more likely than consistent attenders to drop out (Henry, Knight, & Thornberry, 2012). In a study of ninth-grade students in the Chicago Public Schools, attendance was found to be highly predictive of course failure and dropout (Allensworth & Easton, 2005). In fact, freshman absences were 8 times more predictive of course failure than eighth-grade test scores and correctly identified nongraduates 77% of the time. In a review of 25 years of research related to why students drop out of school, Rumberger and Lim (2008) reported that the majority of the 35 studies measuring attendance found that high absenteeism predicted dropout. Likewise, engagement—often reflected by attendance among other social, behavioral, emotional, and cognitive indicators—has emerged as a strong predictor of dropout (Fall & Roberts, 2011; Rumberger, 2011; Rumberger & Lim, 2008). Of the 60 studies in Rumberger and Lim’s review, the majority found that engagement significantly predicted dropout. In addition, school misbehavior is associated with dropout. That is, students who misbehave in school are more likely to drop out than students without disciplinary problems (Ou, Mersky, Reynolds, & Kohler, 2007; Rumberger & Lim, 2008).

Intervening With Students at Risk of Dropping Out

Numerous programs have been designed to improve achievement, attendance, engagement, and behavior for students at risk of dropout. For school practitioners, identifying the most effective programs for reducing dropout and improving graduation rates can be a daunting task. To assist school practitioners in locating evidence-based interventions, national databases such as the What Works Clearinghouse, Blueprints for Violence Prevention, and the National Dropout Prevention Center are available. In addition, systematic reviews and meta-analyses have become more prevalent as a means to assist practitioners in making evidence-based decisions. The Campbell Collaboration is an international research network that prepares, maintains, and disseminates systematic reviews in education, social welfare, crime and justice, and international development to help practitioners and policy makers make well-informed decisions (see www.campbellcollaboration.org). The Campbell Collaboration has produced and published on its website several systematic reviews of education and social welfare interventions relevant to dropout and related risk behaviors (see Maynard, McCrea, Pigott, & Kelly, 2012, 2013; Wilson, Tanner-Smith, Lipsey, Steinka-Fray, & Morrison, 2011). One large systematic review published by the Campbell Collaboration examined effects of dropout intervention programs (Wilson et al., 2011). Findings from this review indicated that dropout programs were on average effective in reducing dropout regardless of the type of program. In addition to national databases and research synthesis to help guide intervention dissemination, some dropout prevention programs have been developed and disseminated on a national scale. One of the largest and most widely disseminated dropout prevention programs is Communities In Schools (CIS; www.communitiesinschools.org).

CIS, founded in 1977 by Bill Milliken, is a nonprofit, nationwide network of nearly 200 independent affiliates delivering a dropout prevention and intervention model in more than 3,000 schools across 28 states. The CIS network communicates the practice model by coordinating national, state, and local affiliate efforts around a mission to “surround students with a community of support, empowering them to stay in school and achieve in life” (Communities In Schools, n.d., heading section). The CIS model uses school-based case managers, hereafter referred to as site coordinators, to develop community partnerships, bring local resources to school campuses, and provide direct services to schools and students at risk of dropout.

CIS site coordinators function as a main point of contact for students and their families, connecting them with resources and supports to address both academic and nonacademic needs. As a primary element of the CIS model, site coordinators conduct annual needs assessments to identify the school’s and its students’ risk factors. The needs assessments are then used to select evidence-based services and organize schoolwide and individualized student service plans to diminish those concerns. School-level services include schoolwide activities or services accessible to all students, regardless of risk status, such as career days, college awareness activities, uniform or school supply assistance, and social service assistance. Individualized case management services provided to at-risk students include basic needs and resources, academic assistance (tutoring), mentoring, enrichment and motivation, life skills and social development, family engagement and strengthening, behavior interventions, community service and service learning, college and career preparation, and professional health and mental health services (CIS, 2011; ICF International, 2010).

CIS uses student and school performance data (i.e., grades, promotion rates, graduation rates) to report outcomes at the local affiliate, state, and national levels. The most recent CIS national annual report suggested that among students exposed to CIS case management services, 97% stayed in school, 84% were promoted to the next grade, and 88% of eligible seniors graduated (CIS, 2011). The data reported in the CIS national annual report, however, are aggregate data from CIS affiliates reported to CIS national; the method by which CIS collects and analyzes the data reported is unclear. In 2011, the results of a 5-year national evaluation of CIS were released, which included a quasi-experimental and three randomized studies conducted in Austin, Texas; Wichita, Kansas; and Jacksonville, Florida (ICF, 2010). The quasi-experimental study—which comprised 602 CIS schools matched with 602 comparable non-CIS schools across the United States, using a propensity score matching method—examined school-level effects of CIS. Small, but positive effects on dropout, attendance, and some academic outcomes were found (ICF, 2010). Effects varied, however, based on the CIS affiliates’ level of implementation, with larger effects found at high-implementing sites on many outcomes. For example, standardized mean difference effect sizes for dropout (.21) and graduation (.08) rates were smaller when calculated with data from all CIS schools than effect sizes calculated for high-implementer sites (.36 for dropout and .31 for graduation rates; ICF, 2010).

Although positive effects were found in the quasi-experimental study, results from the three randomized studies were mixed on most outcomes measured. From baseline to end of year 1, significant positive effects for dropout were found in Austin; positive, though not significant effects were found in Jacksonville; and null or negative effects were found in Wichita. On attendance, null or negative effects were found in Jacksonville and Wichita, and positive, significant effects were found in Austin. Similarly, positive, but not significant, effects on behavioral problems were found in Jacksonville, but negative or null effects were observed in Austin and Wichita. Effects for academics and student attitude and behavior outcomes were similar (i.e., significant; positive, but not significant; and null or negative), both within sites and between sites (ICF, 2010). Although this national 5-year evaluation, which used quasi-experimental and randomized designs, was an improvement over reports of within-group change from data received from local affiliates or states, the evaluation did not meet the What Works Clearinghouse evidence criteria standards, indicating substantial threats to internal validity and thus could not be fully reviewed (see Institute of Education Sciences, n.d.).

The national office of CIS, recognizing that needs and issues of students, schools, and communities vary across regions and states, encourages affiliates to use evidence-based interventions to address local school and individualized student needs. CIS specifies the service delivery model and process; however, the CIS model does not prescribe the specific interventions. Thus, local affiliates are able to select empirically supported interventions based on school and student needs and staff professional expertise and preference.

One local CIS affiliate in Texas sought an intervention to reduce attendance problems—a significant risk factor predicting dropout. The local affiliate partnered with The Meadows Center for Preventing Educational Risk to implement a dropout and engagement intervention, Check & Connect (C&C). C&C was selected, because it was an empirically supported intervention; had been successfully implemented in a prior study with researchers from The Meadows Center for Preventing Educational Risk; was recognized by CIS as an exemplary program to reduce dropout (Hammond, Linton, Smink, & Drew, 2007); and aligned with local goals to improve attendance, behavior, and academics.

C&C (Christenson, Sinclair, Thurlow, & Evelo, 1999; Sinclair, Christenson, Evelo, & Hurley, 1998) is a widely used intervention that is often cited in the literature as a promising intervention for improving school engagement and reducing dropout (Alvarez & Anderson-Ketchmark, 2010; Kelly, Raines, Stone, & Frey, 2010; Lehr, Johnson, Bremer, Cosio, & Thompson, 2004; Stout & Christenson, 2009). The What Works Clearinghouse reviewed evidence of C&C on two occasions related to (1) dropout prevention and (2) students classified as emotionally disturbed (What Works Clearinghouse, 2006, 2011). With regard to dropout prevention, two studies of C&C met evidence standards for reducing dropout—with one study meeting standards and the other meeting standards with reservations. What Works Clearinghouse rated C&C as having positive effects on staying in school, potentially positive effects on progressing in school, and no discernible effects on completing school. Four additional studies were identified for the dropout review; however, those studies did not meet relevance (conducted with elementary students and outcomes were not relevant to the review) or evidence (no control group or nonequivalent comparison group) standards to be considered in the review. With regard to students classified as emotionally disturbed, 24 studies were reviewed; however, no studies met What Works Clearinghouse standards of relevance for the review or evidence for improving outcomes. A recent, but yet unpublished, randomized study on the effects of C&C on school engagement found positive effects on behavioral and psychological engagement but not for cognitive engagement (Roberts, Vaughn, Vaughn, Wexler, Coleman, & Maynard, in press).

In summary, both CIS and C&C are well-known and widely adopted interventions to improve attendance and engagement among students at risk of school dropout. However, independent researchers have not rigorously evaluated either program. All of the underlying evidence supporting the efficacy of CIS was commissioned by the CIS national office or derived from within-group program evaluation designs reported by local CIS affiliates. Furthermore, the one study of CIS that was reviewed by What Works Clearinghouse did not meet evidence criteria. As such, many of the claims that CIS is associated with improvements in engagement, attendance, behavior, and academic performance lack internal validity to make any causal inferences. Similarly, the C&C program developers have conducted the few rigorous studies evaluating the effects of C&C. To bridge this critical gap in the literature, the present study used a randomized design to examine the effects of CIS compared to CIS plus C&C on attendance, academics, and behavior for youth referred for absenteeism.

Purpose of the Present Study

Absenteeism was a significant problem for the schools served by a local CIS affiliate in Texas. To address this concern, CIS staff members partnered with The Meadows Center for Preventing Educational Risk at The University of Texas at Austin to implement and conduct a field-based trial of C&C with students exhibiting absenteeism. As such, the study reported here positions the observations associated with C&C within the broader CIS service delivery model. That is, all participating CIS schools implemented universal, schoolwide interventions and individualized case management services to improve academics, behavior, and attendance. However, within participating CIS schools, students were randomly placed in one of the two conditions: (1) CIS services plus C&C or (2) CIS-only services. The research question guiding this study is as follows: Are there differences in effects on attendance, academics, and behavior for students who receive C&C in addition to CIS from those who receive only CIS services?

Method

Study Design and Procedures

Design

This study used a randomized block design to examine the effectiveness of C&C on academic performance, behavior, and attendance with at-risk middle and high school students during the 2011–2012 school year. Eligible students were randomly assigned to the treatment or control condition within each of the 14 participating CIS schools (9 middle schools, 4 high schools, and 1 middle/high school). We randomized students to condition within schools, as opposed to randomizing schools to condition, for several reasons. Because experiments estimate the average causal effect on the units that have been randomized, and we were interested in estimating effects on student outcomes, it was necessary to randomize students to condition. Moreover, randomizing at the lowest level possible allows for more precise estimates and greater power to detect effects (Rhoads, 2011; Shadish, Cook, & Campbell, 2002). Recruitment of study participants occurred at each school between September and November 2011. The intervention was delivered between November 2011 and May 2012.

Study Inclusion Procedures

Study inclusion criteria were applied at both the school and the student levels. At the school level, three factors determined school inclusion: (1) the school had contracted with CIS to provide case management and dropout prevention services, (2) the school was either a middle or high school, and (3) the school approved both the implementation of the C&C program and the study to assess the effectiveness of the program.

At the student level, CIS site coordinators and school staff members identified students to participate. To be eligible for participation in this study, students must have (1) not been previously enrolled in CIS services, (2) met eligibility criteria for CIS services at the time of study enrollment (i.e., met one or more of the criteria on the Texas Education Agency at-risk eligibility list or referred for family crisis; Texas Education Code §33.151 and §29.081), and (3) demonstrated absenteeism (defined by 20 or more absences during the prior school year or 2 or more absences during the previous month). After school staff members identified and referred students to CIS, the CIS site coordinators at each school provided information to the eligible students and parents about CIS services and the study, including information about participants’ rights to not participate in the study and voluntarily terminate participation at any time. For students from whom both parent consent and student assent were obtained, CIS conducted its standard assessment, which was not part of the study, with all participating students. Following the assessment, students were randomly assigned to treatment or control conditions.

Randomization Procedures

The institutional review board of the university where the study originated, the CIS affiliate, and administrators in all participating schools approved the study and randomization procedures. Randomization occurred within schools at the student level and was conducted by using an online random number generator (Haahr & Haahr, 2010). Student identification numbers (N = 260) were entered into the program that randomly generated numbers. Students were then sorted in ascending order of the randomly generated numbers. The list was then divided in half, with the first half assigned to the CIS plus C&C treatment group (n = 134) and the second half assigned to the CIS-only control group (n = 126). An on-site CIS coordinator trained on C&C acted as the C&C “monitor,” delivering C&C along with typical CIS services to students randomly assigned to the treatment group. A second site coordinator in each school who did not participate in the C&C training delivered only typical CIS services to students randomly assigned to the control condition.

Intervention Procedures

Because all participants received CIS services, this study examined the benefit of combining C&C with CIS compared to CIS alone. C&C is a manualized dropout prevention and intervention program originally developed to reduce drop-out rates for middle school students with emotional and behavioral disabilities (Sinclair et al., 1998). C&C is designed to promote students’ engagement in school through a targeted and individualized approach (for a full description of C&C, see http://checkandconnect.umn.edu and Sinclair et al., 1998). The C&C model comprises two primary components. The check component involves regularly monitoring student data related to alterable risk indicators. The connect component involves building relationships with students and families and facilitating basic or intensive interventions based on student data.

The C&C model is delivered by an adult “monitor” who uses a case management approach to work with students and their families for the duration of the intervention. The primary goal of the monitor is “to keep education a salient issue for the student, his or her family members, and teachers, and to reduce and prevent the occurrence of absenteeism, suspensions, failing grades and other warning signs of school withdrawal” (Sinclair et al., 1998, p. 10). The monitor works with students and families on his or her caseload to promote school engagement by building and maintaining relationships, monitoring student data related to alterable risk indicators, and implementing individualized interventions with students and families, based on the data.

In this study, CIS site coordinators acted as the C&C monitors. The C&C monitors were school-based practitioners who had been employed with CIS for a mean of 4.38 years. All C&C monitors were female (100%) with education and experience in the fields of psychology, counseling, or social work. Most of the C&C monitors had a master’s degree (64%) and the remaining 36% held a bachelor’s degree. None of the C&C monitors had delivered the C&C intervention prior to the training for the present study. The monitors provided C&C to up to 12 students, in addition to providing CIS services to their regular caseload. The monitors recorded attendance, tardiness, behavioral referrals, and academic performance weekly on a form adapted from Sinclair et al. (1998) and sent the form to the CIS program manager for fidelity monitoring. The data were used to provide feedback to students, determine students’ level of risk, and develop and implement individualized interventions based on risk indicators. Monitors met with students weekly to discuss progress, discuss the importance of staying in school, and assist students with problem solving related to current or ongoing issues (Sinclair et al., 1998). For students exhibiting high risk on the indicators being monitored, individualized interventions were developed according to student risk factors and needs. The student and C&C monitor developed these additional interventions, based on the monitor’s professional expertise and judgment, student and family input, and resources in the school and community (Sinclair et al., 1998).

Training

All CIS site coordinators selected for implementation of the intervention received a full-day training on the C&C intervention. The training consisted of didactic components and role-playing. In addition, the session provided training on processes and requirements of the research, including protection of human subjects, informed consent, study and data collection procedures, and problem solving regarding implementation barriers. Additionally, interventionists were provided with a half-day booster training and support session in December 2011, and the co-principal investigator was available throughout the study period to provide consultation and assistance as needed.

Fidelity

Due to limited resources, we could not comprehensively or rigorously measure fidelity of implementation; however, we used several strategies to promote and monitor fidelity over the course of the study to ensure the intervention was being implemented as intended. C&C is a well-specified intervention with an intervention manual that provides for standardization, reduces variability of implementation, and provides sufficient information for the intervention to be replicated and compared across the studies, thus enhancing both internal and external validity (Gearing, El-Bassel, Gesquire, Baldwin, Gillies, & Ngeow, 2011; Smith, Daunic, & Taylor, 2007). As mentioned previously, all CIS site coordinators who implemented C&C were trained and monitored to enhance the competence and adherence of the implementers, reduce implementer drift, and correct any deviations from the intervention in real time over the course of the study (Bellg et al., 2004; Gearing et al., 2011; Perepletchikova & Kazdin, 2005).

Participants

All students participating in this study met Texas Education Agency criteria for eligibility for CIS services (see Texas Education Code sections 29.081 and 33.151). Participating students averaged 5.06 days absent and .23 disciplinary referrals at baseline. About half (56%) of the participants were female, and the majority (89%) of the students were Hispanic. The average age of the participants was 15.1 years, with the majority of students in grades 6 through 9. Students participating in this study were, by large, economically disadvantaged as evidenced by the majority (74%) receiving free or reduced-price lunch.

Attrition

Due to attrition resulting from varied and sometimes unspecified reasons including, but not limited to mobility, transfer to alternative schools, early graduation, suspension/expulsion, incarceration, and other reasons, posttest data were available for only 189 of the 260 students randomized to the treatment and control groups. The final analytic sample consisted of 89 students in the treatment group and 100 students in the control group (see Figure 1). The total (27%) and differential (13%) attrition rates were relatively high, although not uncommon in school-based field research with at-risk students in urban settings (see Wilson & Lipsey, 2006; Wilson et al., 2011).

Figure 1.

Participant flowchart. Participant percentages reported at each stage are calculated using the number of participants in prior stage. N(n) = number of students; J(j) = number of schools.

Analytic Sample

Analysis of selection bias and pretest equivalency of the analytic sample suggested that randomization was successful; that is, the analytic treatment and control groups appear statistically balanced at pretest on all demographic characteristics and outcomes. As seen in Table 1, for students randomized to the C&C or the control condition, contingency tables and χ² tests for dichotomous variables suggest the groups were similar with regard to sex, grade, ethnicity, and free and reduced-price lunch status. Likewise for continuous variables, t-tests suggest the treatment and control groups were statically balanced at pretest by age, grade, family income, and on all outcome variables (i.e., academic performance, discipline, attendance).

Table 1.

Demographic Characteristics and Pretest Equivalence of Treatment and Control Groups.

Variable	Total (N = 189)		C&C (n = 89)		Control (n = 100)
Variable	%	N	%	n	%	n	χ ² (df)	p
Sex
Male	44	84	38	34	50	50	2.66 (1)	.10
Female	56	105	62	55	50	50	2.66 (1)	.10
Grade								.10
6th	20	37	19	17	20	20	1.71 (6)
7th	15	28	15	13	15	15
8th	21	40	24	21	19	19
9th	23	43	21	19	24	24
10th	7	13	6	5	8	8
11th	6	12	8	7	5	5
12th	8	16	8	7	9	9
Ethnicity								.09
African American	11	20	15	13	7	7	2.88 (1)
Hispanic	89	169	85	76	93	93	2.88 (1)
Free or reduced lunch								.52
Yes	74	140	72	64	76	76	0.41 (1)	.52
No	26	49	28	25	24	24
	M	SD	M	SD	M	SD	T (df = 187)^#	P
Age	15.1	2.02	15.2	0.20	15.0	0.21	0.51	.61
Grade	8.35	1.80	8.34	0.19	8.36	0.18	0.04	.97
Income	42.46	36.69	43.03	3.95	41.35	3.63	−0.20	.84
Academic performance	75.63	8.70	76.56	7.73	74.78	9.46	1.36	.17
Discipline	0.23	0.65	0.169	0.65	0.30	0.72	1.38	.17
Attendance	5.06	4.47	4.63	3.83	5.44	4.97	1.25	.22

Note. C&C = Check & Connect; df = degrees of freedom; SD = standard deviation.

Measures

Outcomes of interest in this field-based trial were selected based upon prior studies examining the effects of C&C, the priorities and interests of CIS, and the availability of data already being collected by the school or the CIS. It was important for us to measure proximal outcomes that did not impose additional burdens on the school or staff in terms of time or money and that practitioners could sustain beyond the involvement of external researchers. More specifically, we focused on the intervention’s effect on average student academic performance (Lehr, Sinclair, & Christenson, 2004; Sinclair et al., 1998), student behavior (Todd, Campbell, Meyer, & Horner, 2008), and student attendance (Lehr et al., 2004; Sinclair, Christenson, Lehr, & Anderson, 2003).

Dependent Outcomes

Three dependent variables were used as outcomes in the present study. First, to assess the effect of the intervention on academic performance, we generated performance composites for all students in the data set, using English, mathematics, science, and social studies grades. Pretest performance composites (α = .73) were generated, using grades earned during the marking period prior to enrollment in the study. Posttest performance composites were generated from the grades earned during the study period (α = .82). Second, to assess the intervention effects on student discipline, the total count of office referrals was tallied for each student. That is, the pretest discipline outcome was generated using the total number of office referrals for the marking period prior to enrollment in the study; posttest discipline outcomes consisted of the number of referrals received during the last marking period. Third, to assess the impact of the intervention on student attendance, the pretest measure consisted of the total number of days absent in the marking period prior to the study, whereas the posttest measure consisted of the number of days missed during the last marking period.

Covariates

Covariates were modeled at the student and school levels. At the student level, all models controlled for student free and reduced-price lunch status, race, sex (nominal, 0 = no; 1 = yes), age, grade, and family income. School-level covariates included school size; the percentage of students considered at risk, highly mobile, disadvantaged, or of limited English proficiency; and the percentage of students meeting standards on the state-level achievement test. Following conventions in multilevel modeling, all continuous student- and school-level predictors were grand mean centered and each model controlled for pretest functioning on the outcome of interest (Raudenbush, Bryk & Congdon, 2002; Singer, 1998).

Analysis Strategy: Hierarchical Linear Modeling (HLM)

Because randomization to treatment and control groups occurred at the individual level within staff who provided C&C services in each school setting, the student-level data used to analyze treatment effects were nested within each school to account for variation in outcomes that may have occurred due to school- or staff-level effects. In addition, prior research suggests schools have contextual factors (e.g., average school-level poverty, school size, mobility) that strongly influence individual student performance and behavior (Raudenbush, Bryk, & Congdon, 2002; Singer, 1998). Therefore, HLM was used to account for the nested or random school-level effects when modeling student-level responses to the intervention.

A stepwise HLM model estimation procedure was used to fit each dependent variable (Raudenbush et al., 2002). In Step 1, an unconditional, fixed-effects model was fit to each outcome to estimate the intraclass correlation (ICC)—or the proportion of variance in outcomes attributed to school-level effects. The ICC for academic performance (ρ = .09), discipline (ρ = .15), and attendance (ρ = .12) indicated that 9%, 15%, and 12% of the variation in posttest outcomes were attributed to school-level effects, respectively—sufficient magnitudes to warrant the use of HLM.

In Step 2, conditional models were fit to test the intervention effects. In Figure 2, the Level-1 equation fit each outcome (Y_ij )—average school performance and total number of discipline referrals—for each student (i) in each school (j). Each estimated outcome was derived from the sum of the intercept (β_0j) as a condition of student pretest scores (β_1j), treatment assignment (β_2j), student-level covariates (β_3j – β_7j), and a fixed error term that refers to the unexplained residual within schools (r_ij ). The Level-2 equation models the random intercept for the Level-1 equation as a function of the school average effect (π₀₀) conditioned by school size (π₀₁) and school-level proportions of students considered disadvantaged (π₀₂), of limited English proficiency (π₀₃), highly mobile (π₀₄), academically proficient (π₀₅), and at risk (π₀₆). The Level-2 equation also models a between-school residual term (e _0j).

Figure 2.

Two-level random intercept model.

In Step 3, random slope and intercept models were tested by fitting cross-level interactions and random effects. Cross-level interactions were represented by product terms created by multiplying the treatment assignment variable (i.e., CIS + C&C = 1; CIS only = 0) and student- and classroom-level variables to examine whether the effects of C&C were moderated by those characteristics. Random effects and interaction terms were retained in the models only if significant. All associations were assessed using a two-tailed test and α = .05. Model estimates were generated in STATA 11.0 (StataCorp, 2005), using xtmixed and a maximum likelihood estimator—which performs well when the number of Level-2 units is not large (Bryk & Raudenbush, 1992).

In Step 4, following model estimation, effect sizes were estimated for all outcomes significantly associated with C&C (Cohen, 1988). Effect sizes were calculated for average academic performance and discipline using approaches suggested for multilevel data (Spybrook, Raudenbush, Congdon, & Martínez, 2009). Figure 3 represents the effect size equation, where β is the multilevel coefficient for the treatment variable on the outcome of interest, τ² is the residual variance between schools, and σ² represents the residual variance within schools.

Figure 3.

Effect size estimate for multilevel data structures.

Results

The multilevel model estimates are offered in Table 2. No association was observed between study conditions for the attendance outcome. In addition, no significant random effects or associations were observed for all interactions between C&C and student-level fixed effects. As such, random effects and product terms were not retained in the final models and will not be reported here.

Table 2.

Student-School Fitted Hierarchical Linear Models: The Effects of C&C (N = 189).

		Academic performance		Discipline		Attendance
Level	Effect	β	SE	β	SE	β	SE
Student	Conditional mean	14.025***	7.262	2.019**	1.305	13.537*	7.776
	Pretest	.679***	.052	1.208***	.138	.731***	.123
	Age	−1.447**	.576	.076	.127	.760	.761
	Grade	1.418*	.738	−.188	.154	−1.310	.907
	Sex	.142	.829	−.149	.183	−.796	1.093
	Free lunch	−2.773**	1.125	−.070	.235	.551	1.392
	Income	.014	.011	−.003	.003	−.026*	.015
	African American	1.072	1.400	.544*	.139	−1.736	1.905
	Tx (C&C)	1.547*	.765	−.363*	.173	−.577	1.033
School	School size	.002	.003	−.000	.000	.002	.002
	% disadvantaged	.017	.263	−.009	.039	.235	.224
	% LEP	.415	.255	.035	.036	−.189	.202
	% at risk	−.171	.113	−.007	.016	.013	.089
	% mobility	.414**	.155	.021	.021	.062	.117
	% average performance	.079	.112	.017	.017	.015	.095

Note. C&C = Check & Connect; SE = standard error; academic performance = composite of English, mathematics, science, and social studies; discipline = total number of office referrals; attendance = total number of days missed; LEP = limited English proficiency. Tx = C&C = 1, control = 0; income and percentage of disadvantaged students, LEP, at risk, mobility, and average school-level performance all grand mean centered. All hypothesis tests are two tailed.

*p < .10. **p < .05. ***p < .001.

Significant associations were observed between study conditions on academic performance and discipline. The pretest covariate was significant for all outcomes, and—with the exception of sex (male = 1; female = 0), race (African American = 1; Hispanic Latino = 0), and average school-level mobility—the student- and school-level covariates were not significantly associated with the dependent variables.

Academic Performance

First, controlling for pretest performance and all relevant student- and school-level characteristics, C&C was positively related to improvements in posttest student academic performance, 1.547 (p = .043, 95% CI [.047, 3.048]). That is, on average, students randomized to the C&C condition, compared to control students, evidenced 1.547 percentage points higher on their average academic performance during the study compared to preintervention levels of performance. In addition, the coefficients for age, grade, and free and reduced-price lunch were significantly associated with the average academic performance.

The coefficient for the grand mean centered variable, age, −1.447 (p = .012, 95% CI [−2.575, −.321]), suggests that the average-age student in the study (15.07 years) scored 1.447 percentage points lower on their average academic performance during the study compared to preintervention performance. The coefficient for grade, 1.418 (p = .05, 95% CI [2.86, .028]), suggests the students in grade 8, the average grade of students in the study, scored 1.418 percentage points higher on their average academic performance during the study compared to preintervention performance. Finally, the coefficient for free and reduced-price lunch, −2.773 (p = .01, 95% CI [−4.978, −.568]), suggests that students receiving free and reduced-price lunches scored 2.773 percentage points lower on their average academic performance during the study compared to preintervention levels.

Discipline Referrals

Controlling for pretest performance and all relevant student- and school-level characteristics, C&C was negatively associated with the total number of office referrals received during the study period compared to the marking period before the study, −.363 (p = .036, 95% CI [−.703, −.023]). That is, on average, students randomized to the C&C condition, compared to control students, had .363 fewer office disciplinary referrals at posttest. In addition, students who identified as African American, .544 (p = .088, 95% CI [0.082, 1.17]), compared to Hispanic Latino students, had .544 more office referrals than during the preintervention marking period.

Effect Size Estimates and Improvement Index

The effect sizes representing the changes in the dependent variable attributed to C&C were small, according to Cohen’s metric. For the changes in academic performance, the effect size was δ = .07. For the reductions in disciplinary referrals for the study period compared to the prior marking period, the intervention’s effect size was δ = −.27. Although no significant effects were observed for attendance, the effect size was δ = −.01.

To translate effect sizes into terms that illustrate the practical meaning of the intervention’s effect, we used an improvement index as suggested by the Institute of Education Sciences (2008). An improvement index represents the change observed between the percentile rank of the average student in the intervention group compared to the average student in the control group. Alternatively, an improvement index can be interpreted as the expected change that would be observed if the average student in the control group were to receive the intervention.

To convert an effect size to an improvement index entails using a standard normal curve for z-scores. An improvement index suggests the effect size of .07 for academic achievement translates into a 3% improvement for the average student in the intervention compared to a student in the control condition. An improvement index for a −.27 effect size for disciplinary referrals translates into an 11% reduction in disciplinary referrals for the average student in the C&C group compared to a student in the control group. Finally, an improvement index for a −.01 effect for attendance translates into a .04% improvement in the average student in the intervention compared to one in the control condition.

Discussion and Applications to Social Work

High school dropout is a significant social and public health concern that affects not only dropouts across their life course but also the society as a whole. The drop-out problem in the United States has been referred to as a crisis, and substantial attention and effort have been directed at reducing drop-out rates (Rumberger, 2011). Although the drop-out rate has declined over the past 2 decades, too many students continue to drop out of school. As such, there is a significant need to develop, implement, test, and refine interventions that seek to reduce dropout and improve school engagement.

The present study examined the effects of C&C, a dropout and school engagement intervention, with students at risk of dropout, as defined by high rates of absenteeism, poor school behavior, and below-average academic performance—three critical risk factors in predicting eventual dropout. The current study adds to the evidence base for C&C, supporting the findings of prior studies by the developers of C&C (Sinclair et al., 1998). The current study, being the only independent randomized effectiveness trial of C&C, not only adds rigor to the existing evidence but also was conducted with a predominantly Hispanic population—a student subgroup that prior studies suggest are at an increased risk of dropout (Chapman et al., 2010).

The findings summarized here extend empirical support for C&C as a promising intervention for providing academic and behavioral supports to at-risk students referred for absenteeism. Specifically, students who received C&C had better grades and fewer disciplinary referrals compared to the students in the control group. However, no significant effects were found for attendance. Although the findings are mixed and effects are small, the findings are, in several ways, impressive, considering that C&C was compared in this trial to another dropout prevention program, CIS; that C&C was implemented in a real-world setting by practitioners with minimal support from the university researchers; and that outcomes were assessed after one semester of implementation versus the 2 years recommended by the C&C developers (Sinclair et al., 1998).

Two key outcomes for which we observed statistically significant effects, student grades and behavior, indicate promise in preventing dropout, improving protective factors, and reducing risk factors for at-risk youth. Of the known dropout risk factors, student grades is the strongest predictor of dropout (Allensworth & Easton, 2007), and student misbehavior is a well-established risk indicator for dropout (Battin-Pearson et al., 2000; Rumberger & Lim, 2008). Indeed, prior research on key proximal indicators, such as academic performance and school behavior, suggest that interventions seeking to alter the cumulative impact of poor grades and disruptive behavior can positively influence the developmental sequencing of risk across childhood. That is, despite a collection of risk factors beyond the walls of a school, students who do well academically, have few behavioral problems in school, and healthy relationships with peers and teachers tend to experience proximal school success. Proximal success in school can confer long-term benefits, which promote the likelihood that students will graduate, attend college, and participate in labor markets while simultaneously decreasing the likelihood that those at-risk students drop out, subsist on welfare, or engage in criminal behavior (Burt & Roisman, 2010; Heckman & Kautz, 2012; Wentzel, 2002). As observed in prior school-based prevention studies, early intervention focused on both academic and nonacademic risk factors may interrupt a cascade of events associated with economic disadvantage that eventually result in costly social and health problems by adulthood (Bradshaw, Reinke, Brown, Bevans, & Leaf, 2008; Bradshaw, Zmuda, Kellam, & Ialongo, 2009; Hawkins, Kosterman, Catalano, Hill, & Abbott, 2008). Indeed, as both grades and behavior are significant risk factors for dropout, improving proximal functioning related to these outcomes may translate into long-term benefits, such as improved school-completion rates. These findings are consistent with Sinclair, Christenson, & Thurlow (2005) finding that youth who received C&C had lower rates of dropout than a control group.

The absence of significant effects on attendance found in this study, however, runs contrary to recent experimental studies of C&C that have measured behavioral engagement, of which attendance is often considered an indicator. A prior study observed a large treatment effect on behavioral engagement, as assessed by the School Dropout Risk Inventory, a self-report questionnaire that estimates students’ likelihood of dropping out of school, based on dispositional and contextual sources of risk (Roberts et al., in press). The behavioral engagement subscale of the School Dropout Risk Inventory includes 6 items and measures the extent to which students conformed to classroom rules and norms, such as completing homework, getting in trouble in school, being absent from school, and skipping class. Likewise, Sinclair et al. (2005) found that students who received C&C were more likely to demonstrate greater consistency in school attendance than the control group. In two single-group pretest–posttest studies of C&C, Lehr et al. (2004) found a decline in absences among the elementary students who received C&C, and Christenson et al. (1999) found decreased risk based on a composite measure of absences, course grades, and behavior. The C&C studies that found positive effects on attendance or behavioral engagement were implemented for at least 2 years. Because students in the present study received the C&C intervention for one school semester, it is possible that C&C treatment effects on attendance require the intervention be sustained over a longer period of time.

The positive effects on grades and behavior and the absence of effects on attendance found in this study are somewhat puzzling, given the significant correlation between absenteeism, grades, and behavior found in longitudinal (Henry & Huizinga, 2007) and population-based (Vaughn, Maynard, Salas-Wright, Perron, & Abdon, 2013) studies. As such, we would have expected that if a reduction in disciplinary infractions occurred, an associated increase in attendance would follow. Moreover, no discernible increase in attendance would then result in a lack of effects found for grades. Although attendance is associated with a number of negative outcomes, including poor grades, externalizing behaviors, and dropout, increasing attendance does not necessarily lead to improvement in problems correlated with attendance. For example, although dropout prevention programs, overall, have been found to be effective in reducing dropout (Wilson et al., 2011), meta-analytic findings from a recent systematic review found mixed effects of dropout programs on attendance outcomes (Tanner-Smith & Wilson, 2013). The meta-analysis did not directly test mediating effects of attendance on dropout; however, the lack of consistency in positive outcomes on attendance and dropout in studies that measured both outcomes “cast doubt on the assumption that dropout prevention programs may also decrease absenteeism, or that absenteeism is simply a point along the ‘dropout continuum’” (Tanner-Smith & Wilson, 2013, Discussion section, para. 7). Therefore, an intervention that does not positively affect attendance may nonetheless reduce dropout.

The present study is not without limitations, and findings should be interpreted accordingly. First, an intent-to-treat analysis was used in this study. However, we were not able to complete a full application of an intent-to-treat analysis as planned due to missing data resulting from participant attrition from the study. Total and differential attrition rates experienced in this study were relatively high and exceeded acceptable attrition rates established by the What Works Clearinghouse; thus, this study would not earn the highest rating, “meets evidence standards” from the What Works Clearinghouse (U.S. Department of Education, 2011). In the presence of missing outcome data, an intent-to-treat analysis can yield biased results (Shadish et al., 2002). We followed the What Works Clearinghouse guidelines for randomized controlled trials with high attrition and examined baseline equivalence on student characteristics and outcome variables. Our results indicate that the analytic treatment and control groups were comparable on all observed variables. In addition, we controlled for demographic variables and pretest scores on outcome variables in our analyses. Although the study estimates may be biased by the presence of differential attrition or unobserved heterogeneity, the analytic sample was balanced on observed variables at pretest and a randomized design minimizes threats to internal validity.

A second limitation is the lack of measurement of fidelity of the intervention and rigorous assessment of the counterfactual, due to limited resources. To begin, fidelity is a bifurcated concept consisting of surface features (counting the number of intervention elements students experience) and quality features (a deeper understanding of implementation issues such as language and examples used to teach that impacts student skill acquisition). As such, future research may take into account the bonding or quality of the relational supports provided to students by C&C staff. It may be that certain staff behaviors may lead to increased bonding for students. As a result, it is possible that significant outcomes observed in this study could be attributable to influences of unknown variables. Similarly, the lack of positive effects on attendance and the small effects on behavior and academics could be due to the intervention not being implemented as designed. We did, however, implement several strategies to promote, monitor, and enhance fidelity. These strategies included having a well-defined, manualized, and replicable intervention; providing initial and booster training sessions to the implementers; and monitoring implementation through weekly fidelity monitoring forms completed by the implementers and reviewed by the coinvestigator.

A third limitation is the relatively short period within which the intervention was implemented. In this study, students received the intervention for approximately 6 months, as opposed to the 2 years recommended by Sinclair et al. (1998). As a result, students may not have received the full benefit of C&C, possibly explaining the smaller effects on behavior and grades and null effects on attendance.

Our choice of outcomes—school-reported grades, office disciplinary referrals, and attendance—was both pragmatic, in that the data were readily available, as well as purposeful, in that the schools and the CIS affiliate were interested in positively affecting these proximal outcomes. School data can, however, be idiosyncratic to schools and may not always be reliably tracked and reported, potentially introducing bias. However, because our sample was randomized within schools, we believe that any bias resulting from idiosyncratic practices at individual schools was balanced across the treatment and the control groups. Moreover, school archival data are commonly used in school-based research (Irvin, Tobin, Sprague, Sugai, & Vincent, 2004). Thus, the limitations of this study from the use of school-reported data are not particularly uncommon, although they are important to note.

Despite the limitations, this randomized study is one of a few rigorous intervention studies of C&C, provides evidence of effects with a population different from prior C&C studies, and adds a level of internal validity rarely found in studies of school-based intervention research. Moreover, this study provides evidence of the effects of C&C implemented in a real-world setting by school-based practitioners, situating effect sizes within the context of C&C being implemented under conditions that practitioners would normally experience.

Conclusion

Dropout and related risk factors, such as school disengagement, absenteeism, behavioral problems, and poor academic performance, are issues with which social workers, counselors, and psychologists are frequently confronted and for which they expected to intervene. Generating and using evidence for practice has been a growing mandate in social work and related fields, and using evidence to address drop-out and related issues is of no exception. Implementing evidence-informed interventions to improve behavioral and academic outcomes for at-risk students is of critical importance to reducing dropout and improving social and behavioral health outcomes.

C&C has received significant attention as a promising intervention to improve engagement and reduce dropout but has limited evidence of effects, particularly experimental evidence and evidence with varying populations. This study provides rigorous empirical evidence in support of C&C for a different sample of at-risk students from prior studies of C&C, Hispanic and absentee students. Thus, this study uniquely contributes to the evidence of C&C and provides additional evidence that school-based practitioners can use inform practice decisions.

Although this randomized study adds to the knowledge base of social work intervention research, to fully engage in evidence-based practice as a profession, we need a robust body of rigorous, experimental evidence of effects of interventions (Soydan, 2008). There is, however, a dearth of rigorous intervention research in social work (Horton & Hawkins, 2010; Maynard, Vaughn, & Sarteschi, 2012; Rosen, Proctor, & Staudt, 1999). Challenges of conducting rigorous intervention research in social work have been discussed previously (see Fraser, 2004; Geierstanger, Amaral, Mansour, & Walters, 2004). Although real challenges do exist, real or perceived challenges are often too easily dispensed as barriers, often prematurely thwarting attempts at rigorous intervention research. Methodological and substantive advances, as well as advances in implementation models, provide social work researchers and practitioners with access to more tools and models to conduct intervention research than ever before (Fraser, 2004). Moreover, university–community partnership models have been advanced as a means of bridging practice and research that social work can use to build evidence-based practice and practice-based evidence (Begun, Berger, Otto-Salaj, & Rose, 2010).

This study provides evidence that university–community partnerships can work to build rigorous evidence of effects of interventions in real-world settings, using existing resources. Although this research was supported by a postdoctoral grant through the Institute of Education Sciences to support the position of the principal investigator, the principal investigator could have done the same work as a faculty member at a university without external funding. Resources beyond the time and effort of the university and community personnel involved to implement and conduct the study were not necessary, as the team worked together to use and build upon existing resources, processes, and systems. Although there are many challenges to conducting rigorous research in school settings, the delivery of interventions within the school context by school personnel affords researchers an understanding of the limitations, strengths, and real-world effects of a program. The data collected from such research also position effect size estimates in the context of a real school setting. Although randomization strengthens internal study validity, alternative designs (i.e., switching replications or regression discontinuity) can simultaneously alleviate ethical and methodological issues frequently raised regarding the randomization of children in need of services. Despite potential challenges, conducting rigorous intervention research in social work is not only possible, but critical to advancing the field and building the knowledge needed to provide effective services to improve student outcomes.

Footnotes

Acknowledgments

The authors would like to thank the local CIS affiliate for their commitment to building and using evidence to improve practice, the CIS site coordinators for their hard work throughout the implementation and data collection process, participating schools for their support and commitment to their students, and the students who participated in the study.

Authors’ Note

The content is solely the responsibility of the authors and does not necessarily represent the official views of the supporting entities.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The authors are grateful for support from the Meadows Center for Preventing Educational Risk at the University of Texas at Austin and the Institute of Educational Sciences (grant #R324B080008).

References

Allensworth

Easton

J. Q.

(2005). The on-track indicator as a predictor of high school graduation. Chicago, IL: Consortium on Chicago School Research, University of Chicago. Retrieved from http://ccsr.uchicago.edu/content/publications.php?pub_id=10

Allensworth

Easton

J. Q.

(2007). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Chicago, IL: University of Chicago Consortium on Chicago School Research. Retrieved from http://ccsr.uchicago.edu/sites/default/files/publications/07%20What%20Matters%20Final.pdf

Alvarez

M. E.

Anderson-Ketchmark

. (2010). Review of an evidence-based school social work intervention: Check & connect. Children & Schools, 32, 125–127.

Balfanz

Herzog

Mac Iver

D. J.

(2007). Preventing student disengagement and keeping students on the graduation path in urban middle-grades schools: Early identification and effective interventions. Educational Psychologist, 42, 223–235.

Battin-Pearson

Newcomb

M. D.

Abbott

R. D.

Hill

K. G.

Catalano

R. F.

Hawkins

D. J.

(2000). Predictors of early high school noncompletion: A test of five theories. Journal of Educational Psychology, 92, 568–582. doi:10.1037/0022-0663.92.3.568

Begun

A. L.

Berger

L. K.

Otto-Salaj

L. L.

Rose

S. J.

(2010). Developing effective social work university-community research collaborations. Social Work, 55, 54–62.

Bellg

Borrelli

Resnick

Hecht

Minicucci

D. S.

Ory

… Czajkowski

(2004). Enhancing treatment Fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23, 443–451.

Bureau of Justice Statistics. (2004). Survey of inmates in state and federal correctional facilities, 2004. Ann Arbor, MI: Inter-University Consortium for Political and Social Research. doi:10.3886/ICPSR04572

Burt

K. B.

Roisman

G. I.

(2010). Competence and psychopathology: Cascade effects in the NICHD study of early child care and youth development. Development and Psychopathology, 22, 557–567. doi:10.1017/S0954579410000271

10.

Bradshaw

C. P.

Reinke

W. M.

Brown

L. D.

Bevans

K. B.

Leaf

P. J.

(2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1–26. doi:10.1037/a0012883

11.

Bradshaw

C. P.

Zmuda

J. H.

Kellam

S. G.

Ialongo

N. S.

(2009). Longitudinal impact of two universal preventive interventions in first grade on educational outcomes in high school. Journal of Educational Psychology, 101, 926–937. doi:10.1037/a0016586

12.

Bryk

A. S.

Raudenbush

S. W.

(1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.

13.

Chapman

Laird

KewalRamani

(2010). Trends in high school dropout and completion rates in the United States: 1972–2008 (NCES 2011-012). Washington, DC: National Center for Educational Statistics. Retrieved from http://nces.ed.gov/pubsearch

14.

Christenson

S. L.

Sinclair

M. F.

Thurlow

M. L.

Evelo

(1999). Promoting student engagement with school using the Check & connect model. Australian Journal of Guidance and Counseling, 9, 169–183.

15.

Cohen

(1988). Statistical power analysis for the social sciences. Hillsdale, NJ: Erlbaum.

16.

Communities In Schools. (2011). Unlocking potential: 2011 annual report. Arlington, VA: Communities In Schools National Office. Retrieved from www.communitiesinschools.org

17.

Communities In Schools. (n.d.). Who we are. http://www.communitiesinschools.org/about/

18.

Dee

T. S.

(2004). Are there civic returns to education? Journal of Public Economics, 88, 1697–1720.

19.

Fall

A. M.

Roberts

(2011). High school dropouts: Interactions between social, context, self-perceptions, school engagement, and student dropout. Journal of Adolescence, 35, 787–798.

20.

Fraser

M. W.

(2004). Intervention research in social work: Recent advances and continuing challenges. Research on Social Work Practice, 14, 210–222.

21.

Gearing

R. E.

El-Bassel

Gesquiere

Baldwin

Gillies

Ngeow

(2011). Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review, 31, 79–88.

22.

Geierstanger

S. P.

Amaral

Mansour

Walters

S. R.

(2004). School-based health centers and academic performance: Research, challenges, and recommendations. Journal of School Health, 74, 347–352.

23.

Haahr

(2010). List randomizer. Retrieved from http://www.random.org/lists

24.

Hammond

Linton

Smink

Drew

(2007). Dropout risk factors and exemplary programs: A technical report. Clemson, SC: National Dropout Prevention Center. Retrieved from http://www.dropoutprevention.org/major-research-reports/dropout-risk-factors-exemplary-programs-technical-report

25.

Hawkins

J. D.

Kosterman

Catalano

R. F.

Hill

K. G.

Abbott

R. D.

(2008). Effects of social development intervention in childhood 15 years later. Archives of Pediatric and Adolescent Medicine, 162, 1133–1141. doi:10.1001/archpedi.162.12.1133

26.

Heckman

J. J.

Kautz

T. D.

(2012). Hard evidence on soft skills. Labour Economics, 19, 451–464. doi:10.1016/j.labeco.2012.05.014

27.

Henry

K. L.

Huizinga

D. H.

(2007). School related risk and protective factors associated with truancy among urban youth placed at risk. Journal of Primary Prevention, 28, 505–519.

28.

Henry

K. L.

Knight

K. E.

Thornberry

T. P.

(2012). School disengagement as a predictor of dropout, delinquency, and problem substance use during adolescence and early adulthood. Journal of Youth and Adolescence, 41, 156–166.

29.

Horton

E. G.

Hawkins

(2010). A content analysis of intervention research in social work dissertations. Journal of Evidence-Based Social Work, 7, 377–386.

30.

ICF International. (2010). Communities In Schools national evaluation five-year summary report. Fairfax, VA: ICF International. Retrieved from http://www.communitiesinschools.org

31.

Institute of Education Sciences. (2008). What Works Clearinghouse procedures and standards handbook (version 2.0). Retrieved from http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_standards_handbook.pdf

32.

Institute of Education Sciences. (n.d.). Dropout prevention publications and reviews. Retrieved from http://ies.ed.gov/ncee/wwc/topic.aspx?sid=3

33.

Irvin

L. K.

Tobin

T. J.

Sprague

J. R.

Sugai

Vincent

C. G.

(2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions, 6, 131–147.

34.

Kelly

M. S.

Raines

J. C.

Stone

Frey

(2010). School social work: An evidence-informed framework for practice. New York, NY: Oxford University Press.

35.

Kurlaender

Reardon

Jackson

(2008). Middle school predictors of high school achievement in three California school districts. Santa Barbara, CA: California Dropout Research Project.

36.

Lehr

C. A.

Johnson

D. R.

Bremer

C. D.

Cosio

Thompson

(2004). Essential tools: Increasing rates of school completion: Moving from policy and research to practice. A manual for policymakers, administrators, and educators. Minneapolis, MN: National Center on Secondary Education and Transition.

37.

Lehr

C. A.

Sinclair

M. F.

Christenson

S. L.

(2004). Addressing student engagement and truancy prevention during the elementary school years: A replication study of the check & connect model. Journal of Education for Students Placed at Risk, 9, 279–301.

38.

Levin

H. M.

Belfield

C. R.

(2007). Educational interventions to raise high school graduation rates. In Belfield

C. R.

Levin

H. M.

(Eds.), The price we pay: Economic and social consequences of inadequate education (pp. 177–199). Washington, DC: Brookings Institution Press.

39.

Lochner

Moretti

(2004). The effect of education on crime: Evidence from prison, arrests and self-reports. American Economic Review, 94, 155–189.

40.

Maynard

B. R.

McCrea

K. T.

Pigott

T. D.

Kelly

M. S.

(2012). Indicated truancy interventions: Effects on school attendance among chronic truant students. Campbell Systematic Reviews. 8(10), doi:10.4073/csr.2012.10

41.

Maynard

B. R.

Vaughn

M. G.

Sarteschi

C. M.

(2012). The empirical status of social work dissertation research: Characteristics, trends, and implications for the field. British Journal of Social Work. Advance online publication. doi: 10.1093/bjsw/BCS123

42.

Milligan

Moretti

Oreopoulos

(2004). Does education improve citizenship? Evidence from the United States and the United Kingdom. Journal of Public Economics, 88, 1667–1695.

43.

Oreopoulos

Salvanes

K. G.

(2011). Priceless: The nonpecuniary benefits of schooling. Journal of Economic Perspectives, 25, 159–184.

44.

S. R.

Mersky

J. P.

Reynolds

A. J.

Kohler

K. M.

(2007). Alterable predictors of educational attainment, income, and crime: Findings from an inner-city cohort. Social Service Review, 81, 85–128.

45.

Perepletchikova

Kazdin

A. E.

(2005). Treatment integrity and therapeutic change: Issues and recommendations. Clinical Psychology: Science and Practice, 12, 365–383.

46.

Raudenbush

S. W.

Bryk

A. S.

Congdon

(2002). Hierarchical linear modeling. Thousand Oaks, CA: Sage.

47.

Rhoads

C. H.

(2011). The implications of “contamination” for experimental design in education. Journal of Educational and Behavioral Statistics, 36, 76–104.

48.

Roberts

Vaughn

M. G.

Vaughn

Wexler

Coleman

M. A.

Maynard

B. R.

(In Press) Student Engagement: Malleability and Treatment Effects in a Randomized Trial. Manuscript submitted for publication.

49.

Rosen

Proctor

E. K.

Staudt

M. M.

(1999). Social work research and the quest for effective practice. Social Work Research, 23, 4–14.

50.

Rouse

C. E.

(2007). Quantifying the costs of inadequate education: Consequences of the labor market. In Belfield

C. R.

Levin

H. M.

(Eds.), The price we pay: Economic and social consequences of inadequate education (pp. 99–124). Washington, DC: Brookings Institution Press.

51.

Rumberger

R. W.

(2011). Dropping out. Cambridge, MA: Harvard University Press.

52.

Rumberger

R. W.

Lim

S. A.

(2008). Why students drop out of school: A review of 25 years of research (California Dropout Research Project Report 15). Retrieved from http://www.cdrp.ucsb.edu/pubs_reports.htm

53.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth.

54.

Silver

Suanders

Zarate

(2008). What factors predict high school graduation in the Los Angeles Unified School District? Santa Barbara, CA: California Dropout Research Project. Retrieved from http://www.cdrp.ucsb.edu/pubs_reports.htm

55.

Sinclair

M. F.

Christenson

S. L.

Evelo

D. L.

Hurley

C. M.

(1998). Dropout prevention for youth with disabilities: Efficacy of a sustained school engagement procedure. Exceptional Children, 65, 7–22.

56.

Sinclair

M. F.

Christenson

S. L.

Lehr

C. A.

Anderson

A. R.

(2003). Facilitating student engagement: Lessons learned from Check & Connect longitudinal studies. The California School Psychologist, 8, 29–42.

57.

Sinclair

M. F.

Christenson

S. L.

Thurlow

M. L.

(2005). Promoting school completion of urban secondary youth with emotional or behavioral disabilities. Exceptional Children, 71, 465–482.

58.

Singer

J. D.

(1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 23, 323–355.

59.

Smith

S. W.

Daunic

A. P.

Taylor

G. G.

(2007). Treatment fidelity in applied educational research: Expanding the adoption and application of measures to ensure evidence-based practice. Education and Treatment of Children, 30, 121–134.

60.

Soydan

(2008). Applying randomized controlled trials and systematic reviews in social work research. Research on Social Work Practice, 18, 311–318.

61.

Spybrook

Raudenbush

S. W.

Congdon

Martínez

(2009). Optimal Design for longitudinal and multilevel research: Documentation for the Optimal Design software V.2.0. Retreived from www.wtgrantfoundation.org

62.

StataCorp. (2005). Stata statistical software: Release 9. College Station, TX: Author.

63.

Stout

K. E.

Christenson

S. L.

(2009). Staying on track for high school graduation: Promoting student engagement. The Prevention Researcher, 16, 17–20.

64.

Tanner-Smith

E. E.

Wilson

S. J.

(2013). A meta-analysis of the effects of dropout prevention programs on school absenteeism. Prevention Science. Advance online publication. doi:10.1007/s11121-012-0330-1

65.

Texas Education Code §29.081. Compensatory, intensive, and accelerated instruction. Retrieved from http://www.statutes.legis.state.tx.us/SOTWDocs/ED/htm/ED.29.htm#29.081

66.

Texas Education Code §33.151. Curriculum, programs and services, Chapter 33: Services programs and extracurricular activities; Subchapter E. Communities in Schools program. Retrieved from http://www.statutes.legis.state.tx.us/Docs/ED/htm/ED.33.htm#33.151

67.

Todd

A. W.

Campbell

A. L.

Meyer

G. G.

Horner

R. H.

(2008). The effects of a targeted intervention to reduce problem behaviors: Elementary school implementation of Check In–Check Out. Journal of Positive Behavior Interventions, 10, 46–55.

68.

U.S. Department of Commerce. (2009). Extended measures of well-being: Living conditions in the United States, 2005. Washington, DC: U.S. Census Bureau. Retrieved from http://www.census.gov/population/www/socdemo/extended-05.html

69.

U.S. Department of Education. (2011). What Works Clearinghouse Training Manual- DRAFT September 2011. Washington, DC: U.S. Department of Education.

70.

Vaughn

M. G.

Maynard

B. R.

Salas-Wright

C. P.

Perron

B. E.

Abdon

(2013) Prevalence and correlates of truancy in the US: Results from a national sample. Journal of Adolescence, 36, 767–776. doi: 10.1016/j.adolescence.2013.03.015.

71.

Vaughn

M. G.

Salas-Wright

C. P.

Maynard

B. R.

(In Press) Dropping out of school and Chronic Disease in the United States. Manuscript submitted for publication.

72.

Wentzel

K. R.

(2002). Are effective teachers like good parents? Teaching styles and student adjustment in early adolescence. Child Development, 73, 287–301. doi:10.1111/1467-8624.00406

73.

What Works Clearinghouse. (2006). Dropout prevention intervention report: Check & Connect. Retrieved from http://whatworks.ed.gov

74.

What Works Clearinghouse. (2011). Children classified as having an emotional disturbance intervention report: Check & Connect. Retrieved from http://whatworks.ed.gov

75.

Wilson

S. J.

Lipsey

M. W.

(2006). The effects of school-based social information processing interventions on aggressive behavior, part II: Selected/Indicated pull-out programs. Campbell Systematic Reviews, 6. doi:10.473/csr.2006.6

76.

Wilson

S. J.

Tanner-Smith

E. E.

Lipse

M. W.

Steinka-Fry

K J

Morrison

. (2011). Dropout prevention and intervention programs: Effects on school completion and dropout among school-aged children and youth. Campbell Systematic Reviews, 8. doi:10/4073/csr.2011.8