Evaluating a pregnancy and STI prevention programme in rural,at-risk,middle school girls in the USA

Abstract

Objective:

Young people in urban areas are often the focus of pregnancy and sexually transmitted infection (STI) prevention programmes because of their high risk of unwanted pregnancy and contracting an STI. Young people in rural areas are far less studied but also have a high risk of similar outcomes. This study evaluates Giving Our Girls Inspiration & Resources for Lasting Self-Esteem (GO GIRLS), an after-school pregnancy and STI prevention programme, in a sample of high-risk middle school girls living in rural areas using quasi-experimental methods.

Design:

GO GIRLS was assessed using the same survey administered at three time points: prior to the start of the programme, directly following completion of the programme and at the end of the school year. The sample for this study was drawn from a larger evaluation of a multi-faceted health initiative conducted over the course of the school year in three rural counties in South Central Florida.

Setting:

The GO GIRLS programme was implemented in a rural area of South Central Florida. This area has higher rates of teenage pregnancy and poverty and lower educational attainment in comparison with state averages.

Method:

Propensity scores and nearest-neighbour matching without replacement were used to select a group of girls who did not participate in the programme but were comparable to participants. Programme participants were compared to the propensity score–selected controls on key outcomes using hierarchical linear regression.

Results:

Findings indicate that at follow-up, girls who participated in GO GIRLS had attitudes more accepting of delaying sexual intercourse than controls. However, participants did not differ in their levels of sex refusal skills at follow-up.

Conclusion:

This study shows that the GO GIRLS programme has promise for at-risk, rural girls and highlights where future evaluation research is needed.

Keywords

GO Girls pregnancy prevention programme evaluation propensity scores rural youth STI prevention USA

Numerous preventive interventions have been conducted in order to reduce unwanted pregnancies and instances of sexually transmitted infections (STIs) in girls. Changing sexual behaviour – whether by delaying behaviours until a later time point (e.g. adulthood and marriage) or increasing the use of condoms and other forms of contraception – is currently the most effective and practical way to reduce instances of these outcomes (As-Sanie et al., 2004; Bennett and Assefi, 2005; Goesling et al., 2014; Kirby, 2007; Robin et al., 2004). Thus, prevention programmes target risk and protective factors that have been identified as influencing girls’ sexual behaviours. However, programme evaluations have demonstrated that many programmes are not effective in reducing risk behaviours or changing attitudes. Moreover, many programmes being conducted are not evaluated or have constraints that limit the ability to conduct best practice evaluations (As-Sanie et al., 2004; Bennett and Assefi, 2005; Goesling et al., 2014; Kirby, 2007; Robin et al., 2004). Additionally, the few programmes that have been identified as effective were almost exclusively evaluated in urban and metropolitan settings, overlooking the needs of rural youth (Goesling et al., 2014). The present investigation uses quasi-experimental methods to overcome constraints faced while evaluating programme effects for an intensive, after-school programme aimed at reducing sexual risk. The programme was implemented with an at-risk population of middle school girls living in two rural counties in the Southeastern USA.

Limitations to available programmes and their evaluation

An overwhelming number of STI and pregnancy prevention programmes are available, but many have never been evaluated. Of those that have been evaluated, many evaluations have not consistently used rigorous methods for testing programme effects. This is evident by the large number of studies omitted in systematic reviews of the literature due to methodological issues (Goesling et al., 2014; Kirby, 2007; Robin et al., 2004). Furthermore, programmes supported by US state or federal funds may have only minimal evaluation requirements, even when the receipt of funds mandates partnerships between the community agencies charged with conducting the programmes and researchers.

In response to this, the US Department of Health and Human Services has published a list of 31 prevention programmes that have been shown effective in high-quality evaluations (Goesling et al., 2014). However, none of the programmes included in this list were specifically evaluated for effectiveness in rural communities, and most of the evaluations occurred in exclusively urban or metropolitan settings. Even in the extensive review of pregnancy and STI prevention programme evaluations published by The National Campaign to Prevent Teen and Unplanned Pregnancy, only 10% of the experimentally sound evaluations of curriculum-based programmes were conducted in rural settings (Kirby, 2007). Overlooking rural youth is especially problematic as recent research indicates that while teenage birth rates have been dramatically decreasing in urban areas, they are decreasing at markedly slower rates in rural areas (Stewart Ng et al., 2013). Additionally, programmes that are effective for urban youth are not necessarily effective for rural youth as programmes need to be matched to the characteristics of the targeted population (Demby et al., 2014; Stanton et al., 2006). Thus, this study aims to make an important addition to the small body of research on programmes implemented in and designed or adapted for rural communities.

Even when programme evaluation has occurred, best practices in evaluation, such as random assignment to conditions, may not be feasible, especially in small-scale programme evaluations and in communities where funding agencies require that all young people receive some form of programme. Schools or agencies may further constrain selection and enrolment of youth in programmes by only enrolling particular kinds of young people in certain programmes. Hence, quasi-experimental designs are commonly used as they offer alternatives as first steps in evaluation of programmes in communities. For instance, in the comprehensive review by the National Campaign to Prevent Teen and Unplanned pregnancy, 41% of the evaluations of curriculum-based programmes used a quasi-experimental design (Kirby, 2007).

Propensity scores have emerged as a useful tool in quasi-experimental methods allowing for initial tests of programmes that were not able to use random assignment to condition (Martino et al., 2008; Rosenbaum, 2009; Wiggins et al., 2009); successful studies identified by these means may then form the basis for large-scale trials. A propensity score is the probability of being assigned to a treatment group and is calculated for each individual based on a set of background characteristics (baseline covariates) measured for all participants (Rosenbaum and Rubin, 1983). As such, two participants with the same propensity score tend to have the same distribution of background characteristics. One of a variety of matching techniques is then used to select participants for a comparison group that matches the characteristics of participants in the treatment group. Programme evaluations that want to make causal conclusions but are not able to use random assignment for treatment and control groups can utilise propensity scores to select participants to form a comparison group that has the same distribution of baseline characteristics as the treatment group (Stuart, 2010). Propensity scores have been used in evaluations of a variety of prevention programmes, including evaluations of programmes preventing unwanted teenage pregnancy and STIs (Martino et al., 2008; Rosenbaum, 2009; Wiggins et al., 2009).

The present investigation

The purpose of this study was to evaluate the ability of the Giving Our Girls Inspiration & Resources for Lasting Self-Esteem (GO GIRLS) programme to increase middle school girls’ positive attitudes towards delaying sex and increasing sex refusal skills in an at-risk population. The GO GIRLS programme is one of three pregnancy and STI prevention programmes for middle school students implemented by a local community health agency in two rural counties in Southern Central Florida. The counties have high rates of teenage pregnancy and poverty and lower educational attainment in comparison with state averages (Florida Department of Health, 2011). This study, although it is small in scale, is important because it adds to the relatively small literature about prevention programmes targeted at rural, young people. Attitudes towards delaying sex and sex refusal skills were selected as outcomes because they are both predictive of actual sexual behaviours and are popular targets of prevention programmes (Demby et al., 2014; Young et al., 2004). We focus on these predictors of sexual behaviours because the timing restraints placed upon the programme implementation and evaluation did not allow enough passage of time in order to test for behavioural outcomes. We hypothesise that after completing the programme GO GIRLS, participants will have higher levels of sex refusal skills and will endorse delaying sex more than the comparison group of girls generated by propensity scores.

Methods

Design and participants

The sample for this study is drawn from a larger evaluation of a multi-faceted health initiative conducted over the course of the school year in three rural counties in South Central Florida. The project is part of a federal government initiative that provided funds to community agencies to conduct pregnancy and STI prevention programming. Evaluation/research activities were contracted to the University of Florida, and the protocol was approved by the University’s Behavioural/Nonmedical Institutional Review Board (IRB). The IRB also approved a waiver of active parental consent for all evaluation activities.

Through this health initiative, all students in sixth to eighth grades (aged 11–14 years) received pregnancy and STI prevention programming in school during the 2009–2010 school year. An additional after-school programme, GO GIRLS, was available in two of the three counties to any middle school woman who wished to participate; one county had a single middle school and the other had four middle schools. The third county did not offer the GO GIRLS programme, and in addition, this county chose to implement a different in-school programme. Therefore, the current sample does not include students from the third county.

The GO GIRLS programme is an 8-week, intensive, after-school programme for middle school girls, aimed at reducing sexual risk as well as increasing self-esteem and goal setting. It was adapted from a programme promoting girls’ health that originally focused on eating behaviours (Steiner-Adair et al., 2002). The GO GIRLS curriculum includes sessions on goal setting, media literacy skills, values, teenage pregnancy and STIs, relationships and planning for the future. Each session included an educational lesson on the day’s topic, an activity to reinforce the day’s lesson and time set aside for participants to reflect on the day’s lesson through journaling.

Health educators used a variety of methods for recruiting participants in addition to the participants who opted in to the programme on their own; thus, random assignment was not used. The programme was advertised to all female middle school students. Additionally, school administrators and teachers strongly encouraged girls they felt were at particularly high risk for behavioural or health problems to participate in the programme, thus creating a treatment group that had a higher risk of unwanted pregnancy and STIs than the general school population.

Girls who completed the entire programme received 16 hours of programme activities. However, the average level of involvement was 23.02 hours (standard deviation [SD] range = 10.39–11.76) because many girls were encouraged to re-enrol in the programme multiple times throughout the school year and about half of the participants did so. Several girls chose to repeat GO GIRLS because it gave them something to do after school as there were not many extracurricular options in these rural counties. While we do not have a comprehensive list of other after-school programmes available to the students, we do know the options were limited and the counties were dealing with reduced education budgets during the timeframe in which the programme was implemented.

The programmes were assessed using the same survey administered at three time points: prior to the start of the programme, directly following completion of the programme and at the end of the school year. Students who participated in both an in-school programme and GO GIRLS could have multiple baseline measurements, one for each programme. Data from the first baseline survey, prior to any programming, are used in the current analyses.

A total of 126 girls participated in the GO GIRLS programme and completed at least one of the three surveys. Because not all participants completed both follow-up surveys, this study uses the last follow-up survey that each participant completed and controls for the time between the baseline and the follow-up surveys in all models. In all, there were 58 GO GIRLS participants with reliable baseline and follow-up data.

Possible comparison participants included all of the female middle school students who participated in only the in-school programme in the two counties in which GO GIRLS was implemented. Each of these girls was invited to participate in GO GIRLS but chose to not participate. In order to be included in the pool of possible comparisons, participants also needed to have completed the baseline assessment and at least one follow-up assessment after the in-school programme (n = 422). Using the procedures described subsequently, a propensity score–selected (PSS) comparison group of 58 girls was identified. The final sample used in analyses (n = 116) ranged in age from 11 to 15 years (indicating some participants may have been held back a grade) with an average age of 12.50 years (SD range = 0.96–1.06). Almost half of the participants in the final sample were Latina (41.5%) and about a quarter were Caucasian (25.8%). The remaining participants were African American (20.6%) and from other ethnic groups (12.1%).

Measures

Attitudes towards delaying sex

In all, 15 items assessed attitudes about delaying sex and teen sexual behaviours. Specifically, items from the Views Supportive of Abstinence measure and the Views Unsupportive of Teen Sex measures were used (Mathematica Policy Research [MPR], 2007). Three items were added to assess views towards delaying sex that are targeted in the programmes (Graber and Johnson, 2009). Four additional items developed by the Administration for Children and Families (2009) were also used. All items presented a statement about delaying sex or teen sexual behaviour and asked participants to state their agreement with the statement using a 5-point scale from 0 (strongly agree) to 4 (strongly disagree). Items about engaging in sexual intercourse were reverse-coded to reflect the directionality of the items about delaying sex. All the items were averaged to create an overall measure of attitudes towards delaying sex, with lower scores indicating more favourable views towards delaying sex (T1 α range = .87–.89; T2 α range = .89–.92).

Sex refusal skills

The four-item Refusal Skills Scale (MPR, 2007) was used to assess beliefs in personal efficacy for avoiding sexual situations and sex. The items posed hypothetical situations in which the participant’s significant other wanted to have sexual intercourse and the participant did not. Likelihood of sticking with the decision to not have sex or avoiding situations that lead to sex was rated on a 4-point Likert scale from 0 (definitely would) to 3 (definitely would not). The four items were averaged to create a total refusal skills score, with lower scores indicating higher levels of refusal skills (T1 α range = .72–.80; T2 α range = .74–.85).

Covariates used to create propensity scores

Propensity scores were calculated using 39 covariates (Appendix 1). Covariates included demographic variables, risk behaviours, actual and intended sexual behaviours, predictors of sexual behaviours and life goals. Including these covariates is in line with advice that suggests that including any covariate related to the outcome or treatment group will help reduce the bias of the propensity scores as opposed to increasing the variance of the scores (Stuart, 2010).

Because propensity score matching procedures do not allow for missing data, multiple imputation was used to calculate missing data prior to calculating propensity scores. A dichotomous missing data indicator variable (0 = not missing and 1 = missing) was created for three variables with more than 5% of the data missing prior to multiple imputation (Appendix 1). These indicator variables were included as covariates when calculating the propensity scores.

Analyses

Prior to analyses, the MICE package in R (Van Buuren and Groothuis-Oudshoorn, 2011) was used to impute 10 data sets, and the analyses were pooled across the 10 imputations according to the rules set forth by Rubin (1987). Pooling approaches for some analyses have not yet been defined, and in those cases, the range of values across the 10 imputations is provided. After imputing 10 data sets, the MatchIt package in R was used to calculate logistic regression propensity scores and then use those scores to conduct nearest-neighbour matching without replacement (Ho et al., 2011). This matching selected the untreated participants to be included in the PSS comparison group for each data set.

Once PSS comparison group was created for each of the 10 data sets, the respective comparison group was used to estimate the treatment effect of the GO GIRLS programme on attitudes about delaying sex and sex refusal skills. Treatment effects were calculated using two separate hierarchical linear regression models, one for each outcome. The first block of each model included just the treatment condition (GO GIRLS participation = 1). The second block controlled for the baseline measure of the outcome and the time in months between the baseline and follow-up surveys. The third block added age, ethnicity dummy codes (other ethnicity omitted) and school dummy codes. Finally, following the hierarchical regressions, sensitivity analyses were conducted to determine how robust the resulting effects were to hidden biases (Rosenbaum, 2002).

Results

Propensity score matching procedures do not allow for missing data. Using multiple imputation of missing data, more participants were maintained in the sample than if listwise deletion had been used. The data were assumed to be missing at random; it has been suggested that imputation is a better approach than listwise deletion, even when some of the missingness is nonrandom (Schafer and Graham, 2002). Of the 79 variables in the data set (not including missing data indicator variables), 72 had missing data. In all, 2.1% (778 points of the 37,920 possible points) were missing and subsequently imputed.

Next, a variety of matching methods utilising propensity scores were tested, including nearest-neighbour matching with and without replacement, full matching and optimal matching. Nearest-neighbour matching without replacement produced the best covariate balance out of all the matching methods, and thus, it was used on all 10 of the imputed data sets to select a PSS comparison group. Prior to matching, the standardised mean difference between the control participants and the treatment group on the 39 covariates (which was increased to 54 variables due to dummy coding categorical covariates for propensity score matching) was greater than 0.25 for 14.8%–20.4% of the covariates. After nearest-neighbour matching without replacement, 5.6%–11.1% of the covariates had standardised mean differences greater than 0.25, indicating better balance on the covariates between the treatment and PSS comparison group which allows for causal inferences to be cautiously drawn (Stuart, 2010).

Basic demographic data and indicators of risk for the pool of possible comparison participants, the PSS comparison group and the GO GIRLS participants are shown in Table 1. Nationally, only 5% of girls have had sex by their 14th birthday (Finer and Philbin, 2013), but in our sample, 10.3%–13.5% of the girls aged 13 years and younger have engaged in sexual intercourse at least once and 10.3%–12.9% of all of the girls in the sample had previously had intercourse. The GO GIRLS participants and the possible comparison pool of all eligible girls did not differ significantly on any of the demographics or key indicators of risk presented in Table 1, except for ethnic group, χ² = 11.22–12.65, p < .05. This effect was driven by significantly fewer Caucasian girls and significantly more African American girls in the GO GIRLS group than expected by chance.

Table 1.

Descriptive statistics by treatment group.

Characteristic	Comparison pool (n = 422)^a	GO GIRLS (n = 58)^a	PSS comparisons (n = 58)^a
M age (years)	12.69 (0.91–0.92)	12.41 (1.00–1.16)	12.56 (0.96–1.05)
White (%)	44.8	22.4–24.1	24
Latina (%)	32.9	39.7	49
African American (%)	12.3	24.1	17
Other (%)	10.0	12.1–13.8	10
Live with mother and father (%)	51.4–51.7	39.7–41.4	46
Ever had sexual intercourse (%)	8.3–8.8	8.6–10.3	10
Ever tried cigarettes (%)	12.3–13.0	13.8–15.5	15
Ever had a drink of alcohol (%)	24.6–25.4	32.8–34.5	39
Ever tried other drugs (%)	6.2–6.6	6.9–8.6	7

GO GIRLS: Giving Our Girls Inspiration & Resources for Lasting Self-Esteem; PSS: propensity score–selected; M: mean.

n per data set; when statistic is not pooled, ranges across all 10 data sets are reported.

Despite the lack of statistically significant differences between GO GIRLS participants and the larger comparison pool, visual inspection and sample size constraints indicated that these groups may differ on some dimensions. Hence, we used propensity scores to select a group of comparison participants that matched the GO GIRLS participants’ demographics and key indicators of risk. As indicated above, the analysis of balance after matching confirmed that the PSS comparison group matched the GO GIRLS participants more closely than the full sample of possible comparison girls (see Table 1). Although the PSS comparison group is not exactly the same as the GO GIRLS group, in particular with respect to the ethnic breakdown of the participants, it does represent a group of girls that are at higher risk for unwanted pregnancy and STIs than the full sample of possible comparison girls, and thus, it is a more appropriate comparison group for the evaluation of GO GIRLS.

In preliminary analyses, we conducted t-tests to determine whether baseline measurements of attitudes towards delaying sex and sex refusal skills varied between PSS comparison and GO GIRLS participants. As stated previously, possible scores for attitudes towards delaying sex ranged from 0 to 4, with lower scores indicating more positive attitudes towards delaying sex. The PSS comparison participants had a baseline mean score of 1.15 (SD range = 0.60–0.72) and a follow-up mean score of 1.14 (SD range = 0.60–0.74), while the GO GIRLS participants had a baseline mean score of 0.98 (SD = 0.69) and a follow-up mean score of 0.71 (SD range = 0.69–0.69). The groups did not vary significantly from one another on attitudes about delaying sex at baseline, t(114) = 1.25, p > .05, but they did vary at follow-up, t(114) = 3.26, p < .01.

Possible scores for sex refusal skills ranged from 0 to 3, with lower scores indicating more refusal skills. The PSS comparison participants had a baseline mean score of 0.72 (SD range = 0.64–0.77) and a follow-up mean score of 0.70 (SD range = 0.58–0.70), while the GO GIRLS participants had a baseline mean score of 0.49 (SD range = 0.65–0.71) and a follow-up mean score of 0.44 (SD range = 0.61–0.70). The groups did not differ on refusal skills at baseline, t(114) = 1.67, p > .05, but they did vary at follow-up, t(114) = 2.02, p < .05. We include these t-tests only as baseline statistics; tests of treatment effects that include covariates are reported below. Caution should be used when interpreting the results due to the smaller sample size.

Treatment effects

As indicated, hierarchical linear regression was used to examine treatment effects. The dichotomous indicator for participating in GO GIRLS (GO GIRLS participation = 1) was entered in the first block. The second block included the baseline measure of the outcome as well as the elapsed time in months between the baseline and follow-up data. On average, 4.14 months (SD range = 2.69–2.80) elapsed between baseline and follow-up data collections. Demographic characteristics including a continuous variable for age, dummy codes for race (omitting ‘other ethnic groups’) and dummy codes for four of the five schools were entered into the third block. Previous sexual intercourse was not included as a control because there were only five to nine girls in each treatment group who had engaged in sexual intercourse.

Confirming our hypothesis, the GO GIRLS programme had a significant treatment effect on attitudes about delaying sex (Table 2). Girls who received the programme had attitudes that were more supportive of delaying sex (lower scores) than girls who did not participate in GO GIRLS. This held true throughout all the blocks.

Table 2.

Hierarchical linear regression testing the treatment effect on attitudes towards delaying sex.

Predictor	Block 1	Block 2	Block 3
Intercept	1.14 (0.10)***	0.47 (0.17)***	−2.72 (0.67)***
Treatment	−0.43 (0.13)**	−0.30 (0.09)**	−0.26 (0.09)**
Baseline attitudes		0.66 (0.07)***	0.64 (0.07)***
Elapsed time between measures (months)		−0.03 (0.02)	−0.01 (0.02)
Age			0.21 (0.05)***
School 2			0.40 (0.13)**
School 3			0.57 (0.14)***
School 4			0.39 (0.17)*
School 5			0.49 (0.21)*
White^a			0.18 (0.18)
African American^a			0.11 (0.15)
Latina^a			0.17 (0.18)
r² range	0.08–0.12*	0.49*–0.56*	0.61*–0.69*
r²Δ range	0.08–0.12*	0.41*–0.46*	0.11–0.16*
F range	9.53–15.57*	35.56*–48.32*	14.72*–21.42*

Coefficients and standard errors are reported and were pooled across all 10 data sets.

Other ethnic groups is the reference group.

p < .05; **p < .01; ***p < .001.

As would be expected, participants with attitudes less supportive of delaying sex at baseline had significantly less supportive attitudes at follow-up. There was no effect of elapsed time between baseline and follow-up indicating that follow-up attitudes cannot be predicted by the amount of time between the baseline and follow-up data collections. There was an age effect such that younger participants were more supportive of delaying sex than older participants. There was a significant school effect as the reference school had attitudes that were significantly more approving of delaying sex than all four of the other schools. The analysis was rerun with other schools as the reference school; no other differences between schools emerged. Additionally, the interaction of school by treatment was not significant. Finally, ethnicity did not predict differences in attitudes about delaying sex. No dosage effects of the GO GIRLS programme were found in a follow-up regression using just GO GIRLS participants.

Contrary to hypotheses, the GO GIRLS programme did not have a significant treatment effect on skills for refusing sexual intercourse (Table 3). At follow-up, students who received the programme did not have different levels of refusal skills than participants who only had the in-school programme. The level of refusal skills at baseline did predict the level of refusal skills at follow-up, such that those with more skills at baseline (lower scores) also had more skills at follow-up. Again there was an effect of age such that younger participants had more refusal skills than older participants. Finally, neither the school in which the programme was administered nor ethnicity was a significant predictor of follow-up refusal skills.

Table 3.

Hierarchical linear regression testing the treatment effect on sexual intercourse refusal skills.

Predictor	Block 1	Block 2	Block 3
Intercept	0.70 (0.09)***	0.28 (0.11)**	−2.08 (0.81)*
Treatment	−0.26 (0.13)*	−0.12 (0.10)	−0.09 (0.10)
Baseline refusal skills		0.61 (0.02)***	0.60 (0.08)***
Elapsed time between measures (months)		−0.01 (0.02)	0.00 (0.02)
Age			0.17 (0.06)**
School 2			−0.06 (0.22)
School 3			0.09 (0.22)
School 4			0.22 (0.22)
School 5			0.04 (0.22)
White^a			0.16 (0.21)
African American^a			0.08 (0.18)
Latina^a			0.19 (0.20)
r² range	0.02–0.5*	0.92–0.94***	0.47*–0.55*
r²Δ range	0.02–0.04*	0.38*–0.45*	0.04–0.11**
F range	2.09–5.83*	24.59*–36.60*	8.37*–11.74*

Coefficients and standard errors are reported and were pooled across all 10 data sets.

Other ethnic groups is the reference group.

p < .05; **p < .01; ***p < .001.

Sensitivity analyses investigated how much an unobserved confounder would have to increase the odds of being in the GO GIRLS programme in order to explain away the associations between the GO GIRLS programme and attitudes towards delaying sex. The sensitivity analysis found that an unobserved confounder would have to increase the odds of being in the treatment group between 1.19 and 1.91 times in order to overturn the significant treatment effect.

Discussion

The results of this study indicated that the GO GIRLS programme had positive effects for at-risk, middle school girls. Girls enrolled in this programme, in addition to an in-school programme, maintained more supportive attitudes towards delaying sex than girls who only experienced an in-school prevention programme. This is promising as previous research has demonstrated that maintaining views supportive of delaying sex is predictive of less sexual behaviour (Kirby, 2007). Although both GO GIRLS participants and the comparison group received in-school programming, only the GO GIRLS participants demonstrated change in their attitudes. Because the elapsed time between baseline and follow-up measurements ranged from immediately following programme completion (0.10–0.23 months since baseline) to 8.63 months since baseline measurement, it was controlled in analyses. The elapsed time between measurements was not significant, indicating that attitudes did not vary as the amount of time since baseline, and subsequently the end of the programme, increased. This is important to note as many prevention programmes often see a strong effect immediately following a programme but do not see lasting effects.

About half of the girls who participated in GO GIRLS enrolled in the programme more than once throughout the school year. However, a follow-up analysis indicated that the amount of time spent in the GO GIRLS programme did not affect the attitudes towards delaying sex at follow-up. Thus, repeating the programme does not appear to produce better outcomes than only participating in the programme once.

Although preliminary analyses (t-tests) suggested that there might be a treatment effect for refusal skills, analyses of treatment effects that included covariates did not find an effect. While changing attitudes is an important effect not to be overlooked, attitudes and behaviours do not always align; thus, programmes need to see effects across many areas in order to ensure effectiveness in reducing rates of unwanted pregnancy and STIs. Recent meta-analyses of pregnancy and HIV prevention programme evaluations identify self-reported refusal of sexual intercourse skills as a common outcome across many of their included studies (Johnson et al., 2011; Kirby and Laris, 2009). The programmes that effectively increased refusal skills used more intensive methods, such as role plays and games, than were utilised in GO GIRLS. Altering the aspects of the GO GIRLS programme that are designed to increase these refusal skills to be more like these programmes that have demonstrated success could help increase the effectiveness of the GO GIRLS programme.

One of the main aims of the GO GIRLS programme is to reduce unwanted pregnancies and STIs through the postponement of sexual intercourse. In this study, only predictors of these outcomes were evaluated as opposed to the actual sex, pregnancy and STI rates for two reasons. First, the number of participants who indicated that they had had sex for the first time between baseline and follow-up data collections was very small, just 7 participants enrolled in GO GIRLS and 21 of all of the possible comparison participants (6% of the total sample). Second, it is suggested that in order to see full effects on these outcomes, the time between the programme completion and follow-up needs to be at least 1 calendar year (Kirby, 2007). While we originally planned for additional follow-up data collections after a year had elapsed, changes in federal funding to the community agency limited data collection to a single school year.

Sensitivity analyses investigated how much an unobserved confounder would have to increase the odds of being in the GO GIRLS programme in order to explain away the associations between the GO GIRLS programme and attitudes towards delaying sex. The analysis revealed that the effect of the GO GIRLS programme on attitudes was not very insensitive to unobserved confounders. An unobserved confounder would have to increase the odds of being in the GO GIRLS programme less than twofold in order to eliminate the treatment effect.

Limitations

There are several limitations to this study. Many of these were encountered during data collection, and the evaluation models were specifically chosen to help overcome many of those obstacles. First, attrition was problematic, but for a high-risk sample, such as the one in this study, attrition is typical. The mobility of the sample and our inability to administer make-up surveys for absent students account for a majority of the attrition. It is clear that the full sample for this study is very mobile: at baseline, 16%–17% of the total sample indicated that they had attended more than one school since the school year began in August and 13%–14% indicated that they had lived in more than one place since August. While we do not have school attendance data for the current sample, we do know that absenteeism from school is a common characteristic among at-risk young people and could be a source for our high attrition rate. Due to limitations set forth by the schools, we were not able to administer make-up surveys to students absent on the evaluation days. To overcome our attrition rate, when the follow-up questionnaire from the end of the school year was not available, the follow-up outcomes from the questionnaire administered directly following programme completion were used. The elapsed time between baseline and follow-up measures was then controlled, allowing for more participants to be kept in the sample.

As mentioned previously, the desires and requests of the schools and health providers prevented the use of random assignment to the GO GIRLS programme. In order to overcome this limitation, we used propensity scores to select a comparison group similar to the group who received the GO GIRLS programme. It is important to note that the propensity score distribution for the possible comparison participants did not completely overlap the propensity score distribution of the GO GIRLS participants. This is most likely due to the GO GIRLS recruitment methods because they targeted the girls with the highest risk, and therefore, there were not many girls with these extreme levels of risk left in the possible comparison group. This difference in distribution, or rather the targeting of the highest risk girls for the programme, might also have diluted the programme effects and the subsequent sensitivity analysis.

This was the first evaluation of the GO GIRLS programme, and future evaluation is necessary. Additionally, we hypothesise that the desired increase in refusal skills was not seen due to a limited amount of time devoted to intensive educational methods, such as role plays and games, specifically aimed at increasing refusal skills. Thus, future iterations of GO GIRLS should adjust the programme accordingly and test whether this change produces the desired outcomes. As all participants received an in-school programme, it is unknown whether the same treatment effects are seen when participants only receive the GO GIRLS programme. It should be noted that the in-school only participants did not have significantly different attitudes or refusal skills at the follow-up than they did at baseline data collection. Thus, the in-school programme does not appear to be effective for high-risk girls. With this in mind, we hypothesise that the GO GIRLS effects will remain if the programme is administered without an accompanying in-school programme. The GO GIRLS programme is also intended to have positive effects on girls in more areas than just sexuality; these areas need to be evaluated.

Finally, it can be argued that participation in prevention programmes teaches young people what the socially desirable attitudes about sex are, and that this is the reason behind changes in attitudes. However, as both the GO GIRLS and the comparison group were involved in prevention programming with a goal to change attitudes, there was the possibility for both groups to answer attitude items on the questionnaire in socially desirable ways.

Overall, this study establishes a strong initial evaluation of the GO GIRLS programme. As many programmes conducted in community and school settings are never evaluated, it is important to continue to strive towards programme evaluation, even when best methods for evaluation cannot be implemented. This study adds to the overall literature about the effectiveness of programmes implemented in rural settings which has a much smaller literature than programmes implemented in urban and metropolitan settings. Results suggest that the GO GIRLS programme is effective in altering attitudes about delaying sex in an at-risk, rural sample, and further research should continue investigating the other effects of the programme. Further programme development should better address the aspects of the programme aimed at increasing skills for refusing sexual intercourse.

Footnotes

Appendix 1 Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The evaluation of this program was supported by the following grant to Heartland Rural Health Network: Administration for Children & Families, Department of Health and Human Services CBAE Program, Grant Number: 90AE0276.

References

Administration for Children and Families (2009) Program-specific performance measure. OMB no.: 0970-0272. Washington, DC: Administration for Children and Families.

As-Sanie

Gantt

Rosenthal

(2004) Pregnancy prevention in adolescents. American Family Physician 70(8): 1517–1524.

Bennett

Assefi

(2005) School-based teenage pregnancy prevention programs: A systematic review of randomized controlled trials. Journal of Adolescent Health 36(1): 72–81.

Demby

Gregory

Broussard

. (2014) Implementation lessons: The importance of assessing organizational ‘fit’ and external factors when implementing evidence-based teen pregnancy prevention programs. Journal of Adolescent Health 54(3): S37–S44.

Finer

Philbin

(2013) Sexual initiation, contraceptive use, and pregnancy among young adolescents. Pediatrics 131(5): 886–891.

Florida Department of Health (2011) Florida CHARTS (Community Health Assessment Resource Tool Set). Available at: http://www.floridacharts.com/charts/default.aspx

Goesling

Colman

Trenholm

. (2014) Programs to reduce teen pregnancy, sexually transmitted infections, and associated sexual risk behaviors: A systematic review. Journal of Adolescent Health 54: 499–507.

Graber

Johnson

(2009) CBAE Program Evaluation Survey. Gainesville, FL: University of Florida.

Imai

King

. (2011) MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software 42(8): 1–28.

10.

Johnson

Scott-Sheldon

LAJ

Huedo-Medina

. (2011) Interventions to reduce sexual risk for Human Immunodeficiency Virus in adolescents: A meta-analysis of trials, 1985–2008. Archives of Pediatrics and Adolescent Medicine 165(1): 77–84.

11.

Kirby

(2007) Emerging Answers 2007: Research Findings on Programs to Reduce Teen Pregnancy and Sexually Transmitted Diseases. Washington, DC: National Campaign to Prevent Teen and Unplanned Pregnancy.

12.

Kirby

Laris

(2009) Effective curriculum-based sex and STD/HIV education programs for adolescents. Child Development Perspectives 3(1): 21–29.

13.

Martino

Elliott

Collins

. (2008) Virginity pledges among the willing: Delays in first intercourse and consistency of condom use. Journal of Adolescent Health 43(4): 341–348.

14.

Mathematica Policy Research (MPR) (2007) Survey of Teen Activities and Attitudes: Used for the Final Impact Analysis. Princeton, NJ: MPR.

15.

Robin

Dittus

Whitaker

. (2004) Behavioral interventions to reduce incidence of HIV, STD, and pregnancy among adolescents: A decade in review. Journal of Adolescent Health 34(1): 3–26.

16.

Rosenbaum

(2009) Patient teenagers? A comparison of the sexual behavior of virginity pledgers and matched nonpledgers. Pediatrics 123(1): e110–e120.

17.

Rosenbaum

(2002) Observational Studies. 2nd ed. New York: Springer.

18.

Rosenbaum

Rubin

(1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1): 41–55.

19.

Rubin

(1987) Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons.

20.

Schafer

Graham

(2002) Missing data: Our view of the state of the art. Psychological Methods 7(2): 147–177.

21.

Stanton

Harris

Cottrell

. (2006) Trial of an urban adolescent sexual risk-reduction intervention for rural youth: A promising but imperfect fit. Journal of Adolescent Health 38(1): 55.e25–55.e36.

22.

Steiner-Adair

Sjostrom

Franko

. (2002) Primary prevention of risk factors for eating disorders in adolescent girls: Learning from practice. International Journal of Eating Disorders 32(4): 401–411.

23.

Stewart Ng

Kaye

National Campaign to Prevent Teen and Unplanned Pregnancy (2013) Teen childbearing in rural America. Science Says, January, no. 47, pp. 1–6.

24.

Stuart

(2010) Matching methods for causal inference: A review and a look forward. Statistical Science 25(1): 1–21.

25.

Van Buuren

Groothuis-Oudshoorn

(2011) MICE: Multivariate imputation by chained equations in R. Journal of Statistical Software 45(3): 1–67.

26.

Wiggins

Bonell

Sawtell

. (2009) Health outcomes of youth development programme in England: Prospective matched comparison study. British Medical Journal 339(7713): 148–151.

27.

Young

Donnelly

Denny

(2004) Area specific self-esteem, values, and adolescent sexual behavior. American Journal of Health Education 35(5): 282–289.