Process Evaluation of a Stress Management Program for Low-Income Pregnant Women: The SMART Moms/Mamás LÍSTAS Project

Abstract

Background. The SMART Moms/Mamás LÍSTAS Project was a randomized control trial that tested the efficacy of a prenatal stress management program in reducing stress and cortisol levels among low-income women. The current study is a process evaluation of the stress management program (intervention arm of the original randomized controlled trial) and assessed whether implementation fidelity factors (i.e., intervention delivery, receipt, and enactment) were associated with lower stress (perceived stress, salivary cortisol), improved negative and positive mood states (Positive and Negative Affect Schedule), and increased confidence to use relaxation and coping skills. Method. Fifty-five low-income pregnant women (71% Latina, 76% annual income <$20,000) attended weekly group-based sessions over an 8-week period in which a clinically trained researcher taught relaxation and coping skills. Process evaluation measures were obtained via participant self-report and videotaped class sessions that were coded for delivery, receipt, and enactment of the intervention to determine which implementation factors were associated with changes in program outcomes (stress, mood, confidence) over the 8-week period. Results. Women in stress management showed a significant reduction in their stress and cortisol levels (p < .001), improvements in negative and positive mood states (p < .001) and were more confident in using relaxation and coping skills postintervention (74%). The implementation factors of delivery (i.e., instructor adherence to intervention content; p = .03) and enactment (i.e., participant use of intervention skills; p = .02) were most associated with improvements in program outcomes. Conclusion. These findings highlight that implementation factors should be considered when delivering stress management interventions in underserved communities.

Keywords

cortisol fidelity implementation pregnancy process evaluation stress management

Stress experienced during pregnancy can have significant long-term health consequences, including increased anxiety and postpartum depressive symptoms for mothers (Dunkel Schetter, Niles, Guardino, Khaled, & Kramer, 2016; Yim et al., 2015), and preterm birth and low birthweight for infants (Van den Bergh et al., 2017). Of particular concern are low-income and ethnic minority women, who experience several psychosocial stressors during pregnancy, including unemployment, racial discrimination and little or no prenatal health information (Lefmann, Combs-Orme, & Orme, 2017). These stressors have been associated with altered patterns of the stress hormone cortisol during pregnancy, which in turn, have been associated with several adverse birth outcomes including preterm labor, low infant birthweight, infant hospitalizations at birth, and infant brain cell damage (Glynn, Dunkel-Schetter, Chicz-DeMet, Hobel, & Sandman, 2007; Hodyl et al., 2017; Van den Bergh et al., 2017). Given these heightened health concerns, studies that develop and test prenatal interventions targeting optimal management of stress and cortisol levels in this population are needed.

The Stress Management and Relaxation Training for Moms (SMART Moms)/Mamás LImitando el eSTrés para Ayudar su Salud (Mamás LÍSTAS) Project was a prospective, randomized control trial examining whether a prenatal cognitive behavioral stress management (CBSM) intervention, focused on teaching women coping and relaxation skills, was effective in regulating stress and cortisol levels among low-income pregnant women compared with a control group. Results showed that women receiving CBSM had lower perceived stress levels and more regulated cortisol patterns at 3 months postpartum compared those in the control group (Urizar, Yim, Rodriguez, & Dunkel Schetter, 2019). These results demonstrated the potential of prenatal CBSM interventions to improve stress outcomes. However, research is also needed on the mechanisms by which such interventions work.

Process evaluation studies help to identify these mechanisms by assessing implementation fidelity, or the degree to which an intervention was delivered as intended, to better understand why an intervention was effective (Breitenstein, Gross, Garvey, Hill, Fogg, & Resnick, 2010). A key component in this evaluation is treatment fidelity, referring to the methodological strategies used to monitor and enhance the reliability and validity of behavioral interventions (Bellg et al., 2004). To assist researchers in evaluating intervention fidelity, the National Institutes of Health established the Behavior Change Consortium Treatment Fidelity Workgroup, which published guidelines and recommendations for best practices in fidelity assessment (Bellg et al., 2004). Five areas of fidelity are emphasized: (1) study design (i.e., theoretical framework used, format and dose of intervention, and credentials of intervention facilitators), (2) training protocol for facilitators (i.e., standardization of training), (3) intervention delivery (i.e., content covered), (4) intervention receipt (i.e., ensuring that participants received and understood content), and (5) intervention enactment (i.e., participants’ ability to perform intervention-related skills in relevant, real-life settings). Interventions with high fidelity have shown significant changes in hypothesized mechanisms that, in turn, have led to better program outcomes, such as improved reading scores in literacy programs and increased physical activity in health promotion interventions (Borrelli, 2011; Durlak & DuPre, 2008). Therefore, without careful attention to fidelity, inferences regarding intervention success, utility, and generalizability are difficult to determine.

Despite evidence strongly suggesting that researchers conduct fidelity evaluation, 54% of behavioral health intervention studies fail to do so and only 16% adhere fully to fidelity workgroup guidelines (Borrelli, 2011). Furthermore, few, if any, of these studies focus on low-income populations (Glasgow, 2008). In the realm of CBSM interventions, the few fidelity studies that exist have largely focused on interventions delivered in occupational and school-based settings (Havermans et al., 2016; Kraag et al., 2007). Results are mixed with a few studies showing increased class attendance, increased program dosage, and increased practice of relaxation techniques to be associated with decreased job strain, anxiety, and blood pressure (Murta, Sanderson, & Oldenburg, 2007). However, 54% of these studies fail to present information linking fidelity evaluation variables with study outcomes making it difficult to identify fidelity factors associated with effective CBSM intervention implementation and outcomes (Murta et al., 2007).

The current study conducted a process evaluation of the CBSM intervention in the SMART Moms/Mamás LÍSTAS Project to assess whether implementation fidelity factors (i.e., intervention delivery, receipt, and enactment) were associated with any observed changes in study outcomes (i.e., stress and cortisol, mood states, and confidence to use relaxation and coping skills) from baseline to post-CBSM intervention.

Method

Participants and Setting

Women were recruited between 2011 and 2013 from six prenatal centers in southern California, which serve a predominantly low-income and ethnic minority population representative of the surrounding neighborhoods (38% below poverty level; 55% Latino, 21% African American; United States Census Bureau, 2014). Recruitment occurred either during women’s prenatal clinic visits, through print-based advertising, and/or by health care provider referrals. Women signed a consent form to determine their study eligibility. Eligibility criteria included being 18 years of age or older, less than 17 weeks pregnant, fluent in either Spanish or English, free of any major medical problems (e.g., gestational diabetes, major depression), and free of any medications that may interfere with their cortisol levels (e.g., asthma inhaler, antidepressants).

Procedure and Intervention

One hundred women were randomized to either an 8-week prenatal CBSM intervention (n = 55) or an attention control group (n = 45), where women received printed prenatal health information by mail over an 8-week period versus coming to a group class (see Urizar et al., 2019 for complete details on study enrollment and the CONSORT statement; NIH Clinical Trial NCT03627247). The current paper focuses on the 55 women randomized to the CBSM intervention. Intervention efficacy was evaluated using an experimental pretest–posttest design.

The CBSM program was an 8-week, group-based intervention that taught coping and relaxation skills to address common stressors experienced during pregnancy and motherhood (e.g., financial strain, concerns about the health of the baby; see Table 1 for a description of the intervention study design). Group facilitators taught the classes at a local prenatal clinic (where most women were receiving prenatal services) and were offered in both Spanish and English to reduce language barriers to participating in CBSM. The content for the course was based on CBSM interventions shown to be effective in regulating stress and cortisol among several health populations (Antoni, 2013; Antoni et al., 2005; Urizar & Muñoz, 2011). Class activities were designed to be interactive (e.g., role-playing, use of physical props to introduce class concepts, use of relaxation exercises) to optimize participant engagement and understanding of course material while tailoring class content to commonly reported stressors experienced during pregnancy (e.g., discomforts of pregnancy, receiving adequate support to care for the baby). One to two group facilitators led each class using materials from an instructor training manual (see Table 1 for description of intervention training). Each week, participants were given coping and relaxation skills to practice at home and were asked to record their experiences on an activity log as a weekly homework assignment that was collected and discussed in class the following week (see Table 2 for description of intervention content). The Institutional Review Board at California State University, Long Beach approved all study procedures.

Table 1.

Treatment Fidelity Characteristics for the SMART Moms/Mamás LÍSTAS Project.

Fidelity characteristic	Description
Study design
Theoretical framework	• Cognitive behavior theory for stress management based on Michael Antoni’s B-SMART Program for women with breast cancer (Antoni, 2003) and Ricardo Muñoz’s Mothers and Babies Depression Prevention Course for pregnant women (Muñoz et al., 2007)
Duration/dose of intervention	• Eight weeks (one class per week; each class was 2 hours long)
Intervention format	• In-person, closed group format (i.e., no open enrollment after second week of the 8-week intervention) • Three to eight women in each group (12 total groups); classes taught in Spanish (six groups) or English (six groups)
Pilot testing of intervention	• Two focus groups (one in Spanish, one in English) with four to six pregnant women, clinic directors, and staff for feedback on intervention content (sample class activities were conducted), cultural relevancy, and reading level of materials in both languages
Training
Group facilitator characteristics	• One to two group facilitators • Lead investigator (trained clinical psychologist specialized in designing and teaching CBSM interventions among low income pregnant populations [Muñoz et al., 2007; Urizar & Muñoz, 2011]) and advanced masters students in psychology
Training manual	• An instructor training manual was developed, based on CBSM materials shared with participants (i.e., SMART Moms/Mamás LÍSTAS Participant Manual; Urizar & Kofman, 2012), that outlined class topics and activities
Facilitator supervision	• Weekly group supervision with lead investigator to review class topics and activities, engage in role-playing and methods for delivering CBSM material, and review of class videotapes to promote course adherence and troubleshoot any deviations from protocol • Facilitators required to pass written exam on instructor training manual at preintervention
Delivery
Intervention adherence	• Two CBSM-trained research assistants independently reviewed videotapes of each class and completed checklists to assess whether group facilitators covered the material in the instructor training manual over the allotted time
Quality of instruction	• Immediately after each class, group facilitators completed the FIMP (Sigmarsdóttir, DeGarmo, Forgatch, & Guðmundsdóttir, 2013) to rate their teaching in five areas: (1) demonstration of technical knowledge, (2) how well they taught the course content, (3) how much structure was given in class, (4) use of process skills, and (5) overall quality of instruction • Two research assistants also reviewed videotapes to rate the group facilitators using the FIMP • Participants rated group facilitators on their teaching effectiveness using an evaluation form
Receipt
Class attendance	• Participants signed an attendance sheet prior to each class over an 8-week period
Participant understanding	• Two research assistants observed videotaped CBSM classes to rate participants on their comprehension of coping and relaxation skills, their correct application of course concepts and skills in hypothetical situations, and their ability to distinguish CBSM skills from one another • After each class, participants rated the degree to which the intervention material was confusing using an evaluation form
Enactment
Use of intervention skills	• Two research assistants observed videotaped CBSM classes and rated participants on the degree to which they described using intervention skills in real-world scenarios
Homework assignments	• Participants completed “Personal Growth Assessments” at home each week where they recorded the: (1) stressor experienced that day, (2) use of coping/relaxation skill learned in class that week, and (3) change in their stress level after using the coping/relaxation skill

Note. CBSM = cognitive behavioral stress management; FIMP = Fidelity of Implementation Rating System.

Table 2.

Cognitive Behavioral Stress Management (CBSM) Intervention Aims, Strategies, and Techniques.

Intervention topic	Aims	Strategies	Techniques
Class 1: Stress awareness	• Increase awareness of different stress responses • Learn diaphragmatic breathing	• Identify sources of stress and our reactions to stressors • Provide in-session experiences	• Didactic and self-reflection activities • Self-monitoring exercises
Class 2: Thought awareness	• Recognize how thoughts affect our emotions • Learn mindfulness	• Become aware of different automatic negative thoughts • Distinguish thinking errors	• Apply concepts to real-world examples • Mindful eating exercise
Class 3: Thought replacement	• Learn how to balance overly negative/positive self-talk • Learn guided imagery	• Teach ABCDE model to thought replacement • Provide in-session experiences	• Apply antidotes to negative self-talk to personal experiences • Role-playing ABCDE model
Class 4: Coping awareness	• Define different types of coping • Learn progressive muscle relaxation	• Differentiate controllable vs. uncontrollable stressors • Define active vs. passive coping	• Share stressors of pregnancy, childbirth, and motherhood • Use props for coping resources
Class 5: Matching coping	• Identify steps to matching coping strategies to stressors • Learn “letting go of tension”	• Transactional model of coping • Identify active/passive coping strategies for different stressors	• Apply matching of coping strategies to pregnancy stressors • Self-reflection activity
Class 6: Social support	• Recognize how our interactions with others affect our emotions • Learn ‘supportive imagery’	• Identify types of social support • Learn how to match types of support to different needs	• Identify support network • Use props to match type of support needed to manage a stressor
Class 7: communication	• Define different types of communication • Learn ‘therapeutic touch’	• Learn steps to assertive communication • Provide in-session experiences	• Role-play communication styles • Practice active listening and assertive communication
Class 8: Review of coping and relaxation skills	• Identify coping and relaxation skills learned	• Being a role model for baby • Identify upcoming stressors	• Use props for identifying CBSM skills learned

Measures

Implementation Fidelity Factors

Intervention delivery

Assessment of intervention delivery focused on how the CBSM program was taught to establish that the intervention was delivered as intended, as outlined by a training manual, and implemented competently by the group facilitators. Intervention delivery was operationalized as instructor adherence to intervention content, quality of instruction, and participant ratings of teaching effectiveness (see Table 1). Instructor adherence to intervention content was assessed by two CBSM-trained research assistants who independently observed class videotapes and completed checklists to assess to what degree group facilitators covered the intended class material using a 3-point Likert-type scale of 1 = poor coverage to 3 = thorough coverage per published guidelines and recommendations for best practices in fidelity assessment (Bellg et al., 2004; Borrelli, 2011). An average score was calculated across all classes and raters (intraclass correlation coefficient [ICC] = .65) to provide a final score, with higher scores reflecting greater instructor adherence.

Quality of instruction was assessed with an adapted version of the Fidelity of Implementation Rating System (FIMP), which was originally designed to evaluate the quality of instruction for a parent management training intervention (Sigmarsdóttir et al., 2013). The examples used in the items of the original FIMP were modified to reflect the CBSM intervention used in the current study (e.g., “The instructor adjusted CBSM tools and concepts to participants”). Two research assistants independently observed class videotapes and rated instructors using the FIMP on a 9-point Likert-type scale of 1 = unacceptable work to 9 = good work (scores 1-3 = needs work, 4-6 = acceptable, and 7-9 = good work). An average score was calculated across all classes and raters (ICC = .70) to provide a final score, with higher scores reflecting a greater quality of instruction. Teaching effectiveness was assessed using an evaluation form administered after each CBSM class. Participants responded to one item which asked, “How effective were the instructors in teaching today’s lesson?” on a 5-point Likert-type scale of 1 = not effective at all to 5 = very effective. Participants’ ratings were averaged across all attended classes, with higher scores indicative of greater teaching effectiveness.

Intervention receipt

Assessment of intervention receipt focused on determining whether participants received and understood the information provided in the intervention. Intervention receipt was operationalized as class attendance, participants’ understanding of class material, and degree of course clarity. Class attendance was recorded by having participants sign an attendance sheet prior to beginning each class and then summing the number of classes that each participant attended over the 8-week period. Participants’ understanding of class material was assessed through videotaped observations of CBSM classes where two research assistants independently rated participants on a 3-point Likert-type scale of 1 = low understanding (e.g., verbalized misunderstanding of the material) to 3 = high understanding (e.g., correctly articulated course concepts in their own words). Therefore, participants’ understanding of class material was operationalized as participants correctly describing and distinguishing different coping and relaxation skills from one another, as well as correctly applying course concepts and skills in hypothetical situations discussed in the CBSM classes. Any discrepancies in research assistant ratings were discussed and a final rating of participant understanding was assigned for each class. Ratings were averaged across all classes (weighted κ = .62) to provide a final score, with higher scores reflecting greater levels of understanding. Degree of course clarity was assessed via participant self-report where they were asked to respond to one question from an evaluation form administered after each CBSM class that asked, “How confusing was today’s class?” on a 5-point Likert-type scale of 1 = very confusing to 5 = very clear. Participants’ ratings were averaged across all attended classes, with higher scores indicative of greater clarity of course content.

Intervention enactment

Assessment of intervention enactment focused on monitoring participants’ implementation of CBSM skills in relevant, real-life settings in their daily lives and was operationalized as participants’ use of intervention skills and completion of homework assignments. To assess participants’ use of intervention skills, two research assistants independently observed class videotapes and rated participants’ discussions on their use of CBSM skills outside of the class setting on a 3-point Likert-type scale of 1 = low use (e.g., participant never mentioned using CBSM skills outside of the class setting) to 3 = high use (e.g., participant described experiences of how CBSM skills were used in real-world scenarios). Participants were given time at the beginning of each class to discuss their use of learned CBSM skills from the previous week as part of their weekly homework assignments that they were asked to bring in for review (see description below). Therefore, research assistants focused on these videotaped discussions as part of their ratings for participants’ use of intervention skills (Borrelli, 2011). Any discrepancies in these ratings were discussed and a final rating of participant use was assigned for each class. Ratings were averaged across all classes (weighted κ = .62) to provide a final score, with higher scores reflecting greater levels of use.

Homework assignments were explained at the conclusion of each class to complete at home and turn in the following week. These assignments were collected and marked as complete if the participant wrote an example of how she used the CBSM coping/relaxation skill learned that week. The number of completed homework assignments was then summed over the 8-week period (possible range = 1-7 assignments).

Study Outcomes

Perceived stress

Participants’ changes in perceived stress levels were assessed using an evaluation form administered after each CBSM class. Participants responded to one item that asked, “In comparison to your stress level before class, how would you rate your stress level after this class?” on a 5-point Likert-type scale of 1 = much less stressed to 5 = much more stressed. Participants’ ratings were averaged across all attended classes, with lower scores indicative of lower stress following CBSM.

Salivary cortisol

Participant provided saliva samples (via passive drool) immediately before and after Classes 1 (stress awareness), 3 (thoughts), 5 (coping), and 7 (social support), which represented the completion of a specific CBSM module. Classes were taught at either 10 a.m. or 1 p.m., to minimize the effects of diurnal changes in cortisol. Salivary cortisol was then analyzed from these samples using a time-resolved immunoassay with fluorescence detection. Intra- and interassay variability were both less than 10%. Pre- and postclass cortisol values were averaged across all classes attended. A change score was then calculated (postclass cortisol – preclass cortisol), with lower scores representing a decrease in salivary cortisol following CBSM.

Positive and negative mood

The Positive and Negative Affect Schedule Short Form (PANAS-SF; Thompson, 2007) was used to assess participants’ positive and negative mood states before and after Classes 1, 3, 5, and 7. Participants rated the degree to which they felt 10 different feelings and emotions (five positive, five negative) on a 5-point Likert-type scale of 1 = very slightly or not at all to 5 = extremely. A positive and negative mood score was calculated by summing the items corresponding to each subscale (possible range = 5-25) and averaging these scores across all classes attended. A change score was then calculated (postclass mood – preclass mood), with higher scores on the positive mood subscale representing an increase in positive mood states (α = .78) and lower scores on the negative mood subscale representing a decrease in negative mood states (α = .70) following CBSM.

Confidence in using CBSM skills

Participants’ confidence in using CBSM skills was assessed with an evaluation administered after each CBSM class. Participants responded to one item which asked, “How confident do you feel in using the techniques you’ve learned today?” on a 5-point Likert-type scale of 1 = very unconfident to 5 = very confident. Participants’ ratings were averaged across all attended classes, with higher scores indicative of greater confidence in using CBSM skills.

Analyses

Descriptive statistics summarized results on implementation fidelity factors and relevant study outcomes. Pearson correlations examined associations between study outcomes. Repeated measures analyses examined within-group changes in salivary cortisol from pre- to post-CBSM, controlling for saliva collection time and gestational age. Furthermore, paired-samples t-test analyses tested for within-group changes in negative and positive mood states from pre- to post-CBSM. Five separate hierarchical multiple regression analyses were performed to identify which implementation fidelity factors related to delivery, receipt, and enactment were associated with the program outcomes of perceived stress, salivary cortisol, positive and negative mood, and confidence in using CBSM skills. Case-wise deletion of missing data was used in all analyses (range of missing data = 0-4 participants; 97% retention rate). Participants with missing data did not significantly differ from participants with complete data on any baseline characteristics or study outcomes.

Results

Participant Characteristics

Descriptive characteristics are shown in Table 3. Participants were 26 years of age and 10 weeks pregnant, on average, when they entered the study. The majority of our sample were Latina women who were born outside the United States, were single, unemployed, and had an annual family income of less than $20,000 per year. Most women also had a high school education or less and had at least one other child prior to their current pregnancy.

Table 3.

Sociodemographic, Implementation Fidelity, and Study Outcome Characteristics for Low-Income Pregnant Women in SMART Moms/Mamás LÍSTAS Project (n = 55).

Characteristic	M (SD) or %	Range (Min., Max.)
Sociodemographic
Age (years)	26.3 (±6.4)	(18, 39)
Number of weeks pregnant	10.3 (±4.2)	(4, 17)
Ethnicity
Latina	69.1	—
African American	14.5	—
Asian American	5.5	—
Non-Hispanic white	7.3	—
Mixed ethnicity	3.6	—
Country of birth
Outside of United States	58.2	—
U.S.-born	41.8	—
Marital status
Single	52.7	—
Married	47.3	—
Employment status
Unemployed	72.7	—
Employed	27.3	—
Annual family combined income
<$20,000	79.6	—
≥$20,000	20.4	—
Education
High school education or less	67.3	—
Some college or more	32.7	—
Parity status
Nulliparous	38.2	—
Multiparous	61.8	—
Implementation fidelity
Intervention delivery
Instructor adherence to intervention content	2.9 (±0.8)	(2.7, 3.0)
Quality of instruction	7.4 (±0.7)	(5.9, 8.7)
Participant rating of instructor teaching effectiveness	4.5 (±0.9)	(2, 5)
Intervention receipt
Class attendance	5.0 (±2.2)	(1, 8)
Participant understanding of class content	2.3 (±0.5)	(1, 3)
Participant rating of class clarity	4.7 (±0.5)	(2, 5)
Intervention enactment
Participant use of intervention skills	2.0 (±0.7)	(1, 3)
Completion of homework assignments	3.0 (+ 2.3)	(0, 7)
Study outcomes (postintervention)
Perceived stress	1.6 (±0.8)	(1, 5)
Confidence in using CBSM skills	4.3 (±1.1)	(1, 5)

Note. M = mean; SD= standard deviation; Min. = minimum; Max. = maximum; CBSM = cognitive behavioral stress management.

Program Implementation Fidelity Characteristics

Intervention Delivery

On average, group facilitators demonstrated very thorough coverage of CBSM content and adherence to the manual (M = 2.9 out of 3). The quality of instruction, as measured by the FIMP, was also high and in the “good work” classification for that scale (M = 7.4 out of 9). Participants also rated the quality of teaching as being “very effective” (M = 4.5 out of 5).

Intervention receipt

Participants attended approximately 63% of CBSM classes, which is within the range reported in other CBSM interventions with low-income pregnancy populations (58% to 80%; Field et al., 2000; Urizar & Muñoz, 2011). On average, participants demonstrated a “medium” level of understanding of class material (M = 2.3 out of 3) as measured by videotaped observations. Additionally, participants rated CBSM content as being presented “very clearly” (M = 4.7 out of 5).

Intervention Enactment

On average, participants described a “medium” level of using CBSM skills in real-life scenarios (M = 2.0 out of 3) and completed approximately 43% of their homework assignments (M = 3.0 out of 7 assignments), which is below the range reported in other behavioral interventions with mostly non-Hispanic white mothers (50% to 67%; Goodman et al., 2014; Leermakers, Anglin, & Wing, 1998).

Association Between Study Outcomes

Pearson correlations showed that participants who reported feeling less stressed post-CBSM were associated with having increased levels of positive mood from pre- to post-CBSM, r(50) = −0.315, p = .026. Additionally, participants who reported feeling more confident in using CBSM skills were associated with having decreased levels of negative mood from pre- to post-CBSM, r(51) = −0.280, p = .047. However, changes in salivary cortisol levels from pre- to post-CBSM were not significantly associated with any of the other study outcomes. See Table 4 for correlations between all study outcomes.

Table 4.

Pearson Correlation Analyses Between Postintervention Study Outcomes for Low-Income Pregnant Women Participating in Cognitive Behavioral Stress Management (CBSM).^a

Study outcome	Perceived stress	Salivary cortisol	Positive mood	Negative mood	Confidence in using CBSM skills
Perceived stress	—
Salivary cortisol	0.036	—
Positive mood	−0.315*	−0.191	—
Negative mood	0.039	−0.038	0.125	—
Confidence in using CBSM skills	−0.221	−0.183	−0.092	−0.280*	—

Scores reflect pre- to postintervention changes in study outcomes.

p < .05.

Impact of Implementation Fidelity on Study Outcomes

Perceived Stress

Approximately 74% of participants reported feeling “much less stressed” post-CBSM compared with baseline (see Table 1). Greater instructor adherence to the CBSM manual (p = .042) and greater quality of instruction (p = .027) resulted in lower perceived stress. In contrast, participants who completed more homework assignments reported higher perceived stress post-CBSM (p = .048). The overall variance explained by the final model was 39% (see Table 5).

Table 5.

Hierarchical Regression Analyses for Implementation Fidelity Factors Associated With Postintervention Changes in Perceived Stress and Salivary Cortisol.

	Perceived stress				Cortisol
Variables	R ²	ΔR²	β	SE	R ²	ΔR²	β	SE
	.39*				.20
Intervention delivery
Instructor adherence		.106	−0.287*	0.136		.006	0.062	0.156
Quality of instruction		.089	−0.382*	0.166		.030	−0.047	0.190
Perceived instructor effectiveness		.045	−0.299	0.158		.000	0.042	0.181
Intervention receipt
Class attendance		.005	−0.323	0.185		.056	−0.204	0.213
Participant understanding of CBSM		.009	0.078	0.193		.042	−0.364	0.221
Participant rating of class clarity		.001	−0.040	0.154		.002	−0.127	0.177
Intervention enactment
Participant use of intervention skills		.023	0.204	0.176		.057	0.323	0.202
Completing homework assignments		.112	0.379*	0.185		.002	−0.058	0.213

Note. CBSM = cognitive behavioral stress management; SE = standard error.

p < .05.

Salivary Cortisol

Participants’ salivary cortisol significantly decreased from pre- to post-CBSM, controlling for saliva collection time and gestational age, F(4, 48) = 13.36, p = .002 (see Figure 1a). However, none of the implementation fidelity factors were significantly associated with these decreases in salivary cortisol. A nonsignificant trend was observed with greater participant understanding of CBSM concepts being associated with lower cortisol levels post-CBSM (p = .108). The overall variance explained by the final model was 20% (see Table 5).

Figure 1.

Cognitive behavioral stress management (CBSM) intervention effects on salivary cortisol and positive and negative mood states among low-income pregnant women. (a) Pre- and postintervention cortisol levels. (b) Pre- and postintervention mood levels.

Positive and Negative Mood

Participants’ positive mood significantly increased from pre- to post-CBSM, t(50) = −5.85, p < .001 (see Figure 1b). Contrary to what was expected, participants who completed more homework assignments reported lower positive mood (p = .045). The overall variance explained by the final model was 22% (see Table 6). Participants’ negative mood significantly decreased from pre- to post-CBSM, t(51) = 3.59, p = .001 (see Figure 1b), with participants who used CBSM skills reporting lower negative mood (p = .03). The overall variance explained by the final model was 23% (see Table 6).

Table 6.

Hierarchical Regression Analyses for Implementation Fidelity Factors Associated With Postintervention Changes in Positive and Negative Mood States and Confidence in Using CBSM Skills.

	Positive mood				Negative mood				Confidence in CBSM skills
Variables	R ²	ΔR²	β	SE	R ²	ΔR²	β	SE	R ²	ΔR²	β	SE
	.22				.23				.27
Intervention delivery
Instructor adherence		.001	−0.024	0.156		.018	0.114	0.153		.015	0.137	0.149
Quality of instruction		.013	0.168	0.179		.000	0.094	0.186		.192	0.380*	0.181
Perceived instructor effectiveness		.017	0.220	0.181		.038	0.286	0.178		.000	−0.057	0.173
Intervention receipt
Class attendance		.060	0.015	0.203		.023	0.137	0.208		.053	0.230	0.203
Participant understanding of CBSM		.047	0.075	0.198		.025	0.059	0.216		.000	−0.023	0.211
Participant rating of class clarity		.004	−0.046	0.180		.006	0.005	0.174		.003	0.105	0.169
Intervention enactment
Participant use of intervention skills		.000	0.022	0.194		.109	−0.447*	0.198		.005	−0.092	0.193
Completing homework assignments		.105	−0.430*	0.207		.010	0.018	0.208		.000	0.016	0.203

Note. CBSM = cognitive behavioral stress management; SE = standard error.

p < .05.

Confidence in Using CBSM Skills

Approximately 81% of participants reported feeling “somewhat or very confident” in using CBSM skills (see Table 1). Greater quality of instruction resulted in participants feeling more confident in using CBSM skills (p = .043). The overall variance explained by the final model was 27% (see Table 6).

Discussion

The current study is one of the few to evaluate the implementation fidelity of a prenatal CBSM intervention. Results indicated that intervention delivery was excellent with group facilitators demonstrating thorough coverage of CBSM content, adherence to the treatment manual, and good quality of instruction. Intervention receipt was adequate with participants attending 63% of CBSM classes and demonstrating a good level of understanding of the class material. Intervention enactment was less than desired with participants reporting some use of learned CBSM skills in real-life scenarios and 43% completing homework assignments.

This study also assessed whether implementation fidelity factors were associated with participant outcomes. Intervention delivery emerged as a key component for improved outcomes with greater instructor adherence to CBSM content and quality of instruction resulting in lower perceived stress (74% felt less stressed post-CBSM). Greater quality of instruction also resulted in 81% of participants feeling more confident in using the CBSM skills they learned. These results are consistent with that of a parenting management program that showed greater quality of instruction to be associated with lower parental stress and enhanced parenting skills up to 18 months postintervention (Thijssen, Albrecht, Muris, & de Ruiter, 2017).

Factors related to intervention receipt (i.e., class attendance, participants’ understanding of class material, and degree of course clarity) were not significantly associated with participant outcomes. Despite these nonsignificant results, an interesting pattern emerged with greater participant understanding of CBSM concepts being associated with lower cortisol levels from pre- to post-CBSM (p = .108). These results are consistent with that of a CBSM intervention for breast cancer patients that showed greater improvements in participants’ perceived ability to use CBSM skills to be associated with lower cortisol levels up to 12 months postintervention (Phillips et al., 2011). Together, results of these studies suggest that implementation factors related to intervention receipt need to be examined further to better understand how CBSM interventions may help regulate physiological markers of stress. Results of such studies may vary by the stress outcome used (e.g., blood pressure, cortisol), the time-period examined (i.e., immediately following the CBSM intervention vs. longer term effects over time), and the population studied (e.g., pregnant vs. nonpregnant populations).

Intervention enactment had differing effects on participant outcomes. When assessed through videotaped observations, participants who described using CBSM skills in real-world scenarios reported experiencing lower negative mood post-CBSM. These results are consistent with that of previous studies showing that participants who practiced relaxation skills had reduced anxiety and somatic symptoms of stress (e.g., blood pressure, cortisol), which have been associated with stress-related arousal and negative mood states (e.g., anxiety, distress, fear; Antoni, 2013; Murta et al., 2007). Additionally, participants who completed more homework assignments reported experiencing higher perceived stress and lower positive mood post-CBSM. These results are similar to those by Urizar & Muñoz (2011) who reported that low-income women participating in a prenatal CBSM intervention had higher levels of perceived stress at 6 months postpartum. They speculated that these findings were either due to the homework assignments being burdensome and adding to the competing time demands that low-income mothers face (e.g., work, caring for their children) or due to the assignments increasing their awareness of stress-related symptoms. These findings highlight the need for follow-up qualitative studies to help explain the role of homework assignments in affecting different stress outcomes in this population.

Study Limitations

Several limitations merit mention. First, implementation fidelity was only assessed for those randomized to the CBSM intervention and not those in our attention control group, thereby limiting our sample size (n = 55) and ability to compare changes in these fidelity measures and program outcomes across groups. Women in the attention control group received printed prenatal health information by mail versus coming to a group class, thereby making it challenging to collect comparable class data on implementation fidelity and program outcomes in this group. Larger samples sizes in randomized-control CBSM trials would also increase the power needed to identify associations across different types of implementation fidelity measures on various stress-related outcomes. Finally, there were limitations to using videotaped observations to assess intervention delivery, receipt, and enactment, with more vocal participants during the CBSM classes having greater opportunities to be scored higher on these constructs than less vocal participants. Operationalization of intervention receipt and enactment were particularly challenging and suggest that future studies use additional methodologies, such as quizzes or interviews, to assess participant understanding of CBSM content and allow time for participants to demonstrate understanding and use of CBSM skills during class.

Conclusions

Our results suggest that CBSM interventions can be implemented with fidelity and that certain implementation fidelity factors are associated with improving stress-related outcomes among low-income women learning CBSM skills during pregnancy. In particular, results related to intervention delivery (i.e., instructor adherence to CBSM content and quality of instruction) and enactment (i.e., participant use of CBSM skills in real-world scenarios) were associated with improved participant outcomes (i.e., lower perceived stress, increased confidence to use CBSM skills, and lower negative mood). These results help identify the mechanisms by which CBSM interventions may help to improve health outcomes during pregnancy and enhance the reliability and validity of such interventions.

Our results also highlighted the need to enhance intervention receipt and enactment as some elements of these implementation factors either were not significantly associated with program outcomes (intervention receipt) or were also associated with higher perceived stress and lower positive mood (intervention enactment). To strengthen participant receipt and enactment of core CBSM concepts, delivery of the intervention may need to be culturally tailored for different subgroups of women. More specifically, cultural relevancy can be enhanced by tailoring intervention materials and messages (e.g., modifying images, examples, and labels used for coping and relaxation skills) to make CBSM concepts more salient to the everyday experiences of low-income populations and strengthen the intervention’s impact on program outcomes (Borrelli, 2011). Additionally, studies have begun to examine other modalities for promoting homework adherence in low-income communities, such as the use of text messaging, with promising results to make this component of intervention enactment less cumbersome for participants (Aguilera & Muñoz, 2011).

These results have potential implications for the health of mothers and their infants in helping to reduce stress-related health complications (e.g., postpartum depression, preterm births), as less than 25% of women report actively engaging in stress management during pregnancy (Birdee, Kemper, Rothman, & Gardiner, 2014). Therefore, community-based CBSM interventions are needed to reach underserved women during pregnancy and provide them with the necessary skills and knowledge to facilitate stress management. The CBSM intervention developed as part of the SMART Moms/Mamás LÍSTAS Project could help to address existing barriers (limited resources and training) to providing stress management interventions for English- and Spanish-speaking low-income women by providing a manualized prenatal program that can be delivered by prenatal health care professionals (e.g., social workers, OBGYNs). Cumulatively, our findings support the need to translate evidence-based interventions into clinical practice to optimize the health of low-income mothers and their infants.

Footnotes

Acknowledgements

The authors gratefully acknowledge the contributions of the PRO-Health Research Lab for their instrumental support in data collection and Nicolas Rohleder, PhD, for conducting the cortisol assays.

Authors’ Note

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under Award Number SC2HD068878 (Guido G. Urizar Jr., PI). Additional support was provided by grants from the National Institute of General Medical Sciences of the National Institutes of Health under Award Numbers: UL1GM118979, TL4GM118980, RL5GM118978 (Guido G. Urizar Jr., PI).

ORCID iD

Guido G. Urizar Jr.

References

Aguilera

Muñoz

R. F.

(2011). Text messaging as an adjunct to CBT in low-income populations: A usability and feasibility pilot study. Professional Psychology, Research and Practice, 42, 472-478. doi:10.1037/a0025499.

Antoni

M. H.

(2003). Stress management intervention for women with breast cancer. Washington, DC: American Psychological Association.

Antoni

M. H.

(2013). Psychosocial intervention effects on adaptation, disease course, and biobehavioral processes in cancer. Brain, Behavior, & Immunity, 30(Suppl.), S88-S98. doi:10.1016/j.bbi.2012.05.009

Antoni

M. H.

Cruess

D. G.

Klimas

Carrico

A. W.

Maher

Cruess

. . . Schneiderman

(2005). Increases in a marker of immune system reconstitution are predated by decreases in 24-h urinary cortisol output and depressed mood during a 10-week stress management intervention in symptomatic HIV-infected men. Journal of Psychosomatic Research, 58, 3-13. doi:10.1016/j.jpsychores.2004.05.010

Bellg

A. J.

Borrelli

Resnick

Hecht

Minicucci

D. S.

Ory

, . . . Treatment Fidelity Workgroup of the NIH Behavior Change Consortium. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23, 443-451. doi:10.1037/0278-6133.23.5.443

Birdee

G. S.

Kemper

K. J.

Rothman

Gardiner

(2014). Use of complementary and alternative medicine during pregnancy and the postpartum period: An analysis of the National Health Interview Survey. Journal of Women’s Health, 23, 824-829. doi:10.1089/jwh.2013.4568

Borrelli

(2011). The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry, 71, 552-563. doi:10.1111/j.1752-7325.2011.00233.x

Breitenstein

S. M.

Gross

Garvey

C. A.

Hill

Fogg

Resnick

(2010). Implementation fidelity in community-based interventions. Research in Nursing Health, 33, 164-73. doi:10.1002/nur.20373

Dunkel Schetter

Niles

A. N.

Guardino

C. M.

Khaled

Kramer

M. S

. (2016). Demographic, medical, and psychosocial predictors of pregnancy anxiety. Paediatric Perinatal Epidemiology, 30, 421-429. doi:10.1111/ppe.12300

10.

Durlak

J. A.

DuPre

E. P.

(2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350. doi:10.1007/s10464-008-9165-0

11.

Field

Pickens

Prodromidis

Malphurs

Fox

Bendell

. . . Kuhn

(2000). Targeting adolescent mothers with depressive symptoms for early intervention. Adolescence, 35, 381-414.

12.

Goodman

J. H.

Guarino

Chenausky

Klein

Prager

Petersen

Forget

Freeman

(2014). CALM Pregnancy: Results of a pilot study of mindfulness-based cognitive therapy for perinatal anxiety. Archives of Women’s Mental Health, 17, 373-87. doi:10.1007/s00737-013-0402-7

13.

Glasgow

R. E.

(2008). What types of evidence are most needed to advance behavioral medicine? Annals of Behavioral Medicine, 35, 19-25. doi:10.1007/s12160-007-9008-5

14.

Glynn

L. M.

Dunkel-Schetter

Chicz-DeMet

Hobel

C. J.

Sandman

C. A.

(2007). Ethnic differences in adrenocorticotropic hormone, cortisol and corticotropin-releasing hormone during pregnancy. Peptides, 28, 1155-1161. doi:10.1016/j.peptides.2007.04.005

15.

Havermans

B. M.

Schelvis

R. M. C.

Boot

C. R. L.

Brouwers

E. P. M.

Anema

J. R.

van der Beek

A. J.

(2016). Process variables in organizational stress management intervention evaluation research: A systematic review. Scandinavian Journal of Work Environment Health, 42, 371-381. doi:10.5271/sjweh.3570

16.

Hodyl

N. A.

Aboustate

Bianco-Miotto

Roberts

C. T.

Clifton

V. L.

Stark

M. J.

(2017). Child neurodevelopmental outcomes following preterm and term birth: What can the placenta tell us? Placenta, 57, 79-86. doi:10.1016/j.placenta.2017.06.009

17.

Kraag

Van Breukelen

Lamberts

Vugts

Kok

Fekkes

Huijer Abu-Saad

(2007). Process evaluation of “Learn Young, Learn Fair”: A stress management programme for 5th and 6th graders. School Psychology International, 28, 206-219. doi:10.1177/0143034307078100

18.

Leermakers

E. A.

Anglin

Wing

R. R.

(1998). Reducing postpartum weight retention through a correspondence intervention. International Journal of Obesity, 22, 1103-1109.

19.

Lefmann

Combs-Orme

Orme

J. G.

(2017). Examining the inter-correlated effects of low income, life stress, and race on birth outcomes: A representative state study. Social Work in Health Care, 56, 450-469. doi:10.1080/00981389.2017.1316811

20.

Muñoz

R. F.

H. N.

Ghosh Ippen

Diaz

M. A.

Urizar

Soto

. . . Lieberman

(2007). Prevention of postpartum depression in low-income women: Development of the Mamás y Bebés/Mothers and Babies course. Cognitive & Behavioral Practice, 14, 70-83. doi:10.1016/j.cbpra.2006.10.001

21.

Murta

S. G.

Sanderson

Oldenburg

(2007). Process evaluation in occupational stress management programs: A systematic review. American Journal of Health Promotion, 21, 248-254. doi:10.4278/0890-1171-21.4.248

22.

Phillips

K. M.

Antoni

M. H.

Carver

C. S.

Lechner

S. C.

Penedo

F. J.

McCullough

M. E.

. . . Blomberg

B. B.

(2011). Stress management skills and reductions in serum cortisol across the year after surgery for non-metastatic breast cancer. Cognitive Therapy Research, 35, 595-600. doi:10.1007/s10608-011-9398-3

23.

Sigmarsdóttir

DeGarmo

D. S.

Forgatch

M. S.

Guðmundsdóttir

E. V.

(2013). Treatment effectiveness of PMTO for children’s behavior problems in Iceland: Assessing parenting practices in a randomized controlled trial. Scandinavian Journal of Psychology, 54, 468-476. doi:10.1111/sjop.12078

24.

Thijssen

Albrecht

Muris

de Ruiter

(2017). Treatment fidelity during therapist initial training is related to subsequent effectiveness of parent management training—Oregon model. Journal of Child and Family Studies, 26, 1991-1999. doi:10.1007/s10826-017-0706-8

25.

Thompson

E. R.

(2007). Development and validation of an internationally reliable short-form of the Positive and Negative Affect Schedule (PANAS). Journal of Cross-Cultural Psychology, 38, 227-242. doi:10.1177/0022022106297301

26.

Urizar

Kofman

(2012). The SMART Moms (Stress Management and Relaxation Training for Moms) Course: A prenatal stress management program. Unpublished manual.

27.

Urizar

Muñoz

R. F.

(2011). Impact of a prenatal cognitive behavioral stress management intervention on salivary cortisol levels in mothers and their infants. Psychoneuroendocrinology, 36, 1480-1494. doi:10.1016/j.psyneuen.2011.04.002

28.

Urizar

Yim

Rodriguez

Dunkel Schetter

(2019). The SMART Moms Program: A Randomized trial of the impact of stress management on perceived stress and cortisol in low-income pregnant women. Psychoneuroendocrinology, 104, 174-184. doi:10.1016/j.psyneuen.2019.02.022

29.

U.S. Census Bureau. (2014). Selected economic characteristics: 2010-2014 American Community Survey 5-Year Estimates. Retrieved from https://www.census.gov/acs/www/data/data-tables-and-tools/data-profiles/2014/

30.

Van den Bergh

B. R. H.

van den Heuvel

M. I.

Lahti

Braeken

de Rooij

S. R.

Entringer

. . . Schwab

. (2017). Prenatal developmental origins of behavior and mental health: The influence of maternal stress in pregnancy. Neuroscience & Biobehavioral Reviews. Advance publication online. doi:10.1016/j.neubiorev.2017.07.003

31.

Yim

I. S.

Tanner Stapleton

L. R.

Guardino

C. M.

Hahn-Holbrook

Dunkel Schetter

(2015). Biological and psychosocial predictors of postpartum depression: Systematic review and call for integration. Annual Review of Clinical Psychology, 11, 99-137. doi:10.1146/annurev-clinpsy-101414-020426