Evaluation of an SCCT-Based Intervention to Increase Postsecondary Awareness in Rural Appalachian Youth

Career and college readiness for rural Appalachian youth

The emerging studies of this population yield interesting but sometimes conflicting results. Ali and Saunders (2006) found that rural Appalachian high schoolers connected perceptions of parental support to their career and college plans, emphasizing the importance of family as a value for these students. A later study (Ali & Saunders, 2009), however, found career aspirations were predicted by career self-efficacy and outcome expectations but not perceived support from family and friends. Brown, Copeland, Costello, Erkanli, and Worthman (2009) highlighted the importance of college-educated role models in their study on educational outcomes in Appalachian communities. They learned that students, especially boys, who were exposed to college-educated adults were more likely to plan to attend college themselves. In their study of rural Appalachian high school students, Carrico, Matusovich, and Paretti (2017) noted that interest did not always connect to career choice, often because of the influence of family beliefs or desires. Additionally, career choice was strongly influenced by the desire to stay local for those with longer family ties to their local community, highlighting the importance of localism. Rural Appalachian middle school students reported that academic motivation and positive peer relationships were directly related to higher academic achievement (Hoffman, Anderson-Butcher, Fuller, & Bates, 2017). Also, Wettersten et al. (2005) found that self-efficacy, outcome expectations, and perceived supports and barriers strongly predicted attitudes about school engagement and that parents strongly influenced career and college plans for rural Appalachian youth. It appears family likely influences career and college planning, students need college-educated role models, and believing in their ability to complete the tasks needed to enter into and complete college are vital components for rural Appalachian youth. One theory that directly addresses all of these components is social cognitive career theory (SCCT; Lent, Brown & Hackett, 1994).

Classroom-based career education components

As will be described below, the PiPES classroom curriculum is grounded in SCCT, implemented through a culturally informed lens. On the surface, program components may not appear to be specific to rural Appalachia. The PiPES curriculum is grounded in SCCT and principles of effective career education and as such focuses on reducing proximal barriers and increasing supports, increasing self-efficacy, raising outcome expectations, exploring self and the world of work, and so on—aspects likely to be included in effective career education with any group. However, the specific ways in which these activities are discussed with students and the lens through which these activities are understood by our staff is culturally specific. Indeed, the entire premise of the PiPES program is culturally informed: in a population with historical, cultural, and systemic barriers to postsecondary education, attempts to increase interest in STEMM without first increasing interest in postsecondary education are unlikely to be successful. Thus, whereas many funded programs designed to increase the diversity of the STEMM workforce focus on immersive STEMM experiences for students, PiPES is culturally specific, and thus unique, because of its focus on career education and fostering postsecondary education.

In planning activities, we paid special attention to the unique context of rural Appalachian youth, including strong connection to their local communities and family, lack of college-educated role models, possible PFGCS status, and lower college-going self-efficacy. We emphasize the ways in which biomedical, behavioral, and clinical research sciences can help remedy health inequalities in rural Appalachia, as well as highlight the fact that STEMM careers represent a large portion of the available jobs in these rural communities (Peterson, Bornemann, Lydon, & West, 2015); thus, STEMM careers might allow students to stay in and help their local communities. In addition, we encourage all types of postsecondary education, not just 4-year college. Because military service is particularly highly valued in many Appalachian communities, we also emphasize the ways in which students can pursue postsecondary training in the military. Finally, although the program is designed to address inequalities many students in this region face, we also explicitly train our intervention leaders to be aware of the many ways in which rural Appalachia is a traditionally stigmatized and stereotyped community and to take a strengths-based approach throughout the curriculum. Table 1 provides an overview of the 6-week version of the curriculum and description of how each activity relates to SCCT constructs; a brief description of each SCCT-grounded lesson and an explanation of at least one activity from each week within the 6-week curriculum format is provided below. As we describe specific aspects of the curriculum below, we will highlight additional ways in which these components are conceived or operationalized in culturally specific ways.

OPEN IN VIEWER

Table 1.

PiPES Learning Experiences and Intended Outcomes by Curriculum Week and Targeted SCCT Construct.

Week		Self-Efficacy	Outcome Expectations	Barriers and Supports	Interests	Goals/Aspirations
Week 1: Introduction and rapport building	Activities	Near-peer role model introductions			Icebreakers	10-Year class reunion
	Intended outcome	How confident about PSE and careers are others like me?	Why do others like me think PSE/STEMM is useful?	How have others like me overcome barriers and found support?	What makes me similar to and unique from others?	What do I want my life to be like in ∼12 years?
Week 2: Exploration of self	Activities	Strengths/heroes and who am I? worksheet	—	Discussion	Childhood activity; Holland Career Party; Strengths/Heroes; Who Am I?	Who Am I? Worksheet
	Intended outcome	What am I confident about? Where does that come from?	—	How does knowing myself help me anticipate barriers and supports?	What are my interests, values, and strengths?	How does knowing myself inform my goals?
Week 3: Exploration of the world of work	Activities	O*Net exploration, worksheets, and discussion
	Intended outcome	For what tasks/domains do I have more/less confidence?	What are the salaries, prestige of various career options?	What barriers to/supports for different careers might I have?	What careers might be a good fit, given my interest, values, and so on?	What are some career options I’d like to explore further?
Week 4: PSE options and connections to work/career	Activities	College lingo taboo game; four corners barriers activity	Discussion	Four corners barriers activity	Discussion	Discussion
	Intended outcome	How does understanding basic information about PSE options affect my confidence? How confident are my (near) peers about coping with barriers?	Why PSE is valuable?	What barriers to/supports for PSE do I and others like me anticipate? How might I cope with the barriers and where might I find support?	How does my self-knowledge relate to possible career and PSE options?	What PSE options make the most sense, given my career goals?
Week 5: STEMM	Activities	Discussion of draw-A-Scientist activity with peers and near-peer role models; videos; discussion of STEMM careers
	Intended outcome	How confident are others like me about STEMM?	What does STEMM contribute to society? What do scientists enjoy about their work?	What barriers to/supports for STEMM might I encounter? How have others overcome barriers/found support?	What STEMM career options might be a good fit for my interests, values, and strengths?	What are my PSE and career goals?
Week 6: Goal setting and wrapping up	Activities	Discussion of students’ hopes and fears; integrative reflection over PiPES curriculum; goal-setting discussion and worksheet
	Intended outcome	What am I more/less confident about re: My PSE and career goals?	How would pursuing PSE/STEMM help me meet my goals?	What barriers might I face and where can I find support?	What are my career interests and how can I explore them further?	What are my next steps toward meeting my PSE/career goals?

Note. SCCT = social cognitive career theory; PSE = postsecondary education; STEMM = science, technology, engineering, math, and medical science; PiPES = Possibilities in Postsecondary Education and Science.

Week 1

We introduced the PiPES program within students’ cultural context, acknowledging the health inequalities in the region, the ways in which STEMM careers can allow students to help their communities, and our goals of helping students find educational and career paths they are excited about and that keep them connected to their families and homes. Introductory and goal-setting activities began to address the SCCT variables of self-efficacy, outcome expectations, supports and barriers, interests, and goals/aspirations. For example, students completed a My 10-Year Class Reunion activity sheet to imagine potential career, personal, and postsecondary accomplishments to tell friends at their 10-year high school reunion. They completed sentences such as “the job I want in 10 years is” and “the thing I will be most proud of by then is.” Following this activity, students engaged in a discussion of success and potential steps to make their dreams come true. Although these are common activities in many cultural contexts, our staff were trained to listen for and highlight themes of localism, family and community, and lack of information about opportunities and career possibilities. For example, our staff explicitly asked about how success is defined in students’ families, not just how the students individually defined success. In discussing the activity, students commonly mentioned uncertainty about what kinds of careers are possible for them, the importance of being able to support family members and stay connected to their home communities, and limited job opportunities available in their communities.

Week 2

The goal of this week was self-exploration designed to increase self-efficacy, reduce barriers and increase supports, and help develop interests and goals. Students completed an activity in pairs or small groups to help them identify personal strengths or important values through discussion of childhood heroes or of positive childhood experiences and the characteristics they exemplified. Students were provided with a worksheet to facilitate their thinking and discussion, ² and team members circulated throughout the activity to offer encouragement and guidance as needed, before students came back together to share common themes in the larger group and engage in brief discussion about how this self-exploration informs career exploration. Students then participated in a Career Party activity (generally attributed to Bolles, 1990) to identify personality traits and interests. Six stations corresponding to the Holland Codes (Holland, 1997) were positioned around the classroom, and childhood toys were used to help provide visual representations of interests. For example, the artistic station included toys such as paint and musical instruments, and students who identified as creative or innovative selected this group. The activity resulted in a three-letter Holland (1997) realistic, investigative, artistic, social, enterprising, conventional (RIASEC) code.

Week 3

Students met in their school computer lab to participate in online career exploration designed to address all five SCCT constructs included in Table 1. Using the Career Party codes determined in Week 2, students were introduced to Holland Codes that related to career preferences for work experiences. Students were provided a list of STEMM careers sorted by Holland Code (Holland, 1997) and used this list of STEMM careers to facilitate career searches using an online database (www.onetonline.org). Careers represented all educational levels to appeal to our students. We used the Interests advanced search feature on O*Net to connect the Holland Code activity to their career search. Students completed a career research worksheet while conducting their online search, where they were asked to determine education and training requirements and describe the main responsibilities of the occupation.

Week 4

The goals of this week were to address contextual affordances to postsecondary education and success and to increase awareness of postsecondary options and college-going self-efficacy. We discussed multiple barriers to postsecondary education that might be relevant to rural Appalachian youth (e.g., finances, family support, location, academic readiness) and helped identify supports for addressing these barriers. Knowing that many of our students would be PFGCSs, and that lack of information about postsecondary education is a common barrier throughout the region, we also introduced them to postsecondary options and college lingo. Students were introduced to terms such as major, credit hour, tuition, and financial aid. To reinforce their learning, we played a college lingo game using college-going vocabulary. Students were divided into two teams and alternated having a teammate provide clues to help them guess a college lingo term without saying key terms. For example, students tried to get teammates to guess the term “syllabus” by describing it as a document provided the first day of class that lists assignments and dates for exams.

Week 5

The goal for Week 5 was to increase STEMM interest by connecting postsecondary options to public health needs in local communities, drawing on cultural values of localism and community. Sample activities included a Draw-A-Scientist activity, in which students drew a picture of a scientist and wrote five words about what a scientist looks like and five words about what a scientist does. Typical drawings included men with “crazy hair” wearing lab coats and glasses, working in isolation. This activity provided opportunities to challenge stereotypes of scientists and introduce a personalization of self as scientist. Other activities included discussion of video clips portraying research as exciting and relevant, as well as how public health issues faced by their friends and family related to possible STEMM careers.

Week 6

The final week of the program provided an opportunity for students to create a plan for reaching academic and postsecondary goals and to collect program evaluation data. This included time for answering lingering questions about postsecondary education and STEMM careers, completing and discussing a Putting It All Together activity sheet, and administering instruments for data collection. The Putting It all Together activity had students list three potential career goals, identify how these careers connect with family or community values (an intentional inclusion to reflect the culturally interdependent context in which most of these students make career decisions), and check a list of recommendations for potential behaviors throughout high school that can facilitate accomplishing these goals. This list of recommendations included items such as take a leadership role in a club or activity, start a list of potential training programs or colleges, and register for a foreign language if planning to attend a 4-year college. All recommendations were grounded in what was already available and encouraged at their school.

Program changes

Based on feedback after the initial year of interventions, we made some minor adjustments to the program before Year 2. We did not change the curriculum content, but we incorporated differentiated instruction. For example, we created an advanced career exploration research form used on Day 3 that provided a more detailed career search for academically advanced students. We added strategies such as polling students on Day 1 about postsecondary plans to target discussions each week, focusing more on college as a postsecondary option than apprenticeships and technical schools in academically advanced classes where there were a higher number of students planning to attend 4-year college, and incorporating more small group discussion. We also adjusted scheduling for the intervention by offering each school a 6-week or 8-week curriculum option.

Student participants

School 1 was our largest participating high school with 1,317 students, of whom 60.7% were economically disadvantaged; just over half (∼55%) were female and the vast majority (97%) self-identified as non-Hispanic White. Both years of the intervention at School 1 were delivered to 10th-grade students, approximately 350 students each year. Of these students, approximately 23% were prospective first-generation college students (defined as students for whom neither parent had any postsecondary education at all, not even a semester of community college), 52% could be categorized as nonprospective first-generation college students, and the remaining and 25% could not be categorized because they were unsure of their parents’ educational attainment. School 2 was located in the same county as School 1, with a total enrollment of 387 students. This school had the highest poverty rates of our targeted schools, with 73.9% of students who were economically disadvantaged; gender was nearly equally split. The Year 1 intervention was delivered to approximately 100 eleventh-grade students, and Year 2 intervention was delivered to approximately 85 tenth-grade students. Approximately 43% of students were prospective first-generation college students, another 40% were not, and the remaining 17% could not be categorized. Approximately 98% self-identified as non-Hispanic White. Finally, School 3, located in a neighboring county, had an enrollment of 815 students, of whom 57.4% were economically disadvantaged. Year 1 intervention was delivered to approximately 215 eleventh-grade students, and Year 2 intervention was delivered to approximately 220 tenth-grade students; approximately 54% were male, 96% self-identified non-Hispanic White; and approximately 40% were prospective first-generation college students, and approximately 43% were not. Across the 2 years and three schools, the intervention was delivered to approximately 1,320 students. As part of the larger longitudinal study on the intervention and its effect on students, we delayed implementation of the intervention until Year 2 at two of the schools to create well-matched comparison groups. This will allow us to compare outcomes for students who did and did not receive the intervention.

Project staff participants

Graduate and undergraduate students from the university delivered the multiweek curriculum in teams of two or three. We intentionally recruited students from rural Appalachian backgrounds to serve as near peer role models. Over the first 2 years of the project, 7 doctoral students from counselor education and psychology degree programs served as team leaders (including the third, fourth, and fifth authors), 6 master’s students studying School Counseling and Clinical Mental Health Counseling, and 11 undergraduate students majoring in Psychology or Child and Family Studies also participated in the project. This included 11 project staff divided into four teams in the first semester of Year 1, 12 project staff divided into four teams in the second semester of Year 1, and 16 project staff divided into six teams in the first semester of Year 2. A third of the project staff returned for multiple semesters of PiPES curriculum interventions.

An important component for project staff was prior training on rural Appalachia. All PiPES staff are required to participate in yearly face-to-face trainings that offer information about life in these communities. One training included an overview of the cultural traditions and local economy while another included a panel of speakers who worked in rural Appalachian communities. PiPES staff were also assigned required articles on SCCT, school-based interventions in rural Appalachia, and prospective first-generation college students.

Survey development

The program evaluation was designed to help the PiPES team better understand project impact and effectiveness along with systemic, organizational, and individual processes that impacted PiPES success and sustainability. The evaluation followed the Joint Committee on Standards for Educational Evaluation program evaluation standards, a widely used guide for ensuring reliable and valid program evaluation results (Yarbrough, Shula, Hopson, & Caruthers, 2011). For example, external evaluators helped analyze the data to increase trustworthiness, evaluation included both high school students and PiPES staff, and both qualitative and quantitative data were collected. In addition, the evaluators followed the American Evaluation Association’s (2004) guiding principles for evaluators, which promote professional evaluation practices. Data were collected after each round of the intervention, and all evaluations were anonymous to increase honesty of responses. Survey questions were developed jointly between the program directors (the first and second authors) and the external evaluators, all of whom have training on program evaluation strategies. Lastly, all parts of the evaluation connected to the theoretical frame used in the intervention.

Postcourse survey: Students

Upon PiPES course completion, participants answered an evaluation survey, in which they rated their agreement with a set of statements about their satisfaction with the course and the perceived usefulness of the PiPES curriculum using Likert-type scales ranging from strongly disagree to strongly agree. Sample items included “PiPES class activities helped me learn more about myself” and “PiPES helped me think about new options after high school.” Students also rated the degree to which their interests in postsecondary education had changed (“less,” “the same,” or “more”) as a result of the curriculum. A separate qualitative postcourse evaluation was administered to half of the classrooms that received the curriculum. The qualitative survey contained six open-ended questions, such as “What is the most important thing you learned in PiPES?” and “What else do you wish you had been able to learn in PiPES?” Qualitative data provided project staff with more detailed feedback on the course. All data were coded by at least two evaluation team members using both preset and emergent themes. Codes were eventually combined into broad categories. Qualitative results were analyzed through thematic analysis, using an essentialist method to identify the meaning of the experience for participants (Braun & Clarke, 2006). Key themes appeared across participants and were coded in a deductive manner to align with the research questions (Braun & Clarke, 2006). Potential themes were generated, searched for, reviewed, and refined as needed until a final set presented itself. Coding lists were compared as needed for cohesion, and discussions helped address any differences in coding to increase reliability.

Postcourse surveys and blogs: Staff

Project staff who facilitated the PiPES curriculum were also asked to complete a staff evaluation survey upon completion of the course. This survey included 10 questions, 2 quantitative and 8 qualitative, which evaluated staff perceptions and feedback on course curriculum, implementation, perceived impact, and overall reflections. Quantitative questions used a 10-point Likert-type scale ranging from 1 (not at all) to 10 (completely). Sample questions included “How effective was the PiPES curriculum?” and “How engaging was the PiPES curriculum?” Sample qualitative items included “What was your greatest success this semester?” and “What would you do differently next time?” In addition to the postcourse survey, PiPES staff submitted weekly online blog posts after each day of teaching. Within the blog post, staff members summarized the day’s events and offered any suggestions to staff members who would be teaching the curriculum later in the week. All project staff had access to the online blogs, and the blogs were used as evaluation data.