Learning problem solving to manage school-life challenges: The impact on student success in college

Abstract

Colleges and universities have implemented a broad range of initiatives to support student success. Problem solving courses and course supplements are one approach. Evaluation of these courses has shown positive outcomes in terms of improved academic performance and other benefits. A number of these studies have also reported the largest positive effects with underperforming student groups. To further explore this approach a novel general education academic success course was developed. The course integrated a comprehensive problem-solving model into lectures and assignments as the basis of an active learning instructional strategy. Students were taught the model along with relevant academic skills content. They then applied the model to a personal challenge affecting their success in school and life. Using a matched cohort design, 826 course participants were compared with a campus-wide sample matched on key variables. Generalized linear models were used to estimate between group mean differences, and a Cox proportional hazards model was used to compare time to graduation. Results showed that students who successfully completed the course achieved higher cumulative GPAs overall compared with matched peers. Highest GPAs for students who took the course as freshmen suggested a transfer of knowledge over time. Results also showed that the course significantly benefited students from historically at-risk populations in terms of higher GPAs, units earned, retention, and graduation rates. This study shows that a well designed problem solving course can help students, especially those who struggle academically, to more effectively meet the challenges of college and daily life.

Keywords

academic achievement active learning college retention college students problem solving student success

Student success and problem solving

The college journey can be challenging for any student, even more so for some. Underrepresented, lower socioeconomic status, and first-generation students with less educated parents have been shown to have more challenges with academic performance, progress toward degree completion, retention/persistence, and graduation (Bowen et al., 2005; National Student Clearinghouse Research Center, 2020; Pascarella et al., 2004; Yue and Fu, 2017). Across the United States colleges and universities have implemented a broad range of initiatives to improve recruitment, engagement, retention, and timely graduation. The constellation of interventions designed to impact student success range from tutoring individual students to reorganizing practices across the entire institution. Examples include academic skill building and strategy instruction (Gunn et al., 2011); self-regulation training (MacArthur et al., 2015); problem-based learning (Stinson and Milter, 1996); direct assistance and comprehensive support programs, such as improvements in advising and counseling (Castleman and Goodman, 2018) and remedial instruction (Mokher et al., 2020); short-term campus orientation sessions (Kallison and Stader, 2012); comprehensive first-year experience programs (Schrader and Brown, 2008); and campus climate reforms (Vogel et al., 2008). Many of these interventions also focus specifically on recognized risk factors, such as race-ethnicity and socioeconomic class, and the interaction of these factors with characteristics of the specific institution.

Complex problems

The multiplicity of approaches found in the research literature and in actual practice speaks to the dynamic landscape of challenges facing students and their schools. These real-life challenges can be what cognitive scientists call complex problems (Frensch and Funke, 2014). A simple problem, like a basic math problem, is deterministic. If you follow a known procedure you arrive at one correct answer. Complex problems, by comparison, can reflect multiple concurrent challenges; multiple goals that may be unclear or competing; problem elements that are not transparent, possibly interconnected, or that are dynamically changing in time; unpredictable time lags including delayed feedback; contextual factors that may hinder progress; and issues for which there is no single, clear or superior solution (Brehmer, 1995; Dörner and Funke, 2017).

Problem solving

Many college students face complex problems related to personal/cultural identity, family, school, work, and community environments that can impede their ability to succeed. Given the diversity of these challenges, one potentially useful general intervention would be to teach students effective problem solving strategies. A problem is the gap between a current and a desired state (Reid, 2006). Problem solving is the effort to close that gap, and that effort can be ill-informed, misdirected, and ineffective. Effective problem solving involves a systematic goal-directed approach to problem recognition and conceptualization, strategy reflection and selection, implementation, and the evaluation of efforts to find a useful solution (Allen and Graden, 2002; Deming, 2018; Imai, 1986).

Problem solving is one of the top cognitive domain skills in Gagne’s learning hiearchy (Gagne, 1970), clearly relevant to academic success and life mastery. It is also a quality highly valued by employers in the workplace (Hora, 2019). Research suggests, however, that developing skills in critical thinking and problem solving is not spontaneous, and that many college students do not use these skills to solve real-world, complex problems (Arum and Roksa, 2011). Limitations include lack of information and motivation, incorrect application of strategies, action without self-reflection, and other causes, including how students are taught (Yuriev et al., 2017).

Instruction is important for learning problem solving and for transferring that skill across academic domains and into other life pursuits (Halpern, 1998; Helsdingen et al., 2010). Classroom interventions that include or focus on problem solving skill training have been developed and tested. These courses and course supplements are typically found in science, technology, engineering, and math (STEM) programs, to help those student majors problem solve in their specific domains. These courses/interventions are diverse, and may include collaborative in-class problem solving activities, domain specific problem solving skill training (such as how to solve math or chemistry problems), and the inclusion of other content, including study skills or psycho-emotional training. Results generally show favorable changes in terms of increased problem solving abilities, improved academic performance, greater academic self-efficacy, and other positive outcomes (Armbruster et al., 2009; Heppner et al., 1984; Mestre et al., 1993; Montague et al., 2000; Richards and Perri, 1978; Stanich et al., 2018; Stinson and Milter, 1996; Woods et al., 1997). Results also show that these programs are often comparatively more beneficial for underrepresented and struggling students (Beichner et al., 2007; Freeman et al., 2007; Haak et al., 2011; Theobald et al., 2020).

Teaching problem solving as an academic success strategy

A comprehensive meta-analysis found student retention and academic performance to be associated with academic goals, academic-related skills, academic self-efficacy, and achievement motivation (Robbins et al., 2004). Positive interpersonal connections with faculty and other students was incrementally associated with improvements in retention (Robbins et al., 2006). Many of these characteristics, such as goal clarity, academic skills (including self-regulation and self-reflection), and academic self-efficacy, have the potential to be shaped by targeted classroom interventions.

To further explore curricular-based support a novel academic success course was created. This general education social science course integrated a comprehensive problem-solving model into weekly lectures and assignments as the basis of an active learning instructional strategy. While problem solving skills courses generally work with domain-specific problems—medicine, chemistry, physics—this course uniquely asked students to work on a personal challenge that had the potential to impede their academic/life success. A matched cohort study was conducted to evaluate the effectiveness of this instructional approach in supporting academic persistence and success in college students. It was hypothesized that learning and applying a problem-solving strategy to address a personal challenge affecting school/life, in the context of an academic success course, would have a positive impact on cumulative GPA and related measures of academic performance. It was also hypothesized that these benefits would persist over time and that they would be more significant for student groups that historically underperform in higher education.

Methods

Participants

The Academic Success Course (ASC) was a 16-week lower division general education social science elective open to all students on campus. Students who took the class between fall 2010 and fall 2019 were included in the study (17 semesters, approximately 50 students enrolled in one section of the course per semester, total n = 826 ASC students). The study was conducted at a 4-year, public university located in a large metropolitan area. The student sample for the course was an average age of 21 (range 17–63); female 73%; underrepresented minority 33% (Native American, Latinx, Black); Pell eligible 40% (as a proxy for lower socioeconomic status—Carnevale and Van Der Werf, 2017); all class levels—freshman 30%, sophomore 32%, junior 18%, senior 20%; and all colleges on campus. First generation status was 24% ASC and 28% control (not a matching variable).

For comparison a matched sample was selected from the campus undergraduate student population (n = 826). De-identified data for both groups was obtained with the assistance of the university Office of Institutional Research. ASC and campus students were individually matched on the following variables: demographics (age, sex, underrepresented minority status, Pell grant eligibility); pre-university data (high school or transfer college GPA); and academic details (semester the Academic Success Course was taken, mean cumulative grade point average at the beginning of that semester—2.979 ASC and 2.998 control, year in school, and college of declared major). Human subjects approval was obtained for secondary analysis of the data and for in-class pre-post data collection.

Intervention

The intervention was an academic success course built around a comprehensive problem-solving model. Students selected a personal challenge that was affecting their school/life success and applied the model to explore that problem. Four primary domains were covered over the period of 16 weeks: (1) the problem-solving model; (2) academic skills; (3) self-management skills; and (4) life goals, using a text developed for the course (Burke, 2016). The first several weeks focused on learning a novel problem-solving model designed for the course. The model was based on a Kaizen continual improvement orientation. A core Kaizen tenet is that improvement begins with problem recognition (Deming, 2018; Imai, 1986). The model followed a common problem solving path: define problem, determine goal, choose solution, implement (monitor and collect data), evaluate, and redesign as needed. It included instruction and practice in goal setting, mindful awareness (self-acceptance oriented reflective self-monitoring), and continual improvement concepts and strategies (for additional information on the model see Burke, 2016). The students picked a challenge, defined their goal, and implemented an action plan, integrating new content/strategies learned throughout the semester. They worked individually and in small thematically-related groups (similar personal challenges). For each new step there was related lecture content, feedback and coaching.

Measures

Outcome measures included key metrics of long-term academic performance and student success—cumulative GPA, cumulative credits earned, retention/persistence, graduation, and time to graduation. The data source was de-identified student record information obtained from the campus Office of Academic Research (17 semesters). Data was also collected over a period of six semesters in the ASC classes (n = 229) for pre-post comparisons. The purpose was to examine changes in beliefs and use of effective learning strategies. Measures included the 11-item Self-Efficacy for Self-Regulated Learning Scale (SRL Scale) (Zimmerman et al., 1992), and seven course-specific variables (ASC Items). Although the ASC Items were non-validated they had high face value. The intention was to collect preliminary data to explore the impact of key ASC strategies, such as a specific technique called the Return Method that was taught to help students stay on task while studying (Burke and Hassett, 2020). Collectively, these 18 items provided additional evidence of problem solving—greater use of effective academic strategies related to positive changes in GPA and other academic outcomes.

Statistical analysis

A total of 826 matched pairs (1652 individuals) were included in the analytic sample. ASC students were compared with matched peers with respect to most recent/final cumulative GPA, most recent/final cumulative total units earned, retention (graduated or enrolled Fall 2019), graduation, and time to graduation. Analysis was conducted comparing: (1) all of the matched pairs; (2) a subsample consisting of students who received an A-B grade in the Academic Success Course (621 matched pairs); and (3) a subsample who received an A-C grade in the course (711 matched pairs). The reason for analysis by final course grade (A-B and A-C) was based on the presumption that students who did poorly in the course (D-F, n = 115 matched pairs, 14% of the sample) would not possess a good understanding of the course content and would therefore not show a corresponding improvement in their academic outcomes. In addition to comparison of the whole group, separate comparisons were also conducted for specific demographic subgroups: underrepresented minority (URM), Pell eligible, first generation, female, male, STEM majors from the College of Science and Engineering, as well as by class level—freshman, sophomore, junior, and senior. Differences in outcome means between ASC students and matched controls were estimated using generalized linear models with an identity link; generalized estimating equations (GEE) were used to account for correlation within matched pairs.

To explore group differences in time to graduation an event history analysis was conducted using the Cox proportional hazards model of time to graduation stratified by class level—freshman, sophomore, junior, senior (the Breslow method was used to handle ties). The target event in this study was graduation with a bachelor’s degree. Event history modeling techniques of this type are used with longitudinal data to investigate the occurrence and timing of specific events, such as length of time to disease recovery or remission (Singh and Mukhopadhyay, 2011). These methods are now also being used to examine time-related questions of interest in higher education, including time to degree completion and dropout (Berzenski, 2021; Chen, 2012; Chimka et al., 2007; DesJardins et al., 1999; Ishitani and DesJardins, 2002; Yue and Fu, 2017). Robust standard errors were used in the event history models to account for correlation within pairs. Students who did not graduate were censored in Summer 2019, and students who took the class in Fall 2019 and their matched peers were excluded from this analysis. A Kaplan-Meier survival plot depicts the related time-to-graduation results graphically.

Finally, an analysis of pre-post changes in the use of effective learning strategies was performed using paired sample two-tailed t-test comparisons.

Results

Whole sample comparison

Comparing all students who took the Academic Success Course (grades A-F) with their matched controls (N = 826 pairs, 1652 individuals), there were no significant differences between the two groups for GPA (3.02 vs 3.0, p = 0.25) earned units, retention, or graduation rates.

Grade A-B sample comparison

The between group differences for the A-B grade and the A-C grade subsamples were very similar to each other. The results for the A-B grade comparison are presented in the text below and in Table 1. Notable differences for the A-C grade group are indicated in the text below. See Table 1.

Table 1.

Between group comparisons of ACS students and matched peers on key academic outcome measures by demographic subgroup (grade A-B sample).

	N	ASC students		Matched peers		p-value
	N	Mean	SD	Mean	SD	p-value
Cumulative GPA
All	1242	3.21	0.45	3.10	0.52	0.0001
Freshman	354	3.17	0.48	2.96	0.59	0.0001
Sophomore	406	3.26	0.45	3.17	0.50	0.001
Junior	220	3.23	0.42	3.16	0.47	0.024
Senior	262	3.14	0.40	3.10	0.43	0.102
Female	936	3.23	0.45	3.12	0.51	0.0001
Male	304	3.15	0.44	3.02	0.54	<0.0001
URM	384	3.15	0.47	2.99	0.56	0.0001
Pell eligible	494	3.18	0.48	3.06	0.54	0.0001
First generation	338	3.22	0.45	3.05	0.50	<0.0001
STEM majors	228	3.22	0.45	3.07	0.55	<0.0001
Cumulative units
All	1242	114.59	39.25	111.95	37.91	0.065
Freshman	354	91.05	38.81	89.11	42.10	0.568
Sophomore	406	107.38	32.71	108.38	32.24	0.654
Junior	220	126.20	26.02	119.06	24.66	0.004
Senior	262	147.83	31.84	142.40	24.92	0.055
Female	938	113.73	39.18	111.40	37.42	0.149
Male	304	117.26	39.47	113.66	39.44	0.242
URM	384	108.85	39.89	105.57	39.70	0.220
Pell	494	117.26	40.99	112.99	38.97	0.072
First generation	338	119.89	39.17	110.34	36.78	0.008
STEM majors	228	116.24	44.93	111.77	44.73	0.145
Retention
All	1242	0.80	0.40	0.79	0.41	0.649
Freshman	354	0.75	0.43	0.71	0.45	0.418
Sophomore	406	0.83	0.38	0.82	0.38	0.893
Junior	220	0.95	0.23	0.87	0.33	0.055
Senior	262	0.98	0.15	0.87	0.34	0.001
Female	938	0.85	0.35	0.81	0.39	0.068
Male	304	0.88	0.33	0.82	0.39	0.125
URM	384	0.90	0.31	0.79	0.41	0.003
Pell	494	0.87	0.33	0.78	0.42	0.004
First generation	338	0.92	0.27	0.81	0.39	0.002
STEM majors	228	0.90	0.30	0.82	0.39	0.055
Graduation
All	1242	0.58	0.49	0.58	0.49	0.786
Freshman	354	0.45	0.50	0.45	0.50	0.898
Sophomore	406	0.58	0.49	0.59	0.49	0.893
Junior	220	0.68	0.47	0.65	0.48	0.577
Senior	262	0.92	0.27	0.80	0.40	0.001
Female	938	0.63	0.48	0.60	0.49	0.259
Male	304	0.64	0.48	0.61	0.49	0.316
URM	384	0.59	0.49	0.52	0.50	0.059
Pell	494	0.64	0.48	0.57	0.50	0.040
First generation	338	0.67	0.47	0.53	0.50	0.004
STEM majors	228	0.59	0.49	0.53	0.50	0.158

Grade A-B sample comparison by demographic groups

Cumulative GPA: Looking at the entire subset of students who received an A-B grade in the Academic Success Course and their matched controls (621 pairs, 1242 individuals) the ASC students had a higher cumulative GPA (3.21 vs 3.10, p < 0.0001). Both of these would be in the “B” range (3.00–3.29) at the study site university. Examining cumulative GPA by year in college, we observed that the between group differences in GPAs got progressively smaller at each level (freshmen—3.17 vs 2.96, p < 0.0001; sophomores—3.26 vs 3.17, p = 0.001; juniors—3.23 vs 3.16, p = 0.0240; and no significant difference for seniors—3.14 vs 3.10, p = 0.102). ASC students also had significantly higher cumulative GPAs compared with controls for all demographic groups: first generation (e.g. 3.22 vs 3.05, p < 0.0001), URM, Pell eligible, female, male, and STEM majors. The main difference for the A-C grade subsample was that in addition to seniors the junior year student GPAs were not significantly different from the comparison group.

Cumulative units earned: There were only two instances of significant difference between groups. Juniors who took the course earned more units (126.2 vs 119.1, p = 0.004) as did first-generation students (119.9 vs 110.3, p = 0.008). The same two groups earned more units in the A-C comparisons as well.

Retention: Higher rates of retention were observed for seniors (0.98 vs 0.87, p = 0.001), URM (0.90 vs 0.79, p = 0.003), Pell eligible (0.87 vs 0.78, p = 0.004), and first-generation students (0.92 vs 0.81, p = 0.002). More juniors (0.95 vs 0.87) and STEM majors (0.90 vs 0.82) who took the course were also still in school, but the differences did not reach statistical significance (p = 0.055 for both). For the A-C comparison group the only notable difference was for “All” (n = 621), in which ASC students were more likely to still be in school (retention: 0.84 vs 0.80, p = 0.043).

Graduation: There were three instances of significant differences between groups. Seniors who took the course were more likely to have graduated (0.92 vs 0.80, p = 0.001) as well as Pell eligible (0.64 vs 0.57, p = 0.04) and first-generation students (0.67 vs 0.53, p = 0.004). The A-C comparison was similar, except that there was no difference for the Pell subgroup.

Time to graduation: The Cox proportional hazards model was used to compare the time it took participants to reach the important academic milestone of graduation (following completion of the Academic Success Course). The starting time for each participant was the semester they were enrolled in the course (the same semester was used for the matched controls). There were no significant differences between the two groups overall. However, the Cox proportional hazards model of time to graduation stratified by class level showed that the ASC students who took the class in their senior year tended to graduate more quickly than their matched peers (HR = 1.40, 95% CI 1.13–1.73, p = 0.002). Faster time to graduation was also found for URM students (HR = 1.30, 95% CI 1.03–1.65, p = 0.03) and for first-generation students (HR = 1.33, 95% CI .10.04–1.71, p = 0.026). See Table 2.

Table 2.

Cox proportional hazards ratios and 95% confidence intervals for graduation within 5 years of taking the Academic Success Course, comparing percent graduated ASC students and matched peers (grade A-B sample).

Hazard ratios	N	%ACS	SE	%Peer	SE	HR	95% CI		p-value
A-B sample	1166	82.8	0.019	78.5	0.020	1.096	0.967	1.243	0.151
Freshman	344	67.3	0.046	67.8	0.045	1.185	0.876	1.603	0.271
Sophomore	374	81.8	0.034	80.6	0.034	0.909	0.735	1.125	0.383
Junior	198	89.8	0.036	84.5	0.042	0.942	0.693	1.280	0.701
Senior	250	98.1	0.013	85.3	0.033	1.395	1.126	1.729	0.002
Female	884	82.3	0.022	79.2	0.023	1.087	0.940	1.258	0.261
Male	282	84.9	0.036	76.5	0.041	1.079	0.840	1.386	0.554
URM	348	85.1	0.034	71.8	0.040	1.302	1.026	1.650	0.030
Pell eligible	464	83.7	0.029	73.6	0.034	1.195	0.973	1.468	0.090
First generation	300	90.4	0.031	74.6	0.043	1.332	1.036	1.713	0.026
STEM majors	204	86.3	0.046	75.7	0.053	1.126	0.838	1.512	0.432

The Kaplan-Meier curve depicts the time to graduation trend for the two groups (Figure 1). Comparing the two plot lines, the ASC students had a lower survival rate, that is, they were generally graduating sooner (leaving the study sooner) than their matched controls throughout the 10-year study period. The proportion from each group that graduated within 5 years of the class was 83% (SE 2%) in the ASC group and 79% (SE 2%) in the control group (not a statistically significant overall difference, as noted above).

Figure 1.

Kaplan-Meier plot of time from taking the Academic Success Course to graduation, comparing ASC students and matched campus controls. The period of observation is Fall 2010 to Fall 2019, 105 months.

Use of effective learning strategies

There were significant pre-post changes related to beliefs and use of productive learning strategies (n = 229). This included significant changes in reported academic self-efficacy beliefs (p = 0.007), measured using the Self-Efficacy for Self-Regulated Learning Scale (Zimmerman et al., 1992). Similarly, the 7 items created to examine specific aspects of the course all showed significant pre to post changes, such as using study strategies (p < 0.001) and staying on task while studying (p = 0.007). See Table 3.

Table 3.

Self-efficacy and use of productive learning strategies, ASC within-group pre-post mean comparisons for the SRL Scale and ASC items.

	Pre	Post	Difference	SD	t	df	p-value
SRL scale* (n = 229)
	36.34	37.44	–1.10	5.80	–2.74	207	0.007
ASC items** (n = 229)
Seek help	2.55	2.86	–0.314	1.21	–3.92	228	0.000
Solve problems	3.84	3.99	–0.149	0.91	–2.47	228	0.014
Use strategies	3.25	3.73	–0.482	1.16	–6.22	223	0.000
Stay on task	3.46	3.66	–0.194	1.08	–2.71	226	0.007
Accept mistakes	3.59	3.76	–0.167	1.15	–2.20	226	0.029
Regulate emotions	3.39	3.59	–0.199	1.07	–2.81	225	0.005
Schedule tasks	3.85	4.12	–0.254	1.22	–3.14	227	0.002

11-item Self-Efficacy for Self-Regulated Learning Scale (Zimmerman et al., 1992).

7 ASC course-specific variables.

Discussion

Cumulative GPA and knowledge transfer

ASC and campus control students were initially matched on cumulative GPA (at the beginning of the semester in which the ASC students took the course, ASC GPA = 2.979, matched controls GPA = 2.998). College GPA is a recognized proxy of cognitive development and academic achievement (Nettles et al., 1986). It has been shown to have a large direct effect on college retention, graduation, and time to graduation, as well as being associated with positive outcomes in careers and health (Allen et al., 2008; Berzenski, 2021; Hasl et al., 2019; Yue and Fu, 2017).

Improved academic performance has been found to be associated with active student engagement in courses (Carini et al., 2006; Lee, 2014). As expected, the analysis of the whole sample, which included students receiving a D-F grade in the course (lower engagement), showed no significant differences between ASC students and matched peers on cumulative GPA or other academic performance measures. Looking at the A-C and A-B subsamples, however, the cumulative GPAs for both groups improved and were significantly higher than their matched peers. For the A-B grade sample specifically, the largest cumulative GPA difference between groups was for students who took the course as freshmen (3.17 vs 2.96), followed by sophomores (3.26 vs 3.17), then juniors (3.23 vs 3.16). There was no significant difference between groups for students who took the course as seniors.

This outcome may reflect a transfer of knowledge, temporally and contextually. Students learned important academic/life skills in the course as suggested by changes in cumulative GPA and supported by pre-post changes in measures of self-regulated learning. They then used the ideas and skills acquired in the course across subsequent semesters in new classes, incrementally improving their performance. Students with the most time to practice and refine their approach—freshman—exhibited the most benefit. Transfer of knowledge does not happen automatically (Billing, 2007; Haskell, 2000). Certain instructional practices can help support knowledge retention and transfer, making what is learned more generalizable and useful. Studies suggest that these practices include active learning assignments, application of knowledge to real-world problems, self-discovery, and providing an understanding of underlying principles (Billing, 2007; Haskell, 2000; Weinstein et al., 2000; Wieman and Perkins, 2005). All of these practices were integral to the Academic Success Course. Perhaps one other reason for this apparent knowledge transfer is that unlike domain specific problem solving instruction—how to solve math or chemistry problems—this course focused on resolving personal challenges, a focus that might generalize to life more directly and thereby support a broader transfer of content and skills.

STEM majors from the College of Science and Engineering who took the Academic Success Course earned significantly higher cumulative GPAs compared with matched peers in both the A-B and A-C cohorts. A number of studies have found GPA to be strongly associated with persistence in STEM majors (Dika and D’Amico, 2016; Hachey et al., 2015). There were no significant differences between STEM ASC students and controls for cumulative units earned, retention, or graduation.

A potential benefit of an increasing GPA over time is that incremental successes, or enactive mastery experiences, lead to increases in self-efficacy (Bandura, 1997), and academic self-efficacy is highly associated with greater academic success. In a review of over 100 college outcome studies, academic self-efficacy and achievement motivation were found to be the best predictors of cumulative GPA (Robbins et al., 2004).

Retention, units earned, and graduation for at-risk students

Cumulative GPA: Research has found that teaching self-regulation and active learning strategies can benefit all students, but that it is comparatively more beneficial for URM, female, and struggling students (Beichner et al., 2007; Freeman et al., 2007; Haak et al., 2011; Prince, 2004). Similar results were found in this study. Compared with matched peers, URM, Pell, and first-generation ASC students ended up attaining significantly higher cumulative GPAs.

Cumulative units earned: Cumulative units earned reflect both academic performance and efficient progress toward degree completion. This outcome measure was superior for ASC first-generation students and juniors compared to controls, but there was no difference for URM or Pell students. For juniors the difference was seven more semester units, approximately two courses. In the case of first-generation students it was 10 more units or three additional courses. A survey of first generation students found that being able to complete coursework more quickly was an important factor in the decision to attend college, along with the availability of financial aid and the ability to work while in school (Nunez and Cuccaro-Alamin, 1998).

Retention: In terms of academic retention, freshman and sophomore year is generally the period of highest student dropout (Kirp, 2019; National Student Clearinghouse Research Center, 2020). In the overall sample there were no significant differences in retention between ASC and controls for students who took the course in their freshman or sophomore year. However, URM, Pell, and first-generation students (all class levels), as well as seniors, were more likely to still be in school (or to have graduated). For these students, this was an approximately 10% point higher retention rate compared to matched controls. In a longitudinal analysis of 12,096 college students, Yue and Fu (2017) noted that first-generation students appeared to be less likely to graduate. When the authors controlled for academic performance, differences in their graduation rates became insignificant. If a student’s academic performance can be improved, so potentially can the opportunity for successful completion.

Graduation: Among the ASC students the graduation rates were higher for Pell eligible and first-generation students, as well as for students who took the course during their senior year. In each of these groups the difference in graduation rates between ASC and matched peers was approximately 12% points. The higher graduation rate for seniors also speaks to the importance of courses like this for all college students, not just freshmen and sophomores.

Time to graduation

Previous studies have shown that certain demographic factors, including being minority, first-generation, low-income, or male are associated with slower progress toward degree completion (Bound et al., 2012; Ishitani, 2003). In this case, however, URM and first-generation students were significantly more likely to have completed their degrees, an approximately 13 and 16 percentage point advantage respectively. A 13 percentage point difference in graduation rates was also observed for students who took the course as seniors. Faster time to graduation can free seats and resources for new students, reduce institutional costs per graduate, and have a positive impact on state support tied to improved graduation rates (Hauptman and Merisotis, 1990).

Use of effective learning strategies

Results from the 11-item SRL Scale showed significant changes pre to post in self-efficary toward the use of effective learning strategies. Self-regulated learning strategies have been shown to help students become more motivationally, metacognitively and behaviorally engaged in their own learning (Zimmerman, 1986). Similarly, the 7 ASC items selected to examine specific aspects of the course all showed significant pre-post changes, such as regulating emotions, and seeing oneself as a problem solver. Taken together, these results suggest that the changes in outcomes observed in this study, such as improved GPA and retention rates, may be related to a more deliberate use of effective academic strategies and self-management skills, improved academic self-efficacy, and greater goal clarity.

Personal challenge topics

Finally, ASC students learned the course problem-solving model by selecting and working on a personal challenge that was affecting school/life. Reviewing over 200 final papers from the course, the primary themes students selected were: emotional literacy and mental health (41%); time management and organization (29%); wellness and health practices (24%); and other topics, which included a few papers on study skills (6%). It was rare to have a student focus on specific academic skills for this project. This finding speaks to the diverse needs of contemporary students and the personal histories and challenges they bring with them to campus.

Study limitations

The intervention group was comprised of a self-selected sample. A large review of data from 49 universities examined the relationships between self-selection and first year experience program outcomes. The review noted that despite self-selection clear benefits were observed for Black students related to improved GPA and college satisfaction (Culver and Bowman, 2020). The ASC intervention also combined problem solving, academic/life skills training, and collaborative peer learning, making it harder to specify which element or combinations had the greatest impact. That said, one investigation comparing study skills instruction with and without problem solving training found that the positive effects on academic performance did not endure for the study skills only group (Richards and Perri, 1978). As this was an exploratory study we examined the impact of the intervention on different subgroups resulting in multiple comparisons and an increased risk for Type I errors. Finally, future research should include collecting data with other student populations.

Conclusion

The goal of this study was to evaluate the effectiveness of a novel instructional approach designed to support academic persistence and success in college students. It was found that teaching students a comprehensive problem-solving strategy, one that actively engaged them in navigating a personal challenge, was associated with positive changes in academic effectiveness and efficiency.

Given the number of hours students spend in the classroom, effective pedagogical interventions can make an important contribution toward enhancing student success. The right strategy can empower beliefs, motivation, skills and habits, and lay a foundation for success in school and life. Such interventions may also be of particular value to low income, minority, and first generation students who often struggle academically, ultimately contributing to greater equity in education and society. Strategic instruction in problem solving may be one key for unlocking academic potential by helping students recognize their ability to meet the challenges of daily life with greater curiosity, creativity and commitment.

Footnotes

Acknowledgements

The authors thank Emily Shindledecker and the office of Institutional Research at San Francisco State University for their assistance with data acquisition. We are also grateful to Darryl Dieter, Office of Research and Planning, City College of San Francisco, and Hongtao Yue, Office of Institutional Effectiveness at California State University Fresno, for their helpful feedback on data collection and manuscript review.

Author contributions

AB: Project design, data collection and interpretation, and manuscript preparation. SS: Data analysis and interpretation, and manuscript preparation.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethics approval and consent to participate

Human subjects approval was obtained for secondary analysis of the de-identified campus data and for in-class pre-post data collection in accordance with the university Institutional Review Board policies.

ORCID iD

Adam Burke

Author biographies

Adam Burke is a professor in the College of Health and Social Sciences, San Francisco State University. Research interests include student well-being, metacognitive processes, learner agency, achievement and equity. He was a 2019 recipient of the California State University Faculty Innovation and Leadership Award for his work on student success.

Susan Stewart is a professor of biostatistics at the University of California. Research interests include measurement of health behaviors, risk factors, and outcomes in diverse populations related to cancer prevention, tobacco and drug use, and quality of life. Address: Public Health Sciences, UC Davis, Davis, CA 95616, USA. [ email: slstewart@ucdavis.edu]

References

Allen

Robbins

Casillas

, et al. (2008) Third-year college retention and transfer: Effects of academic performance, motivation, and social connectedness. Research in Higher Education 49: 647–64.

Allen

Graden

(2002) Best practices in collaborative problem solving for intervention design. In: Thomas

Grimes

(eds) Best Practices in School Psychology IV. Washington, DC: National Association of School Psychologists, pp.565–82.

Armbruster

Patel

Johnson

, et al. (2009) Active learning and student-centered pedagogy improve student attitudes and performance in introductory biology. CBE Life Science Education 8: 203–13.

Arum

Roksa

(2011) Academically Adrift: Limited Learning on College Campuses. Chicago, IL: University of Chicago Press.

Bandura

(1997) Self-Efficacy: The Exercise of Control. New York, NY: Freeman.

Beichner

Saul

Abbott

, et al. (2007) The student-centered activities for large enrollment undergraduate programs (SCALE-UP) project. Research-Based Reform of University Physics 1(1): 2–39.

Berzenski

(2021) The when and who of graduation and dropout predictors: A moderated hazard analysis. Journal of College Student Retention Research Theory and Practice 23: 768–92.

Billing

(2007) Teaching for transfer of core/key skills in higher education: Cognitive skills. Higher Education 53(4): 483–516.

Bound

Lovenheim

Turner

(2012) Increasing time to baccalaureate degree in the United States. Education Finance and Policy 7(4): 375–424.

10.

Bowen

Kurzweil

Tobin

(2005) Equity and Excellence in American Higher Education. Charlottesville, VA: University of Virginia Press.

11.

Brehmer

(1995) Feedback delays in complex dynamic decision tasks. In: Frensch

Funke

(eds) Complex Problem Solving: The European Perspective 103–30. Hillsdale, NJ: Lawrence Erlbaum Associates.

12.

Burke

(2016) Learning Life. San Francisco, CA: Rainor Media.

13.

Burke

Hassett

(2020) Evaluating an instructional resource used for teaching and learning meditation: A pilot study. Journal of Cognitive Enhancement 4(4): 412–21.

14.

Carini

Kuh

Klein

(2006) Student engagement and student learning: Testing the linkages. Research in Higher Education 47(1): 1–32.

15.

Carnevale

Van Der Werf

(2017) The 20% Solution: Selective Colleges Can Afford to Admit More Pell Grant Recipients. Washington, DC: Georgetown University Center on Education and the Workforce. Available at: https://cew.georgetown.edu/cew-reports/pell20 (accessed 11 October 2021).

16.

Castleman

Goodman

(2018) Intensive college counseling and the enrollment and persistence of low-income students. Education Finance and Policy 13(1): 19–41.

17.

Chen

(2012) Institutional characteristics and college student dropout risks: A multilevel event history analysis. Research in Higher Education 53(5): 487–505.

18.

Chimka

Reed-Rhoads

Barker

(2007) Proportional hazards models of graduation. Journal of College Student Retention Research Theory and Practice 9(2): 221–32.

19.

Culver

Bowman

(2020) Is what glitters really gold? A quasi-experimental study of first-year seminars and college student success. Research in Higher Education 61(2): 167–96.

20.

Deming

(2018) The New Economics for Industry, Government, Education. Cambridge: MIT Press.

21.

DesJardins

Ahlburg

McCall

(1999) An event history model of student departure. Economics of Education Review 18(3): 375–90.

22.

Dika

D’Amico

(2016) Early experiences and integration in the persistence of first-generation college students in STEM and non-STEM majors. Journal of Research in Science Teaching 53(3): 368–83.

23.

Dörner

Funke

(2017) Complex problem solving: what it is and what it is not. Frontiers in Psychology 8: 1153.

24.

Freeman

O’Connor

Parks

, et al. (2007) Prescribed active learning increases performance in introductory biology. CBE—Life Sciences Education 6(2): 132–9.

25.

Frensch

Funke

(2014) Complex Problem Solving: The European perspective. Hove, UK: Psychology Press.

26.

Gagne

(1970) The Conditions of Learning. New York, NY: Holt, Rinehart and Winston.

27.

Gunn

Hearne

Sibthorpe

(2011) Right from the start: A rationale for embedding academic literacy skills in university courses. Journal of University Teaching and Learning Practice 8(1): 70–80.

28.

Haak

HilleRisLambers

Pitre

, et al. (2011) Increased structure and active learning reduce the achievement gap in introductory biology. Science 332(6034): 1213–6.

29.

Hachey

Wladis

Conway

(2015) Prior online course experience and GPA as predictors of subsequent online STEM course outcomes. The Internet and Higher Education 25: 11–7.

30.

Halpern

(1998) Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American Psychologist 53(4): 449–55.

31.

Haskell

(2000) Transfer of Learning: Cognition and Instruction. New York, NY: Elsevier.

32.

Hasl

Kretschmann

Richter

, et al. (2019) Investigating core assumptions of the “American Dream”: Historical changes in how adolescents’ socioeconomic status, IQ, and GPA are related to key life outcomes in adulthood. Psychology and Aging 34(8): 1055–76.

33.

Hauptman

Merisotis

(1990) The College Tuition Spiral: An Examination of Why Charges Are Increasing. New York, NY: Macmillan Publishing.

34.

Helsdingen

van Den Bosch

van Gog

, et al. (2010) The effects of critical thinking instruction on training complex decision making. Human Factors 52(4): 537–45.

35.

Heppner

Neal

Larson

(1984) Problem-solving training as prevention with college students. The Personnel and Guidance Journal 62(9): 514–9.

36.

Hora

(2019) Beyond the Skills Gap: Preparing College Students for Life and Work. Cambridge, MA: Harvard Education Press.

37.

Imai

(1986) Kaizen: The Key to Japan’s Competitive Success. New York, NY: McGraw-Hill.

38.

Ishitani

(2003) A longitudinal approach to assessing attrition behavior among first-generation students: Time-varying effects of pre-college characteristics. Research in Higher Education 44(4): 433–49.

39.

Ishitani

DesJardins

(2002) A longitudinal investigation of dropout from college in the United States. Journal of College Student Retention Research Theory and Practice 4(2): 173–201.

40.

Kallison

Jr Stader

(2012) Effectiveness of summer bridge programs in enhancing college readiness. Community College Journal of Research & Practice 36(5): 340–57.

41.

Kirp

(2019) The College Dropout Scandal. New York, NY: Oxford University Press.

42.

Lee

(2014) The relationship between student engagement and academic performance: Is it a myth or reality? Educational Research Journal 107(3): 177–85.

43.

MacArthur

Philippakos

Ianetta

(2015) Self-regulated strategy instruction in college developmental writing. Journal of Education & Psychology 107(3): 855–67.

44.

Mestre

Dufresne

Gerace

, et al. (1993) Promoting skilled problem-solving behavior among beginning physics students. Journal of Research in Science Teaching 30(3): 303–17.

45.

Mokher

Park-Gaghan

Spencer

, et al. (2020) Institutional transformation reflected: Engagement in sensemaking and organizational learning in Florida’s developmental education reform. Innovative Higher Education 45(1): 1–17.

46.

Montague

Warger

Morgan

(2000) Solve it! Strategy instruction to improve mathematical problem solving. Learning Disabilities Research and Practice 15(2): 110–6.

47.

National Student Clearinghouse Research Center (2020) First-year persistence and retention 2018 beginning cohort. Available at: https://nscresearchcenter.org/wp-content/uploads/PersistenceRetention2020.pdf (accessed 24 October 2021).

48.

Nettles

Thoeny

Gosman

(1986) Comparative and predictive analyses of Black and White students’ college achievement and experiences. The Journal of Higher Education 57(3): 289–318.

49.

Nunez

Cuccaro-Alamin

(1998) First-Generation Students: Undergraduates Whose Parents Never Enrolled in Postsecondary Education. Washington, DC: U.S. Dept. of Education, Office of Educational Research and Improvement, National Center for Education Statistics.

50.

Pascarella

Pierson

Wolniak

, et al. (2004) First-generation college students: Additional evidence on college experiences and outcomes. The Journal of Higher Education 75(3): 249–84.

51.

Prince

(2004) Does active learning work? A review of the research. Journal of Engineering Education 93(3): 223–31.

52.

Reid

(2006) Productivity and quality improvement: An implementation framework. International Journal of Productivity and Quality Management 1(1-2): 26–36.

53.

Richards

Perri

(1978) Do self-control treatments last? An evaluation of behavioral problem solving and faded counselor contact as treatment maintenance strategies. Journal of Counseling Psychology 25: 376–83.

54.

Robbins

Allen

Casillas

, et al. (2006) Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Education & Psychology 98(3): 598–616.

55.

Robbins

Lauver

, et al. (2004) Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin 130: 261–88.

56.

Schrader

Brown

(2008) Evaluating the first year experience: Students' knowledge, attitudes, and behaviors. Journal of Advanced Academics 19(2): 310–43.

57.

Singh

Mukhopadhyay

(2011) Survival analysis in clinical trials: Basics and must know areas. Perspectives in Clinical Research 2(4): 145–8.

58.

Stanich

Pelch

Theobald

, et al. (2018) A new approach to supplementary instruction narrows achievement and affect gaps for underrepresented minorities, first-generation students, and women. Chemistry Education Research and Practice 19(3): 846–66.

59.

Stinson

Milter

(1996) Problem-based learning in business education: Curriculum design and implementation issues. New Directions for Teaching and Learning 1996(68): 33–42.

60.

Theobald

Hill

Tran

, et al. (2020) Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences 117(12): 6476–83.

61.

Vogel

Holt

Sligar

, et al. (2008) Assessment of campus climate to enhance student success. Journal of Postsecondary Education and Disability 21(1): 15–31.

62.

Weinstein

Husman

Dierking

(2000) Self-regulation interventions with a focus on learning strategies. In: Boekaerts

Pintrich

Zeidner

(eds) Handbook of Self-Regulation. Cambridge, MA: Academic Press, pp.727–47.

63.

Wieman

Perkins

(2005) Transforming physics education. Physics Today 58(11): 36–41.

64.

Woods

Hrymak

Marshall

, et al. (1997) Developing problem solving skills: The McMaster problem solving program. Journal of Engineering Education 86(2): 75–91.

65.

Yue

(2017) Rethinking graduation and time to degree: A fresh perspective. Research in Higher Education 58(2): 184–213.

66.

Yuriev

Naidu

Schembri

, et al. (2017) Scaffolding the development of problem-solving skills in chemistry: guiding novice students out of dead ends and false starts. Chemistry Education Research and Practice 18(3): 486–504.

67.

Zimmerman

(1986) Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology 11(4): 307–13.

68.

Zimmerman

Bandura

Martinez-Pons

(1992) Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal 29: 663–76.