Effects of Research-Related Activities on Graduation at a Hispanic Serving Institution

Abstract

This paper reports on the results of a study of 6,654 unique students on the type of research-related activities (e.g., undergraduate research and internships) they participated in while at California State Polytechnic University, Pomona (Cal Poly Pomona). Results indicate that the odds of graduating for students who participated in research-related activities were almost twice those of students who did not participate in research-related activities. These results differ from and complement studies on the impact of undergraduate research at liberal arts colleges and research-intensive universities. Study results indicate that non-first-generation students, non-low-income students, and non-underrepresented minority (non-URM) students were more likely to participate in research. Participation in internships with industry and with a professional were most predictive of graduation. Students who participated in multiple research-related activities were also more likely to graduate than those who participated in fewer activities; results indicate research participation is equally beneficial across groups with different demographic characteristics including major, sex, first generation and URM status.

Keywords

undergraduate research internships graduation underrepresented minority low income first generation Hispanic serving institution

Introduction

There is a growing consensus in higher education of the benefits of undergraduate research experiences for student persistence and completion. As early as the late 1990s, universities were challenged to make research-based learning part of the curriculum with the goals of resolving attrition issues and increasing the number of graduated students (Boyer Commission on Educating Undergraduates in the Research Laboratory, 1998). Recognizing the value of research-based learning for student success, studies have aimed to evaluate programs that have undergraduate research as a cornerstone activity for persistence and graduation (Fechheimer et al., 2011; Jones et al., 2010; Kuh, 2008; Lopato, 2010). Participation in undergraduate research is associated with persistence because it has been found to increase students’ academic involvement, engagement, sense of belonging, research identity, skill acquisition, and professional preparedness (e.g., Bowman and Holmes, 2018; BrckaLorenz et al., 2017; Collins et al., 2017; Kendricks and Arment, 2011; Kuh, 2008; Linn et al., 2015; Lopato, 2010; Rodenbusch et al., 2016).

Research suggests that student engagement and participation in educationally purposeful activities differs based on student characteristics (e.g., Hu and Kuh, 2002) and the benefits of participation differ based on student characteristics. Some research suggests that the benefits of participation in undergraduate research experiences may be even more pronounced for traditionally underrepresented minority (URM) students (e.g., Chang et al., 2014; Jones et al., 2010; Linn et al., 2015). For example, Chang et al. (2014) found that while URM students were less likely to persist in their science, technology, engineering, and mathematics (STEM) major than their non-URM counterparts, pre-college factors (e.g., SAT score) and college experiences moderated this effect. Participation in undergraduate research was one college experience that was found to make a significant positive difference on URM STEM degree attainment; URM students who participated in undergraduate research were 17.4 percentage points more likely to persist in STEM than those who did not participate in research-related experiences. This finding points to undergraduate research as a pedagogical activity that is associated with student success and achievement with the potential to close the achievement/equity gap.

Acknowledging the potential positive impact of undergraduate research for all students, studies have examined the effect of undergraduate research on college completion across institutions and across disciplines. Collins et al. (2017) conducted one of the first studies of the effects of undergraduate research specifically, across all disciplines, at a Hispanic-Serving Institution (HSI). They found that participation in undergraduate research was associated with perceptions of greater institutional support, greater gains in knowledge and skills, higher grade point average (GPA), more meaningful interaction with faculty, and higher satisfaction with overall educational experience. They also found that the positive effect of participation in undergraduate research on two of the five outcomes (perceptions of support and knowledge/skills) was significantly greater among non-Hispanic White students than among Hispanic students at the HSI. The researchers noted that this latter finding contradicted previous literature, which was established based on research conducted at non-HSIs; more research is needed to explore benefits of participation in undergraduate research in the context of an HSI. This study expands the exploration of the impact of undergraduate research at an HSI by contributing to how such participation impacts students’ graduation rates.

Current Study

The current study extends the research by Collins et al. (2017) to examine an additional student outcome of interest, college completion. Specifically, this study examines the effect of undergraduate research-related activities across disciplines on student graduation in the context of a large, urban, public HSI. This study examines the effects of participation in research-related activities for students campus-wide rather than those in a specific research training program. Given that the conceptualization of undergraduate research and nature of activities that researchers undertake might vary across disciplines, this study expands the definition of undergraduate research beyond research supervised by a faculty member, to include participation in research or research-related activities (e.g., internships, other activities that prepare students to be a professional in their discipline). These research-related activities can provide students with career-initiating experiences in which they can gain skills and apply disciplinary knowledge. Research indicates that participation in internships has benefits for employment outcomes such as enhanced employment opportunities (Adebakin et al., 2015; Callanan & Benzing, 2004), higher job satisfaction (Gault et al., 2000), improved job skills (Gault et al., 2010), and faster promotion rates (D’Abate, 2010). However, to our knowledge there are few studies that examine the benefits of internships for student academic outcomes such as graduation; the current study assesses this relationship and adds to the literature base.

Student engagement is a key predictor of student learning and development during college (e.g., Kuh, 2001; Kuh, 2003). Student engagement is conceptualized as the time and energy students invest in educationally purposeful activities (e.g., research, internships) and the effort institutions commit to offer high quality activities and encourage students to participate in these activities (e.g., Kuh et al., 2010). Theoretically, as students become more involved in educationally purposeful activities (e.g., invest time and effort, practice skills, receive feedback), they are more likely to benefit from these experiences, become integrated into campus life, and persist through degree completion (e.g., Astin, 1984; Kuh, 2009). This idea - the more you put in, the more you get out - suggests that participation in undergraduate research early on and for an extended period may have a positive influence on student outcomes. Consistent with this idea, Jones et al. (2010) found that undergraduate research experiences positively impacted student persistence and had greater effects on students who participated in undergraduate research early on in their college careers. Adedokun and colleagues (2014) found that students reported gains in fewer areas (e.g., research skills, understanding of research process) at the end of a summer research experience than they reported at the end of their yearlong research experience. Furthermore, Gilmore et al. (2015) found that the duration or amount of time that students engaged in undergraduate research was positively correlated with their subsequent performance in graduate school, as measured by the quality of their research proposal. Taken together, the findings from these studies suggest that students are better able to develop and hone their expertise through practicing their research skills over time. In addition to duration of research experience (depth), the current study assesses whether the number of research-related activities (breadth) effects student graduation rates. It is expected that the more opportunities students have to practice research skills, gain disciplinary knowledge, and build relationships with faculty mentors and other professionals in their field of study, the more likely they are to persist and graduate.

Background/Motivation for Study

There is an increased demand for accountability in higher education. The California State University system has responded to that demand through its Graduate Initiative 2025 (GI, 2025). GI 2025 proposes to increase graduation rates for all California State University system students across its 23 campuses while eliminating opportunity and achievement/equity gaps. The California State University system has identified several key factors that impact student's degree completion and the length of time it takes students to graduate. Investment in the initiative comes with the focus on addressing those factors: (1) Academic Preparation, (2) Enrollment Management, (3) Student Engagement and Well-Being, (4) Financial Aid, (5) Data-Informed Decision Making, and (6) Administrative Barriers. In the 2019–2020 academic year, $75 million was set aside to support progress towards the 2025 goals (Graduation Initiative, 2025, p. 2016). Consequently, Cal Poly Pomona is one of the 23 campuses in the California State University system that have prioritized improving graduation rates for all students in the coming years.

Cal Poly Pomona is located in Southern California, in northeast Los Angeles county. One of only six polytechnic universities nationwide, Cal Poly Pomona is a comprehensive public university widely recognized for its polytechnic mission and learn-by-doing philosophy. In fall 2020, the campus had 29,704 students enrolled, including 93% of whom were undergraduate students. As a state university located in a multi-cultural urban metropolis, Cal Poly Pomona provides access to higher education for a large number of Hispanic and low-income students. Cal Poly Pomona is designated by the U.S. Department of Education as an HSI with Hispanic students accounting for 49% (14,493) of the student body. Eighty-five percent (25,304) of its student population identify themselves as non-White. Approximately 55% of the student population are first-generation college students and 46% are considered financially needy/low income by federal standards (Pell-eligible).

Cal Poly Pomona's long-term plan to achieve their individualized GI 2025 goals recognizes several fundamental areas of development and expansion based on the key factors identified by the California State University (CSU) system, including undergraduate research. The Office of Undergraduate Research (OUR) was identified in Cal Poly Pomona's individualized plan as a targeted support service for student engagement in undergraduate research. Cal Poly Pomona looks to leverage the OUR's student connection as a high impact practice (HIP) to potentially increase graduation rates and narrow the gap in graduation rates.

The purpose of this study is to determine whether undergraduate research, and related activities, have a positive impact on graduation rates, at a large, public, urban HSI. This study complements other studies by providing possible impact of undergraduate research-related activities on college completion at a large, public university with a high percentage of Hispanic students as well as first-generation and low-income students. This study includes an analysis of both undergraduate research, specifically defined as faculty-mentored research, and other research-related activities (e.g., internships, other activities that train students to be a professional in their discipline) outside of the classroom. Finally, this study contributes to our understanding of how research and research-related activities contribute to longer-term outcomes by including survey data across 4 years with multiple cohorts.

Three primary research questions guided our inquiry:

Does participation in research-related activities affect student graduation rates?

Does the amount of research-related participation affect student graduation rates?

Does the type of research-related participation affect student graduation rates?

Materials and Methods

Procedure

A multi-year survey design was used to answer the research questions. A voluntary response sample of Cal Poly Pomona students was used to construct the sample over 4 years: 2015, 2016, 2017, and 2018. The OUR in collaboration with Cobblestone Applied Research & Evaluation, Inc. developed a Research Climate Survey to establish a baseline of Cal Poly Pomona student participation in research-related activities. The Research Climate Survey was administered in fall of each year (2015–2018). Students were initially contacted via their Cal Poly Pomona email, which was sent directly to all Cal Poly Pomona students. The survey was open for three to four weeks and a follow-up reminder was sent via email one week before the survey was closed. In addition, posters were created and displayed at multiple locations on campus. Finally, a web banner was used to advertise completion of the survey directly on the Cal Poly Pomona website. An incentive was offered to participants which included entry in a drawing of $100 gift card for each of the 4 years.

Participants first read an informed consent form, at which point they determined their willingness to participate. Participants who consented to participate completed the Research Climate Survey (IRB #20-133). Participants were asked to provide their student identification (ID) as part of the survey. Student ID information was provided by the research team to the Cal Poly Pomona Office of Institutional Research, Planning and Analytics (IRPA) who matched student IDs to academic and demographic variables.

Research Participation. Research participation was measured via student self-report. Participants responded to a question that asked about their participation in undergraduate research or research-related tasks during their time at Cal Poly Pomona (2015 survey administration) or during the previous academic year (2016 survey administration and onward), excluding those experiences that were part of an undergraduate course. Students could select all of the following response options that applied:

Internship with professional in their academic field of interest

Training under professional in their academic field of interest

Internship with industry

Research or project with a Cal Poly Pomona faculty member

Research or project outside of Cal Poly Pomona

Participation in other activities that trained them to be a professional in their discipline (e.g., relevant off-campus employment, community service)

○ Students were asked to provide an example (open-ended)

None

Participants also responded to an open-ended question about how they first learned about research opportunities at Cal Poly Pomona (included on the 2017 and 2018 survey administrations).

Graduation. Individual student graduation was determined on the basis of whether they had graduated from the institution as of Spring 2019. We also took into account whether they should have graduated based on the length of their tenure at the institution. Specifically, first-time freshman students were expected to have graduated within 6 years, and transfer students were expected to have graduated within 4 years.

Academic and Demographic Characteristics. The Cal Poly Pomona Office of IRPA provided academic and demographic information for participating students. These data were used as provided or used to calculate additional variables. The final list of variables includes:

Time to graduate (in number of years)

Academic college (Agriculture; Business Administration; Education; Engineering; Environmental Design; Hospitality Management; Letters & Arts; Science)

Sex (male or female)

Low-income status [Pell award] (yes or no)

Underrepresented minority status (yes or no)¹

First-generation status (first college degree, first Bachelor's degree, not first generation)

Admission basis (first-time freshman or transfer)

STEM major flag (yes or no)

Data Cleaning

A total of 14,699 responses were received across years (2015 N = 3,155; 2016 N = 3,589; 2017 N = 4,276; 2018 N = 3,679). There were 31 responses that did not include a student ID and 251 responses where the student ID did not match with IRPA data; these responses were excluded from the sample. It was determined that 604 responses were provided by non-undergraduate students (e.g., graduate students, visiting students) and these responses were excluded from the sample. There were also 310 students who did not have certain demographic data provided. Specifically, five students were missing prior GPA data, and 305 students did not have first-generation status. These students were not included in any analyses. A number of incoming students who completed a survey in the same term that they had enrolled were removed from analyses, as students were asked to report on research activities during the previous year and incoming freshmen and first-year transfers were not at Cal Poly Pomona the previous year. The percentage of students removed from analyses was high and ranged from 31.0% to 40.3% across years.

Participants

A total of 8,660 completed surveys, from 6,654 unique student respondents, remained (2015 n = 2,017; 2016 n = 2,077; 2017 n = 2,425; 2018 n = 2,141). Based on the total sample of 6,654 unique participants, we saw that the respondents’ demographics (with the exception of first-generation rates) were slightly different from that of the Cal Poly Pomona general population. Most of the respondents were female (57.8%, compared to the overall campus population where 47% are female), entered as freshmen (64.5%, Cal Poly Pomona: 55%), first-generation college students (58.5%, Cal Poly Pomona: 58%), non-URM (53.6%, Cal Poly Pomona: 44%), and low-income students (61.2%, Cal Poly Pomona: 46%).

It was also possible that students completed multiple surveys across years; these students with multiple survey responses were included; however, the number of surveys they completed was included as a control variable in all analyses. Of the total sample, 79.3% (n = 5,278) completed one survey, 17.1% (n = 1,137) completed two surveys, 3.2% (n = 213) completed three surveys, and 0.4% (n = 26) completed surveys in all 4 years.

Results

Prevalence of Student Participation in Research

While the percentage of students reporting participation in research-related activities varied slightly depending on the year, approximately 55.2% (n = 3,674) of unique respondents indicated they had not participated in any research-related activities, while 44.8% (n = 2,980) of unique respondents participated in at least one research-related activity during the period of interest (Table 1).

Table 1.

Student Participation in Research Across Years.

	Participated in research (n)	Did not participate in research (n)
2014–2015	41.2% (831)	58.8% (1,186)
2015–2016	43.6% (905)	56.4% (1,172)
2016–2017	40.7% (988)	59.3% (1,437)
2017—2018	40.6% (870)	59.4% (1,271)
Overall^a	44.8% (2,980)	61.8% (3,674)

Note: Unique students reporting research participation.

Type of Student Research Participation

Students reported the types of research-related activities in which they participated; students were able to provide responses in more than one category. Hence, the total number exceeds 100%. Table 2 provides a summary of research type for each year. The most frequent categories of participation reported were “Internship with a professional in the field” and “Other”, which included responses such as “robotics club,” “job as a therapy aide,” and “Salvation Army volunteer.”

Table 2.

Student Participation in Research-Related Activities Across Years.

	2014–2015 (N = 2,017)	2015–2016 (N = 2,077)	2016–2017 (N = 2,425)	2017—2018(N = 2,141)	Overall (N = 6,654)
Internship with professional in field % (n)	15.2% (307)	15.8% (328)	14.4% (348)	11.8% (253)	16.7% (1,112)
Training under professional in field of interest % (n)	11.8% (239)	13.0% (270)	9.6% (234)	10.0% (214)	13.4% (894)
Internship with industry % (n)	11.1% (223)	11.5% (238)	9.5% (230)	9.2% (198)	12.2% (815)
Research/project with faculty % (n)	10.5% (212)	10.5% (218)	9.0% (218)	9.9% (213)	11.3% (754)
Research/project outside of Cal Poly Pomona % (n)	5.3% (107)	4.1% (85)	3.8% (91)	3.2% (68)	5.5% (368)
Other % (n)	13.2% (267)	13.4% (279)	13.4% (324)	13.2% (282)	13.4% (894)

Student Research Participation Across Demographic Groups

Analyses across years (2015–2018) of the 6,654 unique participating students included an examination of potential differences in research participation by demographic characteristics; whether there were demographic differences in students who reported ever participating in research activities across all years that the survey was administered.

Table 3 provides a summary of research participation across key demographic groups. A Pearson chi-square test found statistically significant differences in research participation across three demographic groups: low-income students were less likely (44.3%) to participate in research-related activities than non-low-income students (50.4%), χ²(1) = 23.27, p<.001. Students who were not first-generation were more likely to participate in research-related activities (51.0%) than either category of first-generation students (47.1% of first Bachelor's degree students and 41.2% of first college degree students), χ² (2) = 47.84, p<.001. Finally, students who were non-URM were more likely to participate in research-related activities (49.3%) than URM students (43.7%), χ²(1) = 20.81, p<.001.

RQ1. Does participation in research-related activities affect student graduation rates?

Table 3.

Comparison of Research Participation Among Key Demographic Groups (N = 6,654).

		Participated in research % (n)	Did not participate in research % (n)	χ² (df)	Cramer's V
Sex	Male	46.8% (1,316)	53.2% (1,494)	0.06 (1)	0.003
Sex	Female	46.5 (1,789)	53.5% (2,055)	0.06 (1)	0.003
Admission status	FTF	46.4% (1,993)	53.6% (2,298)	0.23 (1)	0.01
Admission status	Transfer	47.1% (1,112)	52.9% (1,251)	0.23 (1)	0.01
Financial need	Non-Low Income	50.4% (1,301)	49.6% (1,282)	23.27***(1)	0.06
Financial need	Low Income	44.3% (1,804)	55.7% (2,267)	23.27***(1)	0.06
First-Gen status (N = 4,165)	Not First-Gen	51.0% (1,406)	49.0% (1,353)	47.84***(2)	0.09
	First Bachelor's	47.1% (752)	52.9% (846)
	First College	41.2% (947)	58.8% (1,350)
URM status	Non-URM	49.3% (1,758)	50.7% (1,811)	20.81***(1)	0.06
URM status	URM	43.7% (1,347)	56.3% (1,738)	20.81***(1)	0.06

*p < .05; **p < .01; ***p < .001.

The impact of research-related participation on student graduation rates was assessed using a binary logistic regression analysis. Only students who were expected to graduate by Spring semester 2019 (within at least 6 years for matriculation for fiirst-time freshman (FTF) students and within at least 4 years of matriculation for transfer students) were included in this analysis. The impact of research activity participation on time to graduation (within 4 and 6 years for FTF students, and within 2, 3, and 4 years for transfer students) was also assessed using a binary logistic regression analysis. We controlled for a number of student demographic and background variables that could reasonably be expected to impact graduation as well as student participation in activities (see Collins et al., 2017). These included student admission status (i.e., FTF or transfer), prior GPA (high school GPA for FTF students and transfer institution GPA for transfer students), sex (i.e., male or female), low-income status, first-generation status, and URM status. In addition, as students were able to participate in multiple surveys over the course of the study, the number of surveys completed was controlled for in the analyses.

To ensure that research activities classified as “Other” did not unduly influence the results, analyses were rerun with the “Other” category of research activities excluded. For each of the key analyses, we report findings both including and excluding “other” as a separate category. In each case, excluding the “other” category did not unduly influence the results.

Research-related participation was found to be a significant predictor of graduation. That is, students who reported participating in research-related activities had higher graduation rates (91.3%) than those who did not participate in research-related activities (85.5%), Wald χ²(1) = 24.87, p < .001, OR = 1.71. This represents a nearly six percentage point difference. Notably, this analysis also demonstrated that women and non-URM students were more likely to graduate than men and URM students, respectively, and that students with higher GPAs also had a higher likelihood of graduating than those with lower GPAs. The results were similar when the “other” category of research participation was excluded Wald χ²(1) = 25.70, p < .001, OR = 1.53.

Interaction terms were added to the model to determine if the effects of research-related participation differed across demographic groups. Demographic variables examined included sex, URM status, low-income status, first-generation status, and admission basis. None of the interaction terms were significant and the pattern of main effects remained generally the same, suggesting that research-related participation is equally beneficial across groups (see Table 4).

Table 4.

Impact of Participation in Research Activities on Graduation Rates (N = 3,708).

Predictor	Original model
Predictor	B	S.E. (B)	Wald	OR
Research participation	0.54	0.11	24.87***	1.71
Sex	0.49	0.11	21.43***	1.63
Admission status	−0.15	0.12	1.58	0.86
URM status	−0.39	0.11	11.67**	0.68
Low-income status	−0.22	0.12	3.32	0.80
First Gen status			0.51
First degree	−0.05	0.14	0.10	0.96
First college	0.05	0.14	0.15	1.05
Prior GPA	0.86	0.13	43.65***	2.37
Number of surveys	−0.19	0.10	3.68	0.83
Constant	−0.66	0.44	2.24	0.52
Interaction terms
ResPart × Sex	−0.05	0.14	0.11	0.95
ResPart × URM	0.09	0.21	0.18	1.10
ResPart × LowIncome	0.06	0.23	0.06	1.06
ResPart × FirstDegree	−0.07	0.27	0.07	0.93
ResPart × FirstCollege	−0.03	0.25	0.01	0.98
ResPart × AdmitStat	0.32	0.21	2.22	1.37

Note: *p < .05; **p < .01; ***p < .001. Each interaction term was entered in a separate analysis, except for the first-generation status interaction terms (first degree and first college), which were entered together. As such, first-generation status utilized “not first-generation” as the comparison group.

Analyses examining whether participation in research-related activities had an impact on time to graduation were run separately for FTF (graduation within 4 and 6 years) and transfer students (graduation within 2, 3, and 4 years). Results demonstrated that FTF students who participated in research-related activities were more likely to graduate within 6 years (89.5%) than FTF students who did not participate in research (82.3%), χ²(1) = 22.62, p < .001, Cramer's V = 0.10. There were no significant differences for graduation within 4 years. Similarly, transfer students who participated in research-related activities were more likely to graduate within 4 years (84.4%) than those who did not participate in research-related activities (79.7%), χ²(1) = 5.90, p = .02, Cramer's V = 0.06 (see Table 5). See Tables 4 and 5 for complete results of graduation and time to graduate analyses. When excluding the “other” category, 4-year and 6-year time to graduation rates for FTF are consistent with these results; 4-year time to graduation χ²(1) = 2.96, p > .05, Cramer's V = 0.03; 6-year time to graduation χ²(1) = 21.24, p < .001, Cramer's V = 0.07.

Table 5.

Impact of Research Participation on Time to Graduate (N = 3,708).

Admission status	Time to graduate	Research participation	Graduated % (n)	Did not graduate % (n)	χ² (df)	Cramer's V
FTF (n = 2,097)	4 Years	Yes	45.1% (515)	54.9% (627)	2.04(1)	0.03
	4 Years	No	42.0% (401)	58.0% (554)	2.04(1)	0.03
	6 Years	Yes	89.5% (1,022)	10.5% (120)	22.62***	0.10
	6 Years	No	82.3% (786)	17.7% (169)	22.62***	0.10
Transfer (n = 1,611)	2 Years	Yes	34.6% (266)	65.4% (502)	1.09(1)	0.03
	2 Years	No	37.1% (313)	62.9% (530)	1.09(1)	0.03
	3 Years	Yes	73.3% (563)	26.7% (205)	0.91(1)	0.02
	3 Years	No	71.2% (600)	28.8% (243)	0.91(1)	0.02
	4 Years	Yes	84.4% (648)	15.6% (120)	5.90*(1)	0.06
	4 Years	No	79.7% (672)	20.3% (171)	5.90*(1)	0.06

Amount of Research

RQ2. Does the amount of research-related participation affect student graduation rates?

Binary logistic regressions were conducted to assess whether the amount of research-related participation was linked to graduation outcomes. Amount of research was operationalized in two distinct ways: (a) number of research-related activities in which students participated and (b) research participation across time.

Number of Research-Related Activities

To calculate the number of research-related activities each participant reported, all instances of research participation reported across all categories was totaled. This includes multiple instances of research-related activities in a given year or across years for those who completed the survey multiple years.

A binary logistic regression was used to determine whether the number of research-related activities students participated in predicts graduation. These analyses controlled for the number of surveys completed, student admission status, prior GPA, sex, low-income status, first-generation status, and URM status. The results of this analysis demonstrated that the number of research-related activities students participated in was a significant predictor of graduation (see Table 6) such that students who participated in more research-related activities were more likely to graduate than who participated in fewer research activities, Wald χ²(1) = 16.44, p < .001, OR = 1.23. The results were similar when the “other” category of research participation was excluded Wald χ²(1) = 23.970, p < .001, OR = 1.32.

Table 6.

Number of Research Activities Participated in Predicting Graduation Rates (N = 3,708).

Predictor	B	S.E. (B)	Wald	OR
Number of research activities	0.21	0.05	16.44***	1.23
Sex	0.49	0.11	21.23***	1.63
Admission status	−0.14	0.12	1.46	0.87
URM status	−0.38	0.11	11.24**	0.68
Low income	−0.23	0.12	3.49	0.80
First Gen status			0.44
First degree	−0.04	0.14	0.08	0.96
First college	0.05	0.14	0.13	1.05
Prior GPA	0.88	0.13	45.42***	2.41
Number of surveys	−.022	0.10	5.15*	0.80
Constant	−0.59	0.44	1.81	0.56

Note: *p < .05; **p < .01; ***p < .001.

Research Participation Across Time

A binary logistic regression was used to determine whether the number of years that students participated in research-related activities students predicts graduation. These analyses only included students who completed at least two surveys across years of the study, and controlled for the number of surveys completed, student admission status, prior GPA, sex, low-income status, first-generation status, and URM status. Across all analyses, the number of years of research-related participation was not found to be a significant predictor of graduation (see Table 7).

RQ3. Does the type of research-related participation affect student graduation rates?

Table 7.

Number of Years of Research Participation Predicting Graduation Rates.

Outcome	B	S.E. (B)	Wald	OR
Graduation (N = 767)	−0.02	0.13	0.02	0.98

Note: These analyses only included students who took part in at least two surveys over the course of the study duration.

To assess whether participation in certain types of research activities was most predictive of graduation, a binary logistic regression analysis was performed. This analysis included terms for whether students had ever participated in each of the six types of research activities, and controlled for the number of surveys completed, prior GPA, sex, URM status, financial need, first generation status, and enrollment status. Patterns of activity for participation disaggregated by STEM major versus non-STEM major, college and other demographic characteristics can be found in Table 8.

Table 8.

Activity Participation Percentages (N) Based on STEM status and College, and Other Demographic Characteristics.

	Internship within industry	Internship with professional	Research with faculty	Outside research	Training with a professional	Other
STEM status (N = 6,551)
STEM major	17.5 (525)	15.8 (474)	15.2 (454)	6.5 (193)	11.4 (341)	15.2 (455)
Non-STEM major	7.8 (277)	17.3 (615)	8.0 (283)	4.7 (167)	15.0 (533)	16.9 (602)
College (N = 6,542)
Agriculture	11.6 (82)	23.8 (168)	10.7 (76)	4.2 (30)	20.8 (147)	21.5 (152)
Business administration	10.6 (120)	20.1 (227)	4.2 (47)	3.8 (43)	14.3 (161)	15.4 (173)
Education Integ	2.1 (6)	6.8 (20)	9.6 (28)	5.1 (15)	12.7 (37)	18.2 (53)
Engineering	23.6 (383)	15.7 (255)	15.4 (249)	5.6 (90)	9.0 (145)	14.4 (233)
Environmental design	6.2 (22)	24.9 (88)	5.9 (21)	5.4 (19)	11.0 (39)	12.2 (43)
Hospitality management	18.8 (60)	19.7 (63)	3.8 (12)	2.8 (9)	25.6 (82)	22.5 (72)
Letters, Arts, & Soc Sci	3.7 (36)	12.1 (118)	12.3 (120)	5.7 (56)	12.1 (118)	17.1 (167)
Science	8.1 (93)	12.9 (148)	16.0 (183)	8.5 (97)	12.6 (144)	14.2 (163)
Sex (N = 6,654)
Male	14.7 (413)	15.8 (445)	12.2 (342)	6.2 (175)	12.2 (342)	15.1 (425)
Female	10.5 (402)	17.4 (667)	10.7 (412)	5.0 (193)	14.4 (552)	16.9 (651)
URM Status (N = 6,654)
Non-URM	14.3 (512)	18.2 (648)	12.5 (447)	6.2 (222)	14.3 (510)	16.6 (592)
URM	9.8 (303)	15.0 (464)	10.0 (307)	4.7 (146)	12.4 (384)	15.7 (484)
Financial need (N = 6,654)
Non-low income	14.0 (361)	19.0 (490)	11.5 (298)	5.2 (135)	14.4 (371)	17.1 (442)
Low income	11.2 (454)	15.3 (622)	11.2 (456)	5.7 (233)	12.8 (523)	15.6 (634)
First-Gen status (N = 6,654)
Not First-Gen	14.7 (406)	20.2 (558)	12.0 (332)	6.0 (165)	14.6 (403)	17.6 (486)
First Bachelor's	12.6 (202)	15.3 (244)	12.1 (193)	4.9 (78)	13.8 (220)	16.0 (255)
First College	9.0 (207)	13.5 (310)	10.0 (229)	5.4 (125)	11.8 (271)	14.6 (335)
Admission status (N = 6,654)
FTF	12.3 (527)	17.0 (730)	12.1 (287)	6.7 (159)	14.9 (352)	16.5 (390)
Transfer	12.2 (288)	16.2 (382)	10.9 (467)	4.9 (209)	12.6 (542)	16.0 (686)

The results of this analysis demonstrated that participation in internships, and particularly internships within industry, significantly predicted graduation. That is, those who participated in an internship within industry were more likely to graduate, Wald χ²(1) = 8.96, p = .003, OR = 1.74. Students who participated in an internship with a professional were also more likely to graduate, Wald χ²(1) = 3.96, p = .05, OR = 1.36. Overall, graduation rates can be found in Table 9, and results of the full model can be found in Table 10.

Table 9.

Graduation Rates for Students who Participated and did not Participate in Each Type of Research Activity (N = 3,708).

Type of activity	Participation % graduated (N)	Did not participate % graduated (N)
Internship in industry	93.5 (534)	87.6 (2,748)
Internship w/ professional	91.8 (708)	87.6 (2,574)
Research w/ faculty	90.4 (463)	88.2 (2,819)
Outside research	90.2 (220)	88.4 (3,062)
Training w/ professional	91.4 (511)	88.0 (2,771)
Other activities	90.7 (555)	88.1 (2,727)

Table 10.

Impact of Type of Activity Participated in on Graduation Rates (N = 3,708).

Predictor	Original Model
Predictor	B	S.E. (B)	Wald	OR
Internship in industry	0.56	0.19	8.96**	1.74
Internship w/ professional	0.31	0.15	3.96*	1.36
Research w/ faculty	0.07	0.17	0.17	1.07
Outside research	0.05	0.23	0.05	1.05
Training w/ professional	0.21	0.17	1.39	1.23
Other activities	0.16	0.16	1.10	1.18
Sex	0.50	0.11	22.28***	1.65
Admission status	−0.15	0.12	1.54	0.86
URM status	−0.38	0.11	11.13**	0.68
Low-income status	−0.22	0.12	3.24	0.80
First Gen status			0.49
First degree	−0.03	0.14	0.05	0.97
First college	0.06	0.14	0.20	1.06
Prior GPA	0.88	0.13	44.49***	2.40
Number of surveys	−0.20	0.10	4.12*	0.82
Constant	−0.64	0.44	2.12	0.53

Note: *p < .05; **p < .01; ***p < .001.

Exploratory analyses were conducted to determine if the benefits of participation in internships differed across academic and demographic characteristics; whether certain groups of students are more likely to benefit from participation in internships, which may then be most beneficial in terms of graduation. Specifically, we examined three factors: first, whether students who were in a designated STEM major were more likely to benefit from participation in internships than students in a non-STEM major; second, whether students enrolled in different colleges (e.g., College of Science, College of Business Administration) were more likely to benefit from participation in internships; and third, whether students with different demographic characteristics (i.e., sex, URM status, financial need, first generation status, and admission status) were more likely to benefit from participation in internships. As some students completed the survey in multiple years, their most recent survey was used in defining whether they were in a STEM major, and their associated college (see Table 11). A series of binary logistic regression analyses were conducted to determine whether STEM status, college, and demographics moderated the effect of specific types of research activity participation. All of these analyses controlled for the number of surveys completed, prior GPA, sex, URM status, financial need, first generation status, and enrollment status; college analyses use the College of Science as a comparison group and first-generation status used non-first-generation students as the comparison group. Results indicate that there were no significant interactions of research activity participation based on STEM status, college, or demographics, suggesting that the benefits of participating in internships were equivalent across these student groups.

Table 11.

Graduation Rates by STEM Status and College.

	Graduated % (N)	Did not graduate % (N)
STEM status (N = 3,632)
STEM major	88.4 (1,377)	11.6 (181)
Non-STEM major	89.5 (1,856)	10.5 (218)
College (N = 3,708)
Agriculture	88.4 (327)	11.6 (43)
Business administration	89.0 (654)	11.0 (81)
Education Integ	93.8 (165)	6.3 (11)
Engineering	85.1 (689)	14.9 (121)
Environmental design	77.4 (130)	22.6 (38)
Hospitality management	95.8 (182)	4.2 (8)
Letters, Arts, & Soc Sci	91.6 (481)	8.4 (44)
Science	91.9 (599)	8.1 (53)

Discussion

In this study, less than half of the sample (45%) of students reported participating in some form of research-related activity outside the classroom at Cal Poly Pomona. After controlling for demographic and academic characteristics, the odds of graduating for students who participated in research-related activities were almost twice as much as compared to those who did not participate in research (p < .001, OR = 1.71). This effect was driven mainly by participation in internships, those within industry and with a professional. In addition, students who participated in more research-related activities were more likely to graduate than who participated in fewer research activities. It should be noted that all statistically significant results represent negligible-to-small effect sizes (e.g., 0.10), thus care must be taken not to overstate the impact of research participation on outcomes.

In previous literature, participation in undergraduate research is generally operationally defined as participation in a faculty research lab or team on campus. This study opted to use a more inclusive definition of participation in research that spans beyond disciplinary boundaries. Participation in undergraduate research was broadly defined as participation in undergraduate research or research-related activities including internships and other career-initiating positions (e.g., activities that train students to be a professional in their discipline). Our conceptualization of research-related activities that include undergraduate research experiences, internships, and other experiences allows us to both see larger patterns of outside coursework preparation as well as compare their contribution to graduation outcomes. Our inclusion of activities designated as “other” reflects the reality that that while some disciplines may be conducive to more traditional forms of research (e.g., chemistry), others require that students attain more practical experience in preparation for the workforce (e.g., hospitality management). It is important to note that analyses were conducted excluding the category “other” and the pattern of results remained the same. We believe that inclusion of those activities which promote discipline-specific knowledge and career preparation as we have done in the current study is the most appropriate conceptualization of research-related activities; this serves as a more unifying framework than solely including undergraduate research, internships, or other similar roles.

Study Limitations

One limitation to these findings is that students who pursue opportunities to participate in research-related activities likely differ from those who did not participate in research-related activities on important characteristics not accounted for in the analyses such as motivation. Although motivation was not assessed, the analyses did account for prior institution GPA, which is often correlated with motivation. Characteristics that were unaccounted for likely influenced the observed relationship between research and student outcomes. It is possible that the most motivated students are the most likely to participate in research and participate more frequently (more research activities) in comparison to less motivated students. Furthermore, the choice to participate in undergraduate research is not always fully decided by the students themselves, as faculty and program staff may choose to recruit students who share similar interests, or they believe have the greatest likelihood of success in STEM (NASEM, 2017). The current study does not account for these factors.

Another noteworthy limitation is that some majors at Cal Poly Pomona require students to participate in a research project (e.g., senior capstone in philosophy, engineering, chemistry) and internships (in business) to graduate. However, many of these research or internship requirements are course-based experiences (where students receive course credits) and the survey specifically inquired about extracurricular (without course credits) research-related participation. Furthermore, undergraduate students completed the survey in the fall semester, reflecting on their participation in research-related activities in the previous year. Thus, it is unlikely that required senior-year research/internship comprised a major portion of research-related participation captured in the annual survey. The current study was intended to examine the benefit of research-related activities above and beyond course-based experiences. However, it is possible that students interpreted the question to include course-based research experiences despite the admonition to avoid it.

A final set of limitations to the current study was that participation in research-related activities was self-reported by students. There is no independent way to verify such self-reported activities, no way to monitor the consistency in implementation of such activities, nor was there any confirmation of the length of time for such participation. Further, it is possible that the quality of experience for students varies substantially. The study sample was not recruited from a formal intervention program at Cal Poly Pomona, but rather from the campus community at large. Participants likely engaged in a range of formal to informal research-related situations. Despite a large sample size, it is also likely that the relationship among research-related participation overall, length of participation, and amount of participation and associated graduation outcomes at Cal Poly Pomona is not fully captured within the current study. Future research should address the current study shortcomings to investigate the relevance of these additional factors when research-related activities are operationally defined to include consistent or standardized implementation.

Importance of Providing Students With Research-Related Activities

In this study, certain types of students were more likely to participate in research-related activities than others (i.e., non-low-income students, non-first-generation students, and non-URM students). This is consistent with other research findings related to first-generation students, for example. While Pike and Kuh (2005) did not examine students’ socio-economic status in their study, they found that first-generation students reported lower levels of academic and social engagement and were less likely to integrate diverse college experiences in comparison to their second-generation peers. Although these findings were largely a function of first-generation students being less likely to live on campus and more likely to report lower educational aspirations, student engagement predicted greater gains in learning and intellectual development. Similarly, participation in research-related experiences examined in the current study can also be considered a function of engagement. We did not measure key variables such as living on campus, students’ educational aspirations or other factors related to engagement as part of the current study, however, it is likely that overall engagement in campus activities, including research-related experiences, could differ for the same students who were less likely to participate in research-related activities such as low-income and first-generation students. Future research should examine additional engagement factors to examine these relationships.

Findings from the current study align with previous research in that URM students were less likely to participate in research than non-URM students, however, URM students did not benefit more or less from research participation than non-URM students when they did participate. These results are encouraging and suggest that research participation is more important for students’ success than their status as racial or ethnic minorities. In fact, researchers agree that higher levels of both academic and social engagement were positively related to minority group membership (Finley & McNair, 2013; Garcia, 2016; Gasiewski et al., 2012; Kahu et al., 2020; Pike & Kuh, 2005). What students do in college is more important to their success that their group membership. These data also point to the importance of examining the learning environment at HSIs, that may provide a more inclusive environment for all students, regardless of their URM status.

The literature suggests that students who participate in research have both better academic outcomes (e.g., GPA) and better affective outcomes (more meaningful interaction with faculty, e.g., Collins et al., 2017). Therefore, as a matter of practice, institutions should particularly encourage and provide clearer pathways for URM students to participate in a variety of extra-curricular activities, particularly research-related activities, if they expect to observe similar outcomes (i.e., graduation) and shrink achievement gap discrepancies compared with non-URM students. In our study, students who participated in more research related activities had better outcomes, but those who participated in these activities for a longer duration did not. This provides additional support for the idea that engagement that is accomplished through a variety of academic and social experiences is promising, regardless of a students’ URM status or the length of such engagement. Further, institutions should develop intentional strategies to guide students on how to engage in a variety of college experiences, particularly for students who are less likely to understand the importance of such participation or who perceive the college environment as less supportive.

Future studies should investigate why participation in research-related activities leads to desired outcomes. In addition to engagement, several mechanisms may explain the relationship between research participation and graduation including sense of belonging (Strayhorn, 2012), faculty mentorship (e.g., Crisp and Cruz, 2009), and self-efficacy (e.g., DeWitz et al., 2009). Future research is needed to elucidate the relationship between participation in research-related activities and higher graduation rates, which can inform efforts on how to best support student persistence. Furthermore, additional research is needed to help institutions better understand what constitutes a high-quality research-related experience and who has access to these experiences. Kuh and O’Donnell (2013) identified eight elements that exemplify HIP, some of which are more relevant for research-related activities than others (e.g., substantive interaction with faculty and peers, real-world application). According to a report published by the Lumina Foundation (Kinzie et al., 2020), for institutions to assess the impact of research-related activities and ensure high-quality experiences, they should (1) measure students’ exposure to the elements of quality; (2) establish a threshold of “high” quality; and (3) examine student satisfaction. Given the current study findings, additional research is needed to examine how HSI institutions in particular can increase student participation in a variety of research-related activities for students who are less likely to understand the value of such participation or who have other constraints that prevent them from taking full advantage of the experiences outside of the classroom.

Footnotes

Acknowledgments

This work was supported by the Office of Undergraduate Research at California State Polytechnic University, Pomona.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Winny Dong

Notes

Author Biographies

Winny Dong is the Faculty Director of the Office of Undergraduate Research at Cal Poly Pomona. Dr. Dong has over 20 years of experience mentoring students in undergraduate research, training faculty mentors, and providing resources and support for students to conduct research.

Rebecca M. Eddy is the President of Cobblestone Applied Research & Evaluation, Inc. She received her doctorate in applied cognitive psychology and has served as the primary investigator on more than 100 projects. Dr. Eddy has 20 years of experience teaching and conducting evaluation and applied research studies in K-12 through higher education.

David M. Mendelsohn is the Statistics Consultant at Cobblestone Applied Research & Evaluation, Inc. and also serves as an adjunct faculty member teaching courses in psychology, statistics and psychometrics. He is currently pursuing his doctorate in Organizational Psychology at Claremont Graduate University.

Courtney Koletar is the Assistant Director of Data and Knowledge Management at Cobblestone Applied Research & Evaluation, Inc. Courtney specializes in database management, data analysis and works with several institutions of higher education on evaluating student-directed interventions. She is currently pursuing her doctorate in Evaluation and Applied Research Methods at Claremont Graduate University.

Monique Matelski is the Director of Research and Evaluation at Cobblestone Applied Research & Evaluation, Inc. She received her doctorate in applied social psychology and specializes in working with faculty, administrators, students, and staff to evaluate educational programs in higher education, with a focus on those serving underrepresented and low-income students.

Everardo Barraza is the Assistant Director for the Office of Undergraduate Research at Cal Poly Pomona. He has served multiple roles in assisting in the implementation and coordination of grant programs that serve minoritized students in STEM, to leading research training programs for undergraduates. As a doctoral student in education leadership at Cal Poly Pomona, Everardo focuses his research on understanding equity and social justice leadership of Hispanic-serving institutions (HSIs) grant leaders using a servingness lens.

References

Adebakin

A. B.

Ajadi

T. O.

Subair

S. T.

(2015). Required and possessed university graduate employability skills: Perceptions of the Nigerian employers. World Journal of Education, 5(2), 115–125. https://doi.org/10.5430/wje.v5n2p115

Adedokun

O. A.

Parker

L. C.

Childress

Burgess

Adams

Agnew

C. R.

Leary

Knapp

Shields

Lelievre

Teegarden

(2014). Effect of Time on Perceived Gains from an Undergraduate Research Program, CBE - Life Sciences Education, 13, 139–148. https://doi.org/10.1187/cbe.13-03-0045

Astin

A. W.

(1984). Student involvement: A developmental theory for higher education. Journal of College Student Personnel, 25(4), 297–308.

Bowman

N. A.

Holmes

J. M.

(2018). Getting off to a good start? First-year undergraduate research experiences and student outcomes. Higher Education: The International Journal of Higher Education Research, 76(1), 17–33.

Boyer Commission on Educating Undergraduates in the research university (1998). Reinventing undergraduate education: A blueprint for America's Research universities. Carnegie Foundation for the Advancement of Teaching.

BrckaLorenz

Garvey

J. C.

Hurtado

S. S.

Latopolski

(2017). High-impact practices and student–faculty interactions for gender-variant students. Journal of Diversity in Higher Education, 10(4), 350–365. https://doi.org/10.1037/dhe0000065

Callanan

Benzing

(2004). Assessing the role of internships in the career-oriented employment of graduating college students. Education & Training, 46(2), 82–89. https://doi.org/10.1108/00400910410525261

Chang

M. J.

Sharkness

Hurtado

Newman

C. B.

(2014). What matters in college for retaining aspiring scientists and engineers from underrepresented racial groups. Journal of Research in Science Teaching, 51(5), 555–580. https://doi.org/10.1002/tea.21146

Collins

T. W.

Grineski

S. E.

Shenberger

Morales

Morera

O. F.

Echegoyen

L. E.

(2017). Undergraduate research participation is associated with improved student outcomes at a Hispanic-serving institution. Journal of College Student Development, 58(4), 583–600. https://doi.org/10.1353/csd.2017.0044

10.

Crisp

Cruz

(2009). Mentoring college students: A critical review of the literature between 1990 and 2007. Research in Higher Education, 50, 525–545. https://doi.org/10.1007/s11162-009-9130-2

11.

D'Abate

(2010). Developmental interactions for business students: Do they make a difference? Journal of Leadership & Organizational Studies, 17(2), 143–155. https://doi.org/10.1177/1548051810370795

12.

DeWitz

S. J.

Woolsey

M. L.

Walsh

W. B.

(2009). College student retention: An exploration of the relationship between self-efficacy beliefs and purpose in life among college students. Journal of College Student Development, 50, 19–34. https://doi.org/10.1353/csd.0.0049

13.

Fechheimer

Webber

Kleiber

P. B.

(2011). How well do undergraduate research programs promote engagement and success of students? CBE life Sciences Education, 10(2), 156–163. https://doi.org/10.1187/cbe.10-10-0130

14.

Finley

McNair

(2013). Assessing underserved students’ engagement in high-impact practices. Association of American Colleges and Universities, 68, 1–68. https://www.aacu.org/assessinghips/report

15.

Garcia

G. A.

(2016). Complicating a Latina/o-serving identity at a hispanic serving institution. Review of Higher Education, 40(1), 117. https://doi.org/10.1353/rhe.2016.0040

16.

Gasiewski

Eagan

Garcia

Hurtado

Chang

(2012). From gatekeeping to engagement: A multicontextual, mixed method study of student academic engagement in introductory STEM courses. Research in Higher Education, 53(2), 229–261. https://doi.org/10.1007/s11162-011-9247-y

17.

Gault

Leach

Duey

(2010). Effects of business internships on job marketability: The employers’ perspective. Education + Training, 52(1), 76–88. https://doi.org/10.1108/00400911011017690

18.

Gault

Redington

Schlager

(2000). Undergraduate business internships and career success: Are they related? Journal of Marketing Education, 22(1), 45–53. https://doi.org/10.1177/0273475300221006

19.

Gilmore

Vieyra

Timmerman

Feldon

Maher

(2015). The relationship between undergraduate research participation and subsequent research performance of early career STEM graduate students. Journal of Higher Education, 86(6), 834–863. https://doi-org.proxy.library.cpp.edu/10.1353/jhe.2015.0031

20.

Graduation initiative 2025 equity goals, … - calstate.edu (n.d.). Retrieved June 17, 2020, from https://www.calstate.edu/csu-system/why-the-csu-matters/graduation-initiative-2025/Documents/gi-2025-equity-goals-and-priorities-2021-22.pdf.

21.

Graduation initiative 2025. The California State University (n.d.). Retrieved November 17, 2021, from https://www2.calstate.edu/csu-system/about-the-csu/budget/2019-20-operating-budget/use-of-funds/Pages/graduation-initiative.aspx.

22.

Kuh

G. D.

(2002). Being (dis) engaged in educationally purposeful activities: The influences of student and institutional characteristics. Research in Higher Education, 43(5), 555–575. https://doi.org/10.1023/A:1020114231387

23.

Jones

M. T.

Barlow

A. E.

Villarejo

(2010). Importance of undergraduate research for minority persistence and achievement in biology. The Journal of Higher Education, 81(1), 82–115. https://doi.org/10.1080/00221546.2010.11778971

24.

Kahu

E. R.

Picton

Nelson

(2020). Pathways to engagement: A longitudinal study of the first-year student experience in the educational interface. Higher Education, 79(4), 657–673. https://doi.org/10.1007/s10734-019-00429-w

25.

Kendricks

Arment

(2011). Adopting a K-12 family model with undergraduate research to enhance STEM persistence and achievement in underrepresented minority students. Journal of College Science Teaching, 41(2), 22–27.

26.

Kinzie

McCormick

A. C.

Gonyear

Dugan

Silverstein

(2020). Assessing quality and equity in high-impact practices: Comprehensive report. Indiana University Center for Postsecondary Research.

27.

Kuh

O’Donnell

(2013). Ensuring quality and taking high-impact practices to scale. Association of American Colleges and Universities.

28.

Kuh

G. D.

(2001). Assessing what really matters to student learning inside the national survey of student engagement. Change: The Magazine of Higher Learning, 33(3), 10–17. https://doi.org/10.1080/00091380109601795

29.

Kuh

G. D.

(2003). What we're learning about student engagement from NSSE: Benchmarks for effective educational practices. Change: The Magazine of Higher Learning, 35(2), 24–32. https://doi.org/10.1080/00091380309604090

30.

Kuh

G. D.

(2008). High-impact educational practices: What they are, who has access to them, and why they matter. American Association of Colleges and Universities. https://www.aacu.org/node/4084.

31.

Kuh

G. D.

(2009). The national survey of student engagement: Conceptual and empirical foundations. New Directions for Institutional Research, 2009(141), 5–20. https://doi.org/10.1002/ir.283

32.

Kuh

G. D.

Kinzie

Schuh

Whitt

E. J.

(2010). Student success in college: Creating conditions that matter (1st ed.). Jossey-Bass. ISBN: 978-0-470-59909-9.

33.

Linn

Palmer

Baranger

Gerard

Stone

(2015). Undergraduate research experiences: Impacts and opportunities. Science, 347(6222), 1261757. https://doi.org/10.1126/science.1261757

34.

Lopato

(2010). Undergraduate research as a high-impact student experience. peerReview, 12(2), 24–27.

35.

National Academies of Sciences, Engineering, and Medicine (2017). Undergraduate research experiences for STEM students: Successes, challenges, and opportunities. The National Academies Press. Retrieved November 17, 2021, from https://www.nap.edu/catalog/24622/undergraduate-research-experiences-for-stem-students-successes-challenges-and-opportunities.

36.

Pike

G. R.

Kuh

G. D.

(2005). A typology of student engagement for American colleges and universities. Research in Higher Education, 46(2), 185–209. https://doi.org/10.1007/s11162-004-1599-0

37.

Rodenbusch

S. E.

Hernandez

P. R.

Simmons

S. L.

Dolan

E. L.

(2016). Early engagement in course-based research increases graduation rates and completion of science, engineering, and mathematics degrees. CBE Life Sciences Education, 15(2), ar20. https://doi.org/10.1187/cbe.16-03-0117

38.

Strayhorn

T. L.

(2012). College students’ sense of belonging: A key to educational success for all students. Routledge.