Abstract
This study evaluates the impact of the peer-mentoring program implemented by a Chilean higher education institution on underrepresented students’ academic success. Specifically, it assesses whether freshmen who enrolled in 2018 and took part in this initiative performed better than students with similar characteristics who did not. A quantitative quasi-experimental design was applied, using the Propensity Score Matching method. The results show that students who took part in this peer-mentoring program got better average grades and had better retention rates and attendance levels than those who did not. Strategies for developing successful mentoring initiatives for college students are discussed, with a special emphasis on their potential relevance to underrepresented students.
In recent times, several higher education systems have started to give new social groups—previously without the chance to go on to post-secondary education—the opportunity to continue with formal education after finishing their secondary studies. Although these changes have had positive effects on social mobility, they have also posed important challenges to colleges, since many of these new students go into higher education without the academic and social skills needed for success at this educational level (Altbach et al., 2009; Bailey et al., 2010; Schofer & Meyer, 2005).
In this scenario, government bodies and colleges have created different initiatives to support disadvantaged students and help them complete their post-secondary studies (Archer, 2007). For example, several institutions have introduced counseling services and remedial courses to support students as they transition to college life (Hagedorn et al., 1999; Harrison & Waller, 2017; Venegas-Muggli et al., 2019).
In this context, a specific strategy that has emerged in higher education institutions to support new students adjusting to college involves peer-mentoring programs. Commonly understood as support programs in which more experienced students support first-year ones with the academic, social and emotional demands of college, several institutions have implemented these type of initiatives (see Fox et al., 2010; Honkimäki & Tynjälä, 2018; Lane, 2018; Leidenfrost et al., 2014). Moreover, it has been argued that these types of support programs are particularly important to diverse and underrepresented groups, who face greater challenges when adapting to post-secondary school life (Budge, 2006; Crisp & Cruz, 2009; Rios-Ellis et al., 2015).
On this basis, this article aims to evaluate the impact of a peer-mentoring program on underrepresented students’ academic success. For this purpose, a peer-mentoring program implemented at INACAP—one of Chile’s largest higher education institutions, mainly offering two-year technical degrees—was used. INACAP is also a non-selective institution with no entry requirements, which means it enrolls a significant number of underrepresented social groups, especially first-generation students from low and middle-income families.
This research aims to contribute to the literature on the subject by considering a methodologically rigorous design based on quasi-experimental methods, in a context in which reviews on mentoring research have agreed on the need to use better research designs to measure the effects of these types of initiative (Crisp & Cruz, 2009; Gershenfeld, 2014; Jacobi, 1991). Besides, this study also contributes by focusing on underrepresented students from more socially deprived backgrounds, since almost all of the studies on mentoring have been conducted at institutions where degrees last for four years. There is a need to expand these types of studies to more diverse populations (Budge, 2006; Crisp & Cruz 2009). Finally, studying this phenomenon in the Chilean context is also relevant. The number of Chilean higher education institutions has increased importantly over the last few decades, allowing students from lower socio-economic backgrounds to go to college (MINEDUC, 2017; Organisation for Economic Co-operation Development [OECD] & The World Bank, 2009). However, this increase has affected institutions’ capacity to produce graduates, highlighting the relevance of studying the potential effects of support programs (OECD, 2017). To this effect, this study shows a relevance initiative that can support how higher education institutions face the challenges of this emerging student population.
Chile’s Higher Education System
Chile’s higher education system took on a new shape after a series of substantial reforms implemented in the 1980s during Pinochet’s dictatorship, aimed at establishing a neoliberal socio-economic model. A new higher education system was designed to align with several reforms made to pensions, employment rights, housing and health care (Brunner, 1997). Based on the propagation of new private institutions coexisting with pre-existing traditional (public and private) universities, a new way for post-secondary institutions to function was established. The cost of education was almost totally transferred to students and higher education institutions had to compete for both students and funding (Espinoza, 2008; Matear, 2006).
Based on these reforms, a new institutional framework was established for Chile’s higher education system. This was made up of five different types of institutions: (a) Traditional Public Universities, (b) Traditional Private Universities, (c) New Private Universities, (d) Technical Training Centers (TTC) and (e) Professional Institutes (PI).
This educational format did not undergo significant changes after the return to democracy in 1990. The main changes over the last 30 years have been focused on improving system regulation and increasing funding opportunities. For instance, the State Guaranteed Loan System (CAE) was created in 2005, allowing private banks to offer student loans. Also, the National Commission for Accreditation (CNA) was founded in 2006 to define new ways of evaluating and accrediting higher education institutions (Espinoza & González 2013). In more recent times, a free education policy was established in 2015 as a result of widespread student demonstrations. This new law allowed students from families in the first five income deciles to study for free at 30 universities in 2016, increasing to 32 universities, 6 TTCs, and 6 PIs in 2017 (OECD, 2017; Venegas, 2016).
The results of this system have been both positive and negative. On a positive note, access to colleges in Chile has increased considerably over the last 40 years. This has allowed new social groups to continue studying after completing secondary education (MINEDUC, 2017). However, this system has also had some equity and quality problems. Significant inequalities are still observed when analyzing student access and the capacity to complete higher education, with high dropout rates being an important challenge faced by this system (OECD, 2017; Venegas-Muggli, 2020). What is more, the new pillars of Chilean higher education have also created a highly segregated system, where students’ socio-economic background is relevant to the quality of the institutions they have access to. This is also linked to significant inequalities in Chile’s primary and secondary education, since access to higher-quality colleges depends on students’ results in the University Selection Test (PSU), which in turn are highly linked to students’ socio-economic background and school type (OECD, 2017).
Literature Review
Defining Peer-Mentoring
In the context of higher education, there is agreement on the absence of a consistent definition of mentoring (Crisp & Cruz, 2009; Lim et al., 2017). According to Nicholson et al. (2018) mentoring can broadly be defined as a voluntary relationship among individuals who are in a similar position (i.e. peers) and is based on a reciprocal sharing of experiences for the collective purpose of enhancing personal growth and professional development. As opposed to other types of support programs—such as tutoring—mentoring is assumed to have a wider scope. It not only involves supporting students by supplementing what their courses teach them, but also providing the mentees with advice, motivation, emotional support and knowledge (Colvin, 2015; Colvin & Ashman, 2010). Therefore, it is argued that mentoring promotes student engagement and lower student anxiety (Stigmar, 2016).
Concerning mentoring initiatives in higher education, an important distinction must be made between hierarchical mentoring and peer-mentoring programs. Hierarchical mentoring involves people from two different social standings, such as when students are supported by faculty members, advisers, and/or counselors (Collier, 2017). In contrast, peer-mentoring involves a relationship in which a more experienced student supports a less experienced one to help with their academic and social integration into college life (Collier, 2017; Colvin & Ashman, 2010). As highlighted by Nicholson et al. (2018, p.424), the participants in peer-mentoring have a similar level of knowledge and develop a relationship based on equality and “a reciprocal sharing of experiences for the collective purpose of enhancing personal growth and professional development”.
Another significant distinction between mentoring styles is connected to the level of formality of these initiatives. In this respect, Lim et al. (2017) argue that mentoring can range from formal and structured programs—usually within a larger organization—to informal experiences that spontaneously arise from the interaction between a more experienced person and a less experienced one. Similarly, Leidenfrost et al. (2014) state that formal mentoring relationships are specified by the aims and the structure of a mentoring program, and mentees are assigned to the mentors. On the other hand, informal mentoring relationships are unstructured and emerge of informal interactions between mentor and mentee.
Roles and Benefits of Peer-Mentoring
Having broadly defined what peer-mentoring is, it is now necessary to go into further detail about what literature has highlighted about its main roles and benefits. On this matter, Colvin’s (2015) identification of peer-mentors’ roles is a decent starting point. She argues that peer-mentors in higher education perform the following five roles: connective link, peer leader, learning coach, student advocate, and trusted friend. Being a connective link refers to the fact that mentors help mentees feel comfortable on campus and get to know campus resources. Being a peer leader involves providing leadership to the students by motivating them to study and attend class and providing examples of study habits and classroom participation. As a learning coach, they support students to help them identify their learning strengths and achieve their potential. The role of student advocate is related to helping students by liaising between them and their instructors. Finally, mentors as trusted friends imply that they are someone students can share their experiences from both inside and outside the class.
Based on these definitions of what a mentor’s role is, several authors have emphasized the benefits associated with the effective implementation of these types of initiative. On students’ non-cognitive skills, it has been argued that these initiatives positively affect mentees’ self-esteem, self-regulation, sense of collaboration, future expectations and their general social adaptation to college life (Cornelius et al., 2016; Crisp, 2010; Douglass et al., 2013; Honkimäki & Tynjälä, 2018; Lane, 2018; Yüksel & Bhadır-Yılmazb, 2019). For example, Collings et al. (2014) argue that peer-mentoring has a relevant moderating effect on students’ levels of self-esteem. They found that the self-esteem of students who were not peer-mentored decreased, while peer-mentored one’s self-esteem did not change.
Another impact of peer-mentoring is connected to students’ learning and cognitive development of what they learn in class. Regarding this issue, it is claimed that participants in these initiatives improve, for example, their critical thinking, study habit and time management skills. Likewise, mentees can reflect in more depth on the learning process and receive additional instructional support (Crisp, 2010; Douglass et al., 2013; Lim et al. 2017; ).
By receiving all these benefits through peer-mentoring initiatives, it is argued that mentees should, in turn, enhance their academic performance. Several studies have shown a positive association between being a beneficiary of peer-mentoring programs and different indicators of academic success, such as students’ grade-point average and retention rates (Collings et al., 2014; Fox et al., 2010; Leidenfrost et al., 2014). For instance, Asgari and Carter Jr (2016), in a quasi-experimental study considering psychology students, found that students that received peer mentoring obtain higher exam grades than non-mentored students.
One last issue that should be highlighted is the fact that mentoring should have a greater effect among underrepresented groups, who are also the focus of this study, since they include students from more socially deprived backgrounds. Although this subject has not been fully developed in the literature, some studies have shown the special value that these types of initiative can have on non-traditional students. Crisp (2010) conducted a study to measure the impact of a mentoring program on the success of community college students. The justification for this was that these students face greater obstacles when adapting to post-secondary education, since many of them do not live on campus, have limited opportunities to engage in social activities, work off-campus and have weaker support systems. She found that mentoring had a direct and positive impact on students’ levels of social and academic integration at their institutions. Similarly, Rios-Ellis et al. (2015) developed a peer-mentoring program for Latino students that was shown to improve academic performance and timely graduation.
In this scenario, this study aims to contribute to the literature on the subject by providing a methodologically rigorous evaluation of a peer-mentoring program for the underrepresented students as it is required to deepen on the effects these initiatives can have on them. Also, this contribution is reinforced by the fact that this phenomenon will be studied in the context of the Chilean higher education system where, even though similar initiatives have been applied, studies have been mainly descriptive (see Aranda et al., 2016; Benvenutto et al., 2018; Retamal & Espinoza, 2016).
INACAP’s Peer-Mentoring Program
INACAP’s peer-mentoring program has emerged for senior students to support freshmen, offering academic, social and administrative guidance to help the latter’s social and academic integration and adaptation to higher education. This peer-mentoring initiative was implemented in 2018 at six of INACAP’s 26 campuses countrywide. Specifically, this strategy was implemented at the following campuses: Iquique, La Serena Copiapó, Santiago (downtown), Curicó and Osorno.
Considering Lim et al.’s (2017) distinctions about the level of formality of mentoring initiatives, INACAP’s program can be described as a formal and structured initiative since it was implemented in a large organization following institutional protocols where all mentees are formally assigned to their mentors. The team in charge of the project had a national coordinator, as well as campus coordinators responsible for the program at each of the six campuses. The central coordinator organized everything that was centrally provided on campus, such as mentor training and merchandising and was also in charge of controlling and evaluating the program. For their part, campus coordinators were responsible for each mentor and mentee on their campus.
The process started with the program’s dissemination and mentor recruitment. Each campus coordinator received applications from senior students and interviewed potential candidates, finally selecting the most suitable ones. Two main criteria were used; candidate’s academic results and their social skills. Interviews were adapted to the mentor's profile, facilitating the selection process. The selected students were then trained in support mechanisms and guidance and prepared to help them in their work as mentors. In this process, a special emphasis was put on explaining mentors about the relevance of new student’s social integration to higher education. At this time, the mentor also had to plan the mentoring sessions they would have during the semester. They were expected to hold six sessions with their mentees lasting between 45 to 60 minutes, which were expected to occur on campus. This stage culminated in a qualification ceremony where the mentor was recognized as such, therefore being in a position to meet with their mentees. Each mentor could have up to five mentees, all from the same area of study.
On the other hand, mentees are chosen by prioritizing those that most require this type of help. For this purpose, a mentoring needs index was developed using the following variables: previous higher education experience, place of residence, mother’s educational level, participation in non-academic groups, access to technological tools, years since high school graduation and the student’s perception of going back to study. All this information was collected from the INACAP freshmen survey that every student answers when they complete enrolment. Students with the highest scores in this index were invited to take part in this initiative. Based on the above, participants of the program were not randomly selected but recruited considering their scores in the mentoring needs index.
With both mentors and mentees selected, the program began. Each mentor met with their mentees and each campus coordinator checked that the mentor held all of the planned mentoring sessions. In these sessions, mentors helped mentees with any difficulties they had with their social and academic integration to college, such as supplemental instruction, library use and applications for scholarships, among others.
Methodology
Participants
This study involved first-year students enrolled at INACAP in 2018 at the six campuses where the peer-mentoring program was implemented. The treatment group was formed by all of the freshmen program beneficiaries and was made up of 231 students. As previously stated, beneficiaries were non-randomly selected as participation was defined based on attributes of students associated with the need for this support program. On the other hand, the control group was made up of first-year students from the same field of study and campuses as those of the treatment group. This group was made up of 8,447 students, giving a total study sample of 8,678 students.
After setting out both comparison groups, a data set was created with data about the students’ sociodemographic and academic characteristics, taken from INACAP’s registration databases, information about students’ high school education obtained from Chilean Ministry of Education records and information provided by the national coordinator of the program evaluated.
Variables
The independent variable was a bivariate indicator that specified whether a student had participated in the peer-mentoring program during 2018 or not. Three different indicators were defined as result variables: (a) the student’s average grades from all the courses they took during the second semester of 2018 (on a scale of 1 to 7), (b) the student’s average attendance at all their courses taken during the second semester of 2018 (attendance percentage) and (c) an indicator that specified whether the student had dropped out during 2018 or not.
The analysis also considered the following control variables connected to student attributes: gender, year of high school graduation, campus, type of academic program (daytime/evening), study field, secondary education type (vocational/scientific-humanistic), secondary school administration type (public/subsidized-private) and secondary school socioeconomic level (proportion of vulnerable students). This last indicator was considered as a proxy of students’ socioeconomic level, given Chile’s high level of socioeconomic segregation in secondary education (see Valenzuela et al., 2014).
A descriptive analysis of the study’s variables is presented in Table 1 according to each comparison group. This information shows that both groups have similar attributes. The main differences concern gender and type of academic program, with beneficiaries of the peer-mentoring program more likely to be women and daytime students than those in the control group.
Descriptive Summary of Variables Considered by Comparison Group.
Data Analysis
To evaluate the impact of the implemented peer-mentoring program, a quasi-experimental design was applied as the sample was not randomly selected (Gertler et al, 2016). The effect of this strategy on the three previously defined result variables was estimated in detail using the Propensity Score Matching (PSM) methodology (Heckman et al., 1998). This method uses students’ observable attributes—defined as control variables in the previous section—to estimate the probability of them being treated. Considering these estimations, this method then compares specific results between both comparison groups. This allows for it to be estimated whether beneficiaries of the evaluated initiative perform better than students with similar characteristics who did not take part in this program.
To increase analysis reliability, two PSM algorithms were applied using StataIC 14.0 software: Nearest Neighbor Matching (NNMATCH) and Inverse Probability Weight (IPW). The first estimator pairs individuals with similar characteristics and then compares result variables with the closest characteristics. The second algorithm compares individuals and gives greater weight in the estimations to those with a higher probability of being treated. In both algorithms, the average treatment effect (ATE) was used. This parameter estimates the effect of certain initiatives or programs by considering their effect on all of the sample’s participants (Austin, 2011). These two algorithms were considered as they have been used effectively in different studies focused on evaluating the impact of educational interventions using quasi-experimental methods (Tipton & Olsen, 2018; Venegas-Muggli, 2019).
All these analyses were carried out on the whole sample. Additionally, the impact of the program on student retention rates was also calculated for specific groups, defined by the student’s gender, academic program type and institution. These estimations meant it could be seen whether the program had a greater impact on some groups than others.
Results
The following tables present results of the effect of the evaluated program on the three defined outcome variables: average grades, average attendance and dropout. Concerning the models behind these results, all estimations included the full set of control variables described in the methodology section as pairing variables. That is to say, each of these models containing all control variables presented appropriate results concerning sample size and balance. Thus, it was not necessary to exclude specific variables at any of the estimations.
In Table 2, the results of the effect of the peer-mentoring program on students’ average grades are presented. Only when the Inverse Probability Weight method is used is the program found to have a positive and significant impact, significant at a 1% level. In the case of the NNMATCH method, a positive association is estimated but is not statistically significant. In other words, when the IPW algorithm is considered, the average grades of students who were part of the peer-mentoring program are significantly higher than the grades of those who did not take part in this initiative. In terms of the magnitude of this significant effect, it is seen that mentees’ average grades are 0.17 higher (on a scale of 1 to 7) than those of students not supported by mentors.
Impact on Average Grades.
Note. n: 6,024. PSM = Propensity Score Matching; NNMATCH = Nearest Neighbor Matching; IPW = Inverse Probability Weight.
***p < 0.01. **p < 0.05. *p < 0.1.
The impact of this initiative on student retention rates is shown in Table 3. These coefficients show in detail whether students who were benefited from the peer-mentoring initiative during 2018 were more likely to continue their studies during 2019 than students who did not take part in this program.
Impact on Retention Rates.
Note. n: 7,002. PSM = Propensity Score Matching; NNMATCH = Nearest Neighbor Matching; IPW = Inverse Probability Weight.
***p < 0.01. **p < 0.05. *p < 0.1.
The results show a positive and significant association between being mentored and retention rates. Whatever the estimator considered, there is a positive impact at a 1% level. In other words, students who are part of the program are expected to have higher retention rates in their second year of study than those with similar attributes who are not. When reviewing the magnitude of the effects, both algorithms show that participants in this initiative have retention rates that are 6.8% higher than non-beneficiaries.
Finally, Table 4 shows the effects of the peer-mentoring program on student attendance levels. Both PSM algorithms show the positive and significant effect of the program on student attendance levels at a 1% level. Beneficiaries of the peer-mentoring strategy have significantly higher attendance levels than those who were not. Regarding the magnitude of the effects, it can be observed, for example, that when the Nearest Neighbor method is used, participants in the program have attendance levels that are 6.2% higher than students who are not.
Impact on Attendance Levels.
Note. n: 7,126. PSM = Propensity Score Matching; NNMATCH = Nearest Neighbor Matching; IPW = Inverse Probability Weight.
***p < 0.01. **p < 0.05. *p < 0.1.
Having examined the program’s effects on the whole student sample, the last results presented show its impact on retention rates separately by student gender, type of academic program and institution. For this analysis, only the PSM Nearest Neighbor algorithm was used.
Table 5 first shows the results by student gender. It can be seen that the program has a positive effect on both men and women. However, this effect is only statistically significant for women, at a 1% level. When only female students are considered, it is estimated that the women who participated in this mentoring program had retention rates that are 10% higher than women who did not.
Impact on Retention Rates by Gender, Type of Academic Program and Institution (NNMATCH).
TTC = Technical Training Centers; PI = Professional Institutes.
***p < 0.01. **p < 0.05. *p < 0.1.
In the case of the academic program, it is estimated that this only has a positive and significant effect on daytime students, which is statistically significant at a 10% level. Specifically, it is seen that the retention rates of daytime freshmen supported by mentors are expected to be 5% higher than daytime freshmen who were not. Finally, it is also seen that the program has different effects on student retention rates when these are examined separately according to the type of institution students attend. Specifically, it is seen that the program only has a positive and significant effect on the retention rate of students enrolled in technical-professional programs and that there is no significant impact on the retention rates of students enrolled in university programs.
Discussion and Conclusions
This study evaluated the impact of a peer-mentoring program on students’ academic performance at a higher education institution in Chile. The main conclusion from the findings is that the implemented mentoring initiative had a positive and significant impact on students’ academic results. Specifically, using the PSM quasi-experimental method, it was shown that students who benefited from this strategy had significantly higher average grades, retention rates and attendance levels than students with similar characteristics but who did not take part. Likewise, it was also illustrated that the magnitude of this program’s effects on student retention rates is greater among female, daytime students and students enrolled in technical-professional study programs.
In line with Nora and Crisp (2007), it can be argued that INACAP’s peer-mentoring program successfully improved student levels of social and academic integration into post-secondary studies. Additionally, its effectiveness seems to be the result of the correct application of the different principles highlighted in the literature on the subject: having student peers as mentors, a mentor with multiple roles (including emotional support) and a formal program structure (Collier, 2017; Colvin & Ashman, 2010; Lim et al., 2017; ). Senior students with a wide range of skills and abilities are involved in this program to support freshmen. Moreover, this initiative has a formal structure, with mentors being trained, mentees assigned to a mentor and planned meetings sessions. To this effect, the program’s structure appears to play a key role in improving student engagement with their academic environment.
On a more general level, the results reveal that expanding higher education systems benefit from support programs when encouraging the integration of new students. In this specific article, it was also shown that the mentoring program’s effects could be of a greater magnitude among underrepresented groups who go to college. This study was carried out at an institution that enrolls a significant number of first-generation students from low and middle-income families. In addition, the results indicate that the program has a greater effect on the academic results of students enrolled in technical-professional programs. On this basis, the findings reinforce previous studies that have highlighted the special value of mentoring initiatives for non-traditional students (Budge, 2006; Crisp & Cruz, 2009; Rios-Ellis et al., 2015). At the same time, this article contributes to literature on the subject, as there has been a lack of studies evaluating the effect of mentoring initiatives on underrepresented groups.
Regarding specific strategy recommendations, the evidence examined suggests that higher education institutions that are interested in supporting underrepresented students’ academic success should develop initiatives similar to the one presented in this article—in other words, a formal program led by senior students, in which mentors are required to fulfill multiple roles to support their assigned mentees. Also, these types of programs should be of special interest for daytime enrolled students, women and students enrolled in technical-professional programs as the impact of the evaluated initiative is larger among them. Most of the underrepresented students going on to higher education in systems that are expanding do not have the abilities required to adapt to this new educational level adequately. In this respect, peer-mentoring initiatives are a valid alternative to help tackle this problem, since they encourage both the cognitive and non-cognitive skills needed to improve student levels of academic and social integration into post-secondary education (Crisp, 2010; Douglass et al., 2013; Honkimäki & Tynjälä, 2018).
Finally, recommendations for further research and limitations of the study are discussed. On the first aspect, it is recommended that this study should be complemented by qualitative methods. As emphasized by Pascarella (2006), quantitative impact evaluations do not provide evidence to help understand the processes that account for the estimated effects. Hence, the results discussed should be complemented by qualitative information including the opinions of mentors, mentees and the people in charge of the program, all of whom can shed light on these findings. This will allow for a deeper understanding of both of the specific elements of the mentoring program that explain its success and whether the other elements suggested by literature could be an option that would improve its results. Additionally, qualitative material will also help understand why the program is more effective in certain groups, such as women and/or daytime students.
On limitations of the study, it is recommended to advance in further impact evaluations considering experimental designs where beneficiaries of peer-mentoring programs could be randomly assigned. Despite randomisation presents institutional barriers, it is required to advance to the use of these methods since they produce more reliable results and quasi-experimental methods have received some criticisms concerning their efficiency (King & Nielsen, 2019). In this specific study, however, the use of quasi-experimental methods was an appropriate option considering the context where the program was applied and it was shown that they could deliver reliable results.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
