The Counseling Center Assessment of Psychological Symptoms-62 (CCAPS-62) and College Academic Outcomes

Abstract

College students experiencing psychological distress are at risk for negative academic outcomes. The Counseling Center Assessment of Psychological Symptoms-62 (CCAPS-62) is a symptom inventory designed for and widely used in college counseling centers. However, the relationships between the CCAPS-62 and functional outcomes in the college environment have not been examined. This study examined the validity of the CCAPS-62 in predicting term grade point average (GPA) and dropout. Data from 297 first-year students at a university’s counseling center were analyzed using multiple regression to determine associations between CCAPS-62 subscales, term GPA, and dropout within the subsequent three academic years. Results show that academic distress was predictive of all academic outcomes in the expected directions, social anxiety was associated with higher term GPA and retention, and hostility was associated with lower term GPA and dropout. Results demonstrated support for the instrument’s predictive validity in the identification of students at academic risk.

Keywords

college student mental health academic distress college counseling center GPA dropout

Significance of the Scholarship to the Public

This study found that scores on a widely used psychological symptom inventory in college counseling centers are associated with GPA and dropout. These findings can help college mental health professionals identify students at academic risk early in the treatment process.

Psychological and emotional well-being are among the many nonintellectual factors that contribute to students’ educational success in college (Credé & Niehorster, 2012; Eisenberg et al., 2009). A range of mental health diagnoses including substance use disorders, antisocial personality disorder, anxiety and other mood disorders are associated with college nonentry and dropout (Breslau et al., 2008; Hunt et al., 2010; Kessler et al., 1995; S. Lee et al., 2009). College students who receive mental health care at college counseling centers have higher levels of academic impairment than their peers, making this population an important target for psychosocial interventions (Krumrei et al., 2010; Lockard et al., 2012). Forty-four percent of undergraduate students report that their mental health affected their academic performance in the past month (Eisenberg et al., 2007). However, surprisingly little is known about the relationships between psychological symptomology among students and key academic outcomes, such as grade point average (GPA), graduation, persistence, or retention among students seeking mental health treatment. Despite research indicating that counseling may be academically helpful to students (Choi et al., 2010), there are no standards for identifying students most at risk for undesirable educational outcomes such as poor academic performance or dropout within a counseling context. Brief, routine assessments of symptoms used at counseling centers provide a possible avenue to further understand these relationships and aid in identifying and intervening with students at risk of undesirable academic outcomes.

Counseling Center Assessment of Psychological Symptoms-62

The Counseling Center Assessent of Psychological Symptoms-62 (CCAPS-62) is a brief symptom inventory developed for use in college counseling centers (Locke et al., 2011). It contains subscales measuring eight problem areas relevant to the college setting: depression, generalized anxiety, social anxiety, academic distress, eating concerns, hostility, substance use, and family distress. The measure is free to use, integrated into popular electronic health record systems, used in over 474 college counseling centers (Center for Collegiate Mental Health [CCMH], 2017a), and is being adopted internationally (Ratanasiripong et al., 2015), making it an important instrument for continued study and validation efforts. Continued development and implementation of the CCAPS-62 is facilitated by the CCMH, a practice-research network which provides counseling centers with aggregate data on their student’s mental health and national points of comparison, while serving as a research hub utilizing clinical data from counseling centers (McAleavey et al., 2015). Early validation work on the measure included demonstrating convergent validity with well-established referent measures (Locke et al., 2011) and evaluating the extent to which high scores on certain subscales were predictive of associated diagnoses from the American Psychological Association’s (2013) Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5), which led to the development of cut score criteria (McAleavey et al., 2012). In comparison to these clinical and measurement correlates, little is known about the relationships between CCAPS-62 subscales and domains of adaptive functioning. In the college or university setting, a student’s ability to remain enrolled and work towards a degree, as well as achieve high grades in their coursework, are salient outcomes for students and other stakeholders.

Of the domains assessed by the CCAPS-62, the Academic Distress subscale is ostensibly most related to academic outcomes. The Academic Distress subscale consists of five items pertaining to enjoyment of classes, self-confidence in academic success, concentration, motivation, and ability to keep up with schoolwork. The subscale is strongly correlated with the more heavily-researched academic adjustment scale of the Student Adaptation to College Questionnaire (McAleavey et al., 2012). However, the developers of the CCAPS-62 acknowledge that the Academic Distress subscale may not directly translate to performance measures such as GPA (Lockard et al., 2012). For example, someone may not enjoy their classes, but still earn high marks in them. As such, the utility of the Academic Distress subscale is unclear. Given widespread use of the CCAPS-62 and the academic context of college counseling centers, increased clarity about what inferences can be drawn from the Academic Distress subscale score is of particular interest to clinicians. Examining the predictive validity of the Academic Distress subscale for relevant academic outcomes will provide a basis for its meaningful, accurate, and reliable interpretation.

MacFarlane et al.’s (2015) comparison of three symptom inventories at a college counseling center generated uncertainty about the predictive validity of the Academic Distress subscale. They found that the Academic Distress subscale had only a small (.21) correlation with the Academic Problems subscale of the College Adjustment Scales (CAS), and a much stronger (.71) correlation with the Family Problems subscale of the CAS. Their findings highlighted the possibilities of differences between measures of academic stress and the need for validation of the Academic Distress subscale (and other academic stress scales) against a set of objective external criteria. Educational outcomes have been associated with many forms of symptomology and comorbidity in nonclinical samples, although not in a consistent fashion, depending on the specific measure used (Arria, Caldeira et al., 2013; Breslau et al., 2008). Given its wide use, further validation and expansion of the utility of the CCAPS-62 for predicting education outcomes would be beneficial.

College Success and Mental Health Symptoms

A variety of other symptom inventories measuring traditional domains of psychopathology have been used to examine college student educational outcomes in conjunction with mental health, with mixed findings overall. In a landmark longitudinal study by Eisenberg et al. (2009), the Patient Health Questionnaire-9 (PHQ-9), a measure of depression, the PHQ panic and generalized anxiety screeners, and the SCOFF measure of disordered eating were administered to 2,798 college students. They found that depression was negatively associated with GPA and retention to the university, such that a score of 15 points on the PHQ-9 was associated with a 4.7% increase in the likelihood of dropping out. Arria, Caldeira et al. (2013) examined the effects of scores on the Beck Depression Inventory (BDI), Beck Anxiety Inventory, history of psychiatric diagnosis, childhood conduct problems, and substance use on any gap in college enrollment. They found that BDI scores predicted enrollment interruptions only early in college, where as cannabis and alcohol use predicted discontinuity only later in college, with no effect for Beck Anxiety Inventory scores. In one clinical study, BDI scores in the moderate-to-severe range were associated with self-reported academic impairment, such as absenteeism or diminished productivity (Heiligenstein et al., 1996). In the United Kingdom, a longitudinal study of 351 first-year undergraduates taking the Hospital Anxiety and Depression Scale showed that depression, but not anxiety, was predictive of lower exam scores (Andrews & Wilding, 2004). These studies show a pronounced association between depression and college outcomes, while pointing towards a more complex relationship as it pertains to generalized anxiety.

However, other studies do not demonstrate a clear link between mental health and GPA. A 2012 meta-analysis of 42 psychological correlates of GPA found no statistically significant association between depression symptoms and GPA, although stress, academic stress, and social support had small associations (Richardson et al., 2012). A study of students with a diagnosed mental illness who received services at their school’s office for students with disabilities found them to have similar GPAs to their peers (Brockelman, 2009). This finding likely underscores the extent to which receiving appropriate accommodations, counseling, and other academic support services improve the academic outcomes of students with mental health problems (D. Lee et al., 2009; Pitre & Pitre, 2009). Therefore, assessing mental health symptoms concurrent with academic difficulties may provide a unique picture for practitioners in college counseling centers.

Previous efforts to predict academic outcomes using psychological symptom measures among college students have primarily used nonclinical samples (Arria, Caldeira et al., 2013; Eisenberg et al., 2009). Therefore, existing research provides limited generalizability to the population of students seeking mental health care, particularly in the interpretation of these constructs as measured by the CCAPS-62. Differential effects between clinical and nonclinical populations may also help explain mixed or weak findings between symptoms such as depression and academic outcomes (Richardson et al., 2012).

Colleges typically provide counseling center services with the assumption that students are being helped academically as well as emotionally (Choi et al., 2010; Sharkin, 2004). Research that informs strategies of identifying students at academic risk in counseling centers may help with this mission. Establishing the predictive validity of the Academic Distress subscale of the CCAPS-62 through investigating whether it predicts academic outcomes can inform clinicians’ interpretation of the measure and clinical decision-making. Generally expanding knowledge of the relationships between mental health symptoms and academic outcomes can be used to inform treatment, targeting areas of distress that are most likely to impede a student’s academic functioning.

Current Study

The current study examined the associations between the CCAPS-62, a widely used clinical measure at college counseling centers, and key academic outcomes of term GPA and dropout from college. This study answers two primary research questions. Is the Academic Distress subscale of the CCAPS-62 associated with term GPA and dropout? Do the other seven subscales of the CCAPS-62 make an additional contribution to predicting term GPA and dropout over and above the Academic Distress subscale? It was hypothesized that Academic Distress subscale scores would have a significant negative association with term GPA and significant positive associations with dropout from the university at one, two, and three years following CCAPS-62 administration (i.e., second-year, third-year, and fourth-year dropout, respectively).

Methods

Participants

Participants were 297 degree-seeking, first-year, undergraduate students seeking psychological services who attended an intake session at a student counseling center at a university in the Pacific Northwest during the 2014–2015 academic year. Most (99.0%) of participants were 18–19 years old, 66.6% were female, 3.7% were international students, 68.8% identified their racial and/or ethnic identity as White, 10.2% were Hispanic/Latino/a, 9.2% were Asian American/Asian, 6.1% multi-racial, 1.4% American Indian/Alaskan Native, 1.4% Native Hawaiian or Pacific Islander, 1.0% African American/Black, and 2.0% identified with some other racial–ethnic identity. Nearly 21% reported that they were the first generation in their family to attend college.

Procedure

Archival clinical records and student academic data were utilized for this study. Approval to utilize pre-existing data and protected health information was obtained from the university institutional review board. Students were administered the CCAPS-62 and a demographic and background questionnaire on computers using the Titanium electronic health records system prior to the initial session. Students also consented to treatment and to have aggregate deidentified data used for potential research purposes at this visit. Access to archival student academic data variables was provided by the university’s Office of the Registrar. An assessment specialist at the counseling center (H. Samlan) matched student academic records with clinical records on the basis of secondary student identification numbers to create a deidentified study data set.

Measures

CCAPS-62

The CCAPS-62 is a 62-item measure of common psychological symptoms (Locke et al., 2011). Respondents “indicate how well each statement describes you, during the past two weeks” with partially labeled numeric response options ranging from 0 (not at all like me) to 4 (extremely like me). Higher scores indicate higher levels of psychological distress. Cronbach’s alpha coefficients presented here are based on the 2012–2014 CCMH sample from participating counseling centers (N = 142,560; CCMH, 2015) and are similar to those found in the current sample (see Table 1). The eight subscales, Depression (13 items, α = .92), Generalized Anxiety (9 items, α = .85), Social Anxiety (7 items, α = 0.84), Academic Distress (5 items, α = 0.82), Eating Concerns (9 items, α = 0.89), Family Distress (6 items, α = 0.83), Hostility (7 items, α = 0.86), and Substance Use (6 items, α = 0.85), have established convergent validity with some well-established referent measures, strong factor structure, and internal consistency reliability. However, the Academic Distress subscale shows only a small correlation (.21) with the Academic Problems subscale on the CAS (MacFarlane et al., 2015).

Table 1.

Descriptive Statistics and Internal Consistency Reliability for CCAPS-62

	M	SD	%	α
CCAPS-62
Depression	1.62	0.97	–	.92
Generalized Anxiety	1.68	0.98	–	.87
Social Anxiety	1.98	0.95	–	.84
Academic Distress	1.71	0.98	–	.82
Eating Concerns	1.06	0.91	–	.90
Family Distress	1.27	0.92	–	.82
Hostility	0.94	0.81	–	.83
Substance Use	0.75	0.89	–	.87
Academic Outcomes
Term GPA^a	2.99	0.82	–	–
Second-year dropout	–	–	15.3	–
Third-year dropout	–	–	23.7	–
Fourth-year dropout	–	–	29.2	–
Treatment
Treatment sessions within term	1.77	2.26	–	–
Any treatment sessions	–	–	73.2	–
Time (weeks)
Between CCAPS and end of term	5.57	2.82	–	–
Between CCAPS and end of academic year	19.91	9.84	–	–

Note. n = 295. CCAPS-62 = Counseling Center Assessment of Psychological Symptoms-62; α = Cronbach’s alpha; GPA = grade point average.

n = 282.

Demographics

Students reported their age, gender, racial and/or ethnic identity, and first-generation college student status on the counseling center paperwork. International student status was obtained from university enrollment records.

GPA

Term GPA was calculated for the quarter in which the student took the CCAPS-62. Courses were graded on a scale of 0–4.3, with 0 indicating an F and 4.3 corresponding to an A+. Only grades from the term in which the student took the CCAPS were used in calculation of term GPA.

Dropout

Students not enrolled for any reason in the fall term each of the three years following their initial session in 2014–2015, and who did not graduate or enroll in any subsequent term, were considered to have dropped out at that time point. As such, second, third, and fourth year dropout refers to having dropped out by fall of the second, third, or fourth year, respectively. At the Fall 2017 time point, any student not enrolled in courses who had not graduated was considered to have dropped out from the college. Dropout was dummy coded (0 = not dropped out, 1 = dropped out) such that positive coefficients indicate higher likelihood of dropout from the university.

Data Analyses

Consistent with CCAPS-62 scoring guidelines (CCMH, 2015), items were reverse scored as appropriate and averaged to compute subscale scores. Pearson and point-biserial correlations were calculated to estimate zero-order associations between CCAPS-62 subscales, term GPA, and second-year, third-year, and fourth-year dropout.

Hierarchical multiple regression was employed to determine the associations between the CCAPS-62 subscales and the academic outcomes of term GPA (linear regression) and dropout (logistic regression), while adjusting for effects of treatment and time. For analyses of term GPA, the number of therapy sessions a student attended that same term, subsequent to the initial assessment session, served as a treatment covariate. Because a student’s cumulative counseling center therapy attendance is conditionally dependent on them not dropping out from the college, dropout analyses included a dichotomous (0 = no treatment, 1 = any treatment) control variable to represent any attendance at therapy sessions beyond an initial assessment. To adjust for possible effects of time within the term or time within the academic year on academic outcomes, the number of weeks between when a student took the CCAPS-62 and the end of the term was used as a covariate for analysis of term GPA, and the number of weeks between CCAPS-62 administration and the end of the academic year was used as a covariate in the dropout analyses.

Four sets of primary analyses correspond to the outcomes of interest: term GPA, and second-year, third-year, and fourth-year dropout. For all hierarchical regression analyses, Academic Distress subscale score, a treatment covariate, and a time covariate were entered first. The other seven subscales were added in the next model. Assumptions for multiple regression were met, collinearity statistics were within acceptable limits, and residual plots were examined for normality, linearity, and homoscedasticity. A few extreme outliers for total treatment were present in the data. Because the relationship between therapy dose and outcomes are nonlinear at higher levels (Baldwin et al., 2009, four values for total term treatment were replaced by the highest present value within three standard deviations of the mean value of treatment attendance.

Results

Missing Data

Two participants were removed from analyses. One was missing a student identification number and one was missing two CCAPS-62 subscale scores, yielding 295 cases for analysis.

Descriptive Data

Table 1 displays descriptive statistics for all study variables and internal consistency reliability statistics for CCAPS-62 subscales. Table 2 displays correlations between CCAPS-62 subscales, term GPA, and dropout at all three time points. Most CCAPS-62 subscales had significant positive intercorrelations, with the exception of a small, significant negative correlation (−.14) between Social Anxiety and Substance Use. The Academic Distress subscale had significant small correlations with all academic outcomes in the hypothesized directions, with absolute correlations ranging from .21 to .24.

Table 2.

Correlations Among CCAPS-62 Subscales, Term GPA, and Dropout

	1	2	3	4	5	6	7	8	9	10	11	12
1. DEP	—
2. ANX	.71***	—
3. SA	.68***	.58***	—
4. AD	.61***	.48***	.41***	—
5. EC	.47***	.34***	.30***	.36***	—
6. FD	.44***	.39***	.25***	.31***	.29***	—
7. HOS	.59***	.44***	.35***	.43***	.30***	.45***	—
8. SUB	.12*	.03	−.14*	.19**	.27***	.09	.26***	—
9. tGPA^a	−.05	.01	.10	−.24***	−.06	−.07	−.22***	−.13*	—
10. DO Y2	.09	.04	−.01	.24***	.04	.06	.13*	−.02	−.35***	—	—	—
11. DO Y3	.11	.05	.03	.22***	.02	−.01	.15**	−.03	−.33***	—	—	—
12. DO Y4	.10	.03	.03	.21***	−.01	.00	.11	−.04	−.36***	—	—	—

Note. n = 295. CCAPS-62 = Counseling Center Assessment of Psychological Symptoms-62; DEP = Depression; ANX = Generalized Anxiety; SA = Social Anxiety; AD = Academic Distress; EC = Eating Concerns; FD = Family Distress; HOS = Hostility; SUB = Substance Use; tGPA = Term GPA; DO = dropout at time points Y2 (second-year), Y3 (third-year), and Y4 (fourth-year); GPA = grade point average. Dropout coded as 1 = dropped out, 0 = not dropped out.

n = 282.

p < .05. **p < .01. ***p < .001.

To check whether and how proposed covariates of treatment and time were related to predictors and outcomes of interest, correlations were calculated between covariates and the CCAPS-62 subscales and between covariates and academic outcomes. The Depression, Generalized Anxiety, Social Anxiety, and Family Distress scales had significant small-to-moderate positive relationships with the total number of treatment sessions a student attended within the academic term they took the CCAPS-62. Scores on Eating Concerns, Hostility, Substance Use and Academic Distress scales were not associated with the number of treatment sessions attended. No significant associations were found between any of the eight CCAPS-62 scales and the time of the academic term or time of academic year at which a student took the CCAPS-62. No significant associations were found between any of the academic outcomes (term GPA, second-year, third-year, or fourth-year dropout) and treatment or time.

CCAPS-62 and GPA

To examine the relationships between the CCAPS-62 subscales and term GPA, hierarchical linear regressions were conducted. Table 3 presents the results of both models for term GPA. Model 1, with Academic Distress, explained 6.9% of the variance in term GPA, with a one-point increase in the Academic Distress score associated with a 0.22-point decrease in term GPA. The eight subscales together in Model 2 accounted for 14.7% of the variance in term GPA, revealing that the other subscales add predictive value beyond Academic Distress. Other than Academic Distress, only two other CCAPS-62 subscales had significant independent associations with term GPA. Social Anxiety was positively related to term GPA such that a one-point increase in Social Anxiety was associated with a 0.18-point increase in term GPA. Hostility was negatively related to term GPA, with a one-point increase in Hostility associated with a 0.27-point decrease in term GPA. Additionally, examination of the regression coefficients for Academic Distress indicate a minor suppression effect, as the beta coefficient increased as the other CCAPS-62 subscales were added to the model (Pandey & Elliot, 2010).

Table 3.

Summary of Hierarchical Regression Analysis for CCAPS-62 Subscales Predicting Term GPA

	Model 1				Model 2
	B	SE B	β	p	B	SE B	β	p
Time	−0.02	0.02	−.08	.249	−0.02	0.02	−.06	.380
Treatment	0.05	0.02	.13	.049	0.03	0.03	.08	.258
Academic Distress	−0.22	0.05	−.25	<.001	−0.26	0.06	−.30	<.001
Depression					0.06	0.10	.07	.537
Generalized Anxiety					0.06	0.07	.07	.425
Social Anxiety					0.18	0.07	.20	.015
Eating Concerns					−0.01	0.06	−.01	.829
Family Distress					0.03	0.06	.03	.617
Hostility					−0.27	0.08	−.26	<.001
Substance Use					0.02	0.06	.03	.695
Model Statistics
R²		.07				.15
F for ΔR²		6.84		<.001		3.57		.001

Note. N = 282. Time is the number of weeks between Counseling Center Assessment of Psychological Symptoms-62 (CCAPS-62) administration and the end of the academic term. Treatment is the total number of therapy sessions attended during that term. GPA = grade point average.

Dropout

Three separate hierarchical logistic regressions were conducted to examine the associations between the CCAPS-62 subscales and second-year, third-year, and fourth-year dropout. In Model 1, with Academic Distress and control covariates, Academic Distress significantly predicted dropout all three years. In Model 1, with Academic Distress and control covariates, Academic Distress significantly predicted dropout all three years (second-year: p < .001, OR = 2.03, 95% CI [1.43, 2.88]; third-year: p < .001, OR = 1.71, 95% CI [1.28, 2.29]; fourth-year: p < .001, OR = 1.61, 95% CI [1.23, 2.11]). Table 4 displays the odds ratios, confidence intervals, and select model statistics from Model 2 for second-year, third-year, and fourth-year dropout. Nagelkerke’s R² indicated that Model 2, adding the other seven CCAPS-62 subscales, accounted for 17.4%, 13.7%, and 10.8% of the variance in second-year, third-year, and fourth-year dropout, respectively. Results indicate that the overall predictive power for the CCAPS-62 drops off after year three, although Academic Distress alone remained a significant predictor of dropout at that time point. Increases in Social Anxiety were associated with decreased odds of second-year dropout, while Hostility was associated with decreased odds of third-year dropout.

Table 4.

Summary of Model 2 Hierarchical Regression Analyses for CCAPS-62 Subscales Predicting Dropout

	Second-Year Dropout			Third-Year Dropout			Fourth-Year Dropout
	OR	95% CI	p	OR	95% CI	p	OR	95% CI	p
Time	1.04	1.01, 1.08	.026	1.03	0.99, 1.06	.076	1.02	0.99, 1.04	.296
Treatment	1.24	0.56, 2.78	.599	1.03	0.54, 1.98	.922	0.81	0.45, 1.46	.482
Academic Distress	2.50	1.60, 3.92	<.001	1.94	1.34, 2.81	<.001	1.87	1.32, 2.64	<.001
Depression	1.09	0.57, 2.11	.793	1.21	0.69, 2.12	.508	1.18	0.70, 1.99	.539
Generalized Anxiety	0.79	0.47, 1.33	.384	0.83	0.54, 1.28	.406	0.82	0.55, 1.22	.331
Social Anxiety	0.58	0.35, 0.97	.037	0.72	0.47, 1.10	.125	0.80	0.54, 1.19	.271
Eating Concerns	0.98	0.64, 1.51	.933	0.90	0.62, 1.29	.556	0.85	0.60, 1.20	.355
Family Distress	0.87	0.57, 1.33	.514	0.70	0.48, 1.02	.060	0.79	0.57, 1.12	.183
Hostility	1.57	0.93, 2.64	.090	1.70	1.08, 2.66	.021	1.40	0.92, 2.13	.121
Substance Use	0.69	0.45, 1.07	.100	0.73	0.51, 1.05	.085	0.74	0.53, 1.04	.085
Model Statistics
χ²		31.09	.001		28.25	.002		23.32	.002
Nagelkerke R²		.17			.14			.11

Note. N = 295. CCAPS-62; Counseling Center Assessment of Psychological Symptoms-62; Time = weeks between CCAPS-62 administration and the end of the academic year. Treatment is coded 0 = no treatment 1 = attended at least one treatment session.

In order to translate these findings into probabilities, marginal effects were calculated from Model 2 results, with the covariate of treatment at zero and other variables set to the distribution of observed values within the sample (Muller & MacLehose, 2014). These show that a one-point increase in Academic Distress is associated with a 9.5, 10.7, and 12.5 percentage-point increase in the cumulative risk of second-year, third-year, and fourth-year dropout, respectively. A one-point increase in Social Anxiety is associated with a 5.6 percentage-point decrease in the risk of second-year dropout and a one-point increase in Hostility is associated with an 8.5 percentage-point increase in the risk of third-year dropout.

Discussion

The purpose of this study was to examine the associations between the CCAPS-62 and college academic outcomes within a sample of counseling center clients. Particular emphasis was given to informing the validity of inferences made from scores on the Academic Distress subscale, due its ostensible connections with term GPA and dropout. Results from logistic and linear regression analyses revealed that (a) academic distress was a significant predictor of all outcomes; (b) social anxiety was associated with better short-term academic outcomes, higher term GPA, and lower risk of second-year dropout; and (c) hostility was associated with lower term GPA and higher risk of third-year dropout.

Academic Distress

This is the first study to examine the relationships between academic distress and any objective measure of academic performance or success. The hypothesis that academic distress would be negatively associated with term GPA and positively associated with dropout was supported by the results. The findings align with previous studies showing associations between other measures of academic stress and educational outcomes among college students (Akgun & Ciarrochi, 2003; Baker, 2002; Struthers et al., 2000). Academic distress was the strongest predictor overall of academic outcomes, and the only subscale predictive of fourth-year dropout. In the sample, roughly half of the dropout occurred after the first year, consistent with other studies showing higher dropout risk between first and second years (Chen, 2012). Because a wide variety of personal and institutional factors influence student persistence and dropout decisions (Reason, 2009), it is noteworthy that academic distress remained predictive of dropout long after the CCAPS-62 was administered.

This study lends initial support for interpreting the Academic Distress subscale scores as potentially indicative of both immediate and long-term academic difficulties, rather than mere subjective distress. The Academic Distress subscale of the CCAPS-62 is a distinguishing characteristic of the tool for use with college students when compared to symptom inventories used in other adult mental health settings. Previous studies among counseling center clients raised questions about the predictive utility of the Academic Distress subscale due to its weak associations with the Academic Problems subscale of the CAS (MacFarlane et al., 2015). As the validity of the Academic Problems subscale for predicting academic outcomes has not been established, the results presented here make the CCAPS-62’s subscale of Academic Distress a preferable measure.

Social Anxiety

Surprisingly, scores on the Social Anxiety subscale were found to be associated with higher term GPA and decreased likelihood of second-year dropout. No studies to date have found positive associations between social anxiety and functional outcomes or quality of life indicators. Indeed, the extant literature reports negative or null associations. Classroom participation, public presentations, study groups, and attending faculty office hours are among the academic areas that can be particularly challenging for an individual experiencing social anxiety (Russell & Shaw, 2009; Russell & Topham, 2012). Studies of adult clinical and community samples find that social anxiety disorder in particular is associated with a range of functional impairments, including occupational and student role functioning and high school noncompletion (Aderka et al., 2012; Stein & Kean, 2000). Although there have not been studies among counseling center clients specifically, studies of social anxiety among college students report either no associations (Strahan, 2003; Topham & Moller, 2011) or negative associations (Brook & Willoughby, 2015) between social anxiety and academic outcomes. Because the Social Anxiety subscale results in the present study are contrary to theoretical expectations based on the empirical record, and there was a substantial suppression effect regarding social anxiety (i.e., full-model associations were substantially larger than zero-order associations), the significant associations between social anxiety, term GPA, and second-year dropout may be spurious.

Hostility

Hostility was found to be associated with lower term GPA and higher risk of dropout within two years following CCAPS-62 administration, supporting the practical utility of the CCAPS-62 measure of hostility as an indicator of academic risk among counseling center clients. The Hostility subscale of the CCAPS-62 primarily assesses the internal states of irritability, anger, violent and aggressive impulses, along with externally observable argumentative behaviors. Across diverse samples, hostility has been inversely correlated with educational attainment (Elbogen et al., 2010; Scherwitz et al., 1991). Additionally, mental health problems that are marked by irritability or anger, including bipolar disorder and antisocial personality disorder, are associated with poorer academic outcomes (Breslau et al., 2008; Hunt et al., 2010; King, 2000). Although not a formal symptom, anger and hostility commonly co-occur with unipolar depression (Koh et al., 2002, Posternak & Zimmerman, 2002), the combination of which is associated with greater chronicity, psychosocial impairment, and psychiatric comorbidities (Judd et al., 2013). Although hostility has rarely been examined as a construct of relevance to academic outcomes among college students, some studies have shown relationships between hostility and lower GPA and interruptions in enrollment (Arria, Garnier-Dykstra, et al., 2013; Trockel et al., 2000). Scores on the Hostility subscale may be indicative of specific and particularly impairing disorders, as well as a characteristic that is associated with greater risk, including academic impairment, independent of a given diagnosis.

Other Domains

Results showed that depression, substance use, eating concerns, generalized anxiety, and family distress were not associated with term GPA or retention. Of these domains, depression has been the most heavily researched in connection with college outcomes. This study adds to a body of literature calling this relationship into question (Breslau et al., 2008; Hunt et al., 2010; Richardson et al., 2012). One previous study (Eisenberg et al., 2009) showed that the PHQ-9 item measuring anhedonia was uniquely predictive of GPA, a domain not assessed by the CCAPS-62 Depression subscale, which may also account for the present findings. These disparate findings also serve as a reminder of the importance of validation research on different symptom inventories, even when measuring the same underlying construct. Only a small zero-order association between substance use (which primarily measures alcohol use) and term GPA was found, indicating that the scale does not uniquely capture academic risk within the CCAPS-62. This finding may underscore the unique and complex role of alcohol use among college students (Merrill & Carey, 2016), and adds to a literature that includes many mixed and null findings with relation to alcohol use and academic outcomes (Arria, Caldeira et al., 2013; White & Hingson, 2013).

Limitations

The current findings should be considered within the context of relevant limitations to the study. First, there are two factors that limit the generalizability of the results. Although the first-year student sample examined provided the ability to track retention and term GPA for an extended time, first-year students also face higher levels of academic risk, including higher risk of dropout following the first year (Chen, 2012), and unique stressors associated with the adjustment to the college environment, which may affect relationships between CCAPS-62 scores and academic variables. Therefore, this limits generalizability to students in different academic years or graduate students. The study also utilized a sample taken from a single large public research university in the Pacific Northwest, and therefore may not generalize to all other institutional settings.

Although attempts were made to control for confounding effects of treatment, this is challenging to accomplish with existing clinical data and without the use of a control group. Neither the cumulative and dichotomous covariates used in analyses to indicate the amount or presence of counseling services received accounted for pertinent factors such as the type of interventions delivered, or the amount of clinical improvement, which are likely to affect the relationship between CCAPS-62 scores at initial assessment and later functional outcomes. Nor were nonclinical academic support services, such as tutoring, accounted for. As a result, it is possible that nonsignificant relationships between certain symptoms and academic outcomes, such as more distal dropout, are due in part to successful treatment in the intervening period, that in turn, reduced academic risk. Additionally, treatment was accounted for only if it was received at the university counseling center itself. Consequently, students who received psychiatric medication at the university health center or therapy elsewhere after their initial assessment are considered as not receiving treatment in the study. This could partially account for the minimal effect of treatment covariates in models. Other possible confounders omitted include demographic variables such as first-generation college student status and racial–ethnic minority identity (e.g., African American, American Indian), which are associated with higher rates of drop out (Espinosa et al., 2019; Ishitani, 2003) and may be associated with experiencing greater academic distress.

Implications for Practice, Advocacy, Education/Training, and Research

Academic functioning outcomes are of particular salience to college counseling centers. This study provides initial evidence for the validity of making inferences regarding academic outcomes from scores on the CCAPS-62. The effect sizes in the present study (e.g., 0.26-point decrease in term GPA associated with one-point increase in Academic Distress) are clinically relevant for students. A quarter of a GPA point drop is substantial enough to have potential real-world consequences, such as being put on academic probation, making the dean’s list, qualifying for a scholarship, or being admitted to graduate school. Similarly, the 5%–10% changes in the risk of dropout based on CCAPS-62 scores found in this study translate to a meaningful amount of student dropout at the university level.

Clinicians who are provided with more data about a student’s level of academic risk should be able to harness this information in the assessment, triage, referral, and treatment planning processes. The utility of the present findings for counseling centers will depend on whether the CCAPS-62 is an improvement over existing practices for assessing academic risk, and the usefulness of actions taken as a result of inferences made. There are currently no accepted best practices for identifying academic functioning or risk within college counseling center settings. Known practices range from asking students to self-report their GPA at intake (Center for Collegiate Mental Health, 2020), asking students if they are considering dropping out (Van Brunt, 2008), and directly accessing and reviewing students’ academic records. As a result of the dearth of research in this area, reviewing CCAPS-62 scores may better help counseling center staff identify students who are at increased academic risk. The findings of the full models in this study also demonstrate that other CCAPS-62 subscales provide more information about academic functioning and risk above and beyond the Academic Distress subscale.

We recommend that Academic Distress, Social Anxiety, and Hostility scores be examined together, in conjunction with other data collected in the intake process. High scores on the Hostility and Academic Distress subscales may help alert clinicians to conduct further assessment of a student’s academic needs, which may include exploring their educational history, recent academic performance, academic self-efficacy, learning challenges or disabilities, and current use of academic supports and services.

The results from this study can be used in training college counseling center clinicians to interpret Academic Distress scores as associated with objective risk, rather than simply subjective distress. Early identification of academic risk and engaging in secondary prevention practices are consistent with the values and aims of counseling psychology as a specialty (e.g., Romano & Hage, 2000; Scheel et al., 2018) and the mission of many college counseling centers (e.g., Brunner et al., 2017). Consistent with this mission, CCAPS-62 scores on both Academic Distress and Hostility tend to improve over time with treatment at a counseling center (Ghosh et al., 2017; Lockard et al., 2012). Findings from the present study may be used to support advocacy for investing more counseling center resources in prevention efforts, as counseling centers will be better able to connect their work with the academic mission of an institution. For counseling centers that focus on clinical intervention to the exclusion of preventative interventions (Brunner et al., 2017), development of efficient referral pathways in collaboration with student academic support services on campus may benefit students scoring high on Academic Distress in the absence of other clinically significant scores.

Future research can build on these findings in ways that increase the accuracy and utility of predictive information about academic risk, and that illuminate the causal mechanisms behind these predictions. Adding other psychosocial and demographic information that is traditionally collected at counseling centers to predictive models may improve the ability to detect academic risk. Student background domains such as first-generation student status (Cataldi et al., 2018), nontraditional student status (Kamer & Ishitani, 2019), learning disabilities (Troiano et al., 2010), psychiatric history (Breslau et al., 2008, Hunt et al., 2010), socio-economic status (Reason, 2009), financial stress (Joo et al., 2008), and racial and/or ethnic identity (Musu-Gillette et al., 2017), are factors which may influence students’ academic risk or inform intervention strategies and should be studied. It would also be useful to examine whether the CCAPS-62 is predictive of academic outcomes above and beyond traditional predictors used in admissions, such as high school GPA or standardized test scores, to understand what unique aspects of student functioning are being captured by the CCAPS-62. Using these or other premorbid measures of functioning, including assessments of precollege mental health, would also further research into the causal relationships between distress, GPA, and retention among college counseling center clients. Examining mental health, academic ability, and academic outcomes over time in a way that allows for the examination of reciprocal relationships, such as cross-lagged designs, is needed for a more accurate look into how the reciprocal process of worsening mental health and academic outcomes unfolds. Such investigations should also include attention to use of adjunctive services that support student academic success, in order to tease out the potential benefits of such services from the benefits of counseling center services. Finally, identifying students at risk of low GPA or dropout is helpful only to the extent that intervention improves outcomes. Future studies are needed to examine questions pertaining to downstream effects of the identification process. It is important to know, therefore, if inferences regarding academic risk from the CCAPS-62 can lead to changes in clinical practice that improve student outcomes.

Summary and Conclusion

The purpose of this study was to examine relationships between Academic Distress and other subscales of the CCAPS-62 and academic outcomes among university counseling center clients. Findings revealed that the Academic Distress and Hostility subscales were associated with lower term GPA and increased dropout, while Social Anxiety was associated with higher term GPA and retention. The magnitude of these relationships is likely to be meaningful to students and other campus stakeholders. This study also provides initial validity evidence for interpreting scores on the Academic Distress subscale as indicative of objective academic difficulties and suggests that using multiple CCAPS-62 subscale scores together may aid counseling centers in individualizing services to students with greater academic needs.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Hillel Samlan

Author Biographies

Hillel Samlan, PhD, is a staff psychologist providing services within an inpatient psychiatric unit at the Portland VA Medical Center . He previously conducted program evaluation as an assessment specialist at the University of Oregon Counseling Center.

Ellen Hawley McWhirter, PhD, is the Ann Swindells professor in counseling psychology at the University of Oregon and director of the Spanish Language Psychological Services and Research Specialization. Her scholarly interests include Latinx adolescent career development, critical consciousness, and adolescent risk behavior. She is a fellow of the Society for Counseling Psychology and former chair of the Society for Vocational Psychology.

Brian A. M. Clark, PhD, is assistant director of assessment and research for the Division of Student Life at the University of Oregon, where he conducts applied psychological research on undergraduate wellbeing and success and develops administrative infrastructure to support data science operations.

References

Aderka

I. M.

Hofmann

S. G.

Nickerson

Hermesh

Gilboa-Schechtman

Marom

(2012). Functional impairment in social anxiety disorder. Journal of Anxiety Disorders, 26(3), 393–400. https://doi.org/10.1016/j.janxdis.2012.01.003

Akgun

Ciarrochi

(2003). Learned resourcefulness moderates the relationship between academic stress and academic performance. Educational Psychology, 23(3), 287–294. https://doi.org/10.1080/0144341032000060129

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.).

Andrews

Wilding

J. M.

(2004). The relation of depression and anxiety to life-stress and achievement in students. British Journal of Psychology (London, England: 1953), 95(Pt 4), 509–521. https://doi.org/10.1348/0007126042369802

Arria

A. M.

Caldeira

K. M.

Vincent

K. B.

Winick

E. R.

Baron

R. A.

O’Grady

K. E.

(2013). Discontinuous college enrollment: Associations with substance use and mental health. Psychiatric Services, 64(2), 165–172. https://doi.org/10.1176/appi.ps.201200106

Arria

A. M.

Garnier-Dykstra

L. M.

Caldeira

K. M.

Vincent

K. B.

Winick

E. R.

O’Grady

K. E.

(2013). Drug use patterns and continuous enrollment in college: Results from a longitudinal study. Journal of Studies on Alcohol and Drugs, 74(1), 71–83.

Baker

R. W.

(2002). Research with the Student Adaptation to College Questionnaire (SACQ). https://www.mtholyoke.edu/courses/shilkret/baker/

Baldwin

S. A.

Berkeljon

Atkins

D. C.

Olsen

J. A.

Nielsen

S. L.

(2009). Rates of change in naturalistic psychotherapy: Contrasting dose–effect and good-enough level models of change. Journal of Consulting and Clinical Psychology, 77(2), 203–211. https://doi.org/10.1037/a0015235

Breslau

Lane

Sampson

Kessler

R. C.

(2008). Mental disorders and subsequent educational attainment in a US national sample. Journal of Psychiatric Research, 42(9), 708–716. https://doi.org/10.1016/j.jpsychires.2008.01.016

10.

Brockelman

K. F.

(2009). The interrelationship of self-determination, mental illness, and grades among university students. Journal of College Student Development, 50(3), 271–286.

11.

Brook

C. A.

Willoughby

(2015). The social ties that bind: Social anxiety and academic achievement across the university years. Journal of Youth and Adolescence, 44(5), 1139–1152. https://doi.org/10.1007/s10964-015-0262-8

12.

Brunner

Wallace

Keyes

L. N.

Polychronis

P. D.

(2017). The comprehensive counseling center model. Journal of College Student Psychotherapy, 31(4), 297–305. https://doi.org/10.1080/87568225.2017.1366167

13.

Cataldi

E. F.

Bennett

C. T.

Chen

Simone

S. A.

(2018). First-generation students: College access, persistence, and postbachelor’s outcomes (NCES 2018-421). National Center for Education Statistics. https://files.eric.ed.gov/fulltext/ED580935.pdf

14.

Center for Collegiate Mental Health (2015). CCAPS User Manual. University Park, PA.

15.

Center for Collegiate Mental Health. (2017, January). 2016 Annual Report (Publication No. STA 17-74). https://ccmh.psu.edu/assets/docs/2016-Annual-Report-FINAL_2016_01_09-1gc2hj6.pdf

16.

Center for Collegiate Mental Health. (2020, July). Standardized Data Set (SDS). (Publication No. U.Ed. STA 17-21). https://ccmh.psu.edu/assets/sds/SDS%20Manual%202020.pdf

17.

Chen

(2012). Institutional characteristics and college student dropout risks: A multilevel event history analysis. Research in Higher Education, 53(5), 487–505. https://doi.org/10.1007/s11162-011-9241-4

18.

Choi

K.-H.

Buskey

Johnson

(2010). Evaluation of counseling outcomes at a university counseling center: The impact of clinically significant change on problem resolution and academic functioning. Journal of Counseling Psychology, 57(3), 297–303. https://doi.org/10.1037/a0020029

19.

Credé

Niehorster

(2012). Adjustment to college as measured by the Student Adaptation to College Questionnaire: A quantitative review of its structure and relationships with correlates and consequences. Educational Psychology Review, 24(1), 133–165. https://doi.org/10.1007/s10648-011-9184-5

20.

Eisenberg

Golberstein

Hunt

J. B.

(2009). Mental health and academic success in college. The BE Journal of Economic Analysis & Policy, 9(1). https://doi.org/10.2202/1935-1682.2191

21.

Eisenberg

Gollust

S. E.

Golberstein

Hefner

J. L.

(2007). Prevalence and correlates of depression, anxiety, and suicidality among university students. American Journal of Orthopsychiatry, 77(4), 534–542. https://doi.org/10.1037/0002-9432.77.4.534

22.

Elbogen

E. B.

Wagner

H. R.

Fuller

S. R.

Calhoun

P. S.

Kinneer

P. M.

Beckham

J. C.

(2010). Correlates of anger and hostility in Iraq and Afghanistan war veterans. American Journal of Psychiatry, 167(9), 1051–1058. https://doi.org/10.1176/appi.ajp.2010.09050739

23.

Espinosa

L. L.

Turk

J. M.

Taylor

Chessman

H. M.

(2019). Race and Ethnicity in Higher Education: A Status Report. American Council on Education. https://1xfsu31b52d33idlp13twtos-wpengine.netdna-ssl.com/wp-content/uploads/2019/02/Race-and-Ethnicity-in-Higher-Education.pdf

24.

Ghosh

Rieder Bennett

Martin

J. K.

(2017). Use of the Counseling Center Assessment of Psychological Symptoms 62 (CCAPS-62) as a repeated measure. Journal of College Student Psychotherapy, 32(2), 151–162. https://doi.org/10.1080/87568225.2017.1367630

25.

Heiligenstein

Guenther

Hsu

Herman

(1996). Depression and academic impairment in college students. Journal of American College Health, 45(2), 59–64. https://doi.org/10.1080/07448481.1996.9936863

26.

Hunt

Eisenberg

Kilbourne

A. M.

(2010). Consequences of receipt of a psychiatric diagnosis for completion of college. Psychiatric Services, 61(4), 399–404. https://doi.org/10.1176/appi.ps.61.4.399

27.

Ishitani

T. T.

(2003). A longitudinal approach to assessing attrition behavior among first-generation students: Time-varying effects of pre-college characteristics. Research in Higher Education, 44, 433–449. https://doi.org/10.1023/A:1024284932709

28.

Judd

L. L.

Schettler

P. J.

Coryell

Akiskal

H. S.

Fiedorowicz

J. G.

(2013). Overt irritability/anger in unipolar major depressive episodes: Past and current characteristics and implications for long-term course. JAMA Psychiatry, 70(11), 1171–1180. https://doi.org/10.1001/jamapsychiatry.2013.1957

29.

Joo

S.-H.

Durband

D. B.

Grable

(2008). The academic impact of financial stress on college students. Journal of College Student Retention: Research, Theory & Practice, 10(3), 287–305. https://doi.org/10.2190/CS.10.3.c

30.

Kamer

J. A.

Ishitani

T. T.

(2019). First-year, nontraditional student retention at four-year institutions: How predictors of attrition vary across time. Journal of College Student Retention: Research, Theory & Practice. https://doi.org/10.1177/1521025119858732

31.

Kessler

R. C.

Foster

C. L.

Saunders

W. B.

Stang

P. E.

(1995). Social consequences of psychiatric disorders, I: Educational attainment. The American Journal of Psychiatry, 152(7), 1026–1032. https://doi.org/10.1176/ajp.152.7.1026

32.

King

A. R.

(2000). Relationships between CATI personality disorder variables and measures of academic performance. Personality and Individual Differences, 29(1), 177–190. https://doi.org/10.1016/S0191-8869(99)00186-5

33.

Koh

K. B.

Kim

C. H.

Park

J. K.

(2002). Predominance of anger in depressive disorders compared with anxiety disorders and somatoform disorders. The Journal of Clinical Psychiatry, 63(6), 486–492. https://doi.org/10.4088/jcp.v63n0604

34.

Krumrei

E. J.

Newton

F. B.

Kim

(2010). A multi-institution look at college students seeking counseling: Nature and severity of concerns. Journal of College Student Psychotherapy, 24(4), 261–283. https://doi.org/10.1080/87568225.2010.509223

35.

Lee

Olson

E. A.

Locke

Michelson

S. T.

Odes

(2009). The effects of college counseling services on academic performance and retention. Journal of College Student Development, 50(3), 305–319. https://doi.org/10.1353/csd.0.0071

36.

Lee

Tsang

Breslau

Aguilar-Gaxiola

Angermeyer

Borges

Bromet

Bruffaerts

de Girolamo

Fayyad

Gureje

Haro

J. M.

Kawakami

Levinson

Oakley

M. A.

Browne

M. A.

Ormel

Posada-Villa

Williams

D. R.

Kessler

R. C.

(2009). Mental disorders and termination of education in high-income and low- and middle-income countries: Epidemiological study. The British Journal of Psychiatry: The Journal of Mental Science, 194(5), 411–417. https://doi.org/10.1192/bjp.bp.108.054841

37.

Lockard

A. J.

Hayes

J. A.

McAleavey

A. A.

Locke

B. D

. (2012). Change in academic distress: Examining differences between a clinical and nonclinical sample of college students. Journal of College Counseling, 15(3), 233–246. https://doi.org/10.1002/j.2161-1882.2012.00018.x

38.

Locke

B. D.

Buzolitz

J. S.

Lei

P.-W.

Boswell

J. F.

McAleavey

A. A.

Sevig

T. D.

Dowis

J. D.

Hayes

J. A.

(2011). Development of the Counseling Center Assessment of Psychological Symptoms-62 (CCAPS-62). Journal of Counseling Psychology, 58(1), 97–109. https://doi.org/10.1037/a0021282

39.

MacFarlane

I. M.

Henry

C. L.

Nash

Kissel

Bush

(2015). Clinical utility of the CCAPS, CAS, and OQ-45. Journal of College Student Psychotherapy, 29(1), 11–28. https://doi.org/10.1080/87568225.2015.973821

40.

McAleavey

A. A.

Lockard

A. J.

Castonguay

L. G.

Hayes

J. A.

Locke

B. D.

(2015). Building a practice research network: Obstacles faced and lessons learned at the Center for Collegiate Mental Health. Psychotherapy Research, 25(1), 134–151. https://doi.org/10.1080/10503307.2014.883652

41.

McAleavey

A. A.

Nordberg

S. S.

Hayes

J. A.

Castonguay

L. G.

Locke

B. D.

Lockard

A. J.

(2012). Clinical validity of the Counseling Center Assessment of Psychological Symptoms-62 (CCAPS-62): Further evaluation and clinical applications. Journal of Counseling Psychology, 59(4), 575–590. https://doi.org/10.1037/a0029855

42.

Merrill

J. E.

Carey

K. B.

(2016). Drinking over the lifespan: Focus on college ages. Alcohol Research. 38(1), 103–114. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4872605/

43.

Muller

C. J.

MacLehose

R. F.

(2014). Estimating predicted probabilities from logistic regression: Different methods correspond to different target populations. International Journal of Epidemiology, 43(3), 962–970. https://doi.org/10.1093/ije/dyu029

44.

Musu-Gillette

de Brey

McFarland

Hussar

Sonnenberg

Wilkinson-Flicker

(2017). Status and trends in the education of racial and ethnic groups 2017 (NCES 2017-051). U.S. Department of Education, National Center for Education Statistics. https://files.eric.ed.gov/fulltext/ED574873.pdf

45.

Pandey

Elliott

(2010). Suppressor variables in social work research: Ways to identify in multiple regression models. Journal of the Society for Social Work and Research, 1(1), 28–40. https://doi.org/10.5243/jsswr.2010.2

46.

Pitre

C. C.

Pitre

(2009). Increasing underrepresented high school students’ college transitions and achievements: TRIO educational opportunity programs. NASSP bulletin, 93(2), 96–110. https://doi.org/10.1177/0192636509340691

47.

Posternak

M. A.

Zimmerman

(2002). Anger and aggression in psychiatric outpatients. The Journal of Clinical Psychiatry, 63(8), 665–672. https://doi.org/10.4088/jcp.v63n0803

48.

Ratanasiripong

Wang

C. D.

Ratanasiripong

Hayes

J. A.

Kaewboonchoo

Kathalae

(2015). Development and cross-cultural validation of the Counseling Center Assessment of Psychological Symptoms–Thai Version. The Counseling Psychologist, 43(2), 248–271. https://doi.org/10.1177/0011000014566991

49.

Reason

R. D.

(2009). An examination of persistence research through the lens of a comprehensive conceptual framework. Journal of College Student Development, 50(6), 659–682. https://doi.org/10.1353/csd.0.0098

50.

Richardson

Abraham

Bond

(2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353–387. https://doi.org/10.1037/a0026838

51.

Romano

J. L.

Hage

S. M.

(2000). Prevention and counseling psychology: Revitalizing commitments for the 21st century. Counseling Psychologist, 28(6), 733–763. https://doi.org/10.1177/0011000000286001

52.

Russell

Shaw

(2009). A study to investigate the prevalence of social anxiety in a sample of higher education students in the United Kingdom. Journal of Mental Health, 18(3), 198–206. https://doi.org/10.1080/09638230802522494

53.

Russell

Topham

(2012). The impact of social anxiety on student learning and well-being in higher education. Journal of Mental Health, 21(4), 375–385. https://doi.org/10.3109/09638237.2012.694505

54.

Scheel

M. J.

Stabb

S. D.

Cohn

T. J.

Duan

Sauer

E. M.

(2018). Counseling psychology model training program. The Counseling Psychologist, 46(1), 6–49. https://doi.org/10.1177/0011000018755512

55.

Scherwitz

Perkins

Chesney

Hughes

(1991). Cook-Medley Hostility Scale and subsets: Relationship to demographic and psychosocial characteristics in young adults in the CARDIA study. Psychosomatic Medicine, 53(1), 36–49. https://doi.org/10.1097/00006842-199101000-00004

56.

Sharkin

B. S.

(2004). College counseling and student retention: Research findings and implications for counseling centers. Journal of College Counseling, 7(2), 99–108.

57.

Stein

M. B.

Kean

Y. M.

(2000). Disability and quality of life in social phobia: Epidemiologic findings. American Journal of Psychiatry, 157(10), 1606–1613. https://doi.org/10.1176/appi.ajp.157.10.1606

58.

Strahan

E. Y.

(2003). The effects of social anxiety and social skills on academic performance. Personality and Individual Differences, 34(2), 347–366. https://doi.org/10.1016/S0191-8869(02)00049-1

59.

Struthers

C. W.

Perry

R. P.

Menec

V. H.

(2000). An examination of the relationship among academic stress, coping, motivation, and performance in college. Research in Higher Education, 41(5), 581–592. https://doi.org/10.1023/A:1007094931292

60.

Topham

Moller

(2011). New students’ psychological well-being and its relation to first year academic performance in a UK university. Counselling and Psychotherapy Research, 11(3), 196–203. https://doi.org/10.1080/14733145.2010.519043

61.

Trockel

M. T.

Barnes

M. D.

Egget

D. L.

(2000). Health-related variables and academic performance among first-year college students: Implications for sleep and other behaviors. Journal of American College Health, 49(3), 125–131. https://doi.org/10.1080/07448480009596294

62.

Troiano

P. F.

Liefeld

J. A.

Trachtenberg

J. V.

(2010). Academic support and college success for postsecondary students with learning disabilities. Journal of College Reading and Learning, 40(2), 35–44. https://doi.org/10.1080/10790195.2010.10850329

63.

Van Brunt

. (2008). Retention and college counseling centers. Recruitment and Retention in Higher Education, 22(5), 1–3. https://70daf429-d4ec-4875-9eb5-b28958ffb4fa.filesusr.com/ugd/b64c59_981a1af88ae64c758077ae5c2cfd054a.pdf

64.

White

Hingson

(2013). The burden of alcohol use: Excessive alcohol consumption and related consequences among college students. Alcohol Research: Current Reviews, 35(2), 201–218. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3908712/