Compensatory Cognitive Strategy Use by Young Adults: A Psychometric Evaluation of Self-Report Measures

Abstract

This study examined the reliability and validity of scores from questionnaires assessing compensatory cognitive strategy use by young adults. Participants (N = 783; 79.6% women; 77.4% White; Age: M = 18.7 ± 0.9 years) completed the Compensatory Cognitive Strategies Scale (CCSS), Multifactorial Memory Questionnaire–Strategies (MMQ-S), Neuro-Quality of Life (Neuro-QoL) v2.0 Short Form–Cognitive Function, and Difficulties in Emotion Regulation Scale (DERS). Reliability estimates were acceptable for the CCSS (α = .85) and MMQ-S (α = .86) in the full sample and comparable in subsamples stratified by gender and self-reported mental health conditions. The CCSS and MMQ-S scores showed evidence for convergent validity (r range: .60 to .70) and discriminant validity in the full sample and subsamples (r with DERS range: .13 to .33). Greater compensatory strategy use had small-to-medium associations with greater subjective cognitive concerns on the Neuro-QOL for the full sample and subsamples (r range: −.19 to −.49). The CCSS and MMQ-S scores showed acceptable properties for use with young adults.

Keywords

cognition psychometrics young adult self-assessment compensatory cognitive strategies compensatory cognitive training cognitive remediation

Researchers have conducted intervention studies examining the efficacy of compensatory cognitive strategies at reducing subjective cognitive concerns in patients with neurological conditions (e.g., multiple sclerosis, dementia; Becker et al., 2019; Troyer, 2001), psychiatric disorders (e.g., psychosis, posttraumatic stress disorder, anxiety, or mood disorders; Jak et al., 2019; Mullen et al., 2017; Twamley et al., 2012), and traumatic brain injuries (Huckans et al., 2010; Jak et al., 2019; Storzbach et al., 2017; Twamley et al., 2015). The outcomes for these trials have included objective cognitive performances, subjective cognitive complaints, and the self-reported implementation of cognitive strategies into the everyday lives of participants. Whereas many prior studies have examined the psychometric properties of neuropsychological tests (Lezak et al., 2012; Strauss et al., 2006) and self-report cognitive symptom questionnaires (Cella et al., 2012; Gershon et al., 2012; Lai et al., 2014; Roth et al., 2014), fewer studies have examined the psychometric properties of questionnaires on compensatory cognitive strategy use (Becker et al., 2019; Dixon & Hultsch, 1983; Troyer & Rich, 2002).

These existing studies have specifically focused on patients with neurological, neurodegenerative, or psychiatric conditions. The assessment of compensatory cognitive strategy use may also be helpful for clinicians working with college students. College students experience significant cognitive demands in their coursework. They have high rates of mental and behavioral health problems (e.g., depression, alcohol use disorders; Auerbach et al., 2016; Blanco et al., 2008) that may adversely affect cognitive functioning (McDermott & Ebmeier, 2009; Semkovska et al., 2019; Stephan et al., 2017). Furthermore, they may acquire conditions or injuries that could affect their cognitive functioning during their studies (e.g., sport-related concussion; Belanger & Vanderploeg, 2005; Dougan et al., 2014; Karr et al., 2014). Successful academic performance requires the use of cognitive strategies, such as planning, notetaking, and developing study strategies to remember important material. For those with and without cognitive weakness or impairment, compensatory cognitive strategy use can be helpful for successful academic performance during postsecondary studies.

Compensatory cognitive strategies have been taught in interventions for many populations with conditions that can affect cognitive functioning (Becker et al., 2019; Jak et al., 2019; Troyer, 2001; Twamley et al., 2012) including college students with psychiatric conditions, which resulted in increased frequency of compensatory cognitive strategy use (Mullen et al., 2017). A clinician working with a college student struggling in their courses may consider possible modes of intervention, which could include addressing mental health problems, recommending academic accommodations, or providing psychoeducation on compensatory cognitive strategy use to support academic performance. Clinicians educating patients on compensatory cognitive strategies as part of their intervention would benefit from having an instrument that quantified current strategy use. Such an instrument would allow them to gauge whether psychoeducation was necessary (i.e., determine if the student is or is not currently using strategies in their everyday life) and track changes in strategy use over the course of the intervention. They may also benefit from reference data among college-aged peers in terms of how often students implement compensatory cognitive strategies in their everyday lives.

This study examined the psychometric properties of two compensatory cognitive strategy scales in a college student sample: the Compensatory Cognitive Strategies Scale (CCSS; Becker et al., 2019) and the Multifactorial Memory Questionnaire–Strategies (MMQ-S; Troyer & Rich, 2002), which have previously been examined for use in samples with multiple sclerosis and dementia, respectively. Prior researchers have not examined the psychometric properties of published scales on compensatory cognitive strategy use among college students. The CCSS and MMQ-S were selected because (a) they are brief and have known psychometric properties (i.e., evidence of reliability, convergent validity, and discriminant validity with different populations); (b) the CCSS was recently developed and involved technology use in its item content, which is pertinent to current college education; and (c) the MMQ-S assesses learning and memory strategies highly relevant for successful academic performance, such as making a to-do list, taking notes, using mnemonics, and practicing spaced retrieval (i.e., effortful recall of information without reference to the source material over increasingly longer spaced time intervals).

For the CCSS and MMQ-S, reliability, convergent validity, and discriminant validity were evaluated, along with the correspondence between compensatory strategy use and subjective cognitive concerns. Reliability was assessed by calculating internal consistency for each scale, hypothesizing that the CCSS and MMQ-S scores would show acceptable reliability among a college student sample. Convergent validity was assessed by calculating the correlation between the CCSS and MMQ-S, hypothesizing that—consistent with previous findings (Becker et al., 2019)—these scales would have a moderate and positive correlation. Discriminant validity was assessed by calculating the correlation between the CCSS and MMQ-S and a measure of emotional regulation, hypothesizing that these scales would correlate, at most, modestly. The relationship between compensatory cognitive strategy use and subjective cognitive concerns was also evaluated as a criterion index for concurrent validity, hypothesizing that—consistent with previous findings (Becker et al., 2019; Troyer & Rich, 2002)—the CCSS and MMQ-S would have a small, negative correlation with cognitive complaints (i.e., those with stronger confidence in their cognitive abilities would report less compensatory strategy use). Last, the assessment of reliability and validity for the CCSS and MMQ-S scores was stratified by gender and preexisting self-reported mental health conditions. Prior research on compensatory cognitive strategy training among college students focused specifically on participants with psychiatric conditions (Mullen et al., 2017); and as such, the current study sought to determine whether the psychometric properties of the scales were comparable between those with and without self-reported mental health conditions.

Method

Participants

Of 1,457 eligible students enrolled in one of three low-level undergraduate psychology courses (i.e., Intro to Psychology, Experimental Psychology, or Statistics in Psychology) at a public university in the United States, 977 college-aged participants completed an online survey as a screener for participation in departmental research (i.e., response rate of 67.1%). For inclusion in analysis, participants needed to meet preestablished inclusion criteria. Participants were excluded from analyses if they (a) reported a history of a neurological condition (e.g., brain tumor, stroke; n = 23); (b) reported a diagnosis of a neurodevelopmental disorder other than attention-deficit/hyperactivity disorder (ADHD), such as autism spectrum disorder (n = 4); and/or (c) reported a traumatic brain injury, not including concussion (n = 9). Among those who met inclusion criteria, participants were further removed from analysis if they provided an incorrect response to either of two embedded validity questions (n = 158), which are described in the Measures section below. This resulted in a final sample of 783 participants included in analysis, with demographic characteristics reported in Table 1.

Table 1.

Sample Characteristics.

Characteristic	Statistic
Age (years), M (SD), range	18.7 (0.9), 18-22
Gender, n (%)
Women	623 (79.6)
Men	157 (20.1)
Genderqueer, nonconforming or no response	3 (0.3)
Race, n (%)
White	606 (77.4)
Black or African American	72 (9.2)
Multiracial	41 (5.2)
Latinx	31 (4.0)
Asian	25 (3.2)
Native American or Pacific Islander	2 (0.8)
Another identity or no response	6 (0.8)
Class, n (%)
Freshmen	458 (58.5)
Sophomores	206 (26.3)
Juniors	103 (13.2)
Seniors	13 (1.7)
Other or no response	3 (0.3)
Major field of study, n (%)
Psychology	238 (30.4)
Other	542 (69.2)

The final sample was an average of 18.7 years old (SD = 0.9, range: 18-22), with 79.6% identifying as women (n = 623; Age: M = 18.6, SD = 0.9; 78.8% White), 20.1% identifying as men (n = 157; Age: M = 18.9, SD = 1.0; 72.6% White), and 0.3% identifying as genderqueer/nonconforming or declining to answer (n = 3). The majority of the sample were freshmen (58.5%; n = 458) or sophomores (26.3%; n = 206) in college, with 30.4% (n = 238) declaring a psychology major. In terms of racial and ethnic identity, 77.4% identified as White (n = 606), 9.2% as Black or African American (n = 72), 5.2% as multiracial (n = 41), 4.0% as Latinx (n = 31), 3.2% as Asian (n = 25), 0.3% as Native American or Pacific Islander (n = 2), and 0.8% as another identity or declined to answer (n = 6). In terms of preexisting mental health conditions not used as exclusion criteria, subsamples of participants self-reported a prior diagnosis of depression (21.8%; n = 171; 83.6% women, 84.2% White), an anxiety disorder (28.7%; n = 225; 84.9% women, 83.1% White), ADHD (12.9%; n = 101; 70.3% women, 88.1% White), or another unspecified mental health diagnosis (8.6%; n = 67; 94.0% women, 89.6% White). There was overlap between these samples of participants, with 22.9% reporting a personal history of two or more comorbid diagnoses (n = 179; 84.9% women, 83.8% White). A subsample also reported no preexisting mental health conditions (n = 491; 79.4% women, 73.7% White). Because of the substantial representation of women and preexisting mental health conditions, each analysis was run for the full sample, men, women, the subsample reporting no prior diagnoses of mental health conditions, and the subsamples reporting prior diagnoses of depression, an anxiety disorder, ADHD, and two or more comorbid mental health conditions.

Measures

The questionnaires included as variables in this study included the CCSS (Becker et al., 2019), the MMQ-S (Troyer & Rich, 2002), the Neuro–Quality of Life (Neuro-QoL) Short Form v2.0–Cognitive Function (Cella et al., 2012; Gershon et al., 2012), and the Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2008). A pair of embedded validity assessment questions were included in the CCSS and MMQ-S, informing participants that the question was a validity question and asking them to respond by selecting a specific rating on the item scale (e.g., This is a validity question to check that you are paying attention. Please respond by selecting Often as your response). Prior researchers have recommended face valid instructed response items for online survey data collection as a method for detecting careless and invalid responding (Meade & Craig, 2012).

CCSS

This measure is a 24-item questionnaire querying how frequently participants use compensatory cognitive strategies in their everyday life, including strategies related to processing speed and attention, memory, and executive functions. The questionnaire builds from prior assessments of cognitive strategies by incorporating items on strategies that involve the use of technology. Participants are instructed as follows: “Please rate how frequently you use each strategy to complete cognitive tasks (e.g., those involving thinking, attention, memory) in your daily life.” They respond to each item by rating their frequency of use on a 5-point Likert-type scale: never (0), rarely (1), sometimes (2), often (3), and all the time (4). All items were summed to arrive at the CCSS total score used in analysis, with possible scores ranging from 0 to 96. In patients with multiple sclerosis, previous research has shown acceptable internal consistency for the CCSS total score (α = .90), along with evidence for convergent and discriminant validity (Becker et al., 2019).

MMQ-S

This measure is a 19-item questionnaire that queries about how frequently participants use specific metamemory strategies in their everyday life. Participants are instructed to “Please rate how frequently you use each strategy to help with memory in your daily life,” rating their frequency of use on a 5-point Likert-type scale: never (0), rarely (1), sometimes (2), often (3), and all the time (4). All items are summed to arrive at the MMQ-S total score used in analysis, with possible scores ranging from 0 to 76. Among older adults, the MMQ-S total score has shown acceptable test–retest reliability (r = .88) and evidence for convergent and discriminant validity (Troyer & Rich, 2002).

Neuro-QoL Short Form v2.0–Cognitive Function

This measure is an 8-item questionnaire on subjective cognitive functioning, through which participants rate how much difficulty they have had with cognitive functioning and engagement in different cognitive tasks both currently and over the previous 7 days. Participants rate each item on a 5-point Likert-type scale. The item ratings are summed to arrive a total raw score, with possible values ranging from 8 to 40. Per recommendation by the test developers, the raw scores for the Neuro-QoL are converted to norm-referenced T scores (M = 50, SD = 10) based on comparison with the general United States population (Hanmer et al., 2020). In the general population, the total score for the questionnaire has shown acceptable reliability (α = .92) and evidence for convergent and discriminant validity (Cella et al., 2012).

DERS

The DERS is a 36-item questionnaire that asks about emotional regulation, understanding, and awareness. Participants are instructed to indicate how often the items apply to them on a 5-point Likert-type scale: almost never (1), sometimes (2), about half the time (3), most of the time (4), and almost always (5). All items are summed to arrive at a total score, with possible scores ranging from 36 to 180. This questionnaire has shown acceptable reliability for the total score (α = .93) in a sample of undergraduate students (Gratz & Roemer, 2004). Previous research has shown good fit for a bifactor model of the DERS including a general factor on which all items loaded (Hallion et al., 2018), which supports the use of a total sum score for the measure.

Procedure

Participants completed the questionnaires online via Sona Systems® participant pool management software as part of a screener for participation in research within a university psychology department. The survey is advertised to students enrolled in the three aforementioned undergraduate psychology courses as a way to earn course credit (i.e., Intro to Psychology, Experimental Psychology, and Statistics in Psychology). Participants provided informed consent for their data to be included in this study, and received course credit for participation in the survey. All questionnaires were administered in the same order: a demographic and health history questionnaire, the Neuro-QoL, CCSS, MMQ-S, and the DERS. This study protocol received ethical approval from a university institutional review board.

Statistical Analysis

Missing Data

Less than 2% of cases were missing data for each item on the Neuro-QoL, CCSS, MMQ-S, and DERS, with data missing completely at random per Little’s (1988) test, χ²(5,022) = 5,160, p = .084. Missing data were handled via the expectation–maximization method (Enders, 2010), which retained 80 cases with partial missing data by imputing expected values for individual items on each scale.

Reliability and Validity Assessment

Internal consistency (α) was calculated as an estimate of reliability for each test score (Cronbach, 1951). Convergent validity was assessed by calculating the correlation (r) between the CCSS and the MMQ-S. Discriminant validity was assessed by calculating the correlations between the CCSS and MMQ-S and the DERS. The correlation coefficients from the convergent and discriminant validity analyses were compared (Meng et al., 1992), hypothesizing that the correlation between the CCSS and MMQ-S would be significantly greater than the correlations between either of these scales and the DERS. The relationship between compensatory cognitive strategy use and subjective cognitive concerns was assessed by calculating the correlations between the CCSS and MMQ-S and the Neuro-QoL. All descriptive and inferential statistics were calculated using IBM SPSS version 27.

Factor Analysis

The dimensionality of the CCSS and MMQ-S were evaluated via confirmatory factor analysis using Mplus v.7.3. The items for both the CCSS and MMQ-S are on Likert-type scales and were considered polytomous indicators in the factor model, which requires an alternative estimator in Mplus called weighted least squares with mean and variance adjusted. Multiple fit indices were calculated, including the χ² goodness-of-fit test, the comparative fit index (CFI), the Tucker–Lewis index (TLI), and the root mean square error of approximation (RMSEA). For the χ² test, an insignificant p value suggests adequate model fit; however, with large sample sizes, this test has increased power to reject the null hypothesis (Tanaka, 1987). With the large sample size for the current study (N = 783), alternative fit indices were interpreted to determine model fit. For the CFI and TLI, values ≥.90 or ≥.95 were considered indicative of adequate and optimal fit, respectively (Bentler & Bonett, 1980; Hu & Bentler, 1999). For the RMSEA, values ≤.08 and ≤.06 were considered indicative of adequate and optimal fit, respectively (Browne & Cudeck, 1993; Hu & Bentler, 1999). The RMSEA is presented with an accompanying 95% confidence interval (CI).

Because the MMQ-S assesses a single common construct of metamemory strategy use and has evidenced unidimensionality in a prior principal components analysis (Troyer & Rich, 2002), a one-factor model was examined for this scale. No prior research has examined the factor structure of the CCSS; although the original development study of the CCSS examined a single total sum score for the items (Becker et al., 2019), implying a unidimensional structure. When examining the factor structure of the CCSS, the use of a confirmatory approach was preferred over an exploratory approach to (a) examine the fit of a one-factor model, which was implied by prior research using a total score; and (b) compare this model with an alternative multidimensional factor model, with individual items categorized based on an examination of item content. As such, a one-factor model was examined for the CCSS, in addition to a multidimensional model that separated item content by different abilities associated with different strategies (i.e., attention and processing speed, memory, and executive functions).

Power Analysis

An a priori power analysis was conducted using G*Power 3.1 (Erdfelder et al., 2009). Based on previous findings, a medium positive correlation (r = .65) between the CCSS and MMQ-S was anticipated (Becker et al., 2019), requiring a sample size of 16 for power of .80 with α of .05; and smaller negative correlations (r = −.20) between the cognitive strategy questionnaires and the Neuro-QoL were anticipated (Becker et al., 2019; Troyer & Rich, 2002), requiring a sample size of 153 for power of .80 with α of .05. A very small correlation was hypothesized between the cognitive strategy questionnaires and the DERS (r < .10); but considering the large sample size, there was power to detect a significant finding for even a very small effect. An additional analysis examined whether the correlations between the CCSS and MMQ-S differed from the correlations between the DERS and each of these scales. With a hypothesized convergent correlation of .65 and hypothesized discriminant correlations of .10, a sample size of 35 was required for a z test comparing two correlations to achieve power of .80 with an α of .05. For the confirmatory factor analysis, a priori power analysis was conducted to determine the necessary sample size based on a conservative hypothesized RMSEA of .04 (Hancock & Freeman, 2001). The required sample size to achieve .80 power predicting a RMSEA of .04 for the MMQ-S one-factor model (degrees of freedom [df] = 152) was 600. For the CCSS, again predicting a RMSEA of .04, the sample size required to achieve .80 power for the one-factor model (df = 252) and the three-factor model (df = 249) was 400.

Results

Descriptive Statistics for Full Sample and Subsamples

Descriptive statistics for the CCSS, MMQ-S, Neuro-QoL, and DERS are reported in Table 2 for the full sample, men, women, and for subsamples of participants with and without self-reported diagnoses of mental health conditions. All variables were normally distributed, with skewness and kurtosis values for each variable approximating zero. Descriptive statistics for individual items on the CCSS and MMQ-S for the full sample are reported in Tables 3 and 4, respectively. The frequencies of self-reported compensatory cognitive strategy use (i.e., using the strategy sometimes, often, or all the time) for the full sample are also reported in Tables 3 and 4 for the CCSS and MMQ-S, respectively.

Table 2.

Descriptive Statistics for the Full Sample and Subsamples Stratified by Gender and Self-Reported Diagnoses of Mental Health Conditions.

	M	SD	Min.	Max	Skewness	Kurtosis
Full Sample (N = 783)
CCSS	48.3	12.1	5	90	0.01	0.33
MMQ-S	42.5	10.5	7	70	−0.03	−0.24
Neuro-QoL	43.8	6.9	25.9	64.2	0.24	0.16
DERS	96.7	17.3	56	156	0.65	0.35
Men (n = 171)
CCSS	49.0	13.1	5	81	−0.04	0.35
MMQ-S	44.0	11.0	12	70	−0.01	−0.41
Neuro-QoL	41.2	7.1	26	59	0.26	0.01
DERS	105.9	18.5	65	155	0.43	0.02
Women (n = 623)
CCSS	49.2	11.8	5	90	0.07	0.42
MMQ-S	43.8	10.0	12	70	−0.03	−0.25
Neuro-QoL	43.5	6.9	26	64	0.16	0.06
DERS	97.5	17.1	63	156	0.68	0.37
Depression (n = 171)
CCSS	49.0	13.1	5	81	−0.04	0.35
MMQ-S	44.0	11.0	12	70	−0.01	−0.41
Neuro-QoL	41.2	7.1	26	59	0.26	0.01
DERS	105.9	18.5	65	155	0.43	0.02
Anxiety disorder (n = 225)
CCSS	49.5	12.7	5	81	−0.16	0.60
MMQ-S	44.6	10.7	12	70	−0.18	−0.16
Neuro-QoL	41.3	7.3	26	59	0.28	−0.17
DERS	103.5	19.1	64	155	0.40	−0.20
Attention-deficit/hyperactivity disorder (n = 101)
CCSS	49.7	13.9	5	81	−0.34	0.45
MMQ-S	44.2	12.4	7	70	−0.40	0.26
Neuro-QoL	39.5	6.5	27	59	0.23	−0.09
DERS	100.7	18.8	70	155	0.91	0.68
Comorbid mental health conditions (n = 179)
CCSS	49.3	13.0	5	81	−0.16	0.51
MMQ-S	44.7	11.1	12	70	−0.25	−0.15
Neuro-QoL	40.7	7.2	26	59	0.35	−0.03
DERS	104.7	19.4	65	155	0.43	−0.24
No mental health conditions (n = 491)
CCSS	47.9	11.9	12	90	0.09	0.26
MMQ-S	41.8	10.3	16	70	0.08	−0.42
Neuro-QoL	45.0	6.6	29	64	0.37	0.22
DERS	93.8	16.0	56	156	0.68	0.55

Note. CCSS, MMQ-S, and DERS scores are raw sum scores of all items. Neuro-QoL scores are norm-referenced T scores. CCSS = Compensatory Cognitive Strategies Scale; MMQ-S = Multifactorial Memory Questionnaire–Strategies; Neuro-QoL = Neuro–Quality of Life Short Form v2.0–Cognitive Function; DERS = Difficulties in Emotion Regulation Scale.

Table 3.

The Compensatory Cognitive Strategies Scale: Descriptive Statistics of Self-reported Engagement in Compensatory Cognitive Strategies and Factor Loadings for One-Factor and Three-Factor Measurement Models.

Compensatory Cognitive Strategies Scale (Becker et al., 2019)	M	SD	Mdn	Frequency. of use		One-factor CFA		Three-factorCFA
	M	SD	Mdn	n	%	λ	p	Factor	λ	p
1. Use reminder functions on computer or personal electronic device (e.g., smart phone, tablets)	2.7	1.2	3	660	84.3	.42	<.001	Memory	.44	<.001
2. Break down complex tasks into smaller steps	2.5	1.0	2	671	85.7	.58	<.001	EF	.60	<.001
3. Ask others to review work	1.9	1.0	2	506	64.6	.48	<.001	EF	.49	<.001
4. Pace/conserve energy	1.9	1.0	2	537	68.6	.52	<.001	Attn/PS	.54	<.001
5. Organize to do the most challenging tasks at best (i.e., most energetic) time of day	2.3	1.1	2	616	78.7	.55	<.001	EF	.57	<.001
6. Minimize distractions in your work space by closing the door, and so on	3.0	0.9	3	725	92.6	.38	<.001	Attn/PS	.40	<.001
7. Take extra time to do tasks	2.5	0.9	2	685	87.5	.58	<.001	Attn/PS	.61	<.001
8. Slow the pace of communication in meetings	1.3	0.9	1	318	40.6	.60	<.001	Attn/PS	.63	<.001
9. Confirm what others say by repeating the “gist” back to them	2.1	1.0	2	581	74.2	.45	<.001	Memory	.47	<.001
10. Take notes while listening to or viewing presentations to organize/reinforce information	3.1	0.9	3	740	94.5	.42	<.001	Memory	.43	<.001
11. Recheck work after taking a break	2.4	1.1	2	632	80.7	.56	<.001	EF	.58	<.001
12. Do one thing at a time rather than multitasking	2.4	0.9	2	652	83.3	.27	<.001	Attn/PS	.28	<.001
13. Write out notes (in advance) to help verbally relay a sequence of events	2.2	1.2	2	579	73.9	.57	<.001	EF	.59	<.001
14. Request accommodations as needed (e.g., extra time to complete assignments)	1.0	1.1	1	235	30.0	.53	<.001	EF	.54	<.001
15. Use personal electronic devices to record written documents	2.0	1.3	2	508	64.9	.37	<.001	Memory	.39	<.001
16. Minimize the number of open tabs/windows when on the internet	2.3	1.2	2	587	75.0	.32	<.001	Attn/PS	.33	<.001
17. Log off of email periodically to limit distractions	1.1	1.1	1	282	36.0	.37	<.001	Attn/PS	.39	<.001
18. Request agenda and/or meeting notes	1.5	1.1	2	405	51.7	.57	<.001	Memory	.60	<.001
19. Ask others to minimize the number of interruptions when working	1.7	1.1	2	452	57.7	.55	<.001	Attn/PS	.58	<.001
20. Audio record meetings for later review	0.8	1.0	0	187	23.9	.56	<.001	Memory	.59	<.001
21. Use a digital recorder to make short reminder memos and review them later	0.4	0.8	0	80	10.2	.55	<.001	Memory	.57	<.001
22. Establish a routine of always putting “key” items in the same place in house or workspace	2.6	1.2	3	661	84.4	.40	<.001	Memory	.41	<.001
23. Slow down; avoid situations that require fast reaction time	1.6	1.0	2	435	55.6	.52	<.001	Attn/PS	.54	<.001
24. Prioritize “to do” list to aid time management and organization	2.9	1.0	3	706	90.2	.56	<.001	EF	.57	<.001

Note. Data in this table are based on the full sample of participants (N = 783). Participants responded to items on the Compensatory Cognitive Strategies Scale (Becker et al., 2019) on a 5-point Likert-type scale, rating how often they used a certain strategy in their everyday life: never (0), rarely (1), sometimes (2), often (3), and all the time (4). Frequency of use was calculated as the frequency at which participants reported using a behavior sometimes, often, or all the time. The one-factor and three-factor columns report the loadings for two measurement models evaluated for the Compensatory Cognitive Strategies Scale, with the factor subcolumn designating the factors on which each item loaded in the three-factor model. Attn/PS = Attention and Processing Speed; CFA = confirmatory factor analysis; EF = executive functions.

Table 4.

The Multifactorial Memory Questionnaire–Strategies Subscale: Descriptive Statistics of Self-Reported Engagement in Compensatory Cognitive Strategies and Factor Loadings for a One-Factor Measurement Model.

Multifactorial Memory Questionnaire–Strategies Subscale (Troyer & Rich, 2002)	M	SD	Mdn	Frequency of use		One-factor CFA
	M	SD	Mdn	n	%	λ	p
1. How often do you use a timer or alarm to remind you when to do something?	2.9	1.0	3	715	91.3	.48	<.001
2. How often do you ask someone to help you remember something or to remind you to do something?	2.2	1.1	2	579	73.9	.40	<.001
3. How often do you create a rhyme out of what you want to remember?	1.1	1.2	1	259	33.1	.64	<.001
4. How often do you create a visual image of something you want to remember, like a name and a face?	1.9	1.3	2	484	61.8	.55	<.001
5. How often do you write things on a calendar, such as appointments or things you need to do?	3.1	1.1	3	716	91.4	.58	<.001
6. How often do you go through the alphabet one letter at a time to see if it sparks a memory for a name or word?	0.6	1.0	0	123	15.7	.38	<.001
7. How often do you organize information you want to remember; for example, organize your grocery list according to food groups?	1.6	1.4	2	403	51.5	.49	<.001
8. How often do you say something out loud in order to remember it, such as a telephone number you just looked up?	2.9	1.0	3	709	90.5	.59	<.001
9. How often do you use a routine to remember important things, like checking that you have your wallet and keys when you leave home?	2.9	1.1	3	699	89.3	.50	<.001
10. How often do you make a list, such as a grocery list or a list of things to do?	3.0	1.1	3	708	90.4	.57	<.001
11. How often do you mentally elaborate on something you want to remember; for example, focus on a lot on the details?	2.7	0.9	3	718	91.7	.56	<.001
12. How often do you put something in a prominent place to remind you to do something, like putting your umbrella by the front door so that you will remember to take it with you?	2.9	1.0	3	722	92.2	.62	<.001
13. How often do you repeat something to yourself at increasingly longer and longer intervals so that you will remember it?	2.3	1.1	2	614	78.4	.65	<.001
14. How often do you create a story to link together information you want to remember?	1.4	1.2	1	352	45.0	.53	<.001
15. How often do you write down in a notebook things that you want to remember?	2.8	1.1	3	686	87.6	.60	<.001
16. How often do you create an acronym out of the first letters in a list of things to remember, such as carrots, apples, and bread (cab)?	1.2	1.2	1	259	33.1	.56	<.001
17. How often do you intentionally concentrate hard on something so that you will remember it?	2.7	0.9	3	710	90.7	.64	<.001
18. How often do you write a note or reminder for yourself (other than on a calendar or in a notebook)?	2.6	1.2	3	630	80.5	.63	<.001
19. How often do you mentally retrace your steps in order to remember something, such as the location of a misplaced item?	2.8	0.9	3	725	92.6	.55	<.001

Note. Data in this table are based on the full sample of participants (N = 783). Participants responded to items on the Multifactorial Memory Questionnaire–Strategies Subscale (Troyer & Rich, 2002) on a 5-point Likert-type scale, rating how often they used a certain strategy in their everyday life: never (0), farely (1), sometimes (2), often (3), and all the time (4). Frequency (Freq.) of use was calculated as the frequency at which participants reported using a behavior sometimes, often, or all the time. The one-factor CFA column reports the loadings for a unidimensional measurement model evaluated for the Multifactorial Memory Questionnaire–Strategies subscale. CFA = confirmatory factor analysis.

The most frequently used compensatory cognitive strategies on the CCSS included taking notes during presentations (94.5% of participants reported using this strategy), minimizing distractions in their workspace (92.6%), using a to-do list to aide with time management and organization (90.2%), and taking extra time to do tasks (87.5%). The least frequently used strategies endorsed on the CCSS included using a digital recorder to make short reminders to review later (10.2%), audio recording meetings (23.9%), requesting accommodations (e.g., more time for assignments; 30.0%), and logging off email to reduce distractions (36.0%). On the MMQ-S, the most frequently used compensatory strategies reported by participants included mentally retracing steps in order to remember something (92.6%), placing something in a prominent place as a reminder (92.2%), mentally elaborating on something to support memory (e.g., focusing on details; 91.7%), using a calendar to document appointments and tasks that need completion (91.4%), and using a timer or an alarm as a reminder (91.3%). The least frequently used strategies on the MMQ-S included going through the alphabet a letter at a time to spark a memory (15.7%), using a rhyme to help remember (33.1%), creating an acronym to help remember (33.1%), creating a story to link together information (45.0%), and organizing information to aide with memory (51.5%).

Reliability Estimates

The internal consistency for the CCSS (α = .85), MMQ-S (α = .86), Neuro-QOL (α = .88), and DERS (α = .86) were acceptable in the full sample, as well as men (CCSS: α = .86; MMQ-S: α = 0.87; Neuro-QOL: α = .88; and DERS: α = .86) and women (CCSS: α = .85; MMQ-S: α = .85; Neuro-QOL: α = .88; and DERS: α = .86) analyzed separately. The internal consistency for each measures was also examined in subsamples of participants reporting prior diagnoses of mental health conditions, including depression (CCSS: α = .86; MMQ-S: α = .87; Neuro-QOL: α = .88; and DERS: α = .86), anxiety (CCSS: α = .86; MMQ-S: α = .86; Neuro-QOL: α = .89; and DERS: α = .87), ADHD (CCSS: α = .88; MMQ-S: α = .90; Neuro-QOL: α = .87; and DERS: α = .86), or two or more comorbid mental health conditions (CCSS: α = .88; MMQ-S: α = .90; Neuro-QOL: α = .87; and DERS: α = .86). Internal consistency was also examined in participants who reported no prior diagnoses of mental health conditions (CCSS: α = .86; MMQ-S: α = 0.87; Neuro-QOL: α = .88; and DERS: α = .88). Consistent with the full sample, internal consistency was acceptable for all scales for each subsample of participants.

Convergent and Discriminant Validity Assessment

The Pearson r correlations between the CCSS, MMQ-S, Neuro-QoL, and DERS are presented in Table 5. The correlations between the CCSS and MMQ-S were calculated to gauge the convergent validity of the scales, hypothesizing that the scales would be positively correlated with a medium effect size. Confirming this hypothesis, the correlational analyses indicated that the CCSS and MMQ-S correlated positively and at a moderate magnitude for the full sample (r = .63, p < .01) and at a comparable magnitude for the subsamples stratified by gender and self-reported diagnoses of mental health conditions (r range: .60 to .70, p < .01, for all correlations).

Table 5.

Convergent and Discriminant Validity Analyses for the Full Sample and Subsamples Stratified by Gender and Self-Reported Diagnoses of Mental Health Conditions.

		CCSS	MMQ-S	Neuro-QoL	DERS
Full sample (N = 783)	CCSS	—
	MMQ-S	.63**	—
	Neuro-QoL	−.21**	−.27**	—
	DERS	.17 (z = 12.46)	.29 (z = 8.98)	−.38**	—
Men (n = 157)	CCSS	—
	MMQ-S	.66**	—
	Neuro-QoL	−.24**	−.27**	—
	DERS	.16* (z = 6.10**)	.29 (z = 4.41)	−.35**	—
Women (n = 623)	CCSS	—
	MMQ-S	.60**	—
	Neuro-QoL	−.21**	−.27**	—
	DERS	.17 (z = 10.10)	.27 (z = 7.57)	−.38**	—
Depression (n = 171)	CCSS	—
	MMQ-S	.68**	—
	Neuro-QoL	−.29**	−.29**	—
	DERS	.22 (z = 6.18)	.32 (z = 4.75)	−.40**	—
Anxiety disorder (n = 225)	CCSS	—
	MMQ-S	.67**	—
	Neuro-QoL	−.25**	−.29**	—
	DERS	.13 (z = 7.65**)	.24 (z = 5.98)	−.37**	—
Attention-deficit/hyperactivity disorder (n = 101)	CCSS	—
	MMQ-S	.70**	—
	Neuro-QoL	−.25*	−.49**	—
	DERS	.20* (z = 5.20**)	.33 (z = 3.77)	−.44**	—
Comorbid mental health conditions (n = 179)	CCSS	—
	MMQ-S	.67**	—
	Neuro-QoL	−.28**	−.31**	—
	DERS	.19* (z = 6.44**)	.31 (z = 4.73)	−.36**	—
No mental health conditions (n = 491)	CCSS	—
	MMQ-S	.61**	—
	Neuro-QoL	−.19**	−.23**	—
	DERS	.20 (z = 8.78)	.29 (z = 6.71)	−.34**	—

Note. CCSS, MMQ-S, and DERS scores are raw sum scores of all items. Neuro-QoL scores are norm-referenced T scores. For each group, the correlation between the CCSS and MMQ-S was compared with the correlation between the each compensatory cognitive strategies scale and the DERS (Meng et al., 1992). A significant z value in the parentheses following these correlations indicates that the correlation between the CCSS and MMQ-S was larger than the correlation between each scale and the DERS. CCSS = Compensatory Cognitive Strategies Scale; MMQ-S = Multifactorial Memory Questionnaire–Strategies; Neuro-QoL = Neuro-Quality of Life Short Form v2.0–Cognitive Function; DERS = Difficulties in Emotion Regulation Scale.

p < .05. **p < .01.

The correlations between the compensatory cognitive strategy scales and the DERS were examined to evaluate discriminant validity, hypothesizing that these scales would have low correlations as measures of separate largely unrelated constructs. Again, confirming this hypothesis, the correlations between the DERS and the CCSS (r = .17, p < .01) and the MMQ-S (r = .29, p < .01) were small, albeit significant considering the well-powered sample size. Within the subsamples with and without self-reported diagnoses of mental health conditions, the correlations between the DERS and the CCSS were of similar magnitude, but varied in their absolute value (r range: .13 to .22). The correlation between the DERS and CCSS was nonsignificant in the subsample with anxiety disorders, but significant in the subsamples with depression (p < .01), ADHD (p < .05), comorbid mental health conditions (p < .05), and no mental health conditions (p < .01). The DERS correlated significantly with the MMQ-S for all subsamples at a similar magnitude to the full sample (r range: .24 to .33, p < .01 for all correlations). The correlations between the CCSS and the MMQ-S were compared with the correlations between each scale and the DERS for the full sample and each subsample. For the full sample and all subsamples, the correlation between the CCSS and MMQ-S was significantly higher than the correlation of the CCSS with the DERS (all p values <.01) and the MMQ-S with the DERS (all p values <.01).

Subjective Cognitive Concerns and Compensatory Cognitive Strategy Use

The Neuro-QoL was the only norm-referenced questionnaire used in the current study, and a sizeable minority of the full sample (30.5%) reported moderate subjective cognitive complaints (i.e., T < 40). This sizeable minority was still greater than expectation based on normative comparisons (i.e., T = 40 corresponds to the 16th percentile) even when removing participants with self-reported diagnoses of mental health conditions: 23.8% of the sample without any reported mental health condition endorsed moderate cognitive symptom severity.

The correlation between the compensatory cognitive strategy scales and the Neuro-QoL were examined to determine how closely compensatory cognitive strategy use corresponded to subjective cognitive concerns in everyday life. The correlation between the CCSS and Neuro-QoL was r = −.21 (p < .01), and was similar for subsamples of participants stratified by gender or self-reported diagnoses of mental health conditions (r range: −.21 to −.29, p < .05, for subsample with ADHD, and p < .01, for all other subsamples). The correlation between the MMQ-S and Neuro-QoL was r = −.27 (p < .01) for the full sample. This correlation was moderate for the subsample with ADHD (r = −.49, p < .01), but comparable to the full sample for subsamples stratified by gender and other self-reported diagnoses (r range: −.23 to −.31, p < .01, for all correlations). As hypothesized, the observed correlations aligned with the small negative correlations observed in previous literature.

Factor Analysis

The one-factor CCSS model failed to meet criteria for adequate model fit, χ² = 1,605.80 (p < .0001), df = 252, CFI = 0.805, TLI = 0.787, RMSEA = 0.083 (95% CI [0.079, 0.087]), albeit the RMSEA approached an upper-bound threshold for acceptable fit. The factor loadings for this model are presented in Table 3. The CCSS items were categorized under three factors: attention and processing speed, memory, and executive functions. The item assignments and factor loadings for the three-factor model are provided in Table 3. The three-factor CCSS model showed comparable fit to the one-factor model, χ² = 1,574.66 (p < .0001), df = 249, CFI = 0.809, TLI = 0.789, RMSEA = 0.082 (95% CI [0.079, 0.086]). The attention and processing speed factor correlated highly with the memory factor (r = .88) and the executive functions factor (r = .90). The memory factor and executive functions factor also correlated highly (r = .91). The one-factor MMQ-S model did not meet criteria for adequate model fit, χ² = 1,795.09, df = 152, p < .0001; CFI = 0.777, TLI = 0.749, RMSEA = 0.117 (95% CI [0.113, 0.122]). The factor loadings for this model are presented in Table 4.

Discussion

The CCSS and MMQ-S total scores showed evidence for acceptable reliability and convergent and discriminant validity in a sample of young adults with and without self-reported mental health conditions attending college in the United States. The observed internal consistency for each scale and the correlation between the CCSS and MMQ-S were consistent with prior estimates in samples of older adults and patients with multiple sclerosis (Becker et al., 2019; Troyer & Rich, 2002). Evidence for discriminant validity was also apparent, in that the correlations between compensatory cognitive strategy use and emotional regulation were significantly smaller than the correlations between the CCSS and MMQ-S. The correspondence between compensatory cognitive strategy use and subjective cognitive concerns was also similar to prior studies, with small negative correlations observed for the current sample. These negative correlations provide evidence for concurrent validity, in that participants with greater cognitive concerns use compensatory cognitive strategies at a greater frequency in their everyday life.

A confirmatory factor analysis was run to examine the internal structure of the CCSS and MMQ-S, hypothesizing a one-factor structure for each based on prior use of the total score for the CCSS (Becker et al., 2019) and a previous principal components analysis for the MMQ-S (Troyer & Rich, 2002). The one-factor models did not meet most criteria for adequate fit for either scale. For the CCSS, fit was not improved when an alternative three-factor model was examined, and the three factors (i.e., attention and processing speed, memory, and executive functions) correlated highly with one another (r range: .88 to .91). The strongest evidence supported the one-factor CCSS model, which was parsimonious and had a RMSEA that approximated the lenient threshold for acceptable fit. However, overall, these findings show weak evidence for any of the scale structures evaluated.

Clinical Implications

A sizeable minority of participants reported a moderate severity of subjective cognitive concerns, even after excluding participants with self-reported mental health conditions that may affect cognitive functioning. Even without any neurological or psychiatric conditions, young adults may struggle to engage in cognitively demanding tasks, and subjectively notice personal difficulties with their cognitive functioning that may go undetected by objective neuropsychological tests. Prior research has shown that subjective cognitive concerns tend to be more related to mental health symptoms than objective cognitive functioning (Serra-Blasco et al., 2019; Srisurapanont et al., 2017), and underlying symptoms of depression could explain the high frequency of elevated cognitive concerns observed in the current sample. Of note, both participants who did and did not have prior diagnoses of mental health conditions reported elevated cognitive concerns, but the severity of current symptoms was not directly measured. Considering the significant stress of postsecondary education (Robotham & Julian, 2006) and the mental health problems experienced by college students (Auerbach et al., 2016; Blanco et al., 2008), cognitive remediation may be a helpful style of intervention for clinicians in college counseling centers seeking to support students managing many personal and academic demands.

Compensatory cognitive strategy use was quite common in the current sample. Nearly all participants reported using some strategies (e.g., taking notes, keeping a to-do list, or maintaining a calendar), whereas other strategies were implemented less often, such as spaced retrieval. Some students may not be familiar with spaced retrieval, but this learning strategy has led to success in college math courses (Hopkins et al., 2016); and increased use could lead to greater academic success. Compensatory cognitive strategies have been used by high-functioning students with learning disabilities to support success in postsecondary education (Reis et al., 2000), and a compensatory cognitive training intervention led to reduced academic difficulties in college students with mental health conditions (Mullen et al., 2017). A clinician could apply such an intervention independent of, or supplemental to, more traditional mental health treatment to support academic functioning, potentially using the CCSS or MMQ-S to assess baseline use of compensatory cognitive strategies and monitor change posttreatment.

Limitations

This study had limitations that affect the generalizability of the findings. Although embedded validity questions addressed careless responding during survey administration, other threats to validity were not evaluated, such as social desirability. Participants were current college students; and, for this population, it may be desirable to report, or at least self-perceive, a greater use of strategies that would help with academic performance. However, the survey responses were nonidentifiable and unrelated to the allocation course credit, which both reduce the risk for a social desirable response pattern affecting the validity of responses. The study was also limited by its cross-sectional design, meaning estimates of test–retest reliability could not be estimated. A test–retest estimate of reliability may be helpful for researchers and practitioners who readminister the CCSS or MMQ-S on multiple occasions, such as those who conduct interventions. The availability of a test–retest reliability estimate would also be useful for calculating a reliable change index to determine if an individual participant or patient improved with intervention. A future investigation examining the test–retest reliability for the CCSS, MMQ-S, or another scale assessing compensatory cognitive strategy use would be beneficial for cognitive rehabilitation research and clinical practice.

Another limitation pertains to participant recruitment, which occurred at a single university psychology department. Although the sample included participants of diverse identities, they identified predominantly as White, with an overrepresentation of women. This sample is not necessarily representative of young adults from different states, regions, or countries, or those who do not attend college or university. The enrollment of participants in college may contribute to greater self-perceived cognitive problems and a greater recruitment of cognitive strategies, considering the high cognitive demands associated with college coursework.

The CCSS and MMQ-S are not the only measures available to evaluate compensatory cognitive strategy use. Other assessment instruments, such as the Cognitive Problems and Strategies Assessment (Twamley et al., 2012) and the Metamemory in Adulthood Questionnaire–Strategy subscale (Dixon & Hultsch, 1983), may have valuable items that address compensatory cognitive strategies not included in the CCSS and MMQ-S. However, these additional measures were not administered in the current study. The relationships between these measures, the CCSS, and the MMQ-S remain unknown and could be the focus of future research on the convergent validity of these scales among young adults.

Future Directions

The CCSS and MMQ-S show many sound psychometric properties for use with young adults based on the current findings, but prior researchers did not design these measures for use with healthy young adults. The cognitive strategies applicable to a young adult may not align with those applicable to an older adult. Whereas a college student may need strategies to focus while studying for a test or manage a busy schedule (e.g., sports or exercise, classes, and work), a retired older adult may need strategies to help remember a phone number, a grocery list, or a doctor’s appointment. Compared with the MMQ-S, some added strengths of the CCSS are its recent development and inclusion of items pertaining to technology use (Becker et al., 2019), which may make this scale more appropriate for use with college students. However, both scales showed comparable psychometric properties, including acceptable reliability and evidence for convergent validity and discriminant validity. This study adds to existing research on the psychometric properties of instruments assessing compensatory cognitive strategy use in various populations, and provides evidence for the reliability and validity of the CCSS and MMQ-S for use with college students with and without mental health conditions. The findings do not indicate a preference for either scale; and the absence of a clear factor structure for both scales may suggest that future researchers could further refine these instruments by modifying their content, combining items from different scales, or including new items specifically pertinent to college students.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Justin E. Karr

References

Auerbach

R. P.

Alonso

Axinn

W. G.

Cuijpers

Ebert

D. D.

Green

J. G.

Hwang

Kessler

R. C.

Liu

Mortier

Nock

M. K.

Pinder-Amaker

Sampson

N. A.

Aguilar-Gaxiola

Al-Hamzawi

Andrade

L. H.

Benjet

Caldas-De-Almeida

J. M.

Demyttenaere

. . . Bruffaerts

(2016). Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychological Medicine, 46(14), 2955-2970. https://doi.org/10.1017/S0033291716001665

Becker

Stuifbergen

A. K.

Henneghan

Morrison

Seo

E. J.

Zhang

(2019). An initial investigation of the reliability and validity of the Compensatory Cognitive Strategies Scale. Neuropsychological Rehabilitation, 29(5), 739-753. https://doi.org/10.1080/09602011.2017.1329154

Belanger

H. G.

Vanderploeg

R. D.

(2005). The neuropsychological impact of sports-related concussion: A meta-analysis. Journal of the International Neuropsychological Society, 11(4), 345-357. https://doi.org/10.1017/S1355617705050411

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588-606. https://doi.org/10.1037/0033-2909.88.3.588

Blanco

Okuda

Wright

Hasin

D. S.

Grant

B. F.

Liu

S. M.

Olfson

(2008). Mental health of college students and their non-college-attending peers: Results from the national epidemiologic study on alcohol and related conditions. Archives of General Psychiatry, 65(12), 1429-1437. https://doi.org/10.1001/archpsyc.65.12.1429

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

K. A.

Long

J. S.

(Eds.), Testing structural equation models (Vol. 21, Issue 2, pp. 136-162). Sage. https://doi.org/10.1177/0049124192021002005

Cella

Lai

J. S.

Nowinski

C. J.

Victorson

Peterman

Miller

Bethoux

Heinemann

Rubin

Cavazos

J. E.

Reder

A. T.

Sufit

Simuni

Holmes

G. L.

Siderowf

Wojna

Bode

McKinney

Podrabsky

. . . Moy

(2012). Neuro-QOL: Brief measures of health-related quality of life for clinical research in neurology. Neurology, 78(23), 1860-1867. https://doi.org/10.1212/WNL.0b013e318258f744

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://doi.org/10.1007/BF02310555

Dixon

R. A.

Hultsch

D. F.

(1983). Structure and development of Metamemory in Adulthood. Journals of Gerontology, 38(6), 682-688. https://doi.org/10.1093/geronj/38.6.682

10.

Dougan

B. K.

Horswill

M. S.

Geffen

G. M.

(2014). Athletes’ age, sex, and years of education moderate the acute neuropsychological impact of sports-related concussion: A meta-analysis. Journal of the International Neuropsychological Society, 20(1), 64-80. https://doi.org/10.1017/S1355617712001464

11.

Enders

C. K.

(2010). Applied missing data analysis. Guilford Press.

12.

Erdfelder

Faul

Buchner

Lang

A. G.

(2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149-1160. https://doi.org/10.3758/BRM.41.4.1149

13.

Gershon

R. C.

Lai

J. S.

Bode

Choi

Moy

Bleck

Miller

Peterman

Cella

(2012). Neuro-QOL: Quality of life item banks for adults with neurological disorders: Item development and calibrations based upon clinical and general population testing. Quality of Life Research, 21(3), 475-486. https://doi.org/10.1007/s11136-011-9958-8

14.

Gratz

K. L.

Roemer

(2004). Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment, 26(1), 41-54. https://doi.org/10.1023/B:JOBA.0000007455.08539.94

15.

Gratz

K. L.

Roemer

(2008). Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment, 30(4), 315-315. https://doi.org/10.1007/s10862-008-9102-4

16.

Hallion

L. S.

Steinman

S. A.

Tolin

D. F.

Diefenbach

G. J.

(2018, April 19). Psychometric properties of the Difficulties in Emotion Regulation Scale (DERS) and its short forms in adults with emotional disorders. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2018.00539

17.

Hancock

G. R.

Freeman

M. J.

(2001). Power and sample size for the root mean square error of approximation test of not close fit in structural equation modeling. Educational and Psychological Measurement, 61(5), 741-758. https://doi.org/10.1177/00131640121971491

18.

Hanmer

Jensen

R. E.

Rothrock

(2020). A reporting checklist for HealthMeasures’ patient-reported outcomes: ASCQ-Me, Neuro-QoL, NIH Toolbox, and PROMIS. Journal of Patient-Reported Outcomes, 4(1), Article 21. https://doi.org/10.1186/s41687-020-0176-4

19.

Hopkins

R. F.

Lyle

K. B.

Hieb

J. L.

Ralston

P. A. S.

(2016). Spaced retrieval practice increases college students’ short- and long-term retention of mathematics knowledge. Educational Psychology Review, 28(4), 853-873. https://doi.org/10.1007/s10648-015-9349-8

20.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

21.

Huckans

Pavawalla

Demadura

Kolessar

Seelye

Roost

Twamley

E. W.

Storzbach

(2010). A pilot study examining effects of group-based cognitive strategy training treatment on self-reported cognitive problems, psychiatric symptoms, functioning, and compensatory strategy use in OIF/OEF combat veterans with persistent mild cognitive disorder. Journal of Rehabilitation Research and Development, 47(1), 43-60. https://doi.org/10.1682/JRRD.2009.02.0019

22.

Jak

A. J.

Jurick

Crocker

L. D.

Sanderson-Cimino

Aupperle

Rodgers

C. S.

Thomas

K. R.

Boyd

Norman

S. B.

Lang

A. J.

Keller

A. V.

Schiehser

D. M.

Twamley

E. W.

(2019). SMART-CPT for veterans with comorbid post-traumatic stress disorder and history of traumatic brain injury: A randomised controlled trial. Journal of Neurology, Neurosurgery and Psychiatry, 90(3), 333-341. https://doi.org/10.1136/jnnp-2018-319315

23.

Karr

J. E.

Areshenkoff

C. N.

Garcia-Barrera

M. A.

(2014). The neuropsychological outcomes of concussion: A systematic review of meta-analyses on the cognitive sequelae of mild traumatic brain injury. Neuropsychology, 28(3), 321-336. https://doi.org/10.1037/neu0000037

24.

Lai

J. S.

Wagner

L. I.

Jacobsen

P. B.

Cella

(2014). Self-reported cognitive concerns and abilities: Two sides of one coin? Psycho-Oncology, 23(10), 1133-1141. https://doi.org/10.1002/pon.3522

25.

Lezak

M. D.

Howieson

D. B.

Loring

D. W.

(2012). Neuropsychological assessment (5th ed.). Oxford University Press.

26.

Little

R. J. A.

(1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198-1202. https://doi.org/10.1080/01621459.1988.10478722

27.

McDermott

L. M.

Ebmeier

K. P.

(2009). A meta-analysis of depression severity and cognitive function. Journal of Affective Disorders, 119(1-3), 1-8. https://doi.org/10.1016/j.jad.2009.04.022

28.

Meade

A. W.

Craig

S. B.

(2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437-455. https://doi.org/10.1037/a0028085

29.

Meng

X. L.

Rosenthal

Rubin

D. B.

(1992). Comparing correlated correlation coefficients. Psychological Bulletin, 111(1), 172-175. https://doi.org/10.1037/0033-2909.111.1.172

30.

Mullen

M. G.

Thompson

J. L.

Murphy

A. A.

Malenczak

Giacobbe

Karyczak

Holloway

K. E.

Twamley

E. W.

Silverstein

S. M.

Gill

K. J.

(2017). Evaluation of a cognitive remediation intervention for college students with psychiatric conditions. Psychiatric Rehabilitation Journal, 40(1), 103-107. https://doi.org/10.1037/prj0000254

31.

Reis

S. M.

McGuire

J. M.

Neu

T. W.

(2000). Compensation strategies used by high-ability students with learning disabilities who succeed in college. Gifted Child Quarterly, 44(2), 123-134. https://doi.org/10.1177/001698620004400205

32.

Robotham

Julian

(2006). Stress and the higher education student: A critical review of the literature. Journal of Further and Higher Education, 30(2), 107-117. https://doi.org/10.1080/03098770600617513

33.

Roth

R. M.

Isquith

P. K.

Gioia

G. A.

(2014). Assessment of executive functioning using the Behavior Rating Inventory of Executive Function (BRIEF). In Goldstein

Naglieri

(Eds.), Handbook of executive functioning (pp. 301-331). Springer. https://doi.org/10.1007/978-1-4614-8106-5_18

34.

Semkovska

Quinlivan

O’Grady

Johnson

Collins

O’Connor

Knittle

Ahern

Gload

(2019). Cognitive function following a major depressive episode: A systematic review and meta-analysis. Lancet Psychiatry, 6(10), 851-861. https://doi.org/10.1016/S2215-0366(19)30291-3

35.

Serra-Blasco

Torres

I. J.

Vicent-Gil

Goldberg

Navarra-Ventura

Aguilar

Via

Portella

M. J.

Figuereo

Palao

Lam

R. W.

Cardoner

(2019). Discrepancy between objective and subjective cognition in major depressive disorder. European Neuropsychopharmacology, 29(1), 46-56. https://doi.org/10.1016/j.euroneuro.2018.11.1104

36.

Srisurapanont

Suttajit

Eurviriyanukul

Varnado

(2017). Discrepancy between objective and subjective cognition in adults with major depressive disorder. Scientific Reports, 7(1), Article 3901. https://doi.org/10.1038/s41598-017-04353-w

37.

Stephan

R. A.

Alhassoon

O. M.

Allen

K. E.

Wollman

S. C.

Hall

Thomas

W. J.

Gamboa

J. M.

Kimmel

Stern

Sari

Dalenberg

C. J.

Sorg

S. F.

Grant

(2017). Meta-analyses of clinical neuropsychological tests of executive dysfunction and impulsivity in alcohol use disorder. American Journal of Drug and Alcohol Abuse, 43(1), 24-43. https://doi.org/10.1080/00952990.2016.1206113

38.

Storzbach

Twamley

E. W.

Roost

M. S.

Golshan

Williams

R. M. O

Neil

Jak

A. J.

Turner

A. P.

Kowalski

H. M.

Pagulayan

K. F.

Huckans

(2017). Compensatory cognitive training for Operation Enduring Freedom/Operation Iraqi Freedom/Operation New Dawn veterans with mild traumatic brain injury. Journal of Head Trauma Rehabilitation, 32(1), 16-24. https://doi.org/10.1097/HTR.0000000000000228

39.

Strauss

Sherman

E. M. S.

Spreen

(2006). A compendium of neuropsychological tests: Administration, norms, and commentary (3rd ed.). Oxford University Press.

40.

Tanaka

J. S.

(1987). “How big is big enough?” Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58(1), 134-146. https://doi.org/10.2307/1130296

41.

Troyer

A. K.

(2001). Improving memory knowledge, satisfaction, and functioning via an education and intervention program for older adults. Aging, Neuropsychology, and Cognition, 8(4), 256-268. https://doi.org/10.1076/anec.8.4.256.5642

42.

Troyer

A. K.

Rich

J. B.

(2002). Psychometric properties of a new metamemory questionnaire for older adults. Journals of Gerontology: Series B Psychological Sciences and Social Sciences, 57B(1), 19-27. https://doi.org/10.1093/geronb/57.1.P19

43.

Twamley

E. W.

Thomas

K. R.

Gregory

A. M.

Jak

A. J.

Bondi

M. W.

Delis

D. C.

Lohr

J. B.

(2015). CogSMART compensatory cognitive training for traumatic brain injury: Effects over 1 year. Journal of Head Trauma Rehabilitation, 30(6), 391-401. https://doi.org/10.1097/HTR.0000000000000076

44.

Twamley

E. W.

Vella

Burton

C. Z.

Heaton

R. K.

Jeste

D. V.

(2012). Compensatory cognitive training for psychosis: Effects in a randomized controlled trial. Journal of Clinical Psychiatry, 73(9), 1212-1219. https://doi.org/10.4088/JCP.12m07686