Abstract
University students often experience high levels of stress and, in some cases, the stress leads to tragic outcomes. An important question is whether roommates can perceive the level and change in distress in their peers. We examined self- and other-reports of 187 same-sex undergraduate dyads at two times in a spring semester. Using the truth and bias model, we found that roommates tended to underestimate their partner’s distress at both time points, and that ratings were equally influenced by truth and self-focus bias forces. For change, however, there was no evidence of directional (average) bias, and perceived change was only significantly related to the truth force. There were no consistent moderation effects by closeness or gender. These findings are interpreted in the context of person perception theory and the practical need for early warning about extreme distress in college students.
College years occur at a critical developmental stage, during which students learn to be more independent and to reduce their dependence on their parents. These years are typically full of challenges, including internal pressures such as expectations of self, personal adaptations to college, and external pressures imposed by others (Bishop, Gallagher, & Cohen, 2000; Kadison & DiGeronimo, 2004). Exposure to stressors places students at risk for adverse outcomes, such as mental health problems, withdrawal from school, thoughts of self-injury, and even suicidal ideation and attempts (Gallagher, Sysko, & Zhang, 2001; Kadison & DiGeronimo, 2004; Sher, Wood, & Gotham, 1996).
Educational institutions are usually proactive in reaching out to students, providing counseling services and screening for mental health problems, but students do not always avail themselves of those services (Eisenberg, Golberstein, & Gollust, 2007; Masuda et al., 2009; Rosenthal & Wilson, 2008). There is some evidence that peers are the preferred source of advice or support for young adults in distress (Deane, Wilson, & Ciarrochi, 2001; Sharkin, Plageman, & Mangold, 2003). This has prompted some colleges to develop peer helping programs to assist with issues such as transition and adjustment to college (Mattanah et al., 2010) and alcohol and drug prevention (Larimer & Cronce, 2002; Larimer, Kilmer, & Lee, 2005). Often colleges use resident assistants (RAs) as the first point of contact for distressed college students. RAs can be viewed as older peers to the younger students, and they are typically trained as gatekeepers to recognize, evaluate, and refer students who might be experiencing emotional or psychological problems (Cross, Matthieu, Lezine, & Knox, 2010; Pasco, Wallack, Sartin, & Dayton, 2012; Tompkins & Witt, 2009).
An alternate group of peers that can be considered are students’ roommates. Roommate relationships are unique among students’ interpersonal relationships because roommates maintain a living space together. Although they might not have known each other at the beginning of the semester, they typically have a common goal of getting along, and they often negotiate responsibilities, and share experiences over time. Frequent contact and shared experiences not only necessitate communication but also create an opportunity for roommates to know and understand each other. Considering the nature of roommate relationships, roommates may be in a critical position to help and refer at-risk students although they do not necessarily have the same training and duty as RAs. An important social psychology question that needs to be addressed is whether college roommates can provide an additional source of information about the mental health and distress of their peers.
There is considerable evidence from the person perception literature that peers can provide systematic information about trait assessment (e.g., Bernieri, Zuckerman, Koestner, & Rosenthal, 1994; Hayes & Dunning, 1997; Kurtz & Sherker, 2003), but there is less evidence regarding whether they can accurately assess mood and distress (e.g., Gros, Milanak, & Hershenberg, 2013; Watson & Clark, 1991), and there is virtually no evidence that they can detect changes in distress levels over time. To address this gap, we carried out secondary analyses of data from a project wherein same-sex college roommate dyads were recruited and asked to report distress for themselves and their roommate at two times in the spring semester of an academic year, 2 months apart. 1 Using a latent variable variation of the truth and bias model (TBM; West & Kenny, 2011), we examined the degree to which roommates were accurate in assessing each other’s distress level as well as change in distress. We also explored whether the roommate relationship closeness and gender affected the accuracy in the perception of distress.
Perceiving Personality Traits
The literature on accuracy in personality can inform investigations of accuracy of mental states. Funder (1995, 2012) proposed the realistic accuracy model (RAM) that reveals how and when accurate personality judgments occur. RAM specifies four important moderators of accuracy: (a) the characteristics of the target who is judged, (b) the individuals making the judgments, (c) the specific trait that is judged, and (d) the information upon which the judgment depends. A substantial body of research has examined these moderators. For example, factors such as psychological adjustment, social status, and socialization affect the degree to which a person’s personality can be accurately understood by others (Human & Biesanz, 2013). Judgment ability is associated with several positive individual differences including overall adjustment, social skills, agreeableness, and interpersonal sensitivity (Hall, Andrzejewski, & Yopchick, 2009; Human & Biesanz, 2011; Letzring, 2008). Moreover, traits that are highly visible, such as extraversion, are more accurately judged by informants (Connelly & Ones, 2010; Connolly, Kavanagh, & Viswesvaran, 2007) and the amount and type of information available to judges is also related to accuracy (Blackman & Funder, 1998; Hirschmller, Egloff, Schmukle, Nestler, & Back, 2015; Letzring & Human, 2014).
Perceiving Affective States
Affective states are different from traits in that they can change from moment to moment, and hence the ratings need to emphasize current information. Nonetheless, a person’s affective state level is known to be related to traits such as neuroticism (Leger, Charles, Turiano, & Almeida, 2016) and states are often related to behaviors in ways similar to traits. This suggests that Funder’s RAM principles may also be applicable to state perception. For instance, Gros and colleagues (2013) conducted a study of 139 friendship dyads in which targets completed a standard questionnaire battery that included a wide range of symptoms of anxiety and depression, and their close friends (informants) completed questionnaires on their view of the target participants’ symptoms. They found that the correlations between targets’ and informants’ reports were higher for more visible symptoms (e.g., behavioral: r = .30) compared with less visible symptoms (e.g., physiological: r = .07). This result was consistent with the “good trait” principle of Funder’s RAM (Connelly & Ones, 2010; Connolly et al., 2007).
However, the principles developed in personality assessments do not always apply to assessments of mental states. The magnitude of the correlations found by Gros et al. (2013) between the targets’ and informants’ reports was only small-to-medium in size (the average correlation was .22). This was lower than typically found in the personality literature (Connelly & Ones, 2010), but this pattern of results was consistent with previous research on perceptions of affective states. For example, Watson and Clark (1991) found self–peer correlations ranging from .19 to .41 for six different negative affect scales, such as sadness. Okazaki (2002) reported correlations of .37 and .39, respectively, between peer reports and students’ own reports of social anxiety and depression.
In addition, self–other agreements in personality traits tend to increase with longer and closer acquaintanceship (Biesanz, West, & Millevoi, 2007; Funder & Colvin, 1988; Thomas & Fletcher, 2003), especially when raters are newly acquainted. In contrast, Gros et al. (2013) did not generally find associations between closeness and level of agreement on anxiety and depression. Their result is consistent with other studies in examining assessments of affective states among friendship samples (Okazaki, 2002; Watson, Hubbard, & Wiese, 2000).
Other principles besides those from personality perception are likely to apply to judgments of mental states. There are good reasons to expect that perceivers’ evaluations of their peer’s distress would be influenced by their own experiences with distress. The perceiver’s own experience with distress in the context of a college semester provides one possible working model of the peer’s stress experience. If this is the case, perception may be partially rooted in self-reference in addition to the actual observation of the roommate’s behavior. More specifically, perceivers have a stronger tendency to resort to assumptions of self–target similarity to “fill the gap” when perceivers lack valid information upon which to base their target ratings (e.g., Paunonen & Hong, 2013; Paunonen & Kam, 2014). Because negative affect is often experienced as an internal state that may not be obvious or observable, it seems logical to postulate that perceiver’s own distress level would be associated with the judgment of the roommate. In addition to self-projection explanation, response bias factors might induce correlation between self-ratings and roommate judgments. For example, regardless of describing their own or their roommate’s level of distress, some individuals may consistently use the lower portion of the rating scale, whereas other individuals may consistently use higher ratings. Such response sets could lead to the same pattern of self–other correlation.
One formal way to evaluate the impact of perceiver’s perspective on judgments of the peer is the TBM of West and Kenny (2011). To our knowledge, this perspective has not yet been applied to data on peer assessments of mental states.
TBM
West and Kenny (2011) proposed the TBM to decompose ratings into variance due to rater effects and due to the features of the target of the rating. The TBM makes a clear theoretical distinction among three forms of accuracy/inaccuracy: one due to a “truth force,” another due to a “bias force,” and also an overall “directional bias.” The magnitude of these components can be estimated using regression methods when the rater’s judgment of the target as well as the rater’s and the target’s self-reports are obtained on the same measurement scale.
Figure 1 shows a representation of the path model that operationally defines the TBM components. Suppose that two students, Ming and Nan, are roommates who report their own distress levels and also judge the distress of the other. Let JN:M be Nan’s judgment of Ming’s distress, and JM:N be Ming’s judgment of Nan’s distress. Furthermore, let TN and TM be the self-reports of Nan’s and Ming’s distress. Figure 1 shows that TN and TM might be correlated, perhaps due to some common stressor or mutual influence. Each of these self-reports is used to predict the judgment of Ming by Nan (JN:M) and of Nan by Ming (JM:N). West and Kenny (2011) defined the self-reports as the truth (TM, TN) and the bT paths, which connect TM to JN:M and TN to JM:N, as truth force. Adjusting for the truth force, the relation of Nan’s own distress (TN) to her rating of Ming’s distress (JN:M) and the analogous relation of TM to JM:N are called bias force by West and Kenny (2011). These are represented by bB in Figure 1. In addition to these associations, the TBM also can reveal the tendency for systematic overreporting or underreporting of distress. This is done by centering all four variables, TN, TM, JN:M, and JM:N, around the average of the self-report measures, TN and TM. In Figure 1, the b0 coefficient (the intercept term) is the level of judged-distress made by a rater with average distress for a roommate who also has average distress. The sign and magnitude of b0 reveals directional bias in the TBM. West and Kenny (2011) noted that the bias force might actually contribute to overall accuracy if the self-perceptions of the two partners are highly correlated. This contribution to accuracy is indirect, going through the (TM, TN) correlation and then through the bB path.

Basic truth and bias model estimated with manifest variables.
The application of the TBM assumes that the self-reports (indicators of truth) are measured without error. If one or both of the self-reports are contaminated by error, then the regression estimates are likely to be biased (e.g., Ledgerwood & Shrout, 2011). The estimated direct effect of a predictor with measurement error is often too small, whereas the estimated effect of a covariate might be too large because of inadequate adjustment. One way to adjust for the impact of measurement is to employ structural equation models (SEM; Kline, 2005; see Figure 2).

Truth and bias model estimated with latent variables from structural equation model.
In addition to level of distress at a fixed time point, a friend or peer might note a change in distress level over the course of a week, month, or semester. Noticing that a person has “changed” is a more complex process, as it requires consideration of both past and current states. However, change can also be inferred by collecting repeated judgments of distress state and then comparing the current report with a previous report. Little is known about the truth and bias forces as they apply to inferred changes in psychological states, but clearly, such information is important in monitoring the health and well-being over time such as over the course of a college semester as stress tends to fluctuate.
So far, we have discussed truth and bias forces without considering the processes by which accuracy and bias in judgments occur. To better use peer judgments to screen or monitor the mental states of college students, it is important to understand what circumstances cause a student to be more accurate or biased in perceiving his or her partner. The TBM uses moderators to examine these perception processes, specifically focusing on how moderators change the truth force, the bias force, the directional bias, or all three.
We are particularly interested in relationship closeness. Here, let C be the relationship closeness, and C × TN and C × TM be the product between closeness and the self-reports of Nan’s (TN) and Ming’s distress (TM; see Figure 3). There are two reasons we think closeness would improve the truth force. One is that current distress state has a trait component, and closer friends will have more information about their peer’s general tendency to be higher or lower on distress. Another reason is that closer friends are likely to spend more time together and engage in self-disclosure. Thus, the unusual circumstances of the more state-like influences will be better known to the closer friend. For example, if a student is having trouble with a current course or with a romantic relationship partner, this trouble is more likely to be known by a close friend than a more distant roommate. The bTC paths in Figure 3 that connect C × TM to JN:M and C × TN to JM:N represent the impact of closeness on the truth force.

Truth and bias model including a moderator estimated with manifest variables.
We also acknowledge that there might be bias forces that diminish the accuracy of close friends, such as positivity bias (Fletcher & Kerr, 2010; Murray, Holmes, & Griffin, 1996a, 1996b) but we believe these biases will be smaller than the trait and state informational effects just described. The relation of C × TM to Ming’s rating of Nan (JM:N) and the analogous relation of (C × TN) to (JN:M) estimate the influence of closeness on the bias force. The b0C paths, which connect C to JN:M and JM:N, reflect the moderating effect of closeness on directional bias.
In addition to closeness, gender is another moderator of interest. There is evidence suggesting that gender role affects communication. For example, women, and people higher in femininity, tend to be better at providing others with appropriate and diagnostic cues regarding one’s inner thoughts, feelings, and dispositions as compared with men and people higher in masculinity (Zuckerman, DeFrank, Spiegel, & Larrance, 1982). Women also tend to provide a greater amount of emotional cues than men during interaction (Brody & Hall, 1993; Gross & John, 2003). These differences may make women’s emotional experiences more available and subsequently easier to be read. Moreover, other research generally shows that women are often better decoders of nonverbal behaviors than men (Hall, 1990; Hall, Carter, & Horgan, 2000; Rosip & Hall, 2004). Although some researchers (Kenny & Acitelli, 2001; Watson, 1989) found no effects for gender in the accuracy of ratings, we believe a complete consideration of truth and bias effects on roommate distress should explore the possibility of gender moderation.
Current Research
In this article, we examine the accuracy and bias in assessments of distress states of college roommate dyads on each other and whether the accuracy and bias change as a function of the level of closeness and gender. We decompose the association of the other and self-reports into truth and bias forces according to the TBM of West and Kenny (2011). We extend the literature on TBM in two ways. First, we examine change in distress over two time points, as well as distress level at each time point. Specifically, we study how perceivers’ own distress and their partner’s actual distress influence the perception, to what extent the perception differs from the partner’s actual distress at the mean level, and how closeness and gender affect these three forces. Second, we adjust the TBM analysis for measurement error using SEM. To explore the implications of the findings, we examine the positive and negative predictive value (PPV and NPV; Pepe, 2003; Zhou, McClish, & Obuchowski, 2009) of the roommates’ reports for identifying the persons who have the highest distress levels and highest changes in distress over time.
We make six hypotheses about expected results. First, we predict that there will be a significant truth force in the perception of distress level—the perception will be moderately associated with the target’s self-rated mood states. Second, we predict that one’s own level of distress is an accessible model of normative distress in one’s social network and so we expect to find a significant bias force—the perception will be linked with the perceiver’s own state. These two effects are expected to hold after adjusting one for the other. Distress that is manifested by the roommate as internalizing affect (low positive affect) might not be behaviorally obvious to the rater. This leads to an expectation that the bias force might be relatively larger than the truth force. Nevertheless, the relative importance of the two forces has not been formally tested. We will explore this question as well.
Third, if the roommate peer’s judgments are to have utility in identifying students who have extreme levels of distress, then it will be important to know the degree to which roommates systematically rate their partner as more or less distressed than the partner rates herself. Insofar as roommates become friends, they might succumb to idealization effects observed in intimate relationships (Murray et al., 1996a, 1996b). Based on this literature, we expect to find a negative pattern of directional bias—students will systematically underestimate their roommate’s distress level.
Fourth, we are interested in whether roommates are able to monitor improvements or declines in their partner’s distress during the school year. Prior work on the perception of distress level do not speak to this question. We speculate that individuals with elevated distress might gradually disclose more symptoms like seeking reassurance (Starr & Davila, 2008) and self-verifying negative information (Swann, Wenzlaff, Krull, & Pelham, 1992), which is likely to lead to their roommate to detect this information and interpret it as increasing disorder. Therefore, we predict to see a significant truth force, such that roommates’ perception will be moderately associated with the target’s self-perceived change in affective states. Regarding whether roommates would project their own mood fluctuation to the perception (bias force) and whether roommates would have systematic over- or underrating of their partner’s change in distress (directional bias), we do not have explicit predictions.
Finally, we will examine the moderating effects of relationship closeness and dyad gender. We hypothesize that roommates who report having a closer friendship will be better at identifying their partner’s affective mood than those with more distant relationships (Hypothesis 5). Regarding the moderation of gender, we expect the truth force to be larger for women dyads (Hypothesis 6).
To test these hypotheses, we carry out secondary analyses of survey data obtained from college students and their roommates at two times (separated by 2 months) in a spring semester. The fact that the students had been roommates for 4 months before the February assessment means that they had a chance to become well acquainted.
Method
Participants
Participants were 187 college roommate pairs who were recruited as part of a study of response bias in panel studies (Shrout et al., 2017). The original study included additional participants who enrolled without roommates. To be included in the current analyses, participants needed to be paired with a roommate and both students needed to have provided data in two panel surveys (February and April). For the original study, undergraduate students were recruited in the fall of 2009 via posted flyers in academic and dorm buildings, personal recruitment on campus, and email announcements at two private universities in New York City. Students were encouraged to recruit one of their roommates to also participate and 74.3% of them did so, leading to 371 pairs and 128 single original participants. Participants in the original study were asked to complete up to four surveys and they were compensated US$10 per survey completed 2 for up to US$50 (four bimonthly surveys and the background questionnaire). For each completed survey, participants were also entered into one of five US$250 lotteries designed to increase compliance and reduce attrition throughout the year. Participants were debriefed after completion of the final survey. Of the roommate pairs, there were 187 in which both partners provided usable data in both February and April. The other 184 pairs had at least one partner who missed one or both surveys.
The roommate dyads available for the current analysis were all same-sex; 159 (85%) dyads were female and 28 (15%) dyads were male. More than half (54%) self-identified as White, and the others identified themselves as Asian (27%), biracial (10%), Hispanic (5%), or Black or African American (4%). Nearly half were first-year undergraduate students, 26% were second-year students, 8% were third-year students, and 15% reported being in their final year of college. The participants included in the current analysis (N = 2 × 187 = 374) were not significantly different from dyads in the original study who were not used (N = 2 × 184 = 368) in terms of racial composition and school year. However, about 31% of excluded dyads and 37% of excluded single participants were male, whereas only 15% of the current sample were male. Excluded single participants also had less senior students (1.7%) and Asian students (21%) than the current sample.
Procedures
After providing informed consent in September, participants completed a background questionnaire. As mentioned earlier, some participants completed surveys in October and December, but these data were intentionally missing from subsets of respondents and are not considered further in our analyses. In February and April, a reminder email was sent 1 week before the target survey date. On the following Tuesday, participants were emailed a link to their online survey and had 2.5 days to complete it. Reminder emails and text messages were sent on Wednesday and Thursday mornings.
Each survey included questions about their own and their roommates’ psychological and physical health. The order of the components of the surveys was randomized, such that for each pair of roommates, one target individual described himself or herself first and his or her roommate second, while his or her roommate described the target person first, and then himself or herself second.
Measures
The survey itself took an average of 50 min for participants to complete, and it included 36 measures related to psychological and physical well-being, alcohol use and beliefs, romantic relationships, coping, time usage, demographics, and others. We focused on general distress and closeness of roommate relationships.
Distress
Participants were asked to complete Kessler Psychological Distress Scale (K10; Kessler et al., 2002) for themselves as well as their roommates.
Rating for the self
Participants rated the extent to which they felt a particular feeling or emotion over the past 6 weeks on a scale ranging from 1 (none of the time) to 5 (all over the time). K10 scale had high Cronbach’s alphas in both February (α = .91) and April (α = .92). We calculated the mean scores of the 10 items for the two time points separately (Table 1). Higher scores indicated more distress.
Descriptive Statistics and Correlations for K10 Simple Mean Scores and Relationship Closeness Across 2 Months.
Note. Because M and N are indistinguishable, the means, standard deviations, and correlations of the two are constrained to be the same. February’s correlations appear above the diagonal, April’s correlations appear below the diagonal, and correlations between February and April appear on the diagonal in bold. The means and SDs for February are in the last two columns and the means and SDs for April are in the last two rows. N = 187 dyads. TM = Ming’s distress; TN = self-reports of Nan’s; JM:N = Nan by Ming; JN:M = Ming by Nan; CM = Ming’s reports of relationship closeness; CN = Nan’s reports of relationship closeness.
p < .05. **p < .01.
Rating for the roommate
We modified the K10 scale and asked participants to complete it from the roommate’s perspective. Here is one example item: “Please rate over the past 6 weeks: How often did your roommate feel hopeless?” Rate from 1 (none of the time) to 5 (all over the time). The modified K10 scale also had high alphas in February (α = .91) and April (α = .92). The mean scores of the perceived distress were listed in Table 1.
Closeness
At both assessments, participants completed 17 items about relationship closeness that were developed by West (2008). The items were designed to capture closeness from disclosure, perceived partner understanding, and received empirical support. An example item is, “It has been easy to express who I really am when I was with my roommate over the past few days.” Participants rated from 1 (not at all) to 5 (extremely). Two reverse coded items from the West (2008) measure had low factor loadings in our data and were set aside. The final 15-item measure had good internal consistency at both time points (February: α = .91, April: α = .92) and it was relatively stable over the two surveys (r = .83). Relationship closeness was also correlated with the amount of self-reported time roommates spent together in the past 6 weeks (February: r = .71, April: r = .76), which is an indication of the validity of the measure. The mean scores of relationship closeness were listed in Table 1.
Analytic Strategy
We tested the TBM using structural equation methods while adjusting for any measurement error in the self- and other-reports of distress. To make the adjustments, we need multiple observed indicators of distress measures. These can be obtained using the individual items of the K10, but this would lead to an analysis of 40 items and the 780 correlations among them. Instead, we opted to group items into parcels of two items each. This reduced the variable count by half, but the number of correlations to 190, a more than fourfold reduction.
Creating item parcels
The 10 K10 items are designed to yield a global measure of distress, but these items actually reflect two facets of distress, anxiety and depression. We used an explicit distributive strategy to construct five parcels such that each parcel had one item indicator from each of the underlying two facets. For example, the first parcel was the average response to the items that had the highest loadings of the two facets, the second parcel was the average response to the items that had the second highest loadings, and so on. For distress level, we created parcels with the item responses at two time points, separately.
Estimating TBM for distress level
Following the TBM, we estimated the degree to which roommates’ judgments of each other’s distress level were accurate and biased at each time point. The model is shown in Figure 4 and represented in Equations 1 and 2, where JM:N and JN:M represent the judgments of Nan by Ming and vice versa, TN and TM represent the self-report of Nan and Ming, and C and G represent closeness and gender. The partners’ reports on closeness were highly correlated within dyads (February: r = .62, April: r = .67) and we used the average to represent dyadic closeness. This dyadic-level closeness was centered around the grand mean (February: M = 3.59, April: M = 3.42). G was effect coded with 0.5 representing female dyads and –0.5 representing male dyads.
We noted that Equation 1 and Equation 2 for individual Ming and Nan within the dyad were parallel because the same-sex roommate dyads were treated as indistinguishable. All analyses were conducted using Mplus Version 7.1 with maximum likelihood estimation. Mplus allows the interactions of closeness and gender with latent variables TM and TN to be estimated using quasimaximum likelihood estimation (Klein & Muthn, 2007). The syntax for this model is available in supplemental material.

Truth and bias model including two moderators estimated with latent variables from structural equation model.
According to the TBM, the directional bias (b0), the truth force (bT), and the bias force (bB) all can change as a function of the moderators. The main effects of these forces refer to their effects when the moderators equal zero, and this can be interpreted as the average effect. The coefficients b0C, bTC, and bBC refer to the degree to which the directional bias, the truth force, and the bias force change as a function of a one-unit difference in the closeness. The coefficients b0G, bTG, and bBG reflect after adjusting for the relationship closeness whether the directional bias, the truth force, and the bias force are stronger for male dyads or for female dyads.
Estimating TBM for distress change
To evaluate truth and bias forces for judgments of change in distress, we compared the April reports of both roommates with their February reports. Reliability of change scores is typically lower than for reports of levels (Gollwitzer, Christ, & Lemmer, 2014), but the SEM approach adjusts for unreliability. For the change in distress, we created five parcels with the difference scores between the item responses of two time points, again balancing the anxiety and depression facets of the K10 items. To study the moderation effect of closeness, we used the April closeness. With these adjustments, Equations 1 and 2 can be used to estimate truth and bias forces for distress change.
Evaluating roommate report as a mental health screen: PPV and NPV of the roommates’ reports
Previous research (Andrews & Slade, 2001; Kessler et al., 2002) suggests that people who score 2.5 and over in K10 scale are likely to have a moderate or severe mental disorder. Using this screening criteria, we categorized students into two groups based on their self-reported distress (T+: likely to have disorder vs. T–: unlikely to have disorder). Likewise, roommates’ judgments were split to two groups (S+: perceive their partner to have disorder vs. S–: not to have disorder). For the distress change, we used top 30% as the cutoff to categorize students (T+: likely to have increasing disorder vs. T– unlikely to have increasing disorder) and roommates’ judgments (S+: perceive their partner to have increasing disorder vs. S–: not to have increasing disorder) into two groups, separately.
The PPVs and NPVs are often used to describe the performance of a diagnostic test. The former is the probability that a person with a positive test truly has the condition (T+/S+), whereas the latter is the probability that a person with a negative diagnostic test truly does not have the condition (T–/S–; Pepe, 2003; Zhou et al., 2009). We calculated PPV and NPV for the perception of distress level and change.
Results
Descriptive Statistics of K10 and Relationship Closeness at Two Time Points
Table 1 presents descriptive statistics and correlations between observed self-rated distress scores and roommates’ judgments at two time points. 3 Because roommates were same-sex, dyad members were treated as indistinguishable (Kenny, Kashy, & Cook, 2006). Individuals within dyads were both perceivers and targets. The mean level of self- and other-reports of distress were all relatively low, with means ranging from 1.64 to 1.87 on a 1 to 5 scale. In February, the mean of self-reported distress was 1.84 (SD = .66). As the final exam was getting closer, partners reported feeling slightly more distress in April than February (MApril = 1.87, SD = .70), but the difference between February and April was not statistically significant (95% confidence bound on the difference = [–.07, .02]). In February, the mean of perceivers’ judgments was 1.64 (SD = .65). In April, perceivers rated that their roommate was feeling more distress (M = 1.70, SD = .69). This small difference was marginally significant (p = .06). Perceivers’ perceptions were smaller than targets’ self-reported distress at both time points. 4
All of the February and April reports were strongly correlated, indicating relative stability of distress and judged-distress. There was no evidence of dyadic correlation of distress or judged-distress. Partners’ self-rated distress scores were not correlated within dyads in either February (r = .04) or April (r = .04). There was no correlation between perceivers’ perceptions of their partner’s distress either (February: r = .03, April: r = .01). The correlations between perceivers’ judgments and targets’ self-reports for the distress level were around .30 (February: r = .33, April: r = .27).
Table 1 also displays descriptive statistics of relationship closeness. The mean of closeness was 3.59 (SD = .89) in February, and 3.42 (SD = .99) in April; the difference of these means was statistically significant but small in effect size, t(186) = 4.03, p < .001, d = .17. These two reports were highly correlated (r = .83), indicating relative stability of closeness.
Two-Item Parcels and Measurement Models
Using the explicit distributive strategy, two-item parcels were created such that each parcel involved one item indicator from each of “anxiety” and “depression” facets of K10 (see supplementary material Tables S1 and S2). The same combinations were used in self-reported distress and judgments of distress. 5 Because our model treated dyadic partners as interchangeable, we adjusted the model indices with the procedure described in Olsen and Kenny (2006). For the perception of distress level, the measurement model generally fit the data well in February, χ2(23) = 40.53, p < .05; root mean square error of approximation (RMSEA) = .06; comparative fit index (CFI) = .98; Tucker–Lewis index (TLI) = 1.00, and April, χ2(23) = 44.74, p < .05; RMSEA = .07; CFI = .99; TLI = 1.00. For the perception of distress change, the measurement model was also reasonably consistent with the data, χ2(23) = 55.30, p < .05; RMSEA = .09; CFI = .97; TLI = .99.
Perception of Distress Level
Truth force and bias force
Table 2 shows estimates of the truth force (bT) and bias force (bB) in the perception of distress level. These two forces had similar magnitudes at two time points. Adjusting one for the other, the target and the perceiver’s self-reported mood state both significantly affected the perceiver’s judgment. Specifically, for a unit change in the distress level rated by the target himself, the perceiver judged that the target felt an increase in .36 units of distress in February (bT = .36, SE = .07, p < .001) and .29 units of distress in April (bT = .29, SE = .07, p < .001). Adjusting for truth force, as the perceiver himself experienced one more unit distress, his perception of the target’s distress increased .24 units in both February (bB = .24, SE = .07, p < .001) and April (bB = .24, SE = .07, p < .001). Although truth force was generally larger than bias force, they were not reliably different from each other in either February or April. 6
Truth and Bias Forces for February, April, and the Change From February to April With Estimates From Structural Equation Model Adjustments for Measurement Error.
Note. N = 187 dyads. bT = truth force; bB = bias force; b0 = directional bias.
p < .05. **p < .01.
Directional bias
Directional bias was estimated by estimating the mean of the judgments for a target whose true distress level is zero when all moderators are at the average level. This adjusted mean is the intercept b0 in Equations 1 and 2. If there is no bias, this value would be close to zero. In February, the directional bias b0 was –.23 (SE = .04) and in April it was –.18 (SE = .05). Both were statistically different from zero (p < .001), suggesting that perceivers had a bias to see the target as having less distress than reported by the target.
Perception of Change in Distress
Truth force and bias force
Table 2 also shows the truth and bias estimates for overall change. There was a significant truth force in the perception of change in distress, although its magnitude was smaller than those in the perception of distress level. After adjusting for the perceiver’s own change in distress, the perceiver reported .26 units of change for each unit of change reported by the target (bT = .26, SE = .09, p < .001). In contrast to the perception of distress level, there was no evidence of bias in the reports of change from February to April (bB = .09, SE = .09, p = .32). Although one effect was significant and the other was not, the truth force and the bias force were not reliably different from each other. When they were estimated to have the same value (bT = bB = .18, SE = .07), the model did not fit the data worse (p = .15), thus the test of possible differences in these forces is inconclusive.
Directional bias
The perceiver thought the target was feeling more distress in April as opposed to February. This perceived change was .05 units higher than what the target herself rated over time, but this overreport tendency was not significantly different from zero (p = .25).
Moderating Effects of Closeness
The second block of rows in Table 2 shows the interactions of closeness with the truth and bias forces and the directional bias for both time points and the change from February to April. There was no evidence of moderation of the truth force by relationship closeness for either time point, nor for change. The bias force, however, was moderated by closeness in April, such that the bias force was stronger for roommate dyads who reported being closer (bBC = .16, SE = .05, p = .001). This unexpected pattern was not evident in February (bBC = .03, SE = .05, p = .53). There was also no evidence that closeness affected directional bias at either time or for change.
Moderating Effects of Gender
The bottom block of rows of Table 2 shows results of the analysis of gender as a moderating variable. There was no statistically significant evidence of gender moderation for truth, bias, or directional bias at either time, nor for change. However, the confidence bounds for these estimates included values nearly as large as the overall estimates of all three parameters, and this suggests that these tests were inconclusive. The relatively small numbers of male roommate pairs had an impact on the power to test gender effects.
PPV and NPV
Overall, majority of students did not report K10 distress, but nearly 20% students scored 2.5 or over (February: 16.9%, April: 17.3%) and these were likely to have a moderate or severe distress, if not disorder. The rate of high distress was double among those whose roommates judged them to be high in distress. These PPV estimates were 41.5% in February and 42.3% April. Considering the prevalence rate was generally low, it was not surprising NPVs were much higher: 86.1% in February and 86.3% in April.
Discussion
Over two time points during the spring semester of college, we found that students tended to underestimate the amount of distress being experienced by their roommates. These results are consistent with those obtained for descriptions of personality trait neuroticism measured with the NEO Personality Inventory (Allik et al., 2010). However, there was a robust truth force in the student’s judgments, which means that the students tended to report higher levels of roommate distress when the roommate reported higher levels of distress. This finding confirmed our primary hypothesis. Perceivers also tended to project more distress on the roommate when the perceiver herself was distressed, but this bias force did not dominate the truth force. Because the levels of distress in the roommates in our sample were not correlated, the bias force did not contribute indirectly to the overall accuracy of the judgment.
The accuracy of the reports in February and April, as hypothesized, led to a modest amount of accuracy in the inferred change in distress of the roommate over these 2 months. We found statistically significant evidence for a truth force in the estimation of distress change. Interestingly, we found no evidence of a bias force for change. This implies that the bias effects were fairly stable over time and essentially canceled out when change scores were computed. This is a novel finding that has implications for monitoring change in distress and well-being using systematic cross-sectional reports of peers and relationship partners. We acknowledge, however, that this finding might be limited to situations when assessments focus on rather long periods of a month or 6 weeks, as used in our study. Asking participants to focus on a longer period might induce more trait-like thinking in their assessments than a focus on states. Examining alternate timing of repeated assessments and the focal time frame is clearly a topic for future research.
Based on Funder’s (1995, 2012) RAM of person perception, we considered four possible moderators of affective perception: (a) the target, (b) the perceiver, (c) the specific trait that is rated, and (d) the information on which the judgment is based. Although affective states are different from personality traits in many aspects like stability, we thought these principles might still be applicable.
Gender is an important characteristic of both target and perceiver. Based on the literature (e.g., Brody & Hall, 1993; Gross & John, 2003; Hall et al., 2000; Rosip & Hall, 2004), we expected that there would be a stronger truth force in female dyads than male dyads. However, we did not find evidence of gender differences in perception of distress level or change. These results are not definitive because current sample has relatively small numbers of male roommate pairs, which could influence the power to detect gender effects.
We expected distress to be a more internal and less observable affective state compared with general personality traits, and for this reason we speculated that people may not perceive moods as accurately as personality traits. Using TBM (West & Kenny, 2011), we found statistically significant truth forces in both time points (February: .36, April: .29). The modest truth forces are also consistent with previous research on judgment of affect in which the correspondence between how the targets saw themselves and how the perceivers saw them ranges from .19 to .41 (Gros et al., 2013; Okazaki, 2002; Watson & Clark, 1991) although this earlier work did not adjust for perceiver’s own affective states. Moreover, the truth force we found was smaller in magnitude than personality traits that are high in visibility such as extraversion (r = .55), but was comparable to personality traits that are low in visibility such as emotional stability (r = .41; Connelly & Ones, 2010).
We also considered the quality of information that would be available to the perceiver. Students who report having closer relationships with their roommates were expected to spend more time together and to have more shared information about emotional concerns. We predicted that this extra information would translate into stronger truth forces in both February and April. However, this prediction was not supported. Relationship closeness did not moderate the truth force for either time point, or for change. There was an unexpected moderation of the bias force by closeness only in April, but not February. However, when we averaged both waves, the moderation effect was not evident, and thus we do not have much confidence in this unexpected significant finding in the April wave.
Although close friends might have access to more information about each other’s emotional state, there might be other forces, such as positivity bias, that offset the use of this information when one roommate is reporting on the distress of the other. Insofar as distress is associated with perceived negative traits of neuroticism and disability, close friends might adjust their reports of distress to be less negative.
The fact that the two assessments were obtained in the spring semester, after roommates had been together for at least 5 months, allowed for the possibility that relationships could have developed to be close. We do not necessarily believe that our findings would generalize to the first few months of the fall semester, when students are still adjusting to college and their roommates. Now that we have documented the existence of truth forces in roommates with several months of relationship experience, it will be important to determine when such effects can be observed in new roommate relationships.
Implications for Counseling Psychology
Our research has important implications for monitoring distress by counseling services at colleges. We found that college roommates were able to detect some differences in negative affect, but that they were also inclined to underestimate distress in general. The implications for underestimation are more serious when a student believes his roommate is experiencing more distress than is usual. This is particularly serious if the distress is a level that suggests a risk for suicide attempt. King and colleagues (King, Vidourek, & Strader, 2008) found that less than half of students they surveyed agree or strongly agree with statements that they can recognize the risk of suicide in a friend. Especially in groups that are high risk for suicide and alcohol abuse, such as fraternities and sororities, it may be useful to provide college peers training regarding the identification of serious problems and regarding the steps that should be taken to refer a distressed peer for counseling or treatment (Olson, Koscak, Foroudi, Mitalas, & Noble, 2016). More universal training on how to identify and respond to the distress of peers might have the benefit of encouraging conversations among roommates about what actions each might take if he or she notices another experiencing extreme distress.
Our results underscore the need to consider peer reports as a screening tool rather than a precise source of information. Among the students who are identified by roommates as possibly having severe distress, only about 42% actually reported severe distress. This PPV is twice the risk of severe distress (which we defined as the top 20% of the K10 distribution), but clearly the majority of students identified by the roommates as being distressed do not themselves report problems. Peer reports can be a useful method of determining if some students need further assessment and support, but care needs to be taken not to associate stigma with any screening program.
Relation of Findings to Other Kinds of Relationships
To what extent does the perceptual process of feelings and affect differ between roommate relationships and other kinds of relationships? Students in the current research had been living with their roommates for several months, so they were probably well acquainted with one another or already developed close friendship. Nevertheless, roommates are certainly not as close as intimate partners. As opposed to roommates, intimate partners would have stronger motivation to understand partner’s thoughts, feelings, and intentions and maintain relationship satisfaction. It is not surprising that Fletcher and Kerr’s (2010) meta-analysis reveals higher levels of accuracy within intimate relationships (r = .58) comparing with our findings.
Roommates and intimate partners also differ in terms of the extent to which partners’ self-reports are correlated with one another. In the current project, the levels of distress in the roommate were not correlated. By contrast, Overall and Hammond (2013) found a small but statistically significant correlation between intimate partners’ depressive symptoms (r = .24). They also reported a significant covariance across partners in directional bias: the more one partner underestimated the commitment or overestimated the negative behavior of his or her partner, the less the partner exhibited the same directional bias. These results could suggest that our results would not generalize to roommate dyads among older students, who were less represented in our sample than first-year students. Insofar as college junior and seniors are able to choose their roommates based on friendship ties, the magnitude of the truth and bias forces might resemble patterns found in the literature for intimate partners.
Strengths, Limitations of Design, and Future Directions
This project had several strengths. We collected measures of distress state and associated perceptions among college roommate dyads at two times during a spring semester and distinguished truth forces from bias variation using the TBM of West and Kenny (2011). To our knowledge, it is the first study to apply TBM to the perception of changes in psychological states. Specifically, we compared students’ first and second responses and tested TBM with the change scores between two responses. Although change scores are often criticized as being confounded by measurement error (Gollwitzer et al., 2014), we adjusted for measurement error using a latent variable SEM. This method also permitted the specification of constraints for indistinguishable partners that were needed in our example.
Our project was also limited by the fact that it was based on secondary analysis of existing data. The original project was designed to study patterns of change in longitudinal data for both self-report and reports of others (Shrout et al., 2017). Students’ reports of their roommates were obtained to study response bias rather than person perception per se. A focused study on roommate reports would have contained more information about the psychological characteristics of the targets and perceivers, and would have had a larger array of measures of emotional state than the K10.
The study may also be limited by the fact that all participants resided in dormitories in large private universities in a major metropolitan area. Perhaps roommates in smaller colleges in more rural settings would have spent more time together and perhaps had a different pattern of results. We also limited our analysis to students who (a) were able to recruit a roommate into the study and (b) were members of roommate dyads who provided complete data. The first restriction led to overrepresentation of females in the sample. However, as noted in Note 4, the average level of distress in the included students was not different from that of dyads with incomplete data or students who participated in the survey alone. We also were not able to examine gender effects with much statistical power as the data come primarily from female dyads. However, we believe that our study provides a useful comparison study for future research that goes beyond the limitations just noted.
One important future direction is to determine what factors facilitate or impede college students in identifying periods of intense distress. In the current research, the perceptions were measured over 2-month periods, and this may be too long to give a precise account of roommates’ distress levels. Ideally, new data would measure perceptions on a weekly or daily basis. This would help to determine whether a roommate would perceive a crisis and distinguish it from a normal fluctuation or responses to stressors. Another need for future research is to evaluate universal or targeted training programs that are designed to inform students about steps they should take if they observe a peer in acute distress. Such training programs are commonly used with RAs (Pasco et al., 2012) and have been adapted to more general groups of peer leaders (Shook & Keup, 2012).
Research is also needed to determine the impact of asking students to be more aware of the distress levels of their peers. College students already have to cope with many stressors and adding an additional concern may be detrimental to some students, especially those who are already having difficulty managing their own lives. On the contrary, there is evidence that when it is possible to provide support to others, the providers may experience psychological benefits (Inagaki & Orehek, 2017). More systematic studies on the balance of the costs and benefits are needed.
Conclusion
College students can provide some information about the level of distress being experienced by their peers. Although the student reports are influenced by the perceiver’s own level of distress, there is a reliable truth force signal available. This information needs to take into account a pervasive tendency to underreport the amount of distress being experienced by peers. There is no strong evidence that the quality of information varies across the gender of the students or the degree of closeness reported by the student and the roommate peer. The student reports may be useful in a screening program to monitor the mental health of undergraduates who are facing potent academic and social challenges in college.
Footnotes
Acknowledgements
We thank members of the New York University and Columbia University Couples labs for their comments on an earlier version of the article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The data analyzed in this article were collected with funding by the National Institute of Alcohol Abuse and Alcoholism, Grant R01-AA017672, P.E. Shrout, PI.
Notes
Supplemental Material
Supplementary material is available online with this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
