Construct Validity of the Social Provisions Scale

Abstract

Extant theory posits well-differentiated dimensions of perceived social support as measured using the Social Provisions Scale (SPS). However, evidence is inconsistent with this multidimensionality perspective, with SPS factor correlations near unity and higher between-factor than within-factor item correlations. This article reports on research investigating the internal structure, gender invariance, and predictive validity of SPS scores. The analyses are conducted in a novel bifactor exploratory structural equation modeling (ESEM) framework, which is designed to account for presumed psychometric multidimensionality in SPS items due to (a) their fallibility as pure indicators of the constructs they are purported to measure and (b) the coexistence of general and specific factors. Based on 376 item responses, evidence was obtained for a bifactor-ESEM representation of the SPS data. In addition, support was found for the invariance of item thresholds and the latent mean invariance of six of the seven SPS factors in the retained solution. Only mean levels of Social Integration were found to differ by gender, with men scoring higher than women. Finally, evidence was obtained for the predictive validity of SPS scores with respect to loneliness and psychological well-being. Quite apart from yielding evidence validating the SPS, this research demonstrates the utility of bifactor ESEM for psychological assessment.

Keywords

Social Provisions Scale ESEM bifactor ESEM CFA construct validity

The Social Provisions Scale (SPS; Cutrona & Russell, 1987) is one of the most widely used measures of perceived social support (PSS) in the psychological literature. This instrument comprises 24 items designed to measure individual differences in the perception of the six social provisions proposed by Weiss (1974). The measure has been used with diverse samples, such as public school teachers (Russell, Altmaier, & Van Velzen, 1987), college students (Cutrona, 1982; Perera & DiGiacomo, 2015), therapists (Dunkle & Friedlander, 1996), and spouses of cancer patients (Baron, Cutrona, Hicklin, Russell, & Lubaroff, 1990). Furthermore, the measure has received psychometric support with respect to its reliability and predictive and convergent validity (Cutrona & Russell, 1987). Despite its prevalence in the scientific literature and seemingly robust psychometric properties, there remain concerns about the validity of the SPS. One concern is the appropriate internal structure of item data obtained from the measure (Mancini & Blieszner, 1992). A second, related, concern is the discriminant validity of the factors that constitute the SPS (Cutrona & Russell, 1987; Mancini & Blieszner, 1992; Russell, Cutrona, Rose, & Yurko, 1984).

The aim of the present study is to further investigate the validity of SPS item scores. First, the latent structure underlying SPS data in a sample of incoming college students is examined. The theoretically espoused six-factor multidimensional representation is tested against competing unidimensional, higher order, and bifactor structures. These measurement structures are tested using traditional confirmatory factor analysis (CFA) as well as exploratory structural equation modeling (ESEM). In combining bifactor and ESEM analyses, the study demonstrates a novel integrative approach to accounting for two distinct sources of construct-relevant item psychometric multidimensionality that may be relevant for SPS data (Morin, Arens, & Marsh, 2015). Next, latent means differences in PSS across gender are examined as well as the gender invariance of item thresholds. Finally, the validity of scores generated from the final SPS model for predicting loneliness and psychological well-being (PWB) is investigated.

Theoretical Background

The SPS is predicated on Weiss’s (1974) theory of the function of social relationships. According to this theory, there are six basic functions derived from social relationships: attachment, social integration, guidance, reliable alliance, reassurance of worth, and opportunity for nurturance. Attachment refers to the provision of emotional support typically found in relationships with intimate partners, family, and close friends. Social integration refers to a sense of belonging to a group that shares common interests. Guidance involves the provision of advice or information from trustworthy social partners. Reliable alliance refers to the assurance that social relationships can be relied on for the provision of tangible aid. Reassurance of worth is the recognition of the individual’s competence and skills by others. Finally, opportunity for nurturance involves a sense that the individual is responsible for the care of others. These six dimensions of social support proposed by Weiss are reflected in most other major theoretical models of social support (e.g., Cobb, 1979; Cohen & Wills, 1985).

A central tenet of Weiss’s (1974) theory is that deficits in each provision may have unique psychosocial implications. For example, deficiencies in a sense of attachment may lead to emotional loneliness, whereas the absence of social integration may result in social loneliness (Russell et al., 1984). Furthermore, a lack of guidance may increase an individual’s anxiety and uncertainty, whereas the absence of reassurance of worth may lead to lower self-esteem and well-being (Mancini & Blieszner, 1992). Taken with the theorized distinct functions of these relational provisions, the unique psychosocial consequences of deficits in these provisions suggest that a multidimensional representation of social support is crucial to adequately operationalizing the construct from Weiss’s perspective. Although the SPS is intended to be a multidimensional measure of PSS in line with this theory, extant factor analytic evidence is unclear about the best structural representation of SPS data (Cutrona & Russell, 1987; Mancini & Blieszner, 1992).

Latent Structure

There have been surprisingly few studies of the structural validity of SPS scores. Seemingly consistent with Weiss’s (1974) theory, Cutrona and Russell (1987) reported that a correlated six-factor CFA model provided an acceptable fit to the SPS data. However, the factor intercorrelations were substantial (M = .761, range = .549-.990). In another CFA test of the SPS, Mancini and Blieszner (1992) found support for a four-factor oblique latent structure for the SPS item data, with the Attachment, Reliable Alliance, and Guidance factors collapsed into a general “Intimacy” factor because of higher between than within construct item correlations. In this solution, factor intercorrelations were also substantial, ranging from .550 to .940 (M = .737). Taken together, these findings call into question the discriminant validity of the factors and the multidimensional perspective underlying the SPS. The sizeable SPS factor correlations have also been shown to result in multicollinearity in regression models (Russell et al., 1984), which may lead to erroneous inferences about the predictive validity of the SPS factors (Marsh, Dowson, Pietsch, & Walker, 2004). High between-construct item correlations and strong factor intercorrelations suggest the possibility of a general factor underlying responses to the SPS items, primarily indexed by Attachment, Reliable Alliance, and Guidance items.

Three alternative models that may be suitable structural representations of SPS item responses are the unidimensional, higher order, and bifactor models, which all posit the presence of some common latent trait. Although Weiss’s (1974) theory posits a multidimensional representation of social support, the current scoring key for the SPS, in which researchers are instructed that a total score may be computed by summing scores for all items, implies a unidimensional structure (Cutrona & Russell, 1987). This unidimensional model assumes that there is only one common source of variation (i.e., latent PSS)in the SPS items. Another alternative model is a higher order measurement structure with a global PSS factor at the apex of the hierarchy and the specific social provisions at the first-order level. The rationale for this specification is that the support functions measured by the SPS share sufficient common variation to assume some underlying common cause, namely global PSS. This higher order structure has been shown to provide an acceptable fit to SPS data in a previous CFA (Cutrona & Russell, 1987).

A final alternative structure for the SPS data is a bifactor model in which the covariance among SPS item data is attributable to two chief sources: (a) a general PSS factor that accounts for the common variation shared by all items and (b) specific social provisions factors that account for additional common variance among item clusters over and above the general factor. This bifactor specification is in line with Cutrona and Russell’s (1987) observation that the SPS measures both specific components of PSS as well as an overall sense of support. The bifactor structure also reflects the view that people possess a general sense of support quite apart from their expectations for specific provisions from finite relationships (Davis, Morris, & Kraus, 1998). It may be that an individual believes that others are generally supportive even though a specific provision (e.g., guidance from an instructor) is not perceived as available (Pierce, Sarason, & Sarason, 1991). To the extent that both the specific and general support factors are of substantive interest, the bifactor model is the only straightforward analytic approach to test general and specific factor effects on relevant outcomes (Chen, West, & Sousa, 2006).

Psychometric Multidimensionality due to Item Fallibility

An important consideration in examining the structural validity of data from multidimensional inventories is the suitability of the independent clusters model of CFA (ICM-CFA) typically used to analyze these data. For multidimensional scale data, the ICM-CFA structure may be too restrictive because items often tap more than one dimension (Perera, 2015). Although the bifactor CFA model can account for item multidimensionality due to the coexistence of general and specific factors underlying an item (Morin et al., 2015), psychometric multidimensionality may also be a function of the fallibility of items as pure indicators of the constructs they are purported to measure (Marsh, Morin, Parker, & Kaur, 2014). Thus, these items are likely to have at least small-to-moderate and substantively important cross-loadings. The constraint of true cross-loadings to zero in the ICM-CFA may result in not only model-data misfit as error is propagated by model misspecification but also inflated factor correlations (Perera, 2015). Such misspecification may lead to erroneous inferences about the discriminant validity of factors, the tenability of higher order representations, and structural regression relationships to the extent that multicollinearity is present (Marsh et al., 2014). As the SPS is a multidimensional inventory of 24 dimensionally complex items (Mancini & Blieszner, 1992), and has been shown to produce highly correlated factors (Cutrona & Russell, 1987), the ICM-CFA structure may not be an entirely appropriate analytic model for examining the latent structure underlying its data.

ESEM provides a more flexible analytic framework within which to examine the internal structure of data derived from multidimensional scales. ESEM differs from traditional ICM-CFA inasmuch as (a) all target and nontarget loadings are freely estimated (conditional on the imposition of minimal identifying restrictions; see Morin, Marsh, & Nagengast, 2013) and (b) factors can be rotated (Asparouhov & Muthén, 2009). Accordingly, ESEM provides a much less restrictive framework for the examination of factorial structures that can adequately account for the factorial complexity of items due to their intrinsic fallibility. Previous applications of ESEM to multidimensional data have demonstrated advantages of this analytic approach over the ICM-CFA in terms of both model fit and parameter estimation (Perera, McIlveen, Burton, & Corser, 2015; see Marsh et al., 2014, for a review). Given the known factorial complexity of the SPS items (Mancini & Blieszner, 1992), ESEM may be an especially useful analytic tool for investigating the internal structure of the data.

Gender Differences in Social Support

Although results concerning gender differences in PSS are not definitive (Matud, Ibáñez, Bethencourt, Marrero, & Carballeira, 2003), some gender effects on PSS, as measured by the SPS, have been observed. For instance, Cutrona and Russell (1987) reported that women perceive greater levels of specific support provisions as well as general perceptions of the availability of support. These gender differences have been attributed to the different socialization experiences of women and men, including the divergent communal and agentic social roles of women and men, respectively (Flaherty & Richman, 1989; Matud et al., 2003; Olson & Shultz, 1994). In the present study, latent mean differences across gender in PSS are examined on the basis of the retained measurement structure in a multiple-indicator-multiple-causes (MIMIC) framework. In addition, this assessment of mean differences is extended to an examination of differential item functioning (DIF) for the SPS. The absence of DIF is a necessary condition for valid interpretations of mean differences and has seldom been investigated in the context of evaluating gender differences in PSS as measured by the SPS.

Predictive Validity

Loneliness

There is evidence that higher PSS is associated with lower loneliness in incoming college samples (Cutrona, 1982; Friedlander, Reid, Shupak, & Cribbie, 2007). Theoretically, the experience of loneliness may be expected to arise when desired or expected levels of social provisions, based on existing social needs, are perceived as unavailable or deficient (Marangoni & Ickes, 1989). The criterion of loneliness may be especially applicable to incoming students as they adjust to the college transition.During the adjustment period, individuals are exposed to numerous normative academic and socioemotional stressors, such as academic overload and making new friends (Credé & Niehorster, 2012), which may increase needs for social provisions. To the extent that basic interpersonal needs underlie desired forms of social relationships, students’ expected levels of social provisions may be particularly high as they adjust to the transition. Following from this assertion, the experience of feelings of loneliness may be precipitated when social provisions are perceived as unavailable to meet social needs.

Of the six specific provisions, attachment and social integration have been shown to be the most consistent negative predictors of loneliness (Larose, Guay, & Boivin, 2002; Russell et al., 1984; Vaux, 1988). This is in line with Weiss’s (1974) relationship functions theory, which holds that feelings of emotional and social loneliness are a response to the absence of attachment and social integration provisions, respectively. In addition to these provisions, it is plausible that reassurance of worth may negatively associate with loneliness in the present sample as in other college samples (Vaux, 1988). For incoming undergraduates, it may be that their perceptions that others acknowledge their skills and abilities guard against the experience of loneliness by satisfying the basic interpersonal need to have one’s worth recognized, which may be especially important when transitioning to a new academic environment. However, extant findings concerning loneliness may be obfuscated by the presence of multicollinearity in regression models (DiTommaso & Spinner, 1997).

PWB

There is evidence that PSS fosters PWB (Cohen & Wills, 1985; Davis et al., 1998), including in college samples (Friedlander et al., 2007; Perera & DiGiacomo, 2015). The finding of a positive relationship has been attributed to the stress-buffering role of social support. This account proposes that stress will have potentially pathogenic effects on individuals’ PWB to the extent that they possess little social support or perceptions thereof, whereas stressor effects will be attenuated or diminished for those with strong social support systems (Cohen & Wills, 1985). Of the six SPS dimensions, reassurance of worth has been the strongest and most consistent correlate of PWB (Constable & Russell, 1986; Russell et al., 1987), which is not surprising as it involves the perception that one is valued by others. This provision may be particularly important for the present sample of incoming undergraduates. It may be that providing individuals with a sense that their competence is valued by others in a novel and competitive academic environment guards against the potentially negative effects of academic stressors on PWB (Mallinckrodt & Bennett, 1992). Results are less clear for the remaining specific provisions. For example, although Quimby and O’Brien (2006) found that attachment was significantly and negatively associated with psychological distress in a sample of nontraditional female undergraduates, DiTommaso and Spinner (1997) observed no significant effect of this provision in a sample of psychology undergraduates. Results are also complicated by the presence of multicollinearity in regression models due to the high correlations among the specific provisions (Cutrona & Russell, 1987).

The Present Investigation

The aims of the present study are threefold. First, the expected correlated factors representation of the SPS data is tested against plausible unidimensional, higher order, and bifactor representations using both CFA and ESEM approaches. Next, DIF and latent mean differences in the SPS factor(s) across gender are examined. Finally, scores from the retained measurement structure are tested for predictive validity with respect to loneliness and PWB

Method

Participants and Procedure

Participants were 376 freshmen enrolled in a metropolitan university in Eastern Australia. The mean age of participants was 17.847 (SD = 0.879), and 63.60% (n = 239) of the sample was female. Students were recruited as part of a larger study on the adaptation experiences of undergraduates to college. Participation required the completion of online batteries of questionnaires at three points across the first semester, of which two time points are relevant to the present study. Social support data were collected during the 5th week of an 18-week semester; loneliness, and PWB data were collected 4 weeks thereafter at midsemester.

Measures

PSS

PSS was measured using the SPS (Cutrona & Russell, 1987), which is a 24-item self-report inventory, rated on a 4-point Likert-type scale, ranging from 1 (strongly disagree) to 4 (strongly agree). The scale measures the extent to which respondents perceive their social relationships as providing social support. The SPS is designed to yield both a global PSS score and subscale scores on the six social provisions proposed by Weiss (1974). In the present sample, the internal consistency for the full scale (α = .927) was good. In addition, internal consistencies for the Attachment (α = .791), Social Integration (α = .795), Reassurance of Worth (α = .641), Reliable Alliance (α = .775), Guidance (α = .831), and Opportunity for Nurturance (α = .710) subscales were acceptable given the limited subscale lengths.

Loneliness

Loneliness was measured using the UCLA (University of California, Los Angeles) Loneliness Scale-8 (ULS-8; Hays & DiMatteo, 1987). The ULS-8 is an 8-item self-report inventory, rated on a 4-point Likert-type scale ranging from 1 (never) to 4 (always). This short-form adaptation of Russell, Peplau, and Cutrona’s (1980) revised UCLA Loneliness Scale is designed to measure subjective feelings of loneliness. In the present sample, the internal consistency for the scale score was acceptable (α = .882).

PWB

PWB was measured using the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS; Stewart-Brown et al., 2009). The SWEMWBS is a 7-item self-report inventory designed to appraise affective–emotional, cognitive–evaluative, and optimal functioning aspects of well-being. Respondents rate the extent of their psychological functioning over the previous fortnight using a 5-point Likert-type scale ranging from 1 (none of the time) to 5 (all of the time). In the present sample, the internal consistency for the scale score was acceptable (α = .848).

Statistical Analysis

Analyses were conducted in three phases. In the first phase, CFA and ESEM analyses of the SPS data were conducted to test the absolute and relative fit of the correlated traits, unidimensional, higher order, and bifactor SPS models. For the correlated traits CFA model specification, each item was specified to load onto only the factor it was designed to measure as per the a priori scoring key, with correlations among the six factors freely estimated. In the higher order CFA, the six factors were specified to index a higher order PSS factor, with no disturbance covariances specified. For the bifactor CFA, all SPS items were specified to load onto a general PSS factor as well as one of the six specific provision factors representing conceptually cohesive residual specificity in items after partialling out the general factor. Null relations among the general and specific factors were specified. For the correlated traits ESEM, higher order ESEM, and bifactorESEM models, the same pattern of target item factor loadings and factor relations was specified as per their CFA analogues. However, ESEM solutions were rotated using the target rotation procedure, with all cross-loadings targeted to be approximately zero but not constrained to zero (Asparouhov & Muthén, 2009). For the higher order ESEM, as current implementations of ESEM in software programs do not allow for the specification of higher order models, the ESEM-within-CFA approach was used to test the higher order ESEM model (Morin et al., 2013; Perera, 2015).

The second phase of the analysis involved tests of latent mean differences in PSS across gender based on the retained SPS structure in a MIMIC framework. This approach to evaluating between-group differences in latent variable means should be preferred to structured means modeling using multiple-group structural equation modeling (SEM), where correlation matrices for finite groups may not be sufficiently robust for multiple-group modeling because of small sample sizes as in the present case (i.e., n_male = 136). The MIMIC modeling involved the estimation and comparison of two models (Marsh et al., 2014): (a) a saturated MIMIC model with paths from gender, coded as a dichotomous exogenous covariate (0 = male; 1 = female), to all indicators but not latent factors and (b) a threshold-invariant model with paths freely estimated from gender to the latent factors but not the factor indicators. If the threshold-invariant model does not result in an appreciable decrement in fit relative to the saturated model, support is found for the invariance of indicator thresholds and, accordingly, group differences can be interpreted as entirely attributable to differences on the latent variables and not DIF. The final phase of the analysis involved the examination of the predictive validity of SPS scores with respect to loneliness and PWB based on the retained SPS model in a general SEM framework.

Statistical analyses were conducted using Mplus 7.3 (Muthén & Muthén, 1998-2014). All CFA and ESEM solutions were estimated using diagonal weighted least squares (DWLS) with a mean-and-variance adjusted test statistic, operationalized as the WLSMV estimator in Mplus, under theta parameterization. A holistic approach to model fit assessment was used, comprising an evaluation of fit indices, parameters estimates, and alternative models. As the χ² can be oversensitive to even minor model misspecifications given moderately large samples and contains a restrictive hypothesis test (i.e., exact fit), three approximate fit indices were considered: comparative fit index (CFI) and Tucker–Lewis index (TLI), >.900 and .950 for acceptable and excellent fit, respectively; and root mean square error of approximation (RMSEA), <.050 and .080 for close and reasonable fit, respectively (Marsh, Hau, & Wen, 2004). For nested model comparisons, because the adjusted χ² difference (MD χ²) test appropriate for the WLSMV estimator also tends to be sensitive to even trivial differences in moderately large samples, changes in the CFI (ΔCFI) and RMSEA (ΔRMSEA) were primarily used. A decrease in the CFI and increase in RMSEA of less than .01 and .015, respectively, are suggestive of support for a more restrictive model (Chen, 2007; Cheung & Rensvold, 2002).

Results

Diagnostics

Inspection of bivariate tables revealed cells with zero frequencies, primarily concerning the lowest category of response for the SPS. As empty cells can lead to model convergence problems under WLSMV estimation because of difficulties computing polychoric correlations, the two lowest categories were collapsed across SPS items, yielding a 3-point scale with two thresholds. Across the SPS data, there was very little missingness (<1.00%). This pattern of missingness was found to be nonsystematic, χ² (205) = 174.157, p = .942 (Little, 1988). For the UCLA-8 and SWEMWBS data, there was a moderate amount of missingness due primarily to participant attrition (13.8%-14.4%). This pattern of missingness, including the SPS data, was also found to be nonsystematic, χ² (720) = 680.371, p = .852 (Little, 1988). Accordingly, pairwise present methods that draw on the full information that is available—the default under WLSMV in Mplus when no covariates are included—were used to manage missing data (Asparouhov & Muthén, 2010). Sample estimates of thresholds and polychoric correlations for the 39 observed variables can be found in Supplemental Appendix A (see supplementary materials for all appendices; available online at http://asm.sagepub.com/content/by/supplemental-data). Estimates of polychoric correlations for the 24 SPS items as a function of gender can be found in Supplemental Appendix B.

Internal Structure

Results of the fit of the measurement models are shown in Table 1. Indices describing the data model fit of the unidimensional solution that is common to both the ICM-CFA and ESEM specifications were somewhatinconsistent. Whereas the CFI and TLI were suggestive of acceptable model fit, the upper limit of the confidence bound of the RMSEA exceeded the .080 cutoff for reasonable fit, suggesting that a unidimensional model may not adequately represent the latent structure underlying SPS data. The correlated-traits CFA, higher order CFA, and bifactor CFA models provided much more acceptable fits to the data. However, in all cases, the corresponding ESEM solutions fitted the data appreciably better (e.g., ΔCFI > .01). Although the superior fit of the ESEM solutions provides tentative support for the ESEM structures over the CFA models, the improvement in fit is not as substantial as observed in other ESEM studies. Thus, it is instructive to evaluate parameter estimates to determine whether the ESEM solutions should be preferred on any other basis.

Table 1.

Model Fit Statistics for the ICM-CFA and ESEM Measurement Structures.

Model	χ²	df	CFI	TLI	RMSEA	RMSEA 90% CI
Independence model	10456.037	276
Unidimensional	847.902	252	.941	.936	.079	[.073, .085]
ICM-CFA
Correlated traits	534.683	237	.971	.966	.058	[.051, .064]
Higher order	566.644	246	.969	.966	.058	[.052, .064]
Bifactor	467.081	228	.977	.972	.053	[.046, .060]
ESEM
Correlated traits	238.912	147	.991	.983	.041	[.031, .050]
Higher order^a	236.867	156	.992	.986	.037	[.027, .046]
Bifactor	192.576	129	.994	.987	.036	[.025, .046]

Note. ICM-CFA = independent clusters model of confirmatory factor analysis; ESEM = exploratory structural equation modeling; df = degrees of freedom; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root-mean-square error of approximation; 90% CI = 90% confidence interval for the RMSEA. N = 376.^aThe higher order ESEM specification was conducted in an ESEM-within-CFA framework (see Morin et al., 2013).

There were notable differences in parameter estimates between the CFA and ESEM solutions for the correlated-traits models. Correlated-traits CFA factor loadings (M = .751) were stronger than corresponding ESEM target loadings (M = .510). Furthermore, ICM-CFA factor correlations (M = .786) were appreciably stronger than ESEM estimates (M = .420). For instance, the near-perfect correlation between Reliable Alliance and Guidance (r = .967) in the CFA solution was substantially lower in the ESEM solution (r = .217). Notably, 10 of the 15 estimated factor correlations in the CFA solution exceeded .800 with three over .900. These substantial correlations call into question the discriminant validity of the factors and the multidimensional perspective underlying the SPS. Given the superior fit of the ESEM models to the sample data, and the systematically lower factor correlations obtained in these less restrictive solutions, the ESEM structures were retained as the preferred models for further analysis. Indeed, Marsh et al. (2009) recommend that ESEM models should be retained when factor correlations are appreciably reduced relative to ICM-CFA correlations.

The correlated traits, higher order, and bifactor ESEM models were compared to determine the best-fitting and most theoretically informative solution. The three models showed uniformly excellent fits to the data and could not be distinguished based on model fit alone (see Table1).¹ Although the correlated traits ESEM solution yielded factor correlations that are consistent with the multidimensionality perspective underlying the SPS, across the Attachment, Reliable Alliance, and Guidance factors, there were six instances of target loadings less than the magnitude of concomitant cross-loadings on at least one of the other two factors (see Supplemental Appendix C). This pattern of loadings is suggestive of the presence of a general support factor, primarily loaded by Attachment, Reliable Alliance, and Guidance items.

Of the three well-fitting models, the higher order ESEM structure should perhaps be preferred on the basis of parsimony. However, relying on the parsimony criterion is not always the best solution. Indeed, from a substantive standpoint, the higher order ESEM solution is unappealing because the standardized second-order loading of Reliable Alliance on the global factor was near-zero (γ = .098) due to the complex item–factor relations involving Reliable Alliance in the loading matrix. The bifactor ESEM may better accommodate these complex relations.

The bifactor ESEM provided a marginally better fit to the data than the higher order ESEM and correlated-traits ESEM models. The parameter estimates from this solution are shown in Table 2. The General Support factor was well-defined in this solution with largely moderate to strong item loadings (|λ| = .154-.862, M = .668). In general, Attachment, Reliable Alliance, and Guidance item loadings on the general factor were stronger (|λ| = .652-.862, M = .778) than loadings for Reassurance of Worth, Social Integration, and Opportunity for Nurturance items (|λ| = .154-.787, M = .559). Beyond the general factor, target loadings on the specific factors (|λ| = .011-.681, M = .322) were systematically larger than nontarget loadings (|λ| = .001-.325, M = .076). However, target loadings on the Attachment, Reliable Alliance, and Guidance specific factors were generally small (|λ| = .011-.562, M = .205), indicating the presence of only little residual specificity after accounting for general support. On the contrary, the Reassurance of Worth, Social Integration, and Opportunity for Nurturance specific factors were well-defined with generally moderate target loadings (|λ| = .147-.681, M = .438). It should be noted, however, that across the Attachment, Reliable Alliance, and Guidance specific factors there was at least one substantive (>.25) target loading on each factor, indicating that this residual content specificity in the SPS model should be controlled. Based on the (a) marginally superior fit of the bifactor ESEM, (b) well-defined general factor and specific Reassurance of Worth, Social Integration, and Opportunity for Nurturance factors, (c) less well-defined but, nevertheless, distinguishable Attachment, Reliable Alliance, and Guidance specific factors, and (d) strength of target specific factor loadings relative to nontarget loadings, the bifactor ESEM was retained and used in subsequent tests of gender invariance and predictive validity.

Table 2.

Factor Loadings From the Bifactor-ESEM Solution.

Item	G-factor	ATT	SI	ROW	RA	GUI	OFN
2	.697	.562	.007	−.040	.107	−.172	.064
11	.777	.325	.064	−.020	−.023	.325	−.004
17	.807	.064	−.038	−.139	−.283	.036	−.008
21	.765	.011	−.215	.028	−.343	−.099	.005
5	.687	.078	.510	.005	−.008	−.061	.040
8	.544	.123	.463	.119	.004	.129	.064
14	.778	−.031	.237	.090	.149	−.001	−.050
22	.787	−.216	.322	.027	.041	−.211	−.066
6	.420	.022	−.039	.407	.058	−.207	.027
9	.493	−.107	.107	.681	.043	.036	−.117
13	.708	.164	.072	.214	−.131	.263	.108
20	.635	−.060	.103	.147	−.104	−.069	.211
1	.652	.190	.107	−.038	.202	.044	−.014
10	.727	.038	.065	.083	.263	.027	−.100
18	.862	−.025	−.049	.011	.219	−.099	−.006
23	.790	−.128	.189	−.072	.096	.001	.028
3	.782	.050	.046	−.021	.294	−.137	−.052
12	.828	.053	−.098	−.007	.038	.288	−.043
16	.823	−.005	−.041	−.056	.102	.265	−.043
19	.820	−.088	−.190	−.022	.101	−.031	.002
4	.384	.199	.085	.161	.062	−.063	.568
7	.154	−.002	.001	−.247	−.037	−.040	.606
15	.619	−.010	−.043	.087	−.061	.064	.559
24	.500	−.133	−.014	−.010	−.012	.014	.543

Note. ESEM = exploratory structural equation modeling; G-factor = general support factor; ATT = Attachment; SI = Social Integration; ROW = Reassurance of Worth; RA = Reliable Alliance; GUI = Guidance; OFN = Opportunity for Nurturance. N = 376. All factor loading estimates are standardized and target loadings on specific factors are shown in bold.

Gender Differences

Mean differences in the SPS factors across gender were examined with bifactor ESEM MIMIC models. The test of the saturated MIMIC model resulted in an excellent fit to the data, χ²(129) = 194.449, p < .001, RMSEA = .037, 95% CI = [.026, .047], CFI = .994, TLI = .985. The threshold-invariant MIMIC model also provided an excellent fit to the data, χ²(146) = 215.314, p < .001, RMSEA = .036, 95% CI = [.025, .045], CFI = .993, TLI = .986, and, notably, no appreciable decrement in fit relative to the saturated model, MD χ²(17) = 25.119, p > .05, ΔRMSEA = −.001, ΔCFI = −.001, ΔTLI = +.001. These results indicate that the effects of gender are limited to the latent means and cannot be attributed to DIF. In the retained threshold-invariant model, a significant effect of gender was found for Social Integration (γ = −0.473, p < .01), such that men were higher than women on this specific provision. However, no significant gender differences were found for General Support (γ = 0.131, p > .05), Attachment (γ = −0.200, p > .05), Reliable Alliance (γ = −0.241, p > .05), Reassurance of Worth (γ = 0.210, p > .05), Guidance (γ = 0.077, p > .05), and Opportunity for Nurturance (γ = −0.016, p > .05).

Predictive Validity

A general SEM model was specified to test the predictive validity of SPS scores. The model included the general factor and six specific provision factors as per the retained bifactor ESEM as well as a separate set of ESEM loneliness and PWB factors indicated by the ULS-8 and SWEMWBS items. The ESEM specification for the loneliness and PWB data was considered appropriate as current conceptualizations of PWB (e.g., Ryff, 1989) include positive relations as a core component of optimal functioning. Similarly, SWEMWBS items tapping positive relations may be expected to load on the loneliness factor. For the indicators of loneliness and PWB, two correlated residuals were specified to reflect presumed intradimensional local dependence generated by potential method effects emerging from highly similar item phrasings (i.e., “I feel left out” [ULS-8 Item 4], “I feel isolation from others” [ULS-8 Item5], “I’ve been feeling useful” [SWEMWBS Item 2], “I’ve been feeling optimistic about the future” [SWEMWBS Item 1]). For the structural relations, direct paths from the support factors to loneliness and PWB were specified. A test of this model resulted in an excellent fit to the sample data, χ²(549) = 740.312, p < .001, RMSEA = .030, 95% CI = [.025, .036], CFI = .986, TLI = .981. As shown in Table 3, only General Support was a noteworthy and statistically significant negative predictor of loneliness, suggesting that the remaining specific factors have little validity for the prediction of loneliness above and beyond the general factor. For PWB, General Support and Reassurance of Worth were found to be moderate and statistically significant positive predictors. The remaining factors were virtually unrelated to this criterion.

Table 3.

Standardized Path Coefficients for the Regression of Loneliness and PWB on the SPS Factors Using the Bifactor-ESEM Structure.

SPS factor	Loneliness	PWB
G-factor	−.642***	.336***
ATT	−.036	−.028
SI	−.089	−.039
ROW	−.116	.340***
RA	.153	.067
GUI	.079	.048
OFN	.018	−.049

Note. PWB = psychological well-being; SPS = Social Provisions Scale; ESEM = exploratory structural equation modeling; G-factor = General Support factor; ATT = Attachment; SI = Social Integration; ROW = Reassurance of Worth; RA = Reliable Alliance; GUI = Guidance; OFN = Opportunity for Nurturance. Structural path estimates are standardized.

***

p < .001.

Discussion

The present study has been concerned with examining the construct validity of SPS data. Specifically, the study investigated the latent structure, gender mean invariance, and predictive validity of SPS responses in a novel bifactor ESEM framework. Accordingly, this study extends previous psychometric analyses of the SPS by applying a novel analytic method to SPS data that accounts for their psychometric multidimensionality due to item fallibility and the coexistence of general and specific constructs (Morin et al., 2015). The findings of the present study show that SPS data are best represented by a complex bifactor structure with a General Support factor and six specific provision factors with varying degrees of residual specificity. Furthermore, evidence was obtained for the gender invariance of SPS item thresholds and six of the seven factor means. Only mean levels of Social Integration were found to vary across gender. Finally, evidence was acquired for the predictive validity of SPS scores with respect to both loneliness and PWB. Taken together, the present research contributes to not only the advancement of the measurement and theory of PSS but also the broader methodological literature on approaches to accounting for psychometric multidimensionality in psychological item data.

Consistent with previous studies reporting on ESEM and CFA tests of multifactorial scale data (Perera, 2015; Perera et al., 2015; see Marsh et al., 2014 for a review), ESEM solutions were found to provide a better fit to the SPS data than their ICM-CFA analogues in the present investigation. This superior fit of the ESEM model is largely due to freely estimating nonzero item cross loadings, which, when constrained to zero in the ICM-CFM, represents a source of model misspecification in analyses of multifactorial data (Morin et al., 2013). Thus, ESEM may provide a better analytic approximation to psychological item data that are seldom ever truly unidimensional, and, if permitted to do so, will oftentimes load on more than one construct (Hopwood & Donnellan, 2010). Aside from better model fit alone, ESEM yields important advantages over the ICM-CFA in parameter estimation (Perera, 2015). In the correlated traits ESEM model, estimated factor correlations were substantially lower than those obtained in the corresponding ICM-CFA model and thus seemingly much more in line with the multidimensionality perspective that underlies the SPS (Cutrona & Russell, 1987). Indeed, where an ESEM solution provides an appreciably better fit to that data than the ICM-CFA and yields substantially lower factor correlations relative to the ICM-CFA, the ESEM model should be preferred (Perera, 2015).

Although the correlated traits ESEM model provided an excellent fit to the data and yielded estimates of factor correlations that are consistent with Weiss’s (1974) multidimensionality perspective, evidence was obtained for construct-relevant multidimensionality in the solution attributable to the coexistence of general and specific factors underlying the items, which a correlated traits ESEM model cannot sufficiently accommodate. Across the Attachment, Reliable Alliance, and Guidance factors there were instances of target loadings less than the magnitude of corresponding cross-loadings on at least one of the other two factors. This finding of high cross-loadings converges with the results of Mancini and Blieszner (1992) who found greater between-factor than within-factor item correlations across these factors. In Mancini and Blieszner’s (1992) study, these three factors were collapsed to form an “Intimacy” factor loaded by all 12 items on the basis of the ostensible violation of simple structure. However, in the present study, there was at least one nontrivial target loading per factor, indicating that each factor may possess sufficient content specificity to be scientifically useful. Taken together, this pattern of loadings is suggestive of the coexistence of (a) a General Support factor primarily loaded by Attachment, Reliable Alliance, and Guidance items and to a lesser extent Social Integration, Reassurance of Worth, and Opportunity for Nurturance items and (b) at least partially defined specific provisions factors that may possess some content specificity.

Construct-relevant item psychometric multidimensionality due to the presence of general and specific factors may be sufficiently accounted for by a bifactor solution. In the present study, the retained bifactor ESEM model provided an excellent fit to the sample data and resulted in theoretically defensible parameters. In this solution, the general factor was well-defined with 23 of the 24 general factor loadings exceeding .350. This is remarkable considering the SPS items were intended to measure distinct dimensions of PSS. This bifactor solution is consistent with the theoretical view that people possess a global sense of support in addition to expectations for, and perceptions of, specific provisions obtained from finite relationships (Cutrona & Russell, 1987; Davis et al., 1998; Pierce et al., 1991). For example, following a natural disaster, a person may perceive the provision of tangible assistance via receipt of government food stamps. However, this may not constitute part of the person’s broader schema of general perceptions of support. Likewise, an incoming college student may perceive their worth is reassured by a professor who providesa high assignment grade, quite apart from feeling generally supported. Notwithstanding the theoretical defensibility of the bifactor model, the specific Attachment, Reliable Alliance, and Guidance factors were only weakly defined, with generally low-to-moderate, and in some cases near-zero, specific-factor loadings. The finding of only partially defined factors raises the possibility of another alternative model in which Attachment, Reliable Alliance, and Guidance items contribute only to the general factor. This structure reflects the view that people may not sufficiently discriminate across these support functions, particularly where these support provisions are derived from a common source (e.g., parents; Mancini & Blieszner, 1992). Instead, it may be that these basic functions constitute the core of people’s general perceptions of support. Future researchers would do well to test this alternative structure.

Even after controlling for the general factor, specific factor cross-loadings remained in the bifactor ESEM. In general, these cross-loadings (|λ| = .001−.325, M = .076) were smaller in magnitude relative to those observed in the correlated traits ESEM (|λ| = .001−.498, M = .136). This is because any true general factor underlying all items is expressed via (inflated) cross-loadings in solutions that do not explicitly model the general factor (Morin et al., 2015). However, the cross-loadings were, at times, substantively meaningful. For example, Item 4 (“there are people who depend on me for help”)—an indicator of Opportunity for Nurturance—cross-loaded nontrivially on Attachment, which may reflect the reciprocity of care that is crucial to strong emotional bonds. Similarly, Item 1 (“there are people I can depend on to help me if I really need it”) and Item 13 (“I have relationships where my competence and skills are recognized”), which are designed to primarily index Reliable Alliance and Reassurance of Worth, respectively, had nontrivial cross-loadings on Attachment. This may reflect the possibility that tangible and esteem support serve an emotional support function to the degree that they signal caring behaviors (Schaefer, Coyne, & Lazarus, 1981). Several other substantively meaningful cross-loadings were observed for the specific factors. These cross-loadings serve a construct enhancement function, allowing latent variables to be estimated using all the available indicator-level information (Morin et al., 2015; Perera, 2015).

Taken together, the results of the present investigation contribute to a nascent literature on the use of bifactor ESEM as an integrative analytic framework for the theoretically informed modeling of two sources of construct-relevant psychometric multidimensionality that may be present in multifactorial instruments (Morin et al., 2015). The first source concerns the coexistence of general and specific factors underlying item data, which may be sufficiently accounted for by a bifactor model. The second source concerns the fallibility of items as purely unidimensional indicators of the constructs they are purported to measure. This psychometric multidimensionality due to item fallibility may be accounted for by ESEM. Where both sources of psychometric multidimensionality are expected to be present, a bifactor ESEM may be the most appropriate analytic model (Morin et al., 2015).

Despite these apparent benefits of bifactor ESEM, the finding of several nontrivial and substantively meaningful nontarget loadings raises the possibility that these parameters may have been specified a priori based on theoretical expectations in a more parsimonious CFA model. Indeed, Booth and Hughes (2014) note that the a priori specification of theoretically plausible cross-loadings should be preferred to the unrestrictive ESEM specification of all possible cross-loadings. According to Perera (2015), this preference is for at least two reasons: (a) tests of nontarget loadings specified a priori on the basis of a substantive rationale, which are subsequently supported by the data, yield stronger evidence for the parameter by virtue of their hypothetico-deductive logic and (b) the a priori specification of only theoretically defensible nontarget loadings minimizes the estimation of atheoretical parameters, which may reflect sampling idiosyncrasies alone, toward preserving scientific parsimony.

Although there were several meaningful cross-loading in the retained bifactor ESEM, there were a greater number of trivial nontarget loadings that do not appear to enhance construct estimation. This suggests that a CFA with minimal theoretically defensible cross-loadings may be a more acceptable analytic structure (Booth & Hughes, 2014). However, in complex multidimensional instruments with several conceptually related dimensions, such as the SPS, the a priori identification of all plausible cross-loadings reflecting multidimensionality due to item fallibility may be an unrealistic ideal. Perera (2015) notes the analytic potential of ESEM with target rotation in these circumstances. Given expected psychometric multidimensionality due to item fallibility, it is possible with target rotation in ESEM to advance hypotheses about the pattern of nonzero and approximately zero loadings in a confirmatory fashion, but permit cross-loadings targeted to zero to deviate from zero should this specification be unsuitable (Morin et al., 2015). Cross-loadings targeted to zero that show appreciable deviation from zero may then become the focus of later scrutiny for theoretical defensibility and replication (Perera, 2015), perhaps in a complex CFA framework that is informed by prior ESEM results.

Latent Mean Invariance

The present study yielded evidence for the gender invariance of (a) SPS item thresholds, (b) the general SPS factor mean, and (c) five of the six specific support factor means. Although the focus of the present invariance tests was on evaluating latent mean differences across gender, the MIMIC models were extended to investigate the invariance of indicator thresholds, which is necessary for valid interpretations of mean differences. Support was found for a bifactor ESEM MIMIC model in which paths from gender to the factor indicators were constrained to zero. This result is indicative of item threshold invariance or the absence of DIF across gender in the SPS. The retained threshold-invariant MIMIC model yielded support for the gender equivalence of factor means for General Support, Attachment, Reliable Alliance, Guidance, Reassurance of Worth, and Opportunity for Nurturance. Only mean levels of Social Integration were found to vary as a function of gender, such that men perceived higher levels of the provision than women. Notably, these gender differences in Social Integration were found, partialling out the influence of the general factor. This result converges with evidence showing that men have larger social networks than women (Fuhrer, Stansfeld, Chemali, & Shipley, 1999), and is somewhat consistent with the theoretical view espousing the agentic social role of men (Matud et al., 2003). It may be that the differences in Social Integration reflect men’s general preference for broad network support (i.e., belonging to a group that shares common interests and recreational activities; Belle, 1989), rather than more emotion-based and expressive support functions, which may satisfy the basic need for belonging but foster the maintenance of autonomy and self-reliance that are central to men’s socialization experiences (Matud et al., 2003).

Predictive Validity

Evidence obtained from the tests of the predictive validity of the SPS scores based on the retained bifactor ESEM supports and extends previous findings on the role of PSS in loneliness and PWB. For loneliness, consistent with relational provisions theories, general support was found to be a strong, negative predictor. This result replicates previous findings of the negative associations of general PSS with loneliness, and extends these findings by showing that this effect of PSS is independent of the specific provisions (Davis et al., 1998; Pierce et al., 1991). Indeed, partialling out the influence of the general factor, the specific social provisions were not significantly related to loneliness. For PWB, General Support was a significant positive predictor. This is consistent with the stress-buffering account of social support, though not a direct test of its core postulate (i.e., a moderation hypothesis), and replicates a large body of literature showing that the general perception of the availability of social support is associated with higher well-being in college students (e.g., Brissette, Scheier, & Carver, 2002; Davis et al., 1998; Perera & DiGiacomo, 2015). The findings from this research also replicate previous data indicating that Reassurance of Worth is the only appreciable predictor of PWB of the six specific provisions (Constable & Russell, 1986), and extend these data by showing that this effect is independent of the General Support factor. This finding may reflect the possibility that across the college transition—the focal context of the present investigation—where students’ sense of self-worth is particularly vulnerable to the deleterious effects of academic and social stressors (Friedlander et al., 2007), perceptions that others acknowledge their skills and abilities, quite apart from a general sense of being supported, may be integral to maintaining PWB.

Limitations and Future Directions

Notwithstanding these advances in the measurement and theory of PSS, the study reported herein has several limitations that serve to guide the appropriate interpretation of results and future research. The sample size in this study, though not small by current standards, limits the findings in two ways. First, cross-tabulations revealed several empty cells, leading to the collapsing of the lower categories of the SPS response scale to enhance the WLSMV estimation process. Thus, it is not clear whether the findings obtained would generalize to the original 4-point scale. Second, the small sample for males precluded the assessment of latent mean differences using multiple-group structured means modeling, which would provide a stronger test of latent mean invariance. This is because the multiple-group approach can simultaneously test the invariance of factor loadings and item thresholds, which are sufficient conditions for comparing factor means. On the contrary, the MIMIC model, which was employed in the present study as a more parsimonious alternative, assumes the invariance of factor loadings but cannot accommodate this test in an ESEM framework (Morin et al., 2015). Future researchers would do well to examine the replicability of the present findings in a larger sample that permits retention of the original SPS item response format and multiple-group structured means modeling.

Another limitation of this study concerns the criteria used for model fit evaluation, selection, and comparison. Although model fit was assessed in line with current recommendations, it is not known whether these guidelines are appropriate for ESEM. Likewise, it is not entirely clear whether the criteria used to compare nested models based on changes in approximate fit indices, which were derived in the context of multiple-population CFA models for continuous data (Chen, 2007; Cheung & Rensvold, 2002), generalize to ESEM models for polytomous data under WLSMV estimation. What is clear is that further simulation and empirical work is required before these criteria for evaluating fit can be used with greater confidence. Until the time, it is important to heed the recommendation of Marsh et al. (2010) to use a holistic approach to model assessment, including an evaluation of fit indices, parameter estimates, and alternative models.

A final limitation concerns the evidence obtained for the psychometric robustness of the SPS. Although support was found for the structural and predictive validity of SPS data, this evidence should be considered only an initial step in validating SPS scores. Future researchers would do well to harness the present results, particularly those concerning the supported factor structure, to further examine construct validity of SPS scores. One profitable line of inquiry may be an investigation of the convergent and divergent validity of SPS scores based on the bifactor solution, perhaps in a multitrait–multimethod framework. Another line of inquiry for establishing construct validity may be an investigation of the stability of the General Support factor, relative to the specific provisions, over time. Indeed, one possibility raised in the extant literature is that global support perceptions are a trait-like personality construct reflecting an individual’s stable view or outlook on the general supportiveness of the social environment (Davis et al., 1998; Sarason, Sarason, & Shearin, 1986). On the contrary, provision-specific perceptions of support may be more bound to current relationships and, accordingly, less stable as relationships change. Bifactor ESEM provides an elegant framework for the examination of temporal stability that can accommodate tests of longitudinal measurement invariance required to examine construct-level stability effects over time.

A third important direction for future research on the construct validity of SPS scores is to explore theoretically relevant outcomes of the specific factors. Although the specific factors generally had little validity for predicting loneliness and PWB in the present study, it is plausible that, for some outcomes, the specific factors may yield greater predictive validity and even relations in the opposite direction relative to the general factor. For instance, the social integration that one perceives as a result of involvement in, say, a sporting team, quite apart from their global sense of being supported, may be more important in the prediction of social loneliness (i.e., the absence of a social network) than generalized perceptions of support (DiTommaso & Spinner, 1997). Also, as suggested above, perceptions that one’s worth is reassured, absent of a general sense of being supported, may be as critical to aspects of positive adaptation (e.g., autonomy and personal growth) as general support, especially in contexts where self-worth may be under threat. Furthermore, it may be that opportunity for nurturance, exclusive of a global sense of support, exerts a negative effect on well-being to the extent that the caregiving is stressful. As provision-specific supports may be bound, at times, to finite relationships (Cutrona & Russell, 1987), the specific factors may have the greatest predictive validity for domain-specific or local outcomes (Davis et al., 1998). On the contrary, for high bandwidth outcomes (e.g., general wellness), the global support factor may possess superior predictive power given its conceptual breadth.

The issue of the validity of the general versus specific factors represented in the final bifactor model has important implications for approaches to scoring the SPS. Indeed, the retention of a bifactor structure necessarily complicates the scoring of the SPS, at least in terms of the computation of observed scores. This is because dominant approaches to computing observed scores, namely the total and individual score approaches (Chen, Hayes, Carver, Laurenceau, & Zhang, 2012), cannot easily accommodate the partitioning of item variance into the general and specific factor components implied by the bifactor model. Take, for instance, the computation of a total score. Some investigators may justify calculating a total SPS score, as the sum or average of all 24 SPS items, on the basis of the finding that the General Support factor was relatively well-defined in the present investigation. This total score would, however, confound the variance associated with the general and specific factors (Brown, Finney, & France, 2011). Similarly, the computation of individual facet or subscale scores, as per, for instance, the SPS scoring key, is conceptually problematic as investigators cannot disentangle the unique contributions of the subscale factors from the contributions of the common component shared by all subscales. Furthermore, where items predominantly reflect the general factor and have weak or near-zero specific-factor loadings, it may be that subscales are meaningless (Reise, Moore, & Haviland, 2010). This issue of observed score computations is further complicated by the presence of specific-factor cross-loadings due to item fallibility.

Although replication of the latent structure retained in the present study is required before definitive guidelines on scoring or modeling the SPS data are provided, a few tentative recommendations are advanced. Unless the present bifactor structure is disconfirmed in future research, investigators working with the SPS should use latent variable modeling approaches to explicitly model the bifactor structure. If interest is predominantly in examining general perceptions of support, unlike the total score approach, the bifactor model yields a “pure” support factor, partialling out specific-factor variance. This research would, by implication, yield important evidence on the validity of the specific factors and illuminate their conceptual meaning (Chen et al., 2006). For researchers who primarily work with manifest-variable methods, if proceeding with the computation of total SPS scores, it may be prudent to qualify any results by acknowledging the multidimensionality of the SPS data and discussing implications for reliability and validity occasioned by item score aggregation (Brown et al., 2011).

In summary, the present study makes important advances in validating the SPS and demonstrating the utility of bifactor ESEM for psychological assessment. The study has shown that item data obtained from the SPS are consistent with a complex bifactor measurement structure. Evidence was also obtained for the gender equivalence of item thresholds and six of the seven SPS factors means in the bifactor solution, including General Support. Only mean levels of Social Integration were found to differ as a function of gender, with men scoring higher than women. Furthermore, the study yielded evidence for the predictive validity of SPS scores with respect to substantively important outcomes, including loneliness and PWB. Quite apart from these substantive advances, on a methodological level, the study illustrates bifactor ESEM as integrative framework for the conduct of structural and predictive validity tests, which profitably accounts for two distinct sources of construct-relevant item psychometric multidimensionality likely to be found in psychological data.

Footnotes

Acknowledgements

Thanks are extended to P. McIlveen, G. J. Fogarty, T. Booth, and two anonymous reviewers for useful comments on this submission.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Supplementary Material

Online appendices are available at

References

Asparouhov

Muthén

(2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438. doi:10.1080/10705510903008204

Asparouhov

Muthén

(2010). Weighted least squares estimation with missing data. MplusTechnical Appendix. Retrieved from http://www.statmodel.com/download/GstrucMissingRevision.pdf

Baron

R. S.

Cutrona

C. E.

Hicklin

Russell

D. W.

Lubaroff

D. M.

(1990). Social support and immune function among spouses of cancer patients. Journal of Personality and Social Psychology, 59, 344-352. doi:10.1037/0022-3514.59.2.344

Belle

(1989). Studying children’s social networks and social support. In Belle

(Ed.), Children’s social networks and social support (pp. 1-14). New York, NY: Wiley.

Booth

Hughes

D. J.

(2014). Exploratory structural equation modeling of personality data. Assessment, 21, 260-271. doi:10.1177/1073191114528029

Brissette

Scheier

M. F.

Carver

C. S.

(2002). The role of optimism in social network development, coping, and psychological adjustment during a life transition. Journal of Personality and Social Psychology, 82, 102-111. doi:10.1037//0022-3514.82.1.102

Brown

A. R.

Finney

S. J.

France

M. K.

(2011). Using the bifactor model to assess the dimensionality of the Hong Psychological Reactance Scale. Educational and Psychological Measurement, 71, 170-185. doi:10.1177/0013164410387378

Chen

F. F.

(2007). Sensitivity of Goodness of Fit Indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504. doi:10.1080/10705510701301834

Chen

F. F.

Hayes

Carver

C. S.

Laurenceau

J. P.

Zhang

(2012). Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality, 80, 219-251. doi:10.1111/j.1467-6494.2011.00739.x

10.

Chen

F. F.

West

S. G.

Sousa

K. H.

(2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189-225. doi:10.1207/s15327906mbr4102_5

11.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating Goodness-of-Fit Indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. doi:10.1207/s15328007sem0902_5

12.

Cobb

(1979). Social support and health through the life course. In Riley

M. W.

(Ed.), Aging from birth to death: Interdisciplinary perspectives (pp. 93-106). Boulder, CO: Westview Press.

13.

Cohen

Wills

T. A.

(1985). Stress, social support, and the buffering hypothesis. Psychological Bulletin, 98, 310-357. doi:10.1037/0033-2909.98.2.310

14.

Constable

J. F.

Russell

D. W.

(1986). The effect of social support and the work environment upon burnout among nurses. Journal of Human Stress, 12, 20-26. doi:10.1080/0097840X.1986.9936762

15.

Credé

Niehorster

(2012). Adjustment to College as Measured by the Student Adaptation to College Questionnaire: A quantitative review of its structure and relationships with correlates and consequences. Educational Psychology Review, 24, 133-165. doi:10.1007/s10648-011-9184-5

16.

Cutrona

C. E.

(1982). Transition to college: Loneliness and the process of social adjustment. In Peplau

L. A.

Perlman

(Eds.), Loneliness: A sourcebook of current theory, research, and therapy (pp. 291-309). New York, NY: Wiley Interscience.

17.

Cutrona

C. E.

Russell

D. W.

(1987). The provisions of social relationships and adaptation to stress. In Jones

W. H.

Perlman

(Eds.), Advances in personal relationships (Vol. 1, pp. 37-67). Greenwich, CT: JAI Press.

18.

Davis

M. H.

Morris

M. M.

Kraus

L. A.

(1998). Relationship-specific and global perceptions of social support: Associations with well-being and attachment. Journal of Personality and Social Psychology, 74, 468-481. doi:10.1037/0022-3514.74.2.468

19.

DiTommaso

Spinner

(1997). Social and emotional loneliness: A re-examination of Weiss’ typology of loneliness. Personality and Individual Differences, 22, 417-427. doi:10.1016/S0191-8869(96)00204-8

20.

Dunkle

J. H.

Friedlander

M. L.

(1996). Contribution of therapist experience and personal characteristics to the working alliance. Journal of Counseling Psychology, 43, 456-460. doi:10.1037/0022-0167.43.4.456

21.

Flaherty

Richman

(1989). Gender differences in the perception and utilization of social support: Theoretical perspectives and an empirical test. Social Science & Medicine, 28, 1221-1228. doi:10.1016/0277-9536(89)90340-7

22.

Friedlander

L. J.

Reid

G. J.

Shupak

Cribbie

(2007). Social support, self-esteem, and stress as predictors of university adjustment among first-year undergraduates. Journal of College Student Development, 48, 259-274. doi:10.1353/csd.2007.0024

23.

Fuhrer

Stansfeld

S. A.

Chemali

Shipley

M. J.

(1999). Gender, social relations and mental health: Prospective findings from an occupational cohort. Social Science & Medicine, 48, 77-87. doi:10.1016/S0277-9536(98)00290-1

24.

Hays

R. D.

DiMatteo

M. R.

(1987). A short-form measure of loneliness. Journal of Personality Assessment, 51, 69-81. doi:10.1207/s15327752jpa5101_6

25.

Hopwood

C. J.

Donnellan

M. B.

(2010). How should the internal structure of personality inventories be evaluated? Personality and Social Psychology Review, 14, 332-346. doi:10.1177/1088868310361240

26.

Larose

Guay

Boivin

(2002). Attachment, social support, and loneliness in young adulthood: A test of two models. Personality and Social Psychology Bulletin, 28, 684-693. doi:10.1177/0146167202288012

27.

Little

R. J. A.

(1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198-1202. doi:10.1080/01621459.1988.10478722

28.

Mallinckrodt

Bennett

(1992). Social support and the impact of job loss in dislocated blue-collar workers. Journal of Counseling Psychology, 39, 482-489. doi:10.1037/0022-0167.39.4.482

29.

Mancini

J. A.

Blieszner

(1992). Social provisions in adulthood: Concept and measurement in close relationships. Journal of Gerontology, 47, 14-20. doi:10.1093/geronj/47.1.P14

30.

Marangoni

Ickes

(1989). Loneliness: A theoretical review with implications for measurement. Journal of Social and Personal Relationships, 6, 93-128. doi:10.1177/026540758900600107

31.

Marsh

H. W.

Dowson

Pietsch

Walker

(2004). Why multicollinearity matters: A reexamination of relations between self-efficacy, self-concept, and achievement. Journal of Educational Psychology, 96, 518-522. doi:10.1037/0022-0663.96.3.518

32.

Marsh

H. W.

Hau

K.-T.

Wen

(2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320-341. doi:10.1207/s15328007sem1103_2

33.

Marsh

H. W.

Lüdtke

Muthén

Asparouhov

Morin

A. J.

Trautwein

Nagengast

(2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22, 471-491. doi:10.1037/a0019227

34.

Marsh

H. W.

Morin

A. J.

Parker

P. D.

Kaur

(2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85-110. doi:10.1146/annurev-clinpsy-032813-153700

35.

Marsh

H. W.

Muthén

Asparouhov

Lüdtke

Robitzsch

Morin

A. J. S.

Trautwein

(2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16, 439-476. doi:10.1080/10705510903008220

36.

Matud

M. P.

Ibáñez

Bethencourt

J. M.

Marrero

Carballeira

(2003). Structural gender differences in perceived social support. Personality and Individual Differences, 35, 1919-1929. doi:10.1016/S0191-8869(03)00041-2

37.

Morin

Marsh

Nagengast

(2013). Exploratory structural equation modeling. In Hancock

G. R.

Mueller

R. O.

(Eds.), Structural equation modeling: A second course (pp. 395-436). Charlotte, NC: IAP.

38.

Morin

A. J. S.

Arens

A. K.

Marsh

H. W.

(2015). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling. Retrieved from https://www.statmodel.com/download/COMPLETE%20Morin%20Arens%20Marsh_total_Bifactor%20ESEM_Final.pdf

39.

Muthén

L. K.

Muthén

B. O.

(1998-2014). Mplus user’s guide. Los Angeles, CA: Author.

40.

Olson

D. A.

Shultz

K. S.

(1994). Gender differences in the dimensionality of social support. Journal of Applied Social Psychology, 24, 1221-1232. doi:10.1111/j.1559-1816.1994.tb00555.x

41.

Perera

H. N.

(2015). The internal structure of responses to the Trait Emotional Intelligence Questionnaire Short-Form: An exploratory structural equation modeling approach. Journal of Personality Assessment, 97, 411-423. doi:10.1080/00223891.2015.1014042

42.

Perera

H. N.

DiGiacomo

(2015). The role of trait emotional intelligence in academic performance during the university transition: An integrative model of mediation via social support, coping, and adjustment. Personality and Individual Differences, 83, 208-213. doi:10.1016/j.paid.2015.04.001

43.

Perera

H. N.

McIlveen

Burton

L. J.

Corser

D. M.

(2015). Beyond congruence measures for the evaluation of personality factor structure replicability: An exploratory structural equation modeling approach. Personality and Individual Differences, 84, 23-29. doi:10.1016/j.paid.2015.01.004

44.

Pierce

G. R.

Sarason

I. G.

Sarason

B. R.

(1991). General and relationship-based perceptions of social support: Are two constructs better than one? Journal of Personality and Social Psychology, 61, 1028-1039. doi:10.1037/0022-3514.61.6.1028

45.

Quimby

J. L.

O’Brien

K. M.

(2006). Predictors of well-being among nontraditional female students with children. Journal of Counseling & Development, 84, 451-460. doi:10.1002/j.1556-6678.2006.tb00429.x

46.

Reise

S. P.

Moore

T. M.

Haviland

M. G.

(2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559. doi:10.1080/00223891.2010.496477

47.

Russell

D. W.

Altmaier

Van Velzen

(1987). Job-related stress, social support, and burnout among classroom teachers. Journal of Applied Psychology, 72, 269-274. doi:10.1037/0021-9010.72.2.269

48.

Russell

D. W.

Cutrona

C. E.

Rose

Yurko

(1984). Social and emotional loneliness: An examination of Weiss’s typology of loneliness. Journal of Personality and Social Psychology, 46, 1313-1321. doi:10.1037/0022-3514.46.6.1313

49.

Russell

D. W.

Peplau

L. A.

Cutrona

C. E.

(1980). The revised UCLA Loneliness Scale: Concurrent and discriminant validity evidence. Journal of Personality and Social Psychology, 39, 472-480. doi:10.1037/0022-3514.39.3.472

50.

Ryff

C. D.

(1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology, 57, 1069-1081. doi:10.1037/0022-3514.57.6.1069

51.

Sarason

I. G.

Sarason

B. R.

Shearin

E. N.

(1986). Social support as an individual difference variable: Its stability, origins, and relational aspects. Journal of Personality and Social Psychology, 50, 845-855. doi:10.1037/0022-3514.50.4.845

52.

Schaefer

Coyne

J. C.

Lazarus

R. S.

(1981). The health-related functions of social support. Journal of Behavioral Medicine, 4, 381-406. doi:10.1007/BF00846149

53.

Stewart-Brown

Tennant

Platt

Parkinson

Weich

(2009). Internal construct validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): A Rasch analysis using data from the Scottish Health Education Population Survey. Health and Quality of Life Outcomes, 7, 15-22. doi:10.1186/1477-7525-7-15

54.

Vaux

(1988). Social and emotional loneliness: The role of social and personal characteristics. Personality and Social Psychology Bulletin, 14, 722-734. doi:10.1177/0146167288144007

55.

Weiss

(1974). The provisions of social relationships. In Rubin

(Ed.), Doing unto others (pp. 17-26). Englewood Cliffs, NJ: Prentice Hall.

56.

West

S. G.

Taylor

A. B.

(2012). Model fit and model selection in structural equation modeling. In Hoyle

R. H.

(Ed.), Handbook of structural equation modeling (pp. 209-231). New York, NY: Guilford Press.