Factor Structure and Psychometric Properties of a Brief Measure of Dysfunctional Individuation

Abstract

Individuation is widely considered a fundamental developmental task of adolescence. It is a process through which the adolescent seeks to define new boundaries between his or her self and others, and the failure to do so has been shown to have serious consequences. Given its importance for understanding developmental transitions, it is surprising that there are few assessments of dysfunctional individuation. Over three studies, we provide evidence of a promising new measure of this important construct: the 10-item Dysfunctional Individuation Scale (DIS). Using confirmatory factor analysis and item response theory, we demonstrate that the DIS possesses a strong one-factor structure and excellent psychometric properties. Furthermore, we document the convergent, discriminant, and concurrent validity of the DIS through its relationships with indices of individuation, adjustment, and clinically relevant symptoms. Finally, we examine the incremental validity of the DIS over neuroticism as a predictor of depression (Beck Depression Inventory–II).

Keywords

individuation assessment identity development adolescence item response theory

Separation–individuation is a process by which one establishes self-other boundaries and the internal representations that encourage autonomy. Although the process was first described to account for the “psychological birth” of the infant and toddler (Mahler, Pine, & Bergman, 2000), there is growing recognition that adolescence and young adulthood constitute a “second phase” where issues of self-governance and relatedness are renegotiated in the context of developmental tasks common to this age period (Blos, 1962; Josselson, 1980; Quintana & Kerr, 1993).

The developmental challenge for the adolescent and emerging adult is to construct a sense of self that does not rely excessively on identifications with others. The individuated self successfully manages the tension between separation and connectedness while avoiding the undesirable outcome of fusion and enmeshment, on the one hand, and complete detachment and isolation, on the other (Allen, Hauser, Eickholt, Bell, & O’Connor, 1994; Kins, Beyers, & Soenens, 2013). In other words, the desire for self-assertion, the desire to exercise autonomous agency, must be realized in a way that does not come at the expense of a simultaneous need for communion, bonding, and connection to others. And similarly, the desire for communion cannot be so total that the sense of self becomes enmeshed and enveloped by relationships. The goal of separation–individuation, then, is relational autonomy whereby the sense of self is established on independent footing but in the context of mutually validating relationships (Allen et al., 1994; Josselson, 1988).

Disturbances in separation–individuation are assumed to result in broadband personality and relational dysfunctions (Bleiberg, 2001; Holmbeck & Leake, 1999). For example, pervasive difficulty in regulating social distance is at the heart of personality disorders (Tackett, Baliss, Oltmanns, & Krueger, 2009; Westen & Heim, 2003) and there is reason to believe that the developmental source of self–other pathology can be traced to separation–individuation gone awry (Coonerty, 1986; Dolan, Evans, & Norton, 1992). Poor separation–individuation is also associated with an incapacity for intimacy (Levitz-Jones & Orlofsky, 1985), insecure and pathological dyadic attachment (Lapsley & Edgerton, 2002; Lapsley, Rice & Shadid, 1989), poor college adjustment (Mattanah, Brand, & Hancock, 2004; McClanahan & Holmbeck, 1992), and difficulties with identity development (Koepke & Denissen, 2012).

A number of promising assessment strategies have been devised to measure elements of separation–individuation, although extant measures may be of limited utility to clinicians and researchers. Most instruments consist of many subscales, making them lengthy and cumbersome to administer, or have uneven predictive validity. For example, the Separation–Individuation Test of Adolescence (SITA; Levine, Green, & Millon, 1986) is a well-regarded measure with strong evidence of construct validity (Holmbeck & Leake, 1999), but its six subscales make it difficult to use as a diagnostic screen or for purposes of longitudinal research. Similarly, the Psychological Separation Inventory (Hoffman, 1984) is a 69-item measure tapping four dimensions of independence (functional, attitudinal, conflictual, emotional) that must be administered twice to capture psychological separation from mother and father. Moreover, only the conflictual independence dimension has shown consistent evidence of predictive validity (Lapsley et al., 1989). The Emotional Autonomy Scale (Steinberg & Silverberg, 1986) measures four core components although there is no consensus about the scale’s factor structure (e.g., Beyers, Goossens, Van Calster, & Duriez, 2005) or whether the emotional autonomy measured by the scale is adaptive or maladaptive (e.g., Ryan & Lynch, 1989).

Christenson and Wilson (1985) introduced a measure of pathology of separation–individuation that was intended to guide clinical practice. The original construction of the scale was guided by Mahler’s theory of infantile separation–individuation and by Pine’s (1979) account of clinical disturbances in separation–individuation manifested by adults. Based on clinical-developmental theories, Christenson and Wilson (1985) argued that pathology of separation–individuation is manifested by difficulty in differentiating the self from others (“Often, when I am in a close relationship I find my sense of self gets lost”), in splitting of self–other representations into “good” and “bad” (“I find that either I like someone or I can’t stand them”), and in relationship disturbances marked by aloneness tolerance (“Whenever I am away from my family I feel very uneasy”), coercion (“I am tempted to control other people to keep them close to me”), and object constancy (“I find that people begin to change whenever I get to know them”).

Christenson and Wilson’s (1985) initial attempt to measure differentiation, splitting, and relationship disturbance resulted in a 39-item scale with strong internal consistency (α = .92) and a unidimensional structure. The scale successfully discriminated clinical patients diagnosed with borderline personality disorder from a nonpatient sample, and cut-scores were proposed for diagnostic purposes. Since then a number of studies using the 39-item scale have reported encouraging evidence of internal consistency and concurrent validity (e.g., Kins, Soenens, & Beyers, 2011; Kins et al., 2013; Lapsley & Edgerton, 2002; Lapsley, Varshney, & Aalsma, 2000). Moreover, a series of studies yielded a reliable 19-item scale that showed a strong pattern of association with psychiatric symptoms and family problems, self-esteem problems, and interpersonal problems (Lapsley, Aalsma, & Varshney, 2001). The 19-item scale was denoted by Lapsley et al. (2001) as a measure of dysfunctional individuation that could serve as a useful diagnostic screen in university counseling or clinical settings.

Although the 19-item Dysfunctional Individuation Scale (DIS) is promising, there is a lack of clarity on its factor structure, or whether 19 items is most the most parsimonious reduction of the scale. Previous research has relied on exploratory factor analysis and different analyses have yielded an inconsistent number of scale items (e.g., Lapsley et al., 2001). Moreover, evidence of how well the scale converges with other indices of individuation is not available. Therefore, research is needed to clarify the measurement properties of this promising clinical measure.

Overview of Studies

In Study 1, we examine the factor structure of the DIS using confirmatory factor analysis (CFA) and attempt to document a pattern of convergent and discriminant validity with other measures of dysfunctional and healthy individuation. We also examine the concurrent validity of the DIS by assessing its relationship with various adjustment problems and with clinical symptomatology. In Study 2, we attempt to replicate the factor structure of the DIS in a different sample. In addition, we examine the item properties of the DIS from an item response theory (IRT) perspective and provide support for the use of the DIS as a robust predictor of depressive symptoms. Study 2 also examines the incremental validity of the DIS over neuroticism when predicting depression. Study 3 more closely investigates the item properties of the DIS and shows additional evidence of concurrent validity through the DIS’s pattern of association with college adjustment problems. We also provide further support for the DIS as predictor of depression, stress, and anxiety using measures common in the clinical literature.

Study 1

The purpose of Study 1 was threefold. First, we examined the scale items and factor structure of the 19-item DIS using CFA to determine a parsimonious set of items. Second, we investigated the convergent and discriminant validity of the DIS by examining its associations with subscales of the Separation–Individuation Test of Adolescence (Levine et al., 1986). Third, we assessed concurrent validity by assessing the relationship between the DIS and the Brief Symptom Inventory (BSI; Derogatis, 1993) and College Adjustment Scales (CAS; Anton & Reed, 1991).

Method

Participants and Procedure

Participants were 245 (167 female, M_age = 20.4, SD_age = 3.80) undergraduates recruited from a large, Midwestern public university in the United States. Participants received course credit in exchange for completing the study. All measures were administered in the form of paper-and-pencil assessments.

Measures

Individuation

Participants completed the 19-item DIS (Lapsley et al., 2001). As noted earlier, the DIS includes items that tap differentiation failure, splitting, and relationship disturbances. Participants rated each item along a 10-point Likert-type scale ranging from 1 (Not characteristic) to 10 (Very characteristic). High scores indicated greater dysfunction.

Participants also completed subscales from the Separation Individuation Test of Adolescence (SITA; Levine et al., 1986). The Separation Anxiety subscale (13 items, α = .78) assesses strong fears of losing emotional or physical contact with an important other and feelings of rejection, abandonment, and desertion (e.g., “I frequently worry about being rejected by my friends”). The Engulfment Anxiety subscale (7 items, α = .81) assesses fear of being controlled, overpowered, and enveloped by close personal relationships so that the sense of independent selfhood is lost (e.g., “I can’t wait for the day I can live on my own and am free from my parents”). The Dependency Denial subscale (12 items, α = .87) assesses denial or avoidance of dependency needs by rejecting or failing to understand feelings of closeness or friendship (“I don’t feel that love has much of a place in my life”). These three scales were selected as measures of dysfunctional individuation because of their prior associations with poor psychological adjustment (Holmbeck & Leake, 1999). Finally, the Healthy Separation subscale (7 items, α = .73) assesses healthy appreciation of both dependency and independence (e.g., “I am comfortable with some degree of conflict in my close relationships”). Items were on a 5-point Likert-type scale ranging from 1 (Strongly disagree) to 5 (Strongly agree), with high scores indicating a stronger endorsement of the constructs.

Psychiatric symptoms

The BSI (Derogatis, 1993) was used to assess psychiatric symptomatology. The BSI consists of nine scales that assess depression (6 items, α = .89), anxiety (6 items, α = .82), phobic anxiety (5 items, α = .76), paranoid ideation (5 items, α = .79), obsessive compulsiveness (6 items, α = .89), somatization (7 items, α = .85), hostility (5 items, α = .78), psychoticism (5 items, α = .76), and interpersonal sensitivity (4 items, α = .85). Items were on a 5-point Likert-type scale ranging from 0 (Not at all) to 4 (Extremely), with high scores indicating a stronger endorsement of the symptom.

Adjustment to college

Participants completed three subscales from Anton and Reed’s (1991) College Adjustment Scales. These scales assessed Family Problems (12 items, α = .79; e.g., “My family doesn’t understand me”), Interpersonal Problems (12 items, α = .79; e.g., “A lot of people irritate me”), and Self-Esteem Problems (12 items, α = .85; e.g., “I feel good about myself”). Responses for each item ranged from 1 (Not true at all) to 4 (Very true).

Results

Preliminary Item Analyses

First, we examined the pattern of interitem correlations of the DIS to detect possible redundancy among the DIS items. In those cases where exceedingly high bivariate correlations (r > .70) between items of similar content were observed, we elected to retain only the briefer of the two items in the pair. Three items were removed due to this criterion. In addition, several items were exceedingly skewed in the positive direction with nearly all participants endorsing the lowest possible response categories. These items were eliminated to improve the performance of the scale as well as clarify the construct. Six items were omitted according to this criterion. See the appendix for the remaining 10 items.

CFA Model fitting

The reduced 10-item scale was fitted to a confirmatory factor model using the lavaan package (Rosseel, 2012) in the R statistical language (R Core Team, 2013). Preliminary analyses indicated a violation of the assumption of multivariate normality (Mardia’s coefficient = 12.67), so we proceeded to use robust standard errors and Satorra–Bentler’s scaled χ².

Overall, the 10 items had excellent fit to one latent factor in our confirmatory factor model, comparative fit index (CFI) = .97, Tucker–Lewis index (TLI) = .96, root mean square error of approximation (RMSEA) = .04 (90% CI = [.01, .06]), standardized root mean square residual (SRMR) = .04, Satorra–Bentler χ²(35, N = 245) = 49.87, p = .049. The parameter estimates and standardized parameter estimates appear in Table 1. As illustrated, all items have significant loadings onto the latent factor. Moreover, as indicated by the standardized path coefficients, the relative strength of the factor loadings is comparable across all 10 items. These 10 items also had high internal consistency (α = .82).

Table 1.

Summary of CFA Parameter Estimates From Study 1.

Item	Estimate	SE	Std. estimate	z Value	p Value
1.	1
2.	0.86	0.16	0.52	5.49	<.001
3.	0.93	0.15	0.52	6.16	<.001
4.	1.08	0.17	0.60	6.50	<.001
5.	0.96	0.17	0.50	5.53	<.001
6.	1.02	0.14	0.60	7.16	<.001
7.	0.75	0.14	0.49	5.41	<.001
8.	0.97	0.16	0.57	5.90	<.001
9.	1.19	0.19	0.67	6.22	<.001
10.	1.00	0.14	0.63	7.16	<.001

Demographic Differences

We next examined the possibility that scores on the DIS differed according to age and gender. A two-sample t-test revealed no significant differences between scores on the DIS for males and females, t(239) = −1.19, p = .23. Additionally, bivariate correlation revealed no significant relationship between age and DIS scores, r(239) = −.05, p = .46.

Bivariate Correlations

As indicated in Table 2, the DIS demonstrates a robust pattern of convergence in the expected direction with several SITA subscales. There is a strong correlation between dysfunctional individuation and the Dependency Denial subscale, the Separation Anxiety subscale, as well as the Engulfment Anxiety subscale. Moreover, the DIS was negatively correlated with the SITA’s Healthy Separation subscale. Hence, the DIS shows the expected pattern of convergent and discriminant validity with respect to the SITA scales.

Table 2.

Summary of Bivariate Correlations from Study 1.

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
Individuation
1. DIS	—
2. Separation Anxiety (SITA)	.53	—
3. Dependency Denial (SITA)	.38	.11	—
4. Healthy Separation (SITA)	−.42	−.18	−.36	—
5. Engulfment Anxiety (SITA)	.27	.23	.28	−.12	—
Brief Symptom Inventory
6. Depression	.48	.36	.30	−.27	.21	—
7. Anxiety	.44	.37	.19	−.20	.22	.76	—
8. Hostility	.34	.18	.23	−.17	.20	.69	.69	—
9. Paranoid Ideation	.54	.38	.32	−.26	.25	.77	.70	.62	—
10. Psychoticism	.44	.34	.26	−.22	.19	.81	.71	.69	.77	—
11. Somatization	.31	.24	.11	−.14	.21	.69	.79	.65	.64	.70	—
12. Obsessive Compulsive	.50	.37	.19	−.17	.25	.63	.74	.59	.72	.65	.59	—
13. Interpersonal Sensitivity	.50	.51	.16	−.24	.31	.73	.69	.59	.72	.70	.55	.66	—
14. Phobic Anxiety	.37	.33	.17	−.21	.21	.61	.69	.57	.58	.65	.64	.51	.59	—
College Adjustment Scales
15. Self-Esteem Problems	.56	.53	.28	−.35	.27	.47	.39	.27	.44	.38	.22	.45	.55	.27	—
16. Interpersonal Problems	.60	.36	.55	−.37	.26	.47	.39	.39	.49	.38	.29	.40	.38	.26	−.59	—
17. Family Problems	.35	.28	.37	−.21	.59	.38	.32	.26	.39	.30	.28	.32	.33	.27	−.40	.46	—
Demographic
18. Age	−.05	−.20	.01	−.17	−.19	−.11	−.06	−.07	−.11	−.06	−.05	−.05	−.17	−.12	−.10	−.05	−.13	—
19. Gender	.08	−.22	.33	−.21	−.11	.07	−.13	.09	−.05	.03	−.10	−.09	−.12	−.10	−.03	.14	−.09	.09	—
Mean	34.99	36.82	23.47	44.99	24.19	6.93	6.70	5.53	6.88	6.02	6.45	9.38	6.29	3.41	23.01	21.01	18.12	20.36	1.32
SD	14.53	7.74	7.48	4.91	6.83	5.93	4.84	4.61	4.49	4.65	5.73	4.88	4.14	4.11	6.33	5.53	5.30	3.80	0.47

Note. All correlations of magnitude greater than or .12 are significant at p < .05.

The DIS also shows a pattern of strong positive correlation with indices of psychiatric symptomatology. As evident in Table 2, the DIS was significantly correlated with depression, anxiety, hostility, paranoid ideation, somatization, obsessive compulsion, interpersonal sensitivity, and phobic anxiety. Moreover, with respect to the College Adjustment Scales, the DIS was also strongly associated with interpersonal problems, family problems, and with self-esteem problems. Hence, the DIS showed a strong pattern of concurrent validity with indices of psychiatric symptoms and with college adjustment problems.

Discussion

Study 1 documents a first attempt at the construction of a brief measure of dysfunctional individuation. Using CFA, we were able to fit a one-factor model to the reduced 10-item DIS. The model had excellent fit, as indicated by several fit indices, including CFI, TLI, and RMSEA. Furthermore, the resulting 10-item DIS showed a strong pattern of convergent and discriminant validity with other indices of individuation. In addition, the DIS was also strongly associated with a variety of indicators of psychiatric dysfunction as well as various college adjustment problems.

Study 2

The results of Study 1 give strong evidence of the association between the DIS and measures of pathology and adjustment problems. However, to further establish the concurrent validity of the DIS, we proceeded to investigate its relation to one of the most widely used and respected measures of depression—the Beck Depression Inventory (BDI-II; Beck, Steer, & Brown, 1996). The BDI-II has a long and well-validated history of measuring clinically relevant depressive symptoms that correspond to the criteria for major depressive disorder described by the Diagnostic and Statistical Manual of Mental Disorders (4th ed., DSM-IV-TR; American Psychiatric Association, 2000).

In addition to investigating the relationship between the BDI-II and dysfunctional individuation, we sought to test the incremental validity of the DIS over and above neuroticism. Neuroticism is understood as a generalized trait that disposes individuals toward negative emotional experiences (e.g., fear, anger, sadness; Watson, 2000). Accordingly, it will prove valuable to demonstrate that the effects of dysfunctional individuation are not reducible to trait neuroticism, given that this disposition is linked to similar negative outcomes.

Beyond testing the relationship between the DIS and the BDI-II, Study 2 allowed us to confirm the single-factor structure reported in Study 1 using a new sample. Additionally, we investigated DIS’s properties from the perspective of IRT (Lord & Novick, 1968). IRT is a model-based framework that presupposes an underlying latent trait that is dependent on both a person’s responses and the parameters of the items (Embretson & Reise, 2000). Thus, IRT allows us to simultaneously estimate item parameters while also estimating participants’ trait levels in a manner that is distinct from a total score calculation used in classical test theory.¹ In the present study, the use of IRT will be particularly helpful in estimating the individual item parameters for the 10-item DIS. This helps evaluate the contribution of the item to the total scale as well as the item’s general “performance,” and the extent to which the item is informative across the continuum of the latent trait.

Method

Participants and Procedure

Participants were 449 students (228 female, M_age = 19.8, SD_age = 1.29) recruited from a random sample of undergraduates at a large private university in the Midwestern United States. Participants were recruited via e-mail, and they completed the brief survey measures online in exchange for $5 in compensation.

Measures

Dysfunctional Individuation Scale

The 10-item DIS (α = .82, M = 34.10, SD = 14.07)—derived in Study 1— is a one-factor scale measuring pathological individuation. Items were on a 10-point Likert-type scale ranging from 1 (Not characteristic) to 10 (Very characteristic). The items appear in the appendix.

Beck Depression Inventory–II

BDI-II (α = .90) is 21-item self-report scale designed to assess the manifestation of depressive symptoms (Beck et al., 1996). The item format consists of 21 symptom categories (e.g., Sadness), each of which require the participants to select one of four statements (e.g., 0: “I do not feel sad.” 1: “I feel sad much of the time.” 2: “I am sad all the time.” 3: “I am so sad or unhappy that I can’t stand it.”). Scale scores are calculated by taking the sum across the 21 items.

Big Five Inventory

Neuroticism was assessed using the subscale from the Big Five Inventory (8 items, α = .82; John, Donahue, & Kentle, 1991; John, Naumann, & Soto, 2008). This consists of eight items (e.g., “Gets nervous easily”) on which participants rated the extent of their agreement with the item as self-descriptive. Responses ranged from 1 (Disagree strongly) to 5 (Agree strongly). The scale score was calculated by taking the mean of the eight items.

Results

Factor Structure of DIS

To confirm the factor structure observed in Study 1, we fit a confirmatory factor model to the 10-item DIS. Model fitting was done using the lavaan package (Rosseel, 2012) in the R statistical language (R Core Team, 2013). Preliminary analyses indicated a violation of the assumption of multivariate normality (Mardia’s coefficient = 71.91); therefore, we used robust standard errors, and report Satorra–Bentler’s scaled χ² below.

As in Study 1, the 10 items had excellent fit to one latent factor in our confirmatory factor model, CFI = .95, TLI = .94, RMSEA = .05 (90% CI = [.04, .07]), SRMR = .03, Satorra–Bentler χ²(35, N = 443) = 78.42, p < .001. The unstandardized and standardized factor loadings indicated that all items load strongly onto the latent factor and magnitude of the factor loadings reveal consistency across the 10 items in terms of their contribution to the latent factor. In addition, these items also possessed good internal consistency (α = .84).

IRT Item Properties of DIS

Item parameter estimates

To further investigate the properties of the 10-item DIS, we examined their slope and threshold parameters using a graded response model (Samejima, 1969, 1996) as implemented in the ltm package (Rizopoulos, 2006) in the R statistical language (R Core Team, 2013). Item slopes—also called discrimination parameters—describe how well a given item is able to differentiate between participants having levels of the latent trait above or below the item’s location (Baker, 2001). Threshold parameters—also called difficulty parameters—can be thought of as “cut points” on the latent trait continuum, where a participant with that level of the latent trait is equally likely to select response category j rather than category j + 1. Thus, a J-point Likert-type item will have J − 1 threshold parameters.

The slope and threshold parameter estimates for the DIS appear in Table 3. As the table illustrates, all slope parameters were significantly different from 0. This indicates the items are well suited to differentiating between participants who differ in the underlying level of dysfunctional individuation. Additionally, the similarity across the 10 slope parameter estimates supports the use of an unweighted sum scoring procedure for the scale (Cheng, Yuan, & Liu, 2012). It is also important to note the large standard errors that accompany the threshold parameter estimates from about b₇ to b₉. For several items the standard errors are in the double- and sometimes triple-digits. This suggests a lack of precision in the estimates of these threshold parameters. This is a result of participants not using the full range of the Likert-type scale, and in particular, that the upper categories were not well endorsed. This is further discussed below.

Table 3.

DIS Item Threshold and Slope Parameter Estimates for Study 2.

Item	b ₁	b ₂	b ₃	b ₄	b ₅	b ₆	b ₇	b ₈	b ₉	a
1.	−1.85 (0.19)	−1.20 (0.19)	−0.69 (0.15)	−0.18 (0.13)	0.32 (0.17)	0.81 (0.32)	1.60 (0.71)	2.58 (2.61)	3.10 (9.47)	1.21 (0.12)
2.	−0.46 (0.10)	0.16 (0.09)	0.77 (0.24)	1.27 (0.70)	1.81 (1.89)	2.18 (5.09)	2.57 (11.77)	2.85 (27.48)	3.53 (71.54)	1.63 (0.15)
3.	−1.12 (0.12)	−0.45 (0.11)	0.29 (0.10)	0.67 (0.27)	1.21 (0.58)	1.74 (1.44)	2.57 (5.17)	3.06 (20.40)	3.38 (54.22)	1.49 (0.14)
4.	−0.55 (0.10)	0.26 (0.09)	0.80 (0.28)	1.09 (0.68)	1.57 (1.32)	2.03 (3.21)	2.51 (8.44)	3.40 (38.51)	3.67 (161.31)	1.66 (0.15)
5.	−1.42 (0.18)	−0.66 (0.15)	−0.12 (0.13)	0.33 (0.21)	1.01 (0.37)	1.42 (0.78)	2.04 (1.40)	3.07 (3.74)	4.15 (16.97)	1.03 (0.11)
6.	−0.02 (0.08)	0.51 (0.16)	0.96 (0.47)	1.25 (1.21)	1.59 (2.53)	1.95 (5.73)	2.42 (15.65)	2.76 (46.84)	2.98 (115.37)	1.99 (0.19)
7.	−0.05 (0.09)	0.62 (0.18)	1.05 (0.48)	1.43 (1.01)	1.88 (2.07)	2.28 (4.62)	3.03 (12.90)	3.32 (42.57)	3.77 (88.53)	1.44 (0.15)
8.	−0.79 (0.12)	0.01 (0.09)	0.52 (0.19)	0.89 (0.42)	1.33 (0.76)	1.68 (1.49)	2.15 (2.78)	2.86 (7.17)	3.23 (21.62)	1.33 (0.13)
9.	−1.58 (0.15)	−0.85 (0.15)	−0.15 (0.10)	0.25 (0.15)	0.67 (0.28)	1.27 (0.58)	2.07 (1.78)	2.79 (7.48)	3.47 (30.37)	1.43 (0.13)
10.	−0.24 (0.08)	0.36 (0.11)	0.86 (0.37)	1.06 (1.00)	1.39 (1.84)	1.83 (4.52)	2.32 (15.30)	2.78 (58.13)	3.55 (380.85)	2.13 (0.20)

Note. b indicates threshold parameter, and a indicates discrimination parameter. SE estimates appear in parentheses.

Information function

Figure 1 illustrates the Fisher information functions for the 10 items of the DIS. An item’s information function is calculated as the inverse of its variance along the latent trait continuum. Thus, items with high information across a given length of the continuum provide particularly precise estimates of person parameters across that length of the latent trait continuum (Baker, 2001). As the figure indicates, the majority of the 10 items had especially high information across the 0 to 4 level of the latent trait. This is particularly true for Item 6 and Item 10.

Figure 1.

Item information curves for the 10-item Dysfunctional Individuation Scale in Study 2. These indicate most items are particularly informative between 0 and 4 on the latent trait continuum.

Demographic Differences

We went on to examine the possibility that scores on the DIS differed according to age and gender. A two-sample t test revealed a marginally significant difference between scores on the DIS for males and females, with males having slightly higher scores, t(440) = 1.79, p = .07. Bivariate correlation revealed no significant relationship between age and DIS scores, r(427) = −.08, p = .12.

Dysfunctional Individuation, Depression, and Neuroticism

The relationship between the BDI-II and the DIS was investigated using bivariate correlations. As expected, scores on the BDI-II and the DIS were highly correlated, r(428) = .49, p < .001. We also fit a multiple regression model regressing BDI-II scores on age, gender, and the DIS. Results indicated that the DIS was still a strong predictor of depression after controlling for age and gender, b = 0.28 (.02), β = .50, t(412) = 11.78, p < .001, with the model accounting for 26% of the variance in scores on the BDI-II, F(3, 412) = 47.04, p < .001.

Multiple regression analysis was used to examine the relationship between depression, dysfunctional individuation, and neuroticism. In particular, we fit regression models with scores on the BDI-II regressed on the DIS and the BFI’s neuroticism subscale. The results revealed that even when accounting for the influence of neuroticism, the DIS remains a robust predictor of depression, b = 0.18 (.02), β = .32, t(422) = 7.89, p < .001, with the model accounting for 42% of the variance in scores on the BDI-II, F(2, 422) = 148.10, p < .001.

Discussion

The results of Study 2 echo those from Study 1. Critically, the factor structure observed in Study 1 was replicated here with a new sample. Additionally, using a graded response model (Samejima, 1969, 1996), we estimated item slope and threshold parameters. Of particular interest in these analyses were the nonzero slopes across all items, which indicate these items are well suited to discriminating between participants differing on their underlying levels of dysfunctional individuation. Also of interest in these analyses were the large standard errors associated with the threshold parameter estimates on the upper end of the Likert-type range. This may indicate that participants were not using the full range of response categories. This is addressed in Study 3.

Examining the relationship between dysfunctional individuation and depression revealed a strong bivariate correlation between the DIS and the BDI-II. Furthermore, dysfunctional individuation remained a strong predictor of depression even after accounting for neuroticism, which has previously been shown to be highly associated with depression (Bagby, Joffe, Parker, Kalemba, & Harkness, 1995; Duggan, Sham, Lee, Minne, & Murray, 1995). This finding provides evidence that the negative effects of dysfunctional individuation are not merely a function of this dimension of personality.

Study 3

Based on the large SE estimates of the DIS item threshold parameters observed in Study 2, we elected to slightly modify the response format. Thus, the primary purpose of Study 3 is to examine the item properties of the revised version of the DIS that uses a modified Likert-type scale. In this final version of the DIS the response format was reduced from the previous 10 Likert-type categories to 7 Likert-type categories ranging from 1 (Strongly disagree) to 7 (Strongly agree). Research has shown that reliability, validity, and discrimination power are significantly higher with more response categories, but only to a point, specifically, up to seven categories (Cheng et al., 2012; Preston & Colman, 2000). Beyond this level, test–retest reliability is negatively affected (Preston & Colman, 2000). We also elected to have participants rate their agreement with the 10 items rather than the extent to which the items were characteristic of themselves in order to improve clarity and because it more closely mirrors other similar scales. Additionally, in this version, we included category labels for all 7 Likert-type categories. This has been shown to improve test–retest reliability (Weng, 2004).

Finally, we sought to demonstrate a pattern of convergent, discriminant, and concurrent validity of the DIS with this modified response format. In Study 3, we examine the relationships between the DIS and the Depression, Anxiety, and Stress Scales (DASS; Lovibond & Lovibond, 1995)—a well-respected measure used in the clinical area. We also examine the adjustment to college in a new sample using the CAS subscales (Anton & Reed, 1991) discussed above.

Participants and Procedure

Participants were 443 students (248 female, M_age = 20.1, SD_age = 1.29) recruited from a random sample of undergraduates at a large private university in the Midwestern United States. Participants were recruited via e-mail and completed the brief survey measures online. They were compensated $5 in exchange for their participation in the study.

Measures

Dysfunctional Individuation Scale

The 10-item DIS (α = .81, M = 32.80, SD = 10.01) assesses pathology of individuation. Responses ranged from 1 (Strongly disagree) to 7 (Strongly agree). Scale scores were computed by summing across participants’ responses to the 10 items.

College Adjustment Scales

As in Study 1, participants responded to three subscales of the CAS (Anton & Reed, 1991). The three subscales included the following: Family Problems (12 items, α = .83; e.g., “A lot of people irritate me”), Interpersonal Problems (12 items, α = .82; e.g., “My family doesn’t understand me”), and Self-Esteem Problems (12 items, α = .88; e.g., “I feel good about myself” reversed). Responses for each item ranged from 1 (Not true at all) to 4 (Very true).

Depression, Anxiety, and Stress Scale

The DASS (Lovibond & Lovibond, 1995) is a 21-tem scale with three 7-item subscales measuring participants’ symptoms of Depression (α = 84; e.g., “I felt that I had nothing to look forward to”), Anxiety (α = 91; e.g., “I felt close to panic”), and Stress (α = 82; e.g., “I found it difficult to relax”) experienced in the past week. Responses indicating symptom frequency were on a 0 to 3 Likert-type scale: 0 = Did not apply to me at all; 1 = Applied to me to some degree, or some of the time; 2 = Applied to me to a considerable degree, or a good part of time; 3 = Applied to me very much, or most of the time.

Results

Psychometric Properties of the DIS

Item parameter estimates

As in Study 2, we examined the item parameter estimates using a graded response model (Samejima, 1969, 1996) and the ltm package (Rizopoulos, 2006) in the R statistical language (R Core Team, 2013). The results are presented in Table 4. As this table indicates, all slope parameter estimates are significantly different from 0, indicating the items are all well suited to differentiate between participants across underlying levels of the latent factor. Moreover, the magnitudes of the slope parameters are comparable across the 10 items, which suggests the items’ contributions to the factor score are similar. This supports the use of un-weighted sum scoring to compute scale scores on the DIS (Cheng et al., 2012).

Table 4.

DIS Item Threshold and Slope Parameter Estimates for Study 3.

	b ₁	b ₂	b ₃	b ₄	b ₅	b ₆	a
Item 1	−2.91 (0.30)	−1.47 (0.27)	−0.92 (0.22)	−0.45 (0.20)	0.68 (0.22)	2.15 (0.99)	1.18 (0.12)
Item 2	−1.29 (0.15)	0.21 (0.08)	0.77 (0.25)	1.51 (0.70)	2.74 (4.18)	4.47 (97.45)	1.37 (0.14)
Item 3	−1.70 (0.15)	−0.27 (0.12)	0.26 (0.12)	0.69 (0.25)	1.72 (1.24)	2.95 (20.64)	1.82 (0.17)
Item 4	−1.18 (0.13)	0.30 (0.07)	0.82 (0.28)	1.23 (0.72)	2.28 (3.07)	3.30 (29.61)	1.61 (0.16)
Item 5	−2.58 (0.32)	−1.04 (0.22)	−0.20 (0.16)	0.37 (0.23)	1.64 (0.50)	3.32 (2.74)	0.91 (0.11)
Item 6	−1.09 (0.12)	0.11 (0.07)	0.52 (0.19)	1.03 (0.44)	1.88 (1.66)	2.75 (11.14)	1.66 (0.16)
Item 7	−0.73 (0.12)	0.49 (0.11)	1.15 (0.42)	1.64 (1.15)	2.52 (3.78)	4.46 (89.86)	1.38 (0.15)
Item 8	−1.32 (0.15)	0.11 (0.09)	0.60 (0.22)	1.01 (0.46)	1.98 (1.18)	3.82 (17.68)	1.28 (0.14)
Item 9	−2.31 (0.23)	−1.16 (0.21)	−0.48 (0.16)	0.02 (0.15)	1.51 (0.44)	2.58 (3.28)	1.35 (0.14)
Item 10	−0.65 (0.09)	0.29 (0.07)	0.76 (0.28)	1.04 (0.75)	1.94 (3.05)	2.93 (41.75)	2.03 (0.20)

Note. b indicates threshold parameter, and a indicates discrimination parameter. SE estimates appear in parentheses.

It also is worth noting the SE of the item threshold estimates in Table 4. Recall that in Study 2 we observed substantial SE estimates for thresholds b₇, b₈, and b₉. This was largely a function of the skewed distribution of responses, and participants’ disinclination to endorse the higher Likert-type categories. As Table 4 illustrates, we again have large SE estimates associated with the highest item threshold (b₆), but the reduction from 10 Likert-type categories to 7 categories—and perhaps the addition of more category labels—has increased the precision of our item threshold estimates.

Item information

Figure 2 illustrates the Fisher information curves for the 10 items of the DIS. Items with high information along a certain length of the latent continuum provide particularly precise estimates of person parameters across that length of the latent trait continuum (Baker, 2001). As Figure 2 indicates, the items were particularly informative from −2 to 3 on the latent continuum. Compare this to Figure 1, which displays the item information curves from the previous study. Note that the items in Study 2 were most informative from 0 to 4 on the latent continuum. This comparison suggests that the modifications made to the labels and range of response categories in Study 3 expanded the range over which the items are informative. Whereas the items in Study 2 were primarily informative across the upper range of the latent continuum, the items in Study 3 are quite informative across a broader range.

Figure 2.

Item information curves for the 10-item Dysfunctional Individuation Scale in Study 3. These indicate most items are particularly informative between −2 and 3 on the latent trait continuum.

Demographic Differences

Next, we tested whether scores on the DIS differed according to age and gender. A two-sample t test revealed a no significant difference between scores on the DIS for males and females, t(433) = 0.76, p = .45. Bivariate correlation revealed no significant relationship between age and DIS scores, r(415) = −.01, p = .82.

Bivariate Correlations

Dysfunctional individuation and college adjustment

The DIS proved to be highly related to several indicators of adjustment during college. In particular, dysfunctional individuation was related to interpersonal problems, r(428) = .56, p < .001, and family problems, r(429) = .37, p < .001. Moreover, the DIS also showed a strong relationship with self-esteem problems, r(430) = .51, p < .001. Gender and self-esteem problems were correlated, with females reporting higher levels of self-esteem problems than males, r(436) = .11, p = .018. Thus, we also examined the relationship between self-esteem problems and dysfunctional individuation after accounting for the influence of gender. As expected, even after controlling for gender, dysfunctional individuation continued to be a strong predictor of self-esteem problems, b = −0.34 (.03), β = −.51, t(428) = −12.36, p < .001.

Dysfunctional individuation and clinical symptomatology

As in Studies 1 and 2, the DIS proved to be a strong indicator of clinically relevant symptoms. More specifically, dysfunctional individuation was highly related to self-reported depressive symptoms, r(428) = .41, p < .001. This is similar in magnitude to the relationship between the DIS and the BDI-II observed in Study 2. Additionally, dysfunctional individuation was also a robust predictor of both anxiety and stress, r(433) = .44, p < .001, r(433) = .36, p < .001, respectively.

Discussion

The results of Study 3 indicated that slight modifications to the DIS response categories could improve the psychometric properties of the scale. More specifically, reducing the number of response categories from 10 to 7 and including categories labels across the entire range of categories improved the precision of the item threshold estimates. Additionally, this modification also broadened the range across the latent continuum over which the items were most highly informative.

Moreover, Study 3 showed the DIS to be strongly associated in the appropriate direction with self-esteem problems, family problems, and interpersonal problems. We also found the DIS to be a strong indicator of a variety of clinically relevant indices of dysfunction, including depression, anxiety, and stress.

General Discussion

The DIS measures dysfunctional individuation that results from the failure to balance one’s dependence and independence from others. Recent studies have demonstrated that this construct is critically important to overall well-being (e.g., Kins et al., 2011; Kins et al., 2013; Lapsley & Edgerton, 2002). The aim of the current project was to develop a brief measure of dysfunctional individuation based on Christenson and Wilson’s (1985) original measure.

Over the course of three studies, and using three different samples, we derived a 10-item measure of dysfunctional individuation. The resulting DIS exhibited a strong one-factor structure, excellent psychometric properties, and a pattern of convergent and discriminant validity with other measures of individuation. Moreover, we also demonstrated the concurrent validity of the DIS through its relationships with a variety of indices of adjustment (i.e., CAS), psychiatric symptoms (i.e., BSI), and clinically relevant symptoms (i.e., BDI-II, DASS). Additionally, the results of Study 2 showed that the relationship between dysfunctional individuation and clinical symptomatology cannot be explained as merely a function of dispositional neuroticism.

The results of these studies provide considerable evidence for the construct validity of the DIS. Unlike other measures the DIS is brief, simple to administer, and consistently shown to predict a range of adjustment and psychiatric symptoms. Furthermore, its brevity and strong psychometric properties recommends its use in longitudinal investigations and intervention studies. As such the DIS is a welcome addition to the toolbox of measures that assess various aspects of a critical developmental challenge that has significant implications for successful adaptation.

It should be noted, however, that the trade-off of using a brief measure such as this one is that we may risk overlooking potentially interesting nuances related to the individuation process that might otherwise be picked up through more lengthy assessments such as the SITA. We believe, however, that the DIS takes a broader approach to the measure of dysfunctional individuation. We believe also that there are advantages to the more general approach; in particular, it facilitates interpretation, and it makes the measurement more expedient.

But the evident advantages of the DIS should be understood in light of several limitations. First, our three studies were conducted using undergraduate samples drawn entirely from the Midwestern United States. And in particular, Studies 2 and 3 recruited from a university population that is made up of approximately 74% Caucasian students. Future research should address the generalizability of the present outcomes with more diverse samples. Second, although the present data show that the DIS is not a surrogate measure of dispositional neuroticism, additional research should address how dysfunctional individuation aligns with closely related constructs such as dyadic attachment. Third, the present data were obtained from relatively high-functioning university undergraduates. Although there is a reported increase in the range and severity of presenting mental health problems and clinically significant psychopathology among university students (e.g., Kitzrow, 2003), future research should address the ability of the DIS to discriminate nonreferred community samples from clinical controls. Moreover, it will be useful to know if cut-scores can be identified to reliably indicate clinically significant levels of dysfunction. Finally, at its core dysfunctional individuation is a developmental construct. Hence, tracking patterns of inter-individual variability and intra-individual change using longitudinal methods will be an informative line of research, particularly in the context of a wider range of outcome variables that map onto other developmental challenges, such as relational intimacy and identity, and in terms of potential moderators, including intrusive parenting.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Allen

J. P.

Hauser

S. T.

Eickholt

Bell

K. L.

O’Connor

T. G.

(1994). Autonomy and relatedness in family interactions as predictors of expressions of negative adolescent affect. Journal of Research on Adolescence, 4, 535-552.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., Text revision). Washington, DC: Author.

Anton

W. D.

Reed

J. R.

(1991). College adjustment scales: Professional manual. Odessa, FL: Psychological Assessment Resources.

Bagby

R. M.

Joffe

R. T.

Parker

J. D. A.

Kalemba

Harkness

K. L.

(1995). Major depression and the five-factor model of personality. Journal of Personality Disorders, 9, 224-234.

Baker

(2001). The basics of item response theory (ERIC Clearinghouse on Assessment and Evaluation). College Park, MD: University of Maryland.

Beck

A. T.

Steer

R. A.

Brown

G. K.

(1996). Manual for the Beck Depression Inventory–II. San Antonio, TX: Psychological Corporation.

Beyers

Goossens

Van Calster

Duriez

(2005). An alternative substantive factor structure of the Emotional Autonomy Scale. European Journal of Psychological Assessment, 21, 147-155.

Bleiberg

(2001). Treating personality disorders in children and adolescents. New York, NY: Guilford Press.

Blos

Peter

. (1962). On adolescence: A psychoanalytic interpretation. New York: The Free Press.

10.

Cheng

Yuan

K.-H.

Liu

(2012). Comparison of reliability measures under factor analysis and item response theory. Educational and Psychological Measurement, 72, 52-67.

11.

Christenson

R. M.

Wilson

W. P.

(1985). Assessing pathology in the separation-individuation process by means of an inventory: A preliminary report. Journal of Nervous and Mental Disease, 173, 561-565.

12.

Coonerty

(1986). An exploration of separation–individuation themes in borderline personality disorder. Journal of Personality Assessment, 50, 501-511.

13.

Derogatis

L. R.

(1993). BSI Brief Symptom Inventory: Administration, scoring, and procedures manual (4th ed.). Minneapolis, MN: National Computer Systems.

14.

Dolan

B. M.

Evans

Norton

(1992). The Separation–Individuation Inventory: Association with borderline phenomena. Journal of Nervous and Mental Disease, 180, 529-533.

15.

Duggan

Sham

Lee

Minne

Murray

(1995). Neuroticism: A vulnerability marker for depression evidence from a family study. Journal of Affective Disorders, 35, 139-143.

16.

Embretson

S. E.

Reise

S. P.

(2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.

17.

Hoffman

J. A.

(1984). Psychological separation of late adolescents from their parents. Journal of Counseling Psychology, 31, 170-178.

18.

Holmbeck

Leake

(1999). Separation-individuation and psychological adjustment in late adolescence. Journal of Youth and Adolescence, 28, 563-581.

19.

John

O. P.

Donahue

E. M.

Kentle

R. L.

(1991). The Big Five Inventory—Versions 4a and 54. Berkeley: University of California, Berkeley, Institute of Personality and Social Research.

20.

John

O. P.

Naumann

L. P.

Soto

C. J.

(2008). Paradigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issues. In John

O. P.

Robins

R. W.

Pervin

L. A.

(Eds.), Handbook of personality: Theory and research (pp. 114-158). New York, NY: Guilford Press.

21.

Josselson

(1980). Ego development in adolescence. In Adelson

(Ed.), Handbook of adolescent psychology (pp. 188-210). New York, NY: Wiley.

22.

Josselson

(1988). The embedded self: I and thou revisited. In Lapsley

D. K.

Power

F. C.

(Eds.), Self, ego, and identity: Integrative approaches (pp. 91-106). New York, NY: Springer-Verlag.

23.

Kins

Beyers

Soenens

(2013). When the separation-individuation process goes awry: Distinguishing between dysfunctional dependence and dysfunctional independence. International Journal of Behavioral Development, 37, 1-12.

24.

Kins

Soenens

Beyers

(2011). “Why do they have to grow up so fast?” Parental separation anxiety and emerging adults’ pathology of separation-individuation. Journal of Clinical Psychology, 67, 647-664.

25.

Kitzrow

M. A.

(2003). The mental health needs of today’s college students: Challenges and recommendations. NASPA Journal, 41, 167-181.

26.

Koepke

Denissen

J. J. A.

(2012). Dynamics of identity development and separation–individuation in parent–child relationships during adolescence and emerging adulthood—A conceptual integration. Developmental Review, 32, 67-88.

27.

Lapsley

D. K.

Aalsma

M. C.

Varshney

N. M.

(2001). A factor analytic and psychometric examination of pathology of separation-individuation. Journal of Clinical Psychology, 57, 915-932.

28.

Lapsley

D. K.

Edgerton

(2002). Separation-individuation, adult attachment style and college adjustment. Journal of Counseling & Development, 80, 484-493.

29.

Lapsley

D. K.

Rice

K. G.

Shadid

G. E.

(1989). Psychological separation and adjustment to college. Journal of Counseling Psychology, 36, 286-294.

30.

Lapsley

D. K.

Varshney

Aalsma

(2000). Psychological attachment and attachment style in late adolescence. Journal of Adolescence, 23, 137-155.

31.

Levine

Green

Millon

(1986). The separation-individuation test of adolescence. Journal of Personality Assessment, 50, 123-137.

32.

Levitz-Jones

E. M.

Orlofsky

J. L.

(1985). Separation-individuation and intimacy capacity in college women. Journal of Personality and Social Psychology, 59, 156-169.

33.

Lord

F. M.

Novick

M. R.

(1968). Statistical theories of mental test scores. Reading. MA: Addison-Wesley.

34.

Lovibond

S. H.

Lovibond

P. F.

(1995). Manual for the Depression Anxiety Stress Scales (2nd. ed.). Sydney, New South Wales, Australia: Psychology Foundation.

35.

Mahler

M. S.

Pine

Bergman

(2000). The psychological birth of the human infant symbiosis and individuation. New York, NY: Basic Books.

36.

Mattanah

J. F.

Brand

B. L.

Hancock

G. R.

(2004). Parental attachment, separation-individuation and college student adjustment: A structural equation analysis of mediational effects. Journal of Counseling Psychology, 51, 213-226.

37.

McClanahan

Holmbeck

G. N.

(1992). Separation-Individuation, family functioning, and psychological adjustment in college students: A construct validity study of the separation-individuation test of adolescence. Journal of Personality Assessment, 59, 468-485.

38.

Pine

(1979). On the pathology of the separation-individuation process as manifested in later clinical work. International Journal of Psychoanalysis, 60, 225-242.

39.

Preston

C. C.

Colman

A. M.

(2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104, 1-15.

40.

Quintana

S. M.

Kerr

(1993). Relational needs in late adolescent separation-individuation. Journal of Counseling & Development, 71, 349-354.

41.

R Core Team. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0. Retrieved from http://www.R-project.org/

42.

Rizopoulos

(2006). ltm: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software, 17, 1-25. Retrieved from http://www.jstatsoft.org/v17/i05/

43.

Rosseel

(2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1-36. Retrieved from http://www.jstatsoft.org/v48/i02/

44.

Ryan

R. M.

Lynch

J. H.

(1989). Emotional autonomy versus detachment: Revisiting the vicissitudes of adolescence and young adulthood. Child Development, 60, 340-356.

45.

Samejima

(1969). Estimation of latent ability using a pattern of graded scores (Psychometrika Monograph Supplement, No. 17). Greensboro, NC: Psychometric Society.

46.

Samejima

(1996). The graded response model. In van der Linden

W. J.

Hambleton

R. K.

(Eds.), Handbook of modern item response theory (pp. 85-101). New York, NY: Springer.

47.

Steinberg

Silverberg

(1986). The vicissitudes of autonomy in early adolescence. Child Development, 57, 841-851.

48.

Tackett

J. L.

Baliss

Oltmanns

T. F.

Krueger

R. F.

(2009). A unifying perspective on personality pathology across the lifespan: Developmental considerations for the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders. Development and Psychopathology, 21, 687-713.

49.

Watson

(2000). Mood and temperament. New York, NY: Guilford Press.

50.

Weng

L.-J.

(2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement, 64, 956-972.

51.

Westen

Heim

(2003). Disturbances of self and identity in personality disorders. In Leary

M. R.

Tangney

J. P.

(Eds.), Handbook of self and identity (pp. 643-664). New York, NY: Guilford Press.