Abstract
Guided by the Goldfried and D’Zurilla behavioral-analytic model, we developed and psychometrically evaluated a measure of heterosocial competence and, in doing so, learned more about the construct in young adults. The Measure of Heterosocial Competence (MHC) is a social problem-solving questionnaire assessing competence in challenging other-sex social interactions. Internal consistency was acceptable and component analyses yielded one hierarchical factor and two group factors suggesting that heterosocial competence is comprised of the ability to initiate interactions, resolve conflict, and manage sexual situations. The MHC was significantly related to measures of general social competence, anxiety in heterosexual situations, and relationship quality and conflict. The MHC appears to be a promising and unique measure with potential in both research and applied settings.
Recently, researchers have made some strides in better defining and conceptualizing heterosocial competence. 1 Early definitions focused almost exclusively on the competencies that brought about contact with opposite-sex persons and thus restricted the requisite skill sets to those in the initiation dimension (Hansen, Christopher, & Nangle, 1992; Kelly, 1982). Other attempts to catalog related skill sets expanded the scope to include the initiation, maintenance, and termination of social and/or sexual relationships (Barlow, Abel, Blanchard, Bristow, & Young, 1977; Galassi & Galassi, 1979). Even this broader definition, however, overlooked the fact that many important other-sex interactions occur outside of the relationship context (Grover & Nangle, 2003). More recently, Hansen et al. (1992) suggested a more encompassing definition of heterosocial competence as the ability to select appropriate responses in a wide range of social situations involving the other sex, including casual relationships, friendships, romantic relationships, and sexual relationships. Grover and colleagues refined this definition by replacing response selection with “the ability to effectively negotiate social situations” and the inclusion of acquaintanceships (Grover, Nangle, Serwik, & Zeff, 2007).
Using this broader definition, heterosocial competence is most likely a multidimensional construct requiring a range of social skills, including communication, assertion, and problem-solving, as well as nonverbal behaviors, such as eye contact and gestures (Conger & Conger, 1982; Egland, Spitzberg, & Zormeier, 1996; Goddard & McFall, 1992; Hansen et al., 1992). These skill sets are expected to overlap with those required in same-sex interactions (e.g., initiating a conversation, interpreting and understanding the affective state of others, assessing peer norms and behaviors), but not completely (e.g., negotiating the anxiety associated with other-sex social interactions, establishing deeper levels of intimacy, integrating sexuality and passion; Connolly, Furman, & Konarski, 2000; Nangle & Hansen, 1998). Importantly, the term “heterosocial” should not be interpreted as suggesting that the construct is relevant only for heterosexual individuals. Just as other-sex oriented individuals also engage in same-sex interactions, those with a same-sex orientation also engage in other-sex interactions (Savin-Williams & Ream, 2007).
In this paper, we describe our systematic efforts to develop a measure of heterosocial competence for young adults. There were many reasons to do so. First, major developmental theorists agree that other-sex interactions are critical for long-term adjustment (Erikson, 1963; Sullivan, 1953). These interactions help to fulfill many functions, including companionship, intimacy, social support, and opportunities for sexual activity (Bukowski, Sippola, & Hoza, 1999; Furman & Shaffer, 2003; Hansen et al., 1992). Not surprisingly, success in other-sex interactions is associated with a sense of well-being and fewer internalizing problems (Connolly, Craig, Goldberg, & Pepler, 2004; Connolly & Johnson, 1996; Feiring, 1999; La Greca & Mackey, 2007; Richards, Crowe, Larson, & Swarr, 1998). Second, young adults find these types of interactions to be the most challenging of all social situations. They perceive interpersonal situations with romantic partners as more difficult than those involving close same-sex friends (Buhrmester, Furman, Wittenberg, & Reis, 1988). Relationship difficulties are one of the most frequently reported problems for those presenting at college counseling centers and are associated with increased mood disturbances and suicidal ideation (Benton, Robertson, Wen-Chih, Newton, & Benton, 2003; Erdur-Baker, Aberson, Barrow, & Draper, 2006; Heppner et al., 1994). Moreover, one third of all college-aged males experience high levels of anxiety about dating (Chorney & Morris, 2008; Dodge, Heimberg, Nyman, & O’Brien, 1987).
We previously developed a parallel questionnaire for younger adolescents (Measure of Adolescent Heterosocial Competence; Grover, Nangle, & Zeff, 2005), but felt there were compelling reasons to also target young adults. Although critical for social development at both stages, heterosocial competence likely differs between adolescence and young adulthood due to changing situations and demands. Both adolescents and young adults name dating and sexual concerns as their primary worries (Benton et al., 2003; Kaufman, Brown, Graves, Henderson, & Revolinski, 1993), yet the type of dating and sexual situations that need to be negotiated are likely very different. For instance, young adults are more likely to be involved in serious dating relationships than early adolescents (Feiring, 1999; Neeman, Hubbard, & Masten, 1995). In addition, sexual behavior is both more prevalent and increasingly a part of a committed relationship in young adulthood than in adolescence (O’Sullivan, Mantsun, Harris, & Brooks-Gunn, 2007). Beyond relationships, young adults experience a variety of new social interactions as they move away from home and enter the workforce or attend college. Less structure and adult supervision during these interactions leads to possible ambiguity and uncertainty about appropriate social behaviors (Thompson & Rapee, 2002). Mixed-sex dorms, large classes, and new jobs provide increased opportunities to socially relate to the other sex.
Another reason to develop a measure of heterosocial competence for young adults is that few existing measures target heterosocial interactions with this age group (see Norton, Grills-Taquechel, & Raouf, 2010; Spitzberg, 2003, for reviews) and those that do, consistent with the prevailing construct definitions at the time, tend to focus on initiation situations or a relatively restricted range of interaction types. For example, the Dating and Assertion Questionnaire (DAQ) (Levenson & Gottman, 1978) contains nine items that assess self-reported frequency of achievement of dating goals (e.g., participants rate how often they “Get a second date with someone you have dated once”). Although the DAQ has a fair degree of external validity as item content was informed by issues raised by participants in a dating therapy group, the DAQ assesses frequency rather than quality of behavior. The Heterosocial Assessment Inventory for Women (HAI-W) (Kolko, 1985) offers an increased range of situations (i.e., 12 vignettes that encompass conversation, date initiation, and sexual initiation), as well as additional response dimensions (e.g., likelihood, anxiety, and perceived skillfulness of response), but is limited by its reliance on global ratings of perceived quality of behavior. Especially with less competent individuals, Likert-type self-ratings of skillfulness and potential outcome to a situation may be compromised by a lack of social knowledge or experience. The Survey of Heterosexual Interactions (SHI) (Twentyman & McFall, 1975) is one of the most frequently employed measures of heterosocial competence. An outgrowth of the minimal dating research movement, it focuses primarily on initiation situations with an unknown person and requires participants to rate their ability to initiate or maintain each interaction. Interestingly, a factor analytic investigation by Wallander, Conger, Mariotto, Curran, and Farrell (1980) suggested that it might actually be more of a heterosocial anxiety measure. As social anxiety impedes both the learning process and the execution of previously learned social skills, it makes sense that anxiety in heterosocial situations would have important implications for competence. However, the heterosocial competence construct is likely comprised of more than just the anxiety associated with interaction initiation.
Our measure development process was guided by the oft-cited behavioral-analytic model proposed by Goldfried and D’Zurilla (1969). This model has served as the framework for the development of a number of social competence measures (e.g., Cavell & Kelley, 1992; Dodge, McClaskey, & Feldman, 1985; Goddard & McFall, 1992; Grover et al., 2005; MacNeil & LeCroy, 1997). From a more traditional trait-like perspective, social skills are thought of as being a function of stable, underlying dispositional characteristics and, as such, expected to be relatively stable across time and situation (McFall, 1982). In contrast, consistent with a behavioral perspective, the behavioral-analytic model assumes that social skills are situation-specific and therefore more variable across situations (Goldfried & D’Zurilla, 1969; Mischel, 1968). Thus, measure development necessitates assurances that the full range of relevant situations is adequately sampled. Of course, the view that each situation is wholly unique is untenable and would render any related prediction impossible (Kazdin, 1977; McFall, 1982). In addressing this problem, Goldfried and D’Zurilla (1969) recommend steps to ensure that the sampled situations be particularly meaningful (i.e., relevant, difficult, and common) to the targeted group. Others have suggested additional strategies for clustering critical social situations in meaningful ways (e.g., Dodge & Murphy, 1984; McFall, 1982). Notably, from a more practical perspective, one could still interpret a summary score on a situation-based measure in a more traditional manner. For example, probabilistically speaking, an individual who is more likely to respond in a competent way across more situation types could be viewed as “more competent.” The situation-based approach, however, offers the advantage of allowing for the identification of problematic situation types for a sample or an individual and is therefore thought of as being more clinically useful (Dodge & Murphy, 1984; Goldfried & D’Zurilla, 1969; McFall, 1982). Identification of problematic situations may then be used to guide the situational focus of intervention and to aid in the discernment of the cause of the difficulty (i.e., problem-solving knowledge vs. performance deficit).
Goldfried and D’Zurilla (1969) provide a framework for situation-based measure development. The first step is the identification of all relevant situations for the target population. A variety of methods can be used at this stage, including self-report, interviews, and naturalistic observation. Next, the resulting situations are presented to a different sample to determine through self- or other-ratings whether they are indeed meaningful or critical (i.e., difficult to resolve and commonly experienced). The second step is referred to as response enumeration and involves obtaining a sampling of possible responses to the identified critical situations. Again, this step can be done utilizing a variety of methods, including interview, questionnaire, analog situations, and naturalistic observation. In the third step, the degree of effectiveness for each generated response is evaluated by individuals in direct contact with the target population. In the fourth step, the measure is actually constructed. Items are developed based on the situations identified in step one, and response options are selected that represent a range of effectiveness as determined in the previous step. Finally, in the fifth and final step, the validity of the new measure is evaluated.
The studies that follow describe the development of the Measure of Heterosocial Competence (MHC). Five studies are presented, each following a step prescribed in the Goldfried and D’Zurilla (1969) model. College students were recruited from introductory psychology courses at a large public university in New England. All students signed informed consent; parental consent was waived for 17 year olds enrolled in the university. Study 1 describes situation identification, in which participants were given a survey that asked them to write down a list of difficult heterosocial situations. Study 2 describes the distillation of that list to those situations that a new sample of participants rated as most common and most difficult, thus identifying the situations that may be most “critical” in determining competence. In Study 3, a new sample of participants wrote down what they would do and say in response to the critical situations collected in Study 2. In Study 4, expert judges rated the responses gathered in the prior step for competence. Using the data from Studies 1 through 4, a multiple choice measure was created with critical situations as questions and responses of varied levels of competence as item choices. In Study 5, a new sample of participants completed the resulting measure and a battery of theoretically related and unrelated measures. In this final study, we refined the scale and examined its psychometric properties.
Study 1: Situation identification
Method
Participants (N = 264; 151 females) ranged in age from 17 to 21 (M = 18.76; SD = .87) and the majority were Caucasian (96%). Participants completed a survey that asked them to write down as many difficult situations with the other sex as they could in one hour.
Results
Participants generated over 6000 total responses and over 1000 unique situations. The list of 1000 situations was organized into emergent themes by two graduate students and one advanced undergraduate research assistant working independently (e.g., first meetings, balancing girl/boyfriend and other-sex friends). The researchers then met as a group to discuss and reach a consensus regarding emergent themes. The list was further reduced by collapsing similar situations into broader themes (e.g., “how to turn down a friend who asks you out on a date” and “how to say no to a date when not interested” were collapsed into “turning down a date”). The resulting list was 125 unique situations.
Study 2: Situation validation
Method
Participants (N = 227; 139 females) ranged in age from 18 to 21 (M = 18.72; SD = .79) and the majority were Caucasian (96%). Participants rated the 125 unique situations on five-point Likert-type scales of “commonness” and “difficulty” similar to those used in other studies utilizing the Goldfried and D’Zurilla approach (e.g., Cavell & Kelley, 1992). The commonness scale asked the participant to rate how common the situation was for college students (e.g., “When girlfriend/boyfriend doesn't approve of your friends. How common is this situation for college students in general? Rate on a scale of 1 to 5 where 1 is not common and 5 is very common”). The difficulty scale asked the participant to rate how difficult each situation was to successfully negotiate (e.g., “When girlfriend/boyfriend doesn't approve of your friends. How difficult is this situation to successfully negotiate? Rate on a scale of 1 to 5 where 1 is not difficult and 5 is very difficult”).
Results
Situations that are considered “critical” for competence are those that regularly occur and that represent a challenge. Situations that are infrequent are less relevant and situations that are easily negotiated will not discriminate between more and less competent individuals. As such, responses were retained for the next step if they were rated above the mean commonness score of 3.8 and rated above the mean difficulty score of 2.6. A total of 38 situations met criteria for being both common and difficult, and were used in the next study.
Study 3: Response enumeration
Method
Participants were 260 undergraduates (154 females) who ranged in age from 18 to 23 (M = 19.07; SD = .94). The majority of participants were Caucasian (91%). The Response Enumeration Questionnaire was created using the 38 critical situations identified in Study 2. A small group of undergraduate research assistants aided in turning each critical situation into a brief, two to three-sentence vignette (e.g., You and your boyfriend are at a party and you notice him begin to flirt with someone else. What would you do and say?). Participants were asked to imagine themselves in each of the 38 vignettes and to write down what they thought they would actually say and/or do in response.
Results
All responses were transcribed and edited for redundancy within each response list (e.g., Cavell & Kelley, 1992). For example, if five participants indicated the same response to the same situation (e.g., talk to him calmly, talk it out with him, ask him to talk about it, etc.), only one response was preserved for the final list of responses for that situation.
Study 4: Response evaluation
Method
Collected responses were evaluated by 12 judges expert in college student heterosocial behavior (e.g., college professor with expertise in social behavior, nurse practitioner employed at a college health center, therapist in the college counseling center). Half of the judges were male and 75% were Caucasian. Age of judges ranged from 26 to 51 (M = 40.58, SD = 8.43). Years of experience working with college students ranged from three to 26 (M = 13.5, SD = 7.37). Judges were provided with a questionnaire consisting of the vignettes combined with the college student-generated responses. Judges rated each response on a five-point Likert-type scale ranging from (1) incompetent to (5) very competent using the following definition of a competent response: A competent or effective response is one that solves the present problem, makes future problems of the same type less likely, and does not introduce any new problems for the person (e.g., Cavell & Kelley, 1992; Gaffney & McFall, 1981; Grover et al., 2005). Each judge was sent a randomly selected subset of vignettes and responses due to the length of the survey. Each college student response was rated by at least three judges (e.g., Cavell & Kelley, 1992).
Results
Mean and standard deviations of judges’ ratings were computed for each generated response (e.g., MacNeil & LeCroy, 1997). Vignettes elicited a range of responses in terms of their competency ratings (from one to four, or two to five in most cases). One vignette was dropped from consideration due to insufficient variance in the judges’ competency ratings in the response list, indicating several “competent” responses generated, and no “less competent” responses generated. For each of the 37 remaining vignettes, four responses were selected to comprise the multiple-choice responses on the final measure. Using the mean competency rating, within-situation responses were divided into quartiles. Four responses were selected within each vignette to represent a most competent response (selected from the top 25%), a least competent response (from the bottom 25%), and two mid-range responses (one from each of the middle quartiles) (e.g., MacNeil & LeCroy, 1997). To optimize the differentiation of competence levels, response items that fell on the border of a quartile range were avoided. In addition, agreement among the judges is important as the consensus suggests that the behavior would be consistently reinforced in real-life situations (Goldfried & D’Zurilla, 1969); therefore, response items with higher judge agreement were favored. Order of responses within each vignette was randomly selected to avoid potential response bias. To complete this step, item vignettes were combined with responses to create multiple choice items. The most competent item was scored a 4, whereas the least competent item was scored a 1 (see Appendix A for sample items and scoring).
Study 5: Initial Validation
Method
Over 400 college students (169 males, 238 females) completed a battery of measures to aid in scale refinement and psychometric investigation of the MHC. Participants ranged in age from 18 to 23 years (M = 19, SD = 1.19) and were primarily Caucasian (90%). The majority of participants reported being in a dating relationship (61%) for an average length of 17.31 weeks (range from 1 to 84 weeks; SD = 14.44). The majority were sexually active (85%, with 10 participants not responding) and the mean age of onset of sexual intercourse was 16.48 (range from 11 to 22; SD = 1.5). Rates of dating and sexual activity and onset are similar to those reported in other college student samples (e.g., Ross & Bowen, 2010).
Based on our conceptualization of heterosocial competence as bearing significant overlap with more general, same-sex social competence (Conger & Conger, 1982; Grover et al., 2007; Grover et al., 2005; Hansen et al., 1992; Kelly, 1982), we selected a convergent measure of general social competence (i.e., Social Skills Inventory). A unique feature of heterosocial interactions is the significant anxiety they engender (Chorney & Morris, 2008; Dodge et al., 1987; Grover et al., 2007). As such, we also included both more heterosocially specific (i.e., Survey of Heterosexual Interactions) and more general social anxiety (i.e., Social Anxiety Scale for Adolescents) measures for our convergence tests. Another index of social competence is relationship quality, (Rose-Krasnor, 1997) and we included related measures (i.e., Network of Relationships Inventory) for other-sex relationships. As there is evidence of a link between social skills and aggression (e.g., Marshall, 2010; Nicotera, Steele, Catalani, & Simpson, 2012), we also included a measure of aggressive responding in conflict situations in romantic relationships (i.e., Conflict Tactics Scale) as a measure of convergent validity. Finally, to vary method, we also included a report-by-other (friend report on the SHI). More challenging was the selection of a measure of discriminant validity, as so many constructs are theoretically related to social competence. As such, we followed Foster and Cone’s (1995) recommendation that all new measures be tested to ensure that they are not related to certain pervasive constructs, such as socioeconomic status (SES) and intelligence. We chose the former. The data were collected in single two-hour sessions.
Demographics
A brief personal information sheet asked participants their age, race, if they were currently dating and the length of that relationship, if they had ever engaged in sexual intercourse, and age at first sexual intercourse. Participants were also asked to indicate parent occupation and education to determine socioeconomic status using the Hollingshead (1975) classification system. Possible scores on the Hollingshead range from eight to 66 with higher scores indicating higher SES.
Heterosocial competence
The MHC is a multiple-choice questionnaire designed to assess competence in heterosocial situations. Each item represents one critical heterosocial situation and is paired with four potential responses of varying competence. For each item, participants were asked to select the response that reflects what he/she would do or say in the given situation. Items were scored on a scale of one (least competent) to four (most competent). The MHC male and female versions differ only in pronouns. In the process of scale refinement described below, the number of MHC items were reduced to 31 with internal consistency in the acceptable range (α = .78). Notably, the coefficient alpha most likely underestimates the true reliability of scores on multidimensional measures such as the MHC (Osburn, 2000). Factor analyses (also described below) suggested the utility of two subscale scores: Relationship conflict and sexual situations (α = .74) and Initiation situations (α = .61). As the subscales emerged in the item analysis phase within the same study as the validation, we had no a priori hypotheses involving them.
General social competence
The Social Skills Inventory (SSI) (Riggio, 1986) was administered to assess general social competence. This 90-item self-report instrument generates three subscales of social skill and three subscales of emotional/nonverbal competence, as well as a total score. The current study utilized the total score. Reliability of the total is acceptable with a recent study reporting internal consistency of .94 (Riggio, Riggio, Salinas, & Cole, 2003). In the current study, the coefficient alpha was acceptable at .87. The SSI has correlated positively with measures of non-verbal social skill, emotional intelligence, communication skills, and perceived support and negatively with social anxiety and loneliness (Miczo, Sergin, & Allspach, 2001; Riggio & Kwong, 2009; Schutte et al., 2001). It was hypothesized that higher levels of heterosocial competence would be positively associated with SSI total scores.
Self and peer reports of anxiety in heterosexual social situations
Anxiety in heterosexual social situations was assessed using the Survey of Heterosexual Interactions (SHI) (Twentyman & McFall, 1975; Williams & Ciminero, 1978). The SHI consists of 20 items that assess perceived difficulty in initiating and maintaining social interactions with the other sex. Internal reliability coefficients range from .89 to .91 (Twentyman & McFall, 1975). Scores range from 20 to 140 with higher scores indicating less anxiety, and more comfort in heterosexual social situations. Low SHI scores are associated with increased anxiety and physiological reactivity during heterosocial role-play tests, less assertiveness, and less frequent dating (Twentyman & McFall, 1975). Although often utilized as a measure of heterosocial competence, a factor analytic investigation by Wallander and colleagues (1980) suggested that it more accurately assesses anxiety in heterosexual situations. In addition to self-report, the participant was asked to give a peer-rating version of the SHI to a close same-sex friend. The peer rater then completed the SHI based on his or her observation of the participant and returned it directly to the researchers (return rate 64.9%). In the current study, Cronbach’s alpha was .92 and .95 for the self- and friend-report versions. It was hypothesized that higher levels of heterosocial competence would be associated with lower levels of self- and friend-reported anxiety in heterosexual social situations (note that this would yield a positive correlation).
Social anxiety
The Social Anxiety Scale for Adolescents (SAS-A) (La Greca, 1999) is one of the most widely used measures of social anxiety in youth (Kearney, 2005) and has been used with college students (e.g., Ho & Lau, 2011). The scale consists of 22 self-statements rated on a five-point Likert-type scale ranging from not at all (1) to all the time (5). A total score in addition to three subscale scores (i.e., fear of negative evaluation, social anxiety and distress in general situations, social anxiety and distress in new situations) can be computed. Test–retest reliability for the SAS-A total score was .86 over 10 days (Garcia-Lopez, Olivares, Hidalgo, Beidel, & Turner, 2001) and ranged from .54 to .78 across the subscales over a two-month interval (La Greca, 1999). The SAS-A is related to measures of social functioning (Young & Lo, 2012), dating anxiety (Glickman & La Greca, 2004), and other measures of social anxiety (Anderson, Jordan, Smith, & Inderbitzen-Nolan, 2009). Total scores range from 18 to 90, with higher scores indicating greater social anxiety. In the current study, internal consistency was .92. It was hypothesized that higher levels of heterosocial competence would be associated with lower levels of social anxiety.
Relationship quality
The quality of the participant’s best other-sex friendship, and current or last romantic relationship was assessed by the Network of Relationships Inventory (NRI) (Furman & Buhrmester, 1985). This measure has been used with adolescent and college student samples (e.g., Moilanen & Raffaelli, 2010). Participants rated 33 items on a 5-point scale to measure various relationship qualities (e.g., reliable alliance, intimacy, conflict, satisfaction of relationship, companionship). Analysis of internal consistency has yielded satisfactory reliability for the subscales (Furman & Buhrmester, 1985; Moilanen & Raffaelli, 2010). Furthermore, test–retest reliability has been supported over a one-month period (Connolly & Konarski, 1994). Factor analyses indicated two dimensions of relationship quality including social support and negative interaction (Furman, 1996). Social Support and Negative Interaction scale scores were computed for each of the relationships assessed. Internal consistency coefficients of both scales, and for each relationship, were above .90. It was hypothesized that higher heterosocial competence would be significantly related to increased social support and decreased negative interactions.
Conflict negotiation
The Conflict Tactics Scale – Revised (CTS2) (Straus, Hamby, Boney-McCoy, & Sugarman, 1996) is a 78-item self-report measure used to assess conflict negotiation strategies utilized in relationships. Items on the CTS2 yield scores on five subscales: negotiation, psychological aggression, physical assault, sexual coercion, and injury. Internal consistency is strong across the subscales with coefficients ranging from .86 to .95. The CTS2 correlates in expected directions with other indices of violence (Friend, Cleary Bradley, Thatcher, & Gottman, 2011; Yun & Vonk, 2011) and is considered the gold standard for assessment in this area (see Straus, Hamby, & Warren, 2003). The current study utilized the psychological aggression, physical assault, and sexual coercion subscales and a negative relation was hypothesized between heterosocial competence and the three subscales.
Results: Scale analyses
Cross-validation of scoring.
The measure was sent to an additional five expert judges (e.g., developmental psychologist with expertise in social relationships in emerging adulthood, clinical psychologist with expertise in social skills, clinical psychologist employed in a college counseling center) to cross-validate the scoring key. Intraclass Correlation Coefficients (ICC) (two-way fixed, single measure, absolute agreement) were computed between the measure total and each judge, as well as between each keyed item and each judged item. Cicchetti and Sparrow (1981) categorize ICC coefficients between .60 and .75 as good, and above .75 as excellent. Interrater reliability on the measure total was excellent (range = .85 to .94). Interrater reliability was excellent on 31 out of 33 items (range = .77 to 1.00) and good on one item (range = .72). One item was dropped due to low interrater reliability (r = .43; Item 30: You and a female are discussing whether the two of you are ready for sex. What would you do and say?).
Component analysis
Parallel analysis (Horn, 1965) on the remaining 36 items indicated that the first three eigenvalues exceeded values that would be expected by chance. However, the third eigenvalue just exceeded that expected by chance and the scree plot indicated that two factors, not three, should be extracted. Thus, it was not surprising that when three factors were extracted the third factor did not have any items that loaded heavily solely on it, and those items that did load on it did not seem to have conceptual or methodological similarities. Thus, two factors were extracted using a principal components extraction with an oblique rotation in the Statistical Package for the Social Sciences (SPSS Version 18.0). The first factor (which explained 11.6% of the variance) was named “Relationship conflict and sexual situations,” as items that loaded heavily on this factor were characterized by situations involving problem-solving in negative or difficult situations, such as a romantic partner cheating or breaking up with their partner, addressing jealousy in a romantic relationship, and addressing unwanted sexual advancement. The second factor (which explained 5.8% of the variance) was named “Initiation situations,” as items loading heavily on this factor were characterized by situations involving initiation of heterosocial interactions, such as wanting to approach someone in a group setting, asking for a phone number, or asking for a date (See Table 1).
The Measure of Heterosocial Competence (MHC) hierarchical model parameter estimates: Standardized loadings and standard errors.
Note. A dash indicates that a loading was constrained to equal zero (i.e., the item was not allowed to load on that factor). Values in parentheses are standard errors.
* p < .05.
Confirmatory Factor Analyses (CFA) were conducted using Mplus version 5.0 statistical software (Muthen & Muthen, 1998–2007) to determine if the MHC factor structure was best represented by only two factors or by two group factors plus a general hierarchical factor. Items were assigned to the two group factors based on the factor pattern matrix loadings from the principal components extraction, and all items were also allowed to load on the general factor. Items that did not load significantly on the group factors (Relationship conflict and sexual situations or Initiation situations) were removed from these factors. If an item did not load significantly on either the group factors or the hierarchical factor, it was removed from the model and scale. This resulted in the removal of five items from the MHC scale: “You and a female are out on a first date. You are not sure how intimate to get emotionally with her. What would you do and say?”; “You are going to meet your girlfriend’s family for the first time. What would you do and say?”; “You are hanging out with both your girlfriend and your female friends. What would you do and say?”; “You are in the mall one day when you run into your ex-girlfriend. She begins to have a conversation with you. What would you do and say?”; and “You need to get up and speak in one of your classes. The class has a lot of females in it. What would you do and say?”
After removal of these items, a hierarchical model with a general factor and two group factors was a good fit to the data as indicated by fit indices: the comparative fit index (CFI) (Bentler, 2004) equaled .95, the Tucker–Lewis Index (TLI) (Tucker & Lewis, 1973) equaled .96, and the Root Mean Square Error of Approximation (RMSEA) (Steiger, 1989) equaled .025. Further, all but four items loaded significantly on the general factor and removal of this factor resulted in a significant decrement in model fit, Δχ2 (21) = 244.5, p < .001. Table 1 indicates which items load on each of the group factors and provides standardized factor loadings for each factor an item loads on in this hierarchical model. Estimation of ωh (Zinbarg, Yovel, Revelle, & McDonald, 2006) revealed that the general factor accounted for approximately 54% of the variance in MHC scores.
Measurement invariance was examined to determine if the hierarchical representation of the MHC described above was invariant for males and females. Due to the relatively large number of parameters being estimated, the sample size was not adequate to conduct a multiple-groups CFA. Therefore, configural invariance was examined by testing the model separately in both males and females. The hierarchical model was a good fit to the data for both females: CFI = .94, TLI = .95, RMSEA = .029; and males: CFI = .91, TLI = .92, RMSEA = .035. Partial metric invariance was examined by constraining loadings in the male sample equal to the loadings from the female sample for those loadings whose male sample confidence intervals included the female sample point estimates. Following these guidelines resulted in a model where 75.9% of the loadings in the male sample were constrained to equal those from the female sample. This partially metric invariant model was an acceptable fit to the data: CFI = .90, TLI = .91, RMSEA = .037, and constraining these loadings did not result in a significant decrement in model fit from the unconstrained configural model, Δχ2 (26) = 35.5, p = .10. These findings provide evidence of partial metric invariance of the MHC factor structure for males and females. Thus, there is evidence that the MHC factor structure is the same for males and females and that the majority of items indicate factors with the same strength in both males and females (i.e., most item factor loadings are the same for males and females).
Results: Measure validation
Normative data
Means and standard deviations for MHC total and subscale scores are presented in Table 2. All three scores were compared to the demographic information collected. The initiation situations subscale differed by gender with men (M = 24.76) reporting higher levels of competence than women (M = 22.92), t(397) = 4.61, p < .001. As might be expected, participants in a current dating relationship (M = 94.12) reported significantly higher MHC total scores than single participants (M = 92.23), t(390) = 1.75, p = .041. Participants who reported being sexually active reported higher scores on the initiation situations subscale (M = 23.88) than those who were not sexually active (M = 22.19), t(387) = 2.98, p = .002. Interestingly, the relationship conflict and sexual situations subscale was positively correlated with age of first sexual intercourse, r(327) = .12, p = .012; whereas the initiation situations subscale was negatively correlated with age of first sexual intercourse, r(328) = −.09, p = .045. None of the MHC scores were correlated with age or length of dating relationship and MHC scores did not differ by ethnic group, although the sample size for non-Caucasian participants was quite small.
Means, standard deviations, and ranges for the measures of validity.
1Marked variables were significantly positively skewed; therefore, median values are presented.
MHC: Measure of Heterosocial Competence; SHI: Survey of Heterosexual Interactions; SSI: Social Skills Inventory; SAS-A: Social Anxiety Scale for Adolescents; NRI-NI: Network of Relationships Inventory Negative Interactions; NRI-SS: Network of Relationships Inventory Social Support; CTS: Conflict Tactics Scale.
Discriminant validation
As hypothesized, none of the three MHC scores were significantly related to the Hollingshead classification of socioeconomic status.
Convergent validation
Descriptive data on each measure are summarized in Table 2. Pearson product–moment correlations and Spearman rho correlations (for the skewed CTS variables) were computed between the MHC, MHC subscales, and the validation measures (see Table 3). Only associations that reached a significance level of p < .01 are noted to reduce the chance of Type I error. Results were collapsed across gender as there was little difference in the pattern of associations among the variables between men and women. The few significant differences are noted in the text.
Convergent validity correlations.
**p < .01, ***p < .001; Spearman correlations were used with the CTS psychological aggression, physical assault, and sexual coercion subscales due to skewed data. Variations in sample size are due to missing data.
CTS: Conflict Tactics Scale.
First, we examined the relation among the MHC and measures of social skill and social anxiety. Although not related to the MHC total score, general social skills were positively related to the MHC initiation situations subscale (SSI) (r = .15, p = .001). As hypothesized, higher levels of heterosocial competence were associated with lower levels of anxiety in heterosexual social situations (SHI) (r = .34, p < .001) and lower levels of overall social anxiety (SAS-A; r = −.29, p < .001). MHC initiation situations subscale scores were more strongly aligned with these measures, evidencing significant relations with all of the measures of social skill and anxiety.
Next, we examined the relations among the MHC and measures of relationship quality. As hypothesized, higher levels of heterosocial competence were associated with lower levels of negative interactions in both romantic relationships (NRI-NI-RR) (r = −.24, p < .001) and other-sex friendships (NRI-NI-OF) (r = −.15, p = .002). Also as expected, higher levels of heterosocial competence were associated with higher levels of social support in romantic relationships (NRI-NI-RR) (r = .14, p = .005). However, contrary to hypotheses, heterosocial competence was not related to social support in other-sex friendships. Relations with the MHC subscales revealed that the relationship conflict and sexual situations subscale scores were significantly associated with lessened negative interactions in romantic relationships, whereas the initiation situations subscale scores were significantly associated with increased social support in other-sex friendships. Analyses of gender differences revealed that the initiation situations subscale was significantly associated with lower levels of negative interactions in other-sex friendships for women, but not for men (ZDifference = −2.68, p = .007).
Finally, we looked at relations between heterosocial competence and relationship aggression. As expected, higher levels of heterosocial competence were associated with lower levels of aggressive responses to relationship conflict including psychological aggression (CTS psychological aggression) (r = −.32, p < .001) and physical assault (CTS physical assault) (r = −.20, p < .001). The MHC relationship conflict and sexual situations subscale scores were better aligned with scores on these measures, evidencing significant relations with all of the relationship aggression indices.
Discussion
Following the Goldfried and D’Zurilla (1969) model, we developed and evaluated the Measure of Heterosocial Competence (MHC), a 31 item social problem-solving questionnaire comprised of brief vignettes depicting challenging other-sex interactions for which individuals select response options that most closely match what they would do. A key feature of the model is its emphasis on a comprehensive situational analysis. MHC items were derived from an initial pool of more than 6,000 identified situations. Reliance on the “target audience” in this process built in a level of content and ecological validity. The initial psychometric evaluation of the MHC yielded promising results. Scores were found to be internally consistent, converge with measures of like constructs, and diverge with an index of socioeconomic status. A hierarchical and two group factor structure explained most of the variance in MHC scores. Initial support for incremental validity was demonstrated in that MHC scores contributed uniquely to the prediction of relationship quality after controlling for general social skill and to a lesser extent, heterosexual-social anxiety.
The significance of initiation, relationship conflict, and sexual situations for the college student population was reflected in the MHC factor structure. Confirmatory factor analysis yielded support for a model including a general hierarchical factor (total score) and two group factors (initiation; relationship conflict and sexual). Examining the interrelationships among the MHC and convergent measures adds to our knowledge about the heterosocial competence construct and its possible underlying skill sets. As might be expected, MHC initiation scores were associated with general social skills and anxiety. Given the overlap between same- and other-sex social interactions (Grover et al., 2007; Hansen et al., 1992; Kelly, 1982; Nangle & Hansen, 1998), convergence with the SSI was hypothesized, although we did expect the relationship to be somewhat stronger. Also not anticipated was the lack of association between MHC total scores and the SSI. Not surprisingly, SHI and SAS-A scores bore strong inverse relationships to MHC initiation scores. Based on earlier heterosocial competence definitions, SHI items assess perceived difficulty in initiating and maintaining social interactions with other-sex individuals. These situations are known to engender significant degrees of anxiety in young adults (Chorney & Morris, 2008; Dodge et al., 1987) and low SHI scores (reverse scored) are associated with increased anxiety and physiological reactivity in heterosocial role-play tests and less frequent dating (Twentyman & McFall, 1975). Glickman and La Greca (2004) suggest that the ability to cope with the anxiety associated with other-sex interactions best distinguishes heterosocial from more general social competence. Anxiety explained a portion, but not all, of the overall variance in heterosocial competence (9%), which is consistent with models that posit social competence is multidetermined (e.g., Rose-Krasnor, 1997).
Although MHC total scores were related to the majority of measures assessing interactions in already established relationships (i.e., relationship quality, relationship aggression), the initiation dimension was related to only two and the magnitude of those correlations was relatively low. The poor performance of the initiation scores relative to the relationship variables is consistent with the notion that requisite skill sets may vary as a function of relationship stage (Galassi & Galassi, 1979). That is, it may be that initiation skills determine the successful launch of a relationship, but other skill sets determine the quality (e.g., level of companionship, instrumental aid, antagonism, conflict) of an ongoing relationship. In contrast, increased competence in relationship conflict and sexual situations was associated with decreased negative interactions in romantic and other-sex friend relationships, as well as lower levels of psychological and physical aggression and sexual coercion. Young adults experience an increase in intimacy in close relationships, thus increasing the potential for conflict (Waldinger et al., 2002) and demands on related skill sets. Finally, as discussed by Sullivan (1953) and others (Connolly et al., 2000; Nangle & Hansen, 1998), managing sexual situations is a considerable challenge for older adolescents and young adults. They require an ability to balance sexual desires and intimacy needs and are often fraught with ambiguities, such as misperceived romantic interests on the part of friends and the recent trend toward “friends with benefits” (Koenig, Kirkpatrick, & Ketelaar, 2007; Owen & Fincham, 2011).
The promising initial psychometric results coupled with the unique social-cognitive approach for assessing heterosocial competence suggests that the MHC may have useful clinical applications. D’Zurilla and Goldfried (1971) stress the distinction between the process of problem-solving and executing a response. Although successful problem-solving does not guarantee a competent response, as other factors such as anxiety, motivation, and behavioral deficits can interfere, successful problem-solving is a necessary precursor for competent responding. Therefore, as a situation-based, social problem-solving measure, the MHC could be used at the individual level to help isolate particular problematic situations for further assessment and intervention. In this vein, Goldfried and D’Zurilla recommend teaching social problem-solving skills (e.g., statement of the social problem, generation of alternatives, selecting an action, evaluating that action) to increase overall competence (see Crick & Dodge, 1994; Goldfried & D’Zurilla, 1969, for examples of the social problem-solving approach). Another useful application would be in the discrimination of knowledge vs. performance deficits. In concert with a role-play assessment, the MHC could be used to determine whether an individual knows what to do in a given situation and whether actual behavioral enactment may be more of a concern. Key to future MHC validation will be an investigation of the ability of the measure to relate to actual in vivo performance. At present, whether a respondent’s endorsements predict what he/she would actually do in those situations remains unknown.
The promise of the resulting measure must be considered within the context of the limitations of the described studies. Some hypothesized convergent measures functioned more like divergent ones, either bearing no significant associations or much lower than expected effect sizes. It is possible that the relations among these constructs and the MHC were attenuated by the difference in assessment approach (traditional self-report measures of global perceptions of competence vs. multiple choice social problem-solving measure). On a related note, we struggled to identify a divergent construct in an a priori fashion and thus limited our assessment to an indicator of SES. Another limitation is that because the sample was not large enough to split in half, Exploratory factor analysis (EFA) and CFA were conducted in the same sample, meaning that the structure may be influenced by chance characteristics of this particular data set. Although future work should be done to validate the MHC factor structure in a confirmatory manner in an independent sample, it is promising that the factors discovered in the EFA appear to be conceptually meaningful. An additional limitation was the reliance on self-report measures in the validation study. Notably, scores on the sole “other-report” (SHI friend report) were significantly associated with the MHC initiation, but not relationship conflict/sexual dimension. This suggests that some people may have better access to some competence aspects than others and should be considered in future MHC validation. One significant unknown is the degree to which scores predict in vivo responding. In other words, future studies should investigate whether respondents actually engage in the kinds of enactment strategies selected. Generalizability is yet another concern. The degree to which the situations generated by this sample of college students generalize to other student, as well as non-student, samples is a question. Regarding the latter, recent studies failed to find any meaningful differences between college and non-student samples on measures assessing personality, politics, and attitudes toward crime (Cooper, McCord, & Socha, 2010; Wiecko, 2010).
A discussion of generalizability limits would not be complete without addressing the lack of overall diversity in our samples. Even regional variations amongst college student samples can be important (e.g., Rentfrow, 2010), and surely ethnic and other demographic differences would be thought to impact findings. With regard to heterosocial competence, for example, female Latino teens have a pattern of sexual development characterized by a later onset of sexual activity and an earlier age at marriage and birth of a child than compared to their White Non-Latino peers (Upchurch, Aneshensel, Mudgal, & McNeely, 2001). The MHC, at least in its current form, may not be appropriate for use with more exclusively same-sex oriented samples as many of the current items refer to dating and sexual situations with the other sex. It is likely that same- and other-sex oriented individuals are similar in relationship competence, as past research suggests that mixed- and same-sex committed couples are indistinguishable in terms of self-reported relationship quality and observed ratings of relationship quality in a conflict interaction (Roisman, Clausell, Holland, Fortuna, & Elieff, 2008). In contrast, the construct of heterosocial competence likely differs between other- and same-sex oriented individuals in terms of relevant other-sex situations, appropriate social goals, and what would constitute a competent response to those critical situations. Future research on the heterosocial competence construct should focus on the role and functions of other-sex relationships in the social development of sexual minority youth and young adults.
We are very encouraged by the results of our ongoing efforts to better define and measure heterosocial competence. A validation study in progress is examining the construct validity of the MHC as a social problem-solving measure. Using a multi-trait, multi-method design, we are hypothesizing convergence with self-report and daily diary measures of more general social problem-solving. To examine divergence, we are comparing MHC scores with performance on two cognitive tasks, one tapping into verbal (and hence more social) and the other non-verbal forms of intelligence. We also hope to address sample generalizability concerns by carrying out MHC investigations with more diverse samples. Clearly, there is much work to be done. In closing, we would also like to encourage others to focus their efforts on this relatively overlooked construct. After all, heterosocial interactions are at once crucial for adjustment and uniquely challenging for young adults.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Note
Appendix A
Sample items, responses and scoring of the measure of heterosocial competence male version.
Directions: For each question select ONE item that best matches what you would do in each situation.
1. You and a female friend have been spending a lot of time together. Her boyfriend begins to get jealous of the amount of time the two of you spend together. What would you do and say?
5. There is a female that you are interested in and you want to call her. What would you do and say?
16. You and your girlfriend have a disagreement and you begin arguing. What would you do and say?
27. You want to approach a female but she is with a group of her friends. What would you do and say?