Why Self-Reports of Happiness and Sadness May Not Necessarily Contradict Bipolarity: A Psychometric Review and Proposal

Abstract

It is assumed that bipolarity in happiness and sadness requires mutual exclusion. However, we present psychometric research to show how coendorsements of happiness and sadness do not necessarily constitute evidence against bipolarity. Because individuals have a tendency to endorse emotion terms close to their current state, individuals whose current state is close to the middle of a bipolar continuum would report both happiness and sadness, despite their current state being best represented by a single point. As such, endorsements of happiness and sadness are theoretically expected when individuals’ current state is in the middle of the continuum. Bipolarity however would be violated if endorsements of extreme levels of happiness and sadness can be demonstrated, which has yet to be shown.

Keywords

bipolarity happy–sad measurement valence

Individuals naturally construe happiness as the opposite of sadness (Russell, 1980). Thus, the common experience of mixed feelings—simultaneous experiences of both happiness and sadness—is puzzling. This conundrum has confounded not only the lay person, but for decades has also divided emotion theorists seeking to determine how the affect space should be parsed. Given the common experience of mixed feelings, can the affect space be truly accounted for with a bipolar dimension with happiness and sadness on opposing ends—a model that implies mutual exclusivity (Barrett & Bliss-Moreau, 2009; Russell, 2003)? Or should happiness and sadness be construed as separable and distinct components, eliminating the assumption of mutual exclusivity, to account for mixed feelings (Cacioppo & Bernston, 1994; Cacioppo & Gardner, 1999)?

The theoretical orientation of happiness and sadness is one of the longstanding controversies in psychological science (Larsen & McGraw, 2014). In seeking a scientific resolution of this issue, researchers have proposed that methodological advances—not simply theoretical advances—are required to resolve the issue (Greenwald, 2012). Indeed, the research tools employed have shifted from simple correlations to more sophisticated models that account for measurement error (Barrett & Russell, 1998; Green, Goldman, & Salovey, 1993) and response formats (Russell & Carroll, 1999). While these methodological advances have helped clarify specific issues, the apparent conflict between bipolar (happiness and sadness as opposites) and bivariate (happiness and sadness as orthogonal) perspectives has not been reconciled. In this article, we propose that methodological considerations in psychometric research can help to fundamentally address this question. Specifically, we propose that if happiness and sadness are bipolar, there can still be a lack of mutual exclusion due to how individuals make responses to emotion items. In other words, bipolarity does not require mutual exclusion because coendorsements of happiness and sadness can occur on a bipolar continuum.

To address this issue, we structure our article as follows. We provide a brief background on how bipolarity has assumed the mutual exclusion of happiness and sadness. However, we question this assumption by presenting a psychometric model of item responding which accommodates the phenomenon of mixed feelings but retains an underlying bipolar continuum (Spector, van Katwyk, Brannick, & Chen, 1997; L. Tay & Drasgow, 2012). We review past research to demonstrate that this pattern of responding is common in self-reports and occurs in self-reported emotions; moreover, past evidence showing support of a bivariate view of emotions does not contradict this pattern of responding. In view of this, we propose that evidence stronger than mutual exclusion is required to demonstrate that happiness and sadness are not bipolar.

Background on the Bipolarity of Happiness and Sadness

The bipolarity debate has focused on whether the valence dimension of the affect space is bipolar or not; that is, whether happiness and sadness (at a point in time) are opposites. The early methodological paradigm for testing bipolarity has been correlational but ascertaining the magnitude of the correlation demonstrative of bipolarity has been controversial (see Barrett & Bliss-Moreau, 2009). Factors such as type of emotion scale items used, measurement error, acquiescence, and scale response formats can change the expected value of the correlation demonstrating bipolarity (see Russell & Carroll, 1999, for a review) and it is difficult to obtain a definitive threshold for bipolarity.

Therefore, researchers have adopted the mutual exclusion paradigm which is ostensibly more straightforward. Quoting Russell and Carroll (1999, p. 25), “bipolarity says that when you are happy, you are not sad and that when you are sad, you are not happy.” Therefore, bipolarity would hold when it can be shown that happiness and sadness are not coendorsed (save measurement error) at any moment in time. On the other hand, bipolarity would be incorrect should it be demonstrated that coendorsements of happiness and sadness occur. Research has shown that while happiness and sadness are reciprocally activated, coendorsements occur with regularity (Schimmack & Colcombe, 2007). Further, nonnegligible coendorsements of happiness and sadness occur in naturalistic and experimental settings leading to a growing conclusion that happiness and sadness are not bipolar (Larsen & Green, 2013; Larsen & McGraw, 2011). See Larsen and McGraw (2014) for a review of the evidence obtained from a variety of stimuli (e.g., music, games of chance) against bipolarity, which is predicated on a lack of mutual exclusion.

Mutual Exclusion and Item Responding

The assumption that endorsements of happiness and sadness need to be mutually exclusive is fundamental to our current understanding of bipolarity. However, we propose that mutual exclusion is not a necessary condition for bipolarity. This is because psychometric research shows that individuals use a self-report process (i.e., item response process) where they are most likely to endorse items closest to (i.e., more descriptive of) their current state (Spector et al., 1997; L. Tay & Drasgow, 2012). Items in this context refer to emotion terms (or emotion statements); endorsements here mean having higher ratings (e.g., choosing 5 = “very descriptive of me” over 1 = “not descriptive of me”). This item response process is formally known as the ideal point response process in the psychometric literature (see review by L. Tay & Drasgow, 2012).

In the case of a bipolar continuum, individuals whose current state is closer to the middle of the continuum as shown in Figure 1A would have the highest probability of endorsing items that are located in the middle of a bipolar continuum (e.g., “neutral”); they would have a high probability of endorsing items that are close to their current state (e.g., “slightly sad,” “slightly happy”), a moderate probability of endorsing items that are moderately close (e.g., “moderately happy,” “moderately sad”), and a small probability of endorsing items that are far away from their current state (e.g., “extremely sad,” “extremely happy”). As such, mutual exclusion in the endorsements of happiness and sadness is not necessary for a bipolar continuum to hold.

Figure 1.

Examples of item endorsement patterns.

We seek to elaborate and clarify this model of item responding for the issue of happiness and sadness bipolarity. First, individuals are located at only a single point on the bipolar continuum; that is, the valence state of individuals at any given moment can be accurately and fully described by one location on the bipolar valence continuum. This is consistent with the fundamental notion of bipolarity (Russell & Carroll, 1999). Nevertheless, a single location closer to the middle of the continuum can often give rise to responses on emotion scale items that exhibit “mixed feelings” or coendorsements of happiness and sadness. Making this distinction between the underlying state (i.e., “true” or “latent” state) and responses to scale items is consistent with current measurement theories. Endorsements of multiple emotion items on different locations on a continuum do not necessarily equate to multiple underlying locations. Instead, item endorsements along the continuum are averaged (or triangulated) to the likely underlying state.

Second, it is important to note that this response process follows a probabilistic pattern, and should not be regarded as acquiescence (Bentler, 1969) or random measurement error (Green et al., 1993). According to this response process, it is theoretically expected that individuals have a higher probability of endorsing emotion items closer to their current level of happiness along the continuum; conversely, they have a lower probability of endorsing emotion items further away. Acquiescence on the other hand occurs when individuals tend to endorse all items along the bipolar continuum; and random measurement error occurs when individuals endorse items along the bipolar continuum in an irregular random manner (e.g., carelessness). As such this proposed process is different from other past proposed processes—although it can occur in tandem with acquiescence and certainly random measurement error.

Third, while this pattern of responding allows for observations of mixed feelings, or coendorsements of happiness and sadness, it is not an unfalsifiable proposition. Because individuals tend to endorse items that are closest to their location along the continuum, when individuals experience more extreme feelings of happiness or sadness, we should observe greater mutual exclusion. This pattern is illustrated in Figure 1B where an individual experiencing extreme happiness would be less likely to endorse any unhappy emotion terms. By extension, individuals who experience extreme sadness would be less likely to endorse any happy emotion terms. In fact, this pattern of responding results in a bivariate response plot of happiness and sadness shown in Figure 2A where we observe greatest mutual exclusion at the extreme levels of happiness (or sadness), moderate mutual exclusion at moderate levels of happiness (or sadness), and little to no mutual exclusion at low levels of happiness (or sadness). Therefore, strongest evidence against the bipolar perspective would occur if substantial endorsements of both extreme happiness and extreme sadness can be shown. An endorsement pattern along the bipolar valence continuum that shows a bimodal distribution of happiness and sadness so that a single point on the bipolar valence continuum fails to describe the affective state would also constitute strong evidence against the bipolar perspective. The expected pattern of responding depicted in Figure 2A differs from that implied by the view that experiences of happiness and sadness are separable so that extreme happiness and extreme sadness are allowable as shown in Figure 2B. It also differs from the mutual exclusion view of bipolarity as shown in Figure 2C where endorsement of the opposing emotion is not allowable at all.

Figure 2.

Expected patterns of positive and negative emotions in a bivariate space.

Fourth, the expected patterns proposed are idealized for simplicity of presentation. In Figure 1, we illustrated a perfectly symmetrical item endorsement distribution although it is known that the happiness scores of individuals tend to be skewed so that most people are happy (Diener & Diener, 1996). The main idea is that the endorsement pattern would be unimodal and not necessarily symmetrical. Hence, individuals may display endorsements of high happiness and endorsements of low unhappiness; however, we would expect the endorsement pattern to peak somewhere along the continuum and decline on both sides. In the same manner, we are not proposing that the bivariate endorsement patterns should fall below an exact diagonal line. Rather, we are stating that most of these bivariate endorsements fall closer to the lower left of the bivariate plot because individuals use the proposed item response process common to self-reports. In other words, the proposed item response process constrains coendorsements so they occur close to the valence state of individuals. If affect valence is a bipolar continuum, then the coendorsement patterns would occur close to the lower left of the bivariate plot. Strongest evidence against the bipolar view would require coendorsements on the top right of the bivariate plot (and no coendorsements on the lower left) so that the proposed item response process cannot account for such coendorsements as two locations would be needed to describe individuals for the bipolar continuum. This corresponds to a strong dual-peaked distribution of item endorsements on the bipolar continuum.

Altogether, the proposed model of item responding would mean that bipolarity should not be expressed as mutual exclusion. Past evidence showing a lack of mutual exclusion may not violate bipolarity. Rather, evidence demonstrating that extreme happiness and extreme sadness co-occur at the same moment would constitute the strongest evidence against bipolarity. This proposed model of item responding reorients our understanding of bipolarity so that a single location on a bipolar continuum can give rise to endorsements of happiness and sadness.

Empirical Evidence

Because the proposed item response process underlies the expectation that coendorsements of happiness and sadness can occur on a bipolar continuum, it is important to evaluate the evidence on item responding. In this section, we review psychometric evidence demonstrating that the proposed pattern of responding holds in self-reports of emotions in both general and experimental settings. We then show how prior evidence used to infer a bivariate view of happiness and sadness do not find endorsements of intense happiness and sadness co-occurring. As such, past evidence does not contradict—and can likely be reconciled with—the current proposal that item responding leads to the lack of mutual exclusion although individuals have only a single state on the bipolar continuum.

Evidence for Item Response Process

The proposed item response process in which individuals tend to endorse scale response items most descriptive of themselves (or closest to their current level of the relevant attribute or state) along a latent continuum (as depicted in Figure 1) occurs across a wide range of self-reported attributes. Psychometric research has shown that this pattern of responding has been found in self-reports of different constructs including attitudes (Roberts, Laughlin, & Wedell, 1999), personality (Chernyshenko, Stark, Chan, Drasgow, & Williams, 2001), and interests (L. Tay, Drasgow, Rounds, & Williams, 2009). In fact, more than 80 years ago Thurstone (1928) proposed that our measurement models need to account for this type of item responding—but this idea has been neglected and only more recently resurfaced (L. Tay & Drasgow, 2012).

Not surprisingly, the proposed item response process of endorsing scale items close to the current level of an individual attribute or state also occurs in self-reports of emotions. In fact, early work demonstrated that this type of item response process in self-reports of emotions leads to an incorrect inference that two dimensions underlie the happiness and sadness bipolar continuum (Spector et al., 1997; van Schuur & Kiers, 1994). Specifically, items such as “slightly/moderately sad” and “slightly/moderately happy” in the middle of the bipolar continuum tend to be coendorsed by a sizeable proportion of individuals in the middle of the continuum. As a result, items in the middle of the continuum are more associated with each other than can be accounted for by a single dimension. Indeed, it has been mathematically shown that two orthogonal dimensions can emerge from a single bipolar continuum due to the way individuals respond to items (Davison, 1977).

More recently, it has been directly shown that individuals exhibit the item response patterns characteristic of the proposed item response process (as expected in Figures 2 and 3A). In a study consisting of 412 participants, emotion indicators along a bipolar continuum (yes/no) ranging from “extremely sad” to “extremely happy” were presented (S. C. Tay, 2011). Participants were asked to endorse emotion indicators that reflected their current feelings. In addition, individuals also responded to Likert scale PANAS-X measures of “happy” and “sad” on a 1 (“very slightly or not at all”) to 5 (“extremely”) scale (Watson & Clark, 1994), and responses were used as a proxy for where individuals subjectively located themselves on a bipolar continuum. It was expected that individuals who rated themselves as “very slightly or not at all” happy on the PANAS-X measure would endorse a range of mixed feelings but individuals who rated themselves as “extremely” happy were expected to show mutual exclusion. Indeed, this pattern was found as shown in Figure 3: The peak proportion of endorsements moved from the middle of the continuum to the end as individuals had more intense feelings of happiness on the PANAS-X items; the same was found for sadness (see Appendix, available Online). Moreover, the bivariate plots were consistent with that in Figure 2A in that coendorsement of happiness and sadness items occurred for low to moderate levels of mixed feelings and little to no coendorsements occurred for extreme happiness or sadness as depicted in Figure 4.

Figure 3.

Proportion of emotion item endorsements at differing levels of happiness.

Figure 4.

Bivariate plot showing coendorsements between sadness and happiness.

A subsequent study with 224 participants using more emotion indicators from “extremely sad” to “extremely happy” replicated this finding (see Appendix, available Online; S. C. Tay, 2011). Further, it was also shown that participants who rated themselves as feeling “ambivalent” or “not feeling anything in particular” showed endorsement patterns that had a single peak distribution along the proposed bipolar continuum, implying that a single point on a bipolar continuum sufficiently captures “ambivalence” as well as “not feeling anything in particular.” Further, there were little to no coendorsements of extreme emotions even for individuals who indicated they felt “ambivalent.” These results lend validity to the proposed item response process.

The past evidence reviewed demonstrates that the proposed item response model holds in general settings. However, a stronger test would examine whether the proposed item response process is violated in mood induction protocols where both happiness and sadness are evoked (i.e., mixed feelings). Specifically, the co-occurrence of extreme happiness and sadness under a mixed-feelings mood induction would falsify the bipolarity hypothesis. Alternatively, if bipolarity holds, a mixed-feelings mood induction would move individuals toward the middle of the continuum resulting in more reports of happiness and sadness at a low to moderate level, due to the proposed item response process. Further, we would expect to see a unimodal distribution of indicator endorsements along the bipolar continuum rather than a bimodal distribution, as a bimodal distribution would indicate that a single location is insufficient to capture the valence state of individuals.

In an experimental study, 165 undergraduate students were randomly assigned to a mixed-feelings condition or a control condition (Kuykendall & Tay, 2014). The mixed-feelings condition was a 19-minute film clip from Life is Beautiful featuring a combination of scenes designed by Larsen and colleagues to produce mixed feelings of happiness and sadness (see description in Larsen & McGraw, 2011). The control condition, also designed by Larsen and colleagues, was a 19-minute film clip from Life is Beautiful. This clip was successfully used in previous studies to evoke sadness (Larsen & McGraw, 2011).

We present the distribution of endorsement of the emotion indicators before and after the mood inductions. Because the premood induction distributions were the same across both conditions, these distributions are combined and displayed in the top panel of Figure 5. We see that before the mood inductions, most individuals are a little to moderately happy. After the mixed-feelings induction, we found that the percentage of coendorsements increased, as displayed in the second panel of Figure 5. Importantly, however, the coendorsements were primarily occurring close to lower levels of little to moderate happiness and sadness but not at extreme levels. Moreover, we did not observe a bimodal distribution that would indicate that a single location is insufficient to capture the valence state of individuals. Therefore there was no strong evidence against bipolarity given the proposed item response process. As a comparison, the control condition used to evoke sadness resulted in a stronger leftward shift in emotion endorsement as displayed in the bottom panel of Figure 5. This indicates that the sadness condition moved individuals further toward sadness and led to fewer reports of mixed feelings whereas the mixed-feelings induction moved individuals toward the middle of the bipolar continuum leading to more reports of mixed feelings; both findings are consistent with the proposed item response process. In other words, a single point on a bipolar continuum can adequately capture movements along the valence continuum, corresponding to the bipolar proposition but the proposed item response process led to more reports of mixed feelings when the affective state of individuals was close to the middle of the bipolar continuum.

Figure 5.

Distribution of endorsement of the emotion indicators before and after mood inductions.

Nonconflicting Past Evidence

In the previous section, we emphasized evidence for how, given a bipolar continuum, the proposed item response process can lead to a lack of mutual exclusion. In this section, we discuss how studies that have supported a bivariate perspective of happiness and sadness do not actually conflict with the bipolar hypothesis, given the proposed item response process. First, the same general pattern of responding we propose was observed in prior research. Diener and Iran-Nejad (1986) showed in two studies—through the use of an experimental design manipulating positive or negative mood and through experience sampling across 6 weeks—that when extreme emotions are felt, there is mutual exclusion. However, when either a positive or negative emotion is felt at low intensity, the opposing emotion can be at any level. Therefore, individuals across multiple situations tended not to experience strong positive and negative emotions and did so only when there were low levels of an opposing emotion.

Second, in mood inductions or in settings that are believed to evoke mixed feelings, the observed mixed feelings do not reflect both intense happiness and sadness. Research by Larsen, McGraw, Mellers, and Cacioppo (2004) demonstrated that after watching Life is Beautiful, sadness and happiness ratings were close to or below the midpoint of the scale indicating mixed emotional experiences are on average toward the low end (i.e., slightly to moderate). In naturalistic settings in which respondents report mixed feelings such as on graduation day, the average ratings of happiness and sadness were elevated, but ratings of sadness were low, and sadness averaged 2.07 out of a 0 to 7 scale. Further, when presented with an evocative mixed-feelings situation of disappointing wins or relieving losses, individuals displayed endorsements of mixed emotions (Larsen et al., 2004). However, consistent with the proposed item response process, as the intensity of negative feelings increased positive feelings waned; the strongest occurrence of mixed feelings were lower than the midpoint of the scale (< 3 on a 1 to 5 scale).

Third, when mixed emotions are assessed using simultaneous key presses (Larsen & Green, 2013; Larsen & McGraw, 2011; Larsen, McGraw, & Cacioppo, 2001), the intensity of these emotions is unknown as individuals are simply asked to press buttons corresponding to happiness and sadness if they felt happy or sad, respectively. If individuals were close to the middle of the bipolar continuum, individuals could endorse both emotions simultaneously for any period of time, but it does not necessarily mean that intense positive and negative emotions were felt. Moreover, it has been acknowledged that in the key press study (Larsen & McGraw, 2011), median coendorsements only occur 24 seconds out of a 20-minute clip (Larsen & McGraw, 2014), which is only 2% of the total time. This could be captured by our proposed response process (or by measurement error).

Discussion

The controversy of whether happiness and sadness are bipolar or bivariate has been based on whether endorsements of happiness and sadness exhibit mutual exclusion or not. However, we propose that the simple criterion of mutual exclusion is inappropriate. Instead, we show that the typical way individuals make self-reports (i.e., their item response process) can lead to coendorsements of happiness and sadness despite a single location on a bipolar continuum. Therefore it is critical that we reconsider the use of a lack of mutual exclusion to infer a bivariate perspective.

By accounting for the item response process, we can provide a more integrative and parsimonious account of the “conflicting” evidence found between the bivariate and bipolar perspectives. The bivariate perspective emphasizes the lack of mutual exclusion whereas the bipolar perspective emphasizes the common occurrence of mutual exclusion. We show here that the process of item responding on a single bipolar continuum can explain why responses to happiness and sadness items are not always mutually exclusive (when individuals are close to the middle of a bipolar continuum); however, it also explains why happiness and sadness often reveal mutual exclusion as well (when many individuals are experiencing more intense feelings of happiness or sadness). As it stands, the bivariate perspective allows for all patterns of bivariate responding (e.g., high happiness and high sadness); however, it does not explain why there is frequently mutual exclusion, nor does it explain why happiness and sadness are inversely related. Moreover, past empirical evidence has not shown both high happiness and sadness and so the bivariate perspective model may be overextending on the current empirical data as shown in Figure 1B. On the other hand, the bipolar perspective explains why happiness and sadness are inversely related but does not explain the lack of mutual exclusion. A consideration of the proposed item response process shows that a lack of mutual exclusion in endorsements can still be consistent with bipolarity.

The proposed response process serves as a methodological lens that applies not only to the issue of mutual exclusion in affect, but also serves as an additional factor to explain why the correlation between happiness and sadness does not display a perfect inverse relation apart from acquiescence and measurement error (Green et al., 1993). This is because when there is lack of mutual exclusion, the inverse correlation is naturally lower. Moreover, this issue extends to the bipolarity of other constructs such as attitudes, which has been argued to be better construed as bivariate (Cacioppo & Bernston, 1994). While some have argued that low negative correlations between pro-Black and anti-Black attitudes reveal attitudinal ambivalence (Katz & Hass, 1988), these findings may be a function of the self-report response process proposed here, apart from acquiescence and measurement error (Green et al., 1993). As such, the methodological-substantive contribution of the proposed response process goes beyond mutual exclusion of happiness and sadness—it is a key factor that needs to be considered more generally (cf. Greenwald, 2012).

Further Ways to Address the Controversy

How can we address the bipolarity/bivariate controversy down the road? Because the item response process can give rise to endorsements of positivity and negativity, future research addressing this controversy needs to go beyond mutual exclusion. As we have shown here, evidence against bipolarity would take the form of extreme happiness and sadness. To demonstrate this, it will be important to find stimuli or situations that can evoke nonnegligible occurrence of extreme happiness and sadness. However, such manipulations may be difficult to implement due to ethical concerns with extreme mood manipulations. Another possible piece of evidence against bipolarity would be to demonstrate conditions in which we find bimodal distributions along a bipolar continuum. For example, if we can elicit moderate levels of happiness and sadness, but not low levels of happiness or sadness, this dual-peaked distribution would indicate that a single location on a bipolar continuum would not be sufficient to describe the affective state. Dual locations on a bipolar continuum would indicate that a bivariate space with a single location may better represent valence.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Barrett

L. F.

Bliss-Moreau

(2009). Affect as a psychological primitive. Advances in Experimental Social Psychology, 41, 167–218.

Barrett

L. F.

Russell

J. A.

(1998). Independence and bipolarity in the structure of current affect. Journal of Personality and Social Psychology, 74(4), 967–984.

Bentler

P. M.

(1969). Semantic space is (approximately) bipolar. The Journal of Psychology, 71, 33–40.

Cacioppo

J. T.

Bernston

G. G.

(1994). Relationship between attitudes and evaluative space: A critical review, with emphasis on the separability of positive and negative substrates. Psychological Bulletin, 115, 401–423.

Cacioppo

J. T.

Gardner

W. L.

(1999). Emotion. Annual Review of Psychology, 50, 191–214.

Chernyshenko

O. S.

Stark

Chan

K. Y.

Drasgow

Williams

(2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523–562. doi:10.1207/S15327906MBR3604_03

Davison

M. L.

(1977). On a metric, unidimensional unfolding model for attitudinal and developmental data. Psychometrika, 42, 523–548.

Diener

(1996). Most people are happy. Psychological Science, 7, 181–185.

Diener

Iran-Nejad

(1986). The relationship in experience between various types of affect. Journal of Personality and Social Psychology, 50, 1031–1038.

10.

Green

D. P.

Goldman

S. L.

Salovey

(1993). Measurement error masks bipolarity in affect ratings. Journal of Personality and Social Psychology, 64, 1029–1041.

11.

Greenwald

A. G.

(2012). There is nothing so theoretical as a good method. Perspectives on Psychological Science, 7, 99–108.

12.

Katz

Hass

R. G.

(1988). Racial ambivalence and American value conflict: Correlational and priming studies of dual cognitive structures. Journal of Personality and Social Psychology, 55, 893–905.

13.

Kuykendall

Tay

(2014). Happiness and sadness as opposites: Evidence for an updated view of bipolarity using a mixed feelings mood induction. Unpublished manuscript, Department of Psychological Sciences, Purdue University, West Lafayette, IN.

14.

Larsen

J. T.

Green

J. D.

(2013). Evidence for mixed feelings of happiness and sadness from brief moments in time. Cognition and Emotion, 27, 1469–1477.

15.

Larsen

J. T.

McGraw

A. P.

(2011). Further evidence for mixed emotions. Journal of Personality and Social Psychology, 100, 1095–1110. doi:10.1037/a0021846

16.

Larsen

J. T.

McGraw

A. P.

(2014). The case for mixed emotions. Social and Personality Psychology Compass, 8, 263–274.

17.

Larsen

J. T.

McGraw

A. P.

Cacioppo

J. T.

(2001). Can people feel happy and sad at the same time? Journal of Personality and Social Psychology, 4, 684–696.

18.

Larsen

J. T.

McGraw

A. P.

Mellers

B. A.

Cacioppo

J. T.

(2004). The agony of victory and thrill of defeats. Psychological Science, 15, 325–330.

19.

Roberts

J. S.

Laughlin

J. E.

Wedell

D. H.

(1999). Validity issues in the Likert and Thurstone approaches to attitude measurement. Educational and Psychological Measurement, 59, 211–233.

20.

Russell

J. A.

(1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178.

21.

Russell

J. A.

(2003). Core affect and the psychological construction of emotion. Psychological Review, 110(1), 145–172. doi:10.1037/0033–295x.110.1.145

22.

Russell

J. A.

Carroll

J. M.

(1999). On the bipolarity of positive and negative affect. Psychological Bulletin, 125, 3–30.

23.

Schimmack

Colcombe

S. J.

(2007). Eliciting mixed feelings with the paired-picture paradigm: A tribute to Kellog (1915). Cognition & Emotion, 21, 1546–1553.

24.

Spector

P. E.

van Katwyk

P. T.

Brannick

M. T.

Chen

P. Y.

(1997). When two factors don’t reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23, 659–677.

25.

Tay

Drasgow

(2012). Theoretical and statistical issues in the assessment of construct dimensionality: Accounting for the item response process. Organizational Research Methods, 15, 363–384.

26.

Tay

Drasgow

Rounds

Williams

(2009). Fitting measurement models to vocational interest data: Are dominance models ideal? Journal of Applied Psychology, 94, 1287–1304. doi:10.1037/a0015899

27.

Tay

S. C.

(2011). The psychometric principles of affect: Are they ideal? (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign.

28.

Thurstone

L. L.

(1928). Attitudes can be measured. American Journal of Psychology, 33, 529–554.

29.

Van Schuur

W. H.

Kiers

H. A. L

. (1994). Why factor analysis often is the incorrect model for analyzing bipolar concepts and what model to use instead. Applied Psychological Measurement, 18, 97–110.

30.

Watson

Clark

L. A.

(1994). The PANAS-X: Manual for the Positive and Negative Affect Schedule – Expanded form. Unpublished manuscript, University of Iowa.