Abstract
This article introduces the somatic transform that operationalizes the relation between affect and cognition at the psychological level of analysis by capitalizing on the relation between the cognitive-denotative and affective-connotative meaning of concepts as measured with semantic differential rating scales. Following discussion of levels of analysis, the importance of language at the psychological level, and two principles (inextricability and complementarity) summarizing the relation between affect and cognition that are rendered explicit by the somatic transform, we present affect control theory (ACT) and its Bayesian extension (BayesACT) containing the somatic transform. We conclude by identifying examples of inextricability and complementarity in the social science and neuroscience literatures and discussing how our psychological model might be implemented in a realistic neural model.
Keywords
The relation between cognition and affect has been a long-standing and often contentious issue in both the biological and social sciences, as exemplified by special issues of journals devoted to the topic, Cognition and Emotion in 2007 (Eder et al., 2007a) and Current Opinion in Behavioral Sciences in 2018 (Mather & Fanselow, 2018). Among other things, the closely related distinction between rational and emotional thinking has engendered a number of dual-process theories in the social sciences. 1 The affect–cognition divide has also contributed to fragmentation within academic disciplines, exemplified in the social sciences by the coexistence of cognitive psychology and the psychology of emotions, 2 and of cognitive sociology and the sociology of emotions. 3 In social psychology, the “cognitive–affective crossfire” has been used to describe the debate over the relative motivational power of cognitive self-consistency versus affective self-enhancement (MacKinnon, 2015; Swann et al., 1987), as well as the relative merits of identity theories emphasizing cognitive processes (Burke & Reitzes, 1991; Burke & Stets, 2009; Stryker, 1980) and those emphasizing affective processes (Heise, 1979, 2007; MacKinnon, 1994.) The cognition–affect divide also distinguishes transactional exchange theory, predicated on cognitive-rational assumptions about human motivation and social behavior, from relational exchange theory based on affective ties and commitment (Lawler et al., 2015, 2009).
Fragmentation along the affect–cognition divide has also occurred in neuroscience. Because of its traditional focus on cognitive processes, affective neuroscience (Damasio, 1994; LeDoux, 1996; Panksepp, 1998) emerged as a largely separate development; see the special issue on the emergence of affective neuroscience of Brain and Cognition (Schmidt, 2003).
Like neuroscience, artificial intelligence (AI) began with an emphasis on cognition, conceptualizing the human mind as a cognitive, information-processing system, and treating emotion as epiphenomenal noise or, at best, a functional subordinate of cognition (Johnson-Laird & Oatley, 1992; Simon, 1967). As a consequence, AI has failed to create an adequate model of the human mind, something many believe is essential to guide the successful mapping of perception and information-processing to motivated action. Over time, however, this narrow cognitive view became challenged with the advent of emotional AI or affective computing (Picard, 1997).
Despite the historical prominence and sometimes deleterious effects of the affect–cognition divide for the study of emotion, recent decades have witnessed something of a ceasefire. While maintaining some degree of distinction, most emotion scholars and researchers today accept that affect and cognition are overlapping and functionally related processes of the human mind. What is lacking, however, are more precise ways to operationalize this relationship.
This article introduces the somatic transform 4 that operationalizes the relation between affect and cognition 5 by connecting the cognitive-denotative and affective-connotative meaning of concepts as measured with semantic differential rating scales. We attend to several preliminary considerations to set the stage for introducing the somatic transform, beginning with a discussion of levels of analysis and the importance of language for analysis at the psychological level, followed by the introduction of two principles (inextricability and complementarity) that summarize the relation between affect and cognition and that are rendered explicit by the somatic transform. With these background discussions in hand, we present affect control theory (ACT) and its Bayesian extension (BayesACT), which conjoin social psychological theory with computational tools to make the relation between affect and cognition explicit by connecting the cognitive-denotative and affective-connotative meaning of concepts through the somatic transform. Following the introduction of ACT and BayesACT, we identify examples of the inextricability and complementarity of affect and cognition in the social science and neuroscience literatures to set the somatic transform of BayesACT in a larger theoretical context. We conclude with a discussion of how our psychological model might be implemented in a realistic neural model.
Levels of Analysis
We are not proposing a Cartesian duality in which mind and body, or consciousness and matter, are distinct ontological realities (Walach, 2020). 6 Rather, we maintain that mental activity is a complex system that can be analyzed at the psychological level of the mind or at the neurological level of the brain where mental activity is physically implemented or realized, and both can be useful for gaining a better understanding of human intelligence and consciousness. The fact that the affect–cognition distinction is fuzzier at the level of the physical brain does not invalidate the distinction at the phenomenological level of the mind (Duncan & Barrett, 2007; see also Barrett & Satpute, 2013). We therefore accept as a phenomenological given that humans psychologically experience a distinction between hot and cold cognitions or, alternatively, between affective states mediated or largely unmediated by cognitive processing.
There is nothing unscientific in maintaining that while the mind is ontologically grounded or realized in the brain, it has emergent properties that cannot easily be reduced to neuronal activity. However, many neuroscientists schooled in reductionism would be uncomfortable with the idea of emergence. In the words of Gazzaniga,
[N]euroscientists dislike this kind of thinking. They cling to the idea that understanding the elementary parts of the nervous system will explain how the brain does its magic to produce the psychological states we all enjoy. Letting go of this idea seems dangerous to most of them, who fear that some kind of ghost will be snuck into the brain . . . [in contrast] scientists working on other complex issues have no problem with the idea of emergence. Physicists, chemists, and biologists all know about it. (2010, pp. 1–2)
Among the most important of emergent properties is the reflective and experiential nature of consciousness, the elusive explanation of which has become known among neuroscientists as the “hard problem of consciousness” (Chalmers, 1995). How and why does our subjective or phenomenological experience arise out of the cognitive processing of auditory and visual information? Why and how do we have an inner life in which we can entertain images and thoughts, or experience emotions? And why and how do we experience ourselves as the locus of these experiences? Despite all its achievements, neuroscience has yet to provide a satisfactory and universally accepted answer.
Language
We live in a symbolic as well as a physical environment. While the medium of mental processing at the level of the physical brain is neuronal, the medium at the level of the psychological mind is largely symbolic. Without denying the importance of visual images, it is likely that mind and reflective consciousness emerged in the human species in step with the acquisition of language (Mead, 1934). Thus, approaching the study of mind and consciousness through the medium of language would appear to be the most appropriate and effective method for studying human subjectivity. 7
Following Osgood et al. (Osgood, 1969; Osgood et al., 1957, 1975), we distinguish between the cognitive-denotative and affective-connotative meaning of language, between the labeling of objects and the affective associations evoked by labeling—“the ‘feeling tones’ of concepts as part of their total meaning” (Osgood, 1969, p. 195). Both kinds of meaning are essential to human communication. “Just as it is not possible to use words without communicating affective connotation, it is also not possible to understand language based on denotation alone” (Russell, 2003, p. 1199).
Compared to cognitive-denotative meaning, affective-connotative meaning is dimensionally simple, comprising feelings of evaluation (E), potency (P), and activity (A)—EPA. This Euclidean elegance provides a portal into the dimensionally complex cognitive-denotative world of meaning. “Osgood’s evaluation, potency, and activity [are] not simply . . . dimensions of spoken and written words . . . they are the hidden language, the affective Rosetta stone that allows mind and body to communicate” (Clore & Pappas, 2007, p. 338; see also Clore & Ortony, 2000). As discussed in what follows, ACT and BayesACT employ semantic differential bipolar scales based on the EPA structure of affective-connotative meaning to measure cultural sentiments for social concepts.
EPA space describes a cognitively appraised affect handled by networks mainly in the prefrontal cortex, and this is not the same as “core affect” (Duncan & Barrett, 2007; Russell, 2003), which is handled by a different functional network in the brain and is much closer to interoceptive signals from the body (Beer, 2017). While Beer opines that few studies have “the aim of understanding the role of emotion in social cognition” (2017, p. 3), there is a large literature on the sociology of emotions that addresses precisely this issue.
The Inextricability and Complementarity of Affect and Cognition
On the basis of experimental evidence suggesting that subjects can make affective preferences among stimuli presented below the threshold of cognitive awareness, Zajonc famously concluded that “affect and cognition are separate and partially independent systems” (1984, p. 117), precipitating a heated exchange with Lazarus in the 1980s that became known at the time as the “primacy of cognition versus affect debate” (Zajonc, 1968, 1980, 1984; see also Lazarus, 1984; Zajonc, 2000).
Our position on the issue of primacy and independence of affect and cognition strikes a balance, specifying an intersection while allowing for partial independence. Moreover, we view the relation between cognition and affect as a reciprocal process, rendering the question of primacy or temporal-causal order a moot point. This same idea has been widely suggested in the literature (Forgas, 2008; Lazarus, 1984; Mook, 1987; Turner, 2009). The reciprocity of affective and cognitive processes is also a core assumption of affect control theory (ACT) and BayesACT discussed in the following lines.
This view of the relation between cognition and affect can be expressed as two principles (MacKinnon, 1994): (a) the principle of inextricability proposes that cognition and affect are overlapping constituents or simultaneous processes of the mind or consciousness rather than completely independent systems, a matter of relative preponderance where mind or consciousness at any given moment can be predominantly cognitive or predominantly affective or anywhere in between; and (b) the principle of complementarity proposes that, as only partially independent constituents or processes, both cognition and affect are necessary for an adequate understanding of the human mind. While the principle of inextricability is an ontological statement about the reality of the human mind as currently understood, the principle of complementarity is an epistemological implication of this ontological view.
At its most general level, the principle of complementarity proposes that what appear to be mutually exclusive phenomena are manifestations of some other underlying phenomenon, and that both expressions are necessary to understand it. The physicist Niels Bohr (1950) advanced the principle in the 1920s to explain the contradictory images evoked by the wave–particle duality of subatomic phenomena. The psychologist William James (1890) had developed the principle much earlier to reconcile, in his terms, the substantive and transitive parts of thought—the semantic denotations and syntactic connections of language with its inflections and affective overtones. And since both are essential components of human thought or consciousness, the principle of complementarity must be invoked in psychology just as it has been in quantum physics (Stephenson, 1986a, 1986b).
Because inextricability is a continuum rather than a dichotomy, one can distinguish between “harder” and “softer” views of the principle. Those adopting a harder view sometimes dismiss the distinction between cognition and affect altogether, even at the phenomenological level of the mind (Eder et al., 2007b). Those adopting a softer view, as we do, emphasize an overlap of cognitive and affective processing in the human mind while allowing that these processes may be partially distinct and independent systems (e.g., Storbeck & Clore, 2007). 8
The inextricability and complementarity of affect and cognition are foundational assumptions of ACT and BayesACT. The cognitive-denotative and affective-connotative meanings of concepts are inextricable because one can be recovered from the other. At the same time, the two are complementary because they refer to the same deeper underlying reality and both are necessary to fully understand this deeper reality. And as expounded next, the somatic transform introduced in this article represents inextricability and complementarity explicitly in a simple energy potential connecting the cognitive-denotative and affective-connotative representations within the framework of Bayesian affect control theory.
Following an introduction to ACT and BayesACT, in the next two sections we identify examples of inextricability and complementarity in the social science and neuroscience literatures. This review locates the somatic transform of BayesACT in a larger theoretical context.
Affect Control Theory (ACT)
Affect control theory (ACT; Heise, 1979, 2007; MacKinnon, 1994, 2015; MacKinnon & Heise, 2010; Robinson & Smith-Lovin, 2018; Smith-Lovin & Heise, 1988) was developed from an integration of Osgood’s semantic differential measurement model (Osgood, 1969; Osgood et al., 1957, 1975), Gollob’s (1968) actor–behavior–object (ABO) model of impression-formation, and Powers’s (1973) perceptual control model of behavior, blending these elements from psychology with insights from action theory and symbolic interactionism from sociology.
ACT measures the affective-connotative meaning of social concepts with three bipolar semantic differential scales corresponding to the EPA structure of affective-connotative meaning established by Osgood et al. (1957) The EVALUATION scale is anchored by “bad, awful” on one end to “good, nice” on the other end; the POTENCY scale, by “small, weak, powerless” to “big, strong, powerful”; and the ACTIVITY scale, by “slow, old, quiet” to “fast, young, noisy.” Each scale ranges from infinitely “bad, awful” (−4.3) to infinitely “good, nice” (+4.3), with actual values generally falling between -3 and +3, and where a +2 or -2 is considered a large (positive or negative) value. In ACT, individual scores on EPA scales are averaged to estimate the cultural sentiments of concepts.
Because EPA scales serve as generalized attitude scales (Osgood et al., 1957), they allow for the measurement of all kinds of social objects—social identities, interpersonal behaviors, social settings, social characteristics, personality traits, and emotions—on a single, common metric. Cultural surveys 9 of EPA semantic differential ratings of social objects yield a set of samples from a population distribution in sentiment space, which can be estimated parametrically (e.g., as the mean and variance of a normal distribution). Lists of mean EPA ratings of social objects mapping cognitive-denotative labels to cultural sentiments are referred to as “dictionaries” in ACT and have been constructed for Canada, China, Germany, Japan, Morocco, Northern Ireland, and the United States. 10
Moreover, EPA scales provide a mathematically coherent metric, enabling the transformation of one type of phenomenon into another employing ACT models programmed in Interact, 11 a computer program for simulating social interactional events and their affective and cognitive outcomes. These models implement three fundamental principles of ACT: (a) affective reaction, (b) affect control, and (c) reconstruction.
The affective reaction principle proposes that people respond affectively to events, experiencing transient impressions or feelings for the actors, behaviors, and object-persons involved in them. Transient impressions are estimated by empirically derived impression-formation equations. The equation for actor evaluation, for example, reveals that good acts generate transient impressions of morality for the actor; powerful acts generate impressions of potency; and lively acts, impressions of liveliness. Being the recipient of a negatively evaluated action makes the object-person in an event decline in evaluation, and so on.
The affect control principle proposes that people construct and interpret events to minimize the discrepancy (called deflection in ACT) between internalized cultural sentiments and the transient feelings produced by events or, equivalently, to confirm cultural sentiments for the identities and actions involved in them. Thus, while the affective reaction principle pertains to impression-formation, the affect control principle applies to impression-management.
The reconstruction principle proposes that when people cannot minimize deflection through restorative actions, they attempt do so by reidentifying event participants by labeling them or by attributing explanatory traits and affective moods. Attributions are predicted from attribution equations, which are derived from empirical EPA ratings of identities modified by traits, moods, and other attributions.
Finally, ACT contains a theory of emotions, which proposes that emotions signal how an interaction is unfolding with respect to the identity-confirming or disconfirming impressions created for its participants. Empirically derived emotion equations in ACT reveal that emotions are a product of transient impressions and their deflection from fundamental cultural sentiments, thus serving as an “error signal” of sorts that can be communicated among participants to ensure smooth interaction through the selection and confirmation of situationally appropriate identities.
ACT unifies cognitive and affective processing by establishing a fundamental connection between the cognitive-denotative representations of the social environment and the affective-connotative representations of the sentiments or feelings with which they are associated. For example, when one perceives a person in a white coat in a hospital, a cognitive-denotative impression of this person may be formed and represented with the symbol “doctor,” which has an associated cultural sentiment in three-dimensional EPA affective space. A doctor, for example, usually evokes feelings of goodness, strength, and modest activity, which is reflected in the EPA profile for this identity EPA profile (E: 2.7; P: 3.0; A: 0.23). 12 And given any affective-connotative (EPA) vector, a cognitive-denotative label can be assigned using a simple nearest neighbour method. For example, the closest label to the numerical EPA profile (−1.0; 2.0; 2.0) is “politician,” with an EPA profile of (−0.9; 2.3; 1.5) at a Euclidean distance of 0.59 from this numerical EPA profile.
As stated before, EPA ratings yield a set of samples from a distribution in the sentiment space that represents the cultural group being measured. This distribution can be estimated parametrically (e.g., as the mean and variance of a normal distribution). However, ACT employs only the mean of EPA ratings to link cognitive-denotative and affective-connotative meaning, and only a single identity for each participant in analysis of actor–behavior–object events. This is restrictive because, in many situations, cognitive-denotative identities may not be clear, and sentiments may be more or less precisely defined. In our hospital example, a person in a white coat could be a nurse or a doctor (but probably not a patient). A person appraising this situation may be uncertain about the identity of the white coat wearer and might conclude that the identities of doctor or nurse are equally likely. Furthermore, while doctors are seen as more powerful than nurses, there may be many cultural experiences of powerful nurses and weak doctors, and cultural sentiment surveys may reveal this ambiguity. For example, the variance in the power ratings for a doctor is 1.4, while for nurse, with an EPA profile (2.9; 1.9; 0.7), the variance is significantly higher (2.4), indicating greater cultural uncertainty about the affective meaning of a nurse on the power dimension. A person observing the white coat wearer doing something powerful such as ordering someone to do something might shift her beliefs about the identity of this person, concluding perhaps that this person is much more likely to be a doctor than a nurse. The amount by which this belief is shifted will depend on the ambiguity of the cultural sentiments for the identities. For example, if doctors were more powerful in the mean, but with a very large variance, then an observation of a powerful behavior would decrease the shift, so the person might conclude that the white coat wearer is only a bit more likely to be a doctor. These deficits in ACT are addressed by BayesACT, to which we now turn.
BayesACT and the Somatic Transform
BayesACT 13 (Hoey et al., 2016; Schröder et al., 2016) generalizes ACT by explicitly representing the distribution over a sentiment, taking the variance in sentiment measurements into account and releasing ACT from the constraints of a single identity and a mean sentiment. BayesACT allows for the explicit representation of the connection between denotative and connotative representations through a potential function we label “the somatic potential.” 14 The original BayesACT model presented in Hoey et al. (2016) did not include this function. We introduce the somatic potential in this article and show how it can be used as a simple, expressive model of the principles of inextricability and complementarity applied to the relation between affect and cognition, naturally allowing for the modelling of multiple identities and of ambiguous sentiments.
The somatic potential is defined in terms of an energy 15 function that measures the incoherence (difference) between the denotative state (e.g., doctor) and the connotative state (a distribution in the affective EPA space). Denoting denotative states with x and connotative states with y, we write this energy function 16 as follows:
where M(x) gives the mean of the connotative distribution for a particular x, and where the parameter γ is the (square root of the) variance in sentiments in the population.
The energy function can be turned into a probability distribution, G(x, y) called the somatic transform that can be used to rank the likelihood of different connotative and/or denotative events:
where c is a normalization constant. 17 Thus, the more coherent x is with y, the closer M(x) is to y, the smaller the energy, and the higher the probability, with “closer,” “smaller,” and “higher” all characterized by the variance parameter γ. That is, if γ is larger, then the same amount of coherence is achieved when M(x) is further from y.
The somatic transform can be used to translate between connotative and denotative states in either direction. For some x, G(x, y) is a function over the connotative states representing the sentiments for denotative state x. For example, G(x, y) for each of three identities in a hospital setting (nurse, doctor, and patient) is a normal distribution 18 given by summary statistics—mean M(x), variance γ2—for each identity, as shown in Table 1. Plotting these as probability density functions over the connotative space of power (P) only, we obtain the visualization shown in Figure 1. Figure 1 shows that the identity of a patient is significantly less powerful than either nurse or doctor, and that the distribution for nurse is significantly more spread out, indicating more ambiguity in the cultural sentiments for nurse than for either of the other two identities.
Mean and variance of sentiments for three identities.
Note. E = evaluation; P = potency; A = activity.
Plot of the normal connotative densities in y = power for three identities as estimated from the Georgia 2014 survey (Smith-Lovin et al., 2016).
The somatic transform can also be used to map from connotative to denotative states. For ome set of denotative labels, a probability distribution over these can be formed for each y by normalizing G(x, y) so it sums to 1.0 over the denotative labels. For example, plugging y = 2.0 into Equation 2 using the variances γ from Table 1, we find the unnormalized probability densities for each of the three hospital identities considered before, as shown in the second column of Table 2. A normalization (division by the sum of the three numbers) yields a probability distribution over these three identities, as shown in the third column of Table 2.
Unnormalized somatic transformed values for power sentiment y = 2.0 for each of three identities and normalized probability distribution over those three identities.
Therefore, suppose a person in a hospital was appraising an unknown individual based on sentiments or feelings about her/his power. Referring to the previous example, the person may have observed this individual ordering someone to do something. Starting from only a sentiment of power (P = 2.0), the likely identities can be ranked probabilistically by the somatic transform such that the identity of patient is all but ruled out, while those of nurse and doctor are roughly equivalent, with nurse slightly outranking doctor.
To investigate further, we can compute this probability distribution for every value of y, as shown in Figure 2. As the connotative power value in y moves from low to high, the most likely identity (top curve in Figure 2) changes from patient (for y < 0.1) to nurse (between 0.1 < y < 2.1) to doctor (y > 2.1).
Probabilities over three identities as a function of the connotative value of power, y, as estimated through the somatic transform.
To recap our hospital example, a person may appraise a situation in terms of identities based on stereotypes and other learned meanings of cultural artifacts (e.g., gender, dress). These appraisals take the form of denotative representations (e.g., identity labels, “doctor” or “nurse”). Appraisals may also be in terms of sentiments (power, evaluation, or activity). The somatic transform allows for these elements to be combined in a probabilistic (Bayesian) way such that the resulting appraisals take both denotative and connotative meanings of identities and their variances into account. The somatic transform explicitly represents this inextricability and complementarity in a simple energy potential connecting the two kinds of representations within the framework of Bayesian affect control theory.
Inextricability and Complementarity in the Social Science and Neuroscience Literatures
As discussed before, the principles of inextricability and complementarity are foundational to ACT and BayesACT, and are rendered explicit by the somatic transform of BayesACT. In this section, we identify instances of these principles (explicit or implicit) in the social science and neuroscience literatures. This review sets the somatic transform in a more expansive theoretical context.
Social Science
Social scientists tend to adopt a softer position on inextricability, emphasizing the overlap of cognition and affect while allowing for a partial independence.
The inextricability of cognition and affect is embedded in the fundamental premise of cognitive appraisal theories of emotion, that cognitive processing is a necessary precondition for, or concomitant of, some important aspects of emotional experience. Although this theory has deep historical roots (Arnold, 1960; Lazarus, 1991; Mandler, 1975; Neisser, 1967), we restrict our discussion to the work of Clore, Ortony, and associates because of its affinity with ACT (Clore & Pappas, 2007).
In their earlier research on the “referential structure of the affective lexicon,” Ortony et al. (1987, p. 351) concluded that “Mental conditions always have a significant cognitive component or a significant affective component, and sometimes both.” This led them to differentiate between terms that have affect as their predominant referential focus (e.g., broken-hearted, thrilled) from other words that have both significant affective and cognitive components (e.g., confident, uninterested). And in a related study, Clore et al. (1987) found that people have difficulty in distinguishing pure affective states, such as angry and afraid, from intense cognitive ones, such as astonished and bewildered.
Inextricability is also a central theme in subsequent work by Clore, Ortony, and associates. In the case of specific emotions (e.g., angry, happy), Clore and Ortony (2000) assert that cognition is always implicated in emotional experience, because emotions always attach to objects, which are “necessarily represented, and representation is the essence of cognition” (2000, p. 53). In the case of undifferentiated affective states such as moods, Storbeck and Clore (2007) challenge the primacy, independence, and automaticity arguments for their alleged independence from cognition.
While emphasizing the effect of cognition on affect, Clore and associates also acknowledge the reciprocal effect of affect on cognition, “modulating and mediating” various cognitive processes (Storbeck & Clore, 2007, p. 1230). According to affect-as-information theory (Clore et al., 2018; Clore & Schnall, 2005; Schwarz & Clore, 1983), undifferentiated affective states such as moods play an important role in judgment and misattribution effects.
In addition to inextricability, the principle of complementarity is explicit in the writings of Clore, Ortony, and associates, as evidenced in their acknowledgement that “cognition and emotion are coming to be viewed as complementary rather than antagonistic processes” (Storbeck & Clore, 2007, p. 1225).
Like cognitive appraisal theory, psychological constructionist theory (Barrett, 2006, 2017a, 2017b; Barrett & Russell, 2015; Russell, 2003; Russell & Barrett, 1999) proposes that emotions emerge from a process of cognitive interpretation. While cognitive appraisal theory focuses on the interpretation of an external situation, however, psychological constructionist theory focuses on the interpretation of an internal sensory or affective state in making causal attributions to an external situation (Barrett, 2009; Gendron & Barrett, 2009).
The idea of inextricability is contained in the basic premise of the theory, according to which the specific emotions (happy, sad, etc.) are psychologically constructed from a combination of more general cognitive and affective primitives that are also ingredients of cognitions. The first primitive is “core affect”—that “neurophysiological state consciously accessible as the simplest raw (nonreflective) feelings evident in moods and emotions” (Russell, 2003, p. 148). Core affect is experienced as an ephemeral-to-enduring, free-floating affective event, an “affective substrate” (Duncan & Barrett, 2007, p. 1186) available to consciousness that is a precondition for first-person or phenomenological experience of the world. 19 Cognitive processes such as labeling and attribution conjoin with core affect in the psychological construction of discrete emotions. 20
Psychological constructionist theory contains a harder view of inextricability for the neuronal level of the brain and a softer view for the psychological level of the mind. This is evident in the assertion that while the brain may not “respect” the distinction between cognition and affect, the distinction may be valid at the phenomenological level of the mind (Duncan & Barrett, 2007). The principle of complementarity is explicit in acknowledging that, at this level, “thinking and feeling . . . are actually two sides of the same coin” (Duncan & Barrett, 2007, p. 1202).
The ideas of inextricability and complementarity can also be found in sociological theories of emotion. For example, Franks (1989, 2006) adopts a harder version of inextricability for the neurological level of the brain and a softer version for the phenomenological level of the mind. While asserting that “Emotions are involved in many areas of the human brain and are tightly interwoven with structures of cognition, memory, and motivation” (2006, p. 51), Franks dismisses the affect–cognition divide at the level of the mind as “a fallacy of dualistic contrasts” (2006, p. 55). “Rather than thinking categorically,” he argues, “it may be wiser to see emotion and cognition on a continuum with a very large middle ground” (2006, p. 60). And both inextricability and complementarity are contained in Franks’s statement that a satisfactory resolution to the cognition–emotion issue “will depend on describing how they can be inextricably linked [inextricability] while capable of being in tension [complementarity]” (Franks, 2006, p. 55).
Neuroscience
Neuroscientists tend to adopt a harder version of inextricability, minimizing the distinction between cognition and affect, and emphasizing their overlap or functional integration in the brain. For example, Davidson (2003) includes in his list of “seven sins” to avoid in the study of emotion the assumptions that affect and cognition involve independent and separate neural circuitry, and that affect is mostly subcortical and cognition is mostly cortical. Commenting on the use of neuroimaging technology such as fMRI to locate cognitive and emotional processes in specific areas of the brain, Cacioppo et al. (2003) caution against equating “the beauty of a brain image” with “its psychological significance” (2003, p. 657), calling the idea of a close mapping of cognitive phenomena and underlying neural substrates a “category error” in cognitive neuroscience. And, more recently, Barrett and Satpute (2013) challenged the “faculty psychology” assumption that “emotional, social, and cognitive phenomena are realized in the operations of separate brain regions or brain networks” (2013, p. 1), proposing instead that findings from neuroimaging studies can be more effectively analyzed using a framework based on large-scale intrinsic networks—domain-general, functional information-processing networks—each of which comprises a number of nodes associated with more specific functions.
Other neuroscientists adopt a softer view of inextricability, acknowledging an intersection of cognition and affect without rejecting their distinction. This can be found in Damasio’s theory of somatic markers (1994, 2003), which conjoins the bodily expression of emotion with the cognitive mechanisms of attention and working memory associated with the prefrontal cortices (for a recent update, see Poppa & Bechara, 2018). For Damasio, “emotion is played out under the control of both subcortical and neocortical structures” and “feelings are just as cognitive as any other perceptual image, and just as dependent on cerebral-cortex processing as any other image” (1994, p. 159).
A softer view of inextricability is explicit in LeDoux’s often quoted assertion “that emotion and cognition are best thought of as separate but interacting [emphasis added] mental functions mediated by separate but interacting [emphasis added] brain systems” (1996, p. 69). The basis for distinguishing between cognitive and emotional states of consciousness, LeDoux argues,
is not the system that represents the conscious content . . . but the systems that provide the inputs to the system of awareness. There is but one mechanism of consciousness and it can be occupied by mundane facts or highly charged emotions. (1996, p. 19)
This early proposition is echoed in the later development of a higher order theory of emotions (LeDoux & Brown, 2017; LeDoux & Hofmann, 2018), which proposes that “a general network of cognitions [GNC] underlies both cognitive and emotional states of consciousness” and that what distinguishes them is “the kind of inputs processed” (LeDoux & Hofmann, 2018, p. 69). 21
Pessoa (2008, 2009) focuses on the interaction between emotion and cognitive processing made possible by the compact and extensive connectivity of neuronal networks in the brain. The idea of inextricability is explicit in his rejection of a distinction between cognition and affect based on cortical and subcortical brain areas. Because interaction logically implies some degree of distinction between cognition and affect, Pessoa’s view of inextricability is of a qualified or softer nature. In later work, however, he suggests that we move beyond a focus on the interaction between cognition and emotion to understanding their integration in the brain, representing emotions in the brain as “functionally integrated systems that involve large-scale cortical-subcortical net-works that are sensitive to bodily signals” (Pessoa, 2018, p. 19; see also Pessoa, 2017).
Panksepp’s view on the role of cognition in emotional experience differs significantly from that of other affective neuroscientists such as Damasio, LeDoux, and Pessoa. Based on extensive research with animal models, he argues forcibly for the evolutionary and functional primacy of affect in the mammalian (including human) brain, de-emphasizing the role of cognition in both unconscious and conscious affective experience. In doing so, however, he tries to walk a fine line between these two levels of consciousness. On the one hand, he holds that much of what transpires in the brain is automatic and unconscious, so that a substantial amount of emotional processing results from “neural networks that in themselves probably elaborate no conscious emotional feelings” (Panksepp, 2001, p. 142). On the other hand, he acknowledges the importance of conscious emotional feelings, but views them as “biological in their essential underlying form” (2001, p. 142), “emergent properties that are realized in the dynamic organizations of neuronal networks” (2001, p. 141). Thus, unlike other affective neuroscientists and cognitive psychologists, Panksepp maintains that neuronal processing in the subcortical brain is not only necessary but also sufficient to explain the conscious, subjective experience of emotional feelings, suggesting that we should explore the nature of conscious emotional feelings “subcortically where the cognitive light is dim but the affective light is bright” (2001, p. 159).
The cognitive neuroscience of emotion (hereafter CNE) proposed by Smith et al. (2018) contrasts with Panksepp’s version of affective neuroscience. While Panksepp focuses on subcortical neural processes, CNE views emotional experience as a whole-brain phenomenon (Barrett, 2017a; Pessoa, 2017), a product of large-scale, intrinsic, domain-general, information-processing networks (see our previous discussion of Barrett & Satpute, 2013). The CNE approach of Smith et al. (2018) is embodied in their three-process model of sequential emotional experience: (a) affect response generation involving coordinated automatic changes in body state and cognitive attentional state; (b) representations of situational appraisals, emotion concepts, and body states; and (c) the consciousness of representations. Here, Smith et al. (2018) identify the large-scale, distributed neural system known as the global workspace network as responsible for selecting active representations for global broadcasting throughout the brain, allowing them to become consciously accessible for deliberation and decision-making. Thus, in contrast to Panksepp, the three-stage model of Smith et al. (2018) proposes that subcortical affective processes and cortical cognitive processes are inextricably involved in all three stages of emotional experience. 22
Implementing the Somatic Transform Model as a Realistic Neural Model
We conclude this article with a suggestion of how the somatic transform might be implemented in a biologically realistic neural model, following the lead of the semantic pointer theory of emotion 23 proposed by Thagard and Schröder (2014; see also Kajic et al., 2019). This theory explains how the structural relations between the embodied features of emotion and the affective meaning of linguistic concepts represented by EPA (evaluation, potency, and activity) may be implemented in the biological neural networks of the brain, integrating the neurobiological and phenomenological levels of analysis in the study of emotion.
The connectionist networks of Thagard and Schröder (2014), among others, have the same motivation as the somatic transform: to build a formal model of the mutual influence of affect and cognition. In a connectionist network, this mutual influence is modeled as a weight vector in some population of neurons. In fact, there are biologically plausible ways of computing energy functionals of the weight vectors given a data set (e.g., by “Boltzmann machines”; Ackley et al., 1985), and minimizing an energy functional is the same as finding the posterior probability of the corresponding distribution over possible states. That is, if different neural networks representing cognition and affect are connected via distributions like the somatic transform that are characterized by a Boltzmann distribution, then it means they can be implemented in a neural network representation as in Thagard and Schröder (2014), or in a sample-based (Monte Carlo) representation as in BayesACT (Hoey et al., 2016). While the somatic transform and the conditional distributions could be implemented this way or in any of a number of other ways, such as the Emergent neural modelling system that implements some of the common types of connectionist networks (Aisa et al., 2008), the advantage of the sample-based approach used in Hoey et al. (2016) is a causal structure (a Bayesian network) that can be exploited in reasoning (finding the likely causes of events).
That is, BayesACT is a conceptual and mathematical model that is suitable for many different types of computational implementation, one of which is a Monte Carlo (stochastic simulation) method as in Hoey et al. (2016), while another uses a neural network that is biologically realistic. We are currently working on a third implementation as a nonbiologically realistic deep neural network. BayesACT and the somatic transform are theoretical models, but are computationally very difficult, requiring some form of approximation when implemented. This is a foundational problem in machine learning and artificial intelligence: the implementation of the model introduces a particular learning (model fitting) bias. Some of these implementations may be more useful in some contexts than in others. For example, one may be interested in examining how a real human brain trades off cognition and affect, and then the biologically realistic model would be most appropriate. Monte Carlo methods and deep neural networks, on the other hand, are not biologically realistic, may be more approximate but may also be much more computationally efficient, and would be more appropriate for artificial intelligence applications. Further, research on how to unify causal models such as Bayesian networks with deep neural networks is an ongoing effort (Rohekar et al., 2018).
Another advantage of the Monte Carlo methods from an artificial intelligence perspective is that they can make use of independencies encoded in the associated Bayesian network to accelerate the process of identifying the situation causally and computing strategies for action. On the other hand, it is not as clear how this causal aspect is implemented in a neural network, although there is some recent work on identifying important input features in a neural network, making them more interpretable, something considered essential in AI from an ethical perspective (Acien et al., 2019; Hooker et al., 2019; Stark & Hoey, 2021).
Summary and Conclusion
In summary, we have argued that cognition and affect are overlapping but partially independent systems of mind or consciousness. The principle of inextricability in its softer version represents this view, with the principle of complementarity as an epistemological implication. At the same time, we acknowledged that the distinction between cognition and affect becomes more problematic at the neurological level of the brain. The principle of inextricability in its harder form recognizes this reality. Although the focus of this article has been on the psychological level of analysis, we discussed these principles at the neurological level as well; and in the conclusion, we discussed how our psychological model of the somatic transform might be implemented as a realistic neural model in a variety of different ways for different contexts.
We approached the relation between cognition and affect at the symbolic level of language by capitalizing on the relation between the cognitive-denotative and affective-connotative meaning of concepts established by Osgood and associates. The somatic transform introduced in this article operationalizes this relation because it enables the mapping of cognitive-denotative and affective-connotative meaning onto one another, a mapping that is culturally shared. Connotative and denotative states are inextricable because one can be recovered from the other. At the same time, the two are complementary because they describe the same deeper underlying reality, and both are necessary to fully understand this deeper reality. Clearly, the connotative state will be of limited usefulness on its own; it needs to be translated into something concrete in the world, and, in particular, into concrete motor movements (behaviors). Perhaps less obvious, the denotative state by itself will also be of limited usefulness due to the computational difficulties it presents as environments grow more complex (FeldmanHall & Shenhav, 2019).
While important in itself, the somatic transform introduced in this article is part of a larger theory, BayesACT, a reformulation of affect control theory based on Bayesian probability theory and statistics. The affective dynamics of BayesACT take place at the representations of affective experience and conscious access levels in the three-process model of affective experience of Smith et al. (2018) discussed before, and would be involved in both subconscious and conscious cognitive appraisals. In this regard, Lane (2000) has formulated a five-level hierarchical model of emotional experience based on a continuum from unconscious to conscious processes. A Level 5 theory “yet to be formulated, would involve social cognition and would focus on how differentiated processes of self and other influences [sic] social behavior and the autonomic and neuroendocrine concomitants of emotional responses” (2000, p. 362). In view of the empirical basis of BayesACT in social cognitions and cultural sentiments, and its social psychological unit of analysis (interpersonal events), we believe that this theory approaches a Level 5 theory as defined by Lane (2000).
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
