A Tale of Two Systems

Abstract

In a Russian folktale, a scamp teaches a fool how to prepare a delicious soup that requires nothing but a soup stone. All one has to do is put the soup stone into boiling water. Yet, to achieve a better taste, it is desirable to add some vegetables, some meat to make the soup especially tasty, and some salt and pepper will do no harm, and so forth.¹ A methodical examination of the development of two-system models during the past 10 years is reminiscent of the above folktale. Indeed, inspecting the different labels proposed and the various terminologies employed to characterize the presumed two systems and their corresponding alleged processes strongly suggest that it has become a stone soup where everything goes.

Several authors (e.g., Gigerenzer & Reiger, 1996; Keren & Schul, 2009; Kruglanski & Gigerenzer, 2011; Osman, 2004) have critically assessed the viability and usefulness of the two-system approach. The recent expositions by Evans and Stanovich (2013, this issue, henceforth E&S) in this issue and elsewhere (e.g., Evans, 2009, 2011; Stanovich, 2009) suggest that they recognize many of the problems and seek to rectify them. Unfortunately, I suggest that their attempts are futile and, if anything, only increase confusion due to the addition of new theoretical constructs and the proposition that there may be more than two systems.

My commentary has two different levels. At one level, I directly address E&S’s article, particularly commenting on what they now term a two-process theory. At a higher level, I consider E&S’s new model to be yet another version of what generically is referred to as “two-system” models and ask whether these indeed constitute theories or models as usually conceived by the scientific community.

It is obviously beyond the scope of this article to provide an elaborated discussion of the nature of adequate scientific theories. Yet, it is commonly agreed that proper models should be based on well-defined constructs, motivate new questions, and yield testable predictions that will either corroborate or refute the theory under investigation. The central claim of this commentary is that none of the two-system theories, including the current version from E&S, satisfy the cardinal requirements of a scientific theory. Indeed, a previous article written in collaboration with my colleague Yaacov Schul (Keren & Schul, 2009) was meant to specify some (but certainly not all) basic requirements needed to transform two-system theories into testable ones. We realized that satisfying these requirements may be difficult, if not impossible, because these theories are so broad that they supposedly can account for almost all high-level social-cognitive phenomena. An important feature, in fact a necessary condition, for any scientific theory is a clear and unambiguous statement regarding the theory’s constraints and the corresponding boundaries. Otherwise, the theory becomes nothing but a set of arbitrary nominal definitions as the testable empirical world is not constrained. Inspection of two-system theories, including E&S’s version currently under discussion, unequivocally shows that the selective evidence they state to support their claims is, at best, consistent with the theoretical position rather than being based on direct tests derived from their theories.

E&S’s article boils down to three main points: First, they agree with many of the criticisms of two-system models, yet believe that these were overstated. Second, they point out that most of the criticisms apply to a generic model, but they assert that the various models differ on different dimensions and thus should not be treated identically. Finally, they propose a new two- process theory that they believe is invulnerable to the common criticisms.

In what follows, I briefly comment on each of these points. I close my commentary by raising two questions: First, I ask whether the current model, like its two-system predecessors, is testable in any scientific sense and probe whether the empirical evidence, brought by proponents of the theory, really supports their claims. Second, I question whether the ontological inquiry about the existence of two (or more) systems advances our psychological understanding. To facilitate the discussion, I will begin by examining each of the five criticisms alluded to by E&S.

Multiple and Vague Definitions (Criticism #1)

Basically, E&S agree that two-system researchers have often used a vague and ill-defined terminology. Consequently, after years of considering whether there are two or even three² (Evans, 2009) “systems”, and even considering the existence of several minds (e.g., Evans, 2010, 2011; Stanovich, 2009), they now claim (no doubt temporarily), that there are simply two types of processes.

E&S’s model abandons the traditional list of characteristics that presumably discern the two systems and instead employs only what E&S refer to as the defining characteristics of the two types of processes, which are working memory (WM) for Evans and cognitive decoupling and mental simulation for Stanovich. Unfortunately, these theoretical constructs are amenable to the same criticisms discussed in the past. Both of these constructs are not well defined and their ability to distinguish Type 1 and Type 2 processes remains to be proven. Let me briefly examine these two constructs according to the criteria proposed by Keren and Schul (2009).

The literature on WM indicates that it is not a unitary concept and that there is disagreement regarding its nature and functions (e.g., Baddeley, 2012; Hassin, Bargh, Engell, & McCulloch, 2009; Shah & Miyake, 1999). Although WM is one of the major constructs in their model, E&S do not commit themselves to any particular characterization of WM. Moreover, regardless of the particular WM theoretical formulation, it is agreed that reliance on WM is a matter of degree, as some processes use it more than others. This is incompatible with E&S’s suggestion to dichotomize the extent of WM reliance in order to distinguish between Type 1 and Type 2 processing. Keren and Schul’s (2009) discussion of the problems associated with dichotomization applies here.

The other defining characteristic of Type 1 processing is not less problematic. E&S propose that Type 1 processes are defined by being autonomous, which implies that Type 1 processes are shielded from Type 2 processes. This assumption is inconsistent with current research. I illustrate such inconsistencies with two phenomena that E&S mentioned as examples of Type 1 processes. Eitam, Hassin, and Schul (2008, Eitam, Schul, & Hassin, 2009) showed that implicit learning is affected by goal relevance and by implicit motivation. The work of Tamir and coworkers suggests that affect regulation is influenced by strategic considerations (e.g., Tamir & Ford, 2009 for regulation of fear; Hackenbracht & Tamir, 2010 for regulation of sadness). Both examples suggest that Type 1 processes cannot be shielded from Type 2 processes. Finally, the alignment criterion (Keren & Schul, 2009) could be used for testing whether mental simulation can be used as an attribute to define Type 2 processes. The answer is most likely negative, because simulations are central to processes typically considered to be Type 1, such as perception, categorization, and motor movements (Barsalou, 1999, 2003; Oberman & Ramachandran, 2007).

Attribute Clusters Are Not Reliably Aligned (Criticism #2)

E&S object to the ongoing claims against their model by claiming that Keren and Schul (2009) apply standards that are too high (e.g., Stanovich & Toplak, 2011). The authors’ suggestion that a deterministic level of prediction is too high a standard is a fragile argument— particularly in light of their strong assertion (which is at best controversial and most probably wrong) that the two modes are associated with different parts of the brain.

E&S agree with our observation (Keren & Schul, 2009, p. 538) that the attributes listed in their Table 1 are not aligned. In fact, they are “very concerned that casual assumptions about the attributes of Type 1 and 2 thinking by even sympathetic authors may be damaging to the progress of dual-process research” (p. 226). In their proposed model, almost all the attributes listed in Table 1 are correlates of the two types of processes, meaning that each such attribute can describe each of the two types of processing. However, one of the appealing characteristics of two-system models is the strong (though improbable) assumption regarding two sets of defining attributes. By abandoning this assumption, E&S are left with a stripped down version of a theoretical model, which now begs the question whether it is not simply a test of WM rather than of dual-mind or dual-process theory.

Finally, given the conceptual and empirical doubts regarding the two attributes selected to define the two types of processing, why select these two rather than other correlated attributes? The kind of evidence the authors cite in support of their model might be recruited for other attributes as well. Who is to decide which features are the “true” defining features? For instance, in Kahneman’s (2011 and personal communication, December 2011) framework, Type 1 and 2 processing are defined respectively by automaticity and effort, and in Lieberman’s (2009) framework, Type 1 and 2 processes are viewed respectively as conscious and unconscious, and one can probably come with some compelling arguments for yet a different set of defining features.

Continuum of Styles Versus Discrete Types (Criticism #3)

Both Newstead (2000) and Osman (2004) were correct when they asserted that demonstrations of processing continua undermine dual-process models—a point reiterated by Keren and Schul (2009). The logic is simple: If a particular dimension (e.g., automatic vs. controlled) is continuous, where is the cutting line that separates the two systems? This is a major drawback that makes the theory untestable. E&S are trying to resolve the issue by distinguishing between types, modes, and thinking dispositions postulating that types are discrete but modes are continuous. As we noted already, these different terms are poorly defined, thus obstructing theory testability, and the authors do not provide any classification criteria as to what constitutes type and what constitutes a mode.

Single Process Accounts May Be Offered for Dual-Process Phenomena (Criticism #4)

This issue has been raised by Kruglanski and Gigerenzer (2011) who advocate a single mental system, claiming that both intuitive (Type 1 processes) and deliberate (Type 2 processes) can be driven by the same set of principles. E&S are probably right in suggesting that the fact that intuition and deliberation are both rule-based does not logically rule out the possibility of two processing systems. However, as I claim below, two-system researchers have chosen to compile partial evidence in support of their claims and have never seriously tested alternative hypotheses such as a unified system. In our earlier paper (Keren & Schul, 2009), we did not take any firm stand on either one or two systems. We opted by default for a single system due to considerations of simplicity and the lack of sufficient evidence for two systems. The main purpose of this earlier article was to establish necessary criteria for justifiably adopting a two-system model.

Evidence for Dual Processing Is Ambiguous or Unconvincing (Criticism #5)

As noted already, a serious weakness of two-system theories, including E&S’s current version, is their failure to meet the most basic requirements of a scientific theory. Specifically, major constructs are not well defined, no explicit constrains of the theory are stated (which means it can explain everything), and consequently the theory does not generate testable predictions. In other words, the theory is actually not refutable. Indeed, all the evidence in support of the alleged theory consists of selectively describing results that are compatible with the theory, fulfilling what Wagenmakers, Wetzels, Borsboom, van der Maas, and Kievit (2012) were referring to as “an agenda for purely confirmatory research.” There is a fundamental difference between a posteriori tracing data that fits a theory and deriving a priori predictions that can then be empirically tested (and eventually be falsified). E&S cite three sources of findings in support of their two-process model, which I suggest are based on faulty inferences.

Experimental behavioral studies

The authors present findings from several studies showing that manipulation of WM leads to differences in accuracy of inferences. The question is whether these findings can be taken as evidence in support of the two-process model. E&S claim that correctness or validity of conclusions cannot be used as a criterion for distinguishing between Type 1 and 2. Specifically, they write “that it is a fallacy to assume that Type 1 processing is invariably nonnormative and Type 2 processing invariably normative” (p. 229). Yet, they interpret the fact that the belief bias (Evans, Barston, & Pollard, 1983) increased and logical accuracy decreased when people operate under time pressure or under concurrent memory load (Evans & Curtis-Holmes, 2005) as support for their new formulation, as both manipulations are assumed to inhibit Type 2 thinking. Clearly, a correlative attribute cannot be used as a marker to distinguish models. Although the findings are consistent with their two-process model, they are also consistent with numerous alternative models (cf., Keren & Schul, 2009). A similar point has been made by Kruglanski and Gigerenzer (2011). Finally, Thompson, Newstead, and Morley (2011) also concluded, based on their own extensive research on the belief-bias, that “None of the current instantiations of dual process theory can account for our data” (p. 333).

Neuroimaging studies

E&S state that “Although the [neuroimaging] studies are still relatively few in number, they generally provide strong support for the claims of the dual-process theorists” (p. 233). Not undermining the popularity of neuroscience in current psychological research, the implications and inferences made from studies using fMRI remain highly controversial (e.g., Gold & Stoljar, 1999); thus, due to various methodological obstacles, results should be interpreted with care. Henson (2006) has recently noted that inferences from brain activation to cognitive theories are only as good as the cognitive theories to which they pertain. In particular, differences in neural activation between experimental conditions may reflect any variation between the conditions, including the nature of the cognitive tasks, their extent, and the participants’ own reactions to their performance. Accordingly, the different patterns of brain area activation when responses are belief-based and when they are responsive to the logic of the problems, may reflect the subjective difficulty of the problems, the confidence of the participants, the memory operations activated by the tasks, or something else. Keren and Schul (2009) discussed the conceptual difficulties underlying inferences from studies using dissociation designs to inferences about system models (see also Chater, 2003). Similar concerns come up when one tries to demonstrate qualitative differences in brain activity that presumably allow one to dissociate cognitive processes. Not only should the patterns of activity satisfy several stringent criteria, which are rarely met by neuroimaging studies (see Henson, 2006; Poldrack, 2006), but the experimental conditions should differ only with respect to the hypothetical process of interest.

There are two major problems with the neuroimaging evidence. First, there are many methodological difficulties associated with this type of evidence. For instance, several articles in a recent volume of Perspectives on Psychological Science noted the inflated (and thus unreliable) correlations between brain measures and behavioral criteria (for details, see Vul, Harris, Winkielman, & Pashler, 2009; Yarkoni, 2009). Second, regardless of the shakiness of the evidence, the two-process model might be consistent with the observed neuroimaging findings, yet so are many other models, including single-process models.

Individual differences: The third source of empirical evidence, cited in support of the two-process model, involves analyses of the relationship between WM capacity or intelligence and participants’ responsiveness to instructions and resistance to belief biases. The authors support their model with evidence that reasoning is impaired when mental calculations are constrained. However, there is also evidence that thinking more about an issue produces normatively worse responses in problems that require abstract reasoning. For instance, Kareev (2000, 2005) analyzed estimations of correlations and used simulations and empirical studies to show that estimates based on small samples are more accurate than estimates based on large ones. In particular, WM capacity is negatively correlated with the goodness of the correlation estimate. The amount of thinking might be influenced by variation in individual abilities, external limitations on cognitive resources, or differences in motivation, and obviously it can affect reasoning and decision. The point is that the nature of influence that amount of thinking has on reasoning varies with the task and the environment. In some decision environments, more computations or more informational units are beneficial, but in others they are detrimental. Accordingly, a priori predictions about the relationship between WM capacity and quality of decisions in the framework of a two-process model must be made in light of an a priori decision model about the particular task in the specific decision environment.

In closing this commentary, I briefly reiterate my major concerns. The main problem with all two-system theories, including the current version, is that they are too broad, they fail to specify any clear constraints, and thus they are untestable. The strength of a scientific theory is assessed by making falsifiable predictions with respect to the phenomena under investigation. In particular, a good theory must also be able to disallow events from happening—predictions that permit its refutation. In response to the criticisms of the early dual-system theories, E&S striped their model from many of the constraints imposed by the early dual-system models. With most of the attributes being only correlative (and, as noted by Fiedler, 2011, “voodoo correlations” are everywhere), their model becomes virtually irrefutable. Because correlations are never perfect, any pattern of findings involving the correlated attribute can be accommodated within their model.

A major weakness of both the current as well as previous models is their failure to state concise constrains of the theory—namely the conditions and phenomena to which the theory is applicable (or not). In other words, two-system theories have unlimited degrees of freedom, which implies that, with enough effort, there will always be some data that fits the theory (and what does not fit, can simply be placed in the drawer). In an insightful commentary, Roberts and Pashler (2000) have convincingly shown that solely relying on good fit between data and theory is not sufficient and can often be misleading. As they noted, “the use of good fits for evidence is not supported by philosophers of science nor by the history of psychology; there seem to be no examples of a theory supported mainly by good fits that has led to demonstrable progress” (p. 358). I claim that such a conclusion equally applies to all two-system models, including E&S’s model.

A good theory is also expected to provide new insights and lead to some new questions. None of the two-system theories satisfy these expectations. Dawes (1999) cites Stephan Gould as defining humans as “the primates who tell stories” (p. 29). Dawes has taken it a step further suggesting that humans, including scientists, are primates whose cognitive capacity shuts down in the absence of a good story. Indeed, two-system theories offer a good story—one that “pleases the mind”—yet this attribute is not considered as a prerequisite of an acceptable scientific theory. Whether further ontological deliberations of one, two, or multiple systems will advance our psychological knowledge is highly questionable. It is probably because of such doubts that Fodor (2001), in a similar vein, have questioned the usefulness of the modularity debate. After two decades in which two system-models have blossomed yet added little if any new insights, the time is probably ripe to divert our scientific efforts into new and more promising avenues.

Footnotes

Author’s Note

I am indebted to Yaacov Schul for his invaluable help and comments on previous drafts of this commentary.

Notes

References

Baddeley

(2012). Working memory: Models, theories, and controversies. Annual Review, 63, 1–29.

Barsalou

L. W.

(1999). Perceptual symbol system. Behavioral and Brain Science, 22, 577–660.

Barsalou

L. W.

(2003). Situated simulation in the human conceptual system. Language and Cognitive Processes, 18, 513–562.

Chater

(2003). How much can we learn from double dissociations. Cortex, 39, 167–169.

Dawes

(1999). A message from psychologists to Economists: Mere predictability does not matter like it should (without a good story appended to it). Journal of Economic Behavior & Organizations, 39, 29–40.

Eitam

Hassin

R. R.

Schul

(2008). Non-conscious goal pursuit in novel environments: The case of implicit learning. Psychological Science, 19, 261–267.

Eitam

Schul

Hassin

R. R.

(2009). Goal-relevance and artificial grammar learning. Quarterly Journal of Experimental psychology, 62, 228–238.

Evans

J. St. J. B. T.

(2009). How many dual-process theories do we need: One, two or many? In Evans

J. St. B. T.

Frankish

(Eds.), In two minds: Dual processes and beyond (pp. 31–54). Oxford, England: Oxford University Press.

Evans

J. St. J. B. T.

(2010). Intuition and reasoning: A dual process perspective. Psychological Inquiry, 21, 313–326.

10.

Evans

J. St. J. B. T.

(2011). Dual-process theories of reasoning: Facts and fallacies. In Holyoak

Morrison

R. G.

(Eds.), The Oxford handbook of thinking and reasoning. New York, NY: Oxford University Press.

11.

Evans

J. St. J. B. T.

Barston

J. L.

Pollard

(1983). On the conflict between logic and belief in syllogistic reasoning. Memory & Cognition, 11, 295–306.

12.

Evans

J. St. J. B. T.

Curtis-Holmes

(2005). Rapid responding increases belief bias: Evidence for the dual-process theory of reasoning. Thinking & Reasoning, 11, 382–389.

13.

Evans

J. St. B. T.

Stanovich

K. E.

(2013). Dual process theories of cognition: Advancing the debate. Perspectives on Psychological Science, 8, 223–241.

14.

Fiedler

(2011). Voodoo correlations are everywhere—not only in neuroscience. Perspectives on Psychological Science, 6, 163–171.

15.

Fodor

(2001). The mind does not work that way. Cambridge, MA: MIT Press.

16.

Gigerenzer

Reiger

(1996). How do we tell an association from a rule? Comment on Sloman (1996). Psychological Bulletin, 119, 23–26.

17.

Gold

Stoljar

(1999). A neuron doctrine in the philosophy of neuroscience. Behavioral and Brain Sciences, 22, 809–830.

18.

Hackenbracht

Tamir

(2010). Preferences for sadness when eliciting help: Instrumental motives in sadness regulation. Motivation and Emotion, 34, 306–315.

19.

Hassin

R. R.

Bargh

J. A.

Engell

A. D.

McCulloch

K. C.

(2009). Implicit working memory. Consciousness and Cognition, 18, 665–678.

20.

Henson

(2006). Forward inference using functional neuroimaging: Dissocations versus associations. Trends in Cognitive Science, 10, 64–69.

21.

Kahneman

(2011). Thinking fast and slow. New York, NY: Farrar, Straus and Giroux.

22.

Kareev

(2000). Seven (indeed, plus or minus two) and the detection of correlation. Psychological Review, 107, 397–402.

23.

Kareev

(2005). And yet the small-sample effect does hold: Reply to Juslin and Olsson (2005) and Anderson, Doherty, Berg, and Friedrich (2005). Psychological Review, 112, 280–285.

24.

Keren

Schul

(2009). Two is not always better than one: A critical evaluation of two-system theories. Perspectives on Psychological Science, 4, 533–550.

25.

Kruglanski

A. W.

Gigerenzer

(2011). Intuitive and deliberative judgments are based on common principles. Psychological Review, 118, 97–109.

26.

Lieberman

M. D.

(2009). What zombies can’t do: A social cognitive neuroscience approach to the irreducibility of reflective consciousness. In Evans

J. St. B. T.

Frankish

(Eds.), In two minds: Dual processes and beyond (pp. 293–316). Oxford, England: Oxford University Press.

27.

Navon

(1984). Resources—A theoretical soup stone? Psychological Review, 91, 216–234.

28.

Newstead

S. E.

(2000). Are there two different types of thinking? Behavioral and Brain Sciences, 23, 690–691.

29.

Oberman

L. M.

Ramachandran

V. S.

(2007). The simulating social mind: The role of the mirror neuron system and simulation in the social and communicative deficits of autism spectrum disorders. Psychological Bulletin, 133, 310–327.

30.

Osman

(2004). An evaluation of dual-process theories of reasoning. Psychonomic Bulletin & Review, 11, 988–1010.

31.

Poldrack

R. A.

(2006). Can cognitive processes be inferred from neuroimaging data? Trends in Cognitive Sciences, 10, 59–63.

32.

Roberts

S. A.

Pashler

(2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107, 359–367.

33.

Shah

Miyake

(1999). Models of working memory: Mechanisms of active maintenance. Cambridge, England: Cambridge University Press.

34.

Stanovich

K. E.

(2009). Distinguishing the reflective, algorithmic and autonomous minds: Is it time for a tri-process theory? In Evans

J. St. B. T.

Frankish

(Eds.), In two minds: Dual processes and beyond (pp. 55–88). Oxford, England: Oxford University Press.

35.

Stanovich

K. E.

Toplak

M. E.

(2011). Defining features versus incidental correlates of Type 1 and Type 2 processing. Mind & Society, 11, 3–13.

36.

Tamir

Ford

B. Q.

(2009). Choosing to be afraid: Preferences for fear as a function of goal pursuit. Emotion, 9, 488–497.

37.

Thompson

V. A.

Newstead

S. E.

Morley

N. J.

(2011). Methodological and theoretical issues in belief bias: Implications for dual-process theories. In Manktelow

Over

Elqayam

(Eds.), The science of reason: A Festschrift for Jonathan St. B. T. Evans (pp. 309–338). New York, NY: Psychology Press.

38.

Vul

Harris

Winkielman

Pashler

(2009). Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4, 274–290.

39.

Wagenmakers

E.-J.

Wetzels

Borsboom

van der Maas

H. L. J.

Kievit

R. A.

(2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632–638.

40.

Yarkoni

(2009). Big correlations in little studies: Inflated fMRI correlations reflect low statistical power—Commentary on Vul et al. (2009). Perspectives on Psychological Science, 4, 294–298.