Behaviour versus performance: The veiled commitment of experimental psychology

Abstract

The use and teaching of experimentation in psychology ought to accompany a discussion of what is within and what is beyond the reach of the method. I address this question by outlining the necessary restrictions that are prerequisite for conducting an experiment. The restrictions include establishing a fixed goal, the fulfillment of which represents successful participation in the experiment, a finite set of possible expressions for satisfying the goal, and a fixed assignment between each possible expression and how it should be performed. These restrictions entail rules that determine what counts as good participation (evaluation), and rules that determine whether a behaviour counts as participation at all (inclusion). Participants’ conformity to the rules makes experimentation possible and, more importantly, maintains the experimenter’s attention on features of performance. This selective attention, in turn, neglects participants’ capacities for adopting and violating rules, and possible alterations in their goals and interpretations. While these capacities fall beyond the scope of experimental psychology, their recognition is necessary for encountering what is not already understood at the start of a research project.

Keywords

behaviour conformity experimental psychology ontology

Human behaviour is open to a variety of descriptions. One description may emphasize the people engaged in the behaviour, while others may emphasize what motivates the behaviour, the skills necessary for the behaviour, or the meaning of the behaviour in a sociocultural context (Bergner, 2011, 2016). The type of description tightly attached to the promise of psychology as a natural science, and one that has a central place in experimental psychology, regards behaviour in terms of processes enacted by the body. I use the term performance to refer to the result of this type of description, for example, movements and postures, although the description can extend to neurophysiological correlates of performance or subsequent effects of performance in the environment. My aim in the present article is to clarify the preconditions that enable such descriptions to be of any use, as well as the necessary restrictions imposed by the preconditions.

In what follows, after presenting a brief version of the entire argument, I will provide a conceptual formulation of behaviour that underlies the argument and clarifies the distinction between behaviour and performance. This formulation follows Peter Ossorio’s (1973, 1985, 2013) analysis of behaviour with an awareness of multiple attributes. Finally, I use this formulation to examine the treatment of behaviour in experimental psychology. I then consider some possible objections and, finally, connect the discussion to some related critiques of experimental psychology.

The argument

A description of performance alone is often too ambiguous to be useful (Bergner, 2010; Kattsoff, 1948). Imagine being in a lecture hall and seeing a man who stands up abruptly in the middle of the lecture. Do we know what he is doing? Is he trying to ask a question? Is he informing the speaker that she is out of time? Is he trying to interfere with the lecture? The same movement can be associated with a range of intentions, which demonstrates how the movement alone is not sufficiently informative of the behaviour.

Imagine, however, that we are given the task of ensuring a movement (e.g., standing up) in a given context (e.g., a lecture hall) is always, consistently and unambiguously, informative. How would we accomplish this task? One solution is to have all audience members agree to stand up if, and only if, they cannot hear the speaker. As a result, standing up during the lecture would have a fixed meaning. Those unable to hear the speaker will be obligated to express it, and they will have a pre-specified way of doing so. Furthermore, monitoring who stands up becomes useful in keeping track of whether the lecture is audible. In our description, we can now replace reference to the movement with the meaning assigned to the movement. Instead of asking, “Did someone ever stand up during the lecture,” we can ask, “Was everybody able to hear the speaker?” Instead of asking, “Was there a relation between location in the lecture hall and standing up,” we can ask, “Did the location in the hall affect the ability to hear the speaker?” Of course, to retain our ability to interpret the two states (standing up and remaining seated) we would have to exclude anyone who is unwilling or unable to conform to the set rules.

The above example seems contrived, but a similar kind of arrangement is prerequisite for conducting any psychological experiment. Namely, the participants must be given a goal that remains unchanged throughout the experiment (e.g., monitor and express one’s ability to hear the lecture), they must adopt a finite and pre-specified set of possible expressions (e.g., “I am able/unable to hear the lecture”), and they must adopt a fixed, one-to-one assignment between their possible expressions and how to perform the expression (e.g., remain seated and stand up, respectively, for the first and second expressions). These restrictions ensure that we can interpret performance in a meaningful way and in connection with a task goal. At the same time, the restrictions require bracketing out the participants’ capacity for adjusting or abandoning the given goal, and the possibility that their behaviour might reflect something unforeseen by the experimenter.

Outside of experimental situations, goals can be adjusted or abandoned. Even with a fixed goal, what a person does in relation to the goal is not fixed and pre-specified (Brandtstädter & Rothermund, 2002). There is no one-to-one relation between what a behaviour signifies and how it is performed (Kruglanski et al., 2002). Even if there is a one-to-one relation between the meaning of a behaviour and its performance, the relation can change. These possibilities do not fit within the scope of an experiment and, therefore, their presence within the experiment would be either invisible or seen as disruptions. In short, by deciding in advance how to interpret participants’ performance, experimental psychology restricts its own domain of investigation and it undermines its ability to discover what is not already understood at the start of research.

To flesh out the argument further would require some attention to the concept of behaviour, particularly to highlight the difference between behaviour and performance.

A formulation of behaviour

Peter Ossorio’s approach to behaviour, which I adopt in the present discussion, has to be understood in light of his approach to concepts in general. By writing statements like, “Concepts can’t be told,” he highlights the contrast between concepts and statements. A concept cannot be evaluated as true or false. “I can tell you that the cat is on the mat, but if I say ‘cat’ or ‘mat’ or ‘behavior’ I haven’t told you anything” (Ossorio, 2013, p. 19). Nevertheless, concepts afford the distinctions that, in turn, enable making and evaluating statements or otherwise behaving in accordance with the distinction (Noë, 2012; Wittgenstein, 1953).

Because a concept belongs to a network of interrelated concepts, it affords a variety of distinctions depending on the conceptual connection we emphasize (e.g., Strawson, 1992, Chapter 2). For instance, the cat is not only something we can locate (“the cat is on the mat”), but also something we can feed (“the cat is hungry”), and something that affects its environment (“the cat is being noisy”), and so forth. Depending on the conceptual connections emphasized, the cat concept orients us toward various distinctions, and we may regard these connections as attributes of the cat (cf. Hibberd, 2001, 2016; Smedslund, 1997). Ossorio’s analysis is, in this sense, attentive to the distinctions afforded by the concept, without demanding a set of criteria for determining membership in, say, the “set of all cats.”

What about the concept of behaviour? To find the distinctions this concept affords, we first find the conceptual connections that serve as attributes of behaviour. We begin by asking: If we call an event a behaviour, what else are we inclined to say or ask about this event? Ossorio (2013) outlines a set of eight attributes. For the present purpose, I reformulate them into the following list of five: A behaviour (a) implies an agent with individual characteristics; (b) it implies a goal that motivates the behaviour, although it may or may not be fulfilled by the behaviour; (c) it implies cognitive and sensorimotor capacities; (d) it implies the enacting or performing of the behaviour; and (e) it implies a meaning within a sociocultural context (Bergner, 2010, 2011, 2016; and Shideler, 1988, more faithfully portray Ossorio’s original formulation).

Each attribute could be disputed if we decide to use it to include/exclude members of the “set of all behaviours.” However, the formulation is not intended to impose boundaries around the concept (such an approach to defining behaviour has not been successful, e.g., Allen & Bekoff, 1999, Chapter 2; Hibbard & Henley, 1994). Instead, each attribute corresponds to a question (e.g., Whose behaviour…? What goal…? What capacity…?, etc.) that orients us toward a certain set of distinctions, which also provide a basis for judging two behaviours as similar or different. Disagreements might arise with regard to how each attribute ought to be specified, but what Ossorio articulates is precisely the grounds upon which those disagreements take place. What is crucial about the formulation is, therefore, that it provides the conceptual connections, possible descriptions, and disagreements, which can apply to the types of events we regard as human behaviour.

Each attribute is multiply realizable, i.e., it is logically possible to change one attribute without changing others. For instance, just as pursuing a goal can be performed in different ways, the same performance attribute can be associated with different goals and meanings. The formulation is particularly useful in classifying descriptions of behaviour based on the attributes that are included or removed in the descriptions. Removing an attribute from a description means that our description is insensitive to variations in that attribute. For instance, if movement duration is the target of description, two behaviours are considered the same if they take up the same span of time, regardless of who performed them or why they were performed.

Everyday examples of behaviours that demonstrate and justify the present formulation can be found elsewhere (Bergner, 2011, 2016). Instead of examples from everyday life, I proceed to use the formulation in examining how we regard behaviour in an experimental situation.

Investigating human performance

An experiment presents a task for the participant, though not always explicitly. The participant engages in completing the task, while a set of factors are manipulated with the aim of finding out whether and how task performance is sensitive to them. In a task in which red/green items have to be categorized using left/right keypress responses, we could manipulate a host of factors ranging from what the participant holds in memory to their body posture. Finding that performance is sensitive to certain factors helps us understand what is involved in performing the task (Massaro, 1989).

In order for an experiment to be a valid test, the participants’ behaviour has to meet a set of criteria. For this reason, participating in an experiment belongs to the class of rule-governed human activities, in which there are rules for inclusion and rules for evaluation (Gerrans, 2005; Thomson, 2010). The inclusion rules, in particular, determine whether a behaviour can be evaluated at all. If, instead of a legal chess move, a person pours hot coffee on their opponent’s king and shouts “I win,” we would not say he is playing chess poorly, because the behaviour violates the inclusion rules. A student who draws caricatures of their professor on their exam sheets is also exhibiting a behaviour that cannot be included and evaluated in the examination. Similarly, in a visual psychophysics experiment, if a participant stops looking at the visual displays and begins pressing keys on the keyboard in alphabetical order, her behaviour would cease to be considered participation.

When the inclusion rules are followed consistently, we can take them for granted and focus on evaluation or measurement of performance. In describing the findings, we do not discuss whether participants cooperated with the experimenter, and how they behaved in such a way that made the experiment possible (Fitzgerald & Callard, 2015). That is because their very inclusion in the discussion depends on their conformity. While participants’ conformity to the rules enables evaluating their performance, it also has consequences for how we view the remaining attributes of behaviour.

Individual characteristics of agents

Averaging data enables experimental psychology to bypass discussing individuals and their differences (Billig, 2013). Nonetheless, two approaches can be adopted to distinguish among participants. First, we can use a performance measure to create categories of participants. In a study of the influence of memory on perception, for instance, we divided participants based on how well they performed the mental rotation component of the task (Gozli, Wilson, & Ferber, 2014). In another study examining the relationship between associative memory and attention, we divided participants based on their attention to the first of a series of events (Gozli, Moskowitz, & Pratt, 2014). In these cases, the target of investigation is not individual characteristics, but the relation between different features of task performance. The purpose of dividing the participants is so we could ask how mental rotation affects visual attention, or how attention affects the role of memory. Since performance is used to create categories of participants, the success of this approach consists of understanding the relationship between those very features of performance.

The second approach aims to pinpoint individual differences, determined outside the experimental task, in terms of performance features (e.g., Hommel, Colzato, Scorolli, Borghi, & van den Wildenberg, 2011; Paglieri, Borghi, Colzato, Hommel, & Scorolli, 2013). The motive here is to discover low-level features of performance that may be associated with high-level individual characteristics, or perhaps to “translate” the individual characteristics in terms of performance features. For example, Wilson, Lowe, Ruppel, Pratt, and Ferber (2016) asked whether the Openness personality trait can predict the breadth of visual attention in space. They found attentional inhibition over a wider space in participants who scored higher on Openness. Setting aside the fact that the variance accounted for is too small to warrant the notion of translating Openness to attentional inhibition, and setting aside the conceptual difficulty in relating Openness and inhibition, the researchers might defend their approach because a measurable feature of performance brings a kind of specificity that is lacking in a qualitative description of Openness. However, the measure employed by the authors does not possess clear and agreed-upon meaning. Indeed, Dukewich and Klein (2015) surveyed expert opinions about the attentional measure used by Wilson et al. (2016) and found little agreement on what the measure represents. Consequently, linking the measure to Openness does not add conceptual clarity (see also, Tafreshi, Slaney, & Neufeld, 2016). More fundamentally, the drive behind this approach to personality traits is seriously challenged if we consider the multiply realizable (logically independent) relationship between individual characteristics and performance attributes (Ossorio, 1973, 1985; see also, Angell, 1913, p. 264).

In another series of experiments, Chan, Rajsic, and Pratt (in press) compared participants who volunteered at the beginning of the academic term (“go-getters”) and those who volunteered toward the end of the term (“procrastinators”). The authors found negligible performance differences in a subset of tasks. Aside from reassuring researchers interested in performance, what do the findings say about the individual characteristics of “go-getters” and “procrastinators”? Does the negligible difference in performance correspond to a negligible difference in individual characteristics?

Let us assume that the so-called go-getters and procrastinators are distinguished in how they prioritize among their available goals, with procrastinators favouring short-term goals over long-term goals (Chen & Chang, 2016). Accordingly, placing both groups in a situation where they act on the basis of a fixed goal would be an effective way of minimizing or eliminating the between-group difference. This objection applies more generally to studies that aim to associate individual characteristics with performance attributes within an experiment. To the extent that those characteristics relate to participants’ tendencies in selecting goals, adjusting or abandoning goals, and violating rules, the experimental setup is more likely to conceal than reveal those characteristics (Brandtstädter, 1987; Smedslund, 1991).

In short, an experimental study of performance is an ineffective way of studying individual characteristics, because it either (a) restricts the scope of description to features of performance, (b) fails to bring clarity to already assumed individual characteristics, or (c) systematically suppresses a range of individual characteristics.

Goals

We assume the participants are sufficiently motivated to complete the experiment, although it is regularly acknowledged that this motivation is substantially weaker compared to many everyday situations (Blackman, 2014; Herwig, Beisert, & Prinz, 2013). It should be noted that when experimenters make reference to goals, they tend to refer to the task goals provided by the experimenter, not the goals that bring the participant to the experiment (Hommel, 2015). The distinction is akin to that between wanting to play a game (analogous to participation goal) and wanting to win the game (analogous task goal). The former is simply assumed, which is why we typically bypass the question of motive with regard to participating in experiments. After all, an experiment is informative of those who participate in it. The kind of goal that remains in discussion, therefore, is the one provided by the experimenter and presumed to be shared by all the participants.

Cognitive and sensorimotor capacities

There is typically an overwhelming gap between what participants are capable of doing and what they are asked to do in an experiment. The focus on a small set of factors is a necessary part of the experimental approach, and in and of itself is not a problem if our experiments are parts of a larger theoretical system. Such a system would guide division of labour among researchers and would anticipate their integration. Unfortunately, experimental psychology is currently not grounded in any such system, which means many clusters of empirical findings are isolated fragments with questionable relevance (Allport, 2008; Hommel & Colzato, 2015; Kingstone, Smilek, & Eastwood, 2008; Newell, 1973).

A consequence of this fragmentation is a neglect of what falls outside the immediate scope of the experimenter’s question, because many capacities that enable participating in the experiment can be ignored and lost in the theoretical void. Most relevant to the present argument is the capacity to recognize, follow, and violate rules (Schmidt, Butler, Heinz, & Tomasello, 2016; Tworek & Cimpian, 2016), and the capacity to maintain a social contract with the experimenter. Neglecting these capacities, and that they are exercised in an experimental setting, frequently results in incorrect interpretation of experimental findings (cf. Hacker, 2015; Robinson, 2016).

Possible divergences between how participants and researchers may describe the same behaviour, and the capacities involved, should also be noted. Participants’ own accounts of completing an experiment would include reference to their capacity to distinguish experimental participation from other types of behaviour. The capacity to distinguish types of behaviour is characteristic of deliberate and cognizant behaviour (Ossorio, 2013, pp. 57–58). Furthermore, participants’ own accounts would include the fact that, in an experimental setting, behaving as a participant is itself a goal, regardless of any other outcome. Including behaviour, recursively, as a goal attribute is characteristic of social practices, for example, pronouncement of marriage, making a promise, etc. (Ossorio, 2013, p. 59). What research participants do within an experiment can, therefore, be described as deliberate and cognizant (knowingly behaving as a research participant and according to a set of instructions) social practice (fulfilling a social contract through following a set of instructions), even though such descriptions must be suppressed when the focus is on performance. Such divergence in description highlights the neglected capacities of experimental participants.

Meaning

To ask about the meaning of a behaviour is to ask what the behaviour signifies within a sociocultural context (Bergner, 2010; Ossorio, 2013). What does a person do by participating in an experiment? A reasonable response is that the person enters and fulfills a social contract. Within this contract, the experimenter determines what constitutes appropriate conduct within the experimental setting, and the participant either follows the instructions or is excluded from analysis. Participants’ conformity orients the researchers’ attention toward features of performance that are of interest in the given research project (e.g., reaction time, accuracy, movement trajectory, probability of a given response, etc.), and away from the experiment as social contract (Blackman, 2014; Fitzgerald & Callard, 2015; Robinson, 2016).

Relying on participants who conform with the rules of the experiment is an effective means of concealing the social contract, the rules, and the capacity to resist and violate those rules (De Genova, 2009). As a result, experimental literature includes claims that social factors are absent in experimental research (Eysenck & Keane, 2015, p. 302) or that a process is being investigated in a “non-social” experimental situation (Wirth, Pfister, Foerster, Huestegge, & Kunde, 2016, p. 850). Our focus is drawn away from the social aspect of the experiment, because participants’ conformity renders this aspect unproblematic. Similarly, the constancy of rules for what can be included (and what must be excluded) leads to neglecting the very existence of those rules. What supports this neglect is a form of abstraction and suspension, which results from bringing one attribute (performance) to the foreground and relegating other attributes to the background (Morley, 2010; Stenner, 2008).

The question of meaning is relevant to the question of biological reductionism (e.g., Bergner, 2016; Martin, 2010; Pickersgill, Cunningham-Burley, & Martin, 2011; Slife, Burchfield, & Hedges, 2010). If we restrict our description to performance, it appears much easier to anticipate a biological redescription. Consider the difference between being in a room alone and being in a room with another person. Experiments have revealed how the presence of another person changes the pattern of our eye movements (Laidlaw, Foulsham, Kuhn, & Kingstone, 2011), our hand movements (Quesque, Lewkowicz, Delevoye-Turrell, & Coello, 2013), and our memory for transient visual events (Constable, Pratt, Gozli, & Welsh, 2015). These descriptions orient us away from the relational situation in which the presence of another person has meaning (Hibberd, 2014). Their openness to biological redescription is because they are already restricted to one human body.

In summary, an experimental setup does not target non-performative attributes of behaviour (individual characteristics, goals, capacities, and meaning). This claim does not mean performative and non-performative attributes of behaviour are causally independent. It means that when we rely on participants’ conformity to experimental rules, which requires keeping some attributes constant (goal, meaning), while neglecting others (e.g., individual characteristics, certain cognitive/sensorimotor capacities), we undermine our ability to investigate the relation among the attributes.

Possible objections

Rules are inessential

A reader familiar with experimental literature might think of counter-examples in which participants are apparently not conforming to any rules. For example, studies of imitative movement (e.g., Genschow & Brass, 2015), studies of the role of the presence of other persons in the situation (e.g., Nasiopoulos, Risko, Foulsham, & Kingstone, 2015; Risko & Kingstone, 2011), and studies of the subliminal sources of behaviour (e.g., Bargh & Chartrand, 1999) may require nothing beyond waiting in a room, or at most looking at some apparently irrelevant display. Although researchers in these studies have the opportunity to adopt a more descriptive approach, the participants can still violate some inclusion rule, for example, by closing their eyes, making a phone call, or walking around in the room.

Not conforming to the inclusion rules means behaving in such a way that prevents hypothesis testing. An example can demonstrate how failing the inclusion rule does not require explicit awareness of the rule. In a recent study of learning social norms, 3-year-old children first observed an adult who used a tool in a particular manner. The children were then given the chance to impose the adult’s manner of tool-use on a puppet that used the tool differently (Schmidt et al., 2016). Because interacting with the puppet was necessary for testing the hypothesis, the researchers excluded any child who did not interact with the puppet during an initial phase. In this example, interacting with the puppet was an inclusion rule, even though the children were not aware of this rule. For an experiment to be a valid test, in general, it is required that participants engage with the experimental setup in such a way that enables hypothesis testing, regardless of their explicit awareness of this requirement.

Experimental study of rule violation

I anticipate an objection based on a recent series of studies on rule following and rule violation (Pfister, Wirth, Schwarz, Steinhauser, & Kunde, 2016; Wirth et al., 2016). The possibility of studying rule violation experimentally would be seriously damaging to my argument. If participants can freely adjust and abandon goals without compromising their role as participants, if participants can exert such an authority over the rules of the experiment, then we would have evidence that experimental researchers can target attributes other than performance. It is, therefore, necessary to examine the structure of these experiments.

The experiments involved instructing participants in a rule-governed movement task, consisting of a starting location on the screen (bottom-middle) and two target locations (top-left and top-right). Participants were required to move an avatar to one of the two target locations depending on the shape of the avatar (pawn vs. king). The rule could, for example, require taking pawns and kings, respectively, to the left and right target locations. On a subset of trials, prior to seeing the avatar, participants received a signal that instructed them to violate the task rule. The researchers carefully measured several performance features, and found that performing “rule-violation” trials was less efficient and contained reliable trajectory deviation toward rule conformity. The authors stated that we are beginning to understand the “signature and underlying mechanism of deliberate rule violations” (Wirth et al., 2016, p. 838).

From what perspective is it possible to describe participants’ behaviour as rule violation? From the experimenters’ perspective, participants were following the instructions. Indeed, it is helpful to note that when participants did not follow instructions, either on “rule-following” or “rule-violation” trials, their performance was excluded from analysis (Pfister et al., 2016; Wirth et al., 2016). From the participants’ perspective, the experimenters’ instructions were also being followed. They were given a set of rules that were supposed to be violated on a subset of trials. Participants’ behaviour is, therefore, not representative of rule violation, except perhaps from the perspective of an observer who happens to hear only half of the instructions. Characterizing what the participants do as non-conformity is possible after neglecting the conformity that enables the experiment. Therefore, these experiments do not deviate from, but exemplify, the kind of conformity to experimental rules discussed above.

“It can be done!”

I anticipate another objection, in response to my review of Pfister et al.’s (2016) studies. One could argue that I have, at best, demonstrated that violating experimental rules has not yet been included within an experimental design, but that does not mean rule violation, as a matter of principle, has no place in experimental psychology. Could one envision a variant of the aforementioned experiment, in which participants could violate the rules of the experiment? Of course, we would still need to know whether participants are following or violating the rule, in order to distinguish violations from mistakes. Consider the following variant of the task. At the beginning of each trial, instead of instructing the participant to follow/violate the task rule, we let participants decide whether they will follow/violate the task rule. They would record their decision using one button press, and then execute their decision.

In this new variant of the task, there are still (at least) three ways that we would be forced to exclude participants for not meeting inclusion criteria: A participant will be excluded if they do not maintain consistency between their first response (prior intention to follow/violate) and their second response (follow/violate); a participant will be excluded if they always follow the task rule, because that would not enable the experimenter to compare their performance across the two kinds of trials; and, for the same reason, a participant will be excluded if they always violate the task rule. These necessary exclusions reveal the persistence of inclusion rules in the modified version of the experiment.

Refusal-to-participate as a response

In well-known studies of obedience to authority, the experimenter monitors participants’ decision to terminate the session (Milgram, 1963; for a review see, Blass, 1999; for a critical analysis, see Stam, Lubek, & Radtke, 1998). That is, terminating the experiment is brought to the attention of the researchers as an available option and, indeed, as the expected alternative in a two-choice response (follow vs. refuse). Consequently, it could be argued that these studies represent genuine investigation of non-conformity.

Once again, from what perspective does terminating the session count as violating the rules of the experiment? Milgram set up his design with the explicit intention to elicit refusal, at least in some participants. A refusal, therefore, fits within the pre-specified structure of the experiment as one of the two possible outcomes, i.e., refusing and following instructions both conform to the two-choice structure of the experimental design (Brandtstädter, 1987). From the participants’ perspectives, a refusal appears as genuine violation only because they are misinformed about the experiment. Their experience is akin to someone who is presented with two keys, requested to only press key #1 and never key #2. Throughout the session, the participant presses key#1 and every time receives negative feedback (e.g., “MISTAKE!”). He turns to the experimenter, who deceitfully tells him “it is essential that you continue to press only key #1.” He finally presses key #2 and is then informed that pressing key #2 was always a genuine choice (and not a violation).

In describing the studies of Pfister et al. (2016), I argued that only someone unaware of a portion of the instructions may consider participants’ behaviour as rule violation. The type of design used by Milgram (1963) makes it possible for participants to consider ending the session as genuine non-conformity, precisely by keeping them uninformed about the design. In addition, the choices are restricted to “conformity” and “non-conformity,” which stand in sharp distinction. The instructions ensure that no middle ground is available, for example, negotiating alternative ways of continuing the experiment. This is in part because the experimenter must easily and clearly identify instances of non-conformity (subtle forms of non-conformity might go unnoticed). Aside from its unpleasant and ethically questionable details, the study is representative of the experimental method in its restrictions. One either accepts the given task goal and the rules for fulfilling the goal or altogether refuses to participate.

The argument is trivially true

Another objection to my argument is that it is trivially true. “Yes,” the critic will say, “we all know that experimental psychology requires conformity in order to render performance interpretable. We all know that sometimes non-experimental methods are more appropriate.” Even if my position faces no disagreement, it nevertheless faces a widespread neglect. As a member of the community of experimental psychologists, it is clear to me that the nature and consequences of this neglect are far from arbitrary. By evading the question of the scope and limits of experimental psychology, we can implicitly assume that the method can eventually—given enough time and effort—address all psychological questions. The purview of each experiment is small, the experimenter says, but give us enough time and we will cover the entire field.

Evading the question, furthermore, keeps unchallenged the assumption that ultimately the language that explains behaviour will consist of references to movements, postures, neural functions, stimuli, or—in other words—the language of performance (Hacker, 2015; Robinson, 2016). The neglect, therefore, serves a meta-scientific theory. Given that explicit defenses of this theory have not fared well (e.g., Marken, 1988; Robinson, 1985, Chapter 4; Sampson, 1981; Smedslund, 2015, 2016; Wheeler, 1925), evading the issue is an effective strategy to preserve it. By evading the question, experimental psychology can continue to make promises that it cannot fulfill (Billig, 2013).

Neglecting the scope and limits of experimental psychology is also a neglect of the gap between operational definitions and everyday use of concepts. If an experimenter asks you to violate a task rule, there is no limit to how creatively you could behave. If, however, the experimenter says, “by violation, I mean moving your avatar to point A, instead of point B,” then you recognize that the meaning of the word “violation” is narrowly defined in the context of the experiment, which in turn narrows the interpretation of the findings. Another experimenter interested in volition will present you with a button-press task and the instruction, “You are free to press the button when you want.” We should note, however, the many decisions already made prior to the so-called free act — whether or not to participate, which button to press, and which hand/finger to use (Hommel, 2017, in press). The operational definition of volition has to be taken into account in interpreting findings. The decision to begin a game of chess cannot be examined by analyzing a game, any game, of chess. The decision precedes the game. Similarly, in an experiment, the participant’s decision to begin performing the task stands outside the experimental task. The meaning of experimental participation cannot be found by looking at the pattern of performance (Teo, 2006, Chapter 4).

Awareness of how experimental psychology relies on human performance, and the preconditions that enable the description of performance to be of any use (fixed goal, restriction of possible expressions, determination of how an expression is performed, and conformity), can increase awareness of the method and its limits. It can enable more accurate interpretation of experimental findings, prevent misuse of concepts, and orient attention toward meta-theoretical questions.

Related ideas

Generalizability

Experimental psychologists have discovered that neglecting the context in which performance occurs, i.e., equating behaviour with performance, is harmful even to the study of performance. This is due to neglecting the possibility that the findings might be by-products of the rules and structure of the experiment (e.g., Allport, 2008; Besner, Stolz, & Boutilier, 1997; Hommel, 2000, 2010, 2015; Kingstone et al., 2008; Proctor & Cho, 2006; Proctor & Xiong, 2015). For instance, the conclusion that the visual system is always sensitive to a unique item in a scene often comes from experiments that implicitly prepare the participants to watch out for unique items (Bacon & Egeth, 1994; Li & Yeh, 2007). The conclusion that processing object location is automatic comes from experiments that require some form of localization (Hommel, 2011). Similarly, the conclusion that semantic processing of words is automatic comes from experiments in which participants engage in some form of categorization (e.g., Besner et al., 1997; Hommel, 2000, 2004). In short, experimental findings are, in part, determined by the rules and structure of the experiment (Brandtstädter, 1987), which restricts their generalizability.

Input–output and closed-loop systems

An experimental setup involves similar treatment of participants, not only by giving them the same set of possible responses, but also by presenting them with the same set of events (stimuli) meant to trigger those responses. This setup frames behaviour as the final step (“output”) in a chain of events that begins with the presentation of a stimulus. Critics of this approach do not refute the fact that people are capable of responding to events, nor do they refute that the responses can follow predictable patterns. What they refute is, first, the primacy of stimulus in the causal sequence (Hommel, 2000, in press) and, second, the assumption that behaviour is part of an open causal sequence (with a beginning and an end), instead of a closed sequence that includes feedback (Marken, 2009). The fundamental difference between input–output and closed-loop systems is that the latter enables the system to control and maintain a desired state.

Consider two friends in conversation who are walking together on their college campus. An input–output perspective would ask how each person understands and responds to their friend’s utterances, why one laughs at another’s joke, or how the appearance of an obstacle in their way triggers a change in their walking direction. In addition to these questions, a closed-loop perspective asks what is continually controlled by their behaviour (Marken, 2009). We may notice, for instance, that the two friends speak more loudly as their paths temporarily diverge around an obstacle that increases their distance or as they enter a noisy area. We can claim, roughly speaking, that being audible is a state that the two friends try to maintain, despite their increased distance or environmental noise. Each friend regularly relies on perceptible signs of being heard by the other, particularly when a disturbance is introduced. In this sense, the stimulus (e.g., a perceptible sign that one is heard) is better characterized as the end, and not the starting point, of behaviour (Gozli & Ansorge, 2016; Hommel, in press).

How is the closed-loop approach relevant to the present discussion? The approach is absolutely key in understanding the social nature of an experiment. The possibility that participants’ behaviour might be driven, in part, by the goal of controlling and maintaining certain states in the experimenter (e.g., perceptible signs of positive affect) or their relation with an experimenter (e.g., perceptible signs of a cooperative relation) escapes the scope of the input–output perspective. Yet, we know that factors such as the presence of the experimenter, interactions with the experimenter, and the continuous social feedback, can be critical in the formation of experimental findings (e.g., Blass, 1999; Fetsinger & Carlsmith, 1959). The notion of control is also connected to Stam et al.’s (1998) analysis of the context in which experiments are introduced, particularly to the new members of the discipline. Just as participants’ conformity to the rules of an experiment depends on their understanding of the experiment, larger-scale conformity in groups and institutions can be controlled via perpetuating a particular understanding of the experiment.

Equality and discovery

The issue of inequality between researchers and research participants has also been previously raised (e.g., Orne, 1962). Among others, Amedeo Giorgi (2014) emphasized that, although researchers and participants differ in their roles, whatever we say about the researcher has to also apply to the participant “in terms of human characteristics and possibilities” (p. 235). How do we determine and protect such an equality? Based on the present argument, essential to the equality is the authority over, and awareness of, the rules of the shared situation (Stam et al., 1998). In the experimental situation, we try to preserve equality by reminding participants that they can end the experiment at any time. Exercising that right, however, terminates the researcher’s opportunity to learn from the participant. What Giorgi has in mind, I believe, is to ensure that participants’ goals and interpretations could diverge from those of the researcher without necessarily ending their relationship. This equality implies that the researcher could study behaviour without pre-determining the possible outcomes. It implies that the participant’s behaviour could have meanings or motives or demonstrate capacities that are not initially understood by the researcher.

Interesting psychological questions arise when people deviate from implicitly held rules of behaviour, when multiple rules clash, or when the existing rules are silent about how one should behave (Giorgi, 2013; see also, De Genova, 2009). The behaviour of the chess player who pours hot coffee on the board, and the behaviour of the student who draws caricatures of the professor on their exam sheet may not be of interest to the chess analyst or the person grading the exam, but they are of interest to a psychologist. By deviating from the existing rules, they pose questions about attributes of behaviour other than performance (individual characteristics, goal, capacities, and meaning). The ability to address an unexpected attribute means the researcher is not the sole authority, who restricts the domain of study to pre-specified expressions performed in pre-specified ways. This important form of equality is absent in an experimental setting.

Conclusion

The aim of the present article was to flesh out the scope of the experimental method in human psychology. What I propose is not abandoning the method, but critical and responsible self-evaluation. Given the features of the typical experimental situation, the utility of the experimental method is confined to studying one attribute of behaviour (performance), while other attributes are either neglected or supplied as restrictions of the experimental design. Recognizing that those non-performative attributes are fixed or neglected, but not absent, begins to clarify the place of experimental psychology in the larger context of inquiry. Finally, meta-theoretical questions in psychology, the issue of reductionism and the significance of mechanical models of performance (“input–output”), are directly influenced by how we distinguish behaviour and performance.

Footnotes

Acknowledgements

I am thankful to Zuzanna Rucinska, Hyojin Pak, Alan Kingstone, Nevia Dolcini, Bruno Bocanegra, Fenna Poletiek, Bryant Jongkees, Greg Huffman, Roberta Sellaro, and Bernhard Hommel for helpful conversations. I also thank Hank Stam, Bob Sternberg, Rick Cooper, and four anonymous reviewers for their feedback on earlier drafts.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a Postdoctoral Fellowship from the Natural Sciences & Engineering Research Council of Canada (NSERC) held at Leiden University.

Author biography

Davood G. Gozli is an assistant professor of Psychology at the University of Macau. His experimental work is concerned with biases in perception and action, and what these biases reveal about human cognition. His theoretical work is concerned with the conceptual underpinnings of experimental psychology.

References

Allen

Bekoff

(1999). Species of mind: The philosophy and biology of cognitive ethology. Cambridge, MA: MIT Press.

Allport

(2008). The ups and downs of cognitive psychology: Attention and other “executive functions”. In Rabbitt

(Ed.), Inside psychology: A science over 50 years (pp. 13–26). New York, NY: Oxford University Press.

Angell

J. R.

(1913). Behaviour as a category of psychology. Psychological Review, 20, 255–270.

Bacon

W. F.

Egeth

H. E.

(1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 55, 485–496.

Bargh

J. A.

Chartrand

T. L.

(1999). The unbearable automaticity of being. American Psychologist, 54, 462–479.

Bergner

(2010). What is descriptive psychology? An introduction. In Davis

Lubuguin

Schwartz

(Eds.), Advances in descriptive psychology (Vol. 9, pp. 325–360). Ann Arbor, MI: Descriptive Psychology Press.

Bergner

R. M.

(2011). What is behaviour? And so what? New Ideas in Psychology, 29, 147–155.

Bergner

R. M.

(2016). What is behaviour? And why is it not reducible to biological states of affairs? Journal of Theoretical and Philosophical Psychology, 36, 41–55.

Besner

Stolz

J. A.

Boutilier

(1997). The Stroop effect and the myth of automaticity. Psychonomic Bulletin & Review, 4, 221–225.

10.

Billig

(2013). Learn to write badly: How to succeed in the social sciences. New York, NY: Cambridge University Press.

11.

Blackman

(2014). Affect and automaticity: Towards an analytics of experimentation. Subjectivity, 7, 362–384.

12.

Blass

(1999). The Milgram paradigm after 35 years: Some things we now know about obedience to authority. Journal of Applied Social Psychology, 29, 955–978.

13.

Brandtstädter

(1987). On certainty and universality in human development: Developmental psychology between apriorism and empiricism. In Chapman

Dixon

R. A.

(Eds.), Meaning and the growth of understanding (pp. 69–84). New York, NY: Springer-Verlag.

14.

Brandtstädter

Rothermund

(2002). The life-course dynamics of goal pursuit and goal adjustment: A two-process framework. Developmental Review, 22, 117–150.

15.

Chan

Rajsic

Pratt

(in press). Go-getters and procrastinators: Investigating individual differences in visual cognition across university semesters. Vision Research.

16.

Chen

B. B.

Chang

(2016). Procrastination as a fast life history strategy. Evolutionary Psychology, 14(1), 1–5. doi: 10.1177/1474704916630314

17.

Constable

M. D.

Pratt

Gozli

D. G.

Welsh

T. N.

(2015). Do you see what I see? Co-actor posture modulates visual processing in joint tasks. Visual Cognition, 23, 699–719.

18.

De Genova

(2009). Conflicts of mobility, and the mobility of conflict: Rightlessness, presence, subjectivity, freedom. Subjectivity, 29, 445–466.

19.

Dukewich

K. R.

Klein

R. M.

(2015). Inhibition of return: A phenomenon in search of a definition and a theoretical framework. Attention, Perception, & Psychophysics, 77, 1647–1658.

20.

Eysenck

M. W.

Keane

M. T.

(2015). Cognitive psychology: A student’s handbook (7th ed.). London, UK: Psychology Press.

21.

Fetsinger

Carlsmith

J. M.

(1959). Cognitive consequences of forced compliance. Journal of Abnormal and Social Psychology, 58, 203–210.

22.

Fitzgerald

Callard

(2015). Social science and neuroscience beyond interdisciplinarity: Experimental entanglements. Theory, Culture & Society, 32, 3–32.

23.

Genschow

Brass

(2015). The predictive chameleon: Evidence for anticipated social action. Journal of Experimental Psychology: Human Perception and Performance, 41, 265–268.

24.

Gerrans

(2005). Tacit knowledge, rule following and Pierre Bourdieu’s philosophy of social science. Anthropological Theory, 5, 53–74.

25.

Giorgi

(2013). Reflections on the status and direction of psychology: An external historical perspective. Journal of Phenomenological Psychology, 44, 244–261.

26.

Giorgi

(2014). Phenomenological philosophy as the basis for a human scientific psychology. The Humanistic Psychologist, 42, 233–248.

27.

Gozli

D. G.

Ansorge

(2016). Action selection as a guide for visual attention. Visual Cognition, 24, 38–50.

28.

Gozli

D. G.

Moskowitz

J. B.

Pratt

(2014). Visual attention to features by associative learning. Cognition, 133, 488–501.

29.

Gozli

D. G.

Wilson

K. E.

Ferber

(2014). The spatially asymmetric cost of working memory load on visual perception: Transient stimulus-centered neglect. Journal of Experimental Psychology: Human Perception and Performance, 40, 580–591.

30.

Hacker

P. M. S.

(2015). Philosophy and scientism: What cognitive neuroscience can, and what it cannot, explain. In Robinson

D. N.

Williams

R. N.

(Eds.), Scientism: The new orthodoxy. London, UK: Bloomsbury.

31.

Herwig

Beisert

Prinz

(2013). Action science emerging: Introduction and leitmotifs. In Prinz

Beisert

Herwig

(Eds.), Action science: Foundations of an emerging discipline (pp. 1–33). Cambridge, MA: MIT Press.

32.

Hibbard

Henley

(1994). Is psychology really “the science of behaviour”? A conceptual analysis of “behaviour” and some recommendations on the use of “behaviour” in psychology. Theory & Psychology, 4, 549–569.

33.

Hibberd

F. J.

(2001). Gergen’s social constructionism, logical positivism and the continuity of error: Part 2. Meaning-as-use. Theory & Psychology, 11, 323–346.

34.

Hibberd

F. J.

(2014). The metaphysical basis of a process psychology. Journal of Theoretical and Philosophical Psychology, 34, 161–186.

35.

Hibberd

F. J.

(2016). Is conceptual analysis only an inquiry into rules for the use of concepts? Theory & Psychology, 26, 815–822.

36.

Hommel

(2000). The prepared reflex: Automaticity and control in stimulus-response translation. In Monsell

Driver

(Eds.), Control of cognitive processes: Attention and performance (Vol. XVIII, pp. 247–273). Cambridge, MA: MIT Press.

37.

Hommel

(2004). Coloring an action: Intending to produce color events eliminates the Stroop effect. Psychological Research, 68, 74–90.

38.

Hommel

(2010). Grounding attention in action control: The intentional control of selection. In Bruya

B. J.

(Ed.), Effortless attention: A new perspective in the cognitive science of attention and action (pp. 121–140). Cambridge, MA: MIT Press.

39.

Hommel

(2011). The Simon effect as tool and heuristic. Acta Psychologica, 136, 189–202.

40.

Hommel

(2015). Action control and the sense of agency. In Haggard

Eitam

(Eds.), The sense of agency (pp. 307–326). New York, NY: Oxford University Press.

41.

Hommel

(2017). Consciousness and action control. In Egner

(Ed.), Handbook of cognitive control (pp. 111–123). Chichester, UK: Wiley.

42.

Hommel

(in press). Goal-directed actions. In Waldmann

(Ed.), Handbook of causal reasoning. Oxford, UK: Oxford University Press.

43.

Hommel

Colzato

L. S.

(2015). Learning from history: The need for a synthetic approach to human cognition. Frontiers in Psychology, 6, 1435.

44.

Hommel

Colzato

L. S.

Scorolli

Borghi

A. M.

van den Wildenberg

W. P. M.

(2011). Religion and action control: Faith-specific modulation of the Simon effect but not stop-signal performance. Cognition, 120, 177–185.

45.

Kattsoff

L. O.

(1948). What is behaviour? Philosophy and Phenomenological Research, 9, 98–102.

46.

Kingstone

Smilek

Eastwood

J. D.

(2008). Cognitive ethology: A new approach for studying human cognition. British Journal of Psychology, 99, 317–340.

47.

Kruglanski

A. W.

Shah

J. Y.

Fishbach

Friedman

Chun

W. Y.

Sleeth-Keppler

(2002). A theory of goal systems. In Zanna

M. P.

(Ed.), Advances in experimental social psychology (Vol. 34, pp. 331–378). San Diego, CA: Academic Press.

48.

Laidlaw

K. E.

Foulsham

Kuhn

Kingstone

(2011). Potential social interactions are important to social attention. Proceedings of the National Academy of Sciences, 108, 5548–5553.

49.

Yeh

S. L.

(2007). New objects do not capture attention without a top-down setting: Evidence from an inattentional blindness task. Visual Cognition, 15, 661–684.

50.

Marken

R. S.

(1988). The nature of behavior: Control as fact and theory. Behavioral Science, 33, 196–206.

51.

Marken

R. S.

(2009). You say you had a revolution: Methodological foundations of closed-loop psychology. Review of General Psychology, 13, 137–145.

52.

Martin

(2010). Self-making and the brain. Subjectivity, 3, 366–381.

53.

Massaro

D. W.

(1989). Experimental psychology: An information processing approach. San Diego, CA: Harcourt Brace Jovanovich.

54.

Milgram

(1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378.

55.

Morley

(2010). It’s always about the epoché. In Cloonan

Thiboutot

(Eds.), The redirection of psychology: Essays in honor of Amedeo Giorgi (pp. 207–215). Montreal, Canada: Interdisciplinary Circle of Phenomenological Research, University of Quebec.

56.

Nasiopoulos

Risko

E. F.

Foulsham

Kingstone

(2015). Wearable computing: Will it make people prosocial? British Journal of Psychology, 106, 209–216.

57.

Newell

(1973). You can’t play 20 questions with nature and win: Projective comments on the papers of this symposium. In Chase

W. G.

(Ed.), Visual information processing (pp. 1–26). New York, NY: Academic Press.

58.

Noë

(2012). Varieties of presence. Cambridge, MA: Harvard University Press.

59.

Orne

M. T.

(1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776–783.

60.

Ossorio

P. G.

(1973). Never smile at a crocodile. Journal for the Theory of Social Behaviour, 3, 121–140.

61.

Ossorio

P. G.

(1985). An overview of descriptive psychology. In Gergen

K. J.

(Ed.), The social construction of the person (pp. 19–40). New York, NY: Springer.

62.

Ossorio

P. G.

(2013). The behavior of persons. Ann Arbor, MI: Descriptive Psychology Press.

63.

Paglieri

Borghi

A. M.

Colzato

L. S.

Hommel

Scorolli

(2013). Heaven can wait: How religion modulates temporal discounting. Psychological Research, 77, 738–747.

64.

Pfister

Wirth

Schwarz

K. A.

Steinhauser

Kunde

(2016). Burdens of non-conformity: Motor execution reveals cognitive conflict during deliberate rule violations. Cognition, 147, 93–99.

65.

Pickersgill

Cunningham-Burley

Martin

(2011). Constituting neurologic subjects: Neuroscience, subjectivity and the mundane significance of the brain. Subjectivity, 4, 346–365.

66.

Proctor

R. W.

Cho

Y. S.

(2006). Polarity correspondence: A general principle for performance of speeded binary classification tasks. Psychological Bulletin, 132, 416–442.

67.

Proctor

R. W.

Xiong

(2015). Polarity correspondence as a general compatibility principle. Current Directions in Psychological Science, 24, 446–451.

68.

Quesque

Lewkowicz

Delevoye-Turrell

Y. N.

Coello

(2013). Effects of social intention on movement kinematics in cooperative actions. Frontiers in Neurorobotics, 7, 14.

69.

Risko

E. F.

Kingstone

(2011). Eyes wide shut: Implied social presence, eye tracking and attention. Attention, Perception, & Psychophysics, 73, 291–296.

70.

Robinson

D. N.

(1985). Philosophy of psychology. New York, NY: Columbia University Press.

71.

Robinson

D. N.

(2016). Explanation and the “brain sciences”. Theory & Psychology, 26, 324–332.

72.

Sampson

E. E.

(1981). Cognitive psychology as ideology. American Psychologist, 36, 730–743.

73.

Schmidt

M. F. H.

Butler

L. P.

Heinz

Tomasello

(2016). Young children see a single action and infer a social norm: Promiscuous normativity in 3-year-olds. Psychological Science, 27, 1360–1370.

74.

Shideler

M. M.

(1988). Persons, behavior, and the world: The descriptive psychology approach. New York, NY: University Press of America.

75.

Slife

B. D.

Burchfield

Hedges

(2010). Interpreting the “biologization” of psychology. Journal of Mind and Behaviour, 31, 165–177.

76.

Smedslund

(1991). The pseudoempirical in psychology and the case for psychologic. Psychological Inquiry, 2, 325–338.

77.

Smedslund

(1997). The structure of psychological common sense. Mahwah, NJ: Lawrence Erlbaum.

78.

Smedslund

(2015). The value of experiments in psychology. In Martin

Sugarman

Slaney

K. L.

(Eds.), The Wiley handbook of theoretical and philosophical psychology: Methods, approaches, and new directions for social sciences (pp. 359–373). Malden, MA: Wiley & Sons.

79.

Smedslund

(2016). Why psychology cannot be an empirical science. Integrative Psychological and Behavioral Science, 50, 185–195.

80.

Stam

H. J.

Lubek

Radtke

H. L.

(1998). Repopulating social psychology texts: Disembodied “subjects” and embodied subjectivity. In Bayer

B. M.

Shotter

(Eds.), Reconstructing the psychological subject: Bodies, practices and technologies (pp. 153–186). London, UK: Sage.

81.

Stenner

(2008). A. N. Whitehead and subjectivity. Subjectivity, 22, 90–109.

82.

Strawson

P. F.

(1992). Analysis and metaphysics. Oxford, UK: Oxford University Press.

83.

Tafreshi

Slaney

K. L.

Neufeld

S. D.

(2016). Quantification in psychology: Critical analysis of an unreflective practice. Journal of Theoretical and Philosophical Psychology, 36, 233–249.

84.

Teo

(2006). The critique of psychology: From Kant to postcolonial theory. New York, NY: Springer.

85.

Thomson

J. J.

(2010). Normativity. Analysis, 70, 713–715.

86.

Tworek

C. M.

Cimpian

(2016). Why do people tend to infer ought from is? The role of biases in explanation. Psychological Science, 27(8), 1109–1122.

87.

Wheeler

R. H.

(1925). Persistent problems in systematic psychology II: Psychological datum. Psychological Review, 32, 251–265.

88.

Wilson

K. E.

Lowe

Ruppel

Pratt

Ferber

(2016). The scope of no return: Openness predicts the spatial distribution of inhibition of return. Attention, Perception, & Psychophysics, 78, 209–217.

89.

Wirth

Pfister

Foerster

Huestegge

Kunde

(2016). Pushing the rules: Effects and aftereffects of deliberate rule violations. Psychological Research, 80, 838–852.

90.

Wittgenstein

(1953). Philosophical investigations. New York, NY: Macmillan.