Theory Building,Replication,and Behavioral Priming

Abstract

This article suggests that the field of behavioral priming, which is basically a technique, is in need of theory building. Guidelines from successful theory building by induction in the realm of conscious motivation (namely, goal setting and self-efficacy) are suggested. The process would include replication with variation, identifying the logical relation between a given prime and the action in question, discovering moderators and mediators, and clarifying the relationship between the conscious mind and the subconscious.

Keywords

goal setting self-efficacy priming

The special section in the January 2014 issue of Perspectives on Psychological Science addressed three important and interconnected issues—theory building, replication, behavioral priming—and the authors are to be commended for addressing them. I argue that behavioral priming, which as of now is basically a core idea (viz., priming affects action), is in need of systematic theory building as Dijksterhuis (2014) and Klatzky and Creswell (2014) have suggested. The question is, how does one go about it? Some have argued for a starting theory, but I will argue that this approach is premature. Some priming advocates have a meta-theory to the effect that the conscious mind has very little relevance to life so that the conscious mind can be largely ignored. For example, Bargh and Chartrand (1999) wrote: “…most of a person’s everyday life is determined not by their conscious intentions and deliberate choices but by mental processes that are put into motion by features of the environment and that operate outside of conscious awareness and guidance” (p. 462). Not all behavioral priming advocates would agree with such an extreme statement, however; thus, theory-building efforts would need to at least begin to address the issue of where and how the conscious mind comes into play in relation to priming.

Theory Building

There is a strong bias in our journals for deductive theory building: formulate a theory, deduce hypotheses, and then test them. This procedure has come under increasing scrutiny, especially because just about everyone knows that in practice this often leads to hypotheses being formulated after the tests in order to conform to the results. It has been suggested by some that hypotheses should be registered online in advance of every experiment. I believe that the real problem lies deeper. The deductive procedure leads to premature theory building and closure: If the hypotheses come out, the theory is supported and theory building may stop, but if they do not come out, a new theory has to be invented. I would argue that a more fruitful procedure would be to build theory inductively: constructing it piece by piece and moving from the particular to the general. This process takes time and cannot be done in one or two studies.

Harriman (2010) has shown that progress in physics and chemistry (e.g., Galileo, Copernicus, Kepler, Newton, the discoverers of the atomic theory of matter) was achieved by induction, not deduction (this included the use of the experimental method and mathematics). It is important to note that the premises used in syllogisms typically require previous induction. In the famous Socrates syllogism, the statement that all men are mortal had to be an inductive discovery. A deductive conclusion, even in the right form, cannot be true unless the premises are true. It is hard to see how deductions from some “irrational element” or “creative intuition” as recommended by Popper (2003, p. 8) would not be arbitrary if divorced from previous knowledge (see Locke, 2007). When premises cannot be validated, one can still form hypotheses, of course, but these would be closer to guesswork than actual deductions (of course, one can also do exploratory research with no hypotheses).

How Do We Do Induction?

There is no complete answer as to how we do induction, although Harriman (2010) argues that the contextual base for induction begins with causal identifications at the perceptual level (e.g., seeing that you can move a ball by pushing it) and continues at the conceptual level.

In the left column of Table 1, I suggest some theory-building principles based on induction (see Locke, 2007, for more details). These guidelines take it as a given that there is a real world out there and that the world obeys causal laws. In the right column, I identify how these principles were applied in developing goal-setting theory (see Locke & Latham, 1990, 2002, 2013, for further details). The table provides only a summary of the essentials of goal theory and not a complete statement. Goal-setting theory is a state rather than a trait theory and was developed within industrial/organizational psychology (which overlaps with organizational behavior in the field of management). It was designed to explain task performance based on conscious goals. It was first offered as a theory in 1990 based on 25 years of research involving several hundred studies done by myself, my codeveloper Gary Latham, and many others. As of now, over 1,000 goal setting studies have been conducted and new discoveries are still being made as shown in Locke and Latham (2013).

Table 1.

Guidelines for Inductive Theory Building

General guidelines	Illustrations based on goal-setting theory (mostly from Locke & Latham, 1990)
1. Start with a core idea based on some knowledge	1. Conscious goals regulate action (idea based partly on one previous study done 25 years previously; Locke & Latham, 1990, pp. 13–14)
2. Clearly define the core idea or concept	2. A goal is the object or aim of an action
3. Specify needed measurements	3. The best measure of a personal goal is the lowest score you would be satisfied (or not dissatisfied) with; assigned goals were hard, easy, or “do your best”; performance is quantified based on the task
4. Accumulate a substantial body of research	4. Initial theory based on about 400 studies
5. Show that there are reliable main effects, if possible	5. Meta analyses showed main effects, ds from .42 to .82 (hard > “do your best” goals; hard > easy goals)
6. Identify boundary conditions (moderators)	6. Main moderators: feedback (knowledge of progress), goal commitment, task knowledge or skill, task complexity, situational constraints
7. Identify causal mechanisms (mediators)	7. Mediators: focus of attention, effort, persistence, task knowledge or skill (again)
8. Establish generality	8. Generality across some 40,000 subjects in eight countries; laboratory, simulation, and field settings; correlational and experimental designs; 88 different tasks; time spans from 1 minute to 25 years; self-set, assigned, and jointly set goals; individual, team, unit, and small organizational levels
9. Take the law of contradiction seriously (e.g., use negative results as a signal to look for moderators)	9. All negative results (to 1990) analyzed for possible causes; controversy resolved by joint experiments (Latham, Erez, & Locke, 1988)
10. Connect to other relevant theories (especially if validated)	10. Integrated with self-efficacy aspect of Bandura’s (1997) social-cognitive theory

Goal-setting theory has its base in biology, given that the actions of living organisms are goal directed (unconsciously and/or consciously), rather than in cybernetic engineering as with control theory (Locke & Latham, 2013). Inanimate objects do not have their own goals.

It should be noted that social-cognitive theory (Bandura, 1986), especially the part relating to self-efficacy (Bandura, 1997), was developed inductively in a similar manner to goal setting theory. Self-efficacy (domain-specific confidence) was actually discovered by accident in the process of an experiment designed to cure snake phobias among volunteer subjects. But the subsequent research was systematic and far-reaching, following a parallel theory-building process, and the same degree of generality has been attained as a result of thousands of studies. Self-efficacy has a very reliable main effect on performance; its effect is causal and not just a reflection of past performance. Bandura developed a manual indicating how to measure self-efficacy. He identified its causes (how to build it), identified the mediators through which it operates (which include conscious goals), and moderators (of which feedback is one, just as with goal setting). The concept applies to groups and institutions and in other cultures. Self-efficacy, like goals, affects action in both laboratory and field settings.

Replication

As can be inferred from Table 1, I believe that the now popular push toward “exact” replication (e.g., see Simons, 2014) is not the best way to go. Everyone agrees that literal replication is impossible (e.g., Stroebe & Strack, 2014), but let us assume it is as close as one can get. What has been achieved? You have now done virtually the same study twice. If you are working from a deductive model and that one study is your “theory,” of course, you are done. But in reality you have not created a theory at all even if you replicate the same “exact” study 100 times. Almost none of the criteria I presented in Table 1 could be met with a single study.

How many single studies (or single studies with “exact” replication) in the history of psychology have led to a scientific contribution of lasting significance? There have been plenty of oft-cited studies but inevitably they end up being qualified or even refuted. Milgram’s famous obedience “study” was actually only one of 22 moderator studies but he never tried to integrate them into a theory (Perry, 2013).

Theory aside, I advocate (with Latham) replication with variation. This is similar in meaning to the often-used term “conceptual replication.” The specific purpose of replication with variation is to look for mediators and moderators and to try to establish, in some form, generality across various conditions (see Table 1). If the core idea is wrong or hopelessly vague, you won’t get replication either with variation or without it. If the core idea needs extensive development and refinement, this will require many and varied studies. With a useful core idea and many studies, a useful theory (or preliminary theory) may emerge but one cannot start with such a theory—this would require omniscience.

Priming

There is a terminological issue here. Priming advocates like to use the term unconscious, but to Freud the unconscious (e.g., the id) could never become conscious through introspection, though it could be inferred (e.g., from dreams). He used the term preconscious for material that was accessible to awareness. I prefer the term subconscious and use it to mean “potentially conscious.” By that, I mean material stored in memory that can be brought into awareness. I believe that this happens to everyone several hundred times a day—for example, whenever you have a conversation, the meanings of the words are automatically brought to consciousness. It is true that typically priming subjects cannot identify the purpose of the priming experiments they are in, but it does not follow that they can never be aware of the material that is stored.

There is a special problem involved in doing research on the subconscious and its relationship to action, however. The subconscious is a storehouse containing millions if not trillions of bits of information, and the material is widely interconnected. In contrast, the conscious mind (focal awareness) can only hold about seven disconnected items at the same time (Miller, 1956). The problem is, what, out of so many possibilities, will be activated in any given case? In everyday life, relevant subconscious material is routinely activated based on conscious purpose, though it can be activated by simple association as well (these processes can be revealed by introspection; Locke, 2009).

It gets more complicated: Even priming studies require that subjects be conscious of something, even if it is only a computer screen (used for subliminal priming). For supraliminal priming, subjects have to focus on a task (e.g., unscrambling sentences, observing another person, filling out a questionnaire, etc.). The studies are not done on comatose subjects. This aspect of priming studies has rarely been discussed. For example, Bargh (2014) reported a study in which shoppers in a supermarket were given a flyer containing health-related primes and they then bought fewer snack foods than a control group. But did the shoppers actually read the flyer, and, if so, what did they recall about it? If they did not read the flyer at all, how would one explain the results?

Now let’s expand this to everyday life. We are exposed to thousands if not tens of thousands of potential primes all day, every day of our lives; the environment is ubiquitous and ever-changing. But we don’t lurch from act to act like zombies, passively regulated by every prime that comes our way. This is unthinkable; there are too many possibilities. Nor can one claim that it depends on the strength of the connection between a specific, stored memory and the environment; the strength of the connection depends mainly on the relevance of what is stored to one’s current purpose. We must continually make conscious choices—we have goals and purposes every day, and every day is different in some ways from every other day. There is constant interaction or movement between the conscious and subconscious. (These same points have been made at length by Bandura, 2008, and others.)

Admittedly, people are not purposeful all the time: They can be resting, drifting, or daydreaming. In such cases, they cede their minds to the subconscious and numerous associations will come to mind. These may or may not lead to action in any given case—which brings us to priming research.

Behavioral Priming Research

Let me begin by saying that Latham and I believe that behavioral priming is a real phenomenon (Latham, Stajkovic, & Locke, 2010). Stajkovic, Locke, and Blair (2006) were the first industrial/organizational psychologists to do a behavioral priming experiment. Using the scrambled sentences task with 60% achievement words as primes, priming increased performance on a brainstorming task over and above the effects of a conscious, hard goal and a conscious “do your best” goal, though it did not improve performance over an easy goal. But all was not well. Two attempts at replication in Germany failed, even though they employed the “exact” same method, using the usual retranslation procedure for the prime words (Juergen Wegge, personal communication). The German professor lost interest after that so we do not know the cause of the failures. (Three more failures of achievement priming, attempts to replicate a Bargh study, are reported by Engeser, 2009; a significant, one-tailed effect was obtained post hoc by combining the studies—a dubious procedure.)

Later, Stajkovic, Locke, and Blair (2007) ran an experiment in which the dependent variables (DVs) were speed and accuracy in proofreading. We choose 60% prime words focused on speed and the same number for accuracy and used the same scrambled sentences method of priming. Priming affected speed but not accuracy (conscious goals worked well for both). Then we used the word matrix method (finding words in a matrix of letters) with exactly the same priming words. This time priming worked for accuracy but not well for speed. We checked in an independent sample whether students had a conscious, personal preference for speed or accuracy but found none. We could not explain these results, and, of course, could not publish it. It was especially puzzling that the same primes were used in both studies—only the form of presenting them differed.

Subsequently, Latham began a serious of priming studies using money raised at a university call center as the DV (Latham & Piccolo, 2009; Shantz & Latham, 2009). The prime in this case, however, was a picture of a woman racer. Priming significantly affected money raised, and this result was replicated even with the call center managers being totally blind to the hypothesis. However, using a picture of a call center employee with a headphone as a prime worked somewhat better than the woman racer. In another study, Latham gave the Thematic Apperception Test (TAT; a projective test involving writing stories in response to pictures that are then scored for achievement imagery) after priming. The prime affected the scores but the scores did not mediate the priming effect, even though the TAT allegedly measures subconscious motivation (Bargh, 2014, did not mention the lack of mediation). It turns out that motivation to perform can also be aroused by a geometric figure that implies forward motion (Natanzon & Ferguson, 2012).

What is the point in providing these examples? I believe that, over and above showing that priming can work, they show puzzling gaps in our understanding of priming:

Why would achievement priming work in one western country and not another?

Why would one type of priming affect one DV and not another whereas another type of priming (using the same priming words) would do the opposite?

How could radically different primes (sentences, matrix, two different pictures, and a geometric figure) all affect task achievement in some form?

One thing missing from the theory building process is that we do not know the logical connection between the actual primes and what they arouse. What is the most logical prime for a given DV? What is the most logical DV for a given prime? Why is walking speed the preferred DV for aging (Bargh, Chen, & Burrows, 1996)? Have walking speed primes been tried? Do they work better than the aging prime? Aging has many aspects (e.g., appearance, health, reaction time, memory, emotional tone, etc.)—which aspect will be aroused by which prime? Would the best prime for a given DV be the one that is most specific to that DV (e.g., the call center employee with a headphone)? If so, how far can you move away from independent variable (IV) specificity and still get a priming effect? Why do very dissimilar primes arouse performance (achievement) motivation?

If we were thinking in terms of a conscious goal we would expect that the best prime would be the one that was most specifically matched to the DV (e.g., the best prime for walking slowly should be a “slow” or “slow walk” prime of some type). In goal-setting research, the IVs and DVs are logically matched, although vague goals such as “do your best” are less effective than specific, difficult goals. Similarly, task-specific or domain-specific self-efficacy predicts logically related outcomes better than general self-efficacy. A lack of “similarity” or “specificity” effect (or continuum) creates a challenge for developing priming theory.

In addition, what is the ideal number and/or percentage of prime words for scrambled sentences? Many researchers seem to prefer 60% (Stajkovic et al., 2006, used 12 out of 20), although Chartrand (personal communication) prefers 90%. Would priming strength be proportional to the number of primes? Bargh (personal communication) once said that too many primes would causes reactive effects. But this seems dubious, as hundreds of goal setting studies have used assigned goals without reactive effects.

Presumably the mediator of priming, if it is not a subconscious motive, is a brain process. But to show mediation, one would have to measure the relevant brain process and then show that the degree of activation was related to the degree of performance attainment. Then, we would have to show that partialing out that brain process vitiated the priming effect. Bargh (2014) reported that brain activity has been shown to be affected by primes, but that does not prove that the brain states mediate the effects. Dijksterhuis (2014) reported that one study found priming effects to be mediated by conscious self-efficacy. If this is a replicable finding, it suggests that the mediating process in priming is not necessarily subconscious.

Now, consider the issue of moderators. Moderators, of course, identify limits or contingencies. An interesting report in Cesario’s (2014) article implies a problem for priming. The researchers looked at whether subliminal pictures of black males would prime aggressive responses to provocation. Several moderators of this effect that have been discovered include individual stereotypes of black males, where the white subjects were seated in relation to physical escape opportunities, the physical closeness of white allies, degree of self-monitoring, the source of the prime (self or other generated), and what group the prime was associated with. Bargh (2014) acknowledged that individual differences in traits have been found to be moderators. However, does every primed action depend on different moderators and/or traits? (For further examples, see Gino & Mogliner, 2014.) This suggests the need for a gradual and extremely complex theory-building process. Would a full theory of behavioral priming have to include thousands of moderators?

It should be noted that these same difficulties arose in trait-level research on the effects of subconscious motives on performance, notably in David McClelland’s work on need for achievement (McClelland, Atkinson, Clark, & Lowell, 1958). Spangler (1992) did a meta-analysis of TAT results for need for achievement and found significant effects using a variety of different DVs, but McClelland (1961) argued that the proper domain for achievement motivation was entrepreneurship or entrepreneurial activity rather than the arbitrary conglomeration of DVs that Spangler used. This means that Spangler’s meta-analysis should not have come out, especially because very few entrepreneurial studies were included. A meta-analysis using just entrepreneurial DVs found a significant association between TAT scores and entrepreneurial actions and success; this showed domain specificity. Self-report questionnaires gave the same result, though TAT and self-report measures are uncorrelated (Collins, Hanges, & Locke, 2004). But still, all was not well. A doctoral student of mine wanted to do a longitudinal study of need for achievement and entrepreneurship. McClelland approved of the sample (small business owners in one field). Due to his 50 years of experience, I asked him to select the TAT pictures to use as IVs. He chose four cards. I asked, dubiously, if four were enough, and he assured me they were. He was wrong. We found that one of the TAT card scores correlated negatively with the scores on the other three cards (the failed card involved a picture of circus performers!). The sum of the three cards was unreliable and did not correlate with entrepreneurial success in a 2-year follow up, whereas conscious goals were strong predictors (Tracy, Locke, & Renard, 1999). Further, in a 25-year longitudinal study of AT&T managers’ advancement, the TAT measures McClelland recommended contributed almost nothing, whereas conscious goals predicted the DV over the entire 25-year period (Howard, 2013), even controlling for ability.

Note here the parallel problem to behavioral priming: We don’t know the logical relation between the content of TAT cards and a specific type of performance. We don’t know which cards are the right ones for the domain in question. One would think the best pictures for entrepreneurship would be pictures of people who seem to be engaged in creating something or starting something new, but the typical TAT cards for need for achievement appear to have no central theme. It is not that projective (subconscious measures) are useless, but, to my knowledge, in 50 years no validated theory was developed regarding what were the best cards for measuring entrepreneurial motivation or performance in an organization. The type of TAT card would, of course, be a moderator.

Behavioral Priming Theory: The State of the Art

Now let us examine the state of behavioral priming theory based on the criteria listed earlier.

1&2. There is a core idea: Priming affects action without the individual being able to report being primed.

3. Performance is measured objectively.

4. Many priming studies have been conducted.

5. There are numerous main effects, but much conflict over failures to replicate; no theory of the logical relation of primes to outcomes.

6. Little systematic progress on moderators. (There is an important issue here: Conflicting or null results have to be caused and the causes need to be identified; usually moderators are involved.)

7. Virtually no progress on mediators.

8. Degree of generality (compared with goal setting and self-efficacy) unknown.

9. Little serious attempt to constructively resolve contradictory results.

10. Connection to other theories: neuroscience, automaticity theories.

One reviewer of this article asked whether the process for developing a valid theory of priming might have to be different than the procedures used for goal setting and self-efficacy theories. This is an interesting question, but I do not see any other way of developing a validated, as opposed to a speculative, theory. But it is likely that it will be much more difficult to develop a priming theory than it was to develop goal and self-efficacy theories.

Conclusion

Building a theory of behavioral priming will be a very arduous but nevertheless important undertaking. I believe it can succeed if the inductive method is used and if the focus is on replication with variation. Key tasks will include identifying moderators and mediators, resolving contradictory findings, establishing the logical relationship between what is primed and the outcome, and clarifying the role of the conscious mind.

Latham and I would acknowledge that, until our recent priming studies, goal-setting theory never focused on the subconscious. We never denied its existence, however, but chose to focus on conscious motivation for two related reasons: We thought it would have a more reliable payoff (based on what was known in the 1960s) than would starting with the subconscious, and because we thought it was closer to action than other determinants (based on Ryan, 1970). We now know that the subconscious can affect action without conscious intent in some cases. It is also obvious that goal effects rely on stored knowledge or skill.

In terms of new directions, I want to call attention here to Peter Gollwitzer’s research program, because it explicitly includes both conscious and subconscious elements (Oettingen, Wittchen, & Gollwitzer, 2013). One well-replicated finding from this research is that if one sets a goal and then formulates an implementation intention (“If Situation X arises, then I will do Y”), then the goal will be implemented in Situation X without further conscious intent. Such a procedure is even more effective if combined with mental contrasting (consciously comparing the desired future with difficulties that could be encountered). Mediators and moderators have been identified. This research program could serve as a model for showing how the conscious and subconscious work together.

Footnotes

Declaration of Conflicting Interests

The author declared no conflicts of interest with respect to the authorship or the publication of this article.

References

Bandura

(1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall.

Bandura

(1997). Self-efficacy: The exercise of control. New York, NY: Freeman.

Bandura

(2008). Reconstrual of “free will” from the agentic perspective. In Baer

Kaufman

J. C.

Baumeister

R. F.

(Eds.), Are we free? Psychology and free will (pp. 86–127). New York, NY: Oxford University Press.

Bargh

J. A.

(2014). The historical origins of priming as the preparation of behavioral responses: Unconscious carry-over and contextual influences of real-world importance. Social Cognition, 32, 209–224.

Bargh

J. A.

Chartrand

T. L.

(1999). The unbearable automaticity of being. American Psychologist, 54, 462–479.

Bargh

J. A.

Chen

Burrows

(1996). Automaticity of social behavior: Direct effects of trait construct and stereotype-activation on action. Journal of Personality and Social Psychology, 71, 230–244.

Cesario

(2014). Priming, replication, and the hardest science. Perspectives on Psychological Science, 9, 40–48.

Collins

C. J.

Hanges

P. J.

Locke

E. A.

(2004). The relationship of achievement motivation to entrepreneurial behavior: A meta-analysis. Human Performance, 17, 95–117.

Dijksterhuis

(2014). Welcome back theory! Perspectives on Psychological Science, 9, 72–75.

10.

Engeser

(2009). Nonconscious activation of achievement goals: Moderated by word class and the explicit achievement motive? Journal of Swiss Psychology, 68, 193–200.

11.

Gino

Mogliner

(2014). Time, money and morality. Psychological Science, 25, 414–421.

12.

Harriman

(2010). The logical leap: Induction in physics. New York, NY: New American Library.

13.

Howard

(2013). The predictive value of conscious and subconscious motives on career advancement. In Locke

E. A.

Latham

G. P.

(Eds.), New developments in goal setting and task performance (pp. 246–261). New York, NY: Routledge.

14.

Klatzky

R. L.

Creswell

J. D.

(2014). An intersensory interaction account of priming effects—and their absence. Perspectives on Psychological Science, 9, 49–58.

15.

Latham

G. P.

Erez

Locke

E. A.

(1988). Resolving scientific disputes by the joint design of crucial experiments by the antagonists: Application to the Erez-Latham dispute regarding participation in goal setting. Journal of Applied Psychology (Monograph), 734, 753–772.

16.

Latham

G. P.

Piccolo

R. F.

(2009, April). The effect of a specific versus nonspecific subconscious goal on employee performance. Paper presented at the annual meeting of the Society for Industrial-Organizational Psychology, New Orleans, LA.

17.

Latham

G. P.

Stajkovic

Locke

E. A.

(2010). The relevance and viability of subconscious goals in the workplace. Journal of Management, 36, 234–255.

18.

Locke

E. A.

(2007). The case for inductive theory building. Journal of Management, 33, 867–890.

19.

Locke

E. A.

(2009). It’s time we brought introspection out of the closet. Perspectives on Psychological Science, 4, 24–25.

20.

Locke

E. A.

Latham

G. P.

(1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall.

21.

Locke

E. A.

Latham

G. P.

(2002). Building a practically useful theory of goal setting and task performance. American Psychologist, 57, 705–717.

22.

Locke

E. A.

Latham

G. P.

(2013). New developments in goal setting and task performance. New York, NY: Routledge.

23.

McClelland

D. C.

(1961). The achieving society. Princeton, NJ: Van Nostrand.

24.

McClelland

D. C.

Atkinson

J. W.

Clark

R. A.

Lowell

E. L.

(1953). The achievement motive. New York, NY: Appleton-Century-Crofts.

25.

Miller

G. A.

(1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97.

26.

Natanzon

Ferguson

M. J.

(2012). Goal pursuit is grounded: The link between forward movement and achievement. Journal of Experimental Social Psychology, 48, 379–382.

27.

Oettingen

Wittchen

Gollwitzer

(2013). Regulating goal pursuit through mental contrasting with implementation intentions. In Locke

E. A.

Latham

G. P.

(Eds.), New developments in goal setting and task performance (pp. 523–548). New York, NY: Routledge.

28.

Perry

(2013). Behind the shock machine. New York, NY: New Press.

29.

Popper

(2003). The logic of scientific discovery. New York, NY: Routledge.

30.

Ryan

T. A.

(1970). Intentional behavior. New York, NY: Ronald.

31.

Shantz

Latham

G. P.

(2009). An exploratory field experiment on the effect of subconscious and conscious goals on employee performance. Organizational Behavior and Human Decision Processes, 109, 9–17.

32.

Simons

D. J.

(2014). The value of direct replication. Perspectives on Psychological Science, 9, 76–80.

33.

Spangler

W. D.

(1992). Validity of questionnaire and TAT measures of need for achievement: Two meta-analyses. Psychological Bulletin, 112, 140–154.

34.

Stajkovic

A. D.

Locke

E. A.

Blair

E. S.

(2006). A first examination of the relationship between primed subconscious goals, assigned goals, and task performance. Journal of Applied Psychology, 91, 1172–1180.

35.

Stajkovic

A. D.

Locke

E. A.

Blair

(2007, August). What happens to performance when conscious and subconscious goals are in conflict? Paper presented at the annual meeting of the Academy of Management, Philadelphia, PA.

36.

Stroebe

Strack

(2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9, 59–71.

37.

Tracy

K. B.

Locke

E. A.

Renard

M. K.

(1999, August). Conscious goal setting versus subconscious motives: Longitudinal and concurrent effects on the performance of entrepreneurial firms. Paper presented at the annual meeting of the Academy of Management, Chicago, IL.