Goal Instructions,Response Format,and Idea Generation in Groups

Abstract

This study examined the separate and joint impact of two standard, but seemingly conflicting brainstorming rules on idea generation in interacting and nominal groups: the free-wheeeling rule, which calls for the production of dissimilar ideas, and the build-on rule, which encourages idea combination and improvement. We also tested whether the superior performance of interacting groups found in several previous studies using a brainwriting technique may have been due to the different response formats employed by groups and individuals. Interacting groups and individuals generated ideas for improving their university under one of three sets of instructions. In one condition, participants were given the build-on rule, but not the free-wheeling rule, and in another condition, the reverse was true. In the third condition, both rules were provided. When the two rules were presented separately, interacting and nominal groups responded similarly, generating ideas from more semantic categories in response to the free-wheeling rule, and generating more practical ideas in response to the build-on rule. But when those rules were presented simultaneously, interacting groups generated ideas from fewer semantic categories than did nominal groups. In addition, interacting groups produced more ideas overall than nominal groups, but only when the two used different response formats.

Keywords

brainstorming creativity goals performance

The ideational productivity of groups relative to individuals has been the subject of much research over the past 60 years. The vast majority of that work has focused on the quantity of ideas generated, and, contrary to intuition, has found that interacting groups typically produce fewer ideas than do the same number of individuals who generate their ideas working alone (i.e., nominal groups). A number of factors have been shown to contribute to this unexpected result, including production blocking (e.g., Diehl & Stroebe, 1987, 1991), cognitive interference (e.g., Nijstad & Stroebe, 2006; Nijstad, Stroebe, & Lodewijkx, 2002, 2003), performance matching (Brown & Paulus, 1996; Camacho & Paulus, 1995; Paulus & Dzindolet, 1993), and motivation loss (Collaros & Anderson, 1969; Karau & Williams, 1993; Williams, Harkins, & Karau, 2003; see Goldenberg & Wiley, 2011 and Larson, 2010, for recent reviews). In the present study, we examine an issue that has been largely overlooked in previous research. It concerns the possibility that groups and individuals may pursue different goals when generating their ideas.

It is common in this area of research to begin an idea-generating session by giving both groups and individuals a set of instructions that derive from Osborn’s (1953, 1963) prescriptions for brainstorming. Brainstorming is a formal approach to idea generation in which participants are asked to follow a set of procedural rules designed to help them capitalize on the cognitive stimulation that is presumed naturally to occur when people work collectively to generate solutions to a problem. Although Osborn originally offered more than 20 different recommendations for how groups ought to be organized in order to maximize their ideational productivity, research has typically employed just the following four (Goldenberg & Wiley, 2011; Paulus & Brown, 2007):

No-Criticism: Criticism is ruled out. No evaluative judgments of any ideas are permitted.

Quantity: Quantity is desired. The more ideas the better, as more ideas should increase the likelihood of finding useful ideas.

Free-Wheeling: Free-wheeling is welcomed. The wilder the ideas the better, as it is usually easier to tame down than to think up. Because criticism is ruled out, it is both acceptable and desired that really wild and unusual ideas be shared.

Build-On: Combination and improvement are sought. Besides brand new ideas, suggestions are wanted for how an existing idea can be turned into a better idea, or how two or more ideas might be joined to create still another idea.

These four rules are usually presented together as a set, following Osborn’s assumptions that each in its own way can help to improve the ideational performance of groups, and that their benefits are cumulative. Recently, however, research has begun to critically examine these assumptions, for example by assessing the impact of the rules when presented separately. The quantity rule, particularly as it impacts idea quality, has received perhaps the greatest attention (e.g., Litchfield, 2009; Paulus & Dzindolet, 1993; Paulus, Kohn, & Arditti, 2011; Runco, Illies, & Eisenman, 2005), and some attention has been paid as well to the no-criticism rule (Nemeth & Ormiston, 2007; Nemeth, Personnaz, Personnaz, & Goncalo, 2004).The effects of the free-wheeling and build-on rules, on the other hand, have remained largely unexplored (but see Kohn, Paulus, & Choi, 2011). It is the separate and joint impact of these two rules that is the focus of the present research.

Brainstorming Rules: Competing Goals

It is useful to conceptualize the rules of brainstorming as instantiating a set of goals for participants (Litchfield, 2008). In this regard, it is noteworthy that Rule 3 (free-wheeling) and Rule 4 (build-on) seem to imply rather incompatible goals, and may engage opposing cognitive processes. On the one hand, Rule 4 suggests that the goal of ideation is to generate ideas that build-on and extend, and so resemble in various ways those that have already been proposed. As such, it would seem to encourage what Smith (2003; Kohn & Smith, 2011) refers to as fixation, which can be defined as a perseverance in the mind of features of already suggested ideas. Fixation results in a tendency, whether intentional or not, to include in a newly generated idea features of previously considered exemplars (e.g., Jansson & Smith, 1991). In brainstorming research, fixation is commonly indexed by the number of semantic categories found among the ideas produced: other things begin equal, using fewer semantic categories indicates greater fixation (e.g., Brown, Tumeo, Larey, & Paulus, 1998; Kohn & Smith, 2011; Larey & Paulus, 1999; Ziegler, Diehl, & Zijlstra, 2000). Thus, to the extent that Rule 4 promotes fixation, it should tend to limit the number of semantic categories from which participants draw their ideas.

Rule 3, on the other hand, is an explicit call for novelty and distinctiveness. Thus, whereas Rule 4 seems to require a degree of convergent, incremental thinking (Guilford, 1959; Guilford & Hoepfner, 1971) that should lead brainstormers to draw their ideas from a narrower range of semantic categories, Rule 3 demands a more divergent orientation that should result in their exploring a wider range of semantic categories. When presented together as part of the standard set of four instructions given at the start of a brainstorming session, these two rules would thus seem to establish incompatible goals. As Litchfield (2008) notes, when multiple goals motivate incompatible action tendencies, the result can be performance trade-offs (e.g., Locke, Smith, Erez, Chah, & Schaffer, 1994). Less clear is which of these conflicting orientations will actually be adopted when Rules 3 and 4 are presented simultaneously, and whether groups and individuals resolve this apparent conflict in the same way.

Competing Goals: Groups Versus Individuals

We suspect that interacting groups and individuals may adopt different orientations when Rules 3 (free-wheeling) and 4 (build-on) are presented simultaneously as part of the standard brainstorming instructions. Specifically, it seems likely that interacting groups will be more inclined to pursue the goal suggested by Rule 4, whereas individuals will be more apt to pursue the rule implied by Rule 3. There are at least three reasons for expecting this.

First, when working in groups people may naturally assume that building on, combining, and improving one another’s ideas takes precedence over generating “wild and unusual” ideas. In everyday life people often consult with others when generating ideas to solve a problem precisely because doing so gives them an opportunity to test their ideas and to improve them by adapting and integrating the suggestions of others. The inherently collaborative nature of group brainstorming may thus convey an implicit demand that favors the build-on rule (cf. Orne, 1962). Just the opposite sort of demand may prevail when people brainstorm alone.

Second, when working in groups, brainstormers may prefer to contribute ideas that are roughly similar to rather than different from those of others because doing so provides greater opportunities for mutual validation and building group cohesion (Stasser & Titus, 1985; Wittenbaum & Bowman, 2004; Wittenbaum, Hubbell, & Zuckerman, 1999). This is a form of performance matching that has not previously been considered (cf. Camacho & Paulus, 1995; Paulus & Dzindolet, 1993). By contrast, people who brainstorm alone may actually find it more stimulating to generate a series of distinct, unrelated ideas rather than a long list of similar ones.

Third, fixation effects resulting from exposure to the ideas of others may make it harder for brainstormers to follow the free-wheeling rule than the build-on rule when working in groups. Fixation has been shown in prior research to interfere with the production of novel ideas (e.g., Jansson & Smith, 1991; Kohn & Smith, 2011; Smith, 2003), and is difficult to eliminate simply by instructions to avoid it (Smith, Ward, & Schumacher, 1993, Experiment 3; Wiley, 1998). As such, it may simply be easier when working in groups to pursue the goal of building on and modifying already-suggested ideas than to strive to generate entirely new ideas (cf. Ward, 1994). People who brainstorm alone, by contrast, should be less troubled by fixation.

Thus, there seems ample reason to expect that when Rules 3 and 4 are presented simultaneously at the start of a brainstorming session, interacting groups will show a greater proclivity than individuals working alone to behave in accordance with Rule 4, and as a result to generate their ideas from fewer semantic categories. To test this idea, we had four-person groups and individuals spend 15 min generating ideas for improving their university, under one of three different combinations of the traditional brainstorming instructions. Approximately one third of our participants were given the standard set of four brainstorming rules prior to starting. The remaining two thirds got instructions that eliminated either Rule 3 or Rule 4. This was done to determine the effects of these two rules when presented separately. Thus, some participants were given Rules 1, 2, and 3, but not Rule 4, whereas others were given Rules 1, 2, and 4, but not Rule 3. The former emphasized free-wheeling and the production of ideas that are as different as possible from one another, whereas the latter emphasized combination and improvement and so the production of ideas that build-on one another. We compared the ideational performance of our interacting groups to that of four-person “nominal” groups created after the experimental sessions were completed by aggregating the nonredundant ideas generated by successive sets of four individuals who worked alone under similar instructions. We predicted the following:

Hypothesis 1 (H1): When given Rule 3 but not Rule 4 (the free-wheeling condition), both interacting and nominal groups will generate their ideas from more semantic categories compared to when they are given Rule 4 but not Rule 3 (the build-on condition).

Hypothesis 2 (H2): When given Rules 3 and 4 simultaneously as part of the standard set of four, nominal groups will generate their ideas from more semantic categories than will interacting groups. Thus, nominal groups will tend to perform as if they were given only Rule 3 (the free-wheeling condition), whereas interacting groups will tend to perform as if they were given only Rule 4 (the build-on condition).

In addition to testing the effects of the two rules on the number of semantic categories used, we were also curious about their influence on another aspect of ideational performance—the practicality or usefulness of the ideas generated. Typically, no difference is observed between interacting and nominal groups on this performance measure (e.g., Diehl & Stroebe, 1987; Rietzschel, Nijstad, & Stroebe, 2006, 2007). However, because Rules 3 and 4 imply rather different ideational goals, we wondered if they might also differentially affect idea practicality. Therefore, we investigated the following question:

Research Question (RQ): What will be the separate and joint effects of Rules 3 and 4 on the number of highly practical ideas generated by interacting and nominal groups?

Brainwriting: Importance of Response Format

We examined these issues using an idea-generating procedure introduced by Paulus and Yang (2000, see also VanGundy, 1981) called brainwriting. Interacting groups were asked to record their ideas on small notecards, writing up to four ideas per card, and exchanged their ideas by passing these notecards among themselves in round-robin fashion, reading one another’s ideas before writing a new idea of their own. This procedure reduces the opportunity for production blocking in interacting groups, as compared to oral discussion, by allowing members to contribute ideas simultaneously, without waiting for a speaking turn. Paulus and Yang (2000) found that when using this technique, four-person interacting groups generated 40% or more ideas during a 15-min period than did four-person nominal groups. Similar results using interacting and nominal dyads have been reported by Coskun (2005b) and by Coskun and Yilmaz (2009, Experiment 2). These findings, of course, are just opposite to what is usually found in brainstorming research (Larson, 2010; Mullen, Johnson, & Salas, 1991), and suggest that brainwriting may be a useful method for exploiting the creative capabilities of interacting groups.

However, a methodological detail common to all three of these studies calls into question the validity of this conclusion. Specifically, there was a small, but potentially important, difference in the way interacting groups and the individuals who comprised the nominal groups recorded their ideas. Whereas interacting groups recorded and exchanged their ideas on small slips of paper, individuals wrote their ideas in one long list on a full-size sheet of paper. Consequently, at any given moment during the idea-generating process interacting groups could see no more than four of their previously generated ideas, whereas individuals could see all of the ideas they had previously generated.

It is conceivable that being able to see all of their ideas at once in a single long list caused individuals in these studies to generate fewer ideas than they would have done had they written those ideas on small slips of paper, just as the groups did. There are two reasons for suspecting this, one cognitive and one motivational. First, the full-sheet response format may have caused more fixation among individuals by exposing them to an increasingly larger number of potential fixation targets—the ideas they had already recorded— as they worked. This may have made them more prone to fixation effects than would have been the case had they been able to see no more than four of their previously generated ideas at a time (cf. Kohn & Smith, 2011; Rundus, 1973). Second, the full-sheet response format may have promoted an unanticipated decline in motivation among individuals as they generated more and more ideas. Specifically, as the length of their idea list grew visibly long, it may have been easier for individuals to conclude that they had satisfied the goal of generating as many ideas as possible (Rule 2), and so could relax, as least in comparison to what otherwise would have occurred had they been able to see no more than four of their already-generated ideas at once. This explanation is consistent with well-established theories of self-regulation in goal-oriented performance (e.g., Bandura & Cervone, 1986; Locke & Latham, 1990, 2002) and also findings suggesting that perceptions of performance play a role in productivity (Nijstad, Stroebe, & Lodewijkx, 2006).

It thus seems possible that this difference in response format, although small, nevertheless gave interacting groups in these previous studies an unfair advantage over the individuals (as nominal groups) with whom they were compared. Evidence in support of this idea comes from a study by Coskun (2005a), who required individuals to record their ideas on the same size slips of paper, and in the same manner (four ideas per slip), as did interacting dyads. Using this modified procedure Coskun found no difference between the number of ideas generated by interacting and nominal dyads. However, response format was not actively manipulated in that study. Rather, it was held constant, with all participants recording their ideas in the same way. Consequently, it remains unclear whether it is this procedural difference or something else that actually led to the different pattern of results observed.

In the present study, most participants who worked individually were asked to record their ideas on notecards, and to set each notecard aside once four ideas had been written on it. Thus, the response format they used was identical to that used by interacting groups. However, in order to test directly the effect of response format, we created one additional nominal group condition—completely outside the factorial design described above—in which individuals given the standard set of four brainstorming instructions listed all of their ideas on a full-size sheet of paper. By comparing the performance of nominal groups in this full sheet condition with that of similarly instructed interacting and nominal groups that used notecards, we can better gauge the contribution of response format to the findings previously reported by Paulus and Yang (2000), Coskun (2005b), and Coskun and Yilmaz (2009; Experiment 2). Based on the results of these previous studies, as well as those more typically found in the brainstorming literature, we predict the following:

Hypothesis 3 (H3): Interacting groups that use the brainwriting procedure and record their ideas on notecards will generate more ideas than nominal groups comprised of individuals who record their ideas on letter-size sheets of paper.

Hypothesis 4 (H4): Nominal groups that use the brainwriting procedure and record their ideas on notecards will generate more ideas than nominal groups comprised of individuals who record their ideas on letter-size sheets of paper.

Method

Design and Participants

The study employed a fully crossed 2 (group type: interacting vs. nominal) × 3 (goal instructions: standard vs. free-wheeling vs. build-on) factorial design. In addition, a seventh condition, lying completely outside of the factorial design, was included to assess the response format confound identified in previous research. For convenience, we refer to this as the full sheet nominal group condition. The complete experiment thus involved a (2 × 3) + 1 design. The participants were 572 (59% female) undergraduate students from a Midwestern university who took part in the study in partial fulfillment of a requirement in their introductory psychology course.

Idea Generation Task

Participants generated ideas for improving their university. Suggestions for making any aspect of the university better were acceptable. They did this working either individually or in four-person interacting groups.

Interacting groups generated ideas using the brainwriting technique introduced by Paulus and Yang (2000). Their members were seated around a square table and each was given 25 notecards measuring 7.6 x 12.7 cm. They began by each writing one idea on one of their notecards, and then passed that card to the person on their right. Simultaneously they each received a notecard from the person on their left. Members read the idea written on the received notecard, added a new idea of their own, and passed it as before to the person on their right. Again they received a notecard from the person to their left, this time with two ideas written on it. They read both ideas, added another idea of their own, and passed the notecard to their right. This continued until the members each received a notecard with four ideas written on it, the first of which was their own. They read these four ideas, placed the notecard on top of any previously used notecards in the center of the table, and then started a new notecard with a new idea. This entire process was repeated over and over for 15 min.

If a group member was unable to think of a new idea in a reasonable amount of time (i.e., by the time the other three members had written their ideas), he or she was to pass the notecard on without writing anything down. Conversely, if while waiting for the next notecard someone thought of an extra idea, he or she could write it down on an extra blank notecard. These extra notecards were collected at the end of the session with all of the others, and the ideas they contained were included in the list of ideas the group had generated. Participants were told that when writing their ideas they did not have to use complete sentences, nor did they have to concern themselves with spelling or grammar. Simple phrases were satisfactory, as long as they were clear. The four members each used different colored pens, thereby making their individual contributions identifiable (cf. Paulus & Yang, 2000).

Participants who worked individually also spent 15 min generating ideas for improving their university, but did so working completely independently. They were run with up to three other participants in the same experimental session, and were seated around the same square table as before. Further, in those nominal group sessions that were part of the main, factorial portion of the experimental design, participants wrote their ideas on the same size notecards, four per card, as did the members of interacting groups. After filling one notecard, they set it aside and continued with a new card. By contrast, participants in the full sheet nominal group sessions recorded their ideas in one continuous list on an 21.6 x 27.9 cm sheet of paper, as described in more detail below.Regardless of the response format, however, participants in these nominal group sessions worked individually. Importantly, they did not share their ideas with one another.

Procedure

Four same-gender participants were scheduled for each experimental session. When all four showed up, they were run either as an interacting group or as individuals in one of the three randomly selected goal instruction conditions (described below). When less than four participants showed up, they were always run as individuals.

After being seated around the table, participants were told that the purpose of the study was to learn more about how people generate creative ideas. In the interacting group sessions the brainwriting procedure was then described, including a detailed explanation of how the notecards (and so ideas) were to be shared in round-robin fashion. In the individual sessions, by contrast, participants were simply asked to write down all of the ideas they had, either four per card, or one after another in a continuous list on full-size sheets of paper. Further, those participating as individuals were explicitly told that they were to work independently, and not to share their ideas with one another. The experimenter then introduced the goal instruction manipulation. After doing so, the experimenter answered any questions that participants had, asked them to begin working, and left the room. At the end of 15 min the experimenter returned, asked everyone to stop working, and collected all of the ideas generated.

Goal Instruction Manipulation

Standard condition

In the standard instructions condition, all four of the traditional brainstorming rules were given: (a) criticism is to be avoided, (b) quantity is desired, (c) free-wheeling is welcomed, and (d) combinations and improvements are sought. However, the wording of the first and fourth rules differed slightly depending on whether participants worked individually or in interacting groups. Specifically, whereas the instructions given to participants in interacting groups referred to all of the ideas generated by the members, regardless of who suggested them, the instructions for those who participated as individuals referred only to the participants’ own ideas.

Free-wheeling condition

In the free-wheeling condition, the third rule (free-wheeling) was emphasized, and the fourth (combine and improve) was dropped. Specifically, participants in interacting group sessions were told that they should try to generate ideas that were as different as possible from the ideas written on the notecards received from their neighbor to the left, and from all other ideas that they had read during the session, including those they previously wrote themselves. Likewise, those who participated as individuals were told that they should try to generate ideas that were as different as possible from all of the ideas they had previously written. All participants were told to write down ideas that did not meet this criterion only when they were unable to come up with an idea that was truly different from all of the other ideas already written.

Build-on condition

The build-on condition involved the opposite pattern of brainstorming rules: the fourth rule (combine and improve) was emphasized, and the third rule (free-wheeling) was dropped. Specifically, participants in interacting group sessions were told that they should try to generate ideas that build-on, modify, or improve one or more of the ideas written on the notecards they received from their neighbor to the left, or any other idea they had read during the session, including those they previously wrote themselves. They were cautioned, however, that ideas built on one or more other ideas should still be new ideas. (The build-on rule obviously did not apply to the very first ideas generated). Similarly, those who participated as individuals were told that they should try their best to generate ideas that build-on, modify, or improve one or more of their own previously written ideas. All participants were instructed to write down ideas that did not meet this criterion only when they were unable to think of a way to build-on, modify, or improve one or more of the already-generated ideas.

Full sheet nominal group condition

In the extra-factorial, full sheet nominal group condition participants worked individually under the Standard goal instructions as described above. However, rather than writing their ideas four-at-a-time on notecards, they wrote their ideas in one continuous list on a standard 21.6 x 27.9 cm sheet of paper. Participants were allowed to write on the back of the sheet, and were provided one additional sheet to use if needed. This method of recording ideas is similar to that used by nominal group participants in Paulus and Yang (2000), Coskun (2005b), and Coskun and Yilmaz (2009; Experiment 2), and it is the only difference between the full sheet condition and the nominal group/standard instructions condition that was part of the factorial portion of the experimental design.

Nominal Group Formation and Idea Transcripts

For data analytic purposes, nominal groups were created after the fact by assembling together all of the ideas generated by successive sets of four same-gender participants who worked individually under the same goal instructions and recorded their ideas in the same manner (i.e., using notecards vs. paper). This was done in order of participation (e.g., the ideas generated by the first four females who worked individually with notecards under the standard goal instructions were combined to define the performance of the first nominal group in the standard instructions condition of the factorial portion of the experimental design). All of the ideas generated by each nominal group were then transferred to a single typed list, resulting in 81 separate nominal group transcripts (61 in the factorial portion of the experimental design, and 20 in the full sheet condition). Likewise, all of the ideas generated by each interacting group were transferred to a single typed list, resulting in 62 separate interacting group transcripts.

Dependent Measures

Six coders, all blind to the experimental conditions, independently scored the transcripts for two qualitative features of the ideas (semantic category and practicality) as well as for quantity. Each transcript was scored by two coders,¹ with no one coder scoring more than half of the same transcripts as were scored by another coder. The coders were juniors and seniors attending the same university as the participants, and so were reasonably well qualified to judge the ideas proposed for improving their university. The average intraclass correlation (using Fisher’s r-to-z transformation) between pairs of raters for all measures was $\bar{r}$ = 0.97, with none lower than r = 0.78. All of the analyses reported below are based on scores that averaged across the two coders for each transcript.

Number of Semantic Categories

A preliminary inspection of the entire set of transcripts suggested that the generated ideas fell into 23 mutually exclusive categories. A working definition of each category was developed through discussion by three of the coders. Among the categories were academic policies, athletics, commuting, dorms, food, library, and safety (see Appendix A for a complete list and examples). Extensive training in the use of this category system was provided to all coders prior to scoring the transcripts. Coders then sorted all of the generated ideas by category, and for each transcript they counted the total number of categories (out of 23) sampled at least once.

Number of Highly Practical Ideas

Each transcript was also scored for the number of highly practical ideas it contained. Highly practical ideas were defined as improvements to the university that would (a) be relatively inexpensive to implement and (b) have a high impact on and/or be important to students. Coders rated the practicality of each idea on a 7-point scale (1 = very impractical; 7 = very practical), where each scale point was anchored by multiple examples (see Appendix B). Coders were given extensive training in the use of this scale. The number of highly practical ideas contained in a transcript was defined as the number of nonredundant ideas with a practicality rating of 6 or higher (cf. Reinig, Briggs, & Nunamaker, 2007).

Total Number of Ideas

Finally, coders also tallied the total number of ideas found in each transcript. This was done by first identifying redundant ideas, then subtracting those redundancies from the total number of transcript entries.

Results

Number of Semantic Categories

The number of semantic categories sampled by groups was correlated with the total number of ideas generated, r (121) = 0.52, p < .001. Consequently, within the factorial portion of the experimental design we analyzed the number of semantic categories sampled by means of a 2 (group type: interacting vs. nominal) × 3 (goal instructions: standard vs. free-wheeling vs. build-on) analysis of covariance (ANCOVA), where the total number of ideas generated served as a covariate. This analysis revealed significant main effects for both group type, F (1, 116) = 12.58, p < .001, η² = 0.10, and goal instructions, F (2, 116) = 9.81, p < .001, η² = 0.14. However, these main effects were qualified by a significant group type × goal instructions interaction, F (2, 116) = 3.64, p < .03, η² = 0.06 (see Figure 1). To test our specific hypotheses we conducted a set of follow-up comparisons using the Fisher-Hayter procedure.²

Figure 1.

Number of semantic categories sampled as a function of group type and goal instructions, controlling for the total number of ideas generated.

Hypothesis 1 predicted that both interacting and nominal groups would generate their ideas from more semantic categories when they were given Rule 3 but not Rule 4 (the build-on condition) compared to when they were given Rule 4 but not Rule 3 (the build-on condition). As predicted, both interacting and nominal groups given the free-wheeling instructions generated ideas from a larger number of semantic categories than did those given the build-on instructions; among interacting groups, M = 15.15 (SD = 1.41) versus M = 13.49 (SD = 1.64) for the free-wheeling and build-on instruction conditions, respectively, q_FH (5, 116) = 4.15, p < .05, and among nominal groups, M = 15.81 (SD = 2.12) versus M = 13.95 (SD = 2.39) for the free-wheeling and build-on instruction conditions, respectively, q_FH (5, 116) = 4.60, p < .05.³ Further, there was no difference between the interacting and nominal groups with respect to the number of semantic categories sampled when given either the build-on, q_FH (5, 116) = 1.17, ns, or free-wheeling instructions, q_FH (5, 116) = 1.61, ns. Thus, Hypothesis 1 was supported.

Hypothesis 2 predicted that when Rules 3 and 4 were presented simultaneously (Standard condition), nominal groups would generate ideas from more semantic categories than would interacting groups, and that as a consequence nominal groups would tend to perform as if they were given only Rule 3 (the free-wheeling condition) whereas interacting groups would tend to perform as if they were given only Rule 4 (the free-wheeling condition). As predicted, controlling for the total number of ideas generated, nominal groups in the standard instruction condition (M = 16.07, SD = 1.93) drew their ideas from a larger number of semantic categories than did interacting groups in the standard instruction condition (M = 13.63, SD = 1.19), q_FH (5, 116) = 5.97, p < .01. Further, and as can be seen in Figure 1, nominal groups in the standard instruction condition generated their ideas from approximately the same number of semantic categories as did those given the free-wheeling instructions, q_FH (5, 116) = 0.64, ns, and significantly more than those given the build-on instructions, q_FH (5, 116) = 5.25, p < .05. The reverse was true for interacting groups, where those given the standard instructions offered ideas from approximately the same number of semantic categories as did those given the build-on instructions, q_FH (5, 116) = 0.35, ns, and from marginally fewer categories than those given the free-wheeling instructions, q_FH (5, 116) = 3.72, p < .10. In sum, with respect to the number of semantic categories sampled, when participants were given all four of the traditional brainstorming rules (the standard instructions condition), those who worked in interacting groups performed as if they were pursuing a build-on goal (Rule 4), whereas those who worked individually performed as if they were pursuing a free-wheeling goal (Rule 3). Thus, Hypothesis 2 was supported.

Number of Highly Practical Ideas

We had an open research question about the separate and joint effects of Rules 3 and 4 on idea practicality. Like semantic categories, the number of highly practical ideas suggested by interacting and nominal groups was also significantly correlated with the total number of ideas generated, r (121) = 0.35, p < .05, so here too we controlled for the latter. Because the number of highly practical ideas was a proper subset of the total number of ideas generated, we controlled for the latter simply by analyzing this measure as a proportion score. A 2 × 3 ANOVA yielded only a significant goal instructions main effect, F (2, 117) = 5.20, p < .01, η² = 0.08. Post hoc comparisons revealed that participants in the build-on condition (M = 0.27, SD = 0.11) generated proportionally more highly practical ideas than did those in either the free-wheeling condition (M = 0.21, SD = 0.07), q_FH (2, 117) = 4.34, p < .01, or the Standard instruction condition (M = 0.22, SD = 0.09), q_FH (2, 117) = 3.32, p < .05. No other effects were significant.

Response Format

To gauge whether the results previously reported by Paulus and Yang (2000), Coskun (2005b), and Coskun and Yilmaz (2009; Experiment 2) may have been due to the difference in the way participants recorded their ideas in the interacting versus nominal group conditions of those studies (on small slips of paper in the former, and on a full-size sheets in the latter), we compared the total number of ideas generated by groups in the three conditions of our own experiment that involved the standard brainstorming instructions: (a) interacting groups that wrote their ideas on separate notecards, (b) nominal groups that wrote their ideas on separate notecards, and (c) nominal groups that wrote their ideas on a standard 21.6 x 27.9 cm sheet of paper. The last of these, of course, is our extra-factorial, full sheet nominal group condition.

We predicted that interacting groups (Hypothesis 3) and nominal groups that used notecards (Hypothesis 4) would generate more ideas overall than nominal groups that used letter-size sheets of paper. A one-way ANOVA comparing the total number of ideas generated in these three conditions yielded a significant effect, F (2, 57) = 6.10, p < .005, η² = 0.18. Follow-up comparisons revealed that nominal groups in the full sheet condition generated significantly fewer ideas overall than did either interacting groups, q_FH (2, 57) = 3.26, p < .05, or nominal groups that used notecards, q_FH (2, 57) = 4.84, p < .01 (see Table 1). Consistent with the results reported previously, the latter two conditions did not differ significantly from one another, q_FH (2, 57) = 1.59, ns. Thus, Hypotheses 3 and 4 were both supported. Notably, when interacting and nominal groups both recorded their ideas using notecards, the difference between them disappeared. Importantly, however, interacting groups did not perform worse than nominal ones. These results appear to suggest that the previously reported superiority of interacting over nominal groups using the brainwriting technique may have been due at least in part to the response format confound rather than to the brainwriting procedure per se.

Table 1.

Number of Ideas Generated and Semantic Categories Sampled As a Function of Response Format.

	Number of ideas		Number of categories^a
Response format	M	SD	M	SD
Interacting groups	77.15	27.76	13.57	1.66
Nominal groups: notecards	85.18	17.34	16.02	1.72
Nominal groups: full sheet	60.68	21.55	15.71	1.76

Note: ^aThe means and standard deviations for the number of semantic categories sampled are adjusted for the number of ideas generated.

Finally, to explore the underlying cause of the differences observed in the foregoing analysis, we used a one-way ANCOVA to test the number of semantic categories sampled in these same three conditions, with the total number of ideas generated included as a covariate. This analysis also revealed a significant effect, F (1, 56) = 12.90, p < .001, η² = 0.32. Interestingly, however, follow-up comparisons revealed that while interacting groups sampled ideas from significantly fewer semantic categories than did either the nominal groups that used notecards, q_FH (2, 56) = 6.60, p < .01, or the nominal groups that used a full-size sheet of paper, q_FH (2, 56) = 5.77, p < .01, the two nominal group conditions did not differ significantly from one another q_FH (2, 56) = 0.82, ns.⁴ The fact that there was no significant difference in the number of semantic categories sampled by the two nominal group conditions suggests that the overall response format difference in the total number of ideas generated was likely not due to fixation. A fuller interpretation of this result is offered in the discussion section.

Discussion

This study had two objectives. One was to examine the separate and joint impact of two standard, but seemingly conflicting brainstorming rules on idea generation in interacting and nominal groups. The other was to test whether the superior performance of interacting relative to nominal groups found in several previous studies using a brainwriting technique may have been due to the different response formats employed by groups and individuals in those studies. We address each of these objectives in turn in the two sections below.

The Rules of Brainstorming

Experimental studies comparing the ideational productivity of interacting groups and individuals have typically asked research participants to follow a set of four procedural rules originally proposed by Osborn (1953, 1963). However, two of those rules seem to require rather incompatible cognitive orientations. Rule 3 (free-wheeling) calls for the production of distinctly different and unusual ideas, thus prompting diversity in the ideas generated, whereas Rule 4 (build-on) calls for ideas that combine and improve upon one another, which encourages a degree of similarity in the ideas produced. The present study sought to determine whether interacting groups and individuals resolve this apparent conflict in the same way when the two rules are presented simultaneously at the start of a brainstorming session, using as a frame of reference their ideational performance when these rules are presented separately.

It was found that when Rules 3 and 4 were presented separately, interacting groups and individuals (as nominal groups) responded similarly. Both generated more diverse ideas—ideas drawn from a larger number of semantic categories—in the free-wheeling condition than in the build-on condition. Further, both generated fewer practical ideas in the free-wheeling than in the build-on condition. By contrast, when the two rules were presented simultaneously (i.e., in the standard instruction condition), interacting groups performed as they did in the build-on condition, whereas nominal groups performed as they did in the free-wheeling condition, with the result that interacting groups drew their ideas from fewer semantic categories than did nominal groups. The present study is not the first to suggest that interacting and nominal groups draw their ideas from different numbers of semantic categories when all four brainstorming rules are presented simultaneously (e.g., Brown et al., 1998; Kohn & Smith, 2011; Larey & Paulus, 1999; Ziegler et al., 2000). It is, however, the first to demonstrate that groups are capable of generating ideas from the same number of categories as individuals when the two conflicting brainstorming rules are presented separately, and differentiate themselves only when those two rules are presented together.

Our results thus suggest that interacting groups and individuals pursue somewhat different goals when the build-on and free-wheeling rules are presented together at the beginning of a brainstorming session. Less clear is the reason for this difference. Earlier we suggested several possibilities. One is that the collaborative nature of group brainstorming may convey an implicit demand favoring the build-on rule over the free-wheeling rule, whereas the opposite may hold for those who brainstorm alone. Alternatively, group brainstormers may prefer to contribute ideas that are generally like those that have been contributed by others as doing so provides more opportunities for mutual validation and cohesion-building, whereas individual brainstormers may find greater stimulation in generating ideas that are more distinctly different from one another. A third possibility is that fixation associated with being exposed to the ideas of others may simply make it harder for group brainstormers to adhere to the free-wheeling rule, so that when given the opportunity, they naturally gravitate to the more-easily followed build-on rule. The present study was not designed to test any of these underlying explanations directly, but now that the phenomenon has been clearly demonstrated, doing so seems a worthwhile direction for further research.

More broadly, the perspective taken in this study follows that of Litchfield (2008, 2009), which emphasizes the goal-directed nature of ideation, and that the brainstorming rules originally proposed by Osborn (1953, 1963) are, most fundamentally, assigned goals. Osborn assumed that each of these rules can help to improve the ideational performance of groups, and implicitly, that their benefits cumulate when presented in combination. However, the results of the present study challenge this idea by suggesting that the joint impact of the rules is not necessarily cumulative, and that certain rules may in fact compete with one another. Rules 3 and 4 in particular, when introduced simultaneously, seem to set up conflicting goals for brainstormers. How they deal with this conflict apparently depends on whether they are brainstorming in groups or individually.

In light of the present results, it would seem useful in future research to examine more systematically the potentially interactive effects of the brainstorming rules. Of course, such an approach could quickly get out of hand. To examine all combinations of just the traditional four brainstorming rules, including presenting each in isolation, would require a 16-cell design. Further, there are other rules and goals beyond the traditional four that might reasonably be considered (e.g., Feinberg & Nemeth, 2008; Litchfield, Fan, & Brown, 2011; Paulus et al., 2011). Therefore, a more disciplined, theory-driven approach will be needed to explore potential synergistic and competitive effects among rule sets.

It is also worth exploring whether it might be helpful to stagger the introduction of the rules so that brainstormers have just one goal (or one subset of compatible goals) to pursue at a time. For example, a brainstorming session might be divided into distinct phases, as initially suggested by Osborn (1963), perhaps with the free-wheeling rule emphasized during an initial, divergent brainstorming phase, and the build-on rule emphasized in a separate, convergent brainstorming phase (e.g., de Vreede, Briggs, & Reiter-Palmon, 2010; Kohn et al., 2011). Structuring the overall brainstorming session in this way would presumably push those who brainstorm in groups to generate ideas that are just as diverse as those who brainstorm alone, and so give them a more varied set of ideas to work with once the goal shifts to combining and improving upon the ideas already suggested.

Response Format

A secondary purpose of this study was to test whether the superior performance of interacting relative to nominal brainwriting groups first reported by Paulus and Yang (2000), and replicated by Coskun (2005b), and Coskun and Yilmaz (2009; Experiment 2), may have been due to the different response formats employed by groups and individuals in those studies. The present study too found that interacting groups were superior, but only in comparison to nominal groups that recorded their ideas on full-size sheets of paper. When interacting groups were compared instead to nominal groups that recorded their ideas on notecards, just as the interacting groups did, the effect disappeared.

Interestingly, asking the members of nominal groups to record their ideas on notecards versus full-size sheets of paper did not seem to affect the number of semantic categories they employed: both types of nominal groups sampled their ideas from approximately the same number of categories, and both used more categories than did interacting groups. This result is inconsistent with the idea that the superior performance of interacting relative to nominal groups observed by Paulus and Yang (2000) and others was due to greater fixation in their (full-sheet) nominal group conditions, as fixation should decrease the number of semantic categories used (Kohn & Smith, 2011; Smith, 2003).

Rather, our findings point instead to a motivational explanation for the results of those earlier studies: Continuous exposure to their complete list of already-generated ideas may have reduced the motivation of those in the full sheet nominal condition to continue generating ideas at a rapid pace. We suspect that as they produced more and more ideas, individuals in the full sheet nominal condition, compared to those who wrote their ideas just four-at-a-time on notecards, were more likely to be aware of how many ideas they had generated overall, and as a consequence were more apt to conclude that they had approximately satisfied the goal of generating as many ideas as possible (Rule 2), especially toward the end of the idea generating session. This performance perception is a likely proximal cause of their lower ideational productivity (cf. Campion & Lord, 1982; Matsui, Okada, & Inoshita, 1983; Nijstad et al., 2006). Other possible explanations are that notecard formats might prompt more social comparison (Festinger, 1954), or that being given a stack of cards to complete might prompt different expectations than being given a single blank sheet of paper (Schwarz, 1999). At this point, it is unclear which of these may be driving the current results, and this is worthy of further investigation.

It is worth noting that although the present study did not find an advantage for interacting groups compared to nominal groups that recorded their ideas in the same way (on notecards), neither did it find the more common, opposite result of inferior interacting group performance. Instead, like Coskun (2005a), we found no significant difference between the total number of ideas generated by interacting and nominal groups that all used notecards to record their ideas. This too is a relatively rare finding in the brainstorming literature and suggests that the brainwriting technique may indeed have some of the benefits proposed by Paulus and Yang (2000). Specifically, they argue that brainwriting not only reduces production blocking and social loafing, it promotes cognitive stimulation by encouraging members to both attend to and reflect upon the ideas of others (cf. Dugosh & Paulus, 2005; Dugosh, Paulus, Roland, & Yang, 2000; Kohn et al., 2011). In support of this interpretation, Paulus and Yang (2000) also found that when participants are asked to try to memorize others’ ideas, interacting groups can perform as well as nominal ones, instead of worse. A further important point to note is that the present studies only examined performance during the initial ideation session, whereas Paulus and Yang (2000) demonstrated benefits of group ideation on performance in a subsequent solitary session. In future work, it will be important to test whether such benefits of initial group ideation might still be found when compared to a nominal group condition using the same response format.

Another useful approach for future research would be to track the minute-by-minute rate of idea generation over the full course of a brainstorming session (cf. Coskun & Yilmaz, 2009; Coskun, Paulus, Brown, & Sherwood, 2000; Kohn & Smith, 2011; Paulus & Dzindolet, 1993; Reinig & Briggs, 2008) in order to test the prediction that early on the rate of production will be similar in the full sheet and notecard conditions, but that the rate of production will later drop in the full sheet condition relative to the notecard condition. Unfortunately, the data obtained in the present study were not collected in a manner that permits such an analysis. This data would also permit the investigation of whether brainstorming benefits are more likely to emerge toward the end of sessions when particpants are in need of external stimulation (Dennis et al., 2005). If this is the case, then group ideation benefits may be more likely to be seen in longer sessions, and it is also possible that there may be particular kinds of brainstorming topics that will be more likely to lead to benefits from group ideation and external stimulation.

Limitations

Two limitations of the present study also suggest directions for future research. First, although we interpret our results as indicating that interacting groups and individuals pursued different goals when Rule 3 (free-wheeling) and Rule 4 (build-on) were presented together at the beginning of a brainstorming session, our evidence is somewhat indirect. Specifically, we inferred the goals being pursued from the number of semantic categories found among the ideas generated: a larger number of categories was taken as evidence of a tendency to favor the free-wheeling goal, whereas a smaller number was taken as evidence of a tendency to favor the building-on goal. These interpretations seem reasonable, given the pattern of results obtained from groups and individuals when the free-wheeling and build-on rules were presented separately. Still, it would be beneficial in future research to employ more direct process (as opposed to product) measures when testing hypotheses about the ideational goals pursued by groups and individuals. For example, it would be useful to determine which, if any, of the already-generated ideas participants relied upon when creating each new idea. Kohn et al. (2011) recently tried this using a self-report approach, asking participants to identify which previously generated idea(s) (if any) were the basis for each new idea they proposed. Although not designed to assess the ideational goals pursued when Rules 3 and 4 are presented together, this study nevertheless demonstrated that participants are able to report reliably the source of at least some of their ideas, just as in natural settings informants often seem able to describe the evolution of certain ideas they develop for solving specific problems (e.g., Hargadon & Bechky, 2006). This method might thus be adapted to provide more direct evidence of the degree to which participants pursue build-on versus free-wheeling goals, and may help as well to fill in some of the details about how those goals are actually achieved.

Finally, future research needs to consider the broader motivational context that exists when any form of brainstorming—brainwriting included—is taken up in authentic organizational settings. As typically studied in the laboratory, brainstorming is quite reasonably conceptualized as a purely cooperative activity. When practiced in organizations, however, competitive motives may sometimes creep in. These can occur, of example, as a natural consequence of demographic, disciplinary, and functional background differences that exist among the set of individuals who are asked to generate problem solutions. Such differences are often presumed to carry with them a diversity of task-relevant ideas, information, and perspectives that, when brought to bear on a problem, can benefit the innovativeness of the ideas generated (e.g., Hülsheger, Anderson, & Salgado, 2009; Woodman, Sawyer, & Griffin, 1993). But such differences may also come with diverse allegiance, identity, and turf implications (e.g., Heslin, 2009; Hoever, van Knippenberg, van Ginkel, & Barkema, 2012; Lau & Murnighan, 1998; Pearsall, Ellis, & Evans, 2008), and prompt very different preferences about the sorts of ideas and problem solutions that are appropriate, acceptable, and/or beneficial. To the extent that such differences are strong, cooperative brainstorming sessions may become venues for subtle forms of competitive negotiation, and as such blunt or distort the intended effects of the standard brainstorming rules. A fuller appreciation of the factors that may impact group performance in authentic real-world settings will enrich our understanding of group brainstorming and brainwriting as effective goal-directed ideational activities. In addition to these factors, Heslin (2009) suggested a number of other potential constraints that arise in authentic organizational settings such as personal histories that are worthy of empirical exploration.

Conclusions

After more than 50 years of empirical research on brainstorming, it would appear that there is still much to learn about this technique and its variants, and about ideation in groups more generally. Recognizing that collective idea generation is fundamentally a motivated, goal-directed activity offers a useful framework for moving forward. Gaining a better understanding of the various—and sometime conflicting—motives at play during a brainstorming session, and using that understanding to design better brainstorming procedures, offers some hope of eventually being able to more reliably capitalize on the long-promised benefits of brainstorming in groups.

Footnotes

Acknowledgements

We would like to thank our undergraduate research assistants for their help with data collection and coding.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

Appendix A

Appendix B

Idea Practicality Rating Scale.

Very impractical 1	2	3	4	5	6	Very Practical 7
Build new building(s) New court/field New gym/pool Free tuition Rebuild campus Fire all professors	New transportation modes Free books Free food Remodeling Renovations	New restaurant New bank Cheaper tuition Campus aesthetics More labs and computers Free printing More scholarships School spirit Football team Cheaper housing Buses more often Health insurance New pricy things School jobs Financial aid Sports teams	More/bigger bathrooms More/new windows Classroom sizes Cheaper printing Cheaper books/ bookstore items Teaching Assistant (TA) and professor accents Instruction quality/policies Exams Academic policies Learning/ studying Extracurricular activities Clubs Speakers Entertainment Professors and TAs Research Opportunities	New furniture/Desks Food prices Free food events Advising Commuting Parking Religion Dorm policies Field trips Campus rules and policies	Fixing things (physical & electronic) Landscaping Painting Heating and cooling Facility hours of operation Food options and quality Safety Small convenience things and items Staff Email issues Diversity and equal opportunity	Snow plowing Cleaning and maintenance Recycling Maps and navigation Food court policies Volunteering

Author Biographies

Olga Goldenberg is a doctoral candidate in psychology at the University of Illinois at Chicago. Her research interests are in group creativity and effects of collaboration on learning and performance.

James R. Larson, Jr. is professor and chair of the Department of Psychology at Loyola University Chicago. His research is in the areas of group decision making, problem solving, and performance.

Jennifer Wiley is professor and associate head of the Department of Psychology at the University of Illinois at Chicago. She is a cognitive psychologist investigating creative problem solving, comprehension, and metacomprehension when learning in science.

References

Bandura

Cervone

(1986). Differential engagement of self-reactive influences in cognitive motivation. Organizational Behavior and Human Decision Processes, 38, 92-113. doi:10.1016/0749-5978(86)90028-2

Brown

V. R.

Tumeo

Larey

T. S.

Paulus

P. B.

(1998). Modeling cognitive interaction during brainstorming. Small Group Research, 29, 495-526. doi:10.1177/1046496498294005

Brown

Paulus

P. B.

(1996). A simple dynamic model of social factors in group brainstorming. Small Group Research, 27, 91-114. doi:10.1177/1046496496271005

Camacho

L. M.

Paulus

P. B.

(1995). The role of social anxiousness in group brainstorming. Journal of Personality and Social Psychology, 68, 1071-1080. doi:10.1037//0022-3514.68.6.1071

Campion

M. A.

Lord

R. G.

(1982). A control systems conceptualization of the goal-setting and changing process. Organizational Behavior and Human Performance, 30, 265-287. doi:10.1016/0030-5073(82)90221-5

Collaros

P. A.

Anderson

L. R.

(1969). Effect of perceived expertness upon creativity of members of brainstorming groups. Journal of Applied Psychology, 53, 159-163. doi:10.1037/h0027034

Coskun

(2005a). Cognitive stimulation with convergent and divergent thinking exercises in brainwriting: Incubation, sequence priming, and group context. Small Group Research, 36, 466-498. doi:10.1177/1046496405276475

Coskun

(2005b). The effect of divergent thinking and group composition on idea generation in brainwriting. Turk Psikoloji Dergisi, 20, 25-42.

Coskun

Yilmaz

(2009). A new dynamical model of brainstorming: Linear, nonlinear, continuous (simultaneous) and impulsive (sequential) cases. Journal of Mathematical Psychology, 53, 253-264. doi:10.1016/j.jmp.2009.03.003

10.

Coskun

Paulus

P. B.

Brown

Sherwood

J. J.

(2000). Cognitive stimulation and problem presentation in idea generation groups. Group Dynamics: Theory, Research, and Practice, 4, 307-329. doi:10.1037//1089-2699.4.4.307

11.

de Vreede

G. J.

Briggs

R. O.

Reiter-Palmon

(2010). Exploring asynchronous brainstorming in large groups: A field comparison of serial and parallel subgroups. Human Factors, 52, 189-202. doi:10.1177/0018720809354748

12.

Dennis

A. R.

Pinsonneault

McNamara Hilmer

Barki

Gallupe

Huber

Bellavance

(2005). Patterns in electronic brainstorming. International Journal of e-Collaboration, 1, 38-57. doi:10.4018/978-1-59904-393-7.ch010

13.

Diehl

Stroebe

(1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology, 53, 497-509. doi:10.1037//0022-3514.53.3.497

14.

Diehl

Stroebe

(1991). Productivity loss in idea-generating groups: Tracking down the blocking effect. Journal of Personality and Social Psychology, 61, 392-403. doi:10.1037//0022-3514.61.3.392

15.

Dugosh

K. L.

Paulus

P. B.

(2005). Cognitive and social comparison processes in brainstorming. Journal of Experimental Social Psychology, 41, 313-320. doi:10.1016/j.jesp.2004.05.009

16.

Dugosh

K. L.

Paulus

P. B.

Roland

E. J.

Yang

H. C.

(2000). Cognitive stimulation in brainstorming. Journal of Personality and Social Psychology, 79, 722-735. doi:10.1037//0022-3514.79.5.722

17.

Feinberg

Nemeth

(2008). The “rules” of brainstorming: An impediment to creativity? (Working Paper No. 167-08). Berkeley: University of California, Institute for Research on Labor and Employment. Retrieved from http://www.irle.berkeley.edu/workingpapers

18.

Festinger

(1954). A theory of social comparison processes. Human Relations, 7, 117-140. doi:10.1177/001872675400700202

19.

Goldenberg

Wiley

(2011). Quality, conformity, and conflict: Questioning the assumptions of Osborn’s Brainstorming Technique. Journal of Problem Solving, 3, 96-118.

20.

Guilford

J. P.

(1959). Creativity. American Psychologist, 5, 444-454. doi:10.1037/h0063487

21.

Guilford

J. P.

Hoepfner

(1971). The analysis of intelligence. New York, NY: McGraw-Hill.

22.

Hargadon

A. B.

Bechky

B. A.

(2006). When collections of creatives become creative collectives. Organization Science, 17, 484-500. doi:10.1287/orsc.1060.0200

23.

Hayter

A. J.

(1986). The maximum family wise error rate of Fisher’s least significant difference test. Journal of the American Statistical Association, 81, 1000-1004. doi:10.2307/2289074

24.

Heslin

P. A.

(2009). Better than brainstorming? Potential contextual boundary conditions to brainwriting for idea generation in organizations. Journal of Occupational and Organizational Psychology, 82, 129-145. doi:10.1348/096317908X285642

25.

Hoever

I. J.

van Knippenberg

van Ginkel

W. P.

Barkema

H. G.

(2012). Fostering team creativity: Perspective taking as key to unlocking diversity’s potential. Journal of Applied Psychology, 97, 982-996. doi:10.1037/a0029159

26.

Hülsheger

U. R.

Anderson

Salgado

J. F.

(2009). Team-level predictors of innovation at work: A comprehensive meta-analysis spanning three decades of research. Journal of Applied Psychology, 94, 1128-1145. doi:10.1037/a0015978

27.

Jansson

D. G.

Smith

S. M.

(1991). Design fixation. Design Studies, 12, 3-11. doi:10.1016/0142-694X(91)90003-F

28.

Karau

S. J.

Williams

K. D.

(1993). Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology, 65, 681-706. doi:10.1037//0022-3514.65.4.681

29.

Kirk

R. E.

(1999). Statistics: An introduction (4th ed.). New York, NY: Harcourt Brace.

30.

Kohn

N. W.

Smith

S. M.

(2011). Collaborative fixation: Effects of others’ ideas on brainstorming. Applied Cognitive Psychology, 25, 359-371. doi:10.1002/acp.1699

31.

Kohn

N. W.

Paulus

P. B.

Choi

Y. H.

(2011). Building on the ideas of others: An examination of the idea combination process. Journal of Experimental Social Psychology, 47, 554-561. doi:10.1016/j.jesp.2011.01.004

32.

Larey

T. S.

Paulus

P. B.

(1999). Group preference and convergent tendencies in small groups: A content analysis of group brainstorming performance. Creativity Research Journal, 12, 175-184. doi:10.1207/s15326934crj1203_2

33.

Larson

J. R.

Jr. (2010). In search of synergy in small group performance. New York, NY: Psychology Press.

34.

Lau

D. C.

Murnighan

J. K.

(1998). Demographic diversity and faultlines: The compositional dynamics of organizational groups. Academy of Management Review, 23, 325-340. doi:10.2307/259377

35.

Litchfield

R. C

Fan

Brown

V. R.

(2011). Directing idea generation using brainstorming with specific novelty goals. Motivation and Emotion, 35, 135-143. doi:10.1007/s11031-011-9203-3

36.

Litchfield

R. C.

(2008). Brainstorming reconsidered: A goal-based view. Academy of Management Review, 33, 649-668. doi:10.5465/AMR.2008.32465708

37.

Litchfield

R. C.

(2009). Brainstorming rules as assigned goals: Does brainstorming really improve idea quantity? Motivation and Emotion, 33, 25-31. doi:10.1007/s11031-008-9109-x

38.

Locke

E. A.

Latham

G. P.

(1990). A theory of goal setting and task performance. New York, NY: Prentice-Hall.

39.

Locke

E. A.

Latham

G. P.

(2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57, 705-717. doi:10.1037//0003-066X.57.9.705

40.

Locke

E. A.

Smith

K. G.

Erez

Chah

Schaffer

(1994). The effects of intra-individual goal conflict on performance. Journal of Management, 20, 67-91. doi:10.1177/014920639402000104

41.

Matsui

Okada

Inoshita

(1983). Mechanism of feedback affecting task performance. Organizational Behavior and Human Performance, 31, 114-122. doi:10.1016/0030-5073(83)90115-0

42.

Mullen

Johnson

Salas

(1991). Productivity loss in brainstorming groups: A meta-analytic integration. Basic and Applied Social Psychology, 12, 332-340. doi:10.1207/s15324834basp1201_1

43.

Nemeth

C. J.

Ormiston

(2007). Creative idea generation: Harmony versus stimulation. European Journal of Social Psychology, 37, 524-535. doi:10.1002/ejsp.373

44.

Nemeth

C. J.

Personnaz

Goncalo

J. A.

(2004). The liberating role of conflict in group creativity: A study in two countries. European Journal of Social Psychology, 34, 365-374. doi:10.1002/ejsp.210

45.

Nijstad

B. A.

Stroebe

(2006). How the group affects the mind: A cognitive model of idea generation in groups. Personality and Social Psychology Review, 10, 186-213. doi:10.1207/s15327957pspr1003_1

46.

Nijstad

B. A.

Stroebe

Lodewijkx

H. F. M.

(2002). Cognitive stimulation and interference in groups: Exposure effects in an idea generation task. Journal of Experimental Social Psychology, 38, 535-544. doi:10.1016/S0022-1031(02)00500-0

47.

Nijstad

B. A.

Stroebe

Lodewijkx

H. F. M.

(2003). Production blocking and idea generation: Does blocking interfere with cognitive processes? Journal of Experimental Social Psychology, 39, 531-548. doi:10.1016/S0022-1031(03)00040-4

48.

Nijstad

B. A.

Stroebe

Lodewijkx

H. F. M.

(2006). The illusion of group productivity: A reduction of failures explanation. European Journal of Social Psychology, 36, 31-48. doi:10.1002/ejsp.295

49.

Orne

M. T.

(1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776-783. doi:10.1037/h0043424

50.

Osborn

A. F.

(1953). Applied imagination: Principles and procedures of creative thinking. New York, NY: Scribners.

51.

Osborn

A. F.

(1963). Applied imagination: Principles and procedures of creative problem-solving (3rd revised ed.). New York, NY: Scribners.

52.

Paulus

P. B.

Brown

V. R.

(2007). Toward more creative and innovative group idea generation: A cognitive-social motivational perspective of brainstorming. Social and Personality Compass, 1, 248-265. doi:10.1111/j.1751-9004.2007.00006.x

53.

Paulus

P. B.

Dzindolet

M. T.

(1993). Social influence processes in group brainstorming. Journal of Personality and Social Psychology, 64, 575-586. doi:10.1037//0022-3514.64.4.575

54.

Paulus

P. B.

Yang

H. C.

(2000). Idea generation in groups: A basis for creativity in organizations. Organizational Behavior and Human Decision Processes, 82, 76-87. doi:10.1006/obhd.2000.2888

55.

Paulus

P. B.

Kohn

N. W.

Arditti

L. E.

(2011). Effects of quantity and quality instructions on brainstorming. Journal of Creative Behavior, 45, 38-46. doi:10.1002/j.2162-6057.2011.tb01083.x

56.

Pearsall

M. J.

Ellis

A. P. J.

Evans

J. M.

(2008). Unlocking the effects of gender faultlines on team creativity: Is activation the key? Journal of Applied Psychology, 93, 225-234. doi:10.1037/0021-9010.93.1.225

57.

Reinig

B. A.

Briggs

R. O.

(2008). On the relationship between idea-quantity and idea-quality during ideation. Group Decision and Negotiation, 17, 403-420. doi:10.1007/s10726-008-9105-2

58.

Reinig

B. A.

Briggs

R. O.

Nunamaker

J. F.

(2007). On the measurement of ideation quality. Journal of Management Information Systems, 23, 143-161. doi:10.2753/MIS0742-1222230407

59.

Rietzschel

E. F.

Nijstad

B. A.

Stroebe

(2006). Productivity is not enough: A comparison of interactive and nominal brainstorming on idea generation and selection. Journal of Experimental Social Psychology, 42, 244-251. doi:10.1016/ j.jesp.2005.04.005

60.

Rietzschel

E. F.

Nijstad

B. A.

Stroebe

(2007). Relative accessibility of domain knowledge: The effects of knowledge activation on the quantity and originality of generated ideas. Journal of Experimental Social Psychology, 43, 933-946. doi:10.1016/j.jesp.2006.10.014

61.

Runco

M. A.

Illies

J. J.

Eisenman

(2005). Creativity, originality, and appropriateness: What do explicit instructions tell us about their relationships? Journal of Creative Behavior, 39, 137-148. doi:10.1002/j.2162-6057.2005.tb01255.x

62.

Rundus

(1973). Negative effects of using list items as recall cues. Journal of Verbal Learning and Verbal Behavior, 12, 43-50. doi:10.1016/S0022-5371(73)80059-3

63.

Schwarz

(1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93-105. doi:10.1037//0003-066X.54.2.93

64.

Smith

S. M.

(2003). The constraining effects of initial ideas. In Paulus

P. B.

Nijstad

B. A.

(Eds.), Group creativity: Innovation through collaboration (pp. 15-31). Oxford, UK: Oxford University Press. doi:10.1093/acprof:oso/9780195147308.003.0002

65.

Smith

S. M.

Ward

T. B.

Schumacher

J. S.

(1993). Constraining effects of examples in a creative generation task. Memory & Cognition, 21, 837-845. doi:10.3758/BF03202751

66.

Stasser

Titus

(1985). Pooling of unshared information in group decision making: Biased information sampling during discussion. Journal of Personality and Social Psychology, 48, 1467-1478. doi:10.1037//0022-3514.48.6.1467

67.

VanGundy

A. B.

(1981). Techniques for structured problem solving. New York, NY: Van Nostrand Reinhold.

68.

Ward

T. B.

(1994). Structured imagination: The role of category structure in exemplar generation. Cognitive Psychology, 27, 1-40. doi:10.1006/cogp.1994.1010

69.

Wiley

(1998). Expertise as mental set: The effects of domain knowledge in creative problem solving. Memory & Cognition, 26, 716-730. doi:10.3758/BF03211392

70.

Williams

K. D.

Harkins

S. G.

Karau

S. J.

(2003). Social performance. In Hogg

M. A.

Cooper

(Eds.), Sage handbook of social psychology (pp. 327-346). Thousand Oaks, CA: SAGE. doi:10.4135/9781848608221.n13

71.

Wittenbaum

G. M.

Bowman

J. M.

(2004). A social validation explanation for mutual enhancement. Journal of Experimental Social Psychology, 40, 169-184. doi:10.1016/S0022-1031(03)00091-X

72.

Wittenbaum

G. M.

Hubbell

A. P.

Zuckerman

(1999). Mutual enhancement: Towards an understanding of the collective preference for shared information. Journal of Personality and Social Psychology, 77, 967-978. doi:10.1037//0022-3514.77.5.967

73.

Woodman

R. W.

Sawyer

J. E.

Griffin

R. W.

(1993). Toward a theory of organizational creativity. Academy of Management Review, 18, 293-321. doi:10.2307/258761

74.

Ziegler

Diehl

Zijlstra

(2000). Idea production in nominal and virtual groups: Does computer-mediated communication improve group brainstorming? Group Processes and Intergroup Relations, 3, 141-158. doi:10.1177/1368430200032003