The Cost of Collaboration

Abstract

Prior investigators have asserted that certain group characteristics cause group members to disregard outside information and that this behavior leads to diminished performance. We demonstrate that the very process of making a judgment collaboratively rather than individually also contributes to such myopic underweighting of external viewpoints. Dyad members exposed to numerical judgments made by peers gave significantly less weight to those judgments than did individuals working alone. This difference in willingness to use peer input was mediated by the greater confidence that the dyad members reported in the accuracy of their own estimates. Furthermore, dyads were no better at judging the relative accuracy of their own estimates and the advisor’s estimates than individuals were. Our analyses demonstrate that, relative to individuals, dyads suffered an accuracy cost. Specifically, if dyad members had given as much weight to peer input as individuals working alone did, then their revised estimates would have been significantly more accurate.

Keywords

decision making judgment dyads advice taking egocentric discounting

Although most major decisions in domains as diverse as business, public policy, and international relations are made collaboratively, examples of poor joint judgment abound. And because the failure of many collaborative decisions can be traced to rejection of outside information (Ancona & Caldwell, 1992), researchers have closely investigated which group characteristics foster myopic disregard of alternative viewpoints (Ancona, 1990; Cronin & Weingart, 2007; Janis, 1982; Kane, Argote, & Levine, 2005; Katz, 1982). Whereas such research has identified several factors that increase or decrease this tendency (Ancona, 1990), we pose a novel question: Does the mere act of collaboration discourage the use of outside input?

Research on quantitative judgment has shown that individuals often improve their decision making by integrating outside input into their judgments, in part because they can determine the relative accuracy of their own and other people’s judgments (Soll & Larrick, 2009), and in part because aggregating independently made judgments reduces average error (Armstrong, 2001; Bonaccio & Dalal, 2006; Einhorn & Hogarth, 1978). However, this past work has focused exclusively on individual-level processes (Gino & Moore, 2007; Gino & Schweitzer, 2008; Mannes, 2009; Soll & Mannes, 2011). Although accepting advice can improve performance, the question of whether individual decision makers or collaborators are more willing to accept advice remains unanswered.

It seems possible that collaborative decisions would be more amenable to revision than ones made by individuals working alone. Individuals may underweight peer input because they are too attached to their own views (Harvey & Harries, 2004; Lord, Ross, & Lepper, 1979). In contrast, because collaboration requires individuals to cede prior opinions to reach consensus, collaborators may be less satisfied with joint judgments and therefore more open to revising them. Additionally, collaborative judgment requires discussion, which might enhance decision makers’ recognition that, on average, the judgments of peers are as accurate as their own.

Alternatively, collaborators may devalue outside input more than individual decision makers do. Research on brainstorming has shown that discussion and collaboration can increase the pressure to conform to group members’ opinions (Goncalo & Staw, 2006). Furthermore, collaborators may choose to disregard outside advice to preserve and reinforce feelings of cohesion and rapport (Ancona, 1990; Janis, 1982; Katz, 1982). However, we propose one additional possibility: that the mere act of collaborating enhances confidence in judgment and thereby limits receptivity to outside advice.

The Role of Confidence

People believe that, relative to working alone, working collaboratively allows for acquisition of more resources, greater avoidance of negative outcomes, and greater likelihood of goal achievement (Moreland, 1987). Collaboration increases efficacy beliefs (i.e., beliefs about one’s own ability to produce a desired result), including confidence in decision making (Forsyth, 1999; Park & Hinsz, 2006) and beliefs about overall capability (Stroebe, Diehl, & Abakoumkin, 1992). We thus hypothesized that collaborating on a judgment task would promote confidence in the accuracy of the judgments produced.

In the advice-taking literature, greater confidence has been linked to a greater propensity to disregard outside input (Gino & Moore, 2007; Harvey & Fischer, 1997; Soll & Larrick, 2009). High confidence in one’s own estimates might provide a cue that advice is not needed or appropriate (Sniezek & Van Swol, 2001). And although it may be logical to adhere to judgments one feels confident about, it would be folly to overlook the possibility that one’s peers have similar reasons to feel confident. Therefore, we hypothesized that collaborators would feel more confident than individuals in the accuracy of their initial judgments and would therefore be less open to peer input. We further predicted that this would have deleterious effects for the accuracy of final judgments.

The Benefits and Costs of Collaborative Judgment

The higher confidence experienced by collaborators relative to individuals working alone may be partly justified. The idea that two heads are better than one has received substantial support from experiments revealing the benefit of statistical aggregation when independently made judgments are combined (Soll & Larrick, 2009; Surowiecki, 2004; for a review, see Bonaccio & Dalal, 2006). However, much of this benefit is lost when estimators influence each other’s judgments (Lorenz, Rauhut, Schweitzer, & Helbing, 2011). Therefore, the accuracy of judgments made jointly may be lower than the accuracy of aggregated independent judgments.

Even if collaborators produce highly accurate initial estimates, collaboration may come with an accuracy cost when judgments require outside input. If collaborators integrate less outside advice into their decision making than individuals working alone do, they may lose their initial accuracy advantage. The magnitude of this potential cost can be measured by comparing the accuracy of final collaborative estimates with the accuracy that could have been achieved if collaborators had yielded as much to outside input as individual decision makers had. We predicted that collaborators’ lower receptivity to outside advice would lower their accuracy relative to what they could have achieved had they behaved like individuals working alone. Given that joint decision making requires greater human capital, money, and time than individual decision making does, this lost accuracy would be nontrivial.

In the research reported here, participants made initial judgments and then were given outside advice and the opportunity to revise their judgments. We compared the willingness of individuals working alone and of individuals working with a dyad partner to revise their judgments, their confidence in their initial judgments, and the accuracy of their revised estimates.

Method

Participants

Participants (N = 252) were members of a university research pool and were compensated $10. Depending on their performance, they could receive up to a $30 bonus for completing each of two estimation rounds. This bonus decreased by $1 for each percentage point that any given estimate deviated from the truth.

Design

In this study, we used a 2 × 2 design, manipulating whether individuals made estimates alone or in dyads (judge type) and whether the peer advice came from an individual or a dyad (advisor type). This design allowed us to assess whether the type of judge or type of advisor influenced the use of peer input and influenced accuracy.

Procedure

Participants sat in a partitioned room and answered nine questions related to U.S. geography, demography, and commerce (see Table 1). We worded the questions to solicit percentage estimates so that responses could be combined easily across items.

Table 1.

Estimation Items and Correct Answers

Question	Answer
What percentage of Americans own pets?	63.0%
What percentage of members of Congress are Catholic?	30.1%
In the 2008 presidential election, what percentage of voting-age citizens voted?	64.0%
What percentage of students who entered the high school class of 2002 left high school with a regular diploma?	71.0%
In the United States, what percentage of homeless men are veterans?	40.0%
What percentage of all U.S. undergraduates received some type of financial aid in 2007–08?	66.0%
What percentage of the population in the District of Columbia is White?	38.5%
In 2008, what percentage of corporate officers in Fortune 500 companies were women?	15.7%
What percentage of homes with an iPad have two or more tablets?	17.0%

All participants completed two rounds of estimation: one round in which they made initial estimates and a second in which they could revise their estimates. Participants were randomly assigned to make initial estimates either individually or jointly through discussion with a dyad partner. They were then provided with a set of estimates made by a peer advisor—either another individual or another dyad in the study—and given the option to revise their initial estimates to any degree they felt appropriate.

Variables and analyses

Yielding

To quantify how much participants yielded to advisor input, for each item we divided the amount of the adjustment from the initial estimate by the total distance between the initial estimate and the advisor’s initial estimate.¹ Following methods used in prior research (Bonaccio & Dalal, 2006), we winsorized this measure to be between 0% and 100% and excluded from analysis the 7% of observations on which participants offered the same initial estimate as their advisor.

Confidence

For each item, participants reported their confidence that their estimate fell within 10 percentage points of the correct answer, using a 5-point scale from 1 (not at all confident) to 5 (extremely confident). Although all participants reported their own confidence ratings individually, in the dyad condition we averaged partners’ confidence ratings to arrive at a single rating for each dyad for each item.

Modeling

We used hierarchical linear modeling in STATA (Statacorp, College Station, TX) because each participant (whether working alone or with a partner) provided estimates for nine items. This approach allowed us to control for nonindependence of the multiple observations provided by each participant while maximizing statistical power. For each model, we entered confidence and yielding to peer input as item-level (Level 1) variables and the two independent variables (judge type and advisor type; 0 = individual, 1 = dyad) as participant-level (Level 2) variables.

Results

Use of peer input

Overall, participants yielded an average of 25.8% (SD = 29.5%) to advice from individuals and an average of 26.9% (SD = 30.8%) to advice from dyads (Table 2). Thus, there was no effect of advisor type on yielding (b = 0.01, z = 0.60, n.s.). As predicted, participants working collaboratively yielded significantly less (M = 19.5%, SD = 27.3) than did participants working individually (M = 32.3%, SD = 31.1%), (b = −0.12, z = −6.42, p < .001). There was no significant interaction between judge type and advisor type (b = −0.01, z = −0.26, n.s.).

Table 2.

Mean Yielding, Confidence, and Estimation Error as a Function of Condition

				Estimation error (percentage points)
Condition	n	Distance yielded (%)	Confidence in initial estimates	Initial estimates	Revised estimates
Dyad judge
Dyad advisor	38 dyads	19.9 (28.4)	2.67 (0.78)	39 (45)	34 (41)
Individual advisor	42 dyads	19.1 (26.4)	2.67 (0.77)	41 (51)	37 (44)
Individual judge
Dyad advisor	42	33.2 (31.5)	2.33 (0.88)	43 (49)	35 (40)
Individual advisor	50	31.6 (30.8)	2.22 (0.99)	47 (55)	37 (45)

Note: Standard deviations are shown in parentheses. Participants reported their confidence that their estimates were accurate within 10 percentage points on a scale from 1 (not at all confident) to 5 (extremely confident).

The mediating role of confidence

Participants who made estimates collaboratively were more confident in their initial estimates (M = 2.67, SD = 0.78) than were participants who made estimates individually (M = 2.27, SD = 0.94), b = 0.397, z = 4.30, p < .001 (Table 2). When participants were more confident in their initial estimates, they yielded less (b = −0.069, z = −7.14, p < .001).

To test whether confidence mediated the effect of judge type on yielding, we next regressed yielding to peer input on judge type, advisor type, and initial confidence. We again found a significant effect of confidence (b = −0.059, z = −6.19, p < .001) and no effect of advisor type (b = 0.016, z = 0.78, n.s.). We also found that the effect of judge type (b = −0.106, z = −5.20, p < .001) was significant but decreased by the addition of the mediator to the model. To test the significance of the indirect effect of judge type on yielding via confidence, we used the Monte Carlo bootstrapping method (Selig & Preacher, 2008). The resulting 95% confidence interval for the indirect effect did not contain zero (lower bound: −0.04, upper bound: −0.01). This confirms that confidence significantly mediated the effect of judge type on yielding.

Judging relative accuracy of own and advisor’s estimates

We measured estimation error as absolute percentage-point deviation from the correct answer. Participants who made their initial estimates working with a partner showed marginally less error (M = 40.4 percentage points, SD = 0.02) than did participants who made their initial estimates working alone (M = 45.3 percentage points, SD = 0.02), b = −0.05, z = −1.89, p = .06.

Were dyads superior to individuals at identifying and giving greater weight to those outside estimates that were more accurate than their own estimates? To address this question, we coded the relative accuracy of own and advisor’s estimates (+1 = own estimate more accurate, −1 = own estimate less accurate) and entered this relative-accuracy variable, along with the two independent variables (i.e., judge type and advisor type) and their interactions with relative accuracy, into a model predicting yielding.

The model revealed a significant effect of relative accuracy on yielding (b = −0.05, z = −3.65, p < .01). Participants yielded less when their own estimates were more accurate than those of the advisor than when their own estimates were less accurate than those of their advisor. We again observed that dyads yielded significantly smaller amounts than individuals did (b = −0.129, z = −6.42, p < .001), irrespective of advisor type (b = 0.012, z = 0.58, n.s.). However, neither judge type nor advisor type interacted with relative accuracy to predict yielding (judge type: b = 0.02, z = 1.45, n.s.; advisor type: b = 0.018, z = 1.16, n.s.). Although, on average, participants yielded more to estimates that were relatively more accurate than they did to estimates that were relatively less accurate, dyads were no better at this task than individuals.

The cost of collaboration

Given that dyad members gave significantly less weight to peer input than individuals working alone did and were no wiser in determining whether outside estimates were more or less accurate than their own, it is perhaps not surprising that in offering their revised estimates, they lost their initial accuracy advantage. Neither judge type (b = −0.004, z = −0.18, n.s.) nor advisor type (b = −0.02, z = −0.92, n.s.) significantly influenced the accuracy of the revised estimates. Dyads’ final estimates were no more accurate than those offered by individuals working alone (Table 2).

In order to more directly assess whether dyad members paid a price for weighting peer input less than individuals working alone did, we calculated hypothetical estimates that dyad members could have produced had they yielded to input as much as individuals did (i.e., an additional 12.8 percentage points per item). Those hypothetical revised estimates would have been significantly more accurate than dyads’ actual revised estimates (dyad judge and individual advisor: b = 0.02, z = 2.68, p < .01; dyad judge and dyad advisor: b = 0.02, z = 3.21, p < .01). Thus, dyad members in each condition paid a significant accuracy cost for failing to yield as much as individuals working alone did.

Discussion

Collaboration is not free. Greater time, money, and effort go into making judgments collaboratively than into making them alone. People often collaborate on the assumption that the resulting decisions will be superior to decisions made individually. In our study, dyads were more reluctant than individuals working alone to revise their judgments, and as a result, their revised estimates were less accurate than they could have been had dyad members been more willing to accept peer input. In fact, revised judgments made with the combined inputs of four individuals (i.e., dyad judge and dyad advisor) were no more accurate than those made with three inputs (e.g., one dyad judge and one individual advisor) or even two (i.e., individual judge and individual advisor). This unwillingness to integrate peer input was explained by dyad members’ greater confidence in their estimates. Dyads’ confidence was somewhat warranted given the marginally greater accuracy of their initial estimates, yet it still proved detrimental to the accuracy of their revised estimates. Furthermore, dyads were no better than individuals at giving weight to outside estimates that were more accurate than their own.

In prior research, feelings of confidence or efficacy have been shown to improve performance (Bandura, 1977). However, new research suggests that this may not be the case when individuals or groups are engaged in a novel task in which feelings of high efficacy may inhibit the exploration that results in improved outcomes (Goncalo, Polman, & Maslach, 2010; Moore & Healy, 2008). The research reported here shows that confidence may also reduce the extent to which decision makers consider novel information.

A large literature shows that knowledge transfer is difficult in organizations because groups are resistant to outside information (Kane et al., 2005). This prior work suggests that the quality of collaboration—not the mere act of collaborating—explains why members are reluctant to change their minds. Our study suggests that collaborators may resist incorporating outside input in part because the collaborative process itself increases confidence about the accuracy of one’s own responses, which in turn can minimize some of the benefits of aggregating judgments.

Prior research has suggested that groups may be self-attentive and disregard outside information because of high levels of cohesion (Ancona & Caldwell, 1992). To address this alternative explanation, at the end of the task we measured the extent to which dyads felt cohesive. Supplementary analyses revealed that cohesion within dyads was not correlated with yielding. This finding suggests that rather than wishing to maintain cohesion, dyads may reject outside information simply because they do not believe it adds value.

Many of society’s most important decisions are made collaboratively, following the intuition that “two heads are better than one.” Every aspect of law, policy, and corporate governance relies on the ability of individuals to maximize judgments’ effectiveness through collaboration. Our study demonstrated that collaborators’ reluctance to integrate external input into their decisions may substantially impair their ability to achieve their goals.

Footnotes

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Notes

References

Ancona

D. G.

(1990). Outward bound: Strategies for team survival in an organization. Academy of Management Journal, 33, 334–365.

Ancona

D. G.

Caldwell

D. F.

(1992). Demography and design: Predictors of new product team performance. Organization Science, 3, 321–341.

Armstrong

(2001). Combining forecasts. In Armstrong

J. S.

(Ed.), Principles of forecasting: A handbook for researchers and practitioners (pp. 417–440). Norwell, MA: Kluwer Academic.

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215.

Bonaccio

Dalal

R. S.

(2006). Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences. Organizational Behavior and Human Decision Processes, 101, 127–151.

Cronin

M. A.

Weingart

L. R.

(2007). Representational gaps, information processing, and conflict in functionally diverse teams. Academy of Management Review, 32, 761–773.

Einhorn

Hogarth

(1978). Confidence in judgment: Persistence of the illusion of validity. Psychological Review, 85, 395–416.

Forsyth

(1999). Group dynamics (3rd ed.). Belmont, CA: Wads-worth.

Gino

Moore

(2007). Effects of task difficulty on use of advice. Journal of Behavioral Decision Making, 20, 21–35.

10.

Gino

Schweitzer

M. E.

(2008). Blinded by anger or feeling the love: How emotions influence advice taking. Journal of Applied Psychology, 93, 1165–1173.

11.

Goncalo

J. A.

Polman

Maslach

(2010). Can confidence come too soon? Collective efficacy, conflict and group performance over time. Organizational Behavior and Human Decision Processes, 113, 13–24.

12.

Goncalo

J. A.

Staw

B. M.

(2006). Individualism-collectivism and group creativity. Organizational Behavior and Human Decision Processes, 100, 96–109.

13.

Harvey

Fischer

(1997). Taking advice: Accepting help, improving judgment, and sharing responsibility. Organizational Behavior and Human Decision Processes, 70, 117–133.

14.

Harvey

Harries

(2004). Effects of judges’ forecasting on their later combination of forecasts for the same outcomes. International Journal of Forecasting, 20, 391–409.

15.

Janis

(1982). Groupthink. Boston, MA: Houghton Mifflin.

16.

Kane

A. A.

Argote

Levine

J. M.

(2005). Knowledge transfer between groups via personnel rotation: Effects of social identity and knowledge quality. Organizational Behavior and Human Decision Processes, 96, 56–71.

17.

Katz

(1982). The effects of group longevity on project communication and performance. Administrative Science Quarterly, 27, 81–104.

18.

Lord

C. G.

Ross

Lepper

M. R.

(1979). Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37, 2098–2109.

19.

Lorenz

Rauhut

Schweitzer

Helbing

(2011). How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences, USA, 108, 9020–9025.

20.

Mannes

A. E.

(2009). Are we wise about the wisdom of crowds? The use of group judgments in belief revision. Management Science, 55, 1267–1279.

21.

Moore

Healy

(2008). The trouble with overconfidence. Psychological Review, 115, 502–517.

22.

Moreland

(1987). The formation of small groups. Review of Personality and Social Psychology, 8, 80–110.

23.

Park

E. S.

Hinsz

V. B.

(2006). “Strength and safety in numbers”: A theoretical perspective on group influences on approach and avoidance motivation. Motivation and Emotion, 30, 135–142.

24.

Selig

J. P.

Preacher

K. J.

(2008, June). Monte Carlo method for assessing mediation: An interactive tool for creating confidence intervals for indirect effects [Computer software]. Available from http://quantpsy.org/medmc/medmc.htm

25.

Sniezek

J. A.

Van Swol

L. M.

(2001). Trust, confidence, and expertise in a judge-advisor system. Organizational Behavior and Human Decision Processes, 84, 288–307.

26.

Soll

J. B.

Larrick

R. P.

(2009). Strategies for revising judgment: How (and how well) people use others’ opinions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 780–805.

27.

Soll

J. B.

Mannes

A. E.

(2011). Judgmental aggregation strategies depend on whether the self is involved. International Journal of Forecasting, 27, 81–102.

28.

Stroebe

Diehl

Abakoumkin

(1992). The illusion of group effectivity. Personality and Social Psychology Bulletin, 18, 643–650.

29.

Surowiecki

(2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. New York, NY: Doubleday.