Collective Intelligence and Selecting the NCAA Men’s Basketball Tournaments

Abstract

Every year millions of people fill out brackets, trying to accurately predict the outcome of the NCAA Men’s Basketball March Madness tournament. This study examines how collective swarm intelligence might impact these choices in small groups. Rather than working by themselves, groups of people came together to combine their knowledge and opinions and pick brackets collectively. It is generally agreed that collective intelligence is effective in decision-making. However, how and why collective intelligence augments performance has not been totally agreed upon, and the theoretical explanations have been elusive. This study examines groups that are either highly dedicated or high in expertise to see whether they perform differently based on these dimensions.

Keywords

Collective intelligence audience communication

Introduction

Every year millions of people fill out brackets, trying to accurately predict the outcome of the NCAA Men’s Basketball March Madness tournament. The conventional wisdom would be that experts in this field and those who are highly dedicated fans of NCAA Men’s Basketball would outperform nonexperts and those who were less dedicated to the sport. However, this is not necessarily the case. Despite their best efforts, very few people, if any, choose all of the correct winners. Indeed, put into perspective, those who are basketball fans are estimated to only have a one in 128 billion chance to correctly fill out a bracket (Sarno, 2014).

However, will this result be different when those with a deep knowledge of the sport or a dedication to the sport are able to leverage collective intelligence? This study aims to answer this question as, during the 2016 tournament, rather than working by themselves, groups of people came together to combine their knowledge and opinions and pick brackets collectively. Research has shown that collective intelligence confers benefits such that groups can outperform individuals on certain tasks (Bonabeau, 2009; Malone, Laubacher, & Dellarocas, 2009; Wolf, Krause, Carney, Bogart, & Kurvers, 2015). As such, this study examines the make-up and performance of groups of people that came together to leverage their collective intelligence to pick NCAA Tournament brackets, rather than examine group versus individual task performance. Specifically, this study examines the role that expertise and dedication have.

Collective Intelligence

At a fair in England in 1906, Sir Francis Galton asked the crowd to guess the weight of an ox. When Galton analysed the 787 guesses, he found that the average of the estimates was more accurate than the individual guesses. The Galton experiment has been repeated numerous times with similar results and is popularly known as ‘the Wisdom of Crowds’ (Surowiecki, 2005).

Since then, more recent applications of collective intelligence show that it may overlap with intellectual capital (Secundo, Dumay, Schiuma, & Passiante, 2016), and it can be used for outreach for solutions, recruiting participants, bias avoidance, organization (Bonabeau, 2009) and helping radiologists with decision accuracy (Wolf et al., 2015).

Collective intelligence can be defined ‘as groups of individuals doing things collectively that seem intelligent’ (Malone et al., 2009). More specifically, according to Woolley, Chabris, Pentland, Hashmi, and Malone (2010):

Collective intelligence is the inference one draws when the ability of a group to perform one task is correlated with that group’s ability to perform a wide range of other tasks. This kind of collective intelligence is a property of the group itself, not just the individuals in it.

This phenomenon has existed for many years but has shifted recently due to advances in computing and information technology (Malone et al., 2009). Lévy (1997) stated that it makes all knowledge on a topic available at once and allows for conversation and arbitration on the topic. Further, this productivity can circumvent traditional power structures that might otherwise dictate outcomes for that community externally or even within that community.

Regardless of the definition, it is agreed that collective intelligence is effective in decision-making (Bonabeau, 2009; Malone et al., 2009; Wolf et al., 2015). Jenkins (2002) called it a ‘patchwork much greater than the sum of its parts’. However, how and why collective intelligence augments performance has not been totally agreed upon, and the theoretical explanations have been elusive (Bonabeau, 2009).

One set of researchers have described the ‘c factor,’ which suggests that collective intelligence is effective not as an aggregate of the intelligence of the group but rather a result of group composition and interaction (Woolley et al., 2010). Malone et al. (2009) argued that the configuration of a group will help to determine the efficacy of collective intelligence. Taking the explanations in conjunction, it seems that the make-up of the group is a critical element. Using this explanation as a guide, the current study explores explanations of collective intelligence based on group make-up—with special attention to group makeup as related to expertise and dedication and with two concepts that would be important to any decision-making task.

Software for Collective Intelligence

Unanimous AI has developed a freely available online collaborative platform called UNU that enables individuals to work together as a group and leverage their collective intelligence. The UNU platform has been used previously to examine the performance of groups and collective intelligence (Rosenberg, 2015).

The technology underlying the UNU platform is based on the closed-loop system dynamics found in numerous animal and insect societies that work as a group (Eberhart, Palmer, & Kirschenbaum, 2015; Seeley, 2010). Specifically, Unanimous’ technology models the decision-making processes used by swarming honeybees. While human beings cannot replicate the biological and physiological characteristics that make up the honeybee system, the UNU system aims to approximate this. In Unanimous’ parlance, groups of users working together in UNU are referred to as ‘swarms’. Similar to collective intelligence broadly, prior studies have shown that by working in swarms, human groups can outperform their individual members as well as groups taking traditional votes or polls (Seeley, 2010).

As shown in Figure 1, users log-in from a computer to respond to prompts by manipulating a graphical puck to the group’s preferred answer. Users provide input with a mouse around the puck. Each user influences the movement of the puck as a force vector. The puck is used to select the answer. As such, the input from each user is not an individual selection, but instead a negotiation among group members. Users can push and pull the puck in real time. This allows for users to change their preferences and the strength of their preferences over the course of the decision—this is influenced by the distance from the distance from the puck that the users move their mice.

Figure 1.

Source: Unanimous A.I.

Please note that during participation, the users were only able see other users’ inputs through movement of the puck—not their individual mouse movements. This limits social biasing.

Expertise and Dedication

Based on the noted literature, we expect that expertise and dedication are primary mechanisms through which collective intelligence functions. Expertise is defined by two main components, having knowledge and the ability to apply that knowledge (Gobet, 2015). Competence can be defined in many different ways, typically based on a specific domain (Barnett, 1994), and competence has been suggested as a more effective measure than intelligence (McClelland, 1973). In short, competence refers to the ability to perform a task effectively. For the purposes of this article, competence is used to describe the degree of expertise an individual might have.

According to self-determination theory (SDT), people have a psychological need for feelings of competence (Deci & Ryan, 2011; Huta & Ryan, 2010; Ryan & Deci, 2000). Recently, dimensions of SDT have been used to understand how audience members might enjoy content (Oliver et al., 2016; Tamborini, Bowman, Eden, & Grizzard, 2010) and competence plays an integral role. Consequently, competence may be a key variable for how collective intelligence functions as well as how audience members behave—this has been found in sports spectatorship as well (Rogers, Strudler, Decker, & Grazulis, 2017).

H1: When performing collective intelligence tasks, a group with a greater degree of competence will outperform a group with a lower degree of competence.

Dedication refers to self-motivated behaviours to follow guidelines, meet standards and be proactive (Van Scotter & Motowidlo, 1996). Engagement refers to high involvement, use of high energy and a sense of productivity (Maslach & Leiter, 2008). Thus, dedication can be understood in terms of engagement. A high degree of engagement can be understood as satisfying for the person experiencing it, and it also encourages feelings of efficacy for the task at hand (Csikszentmihalyi, 1997).

H2: When performing collective intelligence tasks, groups with a greater degree of dedication will out-perform groups with a lower degree of dedication.

Method

Participants and Procedures

The participants in this study included 118 individuals recruited from a variety of venues. The participants included university students (recruited through classrooms and other outreach efforts) and members of online communities (recruited through posts in online communities). The majority of the participants were male (86.3%), ranging in age from 18 to 68 (M = 30.63, SD = 11.69). In exchange for participation, people were given the opportunity to win US$1000 split evenly among their group.

In the recruitment postings, participants were provided a URL for an online prequestionnaire and were instructed to complete the questionnaire prior to participating in a collective intelligence task. Towards the end of the questionnaire, participants were asked to note available times for them to participate in a collective intelligence task.

Once the prequestionnaire was completed, the researchers assigned participants to groups based on measures in the prequestionnaire. Groups were designed to maximize or minimize target characteristics such that groups high in expertise and low in expertise were formed. The same was done for dedication, making a total of four groups. These groups were invited to participate in the collective intelligence task at a time when a majority of the participants were available.

Task

Based on the prequestionnaire, participants were all familiar with the activity of ‘picking a bracket’ for the NCAA Men’s basketball tournament. The participants in this study were tasked with picking their bracket as a group, rather than individually in order to explore the questions at hand.

At the scheduled time, members of a group logged into UNU using a standard Web browser (e.g., Chrome or Firefox). When the session began, a moderator provided a brief tutorial to ensure all understood how to express their opinion in the software by manipulating their magnets for the greatest effect and moving the puck as a group to a preferred answer.

The moderator then presented the group with the question ‘Who will win: X vs Y’, where X and Y represented opposing teams, and the group arrived at a collective answer by manipulating the magnets on screen. This was done for each game for the entire bracket. This process was repeated for each of the designated groups such that the high expertise group, the low expertise group, the high dedication group and the low dedication group had completed brackets.

Measures

The prequestionnaire assessed expertise and dedication. These variables were used to split the groups for the collective intelligence task such that those ranking the highest in expertise were placed into that group, those ranking the highest in dedication were placed into that group, and so on. Please note that a participant was only included in one group, not in multiple. ANOVAs associated with these variables showed that the respective groups were different in expertise (F[1, 28] = 4.39, p < 0.05, η² = 0.14, with one (M = 3.43, SD = 1.77) perceiving less expertise than the other (M = 5.67, SD = 1.32). Likewise for dedication (F[1, 28]) = 15.45, p < 0.01, η² = 0.36, with one (M = 5.44, SD = 1.02) perceiving less dedication than another (M = 6.15, SD = 0.82).

Expertise

Expertise was assessed via the Ohanian (1990) five-item semantic differential (α = 0.95). This was measured on a seven-point scale. Sample items include ‘Unknowledgeable – Knowledgeable’ and ‘Unskilled—Skilled’ as related to the Tournament.

Dedication

Dedication was measured with a 5-item scale adapted from Salanova, Agut and Peiró (2005; α = 0.85). The measure used a 7-point scale where 0 = Never and 6 = Always. Sample items included ‘I am proud of my Tournament picks when I make them’ and ‘I am enthusiastic about completing a Tournament bracket and following the Tournament’.

Task Performance

Task performance was measured by how many matchups each group picked correctly such that an incorrect pick was scored as 0 and a correct pick was scored as 1. These results were examined by Tournament round and in terms of upsets picked to provide more granularity to the data. An upset was defined as a weaker seeded team chosen over a stronger seeded team.

Results

In order to explore the data broadly and to give an overview of the data, a series of correlations were performed. Expertise was not correlated with round 1 point, round 2 points, round 3 points, and total points for any group. Rounds 5 and 6 were not analysed because there was little to no variance in the data, and thus, meaningful analysis was not viable. There was partial support for H1 and H2 (see Table 1).

Table 1.

Correlations Between Measures

Measure	Expertise	Dedication	Round 1	Round 2	Round 3	Round 4	Upsets Correct	Upsets Total
Expertise		0.55**	−0.01	−0.07	0.05	0.19*	0.24**	0.42**
Dedication			−.02	0.08	0.19*	0.18	0.12	0.25**
Round 1				0.01	0.13	−0.20*	0.80**	−0.23*
Round 2					0.41**	0.19*	0.04	0.23*
Round 3						0.28**	0.02	0.09
Round 4							−0.10	0.32**
Upsets Correct								0.33**
Upsets Total

Source: The authors.

Note: *p < 0.05. **p < 0.001.

Further analysis revealed a significant multivariate effect of the group that participants were placed in, F(3, 26) = 5.88, p < 0.01, Wilks’ Λ = 0.60, η_p² = 0.40. Based on frequency analysis, the group with less expertise was more successful in picking winners in every round except round 4 and they picked fewer upsets correctly. For dedication, the high and low groups were comparable in task performance with a slight advantage to those with less dedication.

Discussion

In an event as popular and difficult to predict as the NCAA Men’s Basketball tournament, it is not surprising that individuals look to collective intelligence solutions with proven track records to improve predictions. The current study provides empirical evidence that when using a collective swarm intelligence platform, expertise and dedication have an influence on performance.

Not surprisingly, expertise was related to more accurate picks in the later rounds but not significantly so in the earlier rounds. The lack of an advantage for experts in picking the early games reflects the expectation that it is easier to accurately pick the outcome of early round games, while later round games are more difficult.

In sum, we interpret these findings to mean that expertise in collective intelligence confers an advantage when the task is more challenging but not necessarily when the task is easier. That is, a group high in expertise is more effective at accruing less obvious points by picking the harder games. Conversely, a group low in expertise can improve their picks by relying on the diverse experience of the members of the group contributing to the collective intelligence. This indicates that the efficacy of collective intelligence might be a combination of the difficulty of the task at hand in conjunction with the expertise of the group.

As for dedication, there was little evidence that dedication conferred any sort of advantage. It is worth noting that dedication and expertise were correlated at r = 0.55 at p < 0.001. Intuitively, it makes sense that if someone is dedicated, he or she may also be an expert; however, the fact that expertise appears to confer a slight advantage while dedication does not suggests that the concepts are distinct.

This study indicates that groups of individuals can look to collective intelligence solutions, such as UNU’s Artificial Swarm Intelligence, as a means of improving performance on difficult tasks. Notably, this is not always the case, and the swarms with less expertise can have performance indistinguishable from those high in expertise. However, this opens up exciting avenues of exploration for areas of study as diverse as medicine and economics as well as lending credence to the popular wisdom that ‘many heads are better than one’.

Understanding the function by which collective intelligence might function was one of the goals of this study. There is tentative evidence that expertise, more so than dedication, helps explain the function of collective intelligence. This echoes previous conceptualizations of collective intelligence such that the ‘who’ of the group (Malone et al., 2009) and the homogeneity of the group (Mataric, 1992) are critical elements to its success. These findings do not, however, fully account for the ‘c factor’ (Woolley et al., 2010), and future studies should focus on more fully analysing what other attributes are critical to collective intelligence. The current study suggests that task difficulty would be a fruitful area of exploration.

Limitations

There are inherent limitations to a study like this. It is critical to acknowledge the unpredictability of the Tournament itself. While this study aimed to examine the effectiveness of collective intelligence on predicting outcome, the history of this event suggests that there are many results that defy logic and the most conventional forms of intelligence. So while we assume expertise to be a key in predictions for this event, it is quite rational to assume that even the most intelligent selections made by a group of experts might be incorrect in practice. In examining only a singular NCAA Tournament, such random events could limit the effectiveness—or at least its measurement—of swarm intelligence and expertise of such intelligence. Using a longitudinal approach over the course of several years of NCAA Tournaments might mitigate that reality.

Beyond the inherent nature and format of the Tournament itself, this study was also somewhat limited by the number of identified expert swarms. There are likely several reasons for that. First, basketball coaches—perhaps the truest of experts—are typically unable and not allowed to officially participate in such pools. Second, experts such as sports reporters are extremely busy during this time period, making it challenging for them to find time to collectively participate in a swarm. Third, some self-perceived experts may not want to share their particular knowledge in a swarm environment, assuming their individual intelligence might be more valuable on its own. Lastly, some sports experts may not want to put themselves ‘on record’ with a collective swarm, as they may perceive this as a threat to their status as an expert. In the future, we hope to recruit a more robust group of ‘experts’ to participate.

Additionally, the concept of expertise and dedication is increasingly challenging in the mediated landscape of the NCAA Tournament, where knowledge is easily accessed and information is ubiquitous. With the increased popularity of the Tournament, an entire media industry exists around providing tips for making informed choices. As such, the lines between expert and lay person, as well as distinctions between dedicated fans and those less so, are increasingly blurred. Moving forward, it could be worthwhile to examine the impact of media in the decisions made by swarms. Additionally, looking ahead, it would be important to continuously examine the concepts of expert and dedication, two concepts that may be evolving in the interconnected sports media ecosphere, one quite different from the far more simplistic model of generations past.

We also acknowledge the inherent bias that can often be built into making selections for the NCAA Tournament. While it should be assumed that most people would select teams based on their desire to pick correctly, we also recognize that many select teams that they hope will win, something driven by school affiliation, fandom or geography. However, this is a reality of all selections made for this unique sporting event; as such, this experiment provides external validity.

In spite of the limitations of our study, we feel that it is an exciting and important means of examining the viability and boundaries of collective intelligence.

Conclusion

This study has provided an introductory examination into the potential of using swarm intelligence to predict a highly unpredictable sporting event. The findings in this study provide preliminary insight into the characteristics of successful swarms. Clearly, expertise has the potential to be important in predicting outcomes, particularly in cases where the games themselves might be more unpredictable. However, it is unclear whether dedication to sports or a sporting event provides any advantage in predicting through swarm intelligence. Both items deserve additional examination and study.

For future research, it is worthwhile to continue to analyse the true impact of expertise across a wide range of sporting events and contexts. Additionally, it would also be valuable to examine the media consumptions of all participants, which might allow us to better understand the impact of pre-tournament bias. It is possible that our current media environment has created a society of experts, even if only driven by media.

Swarm intelligence has vast potential to harness the collective cognitive abilities of individuals to make ‘better’ decisions and predictions, even for an artefact as highly unpredictable as sports. Through this and future research, we hope to not only better understand how to more effectively predict the outcome of sporting events but also the characteristics of swarms that could be most effective across a variety of domains.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article. David Baltaxe, one of the authors, works at the company that makes the software that we used in the study.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

References

Barnett

(1994). The limits of competence: Knowledge, higher education and society. Bristol, PA: Open University Press.

Bonabeau

(2009). Decisions 2.0: The power of collective intelligence. MIT Sloan Management Review. Retrieved from http://sloanreview.mit.edu/article/decisions-20-the-power-of-collective-intelligence/

Csikszentmihalyi

(1997). Finding flow: The psychology of engagement with everyday life. New York, NY: Basic Books.

Deci

E. L.

, & Ryan

R. M.

(2011). Self-determination theory. Handbook of Theories of Social Psychology, 1, 416–433.

Eberhart

, Palmer

, & Kirschenbaum

(2015). Beyond computational intelligence: Blended intelligence. Paper presented at Swarm/Human Blended Intelligence Workshop, Cleveland, OH.

Gobet

(2015). Understanding expertise: A multi-disciplinary approach. New York, NY: Palgrave Macmillan.

Huta

, & Ryan

R. M.

(2010). Pursuing pleasure or virtue: The differential and overlapping well-being benefits of hedonic and eudaimonic motives. Journal of Happiness Studies, 11(6), 735–762.

Jenkins

(2002). Interactive audiences? In Harries

(Ed.), The new media book (pp. 157–170). London, UK: British Film Institute.

Lévy

(1997). Collective intelligence: Mankind’s emerging world in cyberspace. New York, NY: Plenum Trade.

10.

Malone

T. W.

, Laubacher

, & Dellarocas

(2009). Harnessing crowds: Mapping the genome of collective intelligence (MIT Sloan Research Paper No. 4732-09). Retrieved from http://ssrn.com/abstract=1381502

11.

Maslach

, & Leiter

M. P.

(2008). Early predictors of job burnout and engagement. Journal of Applied Psychology, 93(3), 498.

12.

Mataric

M. J.

(1992). Designing emergent behaviors: From local interactions to collective intelligence. In Meyer

J. A.

, Roitblat

H. L.

& Wilson

S. W.

(Eds.), From Animals to Animats 2, 2nd International Conference on Simulation of Adaptive Behavior (SAB-92) (pp. 432–441). Cambridge, MA: MIT Press.

13.

McClelland

D. C.

(1973). Testing for competence rather than for ‘intelligence’. American Psychologist, 28(1), 1.

14.

Ohanian

(1990). Construction and validation of a scale to measure celebrity endorsers’ perceived expertise, trustworthiness, and attractiveness. Journal of Advertising, 19(3), 39–52.

15.

Oliver

, Bowman

, Woolley

, Rogers

, Sherrick

, & Chung

M.Y.

(2016). Video games as meaningful entertainment experiences. Psychology of Popular Media Culture, 5(4), 390.

16.

Rogers

, Strudler

, Decker

, & Grazulis

(2017). Can augmented reality technology “augment” the fan experience? A model of enjoyment for sports spectators. Journal of Sports Media, 12(2), 25–44.

17.

Rosenberg

L. B.

(2015). Human swarming, a real-time method for parallel distributed intelligence. Paper presented at Swarm/Human Blended Intelligence Workshop, Cleveland, OH.

18.

Ryan

R. M.

, & Deci

E. L.

(2000). The darker and brighter sides of human existence: Basic psychological needs as a unifying concept. Psychological Inquiry, 11(4), 319–338.

19.

Salanova

, Agut

, & Peiró

J. M.

(2005). Linking organizational resources and work engagement to employee performance and customer loyalty: The mediation of service climate. Journal of Applied Psychology, 90(6), 1217.

20.

Sarno

(2014). Don’t take warren Buffett’s bracket challenge. Slate.com. Retrieved from http://www.slate.com/articles/sports/sports_nut/2014/03/billion_dollar_bracket_challenge_why_it_s_a_bad_idea_to_enter_warren_buffett.html

21.

Secundo

, Dumay

, Schiuma

, & Passiante

(2016). Managing intellectual capital through a collective Intelligence approach: An integrated framework for universities. Journal of Intellectual Capital, 17(2). DOI: 10.1108/JIC-05-2015-0046

22.

Seeley

T. D.

(2010). Honeybee democracy. Princeton, NJ: Princeton University Press.

23.

Surowiecki

(2005). The wisdom of crowds. New York, NY: Anchor.

24.

Tamborini

, Bowman

N. D.

, Eden

A. L.

, & Grizzard

(2010). Defining media enjoyment as the satisfaction of intrinsic needs. Journal of Communication, 60, 758–777.

25.

Van Scotter

J. R.

, & Motowidlo

S. J.

(1996). Interpersonal facilitation and job dedication as separate facets of contextual performance. Journal of Applied Psychology, 81(5), 525.

26.

Wolf

, Krause

, Carney

P. A.

, Bogart

, & Kurvers

R. H.

(2015). Collective intelligence meets medical decision-making: The collective outperforms the best radiologist. PLoS One, 10(8). DOI: 10.1371/journal.pone.0134269

27.

Woolley

A. W.

, Chabris

C. F.

, Pentland

, Hashmi

, & Malone

T. W.

(2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686–688. DOI: 10.1126/science.1193147