Evaluating the Interdisciplinary Mission of Small Group Research Using Computational Analytics

Data

We assembled the SGR corpus since its inception in 1990 by downloading all articles reported in Volume 21, Issue 1 (when SGR moved on from being SGB) through Volume 48, Issue 3, from the Business Source Premier scholarly literature database. We saved them as PDF files, converted the PDFs to text data (xml or txt), and assessed them using customized programs written with Python coding language. The resulting nonrelational, schema-less database includes 829 articles, spanning 17,417 pages, and containing 2,793,843 words.

Our conceptual unit of analysis is the written word. Arguably, it is the most significant, most influential, most dynamic innovation in history (Chomsky, 2006). We use words to enact intellectual existence, communicate concepts, define domains, and frame futures. Hence, although elementary, the study of words in a defined corpus has proven robust across many contexts such as computational linguistics, institutions, culture, semiotics, economics, and history.

Our unit of analysis is an n-gram, or more simply a keyword or keywords. Technically, an n-gram is a contiguous sequence of n items from a given sequence of text (Cavnar & Trenkle, 1994). It can take the form of a phoneme, syllable, letter, acronym, word, or base pair. One generates an n-gram, based on keyword and thematic analysis or general interest, and then searches for its use in a text or speech corpus, or in this study, the SGR corpus. So, for example, “group” is a 1-gram, “group discussion” is a 2-gram, “group decision-making” is a 3-gram, and so on.

Analytics

Our first step was to calculate the frequency count of all words with three or more characters in the SGR corpus. We tokenized and stemmed all words to the root word and removed all stop words using the python Natural Language Training Kit (NLTK) package (Bird, Loper, & Klein, 2009). Headers, footers, and references were parsed out to focus on topic content contextualized within the article and to remove duplication of terms from the article and journal titles. These term frequencies were normalized by the total words in the entire corpus consistent with the precedent of the Google n-gram method (Michel et al., 2011). Table 1 represents the top 25 words from the corpus.

OPEN IN VIEWER

Table 1.

The 25 Most Frequently Used Words in SGR.

Unigram	Raw Count	Normalized Count	Unigram	Raw Count	Normalized Count
group	73,512	0.0263	time	10,192	0.0036
team	32,253	0.0115	work	9,893	0.0035
groups	25,953	0.0093	level	9,407	0.0034
members	25,329	0.0091	participants	8,356	0.003
research	23,252	0.0083	process	7,921	0.0028
task	17,512	0.0063	member	7,640	0.0027
performance	14,651	0.0052	model	7,423	0.0027
study	12,488	0.0045	communication	7,351	0.0026
teams	12,422	0.0044	results	7,023	0.0025
small	12,114	0.0043	self	6,931	0.0025
information	11,275	0.004	conflict	6,607	0.0024
social	10,886	0.0039	different	6,456	0.0023
individual	10,627	0.0038

Single words formed a basic vocabulary consistent with the general mission of SGR; however, the unigrams alone were too general to effectively code into an ontology of topic domains. We calculated the bi-gram frequencies to capture sufficient meaning from the word combinations to code them into mutually exclusive and collectively exhaustive categories while remaining sufficiently generic to capture general applicability across articles. The vocabulary needs to be validated by “representatives of the communities active in the application domain” (Missikoff, Navigli, & Velardi, 2002, p. 54; also see Browarnik & Maimon, 2015), so we had a co-editor and an editorial board member code the top 200 bi-grams into the three domains sought by SGR: communication, management, and psychology. Of the top 200 most frequently used bi-grams, the two editorial coders effectively coded 96 bi-grams into one or two but not all three of the domains. The bi-grams that aligned with all three categories did not demonstrate sufficient discrimination and were dropped from the vocabulary. Of the 96 bi-grams coded, 30 were coded into two categories. In all, 41 bi-grams were classified in association with communication, 42 with management, and 46 with psychology. The bi-gram vocabulary by domain is included in Table 2.

OPEN IN VIEWER

Table 2.

Bi-grams by Domain.

Comm Bi-grams (41)	f	Mgmt Bi-grams (42)	f	Psych Bi-grams (46)	f
face face	1224	team performance	2435	individual level	1196
work group	758	team member	1205	group development	1116
group interaction	757	individual level	1196	collective efficacy	882
group processes	722	group development	1116	self efficacy	866
computer mediated	720	team learning	912	relationship conflict	861
group discussion	690	work groups	912	computer mediated	720
task conflict	655	collective efficacy	882	task conflict	655
virtual teams	606	self efficacy	866	social identity	605
social identity	605	team level	857	unshared information	575
unshared information	575	work group	758	group efficacy	543
decision quality	462	task interdependence	718	transactive memory	521
task oriented	457	team effectiveness	687	idea generation	509
task cohesion	430	virtual teams	606	social cohesion	471
shared information	424	group effectiveness	594	mental models	453
information processing	411	work teams	565	conflict management	442
group identity	404	group efficacy	543	within groups	432
group communication	393	role ambiguity	510	task cohesion	430
mediated communication	393	time pressure	494	information processing	411
self help	388	group goals	475	group norms	407
social support	371	decision quality	462	meta analysis	401
virtual team	368	team leader	461	individual performance	375
task satisfaction	365	within groups	432	task satisfaction	365
information sharing	363	team task	412	group composition	365
process conflict	353	project teams	389	self esteem	363
hidden profile	349	individual performance	375	goal setting	355
unique information	349	virtual team	368	process conflict	353
satisfaction group	336	goal setting	355	psychological safety	346
cohesion performance	324	team training	350	therapeutic factors	343
group membership	321	team leaders	349	performance group	329
task related	321	team building	343	cohesion performance	324
group cohesiveness	319	performance group	329	task related	321
team process	318	group membership	321	nominal groups	320
member satisfaction	301	team process	318	group cohesiveness	319
social loafing	301	work team	318	randomly assigned	309
relationship group	298	team size	310	perceptions group	304
ideas generated	295	mental model	302	individual members	303
group leaders	282	transformational leadership	300	mental model	302
team research	277	performance team	293	member satisfaction	301
group social	275	group leaders	282	social loafing	301
group discussions	273	team research	277	ideas generated	295
Comm Bi-grams	f	Mgmt Bi-grams	f	Psych Bi-grams	f
task groups	267	cognitive diversity	275	social psychology	276
		leadership style	274	cognitive diversity	275
				self monitoring	274
				members perceptions	271
				self report	269
				number ideas	268

Based on this vocabulary of bi-grams for each domain, we calculated the term frequencies of each bi-gram for every article. We summed the term frequencies for each domain. Due to the unbalanced set of opportunities across domains, we normalized the counts by the inverse proportion of domain-specific bi-grams to the total set of bi-grams. For example, all communication and management bi-gram term frequencies were multiplied by 96 and divided by 41; all psychology bi-gram term frequencies were multiplied by 96 and divided by 46. The normalized bi-gram term frequencies were converted to proportions to assess the relative representation of each domain within an article.

To identify the sufficient level at which a domain can be considered to be included in an article, we implemented a sensitivity analysis and varied the minimum cutoff thresholds for binary measures of inclusion to examine overarching trends. With three domains (communication, management, and psychology), the maximum inclusion of all three domains would be 33% in which case all three domains would be considered to be an equal part of the article. However, to include interdisciplinary content does not require equal levels of inclusion, only sufficient inclusion to warrant integration. With this in mind, we began with a 10% minimum and coded in 5% increments up to 30%. At the first level, if the sum of term frequencies for bi-grams of a given domain was greater than 10% of the total of all term frequencies for the 96 bi-grams in an article, the article was coded with a 1 for inclusion of that domain indicating at least 10% of the coded terms in an article represented one particular field. The articles were then tallied by number of domains included in the article (e.g., 0, 1, 2, or 3 domains) as well as which domains were included (e.g., communication, psychology, management, communication-management, communication-psychology, management-psychology, and communication-management-psychology). These interdisciplinary content scores range from 0 to 3 where a score of 0 means none of the three domains are present, 1 means the article is unidisciplinary, and a score of 2 or 3 means the article is interdisciplinary; a score of 3 indicating that all of the domains are used together in one article. ³

Computational outputs still need to be manually validated to confirm effectiveness (Short, Broberg, Cogliser, & Brigham, 2010). We randomly selected 70 articles (approximately 8%) and asked an editorial board member to estimate the inclusion of each domain in each article based on the abstracts and introductions. Kappa statistics for interrater reliability between the manual rating and the computational output were maximal when the minimum cutoff threshold was 20% (85.71% agreement or 60 of 70). For this reason, we report the results using 20% as the minimum proportion of bi-gram term frequencies to indicate inclusion of a given domain within an article. Furthermore, we calculated interclass correlation coefficient (ICC) metrics by domain and found high reliability between the two measures (.94 for communication, .89 for management, and .97 for psychology).

Data Adjustments

We use conventional formats to present our findings. Still, we apply adjustments. First, all longitudinal profiles organize data by year of publication. Besides following Google’s n-gram default setting, an annual division mirrors the volume periodicity in SGR. Second, we control for the varying annual levels of scale and scope of SGR by normalizing all counts of articles relative to the total number of articles in a given year. Third, we apply a 3-year simple-moving average to all longitudinal profiles to ease trend recognition and pattern identification.

Overall Interdisciplinary Content

SGR proactively seeks interdisciplinary work that integrates communication, management, and psychology. Interdisciplinary content includes more than one of these domains; thus, articles with interdisciplinary content scores of 2 or 3 are considered interdisciplinary. Given the value of data visualization (Healy & Moody, 2014), the results for a 20% cutoff threshold are summarized in Figure 1 and show that most articles in SGR are interdisciplinary (638 of 829, or 77%), particularly integrating content from two domains (457 or 55% of all articles), though articles that combined all three domains (181 or 22% of all) still outpaced single-domain articles (144 or 17% of all). There were 47 articles (6%) that did not include any of the 96 coded bi-grams. The majority of these articles were editorials and comments which limited the potential to detect disciplinary content.

OPEN IN VIEWER

Figure 1.

Distribution across interdisciplinary levels at 20% cutoff threshold.

Changing the cutoff thresholds for inclusion of a given domain will change the distribution of articles across these categories with lower thresholds increasing interdisciplinary scores and higher thresholds reducing them by requiring more balance. At the lowest threshold of 10%, indicating that an article only needed 10% of its coded content to be from a domain to be counted as belonging to that domain, more than 53% of the articles included all three domains with only 46 articles (6% of total) being from a single discipline. At the highest threshold of 30%, only 18 articles (2%) included all three domains, but 429 (52%) were still considered interdisciplinary within two domains. Although the manual validation most accurately matched a 20% cutoff, results taken from any cutoff threshold from 10% to 30% confirm that SGR is predominantly an interdisciplinary journal. These results provide robust support that SGR is and has been achieving its interdisciplinary mission.

Disciplinary and Interdisciplinary Representation

The initial analyses supported the interdisciplinary focus of SGR in general, but we also assessed which domains and interactions received the most total attention across time. First, to compare the relative utilization of each domain, we summed single-domain and two-domain articles where the domain was included. Three domains were excluded as they added no discriminatory value. Each combination of two domains is also represented. For each level of cutoff inclusion ranging from 10% to 30%, Figure 2 shows the split between domain content areas and two domain interactions.

OPEN IN VIEWER

Figure 2.

Distribution of SGR articles across domains for various cutoff inclusion ranges, 10% to 30%.

The distribution patterns are virtually identical across each cutoff level. Psychology is represented in 55% of the articles whereas communications and management are present in 41% and 32%, respectively. Regarding the 457 articles that integrate two domains based on the 20% threshold, 55% combine psychology and communications whereas 32% combine psychology and management and only 13% combine communications and management.

The previous figures show the overarching balance of interdisciplinary content and relative domains, but it is unclear whether there were key trends at play regarding these balances. So, we mapped the use of each domain and two domain interactions across time as a relative proportion of the total articles in a given year. Figure 3 shows these trends. What we see is that psychology has always been a highly integral component of SGR leading the other domains except for virtual parity in 2013 and the first half of 2017. Similarly, communication was the second leading domain tracking closely with psychology from the beginning through the start of the 2000s where it began to trend toward equal representation with management at 60% to 80% of the utilization of psychology except for parity around 2013. Historically, management was underrepresented in SGR, being represented in just above 10% of articles early in the life of the journal. However, it has grown to be the leading domain of the journal in the first half of 2017.

OPEN IN VIEWER

Figure 3.

Interdisciplinary category trends, 1992 to 2017.

The dual domain trends are similar. Articles combining communication and psychology dominated the early years of the journal, whereas articles combining management with psychology or communication represented less than 10% of articles. These two groups of articles have increased slightly over time, whereas articles combining communication and psychology have dropped considerably to approximately the same levels as the other two dual domains. Articles containing significant elements of all three domains have remained relatively steady throughout the life of SGR, but have recently declined slightly to levels similar to each of the two domain categories.

The Evolution of Psychology, Communication, and Management Within SGR

Historically, manuscripts embedded with psychological traditions or containing significant aspects of psychological theory tended to dominate SGR, notably being more than 4 times as present from 1990 to 1997 as articles embedded within management. However, a shift away from psychology as the dominant discipline within SGR began in 2010, coinciding with a special issue of SGR, co-edited by Poppy McLeod and M. Scott Poole, on interdisciplinary perspectives in small groups. Our analysis cannot speak to the causality between these two events; however, the special issue “represent[ed] the culmination of an effort began in 1998 to undertake an assessment of the state of knowledge on research and theory about small groups” (McLeod & Poole, 2010, p. 661) wherein the authors promoted interdisciplinarity by stressing how well-developed theories on small groups are formed from the same basic components, which may be common to several theories, and are likely applicable across scientific domains (Markovsky, 2010).

Similarly, communication began as a dominant tradition within SGR, but its influence has been allowed to wane over time in the name of a more interdisciplinary SGR corpus. Because of this early dominant influence of both psychology and communication, overlaps in material between psychology and communication also represented the dominant form of interdisciplinarity in the journal from 1990 to 2002. However, currently, there are similar ratios of all three interdisciplinary combinations of two fields: psychology-communication (18.3% from 2014 to 2016; 46.3% from 1990 to 1992), psychology-management (23.3% from 2014 to 2016; 9.7% from 1990 to 1992), and management-communication (12.3% from 2014 to 2016; 3.7% from 1990 to 1992), and articles with combination of all three (13.3% from 2014 to 2016; 26.3% from 1990 to 1992), although combinations of psychology and management currently are leading the way.

In this vein, management research in SGR has experienced a pattern opposite to that of psychology and communication. From 1990 to 1992, management terms were only present in 13% of SGR articles, and in all cases, those articles were interdisciplinary, such that management was never the sole topic of an article. This interdisciplinary nature of management articles has held up over time. In fact, the first article SGR published that mostly (more than 80%) focused on management (and was not an editorial) was William Burpitt and William Bigoness’s 1997 “Leadership and Innovation Among Teams: The Impact of Empowerment.” Perhaps this trend is due to the nature of management itself, as a hybrid science made up of influences from multiple fields including both psychology and communication as well as sociology, anthropology, and others (Robbins & Judge, 2017). Still, over time, and particularly beginning in 1999, management’s influence in SGR became equal to that of psychology, and it has recently surpassed communication as the secondary influence in the SGR corpus.

Directions for Future Work: Within and Outside of SGR

Before we make a few comments (we hesitate to call them recommendations) on the state of interdisciplinarity at SGR and future directions for it, it is important to note that our analyses indicate that any changes in strategy should be minor. Generally, what the editors, editorial board, and reviewers of SGR have been doing over the past 27 years has been working to create an interdisciplinary environment to publish and utilize group research. Therefore, any action or responses to our comments should incorporate minor tweaks instead of sweeping changes.

That being said, as the 1998 project on the interdisciplinary nature of small groups explicitly stated, much of theory we apply to group functioning is interdisciplinary in nature across all three fields germane to SGR. Although SGR is clearly interdisciplinary in nature, there could be more focus on incorporating all three disciplines into single articles. This may be difficult, as indicated by the decline in articles containing significant elements of all three fields since 2014, because authors are motivated to publish work in line with their own research identity, which almost necessarily ties them to a single discipline (Fiore, 2008). Because of this, it may be beneficial for SGR to continue or perhaps increase the use of commentaries or editorials to discuss the implications of dominant theories and findings for all three subfields. Doing this would allow journal-level interdisciplinarity while maintaining space for true between-article multidisciplinarity. Still, efforts such as those by the National Academy of Sciences and the Science of Team Science (SciTS) focusing on the necessity of interdisciplinary research to address the complex problems society faces are making it easier and more personally beneficial for scholars from across disciplines to take interdisciplinary approaches to problems and cultivate relationships across disciplines.

It is also important that SGR maintain its role as an interdisciplinary source for group research, especially considering the number of domain-specific journals that publish work on groups and teams. We feel that this is important to mention explicitly, as, if current trends continue, management has the potential to take the dominant SGR role that psychology and communication once held. However, such a continuation would not benefit group scholarship, as maintaining balance between disciplines is vital to maintaining a broad understanding of small group phenomena as well as a healthy debate and integration among group scholars from different disciplines.

On that note, it is important to more explicitly consider SGR within the broader set of journals that contain research on small groups. When we began our analyses, we did not know whether it would indicate that SGR was indeed meeting its interdisciplinary mission. However, after interpreting the results, it seems that SGR should be considered an exemplar of the effortful dissemination of interdisciplinary work within an important subfield—groups and teams. That being said, many other subfields could likewise benefit from a more interdisciplinary approach. As many group scholars are members of other subfields, and perhaps members of other subfields may read this analysis, it may be beneficial for those interested to try to apply the model of SGR to their own areas of interest. For, to repeat Kuhn (1962), interdisciplinary work is vital to the sociology of knowledge, which all academics strive to contribute to.

Finally, from a broader perspective, BDA helped us analyze and provide insight into the patterns of interdisciplinarity that flow within the SGR corpus, and do so more objectively than has previously been done. We hope this analysis stands as one demonstration that BDA can help fields understand themselves. However, more than that, the techniques utilized could also be applied to a diverse range of questions including interpreting intellectual legacies, domain parameters, design standards, and emerging frontiers. Moreover, this can be applied to not only broad research domains but also specific theoretical or empirical traditions. Thus, BDA may allow any number of research endeavors to move forward by holistically looking back.