Abstract
Small-group pedagogies, such as group research projects, are a common instructional method in undergraduate education. The literature suggests that small-group learning has positive effects on learning outcomes, but some students have negative attitudes toward group work, and student complaints about negative group dynamics, such as free-riding, are common. This study examines the relationship between learning outcomes associated with group research projects, student experiences, and group dynamics, controlling for students’ individual characteristics, group composition, task type, and incentive structures. The sample includes data on course records and self-assessment narratives for 240 students who completed a sociology research methods course at a small, private liberal arts institution between 2004 and 2015. Multivariate analyses indicate that students’ experiences have indirect effects on individual learning outcomes, and some aspects of group composition, task type, and group dynamics predict students’ experiences with group research projects.
Small-group pedagogies, such as group research projects, are widely used in undergraduate education, and evidence from case studies as well as meta-analyses suggests that small-group pedagogies have positive effects on learning outcomes. Yet students often express negative attitudes toward group work or report negative experiences with group members who dominate or withdraw from the group. Do negative experiences and/or dysfunctional group dynamics, such as free-riding, affect students’ learning? 1 Do students have to like group research projects in order to learn from them?
In this study, I review the literature on the learning outcomes associated with group projects and evidence of whether and how these learning outcomes are mediated by students’ experiences and their group’s dynamics. Next I compare cooperative learning and collaborative learning approaches to composing and monitoring groups in order to develop several alternative models of how group composition, task type, and incentive structures may shape group dynamics and learning outcomes of group projects. I then present multivariate analyses of data on the learning outcomes, experiences, and group dynamics associated with group research projects for students who completed one of 14 sections of a sociology research methods course taught over a period of 11 years.
Group Research Projects, Learning Outcomes, and Students’ Experiences
Group research projects combine experiential and small-group learning and typically require students to work together in small groups on all or most of the steps of a research project: review a body of literature, define the research question, design the research project, collect and analyze the data, and write up the research findings. As defined in this article, group research projects might last as little as two weeks (Eglitis, Buntman, and Alexander 2016) or span an entire term or more (Bartholomay 2018); the groups might be subsets of a larger class (Monson 2017), or an entire class might be treated as one small group (Willis and Burns 2011); and the group project may center on locating and analyzing data from secondary sources (Caulfield and Persell 2006) or collecting original data (George 2012). Small-group learning pedagogies, such as group research projects, are widely used at the college level (Davidson and Major 2014; Hillyard, Gillespie, and Littig 2010), and sociology instructors have advocated the use of group research projects in graduate and undergraduate methods courses (Korsching and Peter 2007; Macheski et al. 2008; Schutt, Blalock, and Wagenaar 1984) as well as for introductory courses (Caulfield and Persell 2006) and intermediate electives (Anderson 2017; Bartholomay 2018).
There is evidence, albeit primarily indirect, that group research projects contribute to students’ learning outcomes. These studies typically rely on course evaluations or students’ self-reports rather than direct measures of learning, and they describe students’ perceptions of increased knowledge, ability to work in teams, and research skills as well as instructors’ observations of students’ deep engagement with course material and/or professional development as sociologists (Broughton 2011; George 2012; Keen 1996; Korsching and Peter 2007; Raddon, Nault, and Scott 2008). Some studies that rely on direct measures of learning find increased mastery of course content associated with group research projects (Bartholomay 2018; Monson 2017). These findings are consistent with the broad consensus that small-group learning generally has positive impacts on students’ learning outcomes (Cabrera, Crissman, and Bernal 2002; Davidson and Major 2014; Kilgo, Sheets and Pascarella 2015; Slavin 2013). Recent meta-analyses of studies of cooperative learning (Johnson, Johnson, and Smith 2014; Kyndt et al. 2013) and team-based learning (Swanson et al. 2017) found positive effects on student skills and/or content knowledge.
Most studies that find largely positive correlations between group research projects and students’ learning outcomes, including those that rely on students’ own self-reports of perceived learning, also find some students have negative experiences with or attitudes toward the group research project; these studies typically report informal rather than systematic observations of students’ experiences (but see George 2012). Instructors have noted students’ logistical difficulties, lack of enthusiasm for the group project, interpersonal conflicts, or complaints about grading and free-riding (Broughton 2011; Caulfield and Persell 2006; Edgerton and McKechnie 2002; Korsching and Peter 2007; Longmore, Dunn, and Jarboe 1996; Raddon et al. 2008). These reports are consistent with studies that find significant proportions of undergraduates have negative perceptions of group work more generally (Fiechtner and Davis 1984; Forrest and Miller 2003; Hillyard et al. 2010; Liden, Nagao, and Parsons 1985) and with several articles that offer tips on how to improve students’ experiences with group work (Oakley et al. 2004; Perron 2011; Shimazoe and Aldrich 2010; Stevens 2007).
Correlates of Students’ Attitudes toward and Learning in Group Work
The evidence that students both learn from and often dislike small-group pedagogies, such as group research projects, raises several questions about the relationship between students’ learning and experiences. Are students’ negative experiences with group work an impediment to their learning or unrelated to it (Ulrich et al. 2017)? Are some negative aspects of group projects impossible to eliminate (Longmore et al. 1996)? Are less-privileged students less likely to enjoy, persist in, or benefit from group work (Micari and Drane 2011)? What, if anything, can or should instructors do to minimize students’ negative experiences with group research projects in order to maximize their learning?
Analyses of variation in students’ attitudes toward group work offer some initial clues. A few studies find a correlation between students’ negative attitudes toward group work and some individual characteristics, for example, personality traits (Myers et al. 2009) and higher grade point averages (Liden et al. 1985). There is little evidence that students’ age or gender is correlated with their general perceptions of group work (Cabrera et al. 2002; Hillyard et al. 2010; Pauli et al. 2008; Payne and Monk-Turner 2006) and mixed evidence of an association between race and attitudes toward group work (Cabrera et al. 2002; Payne and Monk-Turner 2006; Strayhorn 2008). There is more evidence that particular kinds of learning groups themselves are correlated with students’ subsequent negative attitudes toward group work (Forrest and Miller 2003; Hillyard et al. 2010), including self-selected or homogenous groups, groups that are too small or too large, groups that did not include peer evaluation and formative feedback, online groups, and the presence of a free-rider in the group (Fiechtner and Davis 1984; Grant-Vallone 2010; Lizzio and Wilson 2005; Payne and Monk-Turner 2006).
Similarly, the literature on the impacts of small-group pedagogies on learning outcomes indicates little variation by students’ individual characteristics. A recent review of the empirical literature on cooperative learning found little evidence of variation by individual students’ ability (or prior levels of academic achievement), and mixed evidence of variation by students’ race-ethnicity, in effects of cooperative learning pedagogy on subsequent achievement (Slavin 2013). There is more evidence of variation in individual students’ learning gains and/or in the achievement of the groups by the characteristics of the learning groups themselves, including group size and whether groups are homogenous in ability, orientation to learning, gender, and race-ethnicity (Baer 2003; Monson 2017; Onwuegbuzie, Collins, and Jiao 2009; Swanson et al. 2017). Finally, broader social contexts also may play a role. A recent meta-analysis found the generally positive effects of cooperative learning on achievement are more pronounced in non-Western (more collectivistic) cultures (Kyndt et al. 2013). And several authors comment that larger cultural norms and values—such as a consumerist orientation to education, competitive individualism, external rather than internal motivations for learning, the social bonds of particular student cohorts, and the rise of online and asynchronous communication—can foster free-rider problems and thus confound the expected impacts of small-group pedagogies on group dynamics and student learning (Collins 2012; Herrmann 2013; Richardson and Harper 1986; Smith et al. 2011).
Taken together, this literature suggests that it is primarily variation in the groups themselves rather than students’ individual characteristics that is correlated with variation in students’ attitudes toward and learning from group projects. If the groups themselves matter, instructors should ask how what is going on in the “group part” of group research projects might affect students’ learning.
How Does Working in Groups Contribute to Learning?
In the absence of control groups it can be difficult to determine whether it is active learning, collaboration with peers, or the combination of the two that leads to any demonstrated learning outcomes associated with group research projects (Monson 2017), and achieving true control groups in a higher education setting is difficult (Grauerholz and Zipp 2008; Huggins and Stamatel 2015). 2 Caulfield and Persell (2006:45) administered pre- and postsurveys to students in courses that used group research projects and in “traditional” courses and argued that the “cognitive competencies” students demonstrated in the courses using group projects “were enhanced by collaborative learning processes.” But it is not clear whether the “traditional” classes were true controls, comparable to the other courses in all relevant aspects, such as course content and student characteristics. Ulrich et al. (2017) compared nursing students’ learning outcomes across six sections of the same course that used different pedagogies (team-based learning; enhanced lecture, which incorporates active learning strategies into a standard lecture format; and standard lecture) and found that both team-based learning and enhanced lecture were correlated with students’ higher scores on a standardized test of obstetrics knowledge. They concluded that the active learning component of team-based learning may be more important than the collaboration with peers for students’ learning (Ulrich et al. 2017:156; see also Huggins and Stamatel 2015). But the six sections of this course were offered over three years with the different pedagogies trialed in different years; concurrent curricular changes made it difficult to rule out confounding factors (Ulrich et al. 2017:157).
In addition, the “group part” of group research projects is complex and includes variation in group dynamics, group tasks, and group composition. Caulfield and Persell (2006:51–52) called for further research that would unpack the “black box” of these intervening variables: “If students involved in group projects do come to understand more and have better skills, can we show how this greater understanding results from the use of groups? How, exactly, does working in groups facilitate the development of greater understanding and skills?” Put another way, what are the small-group processes that are consequential for learning?
Group Processes in Small-group Pedagogies
Recent overviews of various small-group pedagogies emphasize the similarities as well as differences between them (Davidson and Major 2014; Michaelsen, Davidson, and Major 2014; Slavin 2013). All use small groups to encourage students’ interaction and active engagement and thereby foster the development of content knowledge, critical thinking, and related skills. Differences center on the approaches instructors take in creating, structuring, and monitoring the small groups so as to encourage students’ working together (Davidson and Major 2014:30–40). Comparing collaborative learning pedagogy to three varieties of cooperative learning pedagogy usefully fleshes out some of the group processes thought to be central to different kinds of small-group learning and points toward some possible connections between group composition, group tasks, group dynamics, and learning.
Cooperative learning pedagogy
All varieties of cooperative learning pedagogy stress the importance of cooperative group processes because cooperation between members of a small group is thought to lead to better interaction and thus more learning than either competition or individualism (Johnson et al. 2014; Rau and Heyl 1990). Students should work together rather than individually and work toward the group goal “in a mutually helpful manner” rather than competitively; thus instructors typically include team-building activities and often use assigned groups (Davidson and Major 2014:14, 33–34).
Proponents of cooperative learning vary in their emphasis on particular approaches to accomplishing cooperative group dynamics: social interdependence, incentive structures, or task structures. Those who emphasize the centrality of “social interdependence” tend to focus on direct engineering of cooperative group dynamics by establishing goal interdependence (that is, ensuring that each group member believes his or her own goal cannot be achieved unless all members achieve their goals), holding all members of the group accountable not only for their own learning but for every other group member’s individual learning, teaching small-group and interpersonal skills, and monitoring, intervening in, and assisting each group as needed (Johnson and Johnson 2006:487–492). Instructors also can—but need not—employ one or more of several other strategies for encouraging cooperation, including assigning students to specific roles or giving a group grade for the group’s product (Johnson and Johnson 2006). In contrast, those who emphasize the importance of “incentive structures” claim that group rewards are an essential, not optional, component for fostering cooperative learning. They argue that cooperative group processes and positive interpersonal dynamics are much less likely to develop in the absence of an incentive structure that combines group rewards with group goals and individual accountability (Slavin 2013:180, 183, 187) . Finally, some argue that the “type of task” assigned to the group is crucial. If the task is a “true group task” (one that can be accomplished only when all group members work together, not when individuals work separately) with an “ill-structured solution” (that is, no one right answer or solution), then cooperative interaction and productivity is increased, and learning gains result (Cohen 1994:8). With these kinds of tasks, instructors must take care not to overly structure or constrain the interaction between the group members if the interaction is to have the desired characteristics and learning benefits (Cohen 1994:22).
Collaborative learning pedagogy
Of these three variants of cooperative learning, the “social interdependence” approach requires the most engineering by the instructor, while the “incentive structure” and “task type” approaches require somewhat less. Collaborative learning pedagogy is even less structured: here, the essential group process is talking rather than cooperating—discussing, explaining, asking and answering questions with each other and the instructor—and the “focus is on working with each other (but not necessarily interdependently) toward the same goal. . . . [In a] collaborative project . . . students could divide up the task and assemble the individual parts in order to accomplish the common goal” (Davidson and Major 2014:21–22). In contrast to cooperative learning approaches, instructors using collaborative learning allow students to choose their own groups and rely primarily on group goals and task type to foster students’ working together; they tend not to teach group skills or include team-building exercises (Davidson and Major 2014:22, 30).
In sum, different small-group pedagogies tend to emphasize different elements of the concept of a small group itself: the quality of the face-to-face interaction between group members, the agenda or goals that structure their interaction and are their impetus, what is at stake for the members of the group, the group members’ mutual influence and degree of equality, and/or the informal norms and roles to which group members are held accountable (Johnson and Johnson 2006:5–11; Wolff 1950). But all small-group pedagogies regard group processes as important. It is not (just) that the groups are doing something that is consequential for students’ learning: what groups are doing and/or how they are doing it also matter.
Which Aspects of Small-group Pedagogy Contribute to Positive Group Dynamics and Learning Outcomes?
The scholarship-of-teaching-and-learning literature on group research projects in sociology courses supports Davidson and Major’s (2014) contention that in practice, most instructors combine strategies from more than one approach to cooperative learning or from collaborative as well as cooperative learning pedagogies. One instructor assigned students to groups and required students to take one of several specific roles but used a combination of group and individual grades for the group project and did not include team-building exercises (George 2012); another gave students the option of selecting their own groups, used a group grade for the project, and emphasized giving students latitude and responsibility for their work, but required students to designate a team leader (Longmore et al. 1996); another allowed students to choose their own groups, did not require students to take any particular role, but monitored group processes fairly closely and used a combination of individual and group grades for the group project (Bartholomay 2018); and so on. Thus instructors might reasonably prefer evidence of the effectiveness of specific strategies for fostering positive group dynamics and student learning outcomes over evidence of the relative effectiveness of cooperative versus collaborative pedagogy writ large.
However, much of the scholarship on the learning outcomes of small-group pedagogies, such as group research projects, is not well suited to that purpose. First, much of the literature is based on case studies of a single course with so many sources of variation embedded within each case that it is difficult to draw meaningful comparisons across studies even in carefully done reviews of the literature (Cohen 1994; Slavin 2013). Second, meta-analyses leave out many studies that rely on qualitative evidence, critical for understanding group processes and “the why and how a teaching innovation . . . affects learning” (McKinney 2018:128–129). Third, many studies give little or no attention to at least one set of variables—individual characteristics, group characteristics, tasks or rewards of the group project, group dynamics, group achievement, and/or individual learning—that are relevant to the larger question of under what conditions small-group pedagogy fosters positive group dynamics and contributes to learning. For example, several case studies conclude that groups’ achievement on the group task is unrelated to or even hindered by cooperative, mutually helpful, or equitable group dynamics but did not include measures of individual students’ learning (Druskat and Kayes 2000; Onwuegbuzie et al. 2009; Richardson and Harper 1986; Takeda and Homberg 2014). Kyndt et al.’s (2013) meta-analysis of 65 studies that compared cooperative learning to traditional lecture found that the generally positive effects on students’ achievement (as measured by individual students’ grades on exams) did not vary significantly between studies of cooperative learning that used group rather than individual rewards for individual learning. However, it is not clear whether they included studies of cooperative learning methods that used group rewards for group achievement, a common method of structuring rewards in group research projects. Kyndt et al. also found the effect sizes of cooperative learning were larger in science, technology, engineering, and mathematics fields than in the social sciences and speculated that this might have been due to differences in the types of learning tasks characteristic of these domains but did not include measures of task type.
What is needed, then, are analyses that examine whether group dynamics matter for group achievement and individual students’ learning, and which group characteristics matter for group dynamics. Research methods is considered “a complex domain” (Earley 2014:242) and conducting a research project is a classic “ill-structured task” (Cohen 1994; Slavin 2013). Thus, group research projects in research methods courses will tend to require significant student interaction, and studies of this small-group pedagogy can usefully illuminate these research questions. This study analyzes data on learning outcomes and student experiences in group research projects collected from 263 students completing an intermediate-level sociology research methods course between 2004 and 2015. It uses regression analyses to investigate the correlation between students’ experiences with the group research project, their group’s dynamics and achievement on the group task, and their subsequent learning while controlling for their individual characteristics, their group’s homogeneity (by gender, race, and ability), and task type (method of data collection and research topic). I received institutional review board permission to use these data in research.
Data and Methods
The Group Research Project: Learning Goals, Grouping, Task, and Incentive Structure
The group research project was intended to deepen students’ understanding of the basic elements, problems, and potential of social scientific research through trying out research design, data collection, and analysis on a small scale. In the first 10 weeks of the course, students read a standard research methods textbook and a dozen or more examples of sociological research. I administered two or three midterm exams that covered theoretical paradigms, research design, causality, ethics and politics, measurement, sampling, and the like. For the last four weeks of the course, I assigned students to groups of three to five that were diverse by gender and performance on the midterm exams.
Each group designed and conducted a pilot research project on the same general topic (previously chosen by the entire class) but used a different method of data collection (self-administered questionnaires [SAQs], content analysis, experiment, field research, or interviews). Groups were charged with figuring out how to apply their data collection method to the general research question under their time and resource constraints. To guide their work, I supplied each group with a suggested timeline (tailored to their research method) for the completion of particular steps and a checklist of things to keep in mind as they designed and carried out their project. 3 Students also read a few additional textbook chapters on the advantages and disadvantages of specific methods and the basics of qualitative and quantitative data analysis. During the project stage, student groups typically met several times a week (in and out of class, with and without me), and I gave frequent, detailed, critical feedback on their developing ideas and exercised a moderate degree of oversight (e.g., I had to sign off on each group’s final plan for data collection before they began collecting data).
The incentive structure combined a group grade for achievement on the group project and an individual grade to assess each student’s learning. In the final week of the term, each group made a 15- to 20-minute class presentation on their project and findings. Grades on the group project were assigned to the group as a whole and counted for 15 percent of a student’s course grade. In lieu of a final exam, students wrote individual papers describing and critiquing how their research project was designed and carried out, summarizing their findings, and proposing a follow-up study using a different method of data collection. This was an individual grade, worth 25 percent.
Dependent Variable: Individual Student Learning
I used the grade on the final paper to measure individual learning. Because none of the students had prior experience with writing a paper based on original research, I supplied them with a very detailed, five-page outline for the entire paper. This prompt included paragraph-by-paragraph instructions, including which key course concepts they had to address within each section (for example, conceptualization and operationalization, level of measurement, validity and reliability of measures, representativeness of sample, generalizability, univariate and bivariate analyses, and so on). Because the prompt was so detailed, I did not weight the organization of the paper or the quality of the writing when assigning paper grades; instead, grades were based on how comprehensively and correctly students utilized course concepts, and I used the prompt as the rubric.
Main Independent Variable: Group Achievement on Group Project
I used the grade on the research project to measure group achievement. At the same time that students were given the suggested timeline and checklist for completing the research project (see above), they also were given a handout with a checklist of required and optional topics to cover in their in-class presentation (strengths and limitations of their method, hypotheses, measures, sampling, coding processes, key findings, ethical issues, and so on). I used this checklist as a rubric for grading the presentations. In addition, I told students I would grade their projects and papers not according to how perfectly the research had been carried out but rather by what they had learned about research methods. Thus, if the group had made a mistake in their sampling design or had committed a breach of ethics but their self-critique in their presentation or paper gave a correct statement about what the group should have done, they received full credit for that aspect of the project or paper.
Project and paper grades were recorded as letter grades and converted to continuous variables on a 4.0 scale (A = 4.0).
Control Variables: Individual Characteristics, Group Composition, Task Type
For individual students, I recorded the year and term they took the course, their gender and race, their group’s size, and the characteristics of each of their group’s members. I also recorded each student’s midterm exam grades and calculated his or her average midterm exam grade.
For each group, I calculated the mean and range of the group members’ average midterm exam grades. I also recorded whether the group included no, one, or two or more high-achieving students (those who had earned an A or A– on at least one midterm exam); no, one, or two or more men; and no, one, or two or more nonwhite students.
Although all groups completed the same general “ill-structured task” of designing and conducting a research project, I created two additional measures of task type. One is the data collection method (SAQs, experiments, content analysis, field research, or in-depth interviews) since the different kind of work required for each may lend itself to more or less collaboration (Winn 1995). The second is the research topic’s combination of independent and dependent variables; these required different sorts of emotional labor of students. Two thirds (46/70) of the groups chose to study some version of “how does race/class/gender affect students’ experiences at Hobart and William Smith (H&WS)?” Ten groups focused on a somewhat different dependent variable (such as town residents’ perceptions of college students or professors’ teaching styles) but kept the independent variables of race/class/gender. Students often were nervous about approaching town residents they did not know and hesitant about “studying up,” that is, studying those with greater power or higher status than themselves (see Nader 1972). Fourteen groups kept the dependent variable of students’ experiences/interactions at H&WS but focused on a different independent variable having to do with some precipitating event, a cohort effect, or change over time. Nine of these 14 groups focused on the impact of one or more recent sexual assault cases on some aspect of students’ experiences on our campus, and students often found it personally challenging to investigate the effects of these events.
Intervening Variables: Student Experiences, Group Dynamics
Students completed a written self-assessment of their experience with the group research project, due at the same time as their final paper and prior to learning their grade on the group project. The self-assessment was an opportunity for students to reflect critically on their role in the group project, take ownership of their learning, and inform me of any issues they felt were relevant to my grading. Students typically wrote a one- or two-page narrative in response to these five open-ended questions:
Compare yourself and other members of the team on the effort you put into the research project.
Compare yourself and other members of the team on the extent to which you studied course materials and knew what to do for the research project.
Based on the work effort and knowledge level you discuss above, grade yourself and each member of the team.
How did the team process work out? Was it mostly a positive or negative experience for you? Would you do things differently in the future?
Tell me anything else you think I should know in order to be fair in grading the team’s work.
The students’ self-narratives were the source of data on students’ experiences and group dynamics. Relying on students’ individual perceptions of group dynamics is problematic but necessary unless the researcher can observe group processes directly and systematically. To correct for self-report bias, researchers should “consider a group as a whole and . . . examine the similarities and differences in perceptions of group members on the same team” (Grant-Vallone 2010:118–119). I coded each narrative for evidence of that individual student’s experiences in their group and coded each set of narratives in a group for evidence of the group’s dynamics.
Individual students’ overall experience
I coded the student self-assessment narratives (particularly their answers to question D) to create a four-category reverse-coded ordinal measure of individual students’ overall experience with the group process. An example of one student’s narrative that was coded as “mostly negative” (coded as 4) detailed the shortcomings of her teammates; pronounced them “irresponsible, undependable”; and included no mention of any benefits to herself of working on the group project. This narrative ended with a “sad face” emoticon. A “mixed” narrative (coded as 3) by another student characterized the team process “very frustrating” and her team members as being “unwilling to think for themselves” at times but conceded that her teammates had useful insights she would not have thought of. A “mostly positive” narrative (coded as 2) stated that the team process “worked out pretty well” and was “generally positive” albeit “extremely challenging at times.” A student’s “entirely positive” narrative (coded as 1) declared the group process “FABULOUS” and his group a “real joy . . . amazing.” The great majority (88 percent) of narratives were coded as indicating either mostly positive or entirely positive experiences. This limited range of variation may raise questions about whether narratives were coded appropriately. However, I also coded the narratives for mentions of the student’s previous experiences with group work. About one fourth (n = 59) of the students’ narratives offered unsolicited comments on their experiences in other group projects; virtually all (n = 54) stated that the current experience had been better than their previous experience, which supports the validity of the coding scheme.
Groups’ overall average experience
Each group’s average overall experience was calculated as the mean of all individual group members’ experience. These mean scores for the research groups’ overall average experience ranged from 1.0 (in which all members of the group characterized their experience as entirely positive) to 3.5 (half the members said their experience was mixed and the other half said it was mostly negative).
Group dynamics: Free-riders, struggling students, leadership
Because instructors are less likely than students to perceive problems with group processes associated with group work (Liden et al. 1985), prior to coding I searched the literature for conceptualizations of students’ experiences with interpersonal dynamics in group projects. The central theme in both quantitative and qualitative studies of students’ negative experiences with group work was free-riding; this frequently was linked to problematic leadership dynamics—either students’ overt dominance or reluctance to assume leadership (Colbeck, Campbell, and Bjorklund 2000; Gillespie, Roos, and Slaughter 2006; Pauli et al. 2008; Robinson, Harris, and Burton 2015; Soetanto and MacDonald 2017; Stein, Colyer, and Manning 2016). Some studies suggested that students may misread others’ skill deficits or personal struggles as free-riding (Freeman and Greenacre 2011; Gillespie, Rosamond, and Thomas 2006; Micari and Drane 2011; Stein et al. 2016).
I coded the set of narratives for each group for evidence of free-riders and struggling students. About one fourth of the groups were coded as including a free-rider (that is, at least two members’ narratives identified a student who contributed significantly less to the group’s work and characterized the lack of contribution as willful). About one fifth of the groups were coded as including a struggling student (that is, at least two members’ narratives identified a student who struggled to contribute to the group’s work but acknowledged the lesser contribution as beyond that student’s control, such as a student who spoke English as a second language or had a learning difference).
The self-narrative prompt did not ask students to identify particular roles or tasks assumed by each group member, but most students included some specific information about the division of labor and explicitly discussed their own or others’ leadership or, occasionally, failure to lead. Consistent with prior studies (see Stein et al. 2016), I conceptualized leadership as distinct from knowledge and effort and as centered on galvanizing (“took charge,” “kept us focused/on track,” “motivated us,” “the catalyst”) or grounding (“the backbone,” “the rock,” “the glue,” “the mom”) the group in a productive way. I coded the set of narratives for each group to create a three-category nominal measure of leadership dynamics. Forty percent of the groups functioned as an ensemble, in which no students identified a leader(s) and none commented on lack of leadership as a problem for the group. In about one fourth of the groups, one or two students were acknowledged as the clear leader or co-leaders. In about one third of the groups, there was evidence of a leadership failure (students acknowledged that no one stepped up to lead and commented on the problems this caused), leadership conflict (a student tried to assume leadership but other group members blocked her efforts), or a leadership contest (students disagreed who the leader of the group was).
I had data on all 263 students’ grades in the course but self-assessment narratives for only 241 (91 percent) of these students. The 22 missing narratives had either been left stapled to the student’s paper when the student returned to retrieve the paper or (much less commonly) the student had never turned in the final paper and/or self-assessment narrative. In addition, one of these 241 students had been in the sole two-person group, and the partner’s self-assessment narrative was one of the 22 I did not have in my files. This group was removed from the multivariate analyses. The final sample included 240 students in 70 groups. See Table 1 for a summary of some of the sample characteristics for individual students and groups.
Sample Characteristics: Individual Students (N = 240) and Research Project Groups (N = 70).
Earned an A or A– on at least one midterm exam.
On scale of A = 4.0, B = 3.0, and so on.
1 = entirely positive; 2 = mostly positive; 3 = mixed; 4 = mostly negative.
Results
Do More Negative Experiences in Group Research Projects Diminish Learning?
Because the primary reason for assigning group research projects is the expected positive impact on individual students’ learning, I begin by using ordinary least squares regression to estimate several models of variation in students’ individual achievement on the final paper.
Individual achievement on final paper
First, I examined the relationship between individual students’ level of achievement on the final paper and the control variables of individual characteristics and group composition. These are variables that instructors often take into account when assigning research project groups.
The results presented in Table 2 show that in this initial model, none of the coefficients for the group composition variables are significant, but two of the coefficients for individual characteristics are: gender (0.239, p = .017) and average midterm exam grade (0.497, p < .000). The results show that after controlling for race, midterm exam average, and group composition, women’s final paper grades are about one fourth of a letter grade higher than men’s. Moreover, students with an average midterm exam grade of B are likely to earn a final paper grade that is a half-letter-grade higher than similar students with an average midterm exam grade of C. Surprisingly, net of the other variables in the model, nothing about the composition of a student’s group seems to matter for his or her final paper grade: neither the group size nor its gender and racial composition nor the other group members’ level of accomplishment on the midterm exams.
Ordinary Least Squares Regression of Students’ Individual Characteristics, Group Composition, Task Type, Group Achievement, and Overall Experience On Students’ Final Paper Grade (N = 234).
Note: SAQ = self-administered questionnaire; SER = standard error of regression.
Model 4 is the preferred model.
On scale of A = 4.0, B = 3.0, and so on.
SAQ is the excluded category because the distribution of paper grades for this method most closely resembles the overall distribution.
p < .05; **p < .01.
The second model controls for one measure of task type, the data collection method. None of the coefficients for the type of research method are significant, and the model’s fit is slightly worse (the adjusted R2 decreased from .352 to .348). This measure of task type is replaced in the third specification with a different measure, research topic. This time the model fit is slightly improved (the adjusted R2 increases to .360), and the coefficient for one type of research topic is significant (0.284, p = .026). The results suggest that net of other variables in the model, a student whose group researched something other than only H&WS students’ experiences earned a final paper grade that was more than a quarter of a letter grade higher than a student whose group studied the usual topic (how race/class/gender affects H&WS students’ experiences). The coefficients for students’ gender and midterm exam average are virtually unchanged and remain significant.
The fourth model adds the main independent variable (the group’s achievement on the research project) and interactions between this variable and students’ midterm exam average. The results show that students who were in groups with higher project grades did better on the final paper (0.442, p = .001) even after controlling for other variables in the model, although this was somewhat less true for students with average midterm grades in the bottom third of the grade distribution than for students with average midterm grades in the top third (–0.163, p = .024). With the addition of these variables, the group’s gender composition is now significant (–0.339, p = .033): net of other variables in the model, students in all-women groups earned final paper grades that were about a third of a letter grade lower than students in groups with two or more men. Students’ gender and average midterm exam grade remained significant, and the model fit was improved (the adjusted R2 increased to .395).
The fifth model controls for students’ overall experience with the group project. None of the coefficients for the dummy experience variables are significant, and the model fit is slightly worse (the adjusted R2 decreased slightly, to .392). The coefficients for gender, the group’s gender composition, the group grade on research project, and an interaction term between project grade and midterm exam average grade all remain largely unchanged and significant. The results show that after controlling for individual characteristics, group composition, task type, and level of group achievement on the group project, students’ self-reported experiences with the group research project do not explain variation in their final paper grades. In short, students do not have to have a good individual experience with the group project to realize learning gains from it.
Group achievement on group project
Although students’ individual experiences with the group project do not seem to directly affect their learning, negative experiences might still affect students’ learning indirectly by negatively impacting the group project grade, which appeared to be the single largest source of variation in students’ individual final paper grades. As a first cut at examining the correlates of the group project grade, I estimated a model that included several measures of the group’s composition: group size, gender composition, racial composition, and presence of one or more high-achieving students.
The results presented in Table 3 show that several of the coefficients for group composition are significant predictors of group achievement on the project, even though these do not predict individual students’ achievement on their final papers. Net of other variables in the model, groups of three students earned lower project grades than groups of four (–0.219, p = .034), groups with one man earned higher project grades than groups with two or more men (0.342, p = .002), and groups with one high-achieving student earned higher project grades than groups with none (0.421, p = .025). Coefficients for the racial composition of the group are not significant.
Ordinary Least Squares Regression of Group’s Composition, Task Type, and Group Members’ Mean Overall Experience with Group Project on the Group’s Grade on Research Project (N = 70).
Note: SER = standard error of regression.
Model 4 is the preferred model.
On scale of A = 4.0, B = 3.0, and so on.
Field research is the excluded category because the distribution of project grades for this method most closely resembles the overall distribution.
Higher values for this variable indicate more negative experiences: 1 = entirely positive; 2 = mostly positive; 3 = mixed; 4 = mostly negative.
p < .05; **p < .01.
The second model adds controls for one measure of task type, the method of data collection. None of the coefficients for particular methods are significant, but the coefficients for students’ gender, the gender composition of the group, and the presence of a high-achieving student in the group all increase in size and significance, and the overall fit of the model is improved (the adjusted R2 increases from .252 to .295). In addition, the coefficient for the range of variation in group members’ average midterm exam grades is now significant (–0.225, p = .012). This result indicates that groups with a wider range of variation in their members’ average scores on the midterm exams earn lower grades on the group project, net of other variables in the model. 4
The third model introduces an additional measure of task type, the research topic. None of the coefficients for type of research topic are significant, and the model fit is worse (the adjusted R2 declines from .295 to .277). Taken together, the results for Models 2 and 3 suggest that net of other variables in the model, there are no significant differences in the project grades earned by groups using different research methods or investigating different research topics.
The fourth model removes the controls for research topic and adds controls for group members’ mean overall experience. Recall that the overall average experience variable is reverse-coded; that is, higher values indicate more negative average experiences of group members. The addition of this variable improves the model fit (the adjusted R2 increased to .335), and its coefficient is significant. Net of other variables in the model, a group with a “mixed” overall average experience (mean experience score of 3) earned a group project grade that was about one third of a letter grade lower than a group with an “entirely positive” overall average experience (mean experience score of 1) (–0.168, p = .044). Once the group’s mean overall experience is controlled, the coefficient for the method of in-depth interviews becomes significant (0.281, p = .048). This result suggests that, net of other variables in the model and once negative experiences are controlled for, groups that conduct in-depth interviews earn project grades about one fourth to one third of a letter grade higher than groups that conduct field research (the excluded category). This finding is consistent with the somewhat bimodal distribution of project grades for groups using in-depth interviews. 5 The coefficients for groups of three, groups with exactly one man, groups with one high-achieving member, and the range of group members’ average midterm exam grade are somewhat smaller, but all remain significant in this model. In sum, groups’ grades on the research project are diminished by more negative overall average experiences of group members.
Do Group Dynamics Predict Negative Experiences in Group Research Projects?
Since negative overall average experiences with the group project do matter for the group’s grade on the project (even though individual students’ negative experiences do not predict their final paper grades), and since the group’s grade on the project in turn predicted students’ final paper grades, I turned next to an examination of the correlates of groups’ mean overall experiences with the research project. To sort out the relative influence of variables that instructors can control for when assigning groups at the outset versus the group dynamics that emerge once the group project is underway, I first examined the relationship between groups’ mean overall experiences and several aspects of group composition.
Table 4 shows that this initial model’s fit is poor (adjusted R2 = .057), but some of the coefficients are significant: groups of three students have more negative average experiences than groups of four (0.342, p = .030), groups with one man have less negative average experiences than groups with two or more men (–0.409, p = .014), and groups with a wider range of average midterm exam grades have more negative average experiences (0.274, p = .037).
Ordinary Least Squares Regression of Group’s Composition, Task Type, and Group Dynamics on the Group’s Overall Negative Experience with Group Project (N = 70).
Note: Positive coefficients indicate more negative experiences. SER = standard error of regression.
Model 4 is the preferred model.
On scale of A = 4.0, B = 3.0, and so on.
Content analysis is the excluded category because the distribution of group members’ average experience with this method most closely resembles the overall distribution.
p < .05; **p < .01.
The second model adds controls for one aspect of task type, method of data collection. None of the coefficients for research method are significant, and the model fit is very poor (the adjusted R2 decreased, to .009). The removal of this measure of task type and the addition of controls for research topic in the third model dramatically improves the model fit (adjusted R2 increased to .223), and the coefficient for one type of research topic is large and significant (0.610, p < .000). This result suggests that net of other variables in the model, groups whose research topic focused on how some aspect of H&WS students’ experiences were affected by an independent variable other than race/class/gender—most often, one or more recent sexual assault cases on our campus—had much more negative average experiences than groups whose research topic was the usual one (how race/class/gender affected some aspect of H&WS students’ experiences). The coefficients for groups of three and the range of group members’ average midterm exam grades remain significant, but the group’s gender composition is no longer significant, suggesting that the correlation between the group’s gender composition and the group’s overall average experience is mediated by the research topic.
The addition of several controls for group dynamics further improves the model fit (the adjusted R2 increases to .393). The coefficient for presence of a struggling student is not significant, nor is the coefficient for groups that function as an ensemble. These results suggest that there is not a significant difference in the overall average experiences of groups with and without struggling students; nor is there a significant difference in the overall average experiences of groups with and without clear leaders, so long as a group without clear leaders functions as an ensemble. In contrast, the coefficient for presence of free-rider is significant (0.331, p = .042), as is the coefficient for failed/conflictual/contested leadership (0.413, p = .020); both of these aspects of group dynamics are correlated with more negative overall average experiences for groups. Moreover, the coefficient for the range of average midterm exam grades in the group is no longer significant, suggesting that this aspect of group composition influences a group’s overall average experience via the group dynamics of free-riding and/or problematic leadership. The coefficient for groups of three students is smaller but remains significant (0.339, p = .011), suggesting that the more negative overall average group experiences associated with groups of three students only partially operate through these group dynamics.
Discussion
This study examined the relationship between the learning outcomes associated with group research projects, group dynamics, and students’ experiences in these groups, controlling for students’ individual characteristics, group composition, and task type. The answer to the question, “Do students have to like small-group pedagogy in order to learn from it?” is both no and yes. Individual students who reported more negative experiences with the group project were not more likely to earn lower grades on their final papers after controlling for other factors (individual characteristics, group composition, task types, and group project grades) that also predicted final paper grades (Table 2). However, groups whose members reported more negative overall average experiences were more likely to earn lower grades on the research project, even after controlling for aspects of group composition and task types that also predicted the group’s grade (Table 3), and (as noted above) an individual student’s grade on the final paper was influenced by the group’s grade on the project. In sum, students’ negative experiences indirectly affected their individual learning through the intervening variable of group achievement: individual students’ experiences were part of the overall average experience of their group, which predicted the group’s achievement on the project, which in turn predicted individual students’ subsequent learning.
One implication of these results is that instructors who are concerned about maximizing the learning outcomes associated with group projects should shift their focus from reducing individual students’ negative experiences with group projects to reducing groups’ overall average negative experiences. In this study, surprisingly few variables accounted for variation in groups’ mean overall average experience with the group project (Table 4), suggesting that attending to group size, exercising oversight on the research topic, and building in deterrents for free-riding and problematic leadership dynamics can reduce groups’ negative experiences. These results may vary in other institutional contexts; for example, the choice of research topic in group research projects may be less consequential for group dynamics and students’ experiences on larger campuses and commuter campuses, where students’ relationships with their peers may be more impersonal and where they may be less likely to encounter each other in multiple other contexts. It also is likely that the factors identified here as affecting students’ experiences are interrelated in complex ways and will have different implications in different institutional contexts. For example, this study’s results suggest that instructors should avoid composing groups with only three members, since this group size was correlated with lower project grades as well as more negative overall average experiences. On the other hand, larger groups will create exponentially more interpersonal relationships to manage (Johnson and Johnson 2006), and groups larger than five may experience more free-riding and difficulty identifying times to meet as a group (Colbeck et al. 2000). Instructors of larger classes may not have the option of creating groups as small as four or five students, particularly in the absence of additional resources, such as teaching assistants.
One reading of the results presented in Table 4 is that instructors who use group research projects (perhaps especially those who teach at smaller residential campuses) should steer away from highly charged research topics to minimize the likelihood of negative group dynamics and thus groups’ more negative overall average experiences. Perhaps such topics overwhelm some students, causing them to check out and become free-riders; perhaps such topics are particularly likely to lead to leadership battles or leadership vacuums. This interpretation is supported by the findings reported in Table 4, Model 4: the coefficient for the correlation between research topic and groups’ overall average experiences decreased in size once negative group dynamics (free-riding and problematic leadership) were controlled. Some instructors may be reluctant to discourage students from choosing highly charged research topics because these can stimulate students’ greater investment in the group project. These instructors could include a course unit on how researchers’ social locations can influence the kinds of research questions we are, and are not, inclined to ask and equipped to pursue, or include instruction in effective group dynamics either in the course itself (Caulfield and Persell 2006) or in departmental or general curricula (Colbeck et al. 2000; Gillespie, Rosamond, et al. 2006). Finally, some evidence suggests that so long as the instructor offers active, ongoing support, students’ experiences can be positive even if their group’s functioning is poor (Fiechtner and Davis 1984; Hillyard et al. 2010; Lizzio and Wilson 2005).
One challenge is that we do not know enough about how to deter free-riding and problematic leadership dynamics in group projects. Future research should more closely examine the predictors of negative group dynamics, particularly as these may have gendered dimensions (Takeda and Homburg 2014). In this study, free-riding was strongly correlated with students’ gender: more than three fourths (14/18) of the free-riders identified in student narratives were men, even though men were just one third of the overall sample. But in light of the findings presented in Table 4, free-riding may be situationally gendered; perhaps male free-riders are more likely to emerge in group projects that take up particular kinds of research topics (such as examining the effects of a recent sexual assault case on some aspect of students’ experiences). We need to know more about the group characteristics and social contexts, not just the individual characteristics, that are associated with the division of labor and leadership in group projects.
This study’s findings offer partial support to each of the several schools of thought regarding the design of small-group pedagogy and the relative importance of group dynamics. Collaborative learning’s more laissez-faire view of how to compose and monitor small groups is supported by the findings that individual learning (grade on final paper) was not directly influenced by the size of the group, the average level or range of group members’ prior achievement on midterm exams, or individual students’ experiences with the group project. On the other hand, cooperative learning’s general emphasis on the careful engineering of small groups is supported by the findings that individual learning was indirectly influenced by some aspects of group composition (size, gender composition, and members’ prior achievement on midterm exams) and by a group’s mean overall average experience, via the effects on group achievement. Within the broad umbrella of cooperative learning, proponents who particularly emphasize the importance of group dynamics for student learning in small-group pedagogy can point to the findings that free-riding and problematic leadership did diminish groups’ mean overall average experiences, which in turn influenced groups’ level of achievement on the group task. Finally, those who emphasize task type and those who emphasize incentive structures can point to the large majority of students reporting entirely or mostly positive experiences with the group research project in this study, which incentivized cooperation and group rewards by assigning a group grade for the level of achievement on an “ill-structured task,” which was virtually impossible for even the most motivated and competent students to complete on their own, and incentivized individual accountability by giving greater weight to the individual final paper grade than to the group grade on the project.
Footnotes
Acknowledgements
An earlier version of this paper was presented at the 2017 annual meeting of the American Sociological Association. I thank Phil Gleason, Jim Sutton, the editor, and three anonymous Teaching Sociology reviewers for thoughtful comments that improved the manuscript. I thank Benny Calderon and Swellar Zhuo for research assistance. Thanks also to Pamela Oliver of the University of Wisconsin–Madison, who generously shared her group project assignments and other course materials when I taught research methods for the first time.
Editor’s Note
Reviewers for this manuscript were, in alphabetical order, Susan Caulfield, Daina Eglitis, and Mary-Beth Raddon.
Funding
Partial funding for this research was provided by the Sills Family Fellowship and the Office of the Provost at Hobart and William Smith Colleges.
