Measuring University Students’ Group Work Contribution: Scale Development and Validation

Abstract

The purpose of this study is to validate the Group Work Contribution Scale, a scale that was developed for use as a self-assessment at the conclusion of a group project. Specifically, this scale measures the factors related to a learner’s participation within a school-based work team. A two-step validation process was conducted, using first exploratory and then confirmatory factor analysis. A total of 458 undergraduate students participated across two data collection sessions (231 and 227 participants). The refined Group Work Contribution Scale consists of four underlying factors: effort, initiative, responsibility, and backing-up behavior. The final 12-item scale for Group Work Contribution was confirmed via model fit indices.

Keywords

group work contribution self-assessment individual assessment

Over the past decade, researchers and practitioners alike have been interested in how to achieve more productive outcomes from collaborative efforts (Hirschfeld, Jordan, Feild, Giles, & Armenakis, 2006; Rousseau, Aubé, & Savoie, 2006). Collaborative learning environments and pedagogical activities such as team-based learning and group problem solving require students to work together and interact with each other (Hare, 1992; Molyneux, 2001; Mussnug & Hughey, 1997). When students work in groups, what they accomplish as a team is likely greater than each student could accomplish individually, so long as students are able to effectively work with each other.

Group work has several benefits in an education context. A major benefit is the opportunities for students in a group to integrate new ideas from peers who have various perspectives, knowledge, and experiences (Bower & Richards, 2006; Solomon & Schrum, 2007).

Students engaged in group work benefit not only from the synergistic outcomes of their group’s collective actions and resources but also from the opportunity to learn about group work skills (Felder & Brent, 2001; Jaques, 2000). Other advantages of using group work in education include the ability to focus on larger tasks than any one student could do alone, or to accomplish a task more quickly due to a shared workload.

Several researchers have conducted empirical studies of group work benefits on learning outcomes. One key finding in support of requiring group work is that students who work in collaborative groups retained information longer than students who work alone (Johnson & Johnson, 1989; McInnis & Devlin, 2002). In another study, students who participated in group work that included problem-solving activities had better performance on a critical-thinking test than students who individually worked on the same tasks (Gokhale, 1995). Group work has been found to support the development of critical-thinking skills (Fung & Howe, 2012), although teacher facilitation may be needed to fully support that learning (Fung, To, & Leung, 2016). In short, group work can result in meaningful gains in learning outcomes.

Although the benefits of group work and collaboration have been documented in these studies, research on the effectiveness and success of group work is still not conclusive. Other studies have examined adverse effects of group work. For example, individuals may avoid contributing substantially to group tasks by missing group meetings or shifting tasks to other group members (Comer, 1995; Sunwolf & Seibold, 1999). Students who fail to contribute to their groups engaged in social loafing. Social loafing is a behavior pattern in which one or more individuals make less effort or provide less input than their peers when performing in a group as compared with a context in which they would be expected to perform individually (Piezon & Ferree, 2008). In other words, social loafers make less of an effort to achieve a goal when they work together with others (Hallmark & Downs, 1987; North, Linley, & Hargreaves, 2000).

Social loafing can cause various problems in educational contexts. Several studies found that when one or more group members exhibited social loafing phenomena, it negatively influenced the entire group’s work (Aggarwal & O’Brien, 2008; North et al., 2000; Piezon & Ferree, 2008). For example, social loafing has been found to be related to lower performance on group tasks and decreased student satisfaction during group tasks (Bower & Richards, 2006; Piezon & Ferree, 2008). Students who experience difficulties when working with other students, such as teammates who do not participate actively or contribute equally, feel frustrated and anxious toward group work and are discouraged from participating in future group work (Finegold & Cooke, 2006; Gabriel, 2004). In addition, Myers and Anderson (2008) found that groups with social loafers have lower quality work products and morale, and that when one student engages in social loafing, others are prone to follow suit.

One key reason that social loafing occurs is the lack of individual assessment present in a group project scenario. Several studies have identified group assessment, instead of individual assessment, as a context in which social loafing is likely to occur (e.g., Dommeyer, 2007; Stangor, 2004; Sunwolf & Seibold, 1999). In other words, social loafers may believe that their personal work and contribution are not being monitored and evaluated when they have group tasks and feel less need to put forth an effort. At the same time, the assessment of group work most often focuses on a holistic examination of the group’s learning product (Xing, Wadholm, Petakovic, & Goggins, 2015), further promoting a learning environment in which social loafing may be encouraged or at least tolerated.

To identify or monitor students’ contributions in group work, social psychologists have suggested conducting peer evaluations to assess teammates’ individual participation and contribution and to determine the effectiveness of group work (Aggarwal & O’Brien, 2008; Dommeyer, 2007; Myers & Anderson, 2008). Clark (1989) found that using a peer evaluation process enhanced individual performance in collaborative learning group assignments. In this study, students who participated in peer evaluation had greater contributions to their groups, thereby reducing work inequity and conflict. In another study, the presence of a peer evaluation process was found to reduce intergroup conflict and resulted in more evenly distributed workloads (Cook, 1981).

Although using peer evaluation may solve some issues related to group work contribution, there are still limitations of peer assessment. One major limitation is reliance on students’ perceptions of each other’s performance. Students may not be able to judge the value of each other’s contributions to group work and may have their judgment clouded by other factors such as interpersonal conflicts. To fully gauge the magnitude of one’s contribution to a group effort includes reflection on an individual’s internal process; along this dimension, students are only qualified to comment on their own efforts. In consideration of this issue, some researchers have suggested using both self-assessment and peer assessment strategies to promote student participation and to enhance fairness in group work (e.g., Elliott & Higgins, 2005; Hanrahan & Isaacs, 2001; Johnston & Miles, 2004; Stefani, 1994). Self-assessment requires students to make judgments about their own work (Hanrahan & Isaacs, 2001). According to Dochy, Segers, and Sluijsmans (1999), the use of self-assessment increases “the role of students as active participants in their own learning, and is mostly used for formative assessment in order to foster reflection on one’s own learning processes and results” (p. 334). In addition, the use of both types of assessment will provide opportunities for the instructor to triangulate the individual’s contribution to group work (Butcher, Stefani, & Tariq, 1995).

Both forms of assessment have promise, but empirical studies have tended to focus more on peer assessment than self-assessment (e.g., Bamberger, Erev, Kimmel, & Roef-Chen, 2005; Druskat & Wolff, 1999; Erez, Lepine, & Elms, 2002). One reason may be the current lack of a validated tool for measuring an individual’s group work contribution. This study focuses on identifying the factors that are related to group work contribution that might be measured during a self-assessment. Using a two-step exploratory and confirmatory analysis process, a self-assessment questionnaire was developed and validated.

Key Elements of Self-Assessment in Group Work

Most university students are presented with multiple opportunities to engage in group work during their school years. However, when instructors simply assign students group work, the outcomes may not be the end product of a truly collaborative group process. Students may need help learning how to work together effectively, as well as encouragement to do so. If students lack accountability in this area, they may be less likely to collaborate or contribute sufficiently to their group’s work process. Several researchers have identified the relationship between individual assessment in groups and motivation to participate in group work, finding that the accountability of self-assessment motivates student performance in this context (e.g., Druskat & Wolff, 1999; Johnston & Miles, 2004; Lejk & Wyvill, 2001). Essentially, the use of individual assessment not only encourages students to work together but also helps students feel confident that they will be rewarded fairly for their individual contribution (Johnston & Miles, 2004).

This study focuses on three essential elements that explain a student’s behavior in group work: (a) effort, which refers to an earnest attempt to contribute to a group project/assignment; (b) responsibility, which focuses on how each student fulfills her or his duty as a group member; and (c) backing-up behavior, which refers to student contributions that go beyond completing their individual responsibilities and focus on helping other teammates with their share of the work to result in a successful group outcome. These three variables were selected based on the relationships with group work contribution in prior studies, as noted below. The variables are appropriate for inclusion in a self-assessment because students know their own levels of effort, responsibility, and backing-up behavior better than their peers.

Effort

Students participating in a group project may put forth varying levels of effort, but they often receive the same grade regardless of effort. These group grades reflect an assessment of the group’s work product, rather than their process (Johnston & Miles, 2004). It is difficult for instructors to assess individual performance within a group because final work products do not provide indicators about what each student contributed (Jaques, 2000). An individual’s effort toward group work reflects the manner in which a student performed to the best of his or her ability to achieve the shared goal. Because an instructor typically does not observe a group’s work process, students within a group are in the best position to make those judgments (Jaques, 2000; Strom & Strom, 1998), and each student is best poised to comment on his or her own effort. When students put forth their best efforts, the collaborative outcomes of the group should be strongest (Alden, 2011; Cox, 2003). For this reason, a full assessment of a group project that reflects an individual’s learning should not only consider the final group product but also consider the individual’s effort within the group. Within the questionnaire in Group Work Contribution Scale (GWCS), items in the effort category ask the degree of an individual’s effort within the group, such as whether individuals worked to the best of their abilities and were involved in the group discussion and shared their opinions.

Responsibility

Responsibility focuses on how each student fulfills his or her duty to the group, participating in meetings and completing assigned tasks. Whereas individual effort levels help determine the quality of group work outcomes, group work cannot be successfully accomplished if members shirk their responsibility and are not accountable to the group (Karakowsky & McBey, 2001; Kench, Field, Agudera, & Gill, 2009). All group members must be individually responsible to successfully achieve the desired goal of collaborative learning. If one teammate does not complete a part of the project, others must compensate, and the project loses a bit of the depth that is attained through the negotiation of multiple individuals with different knowledge and perspectives (Karakowsky & McBey, 2001).

To best achieve successful group work outcomes, all group members should perform their required duties (Myers & Anderson, 2008). Responsible group members participate in group meetings, clearly communicate and interact with others, share ideas, and complete assigned jobs (Burtis & Truman, 2006; Cohen & Bailey, 1997). Many groups, after collaboratively visualizing their project, divide specific tasks among themselves. Thus, for judging group work contribution, it is important to assess whether students completed their assigned tasks. In this questionnaire, the responsibility factor focuses on how each student performs their required duties as a group member to achieve the common goal. Items related to this factor ask students about whether they fully participated in group meetings, were punctual, and accomplished allocated tasks.

Backing-Up Behavior

When group members work on a common goal, they not only share information, opinions, and perspectives, but through their sharing process, also create a new intellectual product that integrates their ideas and collective resources (e.g., diversity of knowledge, skills, and experience) in a joint effort (Alden, 2011; Johnson, Johnson, & Smith, 2006; Salas, Sims, & Burke, 2005). In addition, students in a successful group support each other with assistance and encouragement (Johnson et al., 2006). This assistance and encouragement constitute backing-up behavior, which is a critical component of successful teamwork (Dickinson & McIntyre, 1997; Marks, Mathieu, & Zaccaro, 2001; McIntyre & Salas, 1995; Porter et al., 2003).

In this study, backing-up behavior refers to student contributions that are greater than the completion of their individual responsibilities. For example, backing-up behavior can be seen when a student stands in for another teammate who is unable to fulfill a role, provides feedback for the teammate, or corrects a team member’s mistakes. These behaviors are visible in the overt actions and verbal statements that occur when group members interact with each other (Rousseau et al., 2006). Backing-up behaviors are required of all group members to achieve a successful collaborative process and product.

Method

Participants and Procedures

The study population was comprised of undergraduate students who experienced team or group work, such as a group assignment or team project, at some point during their college years. Students who had prior group work experiences were eligible to participate, regardless of the class in which the group work occurred. Participants were asked to recall their most recent group work experience (including course subject, length of group project/assignment, and assessment of individual contribution), and to draw upon that experience when participating in this study. The participants were recruited from various courses at a large public university located in Southeastern United States.

The current study includes two sub-studies in which an initial questionnaire in GWCS was validated. Study 1 used exploratory factor analysis (EFA) to support item reduction and identify underlying factors within the questionnaire. Study 2 used a revised GWCS and used confirmatory factor analysis (CFA) to report model–data fit indices, indicating how well the model of GWCS fits to the data. Therefore, there were two separate rounds of data collection (i.e., one for the EFA process, and another for CFA). The EFA questionnaire contained 15 items, and the revised CFA questionnaire contained 12 items.

A total of 458 students participated across both studies, 231 during the EFA sub-study and 227 during the CFA sub-study. Table 1 summarizes the demographic information of these participants. Across both groups, there were slightly more female students (61%), and most were enrolled as juniors (n = 154, 33.62%) or seniors (n = 134, 29.26%). Participants represented a diverse array of majors, with the majority in the social sciences (n = 255, 55.68%), which included communication studies, economics, education, international studies, and political science.

Table 1.

Demographic Characteristics of Participants.

Characteristics	Number	Percentage
Gender
Male	177	38.65
Female	281	61.35
School year
1st year	71	15.50
2nd year	99	21.62
3rd year	154	33.62
4th year or above	134	29.26
Major
Art and humanities	80	17.47
Engineering and technology	46	10.04
Life sciences and medicine	9	1.96
Natural sciences	68	14.85
Social sciences	255	55.68

Total	458	100 (%)

Scale Development and Item Generation

The scale development process had four steps: literature review, expert review, EFA, and CFA. First, scale development began with a review of literature to identify relevant items. Based on the literature review, the three areas discussed above—effort, responsibility, and backing-up behavior—emerged as being both important to a group work contribution and appropriate for inclusion in a self-assessment scale. Across these three areas, 21 items were generated. There were 10 effort questions, five responsibility questions, and six backing-up behavior questions, all rated on a 5-point Likert-type scale.

During the second step, three experts reviewed the initial 21-item questionnaire that was based on the literature. The experts were two professors in instructional systems and a senior-level human performance practitioner. Based on their feedback, the questionnaire was modified, and six items were removed. The refined scale contained 15 items: seven effort questions, four responsibility questions, and four backing-up behavior questions (see Table 2).

Table 2.

The Initial Questionnaire for Group Work Contribution.

Category	Question
Effort	During group work, I
	1. Made the best use of my ability to accomplish a group project
	2. Did my equal share of a group project
	3. Was willing to undertake a task if I have the ability to perform the task
	4. Actively got involved in group discussions (e.g., brainstorming and idea sharing)
	5. Actively expressed my opinion to achieve better group outcomes
	6. Was thoroughly prepared for group meeting(s)
	7. Took notes of key points and important steps in group discussion
Responsibility	8. Never missed the scheduled group meeting(s)
	9. Was punctual for the scheduled meeting(s)
	10. Fulfilled allocated task(s)
	11. Accomplished my assigned task(s) on time
Backing-up behavior	12. Helped teammates who are unable to fulfill their roles
	13. Corrected teammates’ mistakes

The third and fourth steps of the scale development are reported in detail in this article. They comprise larger data collection efforts with undergraduate students as research participants and are reported as Study 1 (third step of scale development, using EFA) and Study 2 (fourth step of scale development, using CFA).

Study 1

The scale items used in this study were newly developed, and although they had undergone an expert review, there was no strong assumption about the number of factors underlying the measured variables. To obtain a condensed factor structure of the GWCS, EFA was performed repeatedly on the initial GWCS for scale purification and identification of underlying dimensions.

Participants and Procedure

Data collection for this study was conducted in multiple undergraduate courses. A total of 231 surveys were collected and analyzed. The initial EFA solution often does not yield simple structure and interpretable factors, so an oblique rotation was used in the analysis. The oblique rotation method is typically used when researchers assume that the underlying factors are related to each other (Thompson, 2004). The factors were extracted based on the following criteria: (a) low communality (i.e., fails to load highly on any factor), (b) weak correlation with other items in the same category, (c) cross-load on an unintended factor or large loadings on the wrong factor, and (d) factors are not interpretable. Data analysis was conducted in several stages using SPSS.

Results

The initial factor analysis indicated that the measured variables had a five-factor structure. Based on low communality, the first criterion of item elimination, Item 7 was deleted from the questionnaire. Then, EFA was run again, and the number of factors was reduced from five to four. In addition, the results showed that Item 6 was cross-loaded on both Factor 1 (.286) and Factor 3 (.252), so that item was deleted from the questionnaire. When EFA was run again, Item 11 was eliminated because of cross-load on an unintended factor (i.e., .485 on Factor 3 and .656 on Factor 1).

In total, the multiple rounds of EFA led to the exclusion of three items, resulting in a 12-item GWCS with a four-factor structure. The final EFA factor matrix is presented (see Table 3). The eigenvalue of each factor was larger than 1, and the total percentage of variance was 67.70. Three items were categorized into Factor 1 with the range of loadings from .60 to .72 (8.78% of the variance). Two items loaded on Factor 2 with the ranges of .74 and .79 (9.75% of the variance). Three items loaded on Factor 3 with the range of loadings from .63 to .77 (15.72% of the variance). The last four items were categorized into Factor 4 with the range of .57 to .92 (33.46% of the variance). Based on the results of the eigenvalues and pattern of loadings, a 4-factor solution was deemed most appropriate.

Table 3.

Finalized GWCS and Pattern Matrix of Final EFA.

	Factor
Questions: During group work, I	1	2	3	4
1. Made the best use of my ability to accomplish a group project.	−.716
2. Did my equal share of a group project.	−.615
3. Was willing to undertake a task if I have the ability to perform the task.	−.597
4. Actively got involved in group discussions (e.g., brainstorming and idea sharing).		−.786
5. Actively expressed my opinion to achieve better group outcomes.		−.740
6. Never missed the scheduled group meeting(s).			.772
7. Was punctual for the scheduled meeting(s).			.718
8. Fulfilled allocated task(s).			.632
9. Helped teammates who are unable to fulfill their roles.				.820
10. Corrected teammates’ mistakes.				.574
11. Provided constructive feedback on teammates’ work.				.675
12. Was willing to help others beyond my assigned tasks.				.924
Eigenvalue	1.053	1.170	1.886	4.015
Percentage of the variance	8.779	9.746	15.720	33.459

Note. GWCS = Group Work Contribution Scale; EFA = exploratory factor analysis.

Each factor was labeled as follows: effort (Factor 1), initiative (Factor 2), responsibility (Factor 3), and backing-up behavior (Factor 4). Factors 3 and 4, responsibility and backing-up behavior, remained the same as in the initial version of the survey. The original effort factor had five items that were divided into two separate factors, effort (three items) and initiative (two items). This latter factor (initiative) signifies the tendency for students to get actively involved in group activities, whereas the former relates to putting forth effort more generally. The overall alpha reliability coefficient was .81, and which is higher than the minimum requirement suggested (.60; Hair, Black, Babin, & Anderson, 2009). The alpha reliability coefficients for each factor were .68, .75, .68, and .85, respectively.

Study 2

Participants and Procedures

The aim of Study 2 was to validate scores on the refined GWCS after Study 1 was complete. The validity of the four-factor structure was tested using CFA. CFA is used to test whether the relationship between indicators and constructs is supported by the data. Specifically, the model hypothesized the following:

Hypothesis 1: Responses to the GWCS can be explained by the four factors (i.e., effort, initiative, responsibility, and backing-up behavior).

Hypothesis 2: Each item measures the assigned factor.

Hypothesis 3: The four factors are correlated to each other.

The finalized questionnaire from the Study 1 was presented to undergraduate students in several courses. Across these classes, 230 surveys were collected. Three surveys were eliminated because they were incomplete, leaving 227 surveys for data analysis. Multiple criteria were used to determine the goodness of fit to the data. Construct reliability and validity were assessed and reported to check the accuracy of GWCS. The data analysis was carried out using Mplus Version 6.12 and SPSS 19.

Model–Data Fit

The data set was checked to discern whether the multivariate normality assumption was met and determine whether maximum likelihood (ML) or robust maximum likelihood (MLR) estimation was most appropriate. This determination was based on skewness and kurtosis of univariate for each item (Curran, West, & Finch, 1996). The distribution of the observed variables should be assessed before conducting factor analysis because using the greater non-normality data with ML estimation produces inaccurate results (Bentler & Wu, 2002). ML is the default estimation procedure, but as explained above, the normality assumption should be met before using the ML approach. To identify the distribution of the observed variables, the mean, standard deviation, skewness, and kurtosis of each item were reported in Table 4 below. As seen in the table, skewness and kurtosis for the items ranged from −2.150 to −.269 and −.454 to 5.894 respectively. Compared with the results of univariate data normality and the cutoff point (skewness and kurtosis as ±2 and ±7), the value of skewness would seem slightly beyond the range. In addition, the value of multivariate kurtosis (54.152) suggests using the robust statistics as the value is out of the range of the recommended cutoff point (multivariate kurtosis as ±3) suggested by Bentler (2006). Based on the results of both univariate and multivariate normality, the MLR was determined as a more appropriate estimation method for further analysis than ML (Finney & DiStefano, 2006).

Table 4.

Mean, Standard Deviation, and Assessment of Normality.

Construct	M	SD	Skewness	Kurtosis
Effort 1	4.264	.863	−1.614	3.546
Effort 2	4.303	.978	−1.634	2.341
Effort 3	4.414	.834	−1.954	4.788
Initiative 1	4.303	.883	−1.715	3.599
Initiative 2	4.154	.896	−1.193	1.574
Responsibility 1	4.321	.972	−1.548	1.951
Responsibility 2	4.308	.918	−1.438	1.900
Responsibility 3	4.480	.806	−2.150	5.894
Backing-up 1	3.973	.954	−.774	.263
Backing-up 2	3.757	.830	−.269	−.454
Backing-up 3	3.660	.900	−.414	−.235
Backing-up 4	3.991	.902	−.635	−.182

The results of CFA are reported in Table 5. The value of chi-square for the four-factor model was tested to evaluate overall model–data fit. The chi-square test evaluates whether the observed covariance matrix is consistent with a specified model. If the chi-square value is zero, the model perfectly fits the data. The higher the chi-square value, the less accurate the model–data fit. In addition to the chi-square test, multiple criteria of fit indices—such as comparative fit index (CFI)/Tucker–Lewis Index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR)—were checked for model–data fit. Acceptable ranges for model–data fit are (Hair et al., 2009; Hu & Bentler, 1999) as follows: (a) CFI/TLI larger than .95 (good), range between .95 and .90 (acceptable), and smaller than .90 (poor); (b) RMSEA smaller than .05 (good), range between .05 and .08 (acceptable), and larger than .08 (poor); and (c) SRMR smaller than .08 (good), range between .08 and .10 (acceptable), and larger than .10 (poor).

Table 5.

Fit Indices of Four-factor Model.

χ²	p	df	CFI/TLI	RMSEA	SRMR
88.602	<.001	48	.967/.955	.061	.043

Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual.

The results of the model-fit tests indicated that the four-factor model fit well with the data, although the p value of the chi-square test is significant (see Table 5). The results of CFI and TLI are .967 and .955 respectively, and those values can be seen to indicate a good fit. The value of RMSEA is .061, which is in-between the acceptable fit range (.05 and .08). The result of SRMR is .043 and indicates the model fits well to the data. The factor structure of the four-factor model is illustrated in Figure 1.

Figure 1.

The standardized factor loadings, factor correlations, and errors variance of each item for the four-factor model with 12 items.

Construct validity

Construct validity determines whether a set of measured items actually represents the intended construct (Hair et al., 2009). To examine the construct validity of the GWCS, convergent validity and discriminant validity were investigated in this study.

Convergent validity

The items that are developed as indicators of a specific construct “should converge or share a high proportion of variance in common, known as convergent validity” (Hair et al., 2009, p. 709). The relative amount of convergent validity can be estimated in several ways. In this study, factor loadings and R², reliability, and average variance extracted (AVE) were used to identify the convergent validity.

The value of each factor loading is an important indicator of convergent validity. That is, the value of standardized loading estimates should be more than .50, and ideally higher than .70 (Hair et al., 2009). This is because the error variance (i.e., the percentage of variance that is not explained by the item) is calculated by one minus the square of a standardized factor loading. For example, if a factor loading score is .70 (R² = .49), the factor explains half the variance in the item (49%) and the other half is the error variance (51%). In other words, if a loading score is below .70, the percentage of pure error is slightly higher than the percentage that the factor explains the variance in the item. The results of factor loadings in Table 6 show that all items successfully converge on the corresponding factors with loadings higher than .70 except two items (Backing-Up Behavior Items 2 and 3). Although two items did not reach .70 (i.e., factor loading score of Item 2 is .671, Item 3 is .683), these measured loading scores were slightly under the determining score, and, more importantly, these scores were greater than .50.

Table 6.

The Results of Factor Loadings, R², Reliability, Corrected Item–Total Correlation, and AVE.

Construct	λ(R²)	Coefficient α	Corrected item–total correlation	AVE
Effort 1	.867 (.752)	.861	.755	.677
Effort 2	.783 (.613)		.681
Effort 3	.817 (.667)		.718
Initiative 1	.868 (.753)	.816	.681	.693
Initiative 2	.795 (.632)		.661	.693
Responsibility 1	.815 (.664)	.869	.614	.699
Responsibility 2	.889 (.790)		.686
Responsibility 3	.802 (.643)		.731
Backing-up 1	.780 (.608)	.830	.650	.552
Backing-up 2	.671 (.450)		.518
Backing-up 3	.683 (.466)		.516
Backing-up 4	.827 (.684)		.633

Note. AVE = average variance extracted.

The alpha reliability coefficient measures the consistency of items on a scale. The 12-item GWCS had an alpha reliability coefficient of .913, which is much higher than the minimum suggested coefficient of .70 (Hair et al., 2009) and suggests that the scale is reliable and contains internally consistent items. The alpha reliability coefficients for each dimension were observed as .861 for effort, .816 for initiative, .869 for responsibility, and .830 for backing-up behavior. The corrected item–total correlation ranged from .516 to .755, which is higher than .50, the point suggested by Hair et al. (2009, see Table 6). Based on these values, it can be said that the GWCS has met expectations for construct validity.

The AVE refers to the average percentage of variation explained among the items of a construct. The recommended cutoff value for AVE is higher than .50 (Fornell & Larcker, 1981). The AVE score for each construct in this study was greater than .50 (see Table 6), denoting that the variance explained by the items in each construct is greater than the variance associated with error variance. In sum, based on the three methods of assessing validity, this 12-item GWCS demonstrates robust convergent validity.

Discriminant validity

The overall construct validity can be identified by evaluating discriminant validity, which examines whether a construct is actually distinct from other constructs. In other words, the high discriminant validity indicates that the items in a construct only measure a specific phenomenon, which the other constructs in the same scale cannot (Hair et al., 2009). The discriminant validity is determined if the AVE value of each construct is greater than the squared correlation of other constructs. Because the values of AVE are more than the surrounding values in the correlation table, it indicates that the necessary condition of discriminant validity is satisfied as well (see Table 7).

Table 7.

Assessment of Discriminant Validity.

Construct	Effort	Initiative	Responsibility	Backing-up
Effort	.677^a
Initiative	.671	.693^a
Responsibility	.635	.420	.699^a
Backing-up	.403	.394	.310	.552^a

Note. AVE = average variance extracted.

AVE value for respective factors.

Discussion

The purpose of this study was to develop and validate the GWCS with undergraduate students. Through two rounds of data collection and factor analysis, a 12-item scale with four factors emerged. The initial model, which included three factors, was modified to a four-factor model because the model with four factors describes students’ group work contribution more accurately (see Appendix). These factors are effort, initiative, responsibility, and backing-up behavior, and they respectively represent a student’s earnest attempts, active involvement, fulfillment of assigned duties, and assistance to teammates during group work.

The findings from model–fit indices suggest that group work contribution is explained well through the 12-item scale with four factors. Each item had high factor loading scores, indicating collectively that these items can accurately measure group work contribution. The accuracy was further affirmed by results from various reliability and validity tests.

Limitations and Future Research

The GWCS was developed and validated through rigorous scale development procedures. Nonetheless, there are some limitations related to the process that might be addressed by future research. First, the scale might be validated with a larger sample. The sample size for each stage of factor analysis exceeded 200 in this study, but larger data sets could increase the generalizability of the findings. Conducting cross-validation, which reproduces the results with a large sample size from the same population, could help to confirm the measurement model as well as to strengthen the generalizability of the results.

Second, although the examination of skewness was not problematic (i.e., all but one item was in the range between −2 and +2), the mean scores and skewness of each item were still high. More accurate results might be obtained by using more segmented answer choices. For this reason, a 7-point Likert-type scale should be considered for more accurate results. A questionnaire with a 7-point Likert-type scale would give a chance for students to choose more interpretable answers as well as for researchers and instructors to obtain better distributed answers.

Third, the students participating in these two validation studies were recruited from across different classes and were asked to recall a prior group work experience when responding to the survey. As such, the participants were not engaged in fully authentic use of the scale. Although we did not inquire about this data point, the temporal distance between the experiences each student recalled and the administration of the survey would have varied, and in some cases, the group experience may have been several months prior. Memories fade with time, and students who had more recently experienced group work might have been able to more accurately respond to the scale items. It would be helpful to revalidate the scale using a large group of students who had recently engaged in the same group project assignment. For example, a future study might focus on collecting data from students in a large course or multiple sections of a course in which a common group project is used, with data collection occurring at a common point (e.g., x weeks post project).

Fourth, this scale is a self-assessment and, as such, represents only an individual’s reporting of his or her own work. It does not reflect outcomes from a student’s actions or how teammates perceived these actions. Individuals may be intentionally dishonest, or may exaggerate or underestimate the magnitude and value of their own contributions. Thus, this scale alone does not generate a full and accurate description of a student’s contribution to a group, but rather provides one perspective on that contribution.

Finally, this scale might be used in a study of student group work, in which survey responses could be triangulated with other data points, such as peer assessments and documents related to the group’s work process and accomplishments over time. This type of study could help ascertain the likelihood that students will respond truthfully to the scale items by cross checking when the instructor asks students for both peer assessments and self-assessments. It also could be used to determine the degree to which students can accurately engage in self-assessment of effort, initiative, responsibility, and backing-up behavior.

Use of the Scale

This scale could be used by instructors in both a formative and a summative sense. By providing the scale to students at the beginning of assigned group work, an instructor would be communicating to students which behaviors are desirable for group members. During long group projects, the scale could be used as a self-assessment at the midpoint of the project work. If results were shared with the instructor at this time, the scale would provide a starting point for a discussion on how, and in which areas, to improve one’s contribution to the group. Finally, instructors might use the self-assessment in conjunction with peer assessment and other relevant data points to individualize student scores on group assignments.

Conclusion

We believe that the GWCS is a useful tool to assess students’ group work contribution across a variety of group types and tasks in academia. The scale is brief and simple to score, which means it could be implemented with relatively little time and effort in a university setting. The use of the GWCS will not only provide instructors with useful information to assist with assessing each student’s contribution to group work, but it also reminds students that their individual contributions are important.

The GWCS can only be used as a self-assessment. An individual’s group work contribution is internal and unstable, meaning that other teammates cannot recognize and accurately report on the degree of another student’s effort or initiative. In some instances, what the individual perceives as a large effort or a high degree of initiative may not be recognized by teammates or may not yield a meaningful contribution to the final product generated by the team. Nonetheless, the individual may have expended a great deal of time and energy in an attempt to contribute to the group. Due to these differences in awareness and perception, we recommend concurrently conducting self-assessment and peer assessment to give an opportunity for instructors to compare and integrate the results from both assessments.

Footnotes

Appendix

Appendix.

The Group Work Contribution Scale (GWCS) With Sub-Categories.

Category	Question
Effort	During group work, I
	1. Made the best use of my ability to accomplish a group project
	2. Did my equal share of a group project
	3. Was willing to undertake a task if I have the ability to perform the task
Initiative	4. Actively got involved in group discussions (e.g., brainstorming and idea sharing)
Initiative	5. Actively expressed my opinion to achieve better group outcomes
Responsibility	6. Never missed the scheduled group meeting(s)
	7. Was punctual for the scheduled meeting(s)
	8. Fulfilled allocated task(s)
Backing-up behavior	9. Helped teammates who are unable to fulfill their roles
	10. Corrected teammates’ mistakes
	11. Provided constructive feedback on teammates’ work
	12. Was willing to help others beyond my assigned tasks

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Min-Ho Joo is an assistant professor in the Department of Educational Technology at Konkuk University, Korea. His research focuses mainly on group behavior, social networking, and collaborative learning environment.

Vanessa P. Dennen is a professor and program coordinator of Instructional Systems & Learning Technologies at Florida State University, USA, and the co-editor in chief of The Internet and Higher Education. Her research investigates the cognitive, motivational, social, and ethical elements of computer-mediated communication.

References

Aggarwal

O’Brien

C. L.

(2008). Social loafing on group projects: Structural antecedents and effect on student satisfaction. Journal of Marketing Education, 30, 255-264. doi:10.1177/0273475308322283

Alden

(2011). Assessment of individual student performance in online team projects. Journal of Asynchronous Learning Networks, 15(3), 5-20.

Bamberger

P. A.

Erev

Kimmel

Oref-Chen

(2005). Peer assessment, individual performance, and contribution to group processes: The impact of rater anonymity. Group & Organization Management, 30, 344-377.

Bentler

P. M.

(2006). EQS 6 Structural Equations Program Manual. Encino, CA: Multivariate Software.

Bentler

P. M.

E. J. C.

(2002). EQS for windows user’s guide. Encino, CA: Multivariate Software.

Bower

Richards

(2006, December). Collaborative learning: Some possibilities and limitations for students and teachers. Paper presented at the 23rd Annual Meeting of Australasian Society for Computers in Learning in Tertiary Education Conference: Who’s learning? Whose technology? Sydney, New South Wales.

Burtis

J. O.

Turman

P. D.

(2006). Group communication pitfalls: Overcoming barriers to an effective group experience. Thousand Oaks, CA: Sage.

Butcher

Stefani

Tariq

(1995). Analysis of peer-, self-, and staff-assessment in group project work. Assessment in Education, 2, 165-185. doi:10.1080/0969594950020204

Clark

G. L.

(1989). Peer evaluation: An empirical test of their validity and reliability. Journal of Marketing Education, 11(3), 41-58. doi:10.1177/027347538901100308

10.

Cohen

S. G.

Bailey

D. E.

(1997). What makes teams work: Group effectiveness research from the shop floor to the executive suite. Journal of Management, 23, 239-290. doi:10.1177/014920639702300303

11.

Comer

D. R.

(1995). A model of social loafing in real work groups. Human Relations, 48, 647-666. doi:10.1177/001872679504800603

12.

Cook

R. W.

(1981). An investigation of student peer evaluation on group project performance. Journal of Marketing Education, 3(1), 50-52. doi:10.1177/027347538100300114

13.

Cox

K. B.

(2003). The effects of intrapersonal, intragroup, and intergroup conflict on team performance effectiveness and work satisfaction. Nursing Administration Quarterly, 27, 153-163.

14.

Curran

P. J.

West

S. G.

Finch

J. F.

(1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29. doi:10.1037/1082-989X.1.1.16

15.

Dickinson

T. L.

McIntyre

R. M.

(1997). A conceptual framework for teamwork measurement. In Brannick

M. T.

Salas

Prince

(Eds.), Team performance assessment and measurement: Theory, methods, and applications (pp. 19-43). Mahwah, NJ: Lawrence Erlbaum.

16.

Dochy

Segers

Sluijsmans

(1999). The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24, 331-350. doi:10.1080/03075079912331379935

17.

Dommeyer

C. J.

(2007). Using the diary method to deal with social loafers on the group project: Its effects on peer evaluations, group behavior, and attitudes. Journal of Marketing Education, 29, 175-188. doi:10.1177/0273475307302019

18.

Druskat

V. U.

Wolff

S. B.

(1999). Effects and timing of developmental peer appraisals in self-managing work groups. Journal of Applied Psychology, 84, 58-74. doi:10.1037/0021-9010.84.1.58

19.

Elliott

Higgins

(2005). Self and peer assessment—Does it make a difference to student group work? Nurse Education in Practice, 5, 40-48. doi:10.1016/j.nepr.2004.03.004

20.

Erez

Lepine

J. A.

Elms

(2002). Effects of rotated leadership and peer evaluation on the functioning and effectiveness of self-managed teams: A quasi-experiment. Personnel Psychology, 55, 929-948. doi:10.1111/j.1744-6570.2002.tb00135.x

21.

Felder

R. M.

Brent

(2001). Effective strategies for cooperative learning. Journal of Cooperation & Collaboration in College Teaching, 10(2), 69-75.

22.

Finegold

A. R.

Cooke

(2006). Exploring the attitudes, experiences and dynamics of interaction in online groups. The Internet and Higher Education, 9, 201-215. doi:10.1016/j.iheduc.2006.06.003

23.

Finney

S. J.

DiStefano

(2006). Non-normal and categorical data in structural equation modeling. In Hancock

G. R.

Mueller

R. O.

(Eds.), Structural equation modeling: A second course (pp. 269-314). Greenwich, CT: Information Age.

24.

Fornell

Larcker

(1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39-50. doi:10.2307/3151312

25.

Fung

Howe

(2012). Liberal studies in Hong Kong: A new perspective on critical thinking through group work. Thinking Skills and Creativity, 7, 101-111. doi:10.1016/j.tsc.2012.04.002

26.

Fung

Leung

(2016). The influence of collaborative group work on students’ development of critical thinking: The teacher’s role in facilitating group discussions. Pedagogies: An International Journal, 11, 146-166.

27.

Gabriel

(2004). Learning together: Exploring group interactions online. Journal of Distance Education, 19, 54-72.

28.

Gokhale

A. A.

(1995). Collaborative learning enhances critical thinking. Journal of Technology Education, 7, 22-30. doi:10.21061/jte.v7i1.a.2

29.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

(2009). Multivariate data analysis. Upper Saddle River, NJ: Pearson.

30.

Hallmark

J. R.

Downs

T. M.

(1987, November). Group participation in the organization: Social loafing as a limitation of group effectiveness. Paper presented at the 73rd Annual Meeting of the Speech Communication Association, Boston, MA.

31.

Hanrahan

S. J.

Isaacs

(2001). Assessing self-and peer-assessment: The students’ views. Higher Education Research & Development, 20, 53-70. doi:10.1080/07294360123776

32.

Hare

A. P.

(1992). Groups, teams, and social interaction: Theories and applications. New York, NY: Praeger.

33.

Hirschfeld

R. R.

Jordan

M. H.

Feild

H. S.

Giles

W. F.

Armenakis

A. A.

(2006). Becoming team players: Team members’ mastery of teamwork knowledge as a predictor of team task proficiency and observed teamwork effectiveness. Journal of Applied Psychology, 91, 467-474. doi:10.1037/0021-9010.91.2.467

34.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55. doi:10.1080/10705519909540118

35.

Jaques

(2000). Learning in groups: A handbook for improving group work (3rd ed.). New York, NY: RoutledgeFalmer.

36.

Johnson

D. W.

Johnson

(1989). Cooperation and competition: Theory and research. Edina, MN: Interaction Book.

37.

Johnson

D. W.

Johnson

R. T.

Smith

K. A.

(2006). Active learning: Cooperation in the college classroom. Edina, MN: Interaction Book.

38.

Johnston

Miles

(2004). Assessing contributions to group assignments. Assessment & Evaluation in Higher Education, 29, 751-768. doi:10.1080/0260293042000227272

39.

Karakowsky

McBey

(2001). Do my contributions matter? The influence of imputed expertise on member involvement and self-evaluations in the work group. Group & Organization Management, 26, 70-92. doi:10.1177/1059601101261005

40.

Kench

P. L.

Field

Agudera

Gill

(2009). Peer assessment of individual contributions to a group project: Student perceptions. Radiography, 15, 158-165. doi:10.1016/j.radi.2008.04.004

41.

Lejk

Wyvill

(2001). The effect of the inclusion of self-assessment with peer assessment of contributions to a group project: A quantitative study of secret and agreed assessments. Assessment & Evaluation in Higher Education, 26, 551-561.

42.

Marks

M. A.

Mathieu

J. E.

Zaccaro

S. J.

(2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26, 356-376. doi:10.2307/259182

43.

McInnis

Devlin

(2002). Assessing learning in Australian Universities: Ideas, strategies and resources for quality in student assessment. Australian Universities Teaching Committee, Centre for the Study of Higher Education. Retrieved from http://www.cshe.unimelb.edu.au/assessinglearning

44.

McIntyre

R. M.

Salas

(1995). Measuring and managing for team performance: Emerging principles from complex environments. In Guzzo

R. A.

Salas

(Eds.), Team effectiveness and decision making in organizations (pp. 9-45). San Francisco, CA: Pfeiffer.

45.

Molyneux

(2001). Interprofessional teamworking: What makes teams work well? Journal of Interprofessional Care, 15, 29-35. doi:10.1080/13561820020022855

46.

Mussnug

K. J.

Hughey

A. W.

(1997). The truth about teams. Training for Quality, 5, 19-25. doi:10.1108/09684879710156487

47.

Myers

S. A.

Anderson

C. M.

(2008). The fundamentals of small group communication. Thousand Oaks, CA: Sage.

48.

North

A. C.

Linley

P. A.

Hargreaves

D. J.

(2000). Social loafing in a cooperative classroom task. Educational Psychology, 20, 389-392.

49.

Piezon

S. L.

Ferree

W. D.

(2008). Perceptions of social loafing in online learning groups: A study of public university and U.S. Naval War College students. The International Review of Research in Open and Distance Learning, 9(2), 1-17. doi:10.19173/irrodl.v9i2.484

50.

Porter

C. O.

Hollenbeck

J. R.

Ilgen

D. R.

Ellis

A. P.

West

B. J.

Moon

(2003). Backing up behaviors in teams: The role of personality and legitimacy of need. Journal of Applied Psychology, 88, 391-403. doi:10.1037/0021-9010.88.3.391

51.

Rousseau

Aubé

Savoie

(2006). Teamwork behaviors: A review and an integration of frameworks. Small Group Research, 37, 540-570. doi:10.1177/1046496406293125

52.

Salas

Sims

D. E.

Burke

C. S.

(2005). Is there a “big five” in teamwork? Small Group Research, 36, 555-599.

53.

Solomon

Schrum

(2007). Web 2.0: New tools, new schools. Washington, DC: International Society for Technology in Education.

54.

Stangor

(2004). Social groups in action and interaction. New York, NY: Psychology Press.

55.

Stefani

L. A.

(1994). Peer, self and tutor assessment: Relative reliabilities. Studies in Higher Education, 19, 69-75. doi:10.1080/03075079412331382153

56.

Strom

R. D.

Strom

P. S.

(1998). Student participation in the evaluation of cooperative learning. Community College Journal of Research and Practice, 22, 265-278. doi:10.1080/1066892980220306

57.

Sunwolf Seibold

D. R

. (1999). The impact of formal procedures on group processes, members, and task outcomes. In Gouran

D. S.

Poole

M. S.

(Eds.), The handbook of group communication theory and research (pp. 395-431). Thousand Oaks, CA: Sage.

58.

Thompson

(2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association.

59.

Xing

Wadholm

Petakovic

Goggins

S. P.

(2015). Group learning assessment: Developing a theory-informed analytics. Journal of Educational Technology & Society, 18, 110-128.