Student Perceptions of Collaboration Skills in an Interprofessional Context: Development and Initial Validation of the Self-Assessed Collaboration Skills Instrument

Abstract

An integral component of interprofessional education (IPE) is the development of a collaboration-ready health-care workforce. While collaboration is a fundamental element of IPE, there is no existing measure of collaboration skills that is not context specific. This article describes the development and initial validation of the Self-Assessed Collaboration Skills (SACS) measure. Items were initially drawn from the Collaboration Skills Assessment Tool rubric, an educational assessment tool. The SACS measure was piloted in a sample of students in an introductory IPE course. Following scale revision, the SACS was piloted a second time in a sample of students in an IPE health systems course and then validated in a sample of students in an introductory IPE course. Exploratory factor analysis was used to assess scale factor structure in Pilots 1 and 2 and confirmatory factor analysis to confirm factor structure in the validation sample. Convergent and discriminant validity were also assessed. The final SACS measure is an 11-item scale consisting of three dimensions of collaboration: information sharing, learning, and team support. The SACS measure demonstrates high internal consistency and both convergent and discriminant validity as a measure of collaboration. The SACS can be implemented in any setting for assessing collaboration in clinical and nonclinical contexts.

Keywords

interprofessional collaboration interprofessional education quantitative methods teams

Background

Interprofessional education (IPE), defined as education that “occurs when students (learners) from two or more professions learn about, from, and with each other to enable effective collaboration and improve health outcomes” (World Health Organization, 2010, p. 10), holds as a fundamental tenet the ability for learners to work together to be “collaborative practice-ready” as they enter the workforce (World Health Organization, 2010, p. 7). Collaboration—“mutual engagement of participants in a coordinated effort to solve the problem together” (Dillenbourg, Baker, Blaye, & O’Malley, 1996, p. 195)—is an integral component of IPE and interprofessional collaborative practice (IPCP), the skill that most fundamentally promotes the “working together” notion of interprofessionalism. While collaboration is often discussed as a fundamental component of IPE (Thistlethwaite & Dallest, 2014), the majority of research on the impact of IPE on student learning outcomes focuses on measures of student attitudes and beliefs about IPE or on readiness to engage in IPE (Parsell & Bligh, 1999). Numerous scholars have urged for more work examining the impact of IPE activities on collaboration and teamwork skills that form the foundation of IPCP (Reeves, Lewin, Espin, & Zwarenstein, 2011; Thistlethwaite et al., 2014).

Despite these numerous calls to action, few researchers are attempting to fill this void in the literature. One of the barriers to assessing collaboration skills as an outcome is the lack of a readily accessible measure of collaboration skills that is not context specific. Most current measures of collaboration or interprofessional collaboration are specific to clinical settings (Baggs, 1994) and do not provide the ability to measure collaboration skills in other settings. Other measures examining perceived competencies in IPE typically examine readiness for or attitudes toward IPE rather than focusing on the specific skills related to collaborative behavior in a team setting (Parsell & Bligh, 1999). For example, the Interprofessional Collaborator Assessment Rubric (ICAR; Curran et al., 2011) was developed to provide educators and practitioners with a method to assess and evaluate health professionals’ competencies in providing collaborative care including communication, collaboration, roles and responsibilities, collaborative patient/client–family-centered approach, team functioning, and conflict resolution/management. The original development of the instrument was completed using focus groups and items were evaluated based on their importance and clarity (Curran et al., 2011), which reflect rigorous methods for test development (Crocker & Algina, 1986). However, no further analysis, such as exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), was completed to examine the factor structure or confirm the dimensions presented. Identifying factor structure and confirming factor dimensions are necessary in the development and validation of a measure. Additionally, the item content in the ICAR is context specific such that the items refer to specific clinical settings. This measure is thus not suitable for use in an educational, undergraduate, and preprofessional setting in which students are yet to have clinical exposure or experience.

The Interprofessional Socialization and Valuing Scale (ISVS) was developed to help educators examine whether transformational learning in IPE had taken place (King, Shaw, Orchard, & Miller, 2010) and examines students’ beliefs, attitudes, and confidence in their ability to work interprofessionally. While useful in gauging students’ confidence in their ability to be collaborative, the scale does not examine the extent to which they use these skills and behaviors in team settings. Additionally, this scale included evidence regarding its initial development, and further analysis regarding its validation including CFA and discriminant/convergent validity was not conduct (King et al., 2010).

The Index of Interdisciplinary Collaboration (IIC; Bronstein, 2002) looks at the level of collaboration between colleagues and examines the extent to which the individual engages in collaborative behavior along five dimensions: interdependence, newly created professional activities, flexibility, collective ownership of goals, and reflection on process. While statistical analyses and results are reported for the development of the scale, the original survey development study included no information on its validation. Moreover, the scale itself examines collaboration specifically in reference to interdisciplinary colleagues and includes items that examine individual behaviors and the behaviors that occur between colleagues. Therefore, it is difficult to assess the extent to which the individual possesses skills in collaboration.

Although it can be used at the baccalaureate level, the Teamwork and Collaboration subscale from Readiness for Interprofessional Learning Scale (RIPLS; Parsell & Bligh, 1999) assesses student readiness for shared learning experiences and focuses on the students’ attitudes toward teamwork and collaboration rather than on the behaviors associated with these skills. Additionally, while statistical analyses and results are reported for the development of the scale, the original survey development study included no information on its validation. McFadyen and colleagues (2005) worked to increase the validity of the RIPLS, including the production of a more stable subscale model; however, the focus of the revised RIPLS remains readiness for learning which, while related to collaboration, is a fundamentally distinct construct.

The Modified Collaboration and Satisfaction about Care Decision tool focuses on the exchanges between nurses and physicians that exemplify collaborative behavior. The tool was developed for use in an intensive care unit and is not suitable for use in more general, nontechnical settings. Other measures of interprofessional collaboration were similarly measured in a specific context and examining the behaviors of the team as a whole rather than focusing specifically on the individual (Orchard, King, Khalili, & Bezzina, 2012).

The organizational psychology literature also provides some insight into the measurement of teamwork and collaboration; however, this area of literature focuses broadly on the teamwork skills as interpersonal processes within teams that contribute to team effectiveness and performance rather than on the specific behaviors associated with collaboration and learning processes, in particular with no measures to that effect (Marks, Mathieu, & Zaccaro, 2001; Morgeson, Reider, & Campion, 2005; Salas, Sims, & Burke, 2005). While measures of collaborative relationships exist within the organizational psychology literature (Aram, Morgan, & Esbeck, 1971; Drach-Zahavy & Somech, 2001; West & Wallace, 1991), they provide measure of the construct at the team or unit level, not the level of the individual. We were unable to identify any existing measures, in the psychological literature or otherwise, that assessed collaborative behaviors at the individual level rather than at the level of group or team. As ability to collaborate is an individual-level trait (albeit one that will likely interact with team-level characteristics when examining team interactions), and one of the skills IPE seeks to develop is for learners to become collaboration-ready at the individual level, group-level assessment is not ideal. Considering these gaps in the measurement of collaboration in the IPE field, the objective of this study was to develop a measure of collaboration skills suitable for use in both nonclinical and clinical interprofessional settings.

The Collaboration Self-Assessment Tool Rubric (CSAT)

Although searches did not uncover existing measures of collaboration for use in nonclinical settings, we did uncover the CSAT rubric (Ofstedal & Dahlberg, 2009), an educational rubric used as a tool to develop awareness of one’s collaboration skills. The CSAT rubric was developed by a team of educators to help generate conversations about the development of collaborative skills and focused on both intrapersonal and interpersonal skills the developers believed to be associated with collaboration. As with many educational rubrics, the CSAT rubric identifies multiple skills associated with collaboration. Each skill is rated from 1 to 4 with examples of each level of proficiency described within the rubric. Upon inspection of the rubric, we determined the foundational components of collaboration behaviors were described within the rubric; however, several problematic issues prevented it from being utilized in empirical settings. First, the CSAT rubric was developed for educational rating to encourage development in students, not for an empirical assessment of collaboration. Second, no examination of the validity of the measure for assessing collaboration was reported. Third, there were additional issues of double-barreled items, coverage of all collaboration-related content areas, and the 4-point response scale that might enlarge floor and ceiling effects of the measure. Given these issues, the research team decided to revise the CSAT to become a measure of collaboration that could be utilized for research purposes as well as individual student assessment. This study was approved by the Saint Louis University Institutional Review Board.

Methods

The original CSAT rubric contained 10 rating areas in two listed domains—intrapersonal skills and interpersonal skills. Rating areas for the intrapersonal skills domain were motivation/participation, quality of work, time management, preparedness, and reflection. Rating areas for the interpersonal skills domain were contribution, team support, team dynamics, interactions with others, and role flexibility. Specific behaviors for each level of proficiency within each rating area were listed in the rubric (see Appendix A—The CSAT Rubric). Based on previous research on collaboration and learning skills, it is argued that the two domains of skills identified here refer to two distinct constructs: one on individual work ethic or conscientiousness (intrapersonal skills) and one on learning and work behaviors in a group setting (interpersonal skills). The analyses below were conducted following this previous research, analyzing the interpersonal and intrapersonal skills items as indicators of two separate dimensions. Only the results regarding the interpersonal skills items are discussed, as these are the items that cover the collaboration skills domain and were used to develop the Self-Assessed Collaboration Skills (SACS) instrument. Considering the importance of domain coverage in psychometric development and validation (Crocker & Algina, 1986), we believed that the interpersonal skills items as identified in the CSAT do not comprehensively cover the collaboration domain. Past research emphasized that perceptions of skills such as time management, motivation, and preparedness relate to conscientiousness and work ethic (Barrick & Mount, 1991; Miller, Woehr, & Hudspeth, 2002) rather than a primary focus on one’s ability to collaborate with others. While these skills may demonstrate convergent validity with measures of teamwork and collaboration, the underlying construct is still conceptually different from collaboration (Tasa, Sears, & Schat, 2011), as conscientiousness may predict one’s level of collaboration but it does not reflect one’s level. Additionally, we chose to develop the measure as a self-report tool for ease of administration within multiple contexts. Self-report measures can provide useful and insightful information on individuals’ interpersonal skills and abilities on constructs such as collaboration (Ferris et al., 2005; Riggio & Riggio, 2001). As such, although the measure is self-report, the SACS is a self-reported assessment of skills and behaviors and not their perceptions about the value or attitude toward those behaviors.

Item Development

We parsed out the components within each CSAT rating area to become the baseline behavior items for the SACS instrument. SACS items were scaled on a 1–7 Likert-type scale from strongly disagree to strongly agree. For example, the CSAT rubric proficiency ratings 1–4 of the motivation/preparation rating area read, (1) “I tend not to participate or remain engaged when a project moves away from my own immediate interests,” (2) “I sometimes make an effort to participate and remain engaged when a project moves away from my own immediate interests,” (3) “I often make an effort to participate and remain engaged even when a project moves away from my own immediate interests,” and (4) “I can be relied on to participate and remain engaged even when a project moves away from my own immediate interests.” The revised item for the SACS became “I remain engaged in group projects even when a project moves away from my own immediate interests.” Items in the SACS Drafts 1, 2, and 3 can be found in Table 1. Three pilot tests were conducted to examine item performance and scale characteristics. Demographic breakdowns of the sample for each pilot are described in Table 2.

Table 1.

Item Pool for SACS Development—Phases 1, 2, and 3.

Dimension	Phase 1	Phase 2	Phase 3
Contribution
1. I freely share ideas	X	X
2. I do not share resources with others easily (R)	X	X	X
3. I share information with others easily	X	X	X
4. It is hard for me to share my ideas with others (R)	X	X	X
5. I share resources with others with ease	X
Team support
6. I represent the team in a positive manner when I am in other settings	X
7. I publicly support the team	X	X
8. I represent the efforts of the team well when I am in different settings	X
9. I publicly support the work of my fellow team members	X	X
Team dynamics
10. I consistently know how to gauge my own impact on the group	X	X
11. I am regularly aware of team dynamics	X	X
Interactions with others
12. I consistently respect the efforts of others	X
13. I routinely listen to the opinions of my fellow team members	X	X	X
14. I regularly acknowledge the efforts of my team members	X	X	X
15. I consistently support the efforts of others	X	X	X
Role flexibility
16. I can move easily between leader and follower assuming either role as needed to accomplish the task	X
17. I am comfortable moving between the role of leader and follower	X
18. I am uncomfortable functioning outside my perceived role (R)	X
Motivation/participation
19. I can be relied on to participate in a group project	X
20. I remain engaged in group projects even when a project moves away from my own immediate interests	X
21. I make an effort to participate in a group project	X
22. I tend to participate in a group project	X
Quality of work
23. My work reflects my best efforts	X
24. I continuously make small changes to improve the quality of my work	X
25. My work reflects a high attention to detail	X
Time management
26. I routinely use time well to ensure things are done on time	X
27. I plan the amount of time I allocate to specific tasks	X
28. I allocate time to tasks appropriately to make sure that projects are done in a timely manner	X
Preparedness
29. I regularly bring the materials I need to complete the work	X
30. I consistently come ready to work	X
31. I routinely come to team meetings without the materials I need (R)	X
32. I frequently forget the materials I need to complete the work (R)	X
Reflection
33. I consistently use self-reflection after collaborative activities	X
34. I regularly use self-reflection after engaging in collaborative activities with team members	X
35. I rarely engage in self-reflection after collaborative activities (R)	X
Team learning
36. I frequently seek feedback from my team members about the quality of my work		X	X
37. I routinely go out and get all the information I can from my teammates		X	X
38. I discuss my views on a team project even if those views are different from the team to engage the team in an open discussion		X
39. I encourage other team members to get involved in the decisions that affect the team		X	X
40. I voice my ideas about how the team could work better together		X	X
41. I value the opinions of others when working on a team project		X	X
42. I seek out different views than my own during team discussions		X	X
43. I encourage others to share their opinions		X

Table 2.

Demographic Characteristics for First, Second, and Third Pilot Samples.

	First Pilot Sample, N = 160	Second Pilot Sample, N = 131	Third Pilot Sample, N = 182
Characteristics	Mean (SD) or n (%)	Mean (SD) or n (%)	Mean (SD) or n (%)
Age	18.77 (0.73)	19.33 (2.80)	18.68 (1.43)
Gender
Female	135 (84.4)	105 (76.6)	142 (84.1)
Male	25 (15.6)	26 (24.4)	16 (15.9)
Race/ethnicity
White	121 (75.9)	113 (82.5)	125 (73.5)
Black	5 (3.1)	1 (0.7)	5 (2.9)
Asian or Pacific Islander	18 (11.1)	4 (2.9)	19 (11.2)
Hispanic or Latino	5 (3.1)	5 (3.6)	5 (2.9)
Other or prefer not to answer	11 (18.3)	8 (5.8)	8 (4.8)
Profession
Pre-medicine	9 (3.1)	0 (0)	6 (3.1)
Nursing	58 (3.1)	49 (35.8)	67 (34.4)
Occupational therapy	25 (14.7)	21 (15.3)	20 (10.3)
Physical therapy	43 (3.1)	47 (34.3)	38 (19.5)
Nutrition/dietetics	6 (3.1)	1 (0.7)	5 (2.6)
Medical laboratory sciences	3 (1.8)	0 (0)	0 (0)
Biology	1 (0.6)	0 (0)	0 (0)
Pre-physician assistant	7 (4.1)	0 (0)	0 (0)
Athletic training	2 (1.2)	10 (1.5)	0 (0)
Radiation therapy	2 (1.2)	2 (7.3)	0 (0)
Other	7 (3.1)	0 (0)	46 (23.6)
Undecided	2 (1.2)	1 (0.7)	0 (0)

Pilot Testing

Sample, pilot test 1

Undergraduate health professions students enrolled in an introduction to IPE course in fall 2015 completed the first draft of the SACS measure. Of the 160 students who completed the measure, the mean age was 18.8 (SD = 0.73), 84% were female, and 76% were White. Professions (student majors) represented in the sample were premedicine, nursing, occupational therapy, physical therapy, nutrition/dietetics, medical laboratory sciences, biology, prephysician assistant, athletic training, radiation therapy, other, and undecided.

Sample, pilot test 2

Undergraduate health professions students enrolled in an interprofessional health systems and health promotion course in spring 2015 completed the second pilot test for the SACS measure. Of the 131 students who completed the pilot 2 measure, the mean age was 19.3 (SD = 2.80), 77% were female, and 83% were White. The health professions (student majors) represented in the sample were radiation therapy, nursing, occupational therapy, physical therapy, athletic training, nutrition/dietetics, and undecided.

Sample, pilot test 3

Undergraduate health professions students enrolled in an introduction to IPE course in fall 2016 completed the Pilot 3 SACS measure. This third pilot was completed in two parts. First, at the beginning of the semester, the students completed the SACS measure for CFA purposes. Of the 181 students who completed the Pilot 3 measure, the mean age was 18.7 (SD = 1.43), 84% were female, and 87% were White. The health professions (student majors) represented in the Pilot Test 3 sample were premedicine, nursing, occupational therapy, physical therapy, nutrition/dietetics, and other. Second, midway through the semester, the students completed the convergent and discriminant validity measures, as these measures refer specifically to the respondents’ behavior within their current team. The time lapse was there to allow students to form their perceptions of their behaviors in their teams. The SACS measure is considered context-free, whereas the team measures are context dependent. Students’ responses from the Time 1 completion of the SACS measure to Time 2 completion of the additional measures were matched through a random identification code that was assigned to each individual student. Of the 182 students who completed the first survey, 140 students completed the second survey.

Statistical Analysis

Descriptive statistics were calculated for each sample and are reported in Table 2. EFA with Promax rotation using SPSS 23 was conducted for Pilot 1. Due to significant revision of the measure and addition of items, EFA was repeated for Pilot 2. CFA using the “lavaan” package in R was conducted for pilot 3 (Rosseel, 2012).

Results

Pilot Test 1

Item analyses

The main objective in this study was scale development, so the analyses focused on measuring collaboration as parsimoniously as possible while still retaining items that best represented the construct. As mentioned above, it was decided that Items 16–35 did not adequately cover the domain of collaboration skills, as perceptions of skills such as time management, motivation, and preparedness reflect the constructs of conscientiousness and work ethic (Barrick & Mount, 1991; Miller et al., 2002) rather than one’s ability to collaborate with other. Thus, only the items falling under the CSAT’s interpersonal scale were included. Upon inspection of the content of the items and discussion among the authors on item content and consistency with the domain of collaboration skills, it was decided to remove Items 5, 6, 8, and 12. For the majority of the remaining items, item–total correlations were above .40 or higher (Nunnally & Bernstein, 1994).

Dimensionality

An EFA using principal factors extraction with Promax rotation was performed on the remaining items to examine the factor structures of the initial SACS. The Promax rotation procedure was used, as theory indicates that the factors or dimensions are likely to be correlated. The EFA extracted three factors (all eigenvalues were above 1.00, ranging from 5.62 to 1.20), and the factors accounted for 69.79% of the total variance. Items related to information sharing load on Factors 1 and 2 includes items related to team impact and boosterism, and the third factor includes items related to team support. Item 1 had low cross-loadings on to Factors 1 and 2, which indicate a possible need for revision or removal of the item. Item 3 had a low cross-loading on factor 2. Items 7 and 10 had a low cross-loading on Factor 2. Finally, Item 11 had cross-loadings above .20 on Factors 1 and 3 with a low loading on Factor 2. These results can be found in the Supplementary materials.

Reliability and factor correlations

Each of the three SACS dimensions indicated acceptable internal consistency estimates. Specifically, the information sharing, boosterism, and team support dimensions had Cronbach’s α estimates of .74, .83, and .93, respectively. All items demonstrate adequate factor loadings with their respective factor. Finally, the three dimensions demonstrated significant correlations with one another. Specifically, the Team Support dimension was positively correlated with the boosterism and information sharing dimensions with r = .64 and r = .52, respectively. Additionally, the boosterism and information sharing dimensions were positively correlated with r = .48.

Pilot Test 2

Scale revision

Due to the results in the pilot study and the tenets of previous research on learning and collaboration, the authors decided that the items originally piloted may not adequately cover the domain of collaboration skills. As team learning is an essential component to interprofessional health-care teams (Thistlethwaite, 2012), it was decided that the collaboration skills scale should include items that reflect the extent to which individuals are able to learn from and with others. These efforts involved integrating three literatures that focus on the skills and behaviors, which individuals enact to challenge the status quo, seek feedback, obtain other view points, and engage in discussions for the sole purpose of improving the team through learning. As interprofessional teams engage in a large amount of information processing, it is necessary for them to engage in learning processes that allow for team members to share information, adapt, and improve in order to be effective (Edmondson, 1999). This research focuses on the team activities that characterize learning which include experimentation, reflection, voicing concerns, and seeking feedback (Edmondson, 1999). Three items were adapted from Edmondson’s (1999) team learning scale by changing the referent to reflect individual learning behaviors rather than perceptions of the activities that occur in one’s team. Additionally, for groups to effectively learn and process information, individuals need to be able to share their ideas, perspectives, and knowledge with other members of their team. An important part of this ability is engaging in Van Dyne and LePine’s (1998) voice behaviors, which involve the actions that are there to constructively challenge the team in order to improve (Van Dyne & LePine, 1998, p. 190). Thus, 2 items from Van Dyne and LePine’s voice behavior scale were implemented in the revisions of the SACS. Finally, organizational learning theory argues that learning encompasses the processes that enable teams to detect, correct, and learn from errors (Argyris, 1977). For teams to do this, team members must encourage and seek out dissenting views to promote learning. Three items were adapted and included in the SACS revision from Garvin, Edmondson, and Gino’s (2008) scholarship on the characteristics of organizational learning activities (Garvin, Edmondson, & Gino, 2008).

Item analyses

In this section of the pilot, we wanted to ensure that the newly revised scale covered the domain of collaboration as completely and as parsimoniously as possible. For the majority of the items, item–total correlations were above .40 or higher (Nunnally & Bernstein, 1994). Upon inspection of the content of the items and discussion among the authors on item content and consistency with theory on team collaboration and effectiveness, it was decided to remove Items 7, 10, 11, and 38. First, Items 7 and 9 had overlapping and possibly redundant information, which, if included, could result in the scale being too narrow and not covering enough of the construct domain (Kline, 1979). Additionally, zero-order correlations between the 2 items were high (r = .83, p < .001), which further supports the possibility of redundancy (Kline, 1979). Thus, Item 7 was removed as the authors argue that item is less representative of the collaboration concept than Item 9. Items 10 and 11 were removed due to the vagueness and ambiguity of the item content. It was determined that these items on team dynamics do not adequately represent the content domain of collaboration, which focuses specifically on the individual behaviors that contribute to coordinated action and effectiveness. Finally, Item 38 was eliminated due to its complexity and specificity.

Dimensionality

An EFA using principal factors extraction with Promax rotation was performed on the remaining items to examine the factor structures of the revised SACS. The Promax rotation procedure was used, as a priori theory indicates that the factors or dimensions are likely to be correlated. Upon inspection of the items, Item 41 loaded on the team support factor contrary to expectations and was further examined in the final phase of the scale validation. The EFA extracted three factors (all eigenvalues were above 1.00, ranging from 6.57 to 1.23), and the factors accounted for 63.84% of the total variance. Item 3 had a low cross-loading on to Factor 2. Items 13 and 41, which primary load on to Factor 2, had low cross-loadings on Factor 1. Item 13 had a low cross-loading on to Factor 3. Items 40, 42, and 43, which primarily load on to Factor 1, had low cross-loadings on Factor 2. Finally, Items 37 and 39 had cross-loadings on Factors 2 and 3 with high factor loadings on Factor 1, which indicates a possible need for revision or removal of the items. These results can be found in the Supplementary materials.

Reliability and factor correlations

Each of the three SACS dimensions indicated acceptable internal consistency estimates. Specifically, the information sharing, team support, and learning dimensions had Cronbach’s α estimates of .77, .84, and .87, respectively. All items demonstrate adequate standardized factor loadings with their respective factor. Finally, the three dimensions demonstrated significant correlations with one another. Specifically, the information sharing dimension was positively correlated with the team support and learning dimensions with r = .50 and r = .35, respectively. Additionally, the team support and learning dimensions were positively correlated with r = .58.

Pilot Test 3: Time 1

Fit statistics

CFAs were conducted to examine the hypothesis presented in the second pilot test that the SACS is multidimensional such that collaboration is comprised of three dimensions including team support, information sharing, and learning behaviors. To evaluate the fit, the χ² value for significance was examined. The criterion for determining acceptable model fit was if the p value was >.05 (e.g., a nonsignificant result; Barrett, 2007). Absolute and approximate fit indices were also used to evaluate model fit. In regard to the root mean square error of approximation (RMSEA), the cutoff value of .06 was utilized (Hu & Bentler, 1999) but interpreted .01 and .05 as excellent and good fit, respectively (MacCallum, Browne, & Sugawara, 1996). The standardized root mean square residual (SRMR) was determined as having good fit if the value is <.08 (Hu & Bentler, 1999). Finally, values .90 or higher were used to indicate acceptable model fit for data for the comparative fit index (CFI) and Tucker–Lewis index (TLI; Hatcher, 1994; Hu & Bentler, 1999; Medsker, Williams, & Holahan, 1994; Mulaik et al., 1989).

An initial CFA was conducted to examine and confirm the factor structure of the previously established 14-item SACS. Fit was less than acceptable, χ²(74, N = 156) = 168.40, p < .001; CFI = 0.90; TLI = 0.88; RMSEA = 0.09; SRMR = 0.06. Thus, through discussions among the authors, a series of decisions were made based on theory. First, Item 41 (“I value the opinions of others when working on a team project”) was removed, as this item reflects an attitude or value toward team learning rather than on the skills that comprise one’s ability to collaborate. Additionally, Item 43 (“I encourage others to share their opinion”) was removed, as the content of the item was deemed too vague. Item 2 (“I do not share resources with others easily”) was removed as the item does not reflect sharing information with others, which was the consideration of the dimension on which it loaded. Finally, Item 9 (“I publicly support the work of my fellow team members”) was removed, as this item may better represent the construct of team loyalty or boosterism (i.e., voluntary behaviors that promote the image of the team to external members; Moorman & Blakely, 1995) rather than representing the domain of collaborative skills.

The revised, three-factor proposed model that excluded Items 41, 43, 2, and 9 (Model 1) was tested for fit and compared to models specifying one- and two-factor models (Models 2 and 3, respectively). Results from CFAs using the “lavaan” package in R (Rosseel, 2012) are presented in Table 3. The results indicate that the three-factor solution was superior to the other two models such that the model met most of the standards of acceptable fit. The one-factor model and two-factor solution had less acceptable fit (see Supplementary materials).

Table 3.

Confirmatory Factor Analysis Results for the SACS.

Item	Information Sharing	Team Support	Learning
3. I share information with others easily	0.67
4. It is hard for me to share my ideas with others (RC)	1.09
13. I routinely listen to the opinions of my fellow team members		0.50
14. I regularly acknowledge the efforts of my team members		0.59
15. I consistently support the efforts of others		0.50
36. I frequently seek feedback from my team members about the quality of my work			0.72
37. I routinely go out and get all the information I can from my teammates			0.78
39. I encourage other team members to get involved in the decisions that affect the team			0.76
40. I voice my ideas about how the team could work better together			0.87
44. I consistently participate in team discussions with an open mind			0.71
42. I seek out different views than my own during team discussions			0.53

Note. χ²(55, N = 165) = 78.57, p < .01. CFI = 0.95; TLI = 0.93; RMSEA = 0.08; SRMR = 0.05. All factor loadings presented are standardized.

A χ² difference test of the one-factor and three-factor models demonstrated that the models were not redundant (χ² difference = 104.45, p < .001). As the three-factor model had better fit indices and was more consistent with theory, it was considered to best represent the dimensionality of the collaboration construct. The two-factor model was tested and compared to the three-factor model. The two-factor model combined the information sharing and learning dimensions to represent one dimension, as previous organizational learning research has suggested that learning involves information sharing processes (Garvin et al., 2008; Shaw & Perkins, 1992). Again, the χ² difference test (χ² difference = 33.77, p < .001) and the fit indices of the two-factor solution both indicate that three-factor model fits the data better than the two-factor model (see Supplementary materials).

Reliability and factor correlations

The majority of SACS dimensions indicated acceptable internal consistency estimates. Specifically, the information sharing, team support, and learning dimensions had Cronbach’s α estimates of .67, .84, and .86, respectively. All items demonstrate adequate standardized factor loadings with their respective factor (see Supplementary materials). Finally, the three dimensions demonstrated significant correlations with one another. Specifically, the information sharing dimension was positively correlated with the team support and learning dimensions with r = .39 and r = .45, respectively. Additionally, the team support and learning dimensions were positively correlated with r = .72.

Pilot Test 3: Time 2

Convergent and discriminant validity measures

Collaboration

Collaboration was measured and collected at Time 1 using the 11-item SACS whose factor structure was confirmed above. This scale demonstrated acceptable levels of internal reliability (α = .87).

Contextual performance

Contextual performance was measured with the instrument from Morgenson and colleagues (2005). This measure assesses the extent to which individuals report helping behavior, facilitation, dedication, and initiative in teams. As these interpersonal behaviors are expected to boost performance and team effectiveness (LePine, Piccolo, Jackson, Mathieu, & Saul, 2008), this measure was used to establish convergent validity of the SACS. This 9-item scale demonstrated acceptable levels of internal reliability (α = .93).

Counterproductive work behaviors (CWBs)

CWBs were measured using Spector, Bauer, and Fox (2010) 10-item short version of the Counterproductive Work Behavior checklist (CWB-C; Spector, Bauer, & Fox, 2010). Items reflect behaviors that are volitional and intended to harm the organization or the team as a whole (Spector & Fox, 2005), and as such this measure was used to establish discriminant validity of the SACS. This 9-item scale demonstrated acceptable levels of internal reliability (α = .93).

Convergent and discriminant validity results

Two main indicators of scale validity were examined. First, the SACS convergent validity, which assesses the extent to which a measure is related to a measure of similar content, was established by comparing it to a measure of extra-role helping and cooperative behavior in a team. Here, the Pearson product–moment correlation between the SACS and the Morgeson and colleagues’ (2005) measure of contextual performance (M = 4.38, SD = 0.45) was computed and examined. As expected, respondents’ self-reported collaboration using the SACS was significantly and positively related to their self-reported contextual performance as assessed using the contextual performance measure (r = .26, p < .01). Next, discriminant validity (i.e., the extent to which a measure of interest is unrelated to a measure of a different construct that is unrelated to construct of interest) was examined. In this study, the short-form CWB-C (M = 11.23, SD = 3.75) was used to establish discriminant validity, as there is no theoretical rationale to suggest these two constructs should be related. This was supported, as there was no relationship between a person’s CWBs and their self-reported collaboration skills (r = .01, ns).

Discussion

This study describes the development process and preliminary validation of the SACS instrument. CFA was consistent with a three-dimensional scale of collaboration: information sharing, learning, and team support. The present study documents high internal consistency within each scale dimension and establishes preliminary construct validity of the measure. Prior measures of collaboration in health or interprofessional settings focus on specific clinical settings and are not useful for nonclinical contexts (Baggs, 1994; Curran et al., 2011).

A desired outcome of IPE initiatives is the preparation of “collaborative practice-ready” practitioners as they enter the workforce upon graduation (World Health Organization, 2010, p. 7). Research suggests that early integration of IPE into health professions education is critical to successful development of future IPCP skills (Thistlethwaite, 2012). However, students need to be provided with the opportunity to practice these skills in intentionally authentic learning experiences. The SACS provides a tool for faculty and researchers interested in understanding the effectiveness of this pedagogy by measuring students’ self-assessed collaboration in both clinical and nonclinical contexts. This will allow for refinement of the educational process, as students move through their health professional programs. Furthermore, the SACS will be critical in evaluating the effectiveness of IPE efforts at the practitioner level in continuing professional education. As system-wide changes are developed to address IPCP across multiple sectors, the need for an appropriate validated measurement tool will become even more important.

Several important limitations must be acknowledged. First, this study provides a preliminary validation of the construct validity of the instrument. Further study is necessary to establish responsiveness across time and its performance in other settings. Efforts to establish the predictive validity of the measure are also needed. Research could examine the extent to which individual scores on this measure predict scores on student performance in IPE courses in which collaboration is a foundational skill in such coursework. Second, although the instrument was piloted in three distinct samples, all samples were part of a convenience sample, undergraduate student population limiting the generalizability. While the measurement of clinical skills and behaviors is inappropriate for the level of learners in this study, collaboration skills can be taught and measured independent of clinical context, and, thus, a valid measure of collaboration skills in this population of learners is necessary. Given the explicit focus on collaboration and skills building for both collaboration and teamwork in interprofessional curriculum, there is a need for a measure that is validated in the undergraduate population. It is possible that the tool may respond differently in older or more developmentally mature samples, and future work is necessary to establish the validity of the measure in other populations. Additionally, the sample sizes, by measurement development standards, are small (Costello & Osbourne, 2005), and future research should be completed with larger samples to replicate and further validate the factor structure identified in the current measure.

Although limitations exist, there are several strengths to this study. The factor structure identified in the EFA (Pilot 2) was confirmed in an independent sample. Additionally, none of the standardized factor loadings in the SACS fall below 0.50. The SACS measure also helps to provide a comprehensive measure of collaboration that assesses not only a student’s ability to contribute to and support fellow team members’ performance but also examines his or her ability to engage in productive conversations and contribute to the learning of the team. Previous measures have often examined these dimensions separately (Edmondson, 1999; Garvin et al., 2008; Ofstedal & Dahlberg, 2009; Van Dyne & LePine, 1998). Thus, the SACS integrates these skills as being representative of the larger collaboration construct.

The SACS is to our knowledge the first measure of collaborations skills for use in both clinical and nonclinical health-care settings. It marks a first step toward measurement of behavioral outcomes of IPCP called for by leaders in the field. Important next steps in the validation of the SACS would be to cross-validate the measure in a different, nonacademic setting or in a setting in which students have more developed knowledge and expertise in interpersonal collaboration (i.e., at the postbaccalaureate and preprofessional level). Finally, research on the SACS would benefit from establishing the predictive validity of the measure by examining its ability to predict performance in an interprofessional team setting.

Supplemental Material

Supplemental_Table - Student Perceptions of Collaboration Skills in an Interprofessional Context: Development and Initial Validation of the Self-Assessed Collaboration Skills Instrument

Supplemental_Table for Student Perceptions of Collaboration Skills in an Interprofessional Context: Development and Initial Validation of the Self-Assessed Collaboration Skills Instrument by Leslie Hinyard, Eileen Toomey, Kathrin Eliot and Anthony Breitbach in Evaluation & the Health Professions

Footnotes

Appendix A

Collaboration self-assessment tool.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Leslie Hinyard

Anthony Breitbach

Supplemental Material

Supplemental material for this article is available online.

References

Aram

J. D.

Morgan

C. P.

Esbeck

E. S.

(1971). Relation of collaborative interpersonal relationships to individual satisfaction and organizational performance. Administrative Science Quarterly, 16, 289–297.

Argyris

(1977). Double loop learning organizations. Harvard Business Review, 55, 115–125.

Baggs

J. G.

(1994). Development of an instrument to measure collaboration and satisfaction about care decisions. Journal of Advanced Nursing, 20, 176–182. doi:10.1046/j.1365-2648.1994.20010176.x

Barrett

(2007). Structural equation modelling: Adjusting model fit. Personality and Individual Differences, 42, 815–824.

Barrick

M. R.

Mount

M. K.

(1991). The big five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1–26.

Bronstein

L. R.

(2002). Index of interdisciplinary collaboration. Social Work Research, 26, 113–123.

Crocker

Algina

(1986). Introduction to classical and modern test theory. Orlando, FL: Holt, Rinehart, and Winston.

Curran

Hollett

Casimiro

L. M.

Mccarthy

Banfield

Hall

… Wagner

. (2011). Development and validation of the Interprofessional Collaborator Assessment Rubric (ICAR). Journal of Interprofessional Care, 25, 339–344.

Costello

A. B.

Osborne

J. W.

(2005). Best practices in exloratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, & Evaluation, 10(7), Available online: http://pareonline.net/getvn.asp?v=10&n=7

10.

Dillenbourg

Baker

Blaye

O’Malley

(1996). The evolution of research on collaborative learning. In Spada

Reiman

(Eds.), Learning in humans and machines: Towards an interdisciplinary learning science (pp. 189–211). Oxford, England: Elsevier.

11.

Drach-Zahavy

Somech

(2001). Understanding team innovation: The role of team processes and structures. Group Dynamics: Theory, Research, and Practice, 5, 111–123.

12.

Edmondson

(1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350–383.

13.

Ferris

G. R.

Treadway

D. C.

Kolodinsky

R. W.

Hochwarter

W. A.

Kacmar

C. J.

Douglas

Frink

D. D.

(2005). Development and validation of the political skill inventory. Journal of Management, 31, 126–152.

14.

Garvin

D. A.

Edmondson

A. C.

Gino

(2008). Is yours a learning organization? Harvard Business Review, 86, 109–116.

15.

Hatcher

(1994). A step-by-step approach to using SAS system for factor analysis and structural equation modeling. Cary, NC: SAS Institute Inc.

16.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.

17.

King

Shaw

Orchard

C. A.

Miller

(2010). The Interprofessional Socialization and Valuing Scale: A tool for evaluating the shift toward collaborative care approaches in health care settings. Work, 35, 77–85.

18.

Kline

(1979). Psychometrics and psychology. London, England: Academic Press.

19.

LePine

J. A.

Piccolo

R. F.

Jackson

C. L.

Mathieu

J. E.

Saul

J. R.

(2008). A meta-analysis of teamwork processes: Tests of a multidimensional model and relationships with team effectiveness criteria. Personnel Psychology, 61, 273–307.

20.

MacCallum

R. C.

Browne

M. W.

Sugawara

H. M.

(1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149.

21.

Marks

M. A.

Mathieu

J. E.

Zaccaro

S. J.

(2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26, 356–376.

22.

McFadyen

A. K.

Webster

Strachan

Figgins

Brown

Mckechnie

(2005). The Readiness for Interprofessional Learning Scale: A possible more stable sub-scale model for the original version of RIPLS. Journal of Interprofessional Care, 19, 595–603.

23.

Medsker

G. J.

Williams

L. J.

Holahan

P. J.

(1994). A review of current practices for evaluating causal models in organizational behavior and human resources management research. Journal of Management, 20, 439–464.

24.

Miller

M. J.

Woehr

D. J.

Hudspeth

(2002). The meaning and measurement of work ethic: Construction and initial validation of a multidimensional inventory. Journal of Vocational Behavior, 60, 451–489.

25.

Moorman

R. H.

Blakely

G. L.

(1995). Individualism-collectivism as an individual difference predictor of organizational citizenship behavior. Journal of Organizational Behavior, 16, 127–142.

26.

Morgeson

F. P.

Reider

M. H.

Campion

M. A.

(2005). Selecting individuals in team settings: The importance of social skills, personality characteristics, and teamwork knowledge. Personnel Psychology, 58, 583–611.

27.

Mulaik

S. A.

James

L. R.

Valnalstine

Bennett

Lind

Stilwell

C. D.

(1989). Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin, 105, 248–292.

28.

Nunnally

J. C.

Bernstein

I. H.

(1994). The assessment of reliability. Psychometric Theory, 3, 248–292.

29.

Ofstedal

K. K.

Dahlberg

(2009). Collaboration in student teaching: Introducing the collaboration self-assessment tool. Journal of Early Childhood Teacher Education, 30, 37–48. doi:10.1080/10901020802668043

30.

Orchard

C. A.

King

G. A.

Khalili

Bezzina

M. B.

(2012). Assessment of Interprofessional Team Collaboration Scale (AITCS): Development and testing of the instrument. Journal of Continuing Education in the Health Professions, 32, 58–67.

31.

Parsell

Bligh

(1999). The develoopment of a questionaire to assess the readiness of health care students for interprofessional learning (RIPLS). Medical Education, 33, 95–100.

32.

Reeves

Lewin

Espin

Zwarenstein

(2011). Interprofessional teamwork for health and social care (Vol. 8). Oxford, England: John Wiley.

33.

Riggio

R. E.

Riggio

H. R.

(2001). Self-report measurement of interpersonal sensitivity. In Hall

J. A.

Bernieri

F. J.

(Eds.), The LEA series in personality and clinical psychology. Interpersonal sensitivity: Theory and measurement (pp. 127–142). Mahwah, NJ: Lawrence Erlbaum.

34.

Rosseel

. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36.

35.

Salas

Sims

D. E.

Burke

C. S.

(2005). Is there a “Big Five” in teamwork? Small Group Research, 36, 555–599.

36.

Shaw

R. B.

Perkins

D. N. T.

(1992). Teaching organizations to learn: The power of productive failures. In Nadler

Gerstein

Shaw

(Eds). Organization Architecture (pp. 175–192). San Francisco: Jossey-Bass.

37.

Spector

P. E.

Bauer

J. A.

Fox

(2010). Measurement artifacts in the assessment of counterproductive work behavior and organizational citizenship behavior: Do we know what we think we know? Journal of Applied Psychology, 95, 781–790.

38.

Spector

P. E.

Fox

(2005). The stressor-emotion model of counterproductive work behavior. In Fox

Spector

P. E.

(Eds.), Counterproductive work behavior: Investigations of actors and targets (pp. 151–174). Washington, DC: American Pyschological Association.

39.

Tasa

Sears

G. J.

Schat

A. C.

(2011). Personality and teamwork behavior in context: The cross-level moderating role of collective efficacy. Journal of Organizational Behavior, 32, 65–85.

40.

Thistlethwaite

(2012). Interprofessional education: A review of context, learning, and the research agenda. Medical Education, 46, 58–70.

41.

Thistlethwaite

Dallest

(2014). Interprofessional teamwork: Still haven’t decided what we are education for. Medical Education, 48, 556–558.

42.

Thistlethwaite

Forman

Matthew

L. R.

Rogers

G. D.

Steketee

Yassine

(2014). Competencies and frameworks in interprofessional education: A comparative analysis. Academic Medicine, 89, 869–875.

43.

Van Dyne

LePine

J. A.

(1998). Helping and voice extra-role behaviors: Evidence of construct and predictive validity. Academy of Management Journal, 41, 108–119.

44.

West

M. A.

Wallace

(1991). Innovation in health care teams. European Journal of Social Psychology, 21, 303–315.

45.

World Health Organization. (2010). Framework for action on interprofessional education and collaborative practice. Geneva, Switzerland: Author.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB