Factorial Invariance of an Integrated Measure of Classroom Sense of Community in Face-to-Face and Online Courses

Abstract

The aim of the present study was to develop a more integrated measure of classroom sense of community (SOC) while testing factorial invariance of the measurement structure across face-to-face and online courses. We incorporated two existing SOC measures to capture both context-specific and context-general characteristics of SOC and developed an integrated scale of classroom SOC with a four-factor structure. Tests of measurement invariance indicated that the new measure of classroom SOC has equivalent patterns of factor loadings, strength of factor loadings, and intercepts across the two course delivery modes, thus making this a viable measure to examine potential differences across these contexts. Latent mean analysis indicated that students in an online course perceived a lower level of SOC.

Keywords

classroom sense of community factorial invariance undergraduate students online learning

Institutions of higher education have recently given more attention to building students’ sense of community (SOC) due to a growing concern about student retention (Summers & Svinicki, 2007; Tovar & Simon, 2010). Considerable research has supported that perception of classroom SOC has significant implications for students’ emotional, social, and cognitive development (O’Connor, 2013; Summers, Beretvas, Svinicki, & Gorin, 2005; Top, 2012). Students experiencing a strong SOC in college tend to exhibit better adjustment and persistence in school (Mannarini & Fedi, 2009; Top, 2012). Despite the educational and psychological benefits of SOC, there are challenges associated with measuring SOC in the post-secondary classroom (McMillan, 1996, 2011).

Many researchers have agreed that the concept of SOC is complex and multi-dimensional (Abdelkader & Bouslama, 2014; Mannarini & Fedi, 2009; Nowell & Boyd, 2010; Warner, Kerwin, & Walker, 2013); however, there is a lack of consensus regarding the conceptual dimensions of the construct. Conceptual divergence has resulted in the development of multiple measures of SOC with different theoretical foundations, which, in turn, generated confusion and problems in comparisons between previous findings and has limited theory building and testing (Nowell & Boyd, 2010, 2011).

Another measurement issue for SOC is related to the rapid growth of online courses because SOC can be experienced in an online learning environment as well as in a traditional face-to-face classroom. Online classrooms differ from traditional face-to-face classrooms in structure and content delivery; though social dynamics are thought to be essential for increasing SOC (Barnard-Brak & Shiu, 2010; Rovai, 2002; Rovai, Wighting, & Lucking, 2004) in both online and face-to-face classrooms. Without examining the measurement invariance of SOC across course delivery methods (i.e., face-to-face vs. online courses), group comparisons remain problematic. Consequently, investigating measurement invariance, or the extent to which factor structures are equivalent across groups, is recommended in all educational and psychological testing (Borsboom, 2006; Tovar & Simon, 2010) as a prerequisite to making cross-group comparisons. No prior research has attempted to examine whether conceptual dimensions of classroom SOC may differ based on course delivery methods. To ensure that we can utilize the same conceptual framework to understand student experience of SOC in different learning environments, it is important to investigate this question empirically. Therefore, the primary purposes of this study were to develop a comprehensive measure of SOC within the context of classroom learning environment by integrating existing theoretical models of SOC and to determine if the measure is invariant across course delivery formats.

A SOC has generally been described as feelings of relatedness, belonging, or connection as a result of common experiences within a group (McMillan, 1996, 2011). Conceptual components of SOC should reflect the contextual nature (e.g., workplace or academic setting) of the specific community being measured (Admiraal & Lockhorst, 2012; Barnard-Brak & Shiu, 2010; Hill, 1996). Earlier research on SOC primarily focused on location-based community (e.g., neighborhood), but subsequent research extended the focus to a variety of settings including school and classroom settings. For instance, Rovai (2002) conceptualized SOC based on two components (i.e., connectedness and learning) in classroom and school contexts and developed the Classroom and Community Scale. The Classroom and School Community Inventory, later developed by Rovai et al. (2004), includes social and learning components as the two primary aspects of classroom learning environment. In addition to the context-specific aspects of SOC, Glynn (1981) suggested that there are some characteristics of SOC that are commonly observed across contexts. Several researchers argued that shared goals and purposes are essential to the development of SOC. Although the importance of shared goals is well addressed in SOC in non-educational settings, available instruments developed for educational settings have failed to consider shared goals as a dimension of SOC. Thus, the development of a more comprehensive, integrated measure of classroom SOC is warranted.

In an effort to incorporate both context-specific and context-general aspects of SOC, we reviewed existing measures. The Perceived Sense of Community (PSC) Scale, devised by Chertok (1990) for non-educational settings (i.e., residential program for recovering alcoholics), consists of three factors: (a) mission (the perception that group members share a common purpose), (b) reciprocal responsibility (the perception that members are mutually responsible to each other), and (c) disharmony (lack of emotional connections among group members). Although the PSC measures shared goals and emotional connections among group members, its use in higher education is limited because it fails to capture the classroom-specific dynamics of SOC.

The Post-Secondary Classroom Community Scale (PSCCS), developed by Bush, Svinicki, Achacoso, and Kim (2004), measures both learning and connection components essential to SOC within higher education but still does not address the domain-general dimension of shared goals and responsibility. Apparent deficits in the content validity of existing SOC measures indicate a need for an integrated measure of SOC that incorporates the domain-general characteristics of SOC (i.e., shared goals and responsibilities) and is sensitive to classroom dynamics specific to the higher education environment.

The goals of the present study provide a more integrated approach to the assessment of SOC, examine measurement invariance across face-to-face and online courses, and compare latent mean differences in SOC across course delivery methods.

Method

Participants

Participants included 819 undergraduate students enrolled in face-to-face (n = 575) and online courses (n = 244) within the College of Education at a large Midwestern state university. Females were disproportionately represented within the entire sample, which is typical of education major courses. Approximately 64% of the face-to-face students and 63% of the online students indicated that they were female. Across both subsamples, nearly 80% indicated that they were Caucasian. About 38% of face-to-face students self-identified as seniors, 23% juniors, 19% sophomores, and 19% indicated that they were freshmen. In the online sample, about 40% identified as seniors, 19% indicated that they were juniors, 15% sophomores, and a little more than 7% indicated that they were freshmen.

Measures

We collected a range of demographic information about class-size, expected grade point average (GPA), cumulative GPA, employment status (1 = employed, 2 = not employed), marital status (1 = married, 2 = not married), and family status (1 = have children, 2 = no children) to test potential differences between the face-to-face and online groups.

Based on an extensive review of the literature, we used two existing measures of SOC with substantially different conceptualizations of SOC to construct and validate an integrated measure of classroom SOC. The first measure is the PSC developed by Chertok (1990) and updated by Bishop, Chertok, and Jason (1997). The PSC includes three conceptual dimensions: Mission, Reciprocal Responsibility, and Disharmony. Sixteen items relevant and applicable to classroom learning environment were adopted from the original scale. The wordings of these items were modified to be appropriate for the classroom setting (see the appendix for sample items). The second measure, the PSCCS, focuses on interactions within the classroom setting (e.g., interaction with the instructor, peers, and content). The PSCCS (Bush et al., 2004) includes 14 items with three conceptual dimensions: Engagement With Content, Student–Instructor Interaction, and Peer Respect/Connectedness (see the appendix for sample items). The items were measured on a 7-point Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree).

Procedures

We received permission from instructors who teach a face-to-face course or an online course to administer surveys to their classes. Students were administered a survey, which included an information sheet, questionnaires, and demographic questions. For students in face-to-face courses, surveys were administered during class time by the research team. Online students were forwarded an online version of the questionnaire by their instructors. All student participation in the study was voluntary.

Analytic Procedure

Students enrolled in face-to-face courses (n = 575) were randomly assigned to a validation (n = 259) and a replication sample (n = 316). An exploratory factor analysis (principal-axis factor analysis using direct oblimin rotation) was conducted on the validation sample using items from the PSC and the PSCCS. Findings from the validation sample were then examined via confirmatory factor analysis (CFA) on the replication sample. Measurement invariance of the final CFA model was then examined across students enrolled in face-to-face courses (n = 316) or online courses (n = 244) using the general procedures outlined by Byrne (2010). First, the CFA models for face-to-face and online students were examined separately. A common measurement model was then tested for invariance by investigating both groups simultaneously. Invariance procedures, as used within this study, examined three nested models. Model 1 is the configural model, which is an initial examination of both the number and pattern of loadings across both groups. Model 2 examined metric invariance, which consists of constraining the factor loadings to be equivalent across both groups. Model 3 investigated scalar invariance, which constrains item intercepts to be equivalent across both groups.

To determine whether imposing equality constraints across models resulted in substantive decrements in model fit, differences in chi-square (∆χ²) and the comparative fit index (∆CFI) were examined. However, prior to comparing nested models using a chi-square difference test, the Satorra–Bentler correction to the chi-square statistic (Satorra & Bentler, 2010) using Mplus (Muthén & Muthén, 2007) was used due to violations of multivariate normality. Chi-square difference values at p < .05 and ∆CFI > .01 indicates a substantial decrease in model fit in comparison with the less constrained models. Overall fit of CFA models was judged based on chi-square goodness-of-fit statistic (χ² / df < 3), Root Mean Square Error of Approximation (RMSEA< .08), Standardized Root Mean Square Residual (SRMR< .06), Tucker–Lewis Index (TLI > .90), and the CFI (excellent at CFI > .95; acceptable at CFI > .90) (Hu & Bentler, 1999; Kline, 2005).

Results

Preliminary Analyses

To accomplish the purposes of this study, the sample was first split into those students enrolled in face-to-face courses (n = 575) and those enrolled in online courses (n = 244). Preliminary analyses compared both subsamples across a range of demographic, academic, and familial variables. There were statistically significant differences in age across online (M = 24.14, SD = 6.75) and face-to-face students (M = 22.03, SD = 4.97), t(790) = 4.55, p < .001, d = .32. Approximately 20% of online students indicated that they were married, compared with only 5% of face-to-face students $χ^{2} (1) = 43.51, p < . 001, ϕ = . 21, p < . 001$ . About 14% of online students reported that they had children, whereas a little more than 5% of the face-to-face students indicated that they had children $χ^{2} (1) = 14.83, p < . 001, ϕ = . 14, p < . 001$ . Approximately 62% of online students and 48% of face-to-face students indicated that they were employed $χ^{2} (1) = 10.63, p < . 001, ϕ = . 12, p = . 001$ . Finally, there were no statistically significant differences in class-size, M_online = 46.20, SD = 44.83, M_F2F = 51.05, SD = 44.16, t(766) = −1.24, p = .519, d = .08; expected GPA, M_online = 3.51, SD = .66, M_F2F = 3.53, SD = .641, t(589) = −.37, p = .877, d = .03; and self-reported cumulative GPA, M_online = 3.19, SD = .56, M_F2F = 3.34, SD = 1.19, t(770) = −1.67, p = .948, d = .09, across the subsamples. In summary, little differences seemed to exist across these subsamples in terms of academic variables, but online students were on average older and more likely to report familial and work obligations than face-to-face students.

Exploratory Factor Analysis

Thirty-one items from the PSC and the PSCCS were submitted to principal-axis factor analysis using direct oblimin rotation on the face-to-face validation sample (n = 259). We considered three extraction criteria including Horn’s (1965) parallel analysis, eigenvalue-greater-than-1-rule, and Velicer’s (1976) minimum average partial (MAP) correlation. Factor solutions were also evaluated for their theoretical meaningfulness and approximation of simple structure. An examination of these criteria, as well as their proposed solutions, eventually led to a four-factor solution.

The first seven eigenvalues before rotation were 13.11, 2.61, 1.77, 0.82, 0.73, 0.64, and 0.38. When conducting Horn’s parallel analysis, 1,000 matrices consisting of the same number of observations (n) and variables (v) as within the present study were randomly generated. We estimated eigenvalues from each matrix to compare the eigenvalues estimated from the sample of face-to-face students with the distribution of each eigenvalue estimated from random data. The first seven eigenvalues located at the 95th percentile from the random data were 1.04, 0.89, 0.79, 0.71, 0.64, 0.58, and 0.38. This analysis indicated that six factors should be extracted because the first six eigenvalues from the sample data were greater than the eigenvalues at the 95th percentile from the randomly generated data.

A six-factor solution was examined and appeared to lead to the extraction of too many factors. We, therefore, used an eigenvalue-greater-than-1-rule as a minimal standard to decide how many factors to extract which indicated that three factors should be extracted. The average squared partial correlation after successively removing the first five components is as follows: 1 = .036, 2 = .027, 3 = .018, 4 = .017, and 5 = .017. These values imply that the MAP correlation is achieved by extracting four factors. Both the three- and four-factor solution were further examined for theoretical meaningfulness, as well as an approximation of simple structure.

The four-factor solution accounts for approximately 58% of the common variance and includes (a) shared goals and responsibility, (b) relationship with the instructor, (c) student engagement, and (d) peer valuation. The primary difference between the four-factor solution and the three-factor solution is that items referring to student engagement were separated from the shared goals and responsibility factor in the four-factor solution, while they were combined with the shared goals and responsibility factor in the three-factor solution. Although student engagement factor and shared goals and responsibility factor were correlated (i.e., r = .50, p < .001), from a theoretical point of view, we judge these distinctions as important to the SOC. Therefore, the four-factor solution was retained as the final model. However, in the rotated four-factor solution, there were nine items with low communality estimates (i.e., h² < .40) or pattern coefficients $< | . 40 |$ .

We conducted a final exploratory factor analysis with these items removed from the analysis. The final four-factor solution accounted for approximately 63% of the common variance. Table 1 presents factor loadings as well as skewness and kurtosis for retained items. Interpretation of the pattern matrix indicated that the first factor is principally defined by seven items referring to Shared Goals and Responsibility (i.e., striving for common purposes). The second factor was defined by five items referring to the Relationship With the Instructor (i.e., support from the instructor), the third factor was defined by eight items referring to Student Engagement (i.e., interest, value of learning), and the fourth factor was defined by two items referring to Peer Valuation (i.e., support and respect from peers).

Table 1.

Exploratory Factor Analysis With Validation Sample (n = 259).

Item	Skewness	Kurtosis	Factor 1	Factor 2	Factor 3	Factor 4
1	−0.56	−0.31	.89 (.87)	−.07 (−.38)	−.11 (.35)	.02 (.04)
2	−0.41	−0.35	.84 (.81)	.04 (−.29)	−.01 (.36)	.03 (.04)
5	−0.58	0.13	.83 (.85)	.05 (.35)	.08 (.25)	.19 (.20)
3	−0.34	−0.45	.70 (.69)	−.11 (−.32)	−.11 (.28)	−.13 (−.11)
4	−0.29	−0.16	.66 (.67)	−.01 (−.29)	.02 (.34)	.12 (.13)
7	−0.44	0.06	.59 (.65)	.01 (−.29)	.14 (.31)	−.05 (−.04)
8	−0.21	−0.28	.57 (.67)	.07 (.27)	.17 (.31)	−.18 (.17)
13	−1.42	1.55	.03 (.35)	.96 (.94)	.02 (.24)	−.01 (.14)
15	−1.37	1.22	.02 (.31)	.89 (.85)	.07 (.35)	−.03 (.15)
12	−1.25	1.20	.01 (.35)	.84 (.86)	−.03 (.24)	−.06 (.18)
11	−1.02	0.30	.03 (.37)	.80 (.86)	−.17 (.35)	.16 (−.04)
14	−0.99	0.83	−.01 (.33)	.80 (.81)	.00 (.29)	−.03 (.14)
16	−0.54	−0.22	.10 (.37)	−.09 (−.36)	.72 (.81)	−.19 (−.15)
20	−0.40	−0.77	−.03 (.31)	−.09 (−.31)	.70 (.68)	−.15 (.09)
17	−0.60	−0.93	−.13 (.21)	−.03 (−.32)	.70 (.65)	.06 (.08)
19	−1.10	0.58	−.06 (.31)	−.04 (−.29)	.73 (.72)	.18 (.20)
18	−0.34	−0.71	.11 (.36)	−.09 (−.34)	.69 (.78)	−.15 (−.12)
10	−0.57	−0.13	.14 (.34)	−.09 (−.38)	.56 (.75)	−.13 (−.10)
6	−0.60	−0.34	.16 (.37)	−.11 (−.36)	.56 (.73)	−.18 (−.15)
9	−0.50	0.00	.14 (.31)	−.12 (−.34)	.47 (.65)	−.03 (.00)
22	−1.23	1.46	.28 (.39)	−.17 (−.36)	.13 (.37)	.43 (.48)
21	−0.88	0.60	.26 (.37)	−.15 (−.36)	.12 (.32)	.40 (.45)
Eigenvalues			9.38	2.31	1.58	0.57
% of variance			42.62	10.52	7.19	2.60

Note. Item content is displayed in the appendix. Skewness standard error = .15; Kurtosis standard error = .30. Factor 1 = Shared goals and responsibility; Factor 2 = Relationship With the Instructor; Factor 3 = Student Engagement; Factor 4 = Peer Valuation; Structure coefficients are in parentheses. Bold values indicate pattern coefficients > .40.

The shared goals and responsibility factor was moderately correlated with the relationship with the instructor factor (r = .40, p < .001) and student engagement factor (r = .53, p < .001), although the correlation with peer valuation factor was not significant (r = .06, p = .31). Relationship with the instructor factor is related to student engagement factor (r = .53, p > .05) and peer valuation factor (r = .18, p < .01). The relationship between student engagement factor and peer valuation factor was not significant (r = .01, p > .05). Cronbach’s alpha reliability coefficients of all factors were favorable: .91 for shared goals and responsibility, .92 for relationship with the instructor, .90 for student engagement, and .84 for peer valuation.

Initial Confirmatory Factor

The factor structure established in the validation sample was then tested via a CFA with the face-to-face replication sample (n = 316) and online samples separately (n = 244). Model fit across both the face-to-face sample, $χ^{2} (203) = 466.44$ , p < .001, $χ^{2} / d f$ = 2.29, RMSEA = .06, SRMR = .06, TLI = .92, CFI = .93, and online sample, $χ^{2} (203) = 459.84$ , p < .001, $χ^{2} / d f$ = 2.91, RMSEA = .07, SRMR = .06, TLI = .89, CFI = .90, was generally adequate, though model fit statistics for the online sample was slightly below acceptable limits. A review of the modification indices across both the face-to-face and online models suggested that the correlation of an error term for two items (i.e., “The course content is not very interesting to me personally” and “In this class everyone seems to be interested in the discussion topics”) within student engagement factor would improve overall model fit. An examination of the content of these items indicated that both referred to student interest, consequently we decided to allow these error terms to be correlated. With this change, the final model fit for the face-to-face, $χ^{2} (202) = 432.71$ , p < .001, $χ^{2} / d f$ = 2.14, RMSEA = .06, SRMR = .05, TLI = .93, CFI = .94, and online samples, $χ^{2} (202) = 418.12$ , p < .001, $χ^{2} / d f$ = 2.06, RMSEA = .06, SRMR = .06, TLI = .90, CFI = .92, were adequate.

For the face-to-face sample, internal consistency reliability estimates were generally high across each factor (i.e., α = .91 for share goals and responsibility, α = .91 for relationship with the instructor, α = .90 for student engagement, α = .84 for peer valuation). Similarly, internal consistency reliability estimates across each factor for the online sample were also favorable (i.e., α = .88 for share goals and responsibility, α = .91 for relationship with the instructor, α = .87 for student engagement, α = .83 for peer valuation).

Measurement Invariance Tests

The results of the measurement invariance tests are presented in Table 2. The configural model, wherein both groups were tested simultaneously on the CFA model discussed above, had acceptable model fit, $χ^{2} (404) = 856.39$ , p < .001, $χ^{2} / d f$ = 2.12, RMSEA = .06, SRMR = .05, TLI = .92, CFI = .93. This implies that both the number and pattern of factors were equivalent across groups. Investigation of metric invariance was conducted by constraining the factor loadings to be equivalent across groups, $χ^{2} (422) = 875.47$ , p < .001, $χ^{2} / d f$ = 2.07, RMSEA = .06, SRMR = .06, TLI = .92, CFI = .93. When applying the scaling factor to observed chi-square values, both the chi-square difference test and changes in CFI values supported metric invariance, $Δ χ^{2} (18) = 15.78, p = . 61; Δ CFI = . 00$ . Next, scalar invariance was examined by constraining intercepts to be equivalent across groups, $χ^{2} (440) = 914.09$ , p < .001, $χ^{2} / d f$ = 2.07, RMSEA = .06, SRMR = .06, TLI = .92, CFI = .93. When applying the chi-square difference test, this model led to decrements in model fit when compared with the model with equivalent factor loadings, $Δ χ^{2} (18) = 38.71, p < . 05$ . Examination of modification indices implied that three items may have different intercepts across groups (i.e., “Students in this class know that they can depend on each other should they need help,” “Students in this class share common values,” and “Students of this class have a clear understanding of mission and purpose”). These intercepts were sequentially freed so that they vary across groups, which resulted in technically statistically significant changes in chi-square, that is, $Δ χ^{2} (1) \approx$ 4.0, p < .05, but failed to substantively alter the chi-square difference test. Given that the scalar invariance model had overall acceptable fit, coupled with evidence from differences across the CFI supporting scalar invariance $Δ CFI = . 002$ , we decided to retain the model with equivalent intercepts across each group. A final CFA combining both groups is presented in Figure 1.

Table 2.

Measurement Invariance Tests for Classroom Sense of Community Scale Across Face-to-Face and Online Courses.

Model	χ²	df	RMSEA	CFI	Model comparison	∆χ²	∆df	p	(∆CFI)
1. Configural	856.39	404	.06	.93	1 and 2	15.78	18	.608	.000
2. Metric	857.47	422	.06	.93	2 and 3	39.95	18	.003	.002
3. Scalar	914.09	440	.06	.93

Note. χ² = Satorra–Bentler adjusted chi-square; df = degrees of freedom, RMSEA = Root Mean Square Error of Approximation; CFI = Comparative Fit Index.

Figure 1.

Final confirmatory factor analysis model.

After investigating metric invariance across face-to-face and online courses, we then examined latent mean differences among the face-to-face and online students. Due to model identification purposes, the latent factor means for the face-to-face group were freely estimated while constraining the latent factor means of the online group to zero (Byrne, 2010). Positive latent means for the face-to-face group would, therefore, indicate that these means were higher than the online group. The latent factor means indicated that the face-to-face students perceived higher levels of Shared Goals and Responsibility, Instructor Relationship, Student Engagement, and Peer Valuation than their online counterparts (see Table 3).

Table 3.

Face-to-Face Latent Mean Estimates for the Four Factors of Classroom Sense of Community Scale.

Factor	Latent mean	SE	CR	p
Shared goals and responsibility	.77	.09	8.08	<.001
Student engagement	.50	.11	4.54	<.001
Instructor relationship	.31	.08	3.58	<.001
Peer valuation	.31	.09	3.36	<.05

Note. Latent means for face-to-face sample are estimated while constraining these values to zero for the online sample. CR = Critical ratio.

Second-Order Factor Analysis

The first-order CFA indicates that substantial correlations exist between each factor (see Figure 1). Given the magnitude of these correlations, Gorsuch (1983) recommended examining a second-order factor structure (Gorsuch, 1983). We therefore conducted a second-order CFA wherein each of the four factors was hypothesized to load onto a single second-order SOC factor. This test was conducted when combining both the face-to-face and online groups. Overall model fit was reasonable, $χ^{2} (204) = 649.68$ , p < .001, $χ^{2} / d f$ = 3.18, RMSEA = .06, SRMR = .05, TLI = .93, CFI = .93, which implies that the covariation between shared goals and responsibility, relationship with the instructor, student engagement, and peer valuation may be accounted for by an underlying SOC factor. Second-order factor loadings ranged from .72 to .83 and were statistically significant at p < .001. Internal consistency, as indicated by Cronbach’s alpha, across the entire set of items was also high (α = .94).

Discussion

The current study aimed to construct an integrated measure of sense of community (SOC) by incorporating both context-specific and context-general characteristics of SOC. Although shared goals and responsibility were considered to be an essential element of SOC (Glynn, 1981), this context-general aspect of SOC has been excluded from the measures of SOC for the higher education classroom. Therefore, we attempted to capture the classroom dynamics of higher education (e.g., social relationships with an instructor and peers and student engagement with content), while incorporating the domain-general features of SOC (e.g., shared goals and responsibility).

The principal-axis factor analysis performed with a validation sample implied that a four-factor structure was optimal. This inference was generally supported by using CFA with a replication sample. The four factors included (a) Shared Goals and Responsibility, (b) Relationship with Instructor, (c) Student Engagement, and (d) Peer Valuation. The result provided empirical evidence that the dimension of “Shared Goals and Responsibility” is a meaningful addition to the assessment of SOC. Striving to accomplish common goals among class members by working together (i.e., shared goals and responsibilities) clearly adds substantive value to understanding the construct of classroom SOC.

Tests of measurement invariance indicated that the factor structure of the integrated measure of classroom SOC is equivalent across face-to-face and online courses, thus providing evidence that the integrated measure of SOC is appropriate to examine group differences across different course delivery methods. The invariant factor structure of this measure across face-to-face and online students enables researchers to conduct research on how best to understand courses to promote students’ classroom SOC.

The latent mean analysis indicated that students taking online courses tend to perceive a lower SOC across each measured dimension than students enrolled in traditional, face-to-face courses. This result might explain the consistent challenges of isolation, retention, and achievement that are observed in online education (Beard & Harper, 2002). Because student–instructor and peer-to-peer interactions are key components of classroom SOC, exploring the differences in social interactions across course delivery methods may be useful for improving the online experience. Lack of socialization opportunities, real-time communication, and direct interaction with the instructor has been reported as disadvantages to the online classroom (Beard & Harper, 2002) and may reduce students’ perceived SOC in the learning environment. However, the current results do not imply that the mean differences are an effect of online classrooms. Reported differences in latent means should be interpreted with caution because of the difficulty of separating the effect of online classrooms on SOC from other, potentially confounding variables such as course design (Twigg, 1996). Future research should aim at systematically manipulating course design and instructional methods in online classrooms to examine the effect on SOC.

Given that online courses and distance education programs are a rapidly growing alternative to traditional face-to-face educational courses (Barnard-Brak & Shiu, 2010), these findings raise important questions about potential intervention strategies that may be used to increase the perception of shared goals, the quality of instructor interaction, student engagement, and the valuation placed on student peers for online learning communities among undergraduate students. Increases in real-time communication and building group activities would encourage peer-to-peer interactions and possibly result in desirable student achievement gains (Beard & Harper, 2002). It is important to note some limitations within the current study. Although our integrated measure of SOC may be used in its current form, it is interesting to note that the error terms of two items within the Student Engagement factor were inter-correlated. The presence of correlated error terms between these two items indicates that reliable variance exists that is unaccounted for by the Student Engagement factor. A review of the content of these items reveals that both refer to student interest; consequently, there may be an unexamined, yet distinct, interest factor that may provide further insight into SOC. As the Peer Valuation factor is only defined by two items, future psychometric work should construct additional items to increase the reliability of this factor.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Abdelkader

Bouslama

(2014). Adolescents sense of community: A new measurement scale. Journal of Marketing Research & Case Studies, 2014, 1-10. doi:10.5171/2014.255150

Admiraal

Lockhorst

(2012). The Sense of Community in School Scale (SCSS). Journal of Workplace Learning, 24, 245-255. doi:10.1108/13665621211223360

Barnard-Brak

Shiu

(2010). Classroom Community Scale in the blended learning environment: A psychometric review. International Journal on E-Learning, 9, 303-311. Retrieved from http://www.editlib.org/p/30426

Beard

L. A.

Harper

(2002). Student perceptions of online versus campus instruction. Education, 122, 658-663. doi:10.1007/BF02763579

Bishop

P. D.

Chertok

Jason

L. A.

(1997). Measuring sense of community: Beyond local boundaries. The Journal of Primary Prevention, 18, 193-212. doi:10.1023/A:1024690424655

Borsboom

(2006). When does measurement invariance matter? Medical Care, 44, 176-181. doi:10.1097/01.mlr.0000245143.08679.cc

Bush

A. M.

Svinicki

M. D.

Achacoso

M. V.

Kim

M. S.

(2004, April). Developing classroom community: Defining dimensions of the classroom community scale. Paper presented at the Annual Meeting of American Educational Research Association, San Diego, CA.

Byrne

B. M.

(2010). Structural equation modeling with AMOS: Basic concepts, applications, and programming (2nd ed.). New York, NY: Taylor & Francis.

Chertok

(1990). A measure of perceived sense of community. Unpublished manuscript.

10.

Glynn

T. J.

(1981). Psychological sense of community: Measurement and application. Human Relations, 34, 789-818. doi:10.1177/001872678103400904

11.

Gorsuch

R. L.

(1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

12.

Hill

J. L.

(1996). Psychological sense of community: Suggestions for future research. Journal of Community Psychology, 24, 431-438. doi:10.1002/(SICI)1520-6629(199610)24:4%3C431::AID-JCOP10%3E3.3.CO;2-Z

13.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185. doi:10.1007/BF02289447

14.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118

15.

Kline

R. B.

(2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford.

16.

Mannarini

Fedi

(2009). Multiple senses of community: The experience and meaning of community. Journal of Community Psychology, 37, 211-227. doi:10.1002/jcop.20289

17.

McMillan

D. W.

(1996). Sense of community. Journal of Community Psychology, 24, 315-325. doi:10.1002/(SICI)1520-6629(199610)24:4%3C315::AID-JCOP2%3E3.0.CO;2-T

18.

McMillan

D. W.

(2011). Sense of community, a theory not a value: A response to Nowell and Boyd. Journal of Community Psychology, 39, 507-519. doi:10.1002/jcop.20439

19.

Muthén

L. K.

Muthén

B. O.

(2007). Mplus user’s guide (5th ed.). Los Angeles, CA: Author.

20.

Nowell

Boyd

(2010). Viewing community as responsibility as well as resource: Deconstructing the theoretical roots of psychological sense of community. Journal of Community Psychology, 38, 828-841. doi:10.1002/jcop.20398

21.

Nowell

Boyd

(2011). Sense of community as construct and theory: Authors’ response to McMillan. Journal of Community Psychology, 39, 889-893. doi:10.1002/jcop.20504

22.

O’Connor

(2013). From isolation to community: Exploratory study of a sense-of-community intervention. Journal of Community Psychology, 41, 973-991. doi:10.1002/jcop.21587

23.

Rovai

A. P.

(2002). Sense of community, perceived cognitive learning, and persistence in asynchronous learning networks. Internet and Higher Education, 5, 319-332. doi:10.1016/S1096-7516(02)00130-6

24.

Rovai

A. P.

Wighting

M. J.

Lucking

(2004). The Classroom and School Community Inventory: Development, refinement, and validation of a self report measure for educational research. Internet and Higher Education, 7, 263-280. doi:10.1016/j.iheduc.2004.09.001

25.

Satorra

Bentler

P. M.

(2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75, 243-248. doi:10.1007/s11336-009-9135-y

26.

Summers

J. L.

Beretvas

S. N.

Svinicki

M. D.

Gorin

J. S.

(2005). Evaluating collaborative learning and community. Journal of Experimental Education, 73, 165-188. doi:10.3200/JEXE.73.3.165-188

27.

Summers

J. L.

Svinicki

M. D.

(2007). Investigating classroom community in higher education. Learning and Individual Differences, 17, 55-67. doi:10.1016/j.lindif.2007.01.006

28.

Top

(2012). Blogging as a social medium in undergraduate courses: Sense of community best predictor of perceived learning. Internet and Higher Education, 15, 24-28. doi:10.1016/j.iheduc.2011.02.001

29.

Tovar

Simon

M. A.

(2010). Factorial structure and invariance analysis of the Sense of Belonging Scales. Measurement and Evaluation in Counseling and Development, 43, 199-217. doi:10.1177/0748175610384811

30.

Twigg

C. A.

(1996). Is technology a silver bullet? Educom Review, 31(2), 28-29.

31.

Velicer

W. F.

(1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41, 321-327. doi:10.1007/BF02293557

32.

Warner

Kerwin

Walker

(2013). Examining sense of community in sport: Developing the Multidimensional “SCS” Scale. Journal of Sport Management, 27, 349-362.