Abstract
The psychometric properties of the scores on the Epistemic Belief Inventory were examined using an exploratory factor analysis (principal axis factor) and confirmatory factor analysis (CFA) on a total sample of 1,242 elementary school teachers. Results of the EFA supported the hypothesized five-factor model but the items had loaded on different factors. Overall, the results showed that the five-factor model did not fit the data and that the EBI was not interpretable with the sample in this study.
Introduction
The Epistemic Belief Inventory (EBI) was developed by Schraw, Dunkle, and Bendixen (1995) who also produced evidence in support of the instrument’s use and interpretation in 2002. Comprising 32 items, the EBI aimed to measure five dimensions of epistemological beliefs: simple knowledge (8 items), certain knowledge (7 items), omniscient authority (5 items), innate ability (7 items), and quick learning (5 items). The score for each dimension is obtained by adding the items scores. Since its development, the Cronbach alphas for the EBI have been reported in research, ranging from .67 to .87 (Bendixen, Schraw, & Dunkle, 1998) and .66 to .83 for the subscales (Ravindran, Greene, & DeBacker, 2005).
Toward greater precision in measurement and factorial validity, Schraw, Bendixen, and Dunkle (2002) recommended examining its psychometric properties of the EBI using modern techniques such as confirmatory factor analysis (CFA). In using CFA, the hypothesized number of underlying factors and the interfactor relationships are specified a priori, either from knowledge of the literature or from theoretical hypothesizing (Hair, Black, Babin, Anderson, & Tatham, 2006). The aim of the present study is to investigate the psychometric properties of the Epistemic Belief Inventory (Schraw et al., 1995) on a sample of school teachers from a culture different from the original sample. The findings of this study can provide evidence of the instrument’s cross-cultural validity.
Method
Participants and Procedure
The total number of participants in this study was 1,242 teachers from 32 elementary schools. Of these, 613 and 629 participants were used for the EFA and CFA respectively. Eighty-three percent (n = 1,036) were females and the majority of them had between 1 and 3 years of teaching service (n = 387, 31.2%). One hundred schools were invited and a total of 32 schools (32%) agreed to participate. These were sent the URL of an online questionnaire comprising the EBI and questions to capture various demographic details. The estimated response rate was 39 per school. On average, participants spent about approximately 15 min to complete the EBI questionnaire and each school was represented by about 38 participants. As English is used as the official language in Singapore, all items in the EBI were presented in English.
Instrument
The 32-item EBI (Schraw et al., 1995) was employed in this study. Participants were asked to indicate the extent to which they agreed or disagreed with each item by using a 6-point scale (1 = strongly disagree; 6 = strongly agree). All items were worded in the same direction except for seven of the 32 items (items 2, 6, 14, 20, 24, 30, 31) and these were reverse-coded at the data analysis stage. A high score on a particular subscale indicates a high level of belief of the factor.
Results
Descriptive Statistics and Test of Univariate Normality
The combined data (n = 1,242) were examined for out-of-range responses (i.e., responses greater than 6), and none were detected. As the data were collected using an online form that prevented submission of uncompleted forms, no missing data were found. The means and standard deviations for each of the 32 items ranged from 2.06 to 5.05 and .96 to 1.76 respectively. To test the assumption of univariate normality, the skew and kurtosis were examined using Kline’s (2005) suggested cutoffs of |3.0| and |8.0|, respectively. The skewness of the 32 items ranged from −1.46 to 1.24, and the values for kurtosis ranged from −1.45 to 2.93, indicating that the responses were fairly normally distributed.
Exploratory Factor Analysis
A principal axis factor (PAF) analysis with promax rotation was performed on the scores of the 32 items of the EBI obtained from 1242 participants. PAF was used in this study because it is appropriate in situations where latent constructs or factors are thought to cause variable responses. Also, PAF analyzes only the common variance that a variable shares with other variables which, unlike principal component analysis, analyzes total variance (Henson & Roberts, 2006). The factor loading and structure Table 1 shows the factor and structure matrices of the 32-item EBI.
Factor and Structure Matrices of the 32-Item EBI
Note: SK = simple knowledge; CK = certain knowledge; IA = innate ability; OA = omniscient authority; QL = quick learning.
The eigenvalues-greater-than-one (K1) rule, scree test, parallel analysis, and the interpretability of different factor solutions served as the criteria to determine the number of factors to extract. The K1 rule retains all factors with eigenvalues greater than 1.0, whereas the scree test illustrates the plotted eigenvalues for drastic changes between adjacent pairs of plotted eigenvalues. In contrast, parallel analysis compares the initially extracted eigenvalues to random data sets that are the same size and other descriptive statistics as the obtained data being evaluated. When the eigenvalue for a component in the random data exceeds the size of the component in the true data set, only the preceding factors are retained for further analysis (O’Connor, 2000). This approach to identifying the correct number of components to retain has been shown to be among the most accurate because the Kaiser’s criterion and Catell’s scree test have a tendency to overestimate the number of components (e.g., Velicer, Eaton, & Fava, 2000).
All of the extraction methods supported a five-factor solution to be retained for the final solution, except the K1 rule, which suggested eight factors should be retained. Using the recommended factor loading of +/− .30 as minimal level for interpretability for a sample size of at least 350 (Hair et al., 2006), three items (6, 18, and 28) were dropped. In addition, items 20 and 32 were in the unexpected direction (negative instead of positive) and these were removed as well, making a total of 5 items being excluded from further analyses. Table 2 shows the results of the exploratory factor analysis with the retained factors and the loadings of 27 items. From the item clustering around each factor, Factor 1 may be named as “Innate Ability,” Factor 2 as “Absolute Knowledge,” Factor 3 as “Simple Knowledge,” Factor 4 as “Knowledge Authorities,” and Factor 5 as “Knowledge Ambiguity.”
Results of the Principal Axis Factor Analysis With Oblimin (Promax) Rotation) of the EBI (27 items)
Notes: (a) Items with factor loadings of .30 or greater (accounting for 10% or more of the item variance) are included. (b) SK = simple knowledge; CK = certain knowledge; IA = innate ability; OA = omniscient authority; QL = quick learning.
This item was reverse-scored.
Confirmatory Factor Analysis
The confirmatory factor analysis (CFA) was performed on the 27-item EBI using a separate sample of 629 participants. In the CFA, the five factors were allowed to correlate and the items/factor relationships were specified as indicated by the EFA (Table 1). Identification of the model was done by using the standardization method (where all covariances were set to 1.0). The fit of the hypothesized five-factor model was assessed by three absolute (χ2, RMSEA, & SRMR) and two incremental (TLI & CFI) fit indices. The chi-square statistic (χ2) assesses the difference between the sample covariance matrix and the implied covariance matrix from the hypothesized model and a statistically nonsignificant χ2 indicates adequate model fit. The RMSEA and SRMR are sensitive to model misspecification, with adequate fit represented by values of .06 and .08 or less, respectively. The incremental fit indices, TLI and CFI with a recommended cutoff of .95 or greater as indicative of acceptable fit (Hu & Bentler, 1999). However, it should be noted that such cutoffs are imperfect although they are widely referred to. Multivariate normality was examined using Mardia’s normalized multivariate kurtosis value. The Mardia’s coefficient for the data in this study was 171.86, and this is lower than that of 899 computed from the formula p(p+2) where p equals the number of observed variables in the model (Raykov & Marcoulides, 2008), indicative of multivariate normality. As such, maximum likelihood (ML) estimation was used to estimate the model’s parameters and fit indices.
Overall, the fit of the five-factor 27-item model was poor (χ2 = 926.332, p ≤ .001; χ2/df = 2.950; TLI = .801; CFI = .822; RMSEA = .056; SRMR = .060). Although the RMSEA and SRMR suggested a reasonable fit to the data, the TLI and CFI were below the recommended value .95 and above.
The lack of fit may be the result of failing to estimate the direct relationships between items and factors, for example, cross-loadings were fixed to zero to model the hypothesized five-factor structure. Bryne (2001) suggested that if a model is correct, the absolute value of most standardized covariances of residuals is expected to be less than three. An inspection of the standardized residual covariances matrix revealed that of the 378 standardized residual covariances, more than 50 had exceeded the absolute value of three. The large number of values exceeding three indicated that these observed variables were not explained well by the proposed model.
Discussion
This study aims to examine the factorial validity of the five-factor model of epistemological beliefs proposed by Schraw et al. (1995) on a sample of elementary teachers in an Asian culture. The results showed that, at the EFA level, there was support for the five-factor hypothesized model that underlies Epistemology Belief Inventory (EBI). However, due to the misspecifications at the item-factor level, removal of five items due to their low factor loadings (less than 0.3), unexpected directional signs (negative instead of positive), and poor model fit from the CFA, there was evidence of complex model misspecification. In other words, a misrepresentation of the relationship between items and factors has occurred. In addition, the items belonging to more than one factor from the original EBI had loaded together and this rendered a solution that was not interpretable.
Overall, the attempts in this study to identify a plausible factor structure for the EBI items were thorough but unsuccessful. Several limitations exist in this study. First, the sample used in this study is different from that of Schraw et al., (1995). While a measure should be validated in different contexts with different populations for greater usability, it is possible that the students used in the original study and the teachers in this study had different conceptions of epistemology. Second, the epistemological beliefs of the teacher participants in this study may have been shaped through the process of teaching and learning that they undertake while at their workplaces, a scenario unlike the prevocational participants in the original study (Schraw et al., 1995).
Future research on the EBI may include additional validation involving participants across different cultures and populations and professions. Furthermore, measurement invariance should be rigorously pursued to establish cross-cultural validity. Given that the EBI was originally developed and tested in North America, its generalizability to other cultures is of interest to empirical researchers and psychometricians alike.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: “Data for this study was collected as part of a research funded by grant LSL 07/05 MJ from the Singapore Learning Sciences Laboratory, National Institute of Education, Nanyang Technological University.”