Abstract
Researcher error is an unfortunate reality that scholars are mindful of and take numerous precautionary steps to guard against. Even still, problems arise, mistakes are made, and errors go undetected, resulting in evidence that can potentially misrepresent the phenomenon under investigation. Fortunately, science, as a process of knowledge formation, functions from a set of design principles that openly and publicly guide the accumulation of evidence and understanding in a way that can reveal inconsistent processes and findings. Such principles have worked as designed for the research on teacher trust in district administration, providing both an opportunity to correct the record, as I do here, and to advance a deeper understanding of the nature of trust, as Lisa Romero and Doug Mitchell do by joining the analysis.
“Teacher Trust in District Administration: A Promising Line of Inquiry,” published in the October 2016 issue, set out to establish a foundation for future research by conceptualizing teacher trust in district administration, developing a useful measure, and testing the relationship between district trust and teacher commitment. On balance, the article accomplishes these objectives: survey items are conceptually aligned, the items are strongly related to trust, and trust has consequences for teacher beliefs. The analysis and results of the empirical test accurately report the relationship between trust and teacher commitment. Regrettably, I made errors in the measurement models that require attention. The intent here is to identify the mistakes and to present results from a reanalysis of the data.
Turning to mistakes, an error was found in the description of the sample data used for the measurement models. Sample data were from 606 teachers in 72 schools, not 849 teachers in 73 schools as originally reported. The sample statistics and item correlations in the article are reported correctly. Another error was made in reporting the measurement models and the test of convergent validity. Residuals for items with a common facet were correlated to account for their common variance. This was not done with the alternative models because items were hypothesized to share variance with a second-order factor. These correlated residuals were not presented in the models. Other errors were found with factor loadings, squared multiple correlations, and residuals. Survey items in the original table followed the order in which the items were listed in the scale development section of the article. The problem was that the actual survey was not organized this way, leading some estimates to be assigned to the wrong survey item.
As a result of these errors, the measurement models were reanalyzed without correlating the residuals to see what happens to the fit statistics and parameter estimates. Fit indices reported in Table 1 show a drop in model fit when residuals are not correlated. For the 10-item model, the original study reported a chi-square of 188.6, root mean square error of approximation (RMSEA) of .05, normed fit index (NFI) of .97, comparative fit index (CFI) of .97, and Tucker–Lewis index (TLI) of .95. The new indices are chi-square of 442.73, RMSEA of .14, NFI of .93, CFI of .93, and TLI of .91. For the five-factor model, the original study reported a chi-square of 384.6, RMSEA of .14, NFI of .93, CFI of .94, and TLI of .91. The new model saw a change in NFI from .93 to .94. For the three-factor model, the original study reported a chi-square of 224.7, RMSEA of .11, NFI of .93, CFI of .94, and TLI of .94. In the new model, chi-square changed to 318.77, RMSEA to .12, NFI to .95, CFI to .95, and TLI to .93. For the five-item single-factor model, the original study reported a chi-square of 5.0, RMSEA of .03, NFI of .99, CFI of .99, and TLI of .98. In the new model, chi-square changed to 87.24, RMSEA to .17, NFI of .97, CFI of .97, and TLI of .94.
Corrected Model Fit Indices for the Hypothesized and Alternative Models.
Note. df = degrees of freedom; RMSEA = root mean square error of approximation; NFI = normed fit index; CFI = comparative fit index; TLI = Tucker–Lewis index. N = 606. The sample size of 606 reflects the original number of cases analyzed in the study. The sample size was incorrectly reported as 849 teachers.
p < .01.
Corrected factor loadings, squared multiple correlations, and residuals are reported in Tables 2, 3, and 4. The primary difference between the original and corrected tables reflects a mistake in assigning proper estimates to the correct survey item. Fixing this mistake changes the respective estimates for survey items and the second-order factors, but the general findings of strong factor loadings, squared multiple correlations above 1.0 for the alternative models, and negative residuals for some second-order factors do not appear substantively different from the original article.
Corrected Sample Items, Factor Loadings, and Squared Multiple Correlations for Hypothesized Model and Trimmed Five-Item Model.
Note. N = 606. Factor loadings and squared multiple correlations were statistically significant at p < .01. Degrees of freedom was 35 for the 10-item model and 5 for the 5-item model.
Corrected Sample Items, Factor Loadings, and Squared Multiple Correlations for Alternative Second-Order Five-Factor Model.
Note. N = 606. Factor loadings and squared multiple correlations were statistically significant at p < .01. Degrees of freedom were 30.
Corrected Sample Items, Factor Loadings, and Squared Multiple Correlations for Alternative Second-Order Three-Factor Model.
Note. N = 606. Factor loadings and squared multiple correlations were statistically significant at p < .01. Degrees of freedom were 32.
I also reexamined the correlation between trust and teacher perceptions of the evaluation process without correlating the residuals. Without correlated residuals, fit indices changed from chi-square of 36.1, RMSEA of .03, CFI of .99, NFI of .99, and TLI of .99 to chi-square of 90.67, RMSEA of .08, CFI of .98, NFI of .97, and TLI of .97. There were slight changes in the factor loadings. Factor loadings for trust ranged from .80 to .89 and for teacher evaluation from .70 to .86. The relationship between trust and teacher evaluation remained strong with a bivariate correlation coefficient of .52 (see Figure 1).
Test of convergent validity with a fully latent structural equation model using maximum likelihood estimation.
With the reanalyzed data in mind, it is important to consider how the understanding of teacher trust in district administration changes based on the new results. Two conclusions can be made. First, survey items maintain their strong association with each other and the latent trust concept. This is the case for both single-factor and second-order models, suggesting that the Teacher Trust in District Administration Scale is useful for future research. Second, it is unclear from these limited results if it is better to conceptualize and measure trust as a single-factor construct or as a multidimensional phenomenon. Each of the four measurement models had fit statistics close, at, and exceeding thresholds for TLI, CFI, and NFI, but all models had a RMSEA over .10. The persistence of negative residuals in the second-order models adds to the uncertainty about the nature of teacher trust in district administration. Negative residuals are a sign of model misspecification, raising questions of the appropriateness of the simple second-order factor structures tested in this study.
In sum, the reexamined measurement models do raise questions about the nature and measurement of trust in district administration. The single-factor model has good fit when the residuals for common trust facets are correlated but fit declines when error variance is uncorrelated. It is unclear what this means for the measurement of trust because a theoretical argument can be made for correlating residuals for items that share a common facet in the single-factor model. Negative residuals persist with the second-order models, suggesting specification problems that were not present with the first-order structures. What is clear is that a closer examination into the factor structure of teacher trust in district administration is needed. Such an examination was beyond the scope of this study.