Pedagogical and Social Climate in School Questionnaire

Abstract

This study evaluated the factorial structure of the Pedagogical and Social Climate in School (PESOC) questionnaire among 307 teachers in Bulgaria. The teacher edition of PESOC consists of 11 scales (i.e., Expectations for Students, Unity Among Teachers, Approach to Students, Basic Assumptions About Students’ Ability to Learn, School–Home Relations, Teacher Cooperation, Teachers’ Professional Development, Teaching Activities, Student Valuation, Principal’s Pedagogical Leadership, and School Management). A confirmatory factor analysis conducted with structural equation modeling supported a bi-dimensional factor model (Students and Teachers; School Leadership and Management). School climate indicators were also significantly and positively correlated. PESOC is an asset to the literature on assessment of school climate with evidence for factorial validity and reliability in an underresearched international context.

Keywords

PESOC school climate CFA Bulgaria

This study examines factorial validity and reliability of the Pedagogical and Social Climate in School (PESOC) questionnaire. PESOC is used to evaluate the pedagogical and social climate of a school (Grosin, 2004). In PESOC, pedagogical and social climate is operationalized as a set of school characteristics such as norms and expectations, values, teaching, and organizational practices. Such facets of school climate have been shown as central to students’ well-being and academic achievement (Thapa, Cohen, Guffey, & Higgins-D’Alessandro, 2013; Zullig, Koopman, Patton, & Ubbes, 2010). The measure is grounded in school effectiveness research to capture both cultural (e.g., objectives, values, and expectations) and structural (e.g., the content and forms of pedagogical activities) aspects of school life through 11 theoretically based subscales: (a) school personnel’s expectations for students’ behavior and academic performance; (b) unity or perceived level of agreement among teachers about school goals, norms, and rules; (c) social rules and assessment regarding students’ behavior and achievement; (d) basic assumptions about students’ ability to learn; (e) school relation and communication with students’ parents; (f) teacher interaction and cooperation; (g) teachers’ professional development; (h) teaching activities; (i) evaluation of students’ academic progress; (j) principal’s pedagogical leadership; and (k) teachers’ perception of school management (e.g., the principal’s involvement and support).

The validity and reliability of the PESOC have been explored in large teacher samples covering several regions in Sweden (Carlsson, 2004; Hultin, Ferrer-Wreder, Karlberg, & Galanti, 2015). Although PESOC was originally developed as a uni-dimensional measure, the scarcely available work on its psychometric properties in Sweden supported a two-factor structure. Even less is known about the cross-cultural applicability of PESOC, an area of research that deserves special attention given the importance of general school climate for adjustment and educational success of youth across nations (UNICEF, 2009). We report the first results of a study where the PESOC was administered to a sample of Bulgarian teachers. This study extends prior work on factorial structure of PESOC in other nations such as Sweden, but with a focus now on an understudied region in Europe. We also explore interrelations among different subscales of PESOC given the lack of such evidence in the literature on school climate and PESOC in particular. We address the above issues by testing (a) internal consistency; (b) interrelations among PESOC subscales in terms of expectations for students, unity among teachers, approach to students, basic assumptions about students’ ability to learn, school–home relations, teacher cooperation, teachers’ professional development, teaching activities, evaluation of students, principal’s pedagogical leadership and school management; and (c) factorial validity of PESOC in a confirmatory factor analysis (CFA).

Method

This study was carried out in 10 schools in Western and Southern Bulgaria (post-communist country in Eastern Europe, bordering with Romania, Greece, Macedonia, Serbia, Turkey, and the Black Sea) with a sample of 307 teachers. Thirty-two percent of the teachers had a bachelor’s degree in education, 68% a master’s degree, and 0, 7% high school degree. Eighty-three percent were teachers at a high school level and 17% at middle school. The selected schools were representative of geographically and ethnically mixed populations across the country and were sampled as part of a larger ongoing study in these regions. All contacted schools agreed to participate, and the teachers’ response rate was very high (up to 95%) due to long-term collaboration with the research team and incentives to participants. PESOC is composed of 67 items rated on a 4-point Likert-type scale (completely agree–completely disagree), which form 11 subscales (see Table 1). Items were translated from English into Bulgarian by four bi-lingual speakers while adhering to standard guidelines to ensure linguistic equivalence by adopting a culturally sensitive translation of items in a committee approach to assure good translation, language features, and cultural applicability of item contents in the local culture (van de Vijver & Leung, 1997). In addition, two separate focus groups of teachers were involved in assessing the appropriateness of the translated items. Teachers filled in PESOC in the school and were assisted by four trained research assistants.

Table 1.

Sample.

	Descriptives
Age, M (SD)	46.79 (9.15)
Gender, n
Male	53
Female	245
Teacher PESOC subscale, M (SD)
Student Expectations	3.18 (0.52)
Unity	3.27 (0.41)
Student Approach	3.23 (0.42)
Basic Assumptions	3.15 (0.52)
School Home	3.11 (0.48)
Teacher Collaboration	3.19 (0.43)
Teacher Development	3.21 (0.52)
Teaching	3.09 (0.50)
Student Evaluation	3.19 (0.52)
Principal’s Leadership	3.22 (0.43)
School Management	3.30 (0.56)

Note. n = total number; PESOC = Pedagogical and Social Climate in School.

Results and Discussion

Preliminary analyses tested for normality and missing data. At the scale level, kurtosis and skewness values were approximately normal and in the range of ±1. At the item level, we found missingness in the range of 1% to 6%. We used full-information maximum likelihood (FIML) estimation (Enders & Bandalos, 2001). We therefore performed the following steps. First, we examined reliability of PESOC using coefficient alpha (recommended value > .70) at scale and overall levels (Cronbach & Shavelson, 2004). The analyses indicated good internal consistency values for each scale and the overall score (see Table 2). Second, bivariate Pearson correlations among all PESOC scales were explored showing that all subscales correlated positively and significantly (Table 3). Third, we estimated a uni-dimensional model of PESOC via CFA in a structural equation model using Mplus (Muthén & Muthén, 2010). Model fit was tested with the Comparative Fit Index (CFI; recommended value > .90) and the root mean square error of approximation (RMSEA; recommended value < .08; Hu & Benter, 1999). The CFA testing a single-factor model controlling for the clustered nature of data by school showed a good fit, χ²(44, N = 306) = 125.36, p < .001, CFI = .941, and RMSEA = .078. Because our sample’s characteristics assume an analysis with clustered data, we used the TYPE=COMPLEX command in Mplus, which uses a sandwich estimator considered in the main substantive analyses that teacher responses are grouped (e.g., by school), and that responses are potentially interdependent. Clustered data are those that can be classified into a number of distinct groups within a study, and our analysis considers the fact that it is critical to take data clustering into account (Sally, James, & Bryce, 2010). In addition, in line with prior work (Hultin et al., 2015) and to provide a stronger evidence of PESOC factorial structure, we tested a two-factor model (Factor 1 = expectations for students, unity among teachers, approach to students, basic assumptions about students’ ability to learn, school–home relations, teacher cooperation, teachers’ professional development, teaching activities, evaluation of students; Factor 2 = principal’s pedagogical leadership and school management). This model showed a slightly improved and acceptable fit with a bi-dimensional factor structure, χ²(43, N = 306) = 106.79, p < .001, CFI = .954, and RMSEA = .070 (Figure 1). The two latent factors were significantly correlated, and the magnitude of the correlation was high (.77), which suggests a strong relation between the two school climate indicators (Tabachnick & Fidell, 2001). We also performed chi-square difference test, indicating that the second “larger” model with more freely estimated parameters fits the data better than the first “smaller” model (Kline, 1998). Therefore, we opted for a two-factor model also in line with prior psychometric investigations on the instrument (Hultin et al., 2015).

Table 2.

Scales, Item Examples, and Internal Consistency Coefficients of Teacher PESOC.

Scale (total items)	Example item	Coefficient alpha
1. Student Expectations (4)	Our principal has high demands and expectations for pupil academic results	.78
2. Unity (10)	There is a high degree of unity among teachers regarding the schools goals	.88
3. Student Approach (6)	It is a well-established policy that teachers show respect for pupils	.78
4. Basic Assumptions (3)	It is well known by the staff that knowledge goals are the most important task of this school	.70
5. School Home (6)	Parents are always informed if a child behaves badly in school	.83
6. Teacher Collaboration (8)	I have the support of my colleagues in selecting teaching content and methods	.88
7. Teacher Development (4)	In this school, you feel that you can develop as an educator	.87
8. Teaching (6)	There is a policy in this school that no pupil will leave this school without gaining basic knowledge and skills	.84
9. Student Evaluation (2)	Regular evaluation of pupil results is used as the basis for planning teaching in this school	.73
10. Principal’s Leadership (13)	In this school, the principal has the final responsibility for teaching quality	.91
11. School Management (3)	There is always a member of school management accessible to discuss and give advice on issues regarding teaching and education	.92
Overall		.97

Note. n = total number; PESOC = Pedagogical and Social Climate in School.

Table 3.

Correlations Among Teacher PESOC Subscales.

Scale	1	2	3	4	5	6	7	8	9	10	11
1. ES	—
2. U	.60	—
3. AS	.51	.61	—
4. BA	.50	.53	.60	—
5. SH	.49	.56	.64	.53	—
6. TC	.49	.60	.61	.51	.64	—
7. TD	.51	.65	.63	.51	.62	.69	—
8. T	.53	.55	.60	.53	.59	.61	.68	—
9. SE	.46	.50	.51	.50	.55	.56	.61	.74	—
10. PL	.57	.56	.58	.47	.49	.54	.51	.57	.58	—
11. SM	.43	.39	.37	.24	.33	.45	.41	.38	.42	.60	—

Note. ES = Expectations for Students; U = Unity Among Teachers; AS = Approach to Students; BA = Basic Assumptions About Students’ Ability to Learn; SH = School–Home Relations; TC = Teacher Cooperation; TD = Teachers’ Professional Development; T = Teaching Activities; SE = Students’ Evaluation; PL = Principal’s Leadership; SM = School Management.

All correlations are significant at p < .01.

Figure 1.

Estimates and residual variance of the two-factor model of the teacher PESOC.

Adaptation of the school climate measures to different languages is needed to allow valid evaluations and cross-cultural comparisons. This study is the first to examine the psychometric properties of PESOC in a group of individuals who speak Bulgarian. This is the first study of the factorial validity and reliability of PESOC with data from different schools in an international context (e.g., Bulgaria) outside of the country where the instrument was developed. Consistent with earlier investigations in Sweden (Hultin et al., 2015), a bi-dimensional factorial structure provided good fit to the data. We were also able to confirm a high level of internal consistency at single and overall scale levels, and that school climate characteristics are significantly and positively related to each other.

Although the Bulgarian version of PESOC demonstrated adequate reliability and factor validity for research purposes, some limitations need to be acknowledged. This is a first-time study in Bulgaria, and replications of our findings are needed prior to generalizing results further in a Bulgarian or any other international school context. Future studies could also explore additional indicators of test–retest stability and predictive validity. In addition, multi-method studies of PESOC (e.g., teacher and student ratings) are important to complement these teacher-reported only study findings. Future research would also extend school climate analysis to mental health outcomes of youth. Given strong evidence that the severity of the mental health problems in the population is related to school setting (Bond et al., 2007), there is need to carry out further studies with a wide measurement net that includes important educational and public health outcomes. Additional promising extension of the present study would be to link school climate domains to teachers’ perceived school satisfaction, which would yield richer information for schools (Zullig et al., 2010).

Nevertheless, this study provides new and preliminary evidence for the validity, reliability, and utility of this scale in a Bulgarian-speaking sample. The Bulgarian version of PESOC adds to the existing school climate literature and provides diversity to a research pool that has been dominated by Western samples. School climate factors can affect student academic success and positive socio-emotional developmental outcomes (Cohen, McCabe, Michelli, & Pickeral, 2009). Measurement tools on these factors such as the one detailed here play important role for educational policy and teaching practices. Therefore, PESOC may be useful in school climate–related interventions.

Footnotes

Acknowledgements

The authors would like to acknowledge the support by a COFAS FORTE (Swedish Research Council for Health, Working Life and Welfare) Marie Curie Grant (Forte Projekt 2013-2669) to the first author. We are also extremely grateful to Dr. Eva Jecheva and the National Agency for Child Protection, all schools and teachers and in particular Neli Filipova, Ivanina Ivanova, Ivanina Noncheva, Radka Kostandinova, Albena Damianova, Stoyka Jekova, Svetla Atanasova, Neli Kiuchukova, Ani Angelova, Lilia Stoyanova, Tianka Kardjilova, Zvetan Terziev, Elena Ianeva, Venizslav Chobanov, Kamelia Mateva and Petko Petkov for their help in carrying out the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: COFAS Marie Curie Grant (Forte project 2013-2669).

References

Bond

Butler

Thomas

Carlin

Glover

Bowes

Patton

(2007). Social and school connectedness in early secondary school as predictors of late teenage substance use, mental health, and academic outcomes. Journal of Adolescent Health, 40, 357. e9-e18. doi:S1054-139X(06)00422-8

Carlson

(2004). Reliabilitetsanalys av Bedömningsinstrument för Pedagogiskt och Socialt Klimat (PESOK) i skolor [Reliability of assessment tools for teaching and social climate (Pesok) in schools] (Research Report No. 2004:6). Department of Statistics, Stockholm University, Sweden.

Cohen

McCabe

E. M.

Michelli

N. M.

Pickeral

(2009). School climate: Research, policy, practice, and teacher education. Teachers College Record, 111, 180-213.

Cronbach

L. J.

Shavelson

R. J.

(2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64, 391-418.

Enders

C. K.

Bandalos

D. L.

(2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8, 430-457. doi:10.1177/0013164401615001

Grosin

(2004). Skolklimat, prestation och anpassning i 21 mellan- och 20 högstadieskolor (Forskningsrapport 71). [School climate, performance and adaptation in 21 middle and 20 high schools] (Research report 71). Pedagogiska institutionen, Stockholms Universitet [Institue of Pedagogy, Stockholm University], Sweden.

Benter

(1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118

Hultin

Ferrer-Wreder

Karlberg

& Galanti

(2015). Psychometric properties of an instrument to measure social and pedagogical school climate among teachers (PESOC). Manuscript in preparation.

Kline

R. B.

(1998). Principles and practice of structural equation modeling. New York, NY: Guilford Press.

10.

Muthén

L. K.

Muthén

B. O.

(2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Author.

11.

Sally

G. S.

James

A. D.

Bryce

(2010). A study of clustered data and approaches to its analysis. The Journal of Neuroscience, 30, 10601-10608. doi:10.1523/JNEUROSCI.0362-10.2010

12.

Tabachnick

B. G.

Fidell

L. S.

(2001). Using multivariate statistics. Boston, MA: Allyn & Bacon.

13.

Thapa

Cohen

Guffey

Higgins-D’Alessandro

(2013). A review of school climate research. Review of Educational Research, 83, 357-385. doi:10.3102/0034654313483907

14.

UNICEF. (2009). UNICEF child friendly schools programming: Global evaluation final report. Retrieved from http://www.unicef.org/education/files/CFS_Executive_Summary_V2r.pdf

15.

van de Vijver

F. J. R.

Leung

(1997). Methods and data analysis for cross-cultural research. Thousand Oaks, CA: Sage.

16.

Zullig

K. J.

Koopman

T. M.

Patton

J. M.

Ubbes

V. A.

(2010). School climate: Historical review, instrument development, and school assessment. Journal of Psychoeducational Assessment, 28, 139-152. doi:10.1177/0734282909344205