Abstract
This comparative analysis applies a distributed leadership framework to data from teachers and leaders taking the Comprehensive Assessment of Leadership for Learning (CALL). Because the policies educators in Denmark and the United States respond to in their daily practice are related through the transnational policy borrowing process, we are better able to understand how these policies impact educators in their respective countries by comparing their leadership practice through a shared lens: the CALL framework. In this exploratory analysis, we take a comparative perspective by asking: How does distributed leadership practice compare in the US and the Danish contexts of schooling? And: How do views on leadership practice vary according to professional roles in specific national and local (school) contexts? Our conceptual framework has three components: neoinstitutional theory, translation theory, and distributed leadership. We use multilevel confirmatory factor analysis and t-tests with SY2015–2016 CALL data to compare and contrast the pattern of leadership practices teachers and school leaders take-up in Danish and US schools. We found that the leadership practices aligned to a school-wide focus on learning are closely associated with the work of monitoring teaching and learning and building nested learning communities, particularly in the US context of schooling.
Keywords
Introduction
Student learning and well-being must be the core issue for the organization and the activities of the school. The municipality and the school leadership team must have a clear vision, goals and action plan for the development of the school with an enduring focus on the learning and well-being of the individual student (Undervisningsministeriet [UVM, Danish Ministry of Education], 2015). “Effective educational leaders develop, advocate, and enact a shared mission, vision, and core values of high-quality education and academic success and well-being of each student” (National Policy Board for Educational Administration (NPBEA), 2015).
In this study, we use data from the Comprehensive Assessment of Leadership for Learning (CALL) to examine how school leaders and teachers experience and evaluate specific leadership practices within their own school. The survey-based assessment is grounded in a conceptual model of leadership for learning that consists of five domains of leadership practice designed to promote student learning, well-being and equity. CALL also builds on the concept of distributed leadership; leaders, teachers and other personnel are asked to assess the leadership practices taken up by everyone in the school community (Halverson and Kelley, 2017; Kelley and Halverson, 2012). We take a comparative perspective by asking the following research questions in this analysis: How does distributed leadership practice compare in the US and Danish contexts of schooling? How do views on leadership practice vary according to professional roles in specific national and local (school) contexts?
Because the policies that Danish and US school leaders respond to in their leadership work in schools are related to each other through the transnational policy borrowing process (Phillips and Ochs, 2003; Steiner-Khamsi, 2006), we are better able to understand how these policies impact leaders in their respective countries by comparing their leadership practice through a shared lens like the CALL framework. A comparative study allows us to learn more about the nature of distributed leadership when we examine how it operates in two different settings. In the context of this study, the USA, Denmark and many other countries exist in a shared or similar set of policy contexts, particularly related to education. Some countries have been pursuing these similar policies for a lot longer than others. Given this shared or similar context on educational reform, CALL serves as a shared device that allows us to examine school leadership practice among educators (teachers, leaders, pedagogues, other personnel) within schools responding to these related reforms.
Since the CALL framework operates from an understanding that leadership is not a character trait but rather a set of tasks and practices distributed in schools (Spillane, Halverson and Diamond, 2004), this comparative study provides an opportunity to understand the nature of distributed leadership in Denmark and the USA. In Denmark, Danish primary schools serving students in grades 0–9 often are led by a team of school leaders consisting of the school’s principal, assistant principal, and the principal of the pedagogues. Pedagogues are a distinct profession in Danish schools; they traditionally work to support students’ social and emotional well-being as part of their educational experience. In the USA, the concept of distributed leadership is only one among many concepts that scholars and practitioners have used to define and make sense of the leadership work required during a time of change and reform.
Educational leaders in schools are uniquely placed between the worlds of policy and practice. As Danish teachers, school leaders, pedagogues, and other personnel begin to take on practices in their daily work that are mandated through a change in policy at the national level, while policies and, to some extent, practices that have been operating in the US space for a longer period of time, this comparative study provides a type of time lag that allows us to trace and follow the changing nature of leadership practice over time in response to external policy demands. Although CALL was developed in the US context of public schools, school leaders in Denmark and the USA are responding to a similar set of education policy reform initiatives that impact their daily leadership work in schools. Though Denmark has adopted these policies more recently than the USA, CALL serves as a counterpoint to the policy environment that surrounds school practitioners in the USA and Denmark by examining leadership practice in an era of school reform. Both the external policies and the CALL framework serve as a common entity between these two contexts of schooling.
Through this comparative study of leadership for learning we can better understand how educational leadership practices develop, both across national contexts, and within the local school organisation. We use quantitative data from the US Elementary and Danish 0–9 versions of CALL (CALL–DK) to explore differences between the two national contexts, and examine whether or not there are differences by organizational role (Leader/Ledelse, Teacher/Lærer, Pedagogue, other personnel) across the two contexts. Our study focuses only on the roles of teacher and school leader and begins with a review of relevant literature on school leadership and recent changes in the policy environment in the USA and Denmark. We then discuss concepts of coupling and de-coupling from new institutionalism that serve, along with distributed leadership, as the conceptual framework for this study. In the Methods section, we describe the CALL instrument, the process of translating CALL from English into Danish, and provide context on the statistical methods used in this analysis: t-test, confirmatory factor analysis (CFA), and multilevel CFA (MCFA).
Policy contexts and related literature
School reform across policy contexts
Education researchers and school-based practitioners in several Western countries have been working to examine and make sense of changing policies, here exemplified by the Nordic countries compared with the USA (Day, 2015; Höög, Johansson and Olofsson, 2005; Moos 2012; Moos, Krejsler and Kofod, 2008a, 2008b; Moos, Möller and Johansson, 2004). Though the contexts in which these latest education policies are implemented may vary by nation, the content and intended outcomes of the policies run parallel to each other in this era of globalization, where national policies on education share similar themes and goals and may call for similar forms of criticism (Lingard, Martino, Rezai-Rashti and Sellar, 2016). Global trends include an increased focus on accountability for school teachers and leaders, augmented testing programs for students, and changes to the working conditions and professional culture for teachers, leaders and other professions. Given the similarity of policy content, scholars are uniquely poised to interpret, analyze and make sense of the impact these similar reform policies are having on their respective contexts.
The Danish Ministry of Education introduced a major school reform in Fall 2014. The reform was the latest initiative in over a decade of national educational policy marked by an increased focus on accountability, ranking systems, and standardized tests influenced by international policy trends (Andersen, 2007; Gustafsson, 2012). These policies mark the influence of organizations like the Program for International Student Assessment; and, greater attention to the Organisation for Economic Co-operation and Development’s policy recommendations in Denmark (Andersen, Dahler-Larsen and Pedersen, 2009). The 2014 reform, however, was not exclusively focused on accountability policy. It covered other key challenges to Danish schools, such as inclusive education, community partnerships, and equality of opportunity, and can therefore be seen as being simultaneously a break from and in continuity with the Danish tradition for a broad host of purposes for public schooling.
The intentions of the latest Danish reform can be seen as a reflection of the original content of the 2002 No Child Left Behind policy in the USA (Rasmussen, 2018). First, the reform introduces national goals for improving learning outcomes and social well-being for all students, thereby focusing on learning outcomes and equity on a systems level. Second, the Danish school reform builds on prior legislation to increase accountability and transparency in education (Andersen, 2007). The school reform is linked to the general quality reform of the public sector, introducing educational quality systems and evaluation tools such as the national standardized testing system, which was first implemented in 2010, as a means to assess the development in learning outcomes. While the accountability measures applied to schools through students’ test results are not as “high stakes” as has been the case in the USA, the Danish reform does aim to augment accountability for students’ learning outcomes. Specifically, there is an increased demand for school leaders to use performance data to monitor and improve student learning. The reform thus aims to strengthen the community’s trust in public schools and work against parents’ growing tendency to favor private schools over public schools.
The reform builds on a concept of school leadership, which awards leaders a relatively high degree of autonomy (Andersen et al., 2009), which is often described as a trait of educational systems in Scandinavia (Gustafsson, 2012: 60). The reform reflects a model of leadership aiming to make school leaders responsible for the interpretation of policies and leading the necessary changes in their school (Rasmussen, 2018). Danish accountability policies have thus aimed to change school leadership to make it more responsible to, and accountable for, school performance. School principals are expected to work closely within the team of formally designated leaders, and distribute leadership tasks throughout the school to increase efficiency and ownership of policy ideas.
School leadership and functional roles in schools
In recent years, the scholarship on school leadership has shifted from a focus on the traits of leaders to an examination of leadership practice. In the US context, this has happened in part because the expectations for the functional role of the school leader has expanded, due to changing policy, from a managerial position to a role of instructional and transformational leader in schools and increasingly entire districts. A similar expansion has also occurred at the municipal level in Denmark. As schools have struggled with a loss of external legitimacy over the past 30-plus years, policy makers external to schools have called for and mandated a slate of reforms directed at schools, designed to change outcomes related to student learning and achievement, which thereby indirectly impacted teacher practice and school leadership. Thus, the role of the school principal in particular has developed along with changing policy and a growing research base on school leaders. Further, scholarship on school leadership has increasingly recognized that leadership work in schools involves multiple people, irrespective of their official roles in schools (Spillane et al., 2004).
This study contributes to earlier work that examines perceptions of leadership practice by differentiating between the unique role-based perspective that teachers and school leaders have on the leadership work occurring in their schools (see: Blitz and Modeste, 2015; Bowers, Blitz, Modeste, Salisbury and Halverson, 2017). The CALL survey is a multi-rater instrument designed to provide formative feedback to school leaders on school-level leadership practice. As such, CALL is part of a broader body of instruments designed to provide feedback, rather than serving as an evaluation tool in itself (London and Smither, 1995). Because scholarship shows that respondents are more candid when providing feedback but may be less forthcoming when their responses impact performance evaluations (Toegel and Conger, 2003), scholars recommend differentiating between the two types of instruments.
Conceptual framework
The conceptual framework for this study is grounded in concepts from organizational theory and has three main components: neo-institutional theory, specifically the concept of coupling (DiMaggio and Powell, 1983; Weick, 1976); translation theory (Røvik, 2007); and the concept of distributed leadership developed by Spillane et al. (2004). Although educational leadership scholars use a range of terminology and definitions to describe distributed leadership in organizations (Harris, 2013), the CALL framework of leadership for learning is supported by the framework of distributed leadership (Spillane et al., 2004). Under this framework, distributed leadership is described as a set of tasks or practices that occur within a given context or situation and require the work of a leader and a follower to carry it out (Spillane et al., 2004). In this study, this distributed leadership framework connects to the methods used for analysis because it is the foundation of the CALL instrument (Halverson and Kelley, 2017; Kelly and Halverson, 2012); and also to the other two components of our conceptual framework because this framework on distributed leadership is informed by the theoretical frameworks of distributed cognition, activity theory, and institutional theory (Spillane et al., 2004).
Organizational sociologists have used the metaphor of “coupling” to describe the relation between the internal technical core of a given institution and the external environment that monitors, assesses, and exerts some determined amount of control through policy development or other means on the technical core (Meyer and Rowan, 1977; Scott, 2004). In education, “coupling” was initially used to describe the nature of the relationship between schools and local policy-making entities at the level of the school district or municipality as in Denmark, the external local community surrounding schools, and the governing bodies at the state level charged in the USA with providing public education for students throughout the state. Schools were described as being “loosely coupled” with these external organizations and benefiting from a significant amount of trust and legitimacy from external policy makers (Weick, 1976).
The neo-institutional theory on alignment, or “coupling,” of practice and policy messages in organizations has been used as a starting point for analyzing such issues (Coburn, 2004; Hallett, 2010; Shen, Gao and Xia, 2017; Spillane and Scherer, 2011). This tradition discusses the nature of problems that may arise when the symbolic, higher-level, or even ceremonial aspects of organization (DiMaggio and Powell, 1983) that are traditionally the realm of strategy and policy directly engage practice. One outcome of this has been described as “de-coupling”; that is, the lack of connection between practice and the various forms of institutional myths or policy narratives that arise in the context of the organization (Coburn, 2004; DiMaggio and Powell, 1983; Weick, 1976).
The concept of coupling allows scholars to examine, understand, and describe the nature and quality of the relationships among external governing bodies and the internal-practice–facing institution of interest. In education, this includes the relationship between schools and federal governing agencies along with transnational organizations like the Organisation for Economic Co-operation and Development and various testing organizations. Further, the coupling metaphor can be used to examine the nature of relationships inside schools as in the relationship between school leaders and classroom-based teachers.
Coupling helps to explain aspects of the reform process occurring in each of the individual countries included in this analysis: the USA and Denmark. In both places, irrespective of when the reforms start, they are initiated in response to a loss of external legitimacy from external entities like the government, the public, and the private sector toward the public schools. Therefore, they are an attempt by external governing bodies to exert greater control over schools and the education sector by crafting and enacting policies that will be implemented with a greater deal of fidelity than in times past.
Translation theory—policy and practice
School leaders occupy a space between practice and policy. They connect what political leaders want to what teachers and other practitioners do. In an age of accountability policy with an emphasis on instructional leadership, this role becomes even more significant because leadership comes to embody both management and instruction; it performs the policy ideals of the school as a simultaneously symbolic and material practice. Neo-institutional theory has argued that the relation between practice and policy may be understood as a kind of translation process (Røvik, 2007).
Røvik (2007) uses translation theory, adapted from literary theory, because it distinguishes between literal translations and the kind of translations that practically rewrite the text in the new language. Unsurprisingly, literal translations are rarely used because so much literary quality is lost using this method. To preserve artistic quality, the original text has to be transformed, Røvik argues, in a translation based on a deep understanding of the particular conditions of the importing language community. These conditions may make it necessary to position the text—or idea—relative to established norms and interests (Røvik, 2007: 255). Similarly, Coburn highlights the importance of understanding how policy “messages” correspond with “preexisting worldviews or practices” (Coburn, 2004: 217). When messages are assimilated in organizations, they sometimes undergo radical transformations (Coburn, 2004).
Tim Hallett (2010) makes a similar point in his case study of an elementary school. When local actors experience de-coupling or “institutional pressure” (Hallett, 2010: 53), Hallett observes how they try to re-create “tighter couplings where loose couplings were once in place” (Hallett, 2010: 54). Professionals in varying degrees need correspondence between what they do and what policy says they ought to do. In the case of educational accountability, Hallet argues, the policy message and norm of accountability becomes “an acceptable ‘solution’ proffered by reformers who deemed loose coupling a form of ‘disorganization’ that limited improvement” (Hallett, 2010: 56–57). This acceptability is key to understanding the variances in how leaders and teachers see their practice align with policy intentions.
Both Hallett and Coburn propose a high degree of local variance in the perception of policy messages. This is supported by scholarship from Gherardi and Nicolini (2000), who posit that we will always observe some kind of transformation process in the transfer of ideas, knowledge, and visions between different levels in the organizational hierarchy. This predicts a wider variety of outcomes than what is envisioned at the policy level. These concepts provide a platform for examining how educators may engage in translation processes between practice and policy.
Methods
This exploratory study uses MCFA and t-tests with data from the 2015–2016 school year of the CALL to compare and contrast the pattern of leadership practices that teachers and school leaders take up in their daily work in schools in the USA and Denmark. The number of elementary school teachers who responded to the CALL items ranged from 1613 to 1695 for the US version of CALL and from 325 to 335 for teachers in schools with grades 0–9 responding to CALL–DK. The number of elementary school leaders who responded to the CALL items ranged from 63 to 81 for the US version of CALL and from 26 to 27 for school leaders who responded to the Danish version of CALL.
Background—the CALL
CALL is an online formative assessment of leadership practice developed by Professors Richard Halverson and Carolyn Kelley at the University of Wisconsin-Madison through a grant from the Institution of Education Sciences. The CALL instrument is a multi-rater assessment grounded in a framework of distributed leadership that recognizes leadership as a set of tasks and practices that occur in response to a given situation (Spillane et al., 2004). CALL measures leadership across five domains of practice: Focus on learning; Monitoring teaching and learning; Building nested learning communities; Effective use of resources; and Maintaining a safe and effective learning environment.
Each domain is then composed of four subdomains of practice, and each subdomain is measured through individual items that participants respond to when they take the survey.
The leadership practices measured through the items in Domain 1, Focus on Learning, are primarily associated with the work of the formal leader or leaders in a school (Kelley and Halverson, 2012). Specifically, these items measure the work required to establish a common mission for instruction throughout the school, the collaborative work required between teachers and school leaders to support a shared vision of student learning. In addition, these items examine how leaders schedule time for teachers to discuss student achievement data, and grading practices as well as the leadership work required to integrate curriculum and instruction across all groups of students in the school. These items also ask about meaningful supports for student learning. The leadership practices measured through the items and practices in Domain 1, Focus on Learning, are related to the work of an instructional leader or leaders in a school.
The leadership practices measured through the items in Domain 2, Monitoring Teaching and Learning, are primarily associated with the leadership work required to evaluate student learning and teaching practice using summative and formative assessments.
Domain 3, Building Nested Learning Communities, is related to professional learning communities (Robinson, Lloyd and Rowe, 2008) and describes the leadership work required to ensure that professional development leads to improved student learning. Items in Domain 3, Building Nested Learning Communities, also measure the extent to which teachers have ample time to collaborate around teaching and learning, and the extent to which teachers have time to conduct peer observations and learn from each other. Further, these items focus on teacher and staff involvement in making decisions related to scheduling, their discretionary budget, and the extent to which instructional coaches and mentors are available to support teachers in their work.
Domain 4, Acquiring and Allocating Resources, measures the leadership work required to effectively manage teachers and staff members as a resource across the school. These items also measure the strategic use of time to support student learning and teacher practice, the use of external consultants to advance the instructional goals of the school, and the work to coordinate effectively with parents and external community members to support the life of the school and student achievement.
The leadership practices measured in Domain 5, Maintaining a Safe and Effective Learning Environment, are related to expectations of student behavior, the cleanliness and safety of the learning environment, and the supports provided to students throughout the school.
Upon completion of the survey, each school receives a detailed report containing quantitative data and qualitative feedback, based on findings from their results, with suggestions for their future work. An interdisciplinary research team validated the CALL instrument using a Rasch model (Rasch, 1960, 1961, 1966) that accounts for the difficulty of each item as respondents take the survey. The following articles provide additional information on the development and validation of CALL: Blitz, Milanowski and Clifford, 2011; Blitz, Salisbury and Kelley, 2014; CALL, n.d.; Camburn and Salisbury, 2012; Kelley and Halverson, 2012; and Kelley and Dikkers, 2016. In the USA, more than 500 schools have taken CALL. Participants who respond to CALL are educators in rural, suburban, and urban schools across the contiguous USA. In Denmark, participants to CALL–DK are educators in urban and suburban municipalities. In Denmark, CALL–DK has undergone pilot testing and 14 schools participated in CALL–DK during the inaugural year. Currently, schools in Denmark respond to CALL items from the first three of the five domains of practice and a subset of items from the fourth domain of practice.
Data sources: CALL and CALL–DK
This study uses data from two versions of CALL: the US-based English language elementary school version of CALL and the Danish 0–9 version of CALL that is the translated and comparable version of the US elementary school CALL survey. In developing the Danish version of CALL (CALL–DK), Danish scholars from University College Capital (UCC) and US scholars from the CALL team worked to translate the CALL items in a way that reflected the Danish context of school leadership, teaching, and learning.
To begin, the entire survey was translated into Danish. Scholars from UCC worked with local Danish practitioners and municipality-level administrators (comparable to district-level central office administration in the USA) to ensure that the items measured practices occurring in Danish schools or emerging practices recently mandated through the Danish Ministry of Education’s policy initiatives. This iterative process involved several municipalities and schools in Denmark. Through this process, two versions of CALL–DK were finalized: the full version of CALL–DK that included items from all five domains of CALL; and an abbreviated version of CALL–DK that included the items and subdomains primarily from the first three CALL domains. The data used in this analysis comes from the first three domains of CALL for both the Danish and US contexts.
Independent samples T-test and MCFA
Independent sample t-tests allowed us to examine differences that may exist between teachers’ and school leaders’ perceptions of leadership practices occurring in their schools, in Denmark and the USA. After examining respondents’ scores on CALL by role (teachers and leaders only) across national contexts, we then turned to MCFA to observe how teachers perceive the leadership practices measured through the CALL subdomains. MCFA is part of a class of interrelated statistical approaches that allow scholars to measure developed constructs and test theories about the relationship and structure of developed constructs, specifically through factor and path analysis (Kline, 2011). In this instance, because CALL has been developed and validated through other statistical procedures, we are primarily interested in examining the pattern of leadership practices that respondents perceive to be occurring in schools, rather than confirming CALL’s ability to accurately measure a defined construct of leadership practice(s).
We conducted both a CFA and an MCFA, with two levels for individuals and schools. CFA is a type of measurement model that allows us to examine theories (factors) about the patterns of leadership practice in the CALL data (indicators measured through subdomain scores). CFA assumes that the individual responses in CALL, the measurement model, are wholly independent of each other, and therefore can be analyzed at a single organizational level. Though teachers, school leaders, pedagogues, and other personnel participate in CALL as independent individuals, they are also nested in schools. Each school has its own culture, climate and other organizational features that affect a given educator’s experience in the school. MCFA accounts for the nested data by estimating the effects of these leadership practices within the individual level of respondents to CALL and across the range of schools participating in CALL (also described as “between level”).
While both components (within and between) of the MCFA account for the nested nature of the CALL data, the parameter estimates from the within level results provide information about the constructs exclusively. The parameter estimates from the between level results provide information about the constructs in relationship to the specific units (here, schools), in which they are nested or clustered. Given the complexity of public education, scholars are increasingly using structural and measurement techniques like structural equation modeling and CFA to examine relationships across multiple variables like leadership, collaboration and efficacy (Dedrick and Greenbaum, 2010; Geijsel, Sleegers, Leithwood and Jantzi, 2003; Miller, Goddard, Goddard, Larsen and Jacob, 2010). The mean subdomain scores are the indicators for the factors (domains) in CALL. Figure 1 lists the CALL domains and subdomains included in this analysis.
Domain and sub-domains.
Procedures and data analysis
We started with the raw item-level data files for all respondents to the CALL elementary school and CALL–DK surveys. We used SPSS Version 23 to calculate subdomain scores for each respondent to CALL from their corresponding items. These individual level subdomain scores were merged across the two contexts into role-specific files (for teachers and school leaders), to facilitate the two t-tests. We conducted t-tests for teachers’ and school leaders’ mean (average) scores across 12 subdomains for the following CALL domains: Focus on learning; Monitoring teaching and learning; and Building nested learning communities.
Following the independent samples t-test for teachers and leaders by country, we sought to better understand how teachers engage in the leadership practices measured through CALL. We used CFA and MCFA to examine the pattern of leadership practice reported by teachers only in the Danish and US contexts for the leadership practices associated with Domain 1, Focus on Learning; Domain 2, Monitoring Teaching and Learning; and Domain 3, Building Nested Learning Communities.
Results
Findings from the t-test analysis suggest that school leaders in the USA and Denmark perceive their school communities to be engaging in the leadership practices distributed in their respective organizations at a consistently higher rate than teachers report. Given that school leaders’ scores across the two contexts are statistically the same in close to half (5 out of 12) of the subdomain scores measured, it appears that Danish and US school leaders may be identifying and interpreting the leadership practices measured through CALL in a shared and consistent way.
Descriptive statistics
Tables 1 and 2 (and see Appendix 1) provide descriptive statistics, including the number of respondents, mean (average) subdomain score, SD, and, for teachers, the intraclass correlation (ICC), for all participants included in this study. The number of respondents included in the t-test analysis for Danish school leaders taking CALL ranges from 26 to 27; their average subdomain scores range from 2.34 (2.4, Summative Evaluation of Teaching) to 4.02 (1.4, Providing Appropriate Services for All Students). For the US school leaders taking the elementary version of CALL, the number of respondents included in the independent sample t-test analysis ranges from 63 to 81. Their average subdomain scores range from 3.12 (2.3, Formative Evaluation of Teaching) to 4.45 (1.2, Formal Leaders Are Recognized as Instructional Leaders).
Teachers: Descriptive statistics for the subdomains included in this analysis.
***p = < 0.001; **p = < 0.01; *p = < 0.05.
SY: survey year; US: United States; DK: Denmark; SD: standard deviation; ICC: intraclass correlation.
Leaders: Descriptive statistics for the subdomains included in this analysis.
***p = < 0.001; **p = < 0.01; *p = < 0.05.
SY: survey year; US: United States; DK: Denmark; SD: standard deviation; ICC: intraclass correlation.
We examined independent samples t-tests for school leaders and teachers separately across the US and Danish contexts. The number of respondents included in the t-test analysis for Danish teachers taking CALL ranges from 325 to 335, and their average subdomain scores range from 1.42 (2.4, Summative Evaluation of Teaching) to 3.46 (1.4, Providing Appropriate Services for All Students). For the US teachers taking the elementary version of CALL, the number of respondents included in the t-test analysis ranges from 1616 to 1695. Their average subdomain scores range from 2.81 (3.3, Socially Distributed Leadership) to 3.93 (1.2, Formal Leaders Are Recognized as Instructional Leaders).
ICC analysis for school-level clustering. ICCs provide information about the extent to which individual scores are clustered within a unit. Because this data set includes responses from individuals in schools, we conducted an ICC analysis to assess the appropriateness of a two-level factor analysis for this inquiry. Each respondent taking CALL has a unique individual-level identifier as well as a school-level identifier. We used the school codes to identify school-level clusters for teachers only. The results of the ICC analysis listed in Table 1 indicate a sufficient amount of clustering for both versions of CALL, warranting a multilevel analysis to account for nested data. Generally, scholars use 0.1 (Kline, 2011) as an indication of clustering within a data set. Specifically, the ICCs for the US teachers range from 0.11 (2.1, Formative Evaluation of Student Learning and 3.3, Socially Distributed Leadership) to 0.43 (3.4, Coaching and Mentoring). The ICCs for the Danish teachers range from 0.07 (2.1, Formative Evaluation of Student Learning) to 0.31 (3.4, Coaching and Mentoring).
Independent samples t-test: Teachers and leaders—USA and Denmark
With the exception of subdomain 3.2, Professional Learning, teachers’ mean subdomain scores are statistically different across the two international contexts of CALL. Most (10 out of 12) of the subdomain scores are different at p < 0.001 and 1 (subdomain 1.4, Providing Appropriate Services for All Students) is different at p < 0.05. Table 1 includes the mean, SDs and ICC for teachers’ subdomain scores.
Conversely, for school leaders, close to half of the mean subdomain scores (5 out of 12) are not statistically different from each other. Two mean subdomain scores are statistically different at p < 0.05 (subdomain 1.4, Providing Appropriate Services for All Students and subdomain 2.2, Summative Evaluation of Student Learning). Five mean subdomain scores are statistically different at p < 0.001 (subdomains 1.1, Maintaining a School-Wide Focus on Learning; 1.2, Formal Leaders Are Recognized as Instructional Leaders; 2.1, Formative Evaluation of Student Learning; 2.4, Summative Evaluation of Teaching; and 3.1, Collaborative School-Wide Focus on Problems of Teaching). Table 2 includes the mean and SD for school leaders’ subdomain scores.
Model estimation
We used Mplus Version 7.31 (Muthén and Muthén, 1998–2015) to compare and contrast the pattern of leadership practices across the US and Danish versions of the CALL survey. The data used in this study consisted of respondent-level mean subdomain scores from 12 subdomains measured through the items in CALL. The respondent-level mean scores were calculated from the CALL items using SPSS Statistics Version 23. The resulting data files were analyzed in Mplus using the maximum likelihood estimation with robust standard errors to account for missing data.
The results of our initial CFA model (Model 1) suggest a reasonable level of fit. The Danish model is stronger overall than the US model. However, because the CFA model does not account for the clustering of teachers’ responses within their schools, we conducted an MCFA (Model 2) to better understand the patterns of leadership practice teachers experience in their respective contexts. For both models, we used respondents’ individual-level subdomain scores as indicators for the overarching factor at the domain level. Tables 3 and 4 provide information on the fit indices for both the single and multilevel models. The results of both models are included in Appendix 1.
Fit indices for Model 1.
SY: survey year; US: United States; DK: Denmark; CFA: confirmatory factor analysis; χ2: Chi-squared; df: degrees of freedom; CFI: comparative fit index; RMSEA: root mean squared error of approximation; SRMR: standardized root mean squared residual.
Teacher’s responses to CALL—CFA fit indices for Model 1.
CALL: Comprehensive Assessment of Leadership for Learning; CFA: confirmatory factor analysis; SY: survey year; US: United States; DK: Denmark; MCFA: multilevel confirmatory factor analysis; χ2: Chi-squared; df: degrees of freedom; CFI: comparative fit index; RMSEA: root mean squared error of approximation; SRMR: standardized root mean squared residual.
While the overall model fit strengthens for the US survey at Model 2 (where we account for school-level clustering), on the Danish side, the results suggest weaker model fit at the multilevel in comparison to the original single-level model. This may indicate a greater degree of homogeneity across Danish schools currently taking CALL, in comparison to elementary schools in the USA; which, in combination with the smaller sample size, may render the MCFA slightly less helpful in accurately describing the patterns of leadership practice in Denmark through CALL at this time. Overall, the fit indices indicate a reasonable level of model fit for the subdomain scores (indicators) loading onto the domains (factors), thereby allowing us to analyze these models. For Model 2, we used Hu and Bentler’s (1999) recommendation and assessed model fit using the following indices and thresholds: standardized root mean squared residual (SRMR), less than or equal to 0.08; comparative fit index (CFI), greater than or equal to 0.95; and root mean squared error of approximation (RMSEA), less than or equal to 0.06.
Model specification
In this study, we examine, compare, and contrast two three-factor models that reflect the CALL theory of leadership practice for schools in Denmark and the USA. Model 1 represents a single-level CFA for the US elementary and Danish 0–9 versions of CALL. Model 2 is a multilevel model that provides model specifications, or results, at two levels: a within, or individual level comparable to Model 1; and a between level that accounts for school-level characteristics associated with teachers’ responses.
The factors for these models align to the first three domains of CALL: Domain 1, Focus on Learning; Domain 2, Monitoring Teaching and Learning; and Domain 3, Building Nested Learning Communities. Thus, the factors in both models are defined by observed variables only. Because this is an exploratory analysis, the parameter estimates were all freely estimated to the factors. The first estimate for each factor was constrained to 1.00 for the unstandardized estimates. Each of the three factors across both versions of the survey are identified with four indicators through the mean subdomain scores. Both models are therefore identified given that they have more than two factors and each factor has more than two indicators (Kenny, 1979; Kline, 2011).
Model evaluation—multilevel Model 2: Teachers in Denmark and the USA
Although CALL has been validated using the Rasch model, here we assess model fit to determine the degree to which the results of the model provide reliable information. We therefore used the following indices to assess model fit: the chi-squared (χ2) test of model fit, the CFI, the RMSEA, and the SRMR. The χ2 value is a basic measure of model fitness. We chose to examine both the CFI and the RMSEA along with the SRMR to assess model fit. Scholars have yet to reach consensus on which fit indices to use (Kline, 2011).
The χ2 value for the two-level three-factor MCFA Model 2 for teachers responding to the US elementary version of CALL, χ2 (102, N = 1696) = 621.494, p < 0.000, indicated a reasonable amount of fit. The SRMR of 0.032 (within or individual level) and 0.101 (between-level) and the RMSEA of 0.055 are less than the cutoff values recommended by Hu and Bentler (1999) of 0.08 and 0.06, respectively, for the within level of the SRMR. The χ2 value for the two-level three-factor MCFA Model 2 for teachers responding to the Danish 0–9 version of CALL, χ2 (102, N = 335) = 356.039, p < 0.000, indicated a reasonable amount of fit. The SRMR of 0.046 (within or individual level) and 0.168 (between level) and the RMSEA of 0.086 indicate a reasonable amount of fit given that the within level of the SRMR is less than the recommended cutoff value (Hu and Bentler, 1999). These statistics indicate a reasonable degree of model fit and thereby allow us to examine the results from the indicators loading onto the factors, or parameter estimations, for the Danish and US versions of Model 2 in this analysis. Tables 2.1 and 2.2 contain the fit indices for the models.
Convergent and discriminant validity. Convergent validity is a measure of how well a given indicator explains the factor onto which it is loading. Each indicator assigned to a factor should ideally load onto that factor at 0.70 or higher. Discriminant validity is a measure of how well a factor in a given model is measuring a portion of the model distinctly from the other factors. In order to assert that any two factors are distinct from each other, correlation estimates for factors in a model should not be greater than 0.90. We also consider the error terms before summarizing findings for each factor.
Factor 1: Focus on Learning—Model 2: Parameter estimates and measures of validity
The parameter estimates for indicators at the within or individual level, among US teachers taking the elementary version of CALL for Factor 1, Focus on Learning, have a different pattern of factors than the within level for Danish teachers on Factor 1. The Factor-1 loadings for the parameter estimates of the within level for the US elementary survey teachers range from 0.46 (indicator 1.4, Providing Appropriate Services for Students Who Traditionally Struggle) to 0.77 (1.1, Maintaining a School-Wide Focus on Learning). The mean score for subdomain 1.1, Maintaining a School-Wide Focus on Learning, was the strongest indicator for Factor 1, loading onto the factor at 0.77 for the within level from US teachers responding to the elementary survey. For Factor 1 on the Danish 0–9 survey, teachers’ factor loadings for the parameter estimates range from 0.30 (indicator 1.4, Providing Appropriate Services for Students Who Traditionally Struggle) to 0.72 (1.3, Collaborative Design of Integrated Learning Plan). For Danish teachers responding to the 0–9 survey, the mean score for subdomain 1.3, Collaborative Design of Integrated Learning Plan, is the strongest indicator for the Danish version of CALL at the individual or within level.
The parameter estimates for indicators at the between level, across schools with US teachers’ responses to the elementary version of CALL for Factor 1, Focus on Learning, have a different pattern of factors than the within level for Danish teachers on Factor 1. However, Danish teachers’ responses have the same pattern of factor loadings across both portions of Model 2 (single and between levels). The US elementary teachers’ Factor-1 loadings for the between level parameter estimates range from 0.54 (indicator 1.4, Providing Appropriate Services for Students Who Traditionally Struggle) to 0.83 (1.2, Formal Leaders are Recognized as Instructional Leaders). The mean score for subdomain 1.2, Formal Leaders Are Recognized as Instructional Leaders, was the strongest indicator for Factor 1, loading onto the factor at 0.83 for the between level from US teachers responding to the elementary survey. For Danish teachers’ responses to Factor 1, the factor loadings for the between-level parameter estimates range from 0.75 (indicator 1.4, Providing Appropriate Services for Students Who Traditionally Struggle) to 0.99 (1.3, Collaborative Design of Integrated Learning Plan). Danish teachers’ mean score for subdomain 1.3, Collaborative Design of Integrated Learning Plan, is the strongest indicator for the Danish version of CALL at the between level, which accounts for school-level characteristics.
The within-level factor loadings for Factor 1, Focus on Learning, for both US and Danish teachers responding to CALL, indicate low convergent validity because only one indicator, Maintaining a School-Wide Focus on Learning (1.1), for the US survey and Collaborative Design of Integrated Learning Plan (1.3), for the Danish survey has factor loadings greater than 0.70. Further, the residual variance for US and Danish teachers at the within level for Factor 1, Focus on Learning, yields results that are greater than 0.70 for the two weakest indicators in each survey: Collaborative Design of Integrated Learning Plan (1.3) and Providing Appropriate Services for Students who Traditionally Struggle (1.4) for teachers in the US; and Formal Leaders are Recognized as Instructional Leaders (1.2) and Providing Appropriate Services for Students who Traditionally Struggle (1.4) for teachers in Denmark. These variance estimates are statistically significant at p < 0.001.
However, when we examine the between-level portion of Model 2, which accounts for school-specific characteristics, we see a high level of construct validity for Factor 1, Focus on Learning in Danish teachers’ responses, whereas their pattern of factor loadings remains the same as that of the within level. All indicators measuring or loading onto Factor 1, for Focus on Learning at the between level for Danish teachers, had factor loadings ranging from 0.75 (1.4) to 0.99 (1.3). This suggests that when school-level organizational characteristics are taken into consideration, the leadership practices measured through Domain 1, Focus on Learning, may be what distinguishes one school from another in the Danish context of schooling. For the US teachers, the between-level portion of Model 2 yields stronger convergent validity with two indicators loading onto Factor 1: 1.2, Formal Leaders are Recognized as Instructional Leaders (0.83) and 1.1, Maintaining a School-Wide Focus on Learning (0.81). The parameter estimates for all indicators measuring the three-factor Model 2 are reported in Appendix 2.
Factor 2: Monitoring teaching and learning—Model 2: Parameter estimates and validity
For both levels of Model 2 (“within” and “between”), the parameter estimates for all indicators for US teachers taking the elementary version of CALL load onto Factor 2, Monitoring Teaching and Learning. These factor loadings facilitate a better understanding of the leadership practices associated with the summative and formative assessments of student learning and teaching practice. The parameter estimates for US elementary school teachers’ responses suggest that schools engaging deeply in these practices prioritize the formative assessment of student learning (0.91) and the formative evaluation of teaching practice (0.89) over summative evaluations for student learning (0.87) and teaching practice (0.85).
US and Danish teachers
For Factor 2, Monitoring Teaching and Learning, on the US elementary survey, teachers’ factor loadings for the parameter estimates of the within level range from 0.71 (2.2, Summative Evaluation of Student Learning) to 0.77 (2.4, Summative Evaluation of Teaching). At the within level for US teachers, the mean score for subdomain 2.4, Summative Evaluation of Teaching, was the strongest indicator for Factor 2. This suggests that teachers consistently report engaging in summative evaluation practices like a formal observation at least once or twice a year, which are typically mandated by local policy or provided for in union contracts.
When we examine these factor loadings for Danish teachers, we see different results that reflect the relatively recent emergence of some of these practices mandated through changing Danish education policy. The parameter estimates for indicators at the within level, among Danish teachers taking CALL, range from 0.51 (2.4, Summative Evaluation of Teaching) and 0.70 (2.3, Formative Evaluation of Teaching). For Danish teachers responding to the 0–9 survey, the mean score for subdomain 2.3, Formative Evaluation of Teaching, is the strongest indicator at both the within level and the between level, which suggests that teachers and school leaders in Denmark are engaging in these leadership practices that distinguish schools from each other in their leadership work. The Factor-2 loadings for Danish teachers’ between-level parameter estimates range from 0.45 (2.1, Formative Evaluation of Student Learning), to 1.02 (2.3, Formative Evaluation of Teaching).
The factor loadings for the US indicators for Monitoring Teaching and Learning (Factor 2) indicate a high amount of convergent validity on the US side of Model 2. This suggests that each indicator (or subdomain) is measuring a distinct construct that informs Factor 2 (Monitoring Teaching and Learning). This is evident in the low residual variance, less than 0.30 at the between level, on the US survey. When we consider convergent validity for Danish teachers’ responses to CALL, it is important to remember that the indicators measuring constructs in Factor 2 are measuring emerging practices recently mandated by Danish policy. The indicators measuring the Formative (2.3) and Summative Evaluation (2.4) of teaching at the within and between levels loaded onto Factor 2 (Monitoring Teaching and Learning).
Factor 3: Building nested learning communities—Model 2: Parameter estimates and validity
When we compare the Danish and US versions of CALL at teachers’ individual level responses in Model 2, we find that teachers report the same pattern of leadership practices for the work of building nested learning communities. In addition, at the within level of Model 2, the same two indicators load onto Factor 3 in both the Danish and US contexts of schooling. Yet, this similarity falls away when we account for school-specific organizational features for both contexts at the between level of the model.
The parameter estimates for indicators at the within or individual levels among US teachers taking the elementary version of CALL for Factor 3, Building Nested Learning Communities, have the same pattern of factor loadings at the within level as Danish teachers on Factor 3. US elementary teachers’ Factor-3 loadings at the within-level range from 0.46 (3.4, Coaching and Mentoring) to 0.79 (3.1, Collaborative School-Wide Focus on Problems of Teaching). On the Danish survey, Factor-3 loadings for teachers range from 0.60 to 0.77 for the same indicators as their counterparts in the USA. At the individual level of Model 2 for Factor 3, Building Nested Learning Communities, the mean score for 3.1, Collaborative School-Wide Focus on Problems of Teaching, was the strongest indicator—suggesting that teachers recognize these practices as central to the work of maintaining a professional learning community.
However, the pattern of leadership practices at the between level for Danish and US schools suggests that teachers in their respective contexts engage in these practices differently across schools. Danish teachers’ factor loadings for the between-level parameter estimates range from 0.91 (3.4, Coaching and Mentoring) to 0.97 (3.3, Socially Distributed Leadership). For teachers in Denmark responding to the 0–9 CALL survey, the mean score for subdomain 3.3, Socially Distributed Leadership, is the strongest indicator for the Danish version of CALL when we control for school-specific characteristics across schools. The factor loadings for US elementary teachers responding to CALL for Factor 3 range from 0.60 (3.4, Coaching and Mentoring) to 0.92 (3.2, Professional Learning). Thus, the average score for the leadership practices associated with Professional Learning was the strongest indicator for Factor 3, looking across elementary schools taking CALL.
Convergent validity
For both models, Factor 3, Building Nested Learning Communities, has stronger predictive ability on the Danish survey than on the US survey. The parameter estimates for Factor 3, Building Nested Learning Communities, at the within level, for both versions of CALL, indicate some convergent validity because two indicators, Collaborative School-Wide Focus on Problems of Teaching (3.1), and Professional Learning (3.2) for both the US and Danish surveys have factor loadings greater than 0.70. Further, when we look at the factor loadings for Factor 3 in Model 1, we see that a third indicator loads on the Danish survey, Coaching and Mentoring (3.4).
When we examine the between-level portion of Model 2, we see high construct validity for Factor 3 in Danish teachers’ responses. All four indicators load onto Factor 3 for Danish teachers. For the US survey only three indicators load onto Factor 3 at the between level: Collaborative School-Wide Focus on Problems of Teaching (3.1), Professional Learning (3.2), and Socially Distributed Leadership (3.3). This suggests that when we consider school-level organizational characteristics, the leadership practices measured through Domain 3, Building Nested Learning Communities, especially for schools in Denmark, may be what distinguishes one school from another.
Discriminant validity for all factors
Discriminant validity allows research scholars to assess a given instrument’s ability to measure distinct factors that are related to other constructs in their model. The results from Model 2 show that Factor 1, Focus on Learning, is correlated with Factor 2, Monitoring Teaching and Learning, at the within level for both the US and Danish versions of CALL. However, when we examine schools at the between level, and thereby account for school-level characteristics, this correlation between Focus on Learning (Factor 1) and Monitoring Teaching and Learning (Factor 2) decreases, particularly for Danish schools where the association between Factor 1 (Focus on Learning) and Factor 2 (Monitoring Teaching and Learning) is 0.89.
The leadership practices measured through Domain 1 are associated with the leadership practices that pertain to Domain 3 at the within and between levels for the US version of CALL, suggesting that leadership practices associated with formal leadership have a meaningful role to play in establishing, building, and maintaining nested learning communities in the USA. However, when we examine the Danish data, we find almost the opposite: no correlation at the within level, with high correlation across schools. This suggests that all schools engage in these practices at some level in Denmark, and the exceptional work requires support from formal school leadership practices. This may reflect the lack of a long-lasting practice in Denmark of formal school leaders engaging in the leadership of teams among the staff. These teams have been strongly self-governing. Further, these teams have primarily worked as an organizational unit of coordination rather than as a professional learning community.
The discriminant validity results for Domain 2 and Domain 3 indicate a weaker association with each other than with Domain 1. Particularly at the between level, these are distinct groups of practices that happen uniquely across different types of school, for both national contexts. So, monitoring teaching and learning and building nested learning communities are distinct groups of practice that require, or are correlated with, the work of formal school leadership practices. The individual level results for teachers in the USA and Denmark demonstrate this clearly. While we still see this association on the US version of CALL when we control for school-level characteristics (0.907 for Monitoring Teaching and Learning, Factor 2 and Focus on Learning, Factor 1), we do not for Danish teachers, which points to the emergent nature of practices associated with monitoring teaching and learning in the Danish context of schooling. Further, the discriminant validity statistics for Building Nested Learning Communities, Domain 3, underscore what we see through the convergent validity statistics. Particularly for Danish teachers, these are distinct practices measured through CALL. While they require the work of formal leadership practices at both levels (across teachers and schools) in the USA, they happen independently in Danish schools across all teachers, yet require support from formal leaders at higher levels of practice.
Discussion
Through this analysis, we found that the leadership practices that align to a school-wide focus on learning are closely associated with the work of monitoring teaching and learning and building nested learning communities particularly in context of schooling in the USA. Further, we found that whereas school leaders across both national contexts may have a unique level of access to external policy messages that inform their perception of leadership practices, teachers do not. Teachers’ responses to CALL may provide an opportunity to understand how particular policy-initiated reforms may be interpreted, translated, and transformed before they reach the classroom and impact practice.
School leaders occupy a liminal position, as translators and mediators, between the world of practice embodied through the teaching, leadership, and learning that happens in their school, and the policy actors that operate outside of, yet impact, schools through policy messages. US school leaders receive, interpret, and implement policy from local school district governing bodies, as well as the state and federal levels. In Denmark, the school leadership team is typically responsible for receiving, interpreting, and implementing policy from their local municipality as well as state administrators. Because education policies increasingly reflect mimetic isomorphism (DiMaggio and Powell, 1983) at the cross-national level, school leaders in their respective nations may be responding to similar education policies across contexts. Mimetic isomorphism is one of three processes described by DiMaggio and Powell (1983) whereby organizations in a particular sector or field adopt or adapt behavioral practices for conducting work. In addition, mimetic isomorphism may involve incorporating specific structural positions within an organization to demonstrate and thereby maintain a certain amount of professionalism and legitimacy with similar organizations in a given sector or field (DiMaggio and Powell, 1983).
Teachers’ responses to CALL in Denmark and the USA present a nuanced picture of distributed leadership practice. When we contrast school leaders’ responses to CALL to that of teachers in the same schools, we see a consistent and markedly different response, both in the rate at which teachers perceive these practices occurring, and the level of congruence reported across the two contexts. Perhaps because teachers typically are removed by at least one organizational degree from the direct interpretation of these policies, they are less able to recognize them in the practices occurring in their schools. The concept of coupling is helpful here. Teachers may remain loosely coupled from the external policy environment while school leaders may simultaneously experience tighter coupling—at least in comparison to teachers—given leaders’ closer relationship to the external policy environment that seeks a closer relationship with schools.
Our analysis suggests that US teachers may be engaging in practices that have been shaped by external policy to a greater degree than their counterparts in Denmark. For example, one of the goals of recent education policy reform in Denmark is to increase the amount of testing and evaluation that students and teachers experience of their learning and practice, respectively. In CALL, these leadership practices are measured through four subdomains that fall under Monitoring Teaching and Learning (Domain 2). The US results for these subdomains are consistently higher at the individual level than that of their counterparts in Denmark. However, we do see Danish teachers engaging in these emerging practices, mandated through Danish policy, particularly at the between level of the multilevel analysis. Conversely, we find that Danish teachers may have a deeper and more nuanced tradition of distributed leadership practices, with democratic governance and decision-making, than do US teachers.
Comparing and contrasting leadership practice
We find that the tasks and practices associated with a focus on learning are correlated with monitoring teaching and learning and building nested learning communities, for Danish and US schools. In the Danish context of schooling, we see a strong set of practices associated with the work of Building Nested Communities measured through Factor 3. This factor, Building Nested Learning Communities, is not highly correlated with the other two factors in either of the models in the Danish context of schooling.
Further, we find that mandated education policy in the US context, has with increased time, had a greater level of integration with US leadership practice than recent Danish policies have had in Danish schools. This corresponds with an institutionalist perspective and the conceptual logic of translation and transformation. Incremental change through translation takes time and requires active work of transformation of both teachers and leaders. However, our findings here are nuanced. When we consider responses from school leaders across contexts, we see agreement about the degree to which they are engaging in the same leadership practices, while teachers disagree on the level of engagement.
Research, policy, and practice
This study focuses on school-level practice in response to external policy environments in Denmark and the USA. Comparative studies should examine the context of local practitioners who are on the receiving end of policies they did not necessarily craft or call for. Comparative analyses, be they normative or critical (Steiner-Khamsi, 2006), are helpful in examining the policy landscape in a given sector for a set of countries, both to compare practices and to examine the process that results from the implementation of a given policy. In addition, it may be helpful to use comparative analysis to examine the work educators engage in as part of their daily process, in response to a context involving implementing external policies resembling policies that other nations are pursuing in their education reform work. Policy development and implementation is meant to impact teaching, learning, and leadership in schools. Studying the local response of school-based practitioners serves to highlight the amount of time it may take for an initiative to become imbedded in practice in a given national context of education.
When top-down, rational reform is implemented with greater testing and accountability, and originating (perceived or otherwise) from external entities at the federal level, it may not go well at the level of school-level practice and may be rejected by the local community—as is the case with the Common Core in several US states (Kornhaber, Barkauskas, Griffith, Sausner and Mahfouz, 2017). However, as Kornhaber et al. (2017) point out, the initial implementation process of the Common Core in the US state of California was largely a success because it resulted in a decrease in accountability and an increase of resources. In addition, in California it started with meaningful engagement with local actors.
This suggests that difficulties with initially implementing Common Core were less about the content and more about the approach to implementation that each state took. The results of this study point to the importance of capacity-building, particularly when governments are attempting to enact a set of policies with what could be described as a “top-down” approach from the federal level, to the state and or municipal levels, into schools and classrooms (Fullan, 2005, 2009). In their study of the relationship between professional development and school capacity, Newmann, King and Youngs (2000) advance a conceptual framework that describes the school-level organizational features that comprise school capacity and the external factors, which, together with school capacity, impact student achievement. In descending order, they posit that student achievement is effected by instructional quality, school capacity, and external policies and programs (Newmann et al., 2000). Further, they hold that school capacity is a multi-faceted construct or entity in schools that is the result of school leadership directly impacting the synergistic relationship among four central dimensions: (a) the professional community among teachers and staff; (b) their instructional competency related to teaching and learning; (c) the degree to which the school’s programs for student and teacher learning are coherent, aligned, and integrated into the broader school organization; and (c) the quality of technical resources such as curriculum, books, laptops, software, classroom and laboratory space to support instruction and thereby augment student achievement (Newmann et al., 2000). More recent scholarship on capacity building has examined the ways in which capacity building can advance efforts toward change in schools (Malen et al., 2015), including the role of the department chair in advancing instructional capacity (Klar, 2012); cultural aspects of schools such as trust (Cosner, 2009) and overall school capacity (King and Bouchard, 2011); and the role of the formal school leader (Youngs and King, 2002).
Limitations
This analysis has two limitations. Although our sample size was large enough to conduct the MCFA using teachers’ responses only, we did not have enough schools to examine these data for school leaders separately. Given the level of agreement between school leaders in Denmark and the USA, it would be helpful to see in greater detail the relationship these practices have with each other across national contexts from a leadership perspective. In future studies, it might be instructive to load the specific items onto the factors defined by the CALL domains. The subdomain scores are a slightly removed measure of leadership practice.
Implications and conclusion
This study has implications for distributed leadership—the framework that undergirds CALL—and policy interpretation and implementation. School leaders must consider the sense-making that teachers will have to engage in for their practice to meaningfully change; and, engage in that learning themselves, so they can scaffold and model it for their teachers (Shaked and Schecter, 2017; Stein and Coburn, 2008). Improved leadership practice in this area will reduce the risk of detrimental gaps in shared understanding between teachers and leaders.
School leaders need to create the organizational structures (time, resources, etc.) that foster meaningful communities of practice. Further, they have to provide instructional leadership on what should be occurring within these communities of practice. School leaders are an integral link between external policy changes and messages, and the leadership required for implementation inside schools.
This study also points to the work required to implement a specific type of policy. Standardized testing mandates that come from external entities such as local, regional, and federal levels of government may be easier to implement than changing teachers’ or leaders’ practices around more complex practices such as coaching or professional learning (Kruse and Johnson, 2017). Standardized tools, by their very nature, are transformed less as they pass through the translation process. This suggests that some kinds of organizational change can be implemented without the need to change practice in a meaningful way while still adhering to policy. However, meaningful change in schools will require a deeper level of change in practice.
Footnotes
Acknowledgements
Carolyn Kelley has an ownership interest in Leadership for Learning, Inc., which owns the copyright for the CALL survey.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Appendix 1
| CALL - US teachers elementary school survey | CALL - DK teachers 0-9 school survey | |||||||
|---|---|---|---|---|---|---|---|---|
| Model 1 - CALL teachers US and DK comparison | Confirmatory factor analysis (CFA: single-level) | Confirmatory factor analysis (CFA: single-level) | ||||||
| Focus on learning – factor 1 | Monitoring teaching and learning – factor 2 | Building nested learning communities – factor 3 | Residual variance | Focus on learning – factor 1 | Monitoring teaching and learning – factor 2 | Building nested learning communities – factor 3 | Residual variance | |
| UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | |||
| Focus on learning – Factor 1 | ||||||||
| Maintaining a school-wide focus on learning (1.1) | 1.000 (0.000) 0.770 | 0.407 (0.019) | 1.000 (0.000) 0.716 | 0.488 (0.046) | ||||
| Formal leaders are recognized as instructional leaders (1.2) | 0.836 (0.030) 0.672 | 0.549 (0.021) | 0.798 (0.081) 0.587 | 0.656 (0.047) | ||||
| Collaborative design of integrated learning plan (1.3) | 0.733 (0.038) 0.481 | 0.769 (0.019) | 1.188 (0.089) 0.802 | 0.357 (0.044) | ||||
| Providing appropriate services for students who traditionally struggle (1.4) | 0.520 (0.028) 0.467 | 0.782 (0.019) | 0.497 (0.082) 0.363 | 0.868 (0.038) | ||||
| Monitoring Teaching and Learning – Factor 2 | ||||||||
| Formative evaluation of student learning (2.1) | 1.000 (0.000) 0.757 | 0.427 (0.019) | 1.000 (0.000) 0.547 | 0.700 (0.049) | ||||
| Summative evaluation of student learning (2.2) | 1.042 (0.034) 0.743 | 0.448 (0.019) | 1.005 (0.127) 0.578 | 0.666 (0.049) | ||||
| Formative evaluation of teaching (2.3) | 1.271 (0.043) 0.767 | 0.412 (0.019) | 1.146 (0.128) 0.761 | 0.421 (0.050) | ||||
| Summative evaluation of teaching (2.4) | 1.180 (0.037) 0.788 | 0.379 (0.018) | 0.946 (0.122) 0.591 | 0.650 (0.051) | ||||
| Building nested learning communities – Factor 3 | ||||||||
| Collaborative school-wide focus on problems of teaching (3.1) | 1.000 (0.000) 0.779 | 0.393 (0.018) | 1.000 (0.000) 0.806 | 0.351 (0.040) | ||||
| Professional learning (3.2) | 1.131 (0.033) 0.813 | 0.338 (0.018) | 1.294 (0.091) 0.763 | 0.419 (0.043) | ||||
| Socially distributed leadership (3.3) | 0.900 (0.038) 0.590 | 0.652 (0.021) | 0.837 (0.067) 0.674 | 0.546 (0.047) | ||||
| Coaching and mentoring (3.4) | 0.870 (0.049) 0.458 | 0.790 (0.020) | 1.183 (0.091) 0.703 | 0.506 (0.046) | ||||
CALL: Comprehensive Assessment of Leadership for Learning; US: United States; DK: Denmark; CFA: confirmatory factor analysis; UnStd: beta; S.E.: standard error.
Appendix 2
| CALL - US teachers elementary school survey | CALL - DK teachers 0-9 school survey | |||||||
|---|---|---|---|---|---|---|---|---|
| Model 2 - CALL teachers US and DK comparison | Within level | confirmatory factor analysis (CFA: single-level) | Within level | confirmatory factor analysis (CFA: single-level) | ||||||
| Focus on learning – factor 1 | Monitoring teaching and learning – factor 2 | Building nested learning communities – factor 3 | Residual variance | Focus on learning – factor 1 | Monitoring teaching and learning – factor 2 | Building nested learning communities – factor 3 | Residual variance | |
| UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | |||
| Focus on learning – Factor 1 | ||||||||
| Maintaining a school-wide focus on learning (1.1) | 1.000 (0.000) 0.767 | 0.412 (0.024) | 1.000 (0.000) 0.639 | 0.592 (0.050) | ||||
| Formal leaders are recognized as instructional leaders (1.2) | 0.776 (0.038) 0.636 | 0.596 (0.032) | 0.850 (0.127) 0.527 | 0.722 (0.036) | ||||
| Collaborative design of integrated learning plan (1.3) | 0.703 (0.041) 0.467 | 0.782 (0.023) | 1.165 (0.057) 0.722 | 0.479 (0.061) | ||||
| Providing appropriate services for students who traditionally struggle (1.4) | 0.502 (0.030) 0.461 | 0.787 (0.022) | 0.501 (0.104) 0.304 | 0.908 (0.036) | ||||
| Monitoring teaching and learning – Factor 2 | ||||||||
| Formative evaluation of student learning (2.1) | 1.000 (0.000) 0.740 | 0.453 (0.026) | 1.000 (0.000) 0.563 | 0.683 (0.048) | ||||
| Summative evaluation of student learning (2.2) | 0.960 (0.037) 0.709 | 0.497 (0.026) | 1.019 (0.132) 0.605 | 0.634 (0.047) | ||||
| Formative evaluation of teaching (2.3) | 1.221 (0.052) 0.738 | 0.456 (0.021) | 0.946 (0.095) 0.698 | 0.512 (0.042) | ||||
| Summative evaluation of teaching (2.4) | 1.067 (0.035) 0.769 | 0.409 (0.020) | 0.736 (0.171) 0.510 | 0.740 (0.075) | ||||
| Building nested learning communities – Factor 3 | ||||||||
| Collaborative school-wide focus on problems of teaching (3.1) | 1.000 (0.000) 0.789 | 0.377 (0.023) | 1.000 (0.000) 0.774 | 0.401 (0.081) | ||||
| Professional learning (3.2) | 1.028 (0.033) 0.788 | 0.379 (0.027) | 1.279 (0.136) 0.716 | 0.487 (0.044) | ||||
| Socially distributed leadership (3.3) | 0.817 (0.048) 0.539 | 0.710 (0.027) | 0.880 (0.108) 0.640 | 0.591 (0.047) | ||||
| Coaching and mentoring (3.4) | 0.699 (0.054) 0.460 | 0.789 (0.029) | 0.961 (0.087) 0.598 | 0.643 (0.044) | ||||
| CALL - US teachers elementary school survey | CALL - DK teachers 0-9 school survey | |||||||
| Between level | multilevel confirmatory factor analysis (MCFA) | Between level | multilevel confirmatory factor analysis (MCFA) | |||||||
| Focus on learning – factor 1 | Monitoring teaching and learning – factor 2 | Building nested learning communities – factor 3 | Residual variance | Focus on learning – factor 1 | Monitoring teaching and learning – factor 2 | Building nested learning communities – factor 3 | Residual variance | |
| UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | UnStd. | S.E. | Stand. | |||
| Focus on learning – Factor 1 | ||||||||
| Maintaining a school-wide focus on learning (1.1) | 1.000 (0.000) 0.813 | 0.338 (0.126) | 1.000 (0.000) 0.966 | 0.067 (0.087) | ||||
| Formal leaders are recognized as instructional leaders (1.2) | 1.144 (0.253) 0.826 | 0.317 (0.102) | 0.700 (0.187) 0.920 | 0.153 (0.280) | ||||
| Collaborative design of integrated learning plan (1.3) | 0.960 (0.384) 0.565 | 0.680 (0.210) | 1.211 (0.162) 0.987 | 0.027 (0.127) | ||||
| Providing appropriate services for students who traditionally struggle (1.4) | 0.698 (0.334) 0.537 | 0.712 (0.230) | 0.512 (0.148) 0.754 | 0.432 (0.190) | ||||
| Monitoring teaching and learning – Factor 2 | ||||||||
| Formative evaluation of student learning (2.1) | 1.000 (0.000) 0.908 | 0.176 (0.119) | 1.000 (0.000) 0.453 | 0.795 (0.196) | ||||
| Summative evaluation of student learning (2.2) | 1.392 (0.142) 0.869 | 0.245 (0.126) | 1.450 (1.123) 0.636 | 0.596 (0.440) | ||||
| Formative evaluation of teaching (2.3) | 1.472 (0.382) 0.887 | 0.213 (0.112) | 3.558 (2.116) 1.017 | -0.034 (999.000) | ||||
| Summative evaluation of teaching (2.4) | 1.667 (0.431) 0.851 | 0.276 (0.103) | 3.533 (1.888) 0.967 | 0.064 (0.072) | ||||
| Building nested learning communities – Factor 3 | ||||||||
| Collaborative school-wide focus on problems of teaching (3.1) | 1.000 (0.000) 0.773 | 0.402 (0.167) | 1.000 (0.000) 0.964 | 0.072 (0.100) | ||||
| Professional learning (3.2) | 1.759 (0.473) 0.922 | 0.151 (0.098) | 1.197 (0.258) 0.928 | 0.138 (0.124) | ||||
| Socially distributed leadership (3.3) | 1.395 (0.297) 0.918 | 0.158 (0.101) | 0.653 (0.120) 0.970 | 0.059 (1.115) | ||||
| Coaching and mentoring (3.4) | 2.363 (1.192) 0.598 | 0.643 (0.175) | 1.509 (0.589) 0.909 | 0.174 (0.164) | ||||
CALL: Comprehensive Assessment of Leadership for Learning; US: United States; DK: Denmark; CFA: confirmatory factor analysis; UnStd: beta; SE: standard error.