Measures of Early Adolescent Development and School Contexts

Creating Conceptual and Methodological Rigor

Most developmental scientists are trained to place great emphasis on methodological and conceptual rigor, ensuring that the measures they use tap into phenomenon in ways that are both comprehensive and precise. Methodological rigor is expected in developmental science, and exemplified in the study designs, measurement tools, and analytic approaches of the articles in this special issue. Equally important, each study presented here is rooted in a strong theoretical and empirical rationale. For example, in their development of a new tool to assess classroom contexts, Frazier et al. (2015) draw from decades of research in the organizational psychology literature on the construct of psychological climate. Kosovich et al. (2014) make the case that their expectancy-value-cost approach to understanding students’ motivation best captures the latest science from the broad and diverse literature on this construct. Taken together, these articles illustrate the ways in which early adolescent learning is embedded in a complex system that includes interactions with teachers and peers both within and beyond the classroom. These measures move us toward a better understanding of the elements of this complex system and the ways in which each element separately or interactively contributes to development.

This conceptual and methodological rigor offers opportunities and challenges alike. Attending to the complexity of school contexts provides new ways to approach the essential work of supporting student learning and development. For example, with tools to help teachers understand their students’ peer networks, teachers may be more likely to engage in grouping practices that promote new peer connections or develop activities to motivate students via existing peer connections. Measures of behaviors in hallways and schoolyards could lead to better out-of-class monitoring and reinforcement, and help create safer and more productive peer and teacher-student interactions across the school. Yet, as detailed below, rigorous science does not always enhance practice.

Finding Relevance for Researchers and Practitioners

There is increased recognition that the types of social, emotional, and behavioral skills and contexts measured in this special issue are relevant to schools. Tooley and Bornfreund (2014) call these “skills for success” and emphasize the importance of developing, testing, and using assessments of these skills in today’s schools. Yet few of these measures are actually being used by practitioners on a regular basis. And, while many of the measures may appear relevant to those trained to view classrooms and schools through a developmental lens, this may not be readily apparent to educators. Below we suggest several ways in which the relevance of developmental science measures can be made more evident to school personnel.

The terminology we use can hinder communication with practitioners. As we reviewed for this special issue, we were struck by the heavy use of jargon. Too often, researchers, including the authors of this introduction, use words that require further explanation, such as homophily, expectancy-value-cost, psychological climate, and teacher sensitivity. These terms are chosen for purposes of definitional precision and relevance to the scientific literature. However, the use of these terms may limit practitioners’ attention to phenomena they would otherwise find interesting. As researchers partner with schools, it will be important to not only communicate these ideas in ways that maintain the precision but also speak more easily to teachers, principals, school counselors, and other school staff.

Another key aspect of relevance is acknowledging that for most schools the primary mission is academic. For researchers who focus on understanding students’ social, emotional, and behavioral experiences in schools, this may be experienced as a barrier. These researchers want school personnel to see the support of the whole child as central to their academic agenda. However, we must acknowledge how far this vision is from the reality of most schools, where staff members experience exceptional pressure to prepare students in core academic subjects. Ideally, we will get better at integrating social, emotional, and behavioral phenomenon into the daily work of schools and their staff (Jones & Bouffard, 2012). Many of the authors in the special issue note this concern and offer ways it might be addressed, such as the suggestion to use homophily measures to inform the creation of peer learning groups (McCormick et al., 2014) or to use information about students’ motivation to tailor instructional approaches (Kosovich et al., 2014). Other authors, such as Coplan et al. (2015) and Cash et al. (2014), present measures relevant to school personnel such as schoolyard and hallway monitors who are primarily responsible for maintaining students’ social, emotional, and physical health.

Measures will be most relevant to schools if they provide feedback on or link to other tools that can guide changes to practice. For example, the CLASS has been used as a framework for understanding teachers’ practice within a video-based coaching model called “MyTeachingPartner” (Allen, Pianta, Gregory, Mikami, & Lun, 2011), as well as within a teacher consultation model for mental health professionals (Cappella et al., 2012). Integral to the success of these approaches is a video library aligned with each of the CLASS dimensions. The video library helps make the CLASS more relevant to teachers by bringing these teaching practices to life; recent evidence suggests that seeing examples of effective teaching may facilitate teachers’ adoption of these practices (Hamre et al., 2012). One can easily image how similar resources might be developed to align with other measures described in this special issue in ways that can enhance their relevance and impact.

One final area of relevance is the extent to which these measures are aligned with broader educational policy goals. Sometimes a simple reframing of research can drive home the policy relevance of measures. For example, Ruzek et al. (2014) frame their article on motivation using the value-added approach commonly applied to standardized test score data as a part of teacher evaluation. This framing helps create an immediate relevance for school practitioners, which might have been missed had the authors used a different analytic technique.

Developing Feasible Approaches to Use

Even when the relevance of measures is apparent, their feasibility during regular school practice may not be. If schools do not have the resources to collect, enter, manage, and analyze data, the measures will not be used. Given the economic realities in many schools and districts, it is incumbent upon researchers to be cognizant of resource needs and work to reduce them, while maintaining the rigor of the measure. The articles in this special issue consider a range of resources needed for these measures, including time, human, technical, and financial.

For example, Kosovich and colleagues (2014) pay acute attention to the time needed to collect and analyze data. Their student-report survey has an extremely short administration time and a strong overlap between observed and latent means, meaning educators can use raw scores rather than latent values, thus simplifying analysis. Similarly, Frazier and colleagues (2015) end with a set of 18 student-report items that create a scientifically sound factor and are feasible to collect. Although there are trade-offs between the number of items (for a survey) or the amount of time in training/observation (for an observation) and the internal reliability of the measure, we suggest that feasibility in practice, and not only scientific rigor, is a critical consideration for developmental scientists working in and with schools.

Beyond time, other resources—such as human, technical, and financial resources—are critical. In their measurement of afterschool program quality, Oh et al. (2014) suggest the need to conduct multiple, short observations of a program, but their analysis of observer effects indicates a single observer may be sufficient. McCormick and colleagues (2014) present an analysis of same- and cross-group friendships in early adolescence. This analysis is conducted using code in Microsoft Excel, which is widely available and requires little technical support or advanced training. Although the authors of these articles do not conduct a cost analysis, financial costs of training, materials, or software must be considered, and efforts should be made to reduce these costs (such as creating online training and no or low-cost materials).

Future Directions

This special issue documents the ways in which developmental scientists can use measurement to better understand young adolescents’ experiences in schools. Students’ motivation, feelings about the classroom, and connections with peers contribute to their engagement across multiple spaces in schools, including classrooms, playgrounds, hallways, and afterschool settings. We also see variability in the quality and nature of these settings in ways that raise concerns about how well some schools are supporting students’ social-emotional and academic development. And, despite some of the limitations noted above, measure developers are paying attention to whether these tools are relevant and useful for educators.

True impact on educational practice, however, requires further refinement of measures and collaboration with practitioners. It is notable that although the work reported here is helping to bridge the gap from research to practice, most of it has started with researchers using grant funds to develop new measures or refine existing ones. Although this work inherently requires partnerships with schools in the piloting of measures, it often falls far short of true collaboration and leads to the issues with relevance and feasibility noted above. An alternative approach is to begin the measurement development process in a researcher-practitioner team. An increasing number of opportunities for such teamwork exist. For example, the California Office to Reform Education (CORE) is a partnership among 10 districts that calls for a reorientation of districts’ work with a much greater emphasis on social-emotional outcomes and the school culture and climate that contribute to these outcomes. The districts are piloting new measures to assess these skills, including growth mind-set, self-efficacy, self-management, and social awareness. Research collaborations with groups such as CORE can help ensure these measures best reflect the latest developmental science while the ongoing piloting in practice settings will provide checks and feedback on relevance and feasibility.

One striking omission in this special issue with great relevance to this task is direct assessment of students’ social, emotional, or behavioral development. Direct assessments are the gold standard in capturing students’ learning in areas such as reading and math, and direct assessments of children’s social and emotional learning are often used in lab-based studies. But issues with relevance and feasibility have often prohibited the use of direct assessments of these skills in schools. Several innovative approaches are currently underway. Clark McKown and colleagues at Rush University are developing a web-based direct assessment of emotional recognition, perspective taking, social problem solving, and self-control, and are piloting the measure with partnering schools and districts throughout Illinois and Virginia (McKown, Allen, Russo-Ponsaran, & Johnson, 2013). The 3C Institute is developing a game-based tool that assesses similar skills as students play a game in which they confront real-world scenarios such as dealing with a bully in the hallway or working with a peer to solve a problem (DeRosier, Craig, & Sanchez, 2012).

Ultimately, the greatest advancements in this work to better align developmental science and educational practice will come as a result of social networks among professionals on both sides of the divide. Educators need to be more deeply engaged, from the beginning, in major research initiatives. Researchers can no longer see schools simply as places to obtain participants. Working together, developmental scientists and school practitioners can define the most important problems of practice and engage in rapid prototyping of measures that may be helpful in addressing these problems. This work will require new approaches to research funding, such as the Institue of Education Science’s Researcher-Practitioner Partnerships in Education Research program. It will also require developmental scientists to learn new methods, such as using design-based research to help increase the transfer of research findings to practice (Anderson & Shattuck, 2012). There are many examples of research-practitioner collaborations, such as the University of Chicago Consortium on Chicago School Research, which has produced important advances in educational practice. The work described in this special issue suggests fertile ground for the establishment of these types of partnerships between scientists and practitioners.

This social networking is not a one-way process where researchers ask school personnel to become more engaged in research. It is equally important for researchers to have more on-the-ground experience in school settings. With funding from IES, our own academic institutions are working to ensure that newly trained education scientists have meaningful experiences in schools and other practice and policy settings relevant to education. For example, doctoral student fellows are required to spend time doing a mentored internship in a local school division or other educational practice or policy organization. There, they get hands-on experience responding to the daily challenges faced by teachers, administrators, and policymakers, and conduct relevant research aimed at a specific, actionable policy or practice question. These experiences can change their perspective on how research contributes to practice.

The research to practice divide is not likely to narrow soon. However, as the articles in this special issue suggest, developmental scientists and education practitioners have much to learn from one another about how to best support students as they transition from childhood to adolescence. Thoughtfully designed measures of early adolescent development and educational contexts have the potential to transform educational practice, particularly if they provide educators with relevant and actionable data in an efficient and reliable way. We must work together to create and refine these measures, develop practical ways to use them, and monitor use to maintain precision and relevance over time. This work has only just begun.