Reframing Assessment as Relational: How Do Teachers of Incarcerated Youth Understand Assessment?

Conceptual Framework

The authors draw upon relational theories of pedagogy to understand how teachers conceptualize assessment with incarcerated youth. Relational theories of pedagogy put relationships between teacher and students at the center of the educational enterprise. In doing so, they highlight aspects of assessment that are often obscured by a narrow focus on assessment instruments and technologies.

Crownover and Jones (2018) define relational pedagogy as the “systematic construction of appropriate relationships embedded with the schooling process. Such relationships develop organically through social interactions and deliberate instructional methods” (pp. 18–19). Relational pedagogy is characterized by mutual respect for teacher and student (Bryk & Schneider, 2003), connecting learning to students’ lived experiences (Brownlee & Berthelsen, 2006), and students and teachers jointly constructing knowledge (Magolda, 1996).

Biesta (2004) defines education as the “communicative interaction between the teacher and learner . . . [education] does not exist in any other sense than as a relation and ‘in relation’” (p. 21). For Biesta, the gap in knowledge and skills between student and teacher is never closed but instead affords the necessary conditions for education. Relational theories of pedagogy highlight and problematize components of communication, especially dialogue and listening. Shor and Freire (1987) describe dialogue as “the sealing together of the teacher and the students in the joint act of knowing and re-knowing the object of study” (p. 100). Brookfield and Preskill (2005) prioritize discussion as the primary means to “nurture and promote human growth” (p. 3). Burbules (1993) connects the power of communicative dialogue to the construction of knowledge and creation of unforeseen outcomes: “It is the nature of this dialogic relation to be able to ‘carry away’ its participants, to ‘catch them up’ in an interaction that takes on a force and direction of its own” (p. 21).

Relational pedagogy theorists also attach particular importance to individual’s capacity for listening. Davis (1997) contrasts evaluative listening, in which the teacher is listening to a student to determine if he or she has provided the correct answer, with interpretive listening, in which the teacher is “interested in ‘making sense of the sense they are making’” (p. 365). Schultz (2003, 2009) identifies four components of listening: listening to know particular students; listening to the rhythm and balance of a classroom; listening for the social, cultural, and community contexts of students’ lives; and listening for silence.

Researchers have demonstrated the academic benefits of dialogic teaching for students, including on standardized assessments (Resnick et al., 2015). Relational pedagogy has also been shown to positively affect students’ motivation to learn (Wentzel, 2009) and provide teachers with resources to combat bullying (Crownover & Jones, 2018). Relational approaches have been shown to affect not only students’ classroom behavior but also teachers’ academic expectations for disruptive students. One study demonstrates that professional development for teachers that focused on increasing developmentally sensitive interactions with high school students can counteract teachers’ low expectations for student academic achievement, as well as reduce disruptive behavior (Hafen et al., 2015). A study of middle school students found that an emotional climate characterized by positive teacher–student relationships correlated with higher student grades (Reyes et al., 2012).

Given the prevalence of severe trauma exposure among students within juvenile justice systems (Dierkhising et al., 2013), there is good reason to believe that relational pedagogy is especially important for these students. Establishing relationships of safety and trust with traumatized adolescents should be seen as the foundation for supporting students’ academic learning and cognitive development (Brunzell et al., 2015; Downey, 2007; Morgan et al., 2015).

Although relational theories of pedagogy have been applied to classroom instruction, from early childhood to higher education, they have more rarely been employed to investigate or theorize assessment practices. One form of assessment in which aspects of relational pedagogy are invoked is formative assessment, sometimes called assessment for learning. Although definitions of formative assessment vary, many identify it as a process through which teachers obtain information about the students’ learning that serves to inform subsequent instruction through frequent and targeted feedback to students (Marzano, 2010; Popham, 2011b; Shepard, 2006). Black and Wiliam (2009) describe formative assessment as a “genuinely dialogic process” in language that resonates with definitions of relational thinking above: “formative interaction is contingent. In such a mode, the teacher’s attention is focused on what she or he can learn about the student’s thinking from their response—what Davis (1997) terms ‘interpretive listening’” (p. 12). Class discussion and informal questioning of students are two common forms of formative assessment (Trumbnell & Lash, 2013).

Although formative assessment is increasingly associated with effective instructional practices and academic outcomes for students (Graham et al., 2015; Kingston & Nash, 2011), it is just one component of the larger assessment landscape that affects students, teachers, and schools. Individually administered standardized assessments, in particular, continue to play a significant role in determining the content of the curriculum and the instructional methodologies students will encounter. Despite research demonstrating the educational benefits of relational pedagogy, assessment is typically viewed as separate from teacher–student relationships.

Methods and Data Sources

The study was conducted as a single case study of an educational facility within a juvenile justice residential setting. Below, the authors describe the setting and participants, as well as methods for data collection and analysis.

Methodology and Data Sources

The authors employed a qualitative case study approach to investigate how teachers make sense of assessment concepts and practices and apply their understandings within their instruction. Case study methodology is well suited to the purposes for this research as it focuses on “the particularity and complexity of a single case, coming to understand its activity within important circumstances” (Stake, 1995, p. xi). In this case, the authors sought to understand how educators within the school setting conceptualized assessment in relation to their unique population of students. It was especially important to investigate teachers’ perceptions of challenges to assessment in order to better understand the context for their perceptions.

Case study methodology calls for “experiencing the activity of the case as it occurs in its context and in its particular situation” (Stake, 2006, p. 2). Beginning in December 2016, the authors conducted regular visits to the site, meeting monthly or bimonthly with five teachers and the social worker who works with the team, and less frequently with the principal. Typical of case study methods, the authors collected and analyzed multiple data sources (Merriam, 2009; Stake, 2006; Yin, 2014). These included 13 meetings (typically 60–90 min each) with teachers and the social worker (not audio-recorded but documented in field notes); individual semi-structured interviews (audio-recorded and transcribed) with five teachers, the social worker, and the principal, as well as with the district superintendent and assistant superintendent supervising the school; review of materials related to assessment, curriculum, and instruction; and field notes from all visits to the facility. Table 1 represents participants’ titles, race, years at the current district/school, years of experience, and licensure areas.

OPEN IN VIEWER

Table 1.

Participant Demographic Information.

Participant	Title	Gender	Race	Years serving at the current district/school	Years serving in alternative setting	Years of experience	License
A	Superintendent	Female	Black	2 years	2 years	12 years (administrator)	N/A
B	Assistant superintendent	Male	Black	2 years	2 years	4 years (teacher) and 20 years (administrator)	Elementary school teacher, building administrator, principal, assistant superintendent
C	Principal	Male	Black	2 years	15 years	3 years (teacher) and 15 years (administrator)	Teacher, building administrator, principal, assistant superintendent
D	School social worker	Male	White	17 years	19 years	19 years	Licensed social worker
E	General education teacher	Male	Black	5 years	5 years	3 years	Social Studies (6–12)
F	General education teacher	Male	Black	2 years	2 years	2 years	Language Arts (K–12)
G	General education teacher	Female	Black	18 years	18 years	20 years	Social Studies (K–12)
H	General education teacher	Female	Black	3 months	8 years	15 years	Biology
I	Special education teacher	Male	White	4 months	6 years	9 years	Special Education (K–12) and Mathematics

Note. Years of experience as a teacher includes experiences as a teacher and a substitute teacher.

The researchers independently coded the interviews and field notes from meetings, using deductive codes derived from the interview protocol (Miles et al., 2014). These included prior experiences working with similar student populations; motivation for working in the setting; challenges in working with student population; forms of educational data provided to teachers; use of assessment data in planning and implementing instruction curriculum; resources that would help teachers in supporting incarcerated youth; and others. In addition, inductive codes were derived from the data set (e.g., motivational strategies, uses of teacher questions). The researchers verified their independent coding, then consolidated discrete codes into thematic codes at a higher level of abstraction (Boyatzis, 1998; Nowell et al., 2017). The codes were used to frame the findings summarized below.

Findings

In the sections below, findings are reported from the interviews and meetings with teachers, the social worker, and administrators. The findings begin with the challenges the participants identified; these are categorized as (a) student-level challenges, (b) structural challenges, and (c) assessment-specific challenges. The authors then describe how teachers conceptualized assessment with the incarcerated youth they serve.

Conceptions of Assessment

The interviews and meetings with teachers, the social worker, and administrators revealed considerable coherence in how individual teachers conceptualize assessment within their unique educational setting and with their student population. Teachers viewed assessment as a process or activity that reveals (or should reveal) students’ knowledge and skills. They believed that knowing students’ knowledge and skills was best achieved through dialogue with individual students rather than through standardized assessments.

Discussion and Implications

As the findings reported above demonstrate, educators of incarcerated youth confront substantial challenges. Many are directly related to assessment, the focus of this study. One of these is students’ negative response to taking group-administered (often computerized) tests. This phenomenon poses a threat to fair and valid interpretation of assessment results (American Educational Research Association [AERA], 2014). Specifically, it is categorized as a threat in test context, which means “multiple aspects of the test and testing environment that may affect the performance of an examinee” (p. 55).

Another finding highlights the limitations of the student records and assessment data with which teachers provided on students. These limitations were most pronounced for students with IEPs, which were often outdated, incomplete, or simply did not seem to accurately describe the student’s academic capacities. Teachers were also limited in their own capacities to interpret the assessment data, again, especially data included in IEPs.

Even challenges not explicitly tied to assessment can be seen as affecting the validity of assessments administered within the school. Students’ emotional condition, commonly referred to by teachers, administrators, and others as related to PTSD, made it unlikely to them that students could focus on assessments, indicating another threat to assessment results in test context. Notably, students’ emotional condition also applies to their engagement with instructional generally.

Problems with technology and the requirement for social media filters may impinge upon students’ access to the online learning platform the school has begun using, along with its embedded assessments. As noted, teachers often provided students with hard copies of forms designed to be completed online. Such solutions may undercut the value of using technology to begin with. More significantly, the emphasis on instruction and assessment delivered through online platforms may detract from teachers’ ability to develop trusting relationships with students, which afford opportunity for formative assessment.

These challenges, understandably, contribute to how teachers and other educators conceptualize assessment. Our findings suggest that educators understand assessment as serving the purpose of informing them about students’ present level of skills and knowledge, and thus to guide subsequent instruction. This intersects with a shared belief that talking with students is the primary means of learning about students, both who they are and what they can do academically. The teachers believed that talking to students individually, in particular, asking them questions, was effective in assessing their capacities.

At the same time, teachers expressed skepticism that formal, group-administered assessments used by the school would provide an accurate picture of students’ skills and knowledge. It is common for teachers and other educators to view assessment in opposition to getting to know students. For example, literacy scholar Peter Smagorinsky (2018) writes:

Tests intervene in relationships by warping instruction toward the most reductive, least compelling ways of thinking: picking the best of four multiple choice items in response to a question the test-taker has little interest in answering.

If this negative interaction of tests and relationships is true in more conventional settings, it is likely to be even more pronounced in juvenile justice settings.

Opposing the value of administering standardized assessments with that of developing relationships with students will not contribute to effective assessment practice in the service of students’ learning. For this reason, it is important to take teachers’ skepticism of assessments seriously. For instance, teachers’ observations about the lack of seriousness with which many students take group-administered standardized assessments, including how little time some students take to complete a test, represents teachers’ tacit understanding of test context as a threat to the validity of the assessment. It is likely that teachers recognize other threats to valid assessment that are likely to be particularly prevalent in juvenile justice settings. Teachers also have multiple opportunities to observe gaps between the results of group-administered standardized assessments and students’ actual performances, resulting in skepticism of the value of the assessment results.

Nor are teachers wrong in their valuing of more informal interaction with students as a valid form of assessment. A recent report from the American Institutes for Research, National Technical Assistance Center for the Education of Neglected or Delinquent Children and Youth points out that, in terms of assessing reading comprehension, an effective assessment can consist of simply asking a student to read short passage from leveled text of increasing levels of difficulty, observe how he or she reads, hesitates, and so on, then ask a few comprehension questions (Salinger, 2010). The validity of such an assessment will be enhanced by the quality of the relationship that the teacher develops with the students.

In fact, teacher–student interaction, which is anathema to most standardized assessments, is at the heart of some conceptualizations of formative assessment. Black and Wiliam (2009), in particular, highlight the importance of listening to students. “In a formative mode . . . the teacher’s attention is focused on what she can learn about the student’s thinking from their response—what Davis (1997) terms “interpretive listening’” (p. 12). When assessment is understood relationally, educators are encouraged to apply their skills in developing trusting relationships with students.

With the current emphasis on standardized assessments, most schools in this country administer multiple group-administered assessments. However, it is essential for schools and teachers to consider the validity of these assessment results. This is true everywhere, but especially so in settings such as educational facilities within juvenile correctional systems, in which multiple threats to validity exist. One potential solution would be to employ individually administered standardized assessments, that is, assessments designed to be administered by an educator (e.g., teacher, school psychologist) with an individual student. Such assessments allow teachers to attend to relational dimensions of assessment; in doing so, threats to the validity of the assessment results are minimized. These assessments are typically employed by teachers of special education; for general education teachers to use them effectively would require technical professional development on their administration and analysis of the results. Teachers in our study expressed interests in learning how to use such assessments because they reflected the teachers’ values of developing relationships with students.

As school’s social worker affirmed, teachers and counselors of incarcerated students will benefit from professional development in trauma-informed practice. As we have demonstrated, the goals and practices of relational pedagogy are consistent with the principles of trauma-informed practice. In particular, professional development in assessment should focus on the use of formative assessments that emphasize dialogue with students. This is particularly true as post-traumatic stress symptoms are known to have a negative impact on group-administered standardized assessment performance (Rutkowski et al., 2010), and thus standardized assessments do not provide teachers with a valid picture of student performance.

For many educators, assessments that report numbers (i.e., scores) are considered the most precise, whereas information from other forms of assessments, such as dialogue with individual students, tend to be dismissed (AERA, 2014). Although teachers in the study value discussions with students, they would benefit from developing specific questioning and facilitation strategies that would expand their capacities to use discussion as a valid form of assessment. Professional development should also support teachers in understanding various threats to valid assessment; in doing so, teachers are likely to recognize the significance of the student information they obtain through conversations with students.

It would also be beneficial for teachers to document and share the information they gather through conversations with students with colleagues. Protocols, such as the Descriptive Review processes developed by the Prospect Center (Himley & Carini, 2000), could provide resources for teachers to collectively exploit the information they obtain from and about students to inform instruction and support student learning.

McDonald et al. (2018) use the term “intimate data” to refer to the student performance data that teachers collect themselves. Some of the data are associated with standardized assessment, that is, it is “collected and analyzed in formally specified ways, and often stored in online management systems designed to analyze it” (p. 2). Equally important are the formal classroom assessments, including, quizzes, tests, projects, and so on, and informal classroom assessment, for example, “teacher observation and questioning, or post teaching reflection” (p. 3). For the authors, what is most critical is how teachers use these data—and talk about them with colleagues—to determine what students know or do not know, but also about students’ thinking processes, and students’ misconceptions.

Conclusion

Teachers of incarcerated youth experience significant challenges to providing their students with the educational experiences they need and deserve. More effective assessment practices will not remove or even mitigate all of these challenges; however, they do hold the promise of more efficiently and effectively guiding teachers in instructional approaches that have the best chance to help the students develop academically.

One teacher in the study commented, “I’m not really an assessment guy.” By that, he did not mean to convey that learning about students’ skills, knowledge, and interests is not important, but rather that the group-administered standardized assessments used at the school were less informative than his one-on-one interactions with students. His view was shared by other teachers in the facility.

By viewing relationship-building with students and assessments as separate—and even conflicting—priorities, teachers and administrators disregard important assets they already manifest, that is, their knowledge of students and the relationships they develop with students. When these two priorities are seen as integrated—through the selection of assessment instruments and use of assessment practices that emphasize dialogue and listening, along with professional development related to these—the potential for more authentic and effective assessment, instruction, and student learning is heightened.

McCray et al. (2018), in a study of the implementation of a structured reading program within a study of a juvenile correctional facility, recognized that the success of any such program would be influenced by “the physical environment, administration, instructional personnel, security personnel, and the students’ academic, social, and mental health histories—and the complexity of relationships among these factors” (p. 277). In addition to these factors, the findings from this study suggest that for the implementation of assessment program to be effective with students in such settings, administrators and teachers must recognize and engage the significance and potential of relationships between teachers and students.