Abstract
The purpose of this study was to examine how teacher delivery, student engagement, and observation focus influenced preservice teachers’ ratings of teaching effectiveness. Participants (N = 84 preservice teachers) viewed short teaching excerpts of orchestral and choral rehearsals wherein the teacher displayed either high or low teacher delivery, the students displayed high or low student engagement, and the camera was focused on either the teacher or the students. Participants were asked to rate the overall effectiveness of the lesson on a 7-point Likert-type scale. Results indicated that lessons with high teacher delivery and high student engagement, with a view of the teacher, were rated as most effective, and lessons with low teacher delivery and low student engagement, with a view of the students, were rated as least effective. There were significant main effects for observation focus, teacher delivery, and student engagement, and significant two-way interactions between each of the variables.
Keywords
Music teacher educators strive to provide preservice teachers with the necessary tools to become effective in the classroom. However, effective teaching can seem elusive to a preservice teacher because numerous variables affect the quality of any given lesson: teacher delivery, student engagement, effective pacing, feedback, sequencing, instruction, and the ability to assess students both academically and socially in the moment. Frequently preservice teachers are required to video record teaching episodes and are encouraged to reflect on these lessons. With so many important variables, on what should preservice teachers focus? The students? Themselves? Furthermore, third parties who observe these teaching episodes also focus on different things, depending on whether they are looking at the teacher or the students, and they sometimes draw different conclusions about teaching effectiveness accordingly. The purpose of this study was to examine how teacher delivery, student engagement, and observation focus influenced preservice teachers’ perceptions of teaching effectiveness.
There is an abundance of research that has attempted to define effective music teaching or to have educators at various levels share their perceptions of what skills and behaviors are important to successful music teaching, including personal, teaching, and music skills (Miksza, Roeder, & Biggs, 2010; Taebel, 1980; Teachout, 1997). Some of these studies have isolated teacher delivery skills, such as posture, proximity, gestures, facial expression, voice volume and modulation, eye contact (Hamann, Baker, McAllister, & Bauer, 2000; Yarbrough, 1975), teacher intensity (Cassidy, 1990; C. K. Madsen & Geringer, 1989; C. K. Madsen, Standley, & Cassidy, 1989), pacing (Duke, Prickett, & Jellison, 1998; Single, 1990; Younger, 1998), accurate instruction (K. Madsen, 2003), and appropriate feedback (MacLeod & Napoles, 2012; Price, 1989, 1992), while others have focused on assorted teacher traits: social intelligence (Juchniewicz, 2010), the ability to motivate and have a positive rapport with students (Teachout, 1997), and enthusiasm (Miksza et al., 2010).
Consistently, preservice and beginning teachers cite classroom management as a concern when entering the field (Conway, Hansen, Schulz, Stimson, & Wozniak-Reese, 2004; Kelly, 2000; Miksza et al., 2010; Teachout, 1997). In fact, many teachers leave the field within the first few years if they are not successful at maintaining student behavior (C. K. Madsen & Madsen, 1998). Classroom management also affects perceptions of teaching effectiveness. Participants rated teaching excerpts that contained high student off-task behavior as less effective than teaching excerpts with high student on-task behavior (Yarbrough & Henley, 1999; Yarbrough & Madsen, 1998; Yarbrough & Price, 1981). Poor classroom management was the most frequently listed attribute for why teachers were perceived as ineffective in Juchniewicz’s (2010) study. In an attempt to assist undergraduate music majors with behavior challenges, Brand (1977) provided simulated encounters with student behavior issues. Clearly, student engagement is an important element of effective teaching and, perhaps, a minimum precursor for learning and student progress.
There is evidence to suggest that focusing on student behaviors requires a higher level of development than focusing on teacher behaviors. The Fuller and Brown model (Fuller, 1969, 1974; Fuller & Brown, 1975) provides a framework whereby preservice teachers gradually progress from being concerned with the self, to dealing with problem behaviors of students, and then to considering teacher impact through pupil learning. Berg and Miksza (2010) and Miksza and Berg (2013) examined the concerns of preservice music teachers and found that they were congruent with this model. Killian, Dye, and Wayman (2013) also found that the model fit for students before and after their student teaching across a 5-year period. Campbell and Thompson’s (2007) participants, however, rated student impact concerns higher than other concerns and thereby did not adhere to the Fuller and Brown model of teacher development. More research is required to determine how teacher and student concerns affect preservice teachers’ perceptions.
Observation focus has affected perceptions of teaching effectiveness in some contexts but not in others. Duke (1987), K. Madsen and Cassidy (2005), and Prickett and Duke (1992) found that participants directed their attention more toward teacher behavior than toward student behavior when observing and evaluating teaching settings, even when they were specifically told to focus on the students. Prickett and Duke (1992) had 120 music majors and 120 nonmusic education majors observe a videotaped excerpt of a private lesson in violin. They were asked to notice as much as they could about the teacher, the student, and the lesson, in general. The researchers noted that participants tended to focus on the teacher regardless of the instruction. This trend was also present when participants evaluated teacher feedback (Duke & Prickett, 1987). The same video of a violin lesson was presented to nonmusic education majors, who were asked to attend to specific aspects of instruction, with attention on the student, the teacher, or both. Responses pertaining to effectiveness and attitudes were very different depending on the focus of attention, leading researchers to conclude that “there are aspects of perception of teacher/student interactions that may be affected considerably by the observer’s focus of attention” (p. 33). University music majors rated lessons focused on teachers higher than the same lessons focused on students in Yarbrough and Henley’s (1999) study. Duke and Blackman (1991) and K. Madsen and Cassidy (2005), however, reported that differential observation tasks did not affect observers’ evaluations of teachers.
Given the dearth of research studies using student behaviors and achievement as a dependent measure (Duke, 2000) and the general trend for teacher educators to use video cameras focused on the teacher during microteaching in methods classes, it appears that little importance has been placed on students when defining teaching effectiveness. Furthermore, even when preservice (Napoles & MacLeod, 2013) and experienced teachers (MacLeod & Napoles, 2014) recognized that student progress had occurred, teacher delivery was a stronger predictor for both groups of teachers’ perceptions of overall teaching effectiveness. In the present study, we examined whether preservice teachers’ perceptions of teaching effectiveness might be influenced by viewing teaching excerpts of contrasting teacher delivery and student engagement conditions that were focused exclusively on the teacher compared to excerpts that were focused exclusively on the students. The following research questions guided this study:
Method
Participants
Participants (N = 84) were preservice teachers from two large state universities: 34 females and 48 males. Two participants did not classify their gender. There were 17 students with a choral music education emphasis (16 vocalists, 1 pianist) and 67 with an instrumental focus (23 woodwind players, 25 brass players, 13 string players, 4 percussionists, and 2 participants with dual instruments). All were recruited from the current semester’s offerings of music education courses.
Preparing the Stimulus Recording
Two experienced female teachers at a summer music camp (in a different state from those of the participants) were enlisted to participate in the study. One was an orchestra teacher, the other a choral teacher. Both were videotaped teaching a small group of their own students, simulating a sectional rehearsal. Two cameras were used, one with a view of the students and the other with a view of the teacher. The teachers instructed students for approximately 2 to 3 minutes using their regular music, in two ways: using high teacher delivery and low teacher delivery. High teacher delivery was defined according to Hamann et al. (2000)’s definition: frequent eye contact, use of gestures, engaged facial expression, and modulation of voice volume and inflection. Low delivery was defined as the absence of these traits. Both teachers were asked to keep their approval ratios constant at four approvals to one disapproval, regardless of delivery condition.
Students were asked to respond to lessons in two ways, either on- or off-task. On-task was defined as behaving appropriately without talking, looking at the teacher when she or he gave instructions, sitting with excellent seated posture, and focusing on the task at all times. In the off-task condition, students were told to look not at the teacher, turn to their neighbors and play with their hair, take out their cell phones, slouch in their chairs or walk around, and never focus on the director. Students were also asked to make these behaviors inaudible, so as not to unwittingly be heard when the camera was not focused on them. The combination of teacher and student behaviors yielded four conditions: (1) high teacher delivery/on-task students (High/On), (2) high teacher delivery/off-task students (High/Off), (3) low teacher delivery/on-task students (Low/On), and (4) low teacher delivery/off-task students (Low/Off). Since each lesson was videotaped from two perspectives (students and teacher), there were 8 conditions total per teacher, for a grand total of 16 excerpts.
Each rehearsal had several “takes” for each condition, and these were recorded for later review by independent observers, who were asked to choose a single excerpt that best reflected high/low teacher delivery and on-/off-task student behavior while exhibiting all stipulations listed above. Trials not meeting criteria were discarded.
Validity Check/Implementation of the Independent Variable
To determine whether the appropriate teacher delivery (high/low) and student engagement (on-/off-task) conditions had been executed, 12 experienced teachers, unfamiliar with the research study and the teachers in the video, were invited to serve as independent judges. Judges viewed paired comparisons of the students from the mini teaching excerpts and were asked to determine whether on- or off-task behavior had been displayed by circling the appropriate response. Similarly, the expert panel viewed two excerpts of the same teacher (one of high delivery and one of low delivery) in a paired comparison format and was asked to select the version they felt displayed high delivery.
Reliability among the 12 observers—computed using the formula [(agreements)/(total observations)]—was 98% for teacher delivery and 100% for student engagement. This high level of agreement indicated that the teacher delivery and student engagement variables were implemented correctly and could be perceived as intended. A separate pilot study was conducted with three different experienced teachers, and they identified wording in the survey instrument that was confusing and titles for the video that were unclear. All corrections were made on the final video.
There were 16 excerpts and 1 practice example total. We then created two versions of eight excerpts each, since maturation could have potentially served as a threat to internal validity. Eight excerpts, with 24-second transitions, yielded a 21 to 22-minute video. We included excerpts of the orchestra teachers’ low teacher delivery lessons and the chorus teachers’ high teacher delivery lessons in one version, and the reverse in the other, so that participants would not be confused by the extreme change in teacher personality. Purposeful orders were chosen that alternated teachers and observation focus. We then reversed the order in each of the two versions to control for probable order effects.
Design and Procedure
Participants viewed one of the four discs and were given the following instructions: “You will be viewing eight short video clips of music teaching. Please circle the rating that best represents your perception of the following elements of the lesson.” The five indicators were “the pacing of the lesson was effective,” “the instruction was accurate,” “the students made progress,” “the teacher feedback was appropriate,” and “this lesson was effective.” These indicators were chosen based on the research literature’s findings on effective music teaching (cited above); our intention was to simulate a realistic teaching evaluation by including five indicators. However, only the lesson effectiveness rating was included in the analysis. Seven-point Likert-type scales were anchored with the words strongly disagree above the first number and strongly agree above the last number. Following a practice example designed to accustom them to the task, participants were given the opportunity to ask any questions prior to beginning the actual study. None did, so the video continued.
Results
Data for the two individual teachers’ ratings were combined (added) for each of the eight conditions. Descriptive data indicated that lesson effectiveness was rated highest for the high teacher delivery, student on-task, teacher focus condition (M = 5.85, SD = 1.12), and lowest for the low teacher delivery, student off-task, student focus condition (M = 2.61, SD = 1.47). Table 1 lists all means and standard deviations by condition. Three of the four highest ratings occurred when teacher delivery was high, and three of the four lowest ratings occurred when teacher delivery was low.
Means and Standard Deviations of Lesson Effectiveness Ratings by Condition.
To answer our three research questions, we conducted a three-way analysis of variance with repeated measures. There were three within-subjects variables, delivery, engagement, and observation focus, each with two levels. Lesson effectiveness rating was the dependent measure. Results revealed significant main effects for delivery, F(1, 83) = 93.41, p < .001,
The interaction between teacher delivery and observation focus, illustrated in Figure 1, shows greater variation in participant ratings for lesson effectiveness when viewing the teacher than when viewing the students. Similarly, as shown in Figure 2, the interaction between engagement and observation focus shows greater variation in participant ratings for lesson effectiveness when viewing the students than when viewing the teacher. When students were off-task (low student engagement) and participants saw the students only, they tended to rate lessons as less effective than when they saw the teacher only. Figure 3 demonstrates the interaction between delivery and engagement. It can be seen that participants were less differentiated in their responses by engagement levels when the teacher displayed low delivery.

Interaction between teacher delivery and observation focus.

Interaction between engagement and observation focus.

Interaction between teacher delivery and engagement.
The main effect of delivery was significant, as lessons with high delivery (M = 4.91, SD = 1.72) were rated higher than lessons with low delivery (M = 3.22, SD = 1.78). Lessons with high student engagement were also perceived as significantly different from lessons with low student engagement (on-task M = 4.71, SD = 1.86; off-task M = 3.41, SD = 1.80). When viewing the teacher only, lesson effectiveness was rated higher (M = 4.20, SD = 1.98) than when viewing the students (M = 3.93, SD = 1.90), and this difference was significant.
Discussion
The purpose of this study was to examine how teacher delivery, student engagement, and observation focus influenced preservice teachers’ perceptions of teaching effectiveness. There were significant main effects for each of the variables, indicating that participants were influenced by all three of these factors when rating teaching effectiveness. Analysis of mean overall ratings revealed that the two excerpts with high teacher delivery and on-task students were rated highest, and the two excerpts with low teacher delivery and off-task students were rated lowest. These findings are logical and are congruent with the literature indicating that student engagement (Yarbrough & Henley, 1999; Yarbrough & Madsen, 1998; Yarbrough & Price, 1981) and teacher delivery (Hamann et al., 2000; K. Madsen, 2003; MacLeod & Napoles, 2014, Napoles & MacLeod, 2013; Yarbrough, 1975) are important elements of effective teaching.
Perhaps more surprising was the large discrepancy (over one full point) in ratings of teaching excerpts with high teacher delivery and off-task students (M = 4.66, SD = 1.73 for teacher observation focus; M = 3.50, SD = 1.78 for student observation focus). These excerpts were the same, but perceptions were very different depending on where the camera was focused. Ratings of excerpts with low teacher delivery and on-task students were also different (M = 3.40, SD = 1.97 for teacher observation focus, M = 3.97, SD = 1.74 for student observation focus), albeit less markedly so. The main effect of observation focus was evidenced consistently, although the interactions provide a more accurate reflection of the findings. Caution is warranted, however, when interpreting these results because of the limited sample size, the context of only two teachers (where ratings for each teacher were combined), and the potential variability of the teaching effectiveness throughout the lesson.
The first two interactions, presented in Figures 1 and 2, illustrate similar ideas. Observation focus determined which variable influenced participant lesson effectiveness ratings. In other words, when viewing the teacher, the teacher’s behavior (or delivery) influenced lesson effectiveness ratings. Likewise, when viewing the students, the students’ behavior (engagement) influenced lesson effectiveness ratings. The third interaction, between teacher delivery and student engagement, was less pronounced, and the effect size was small
Observation focus significantly affected participants’ perceptions of overall lesson effectiveness. This is consistent with some research (Yarbrough & Henley, 1999) and inconsistent with others (Duke & Blackman, 1991; K. Madsen & Cassidy, 2005). In general, participants rated lessons with a view of the teacher higher (more effective) than lessons with a view of the students. However, this trend was not evidenced in the low teacher delivery, student on-task condition. In these cases, overall, participants rated lessons with a view of the teacher lower (less effective) than those with a view of the students. One possible interpretation is that, without a view of the students, the teacher’s low delivery became the sole focus of participants’ attention, and they based their effectiveness rating on delivery alone. Interestingly, though, this trend did not hold true when the teacher’s delivery was low and the students were off-task. In that example, watching students misbehave had a more deleterious effect on ratings than the low teacher delivery. Delivery and student engagement appear to be intertwined in such a way that one variable carries greater importance than the other in some settings but not in others.
The results of this study carry important implications for preparing preservice music teachers. Many teacher education programs require preservice teachers to video record and reflect on their teaching. Frequently, these videos are focused exclusively on the teacher and do not include a view of the students. Exclusive focus on the teacher may provide an incomplete experience when making evaluations. This practice may inadvertently reinforce preservice teachers’ predisposition (according to Fuller, 1969, 1974; Fuller & Brown, 1975) to focus on elements of self-concerns rather than student impact concerns. When possible, it would seem beneficial to have preservice teachers video record and reflect on the students’ engagement and progress during the lesson, prompting preservice teachers to focus on the students in a variety of settings, including observations, peer teaching, and field experiences. This approach may increase their ability to be student centered, thus more focused on student impact concerns in the classroom.
Teacher educators should continue to spend time helping preservice teachers develop high teacher delivery through observation and self-reflection. Eye contact, voice modulation, proximity, and facial engagement are associated with high teacher delivery and can be observed directly in field observations and video recordings. Preservice teachers should be given the opportunity to practice and reflect on these specific teacher traits associated with teaching effectiveness.
Findings from this study are also important for inservice music educators. Because most public school teachers are regularly observed by administrators outside of the music field, it is likely that administrators will focus on teacher delivery and student engagement during these teaching evaluations. Given the results of this study, both of those variables can strongly affect perceptions of lesson effectiveness. Furthermore, if the students are being observed, ratings may be lower. Because the smallest discrepancies in ratings occurred when both the teacher and student behaviors were congruent (high delivery/on-task or low delivery/off-task), it stands to reason that observation focus will not be as consequential in these contexts as when there is some incongruence in their behaviors. Student off-task behavior will drastically affect perceptions of overall lesson effectiveness, even when the teacher is displaying high delivery, if the focus is on the students.
Focus on student learning and progress is crucial, as is training in classroom management skills to keep students engaged. All these variables are consistently cited as important elements of effective teaching, and teachers will continue to be evaluated using these measures. Research that investigates effective models for teaching delivery skills and classroom management is needed. Teacher preparation practices that promote increased focus on the students may be beneficial when designing assignments in teacher education curricula.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
