Measures of music teacher expertise in three early career levels in America

Abstract

The purpose of this study was to observe the teaching of five early career music teachers in three early career levels (preservice, student teaching, and novice teaching) to determine if teaching expertise changes over time. A secondary purpose was to find if any specific teacher behaviors associated with effectiveness were more prominent when teaching their peers in pre-service, another educator’s students in student teaching, and their own students in novice teaching. Data consisted of rehearsal frames and instructional targets identified from teaching video transcriptions along with summative evaluations of the videos by expert teachers. Corroborative findings indicated a lack of specificity across all levels which improved somewhat during student teaching and novice teaching. Directives were the most frequently observed teacher verbal category and information and demonstrations were the highest ranked items across all levels in the summative evaluation. However, instructional directive scores were inconsistent. Teacher modeling doubled during novice teaching. Many of the evaluators commented on the overuse of rote teaching, concurrent performance model, and lack of competency-based education techniques. Implications for higher education training including self awareness and accountability are discussed.

Keywords

Early career expertise novice rehearsal frames teacher effectiveness

Researchers have emphasized that skill acquisition in early career educators is paramount to developing expertise (Chingos & Peterson, 2011; Marsh, 2007; Rivkin et al., 2005). The concept of expertise has been studied by reviewing individual career levels (Hester, 2013; Patterson, 2009; Schleuter, 1991; Worthy, 2005; Worthy & Thompson, 2009) and comparing early career and experts (Bergee, 2005; Goolsby, 1999; Pike, 2014). No quantitative studies to track growth amongst the three early career levels of the same participants were found after an exhaustive search of the literature. Therefore, this preliminary study outlined the teaching of five music teachers in three early career teaching levels (preservice, student, and novice) using video recordings to determine if teaching expertise changed over time. A secondary purpose of this study was to determine if any identifiable teacher behaviors associated with expertise were represented with higher frequency counts in one or more of the early career levels. I examined how participants utilized instructional targets during peer-teaching, student teaching, and novice teaching settings. I also identified duration and frequencies of student and teacher behaviors in the three levels and also identified trends within a rating scale related to problematic novice teacher behaviors. Through the lens of descriptive statistics, I reviewed teacher behavior trends and the role of expertise development strategies utilized in teacher training programs. Additionally, possible differences between early career levels and their implications for teacher education form part of the examination.

Expertise perceptions

Hopefully, early career educators have preservice coursework and experiences such as student teaching to place them on a trajectory of self-reflection and growth in expertise and effective practice. Findings about early career growth in expertise are mixed (Murnane & Phillips, 1981). Classroom experience may not be the sole means of efficacy in that some success may be attributed to alternate experience outside of the field of traditional education (Murnane & Phillips, 1981; Siedentop & Eldar, 1989). Numerous researchers and authorities question if expertise and effectiveness should be equated with experience (Chingos & Peterson, 2011; Marsh, 2007; Rivkin et al., 2005; Siedentop & Eldar, 1989).

Dreyfus and Dreyfus (1980) conceived a development theory in which individuals undergo a five-stage process of expertise: (a) novice, (b) advanced beginner, (c) competent, (d) proficient, and (e) expert. Berliner (1988) transferred Dreyfus and Dreyfus’ theory to teaching expertise. According to Berliner, the most significant growth period is the first 5 years of teaching. Other studies indicated validity in Berliner’s theory (Chingos & Peterson, 2011; Marsh, 2007; Rivkin et al., 2005).

Early career teaching behaviors

Berliner’s theory revealed differences between teaching procedures of novices and experts. Student teachers and novice teachers spent more time using superfluous teacher talk than expert teachers (Goolsby, 1999; Pike, 2014). Bergee’s (2005) study of three levels of conductors, Pike’s (2014) case study of teachers in an elementary class piano setting, and Schleuter’s (1991) case study of elementary general music student teachers found that novice teachers focused on surface aspects (i.e. rhythm) over other performance aspects and were more preoccupied on student enjoyment than cognitive process. Schleuter (1991) and Hester (2013) found that instructional sequencing and assessment were weaknesses in elementary general music student teachers as well.

Goolsby (1999) observed that in-service novice band directors started and stopped more often without providing feedback while experts spent most of their time on performance and addressing interpretive elements. Similar results can be found in the study of preservice and expert teachers giving lessons to adult beginners on a wind instrument (Henninger et al., 2006). Cavitt’s (2003) study of expert middle school and high school band directors indicated higher frequency of nuanced learning targets with error detection planned prior to rehearsals and persistence in correction of problematic sections. Teacher talk and modeling durations were brief but modeling occurred frequently and negative feedback was utilized in a non-threatening and business-like manner. In an investigation of three levels of conductors, Bergee (2005) determined that the intermediate conductor focused less on superficial learning and conducted more artistically. Novice teachers in Goolsby’s (1997) study spent most of the time tuning individual instruments whereas student teachers spent much of their time focused on adjusting pitch accuracy. Of particular interest was a finding in which the greatest gains were in student teachers whose lessons moved from the focus of one-dimensional knowledge to working on interpretative areas as well (Goolsby, 1997). Additionally, the amount of sequential instruction tripled in student teachers over time. Similar findings were indicated in Biddlecombe’s (2012) study related to training in feedback specificity in choral ensembles with graduate-level and in-service conductors. However, Schleuter (1991) indicated the opposite in skills transition from preservice to student teaching in the elementary general music setting. Therefore, some studies indicated that early career teachers can increase in expertise which strengthens the argument that effective teaching improvements begin early in many careers (Chingos & Peterson, 2011; Marsh, 2007; Rivkin et al., 2005).

If the highest gains of effectiveness seem to occur in the first 5 years of in-service teaching (Berliner, 1988), then additional exploration is necessary to discover and track specific behaviors during preservice teaching and into the first 5 years. Specific behaviors which are problematic for early career teachers include pacing, use of non-specific feedback, superfluous teacher talk, and either no modeling or heavy modeling related to rote teaching or concurrent performance (Bergee, 2005; Goolsby, 1999; Henninger et al., 2006; Pike, 2014; Schleuter, 1991).

Using videos to assess teacher behavior and identify rehearsal frames and instructional targets has been well-documented and can provide more information about the development of expertise (Bergee, 2005; Bonds, 2015; Cavitt, 2003; Derby, 2001; Goolsby, 1997, 1999; Orman, 2002; Patterson, 2009; Taylor, 2006; Worthy, 2005; Worthy & Thompson, 2009). Utilizing video as a main source of analysis for measurement from a longitudinal perspective are currently lacking in extant literature. I posit that an approach utilizing a summative evaluation form and in-depth video analysis to collect data will provide richer opportunities for triangulation.

Terminology, purpose, and research questions

The purpose of this preliminary study was to observe the teaching of five music teachers in three early career teaching levels (preservice, student, and novice) using video recordings to determine if teaching expertise changed over time. The secondary purpose of this study was to determine if any identifiable teacher behaviors associated with expertise were represented with higher frequency counts in one or more of the early career levels. Table 1 provides pertinent terminology and definitions related to the study.

Table 1.

Useful terminology and definitions.

Terminology	Definition
Preservice teacher	An aspiring teacher who is currently attending a college/university in preparation for in-service teaching. Teaching experiences offered through the college/university may include peer teaching and field teaching.
Student teacher	A preservice teacher who is currently teaching in the field under a supervising teacher. Their capacity, responsibility, and involvement may be limited, depending on the situation.
Novice teacher	An in-service teacher who is credentialed by the state in which they teach but who has less than 5 years of teaching experience.
Rehearsal frames	A sequence in instruction that begins when a teacher explicitly or implicitly identifies one or more problematic aspects of a performance which needs improvement (instructional targets). This is followed by isolated or decontextualized performances of the musical segment and could include verbal instruction, modeling, approximations, and repetitions. A rehearsal frame concludes with a re-contextualized performance of the musical segment.
Instructional target	An objective that guides learning.

The following research questions guided the study:

What are the frequencies of instructional target categories in rehearsal frames of single performance trials for the three early-career levels?

What are the frequencies and durations of specified teacher and student behaviors observed in rehearsal frames with two or more performance trials for the three early-career levels?

What are the ratings of teaching effectiveness at each level of teaching?

Method

Participants

Participants were five novice teachers who earned an undergraduate degree in music education from the same large university in the Southern region of the United States between 2016 and 2017. Participants were selected using (a) convenience sampling, (b) the number of years they had taught, (c) longevity at the same school, and (d) recommendations of university faculty who were familiar with a participant’s work. All participants were traditional students and were in their mid-20s. Four participants had taught for 2 years and one of the participants had taught for 3 years when this study was conducted. Because the purpose of the study was to identify trends of early career teaching expertise, I selected participants from a broad range of teaching levels. I selected a small sample size, as this was a preliminary study. Additionally, numerous studies examining expertise have utilized small sample sizes successfully (Bergee, 2005; Patterson, 2009; Pike, 2014; Schleuter, 1991).

Upon approval from the International Review Board (IRB), I gathered permissions from the participants. I provided participants with a recruitment letter and consent form and each participant was coded using letters to ensure anonymity. Table 2 provides further information regarding the participants’ preservice focus area and teaching level.

Table 2.

Participant information related to focus and teaching level.

Participant letter	Preservice focus	Student teaching	Novice teaching
A	Elementary (K-5)	Rural intermediate (Gr. 5–6)	Rural primary (Gr. K-2)
B	Elementary (K-5)	Rural middle grades (Gr. 3–4)	Suburban elementary (Gr. K-5)
C	Band (Gr. 6–12)	Rural middle (Gr. 6–8)	Suburban middle (Gr. 6–8)
D	Band (Gr. 6–12)	Suburban middle (Gr. 6–8)	Suburban middle (Gr. 6–8)
E	Choir (Gr. 6–12)	Suburban middle (Gr. 6–8)	Suburban middle (Gr. 6–8)

Procedure

As a part of their degree requirements, all participants took a capstone course prior to the semester of student teaching. One segment of the course was a video recorded 3- to 4-week lab peer teaching mini-unit. All teaching segments were approximately 10 minutes in length. The purpose of the mini-unit was for students to plan, execute, and self-evaluate their rehearsal segments. The videos observed for the study were the first and last in the mini-unit. Teachers A, B, and E taught a three- to four-part choral selection and teachers C and D taught a selection for three-part recorder ensemble. I had access to these videos through an archive from the professor who taught this course.

In student teaching, participants recorded themselves as a part of their cumulative seminar portfolio. Because the videos were recorded independently on varying types of equipment, there were a variety of video camera positions in relation to the student teachers and thereby making some teacher behaviors difficult to decipher (i.e. eye contact or facial expression). Additionally, lengths were not standardized. I used participants’ first video early in the semester and their last video late in the semester for analysis. I had access to these videos via their university files kept by the music department.

I subsequently visited all participants twice during their second or third year of employment as a teacher and video recorded four entire class periods. I asked participants if I could record the grade level, class, or ensemble they felt the most comfortable teaching. Class periods ranged from 30 minutes to an hour. The recordings focused primarily on the teacher; therefore, no parental or student consent were necessary. I used a Zoom Q2n Handy Video Recorder to record all teaching episodes. The first observation occurred in late August of 2018. The second set of observations occurred between November of 2018 and January of 2019. I recorded each participant over a 3-day period. Due to the nature of their jobs, I observed Participants A and B teaching three different grade levels during the same time span, as they were elementary general music specialists. Participants C, D, and E taught at secondary schools and I video recorded them teaching the same grade level over a 3-day consecutive period.

Data collection and analysis—summative evaluation form

Analysis of the video recordings included a summative evaluation form (available in Supplemental Figure 1) called the Teacher Effectiveness Evaluation Form (TEEF).

I used a truncated version of Hamann and Baker’s Survey of Teaching Effectiveness (see Hamann et al., 1998) as the original Survey of Teaching Effectiveness (STE) includes a much wider range of teaching behaviors. Areas on the STE form unsuitable included posture, eye contact, gestures, conducting, and lesson planning.

I recruited six observers to evaluate the participant’s videos. The criteria for selection included (a) convenience sampling, (b) teaching expertise area (band, choir, general music), (c) a minimum of 8 years’ teaching experience, (d) showed signs of rigorous programs (high ratings at contests, award-winning programs, etc.), and (e) advanced degrees and certifications in the area of expertise (i.e., Orff/Kodály levels, advanced degrees in music education, etc.). All observers had experience working in teacher training either as a university supervisor or cooperating teacher as well. Observers responded to six items using a 5-point Likert scale with 1 marked as poor and 5 marked as excellent with anchor definitions. Those items were information and demonstrations, musical model, flow (related to pacing), instructional directives, feedback, and teaching style. Observers also gave an overall rating of the lesson and could provide written comments. While the nature of comments did not relate to my research questions, I believed it necessary to include the option for observers to provide further clarity. While the comments were not coded or necessarily examined with scrutiny, some of those comments are included in the discussion section. Many observer comments provided more detail regarding their TEEF ratings. Each observer evaluated one participant’s videos (N = 8) and one video from each level of another participant’s videos for reliability purposes. I assigned participants to observers with similar teaching expertise so as to provide a deeper understanding of pedagogy observed. Observers had no prior contact and were unfamiliar with participants.

Observers trained prior to watching videos and using the TEEF. During training, they were provided basic background information regarding the participants they observed (numbers of years taught and information regarding teaching situation at each level as provided in Table 1) and background information about each class setting (grade level or ensemble). Training included in-depth information, definitions, and examples regarding each item on the TEEF and were given the form for review and given additional time for questions regarding each item. Observers watched the videos in chronological order over a 1-month period. While randomizing may have provided objectivity, I purposely decided to have observers watch the videos in this order so that the observers would get a better understanding of the participants and could receive a clearer snapshot of where participants started and where they ended. Inter-rater reliability (agreements/[agreements + disagreements]) was 83%.

Data collection and analysis—rehearsal frames

Next, I analyzed the teaching segments to identify rehearsal frames and their instructional targets using the outlines discussed by Duke (1994). I identified and categorized instructional targets for each rehearsal frame (targets available in Supplemental Table 1). I adapted instructional targets for elementary settings from observational studies (Hendel, 1995; Orman, 2002; Price & Hardin, 1990), as only one study (Taylor, 2006) had been performed using rehearsal frames as a tool for analysis in elementary general music classrooms. Taylor’s (2006) study of upper-elementary Orff ensembles utilized rehearsal frames with success.

Rehearsal frames are appropriate for analysis in elementary general music classes in that the refinement of musical skills and performance which is evidenced by using rehearsal frames, can and should be demonstrated in all classrooms, regardless of levels. While the wording of “rehearsal frame” and “performance trial” may be questionable in an elementary general music setting, this verbiage will continue to be used throughout this manuscript with the understanding that a rehearsal frame is simply a refined teaching sequence. However, certain classes or ensembles of any level may have more or less rehearsal frames depending on the objectives to be accomplished in the given time period.

I adapted instructional target categories for choir from Derby (2001) and Patterson’s (2009) studies and adapted instructional target categories for instrumental settings from the Cavitt (2003) and Worthy and Thompson’s (2009) studies. Two instructional targets worth mentioning outside of Supplemental Table 1 are unidentified targets and multiple targets. Cavitt (2003) defined an unidentified target as when “the teacher directs the ensemble to repeat a single passage of music without verbalizing any specific directives or feedback” (p. 221) and multiple targets as “several aspects of performance in a passage which may be addressed simultaneously” (p. 221). I analyzed rehearsal frames with two or more student performance trials using the software program SCRIBE 5 beta version for observation and assessment (Duke, 2019) by observing teacher and student behaviors specified in previous research for duration and frequency counts (Cavitt, 2003; Derby, 2001; Taylor, 2006).

I identified teacher talk and modeling as the primary teacher behaviors for duration measures. Verbal behaviors, a subset for teacher talk, were categorized for frequency using directives, information, questions, specific positive feedback, specific negative feedback, general positive feedback, general negative feedback, and off-task talking. Teacher modeling, a subset for modeling, were categorized for frequency using positive modeling, negative modeling, and concurrent performance model. Student duration behaviors included whole group performance, small group performance, individual performance, on-task student talk, and off-task student talk. Operational definitions for behaviors can be found in Supplemental Table 2. A second researcher provided reliability for 25% of the rehearsal frames selected for further analysis by independently identifying and categorizing the instructional targets in rehearsal frames with two or more student performance trials. Inter-rater reliability (agreements/[agreements + disagreements]) was 80%.

Results

I obtained video recordings of five early career teachers in an undergraduate peer teaching setting, student teaching experience, and second/third year of teaching to determine changes in teaching behaviors and instructional time use among the three levels of expertise using rehearsal frames analysis and a summative evaluation form. The data will be discussed in relation to the research questions provided.

Research question #1—What are the frequencies of instructional target categories in rehearsal frames of single performance trials for the three early-career levels?

I transcribed all video recordings (N = 40) for the purpose of identifying instructional targets and rehearsal frames. Analysis of video transcripts identified 302 rehearsal frames, 244 of which concluded after a single performance trial. Table 3 reports frequency counts of instructional targets related to the 244 single performance trials. The most frequently observed targets in peer teaching single performance trials were pitch accuracy (43%) and unidentified target (26%) with a steep decline in percentages and frequencies after this. The most frequently observed targets in student teaching single performance trials were pitch accuracy (26%), rhythmic accuracy (16%), reading/notating (13%), and multiple targets (11%). The highest observed targets in novice teaching single performance trial rehearsal frames were pitch accuracy (27%), rhythmic accuracy (22%), and technical facility (14%). Technical facility is defined rather broadly in relation to elementary and secondary settings, but is related to items such as fingering agility in rapid passages, percussion sticking technique, vocal agility, and other aspects of performance related to motor skills. Observed percentages declined sharply after these three instructional targets.

Table 3.

Frequency and percentage of instructional targets for rehearsal frames with single performance trials (N = 244).

Target category	Peer teaching		Student teaching		Novice teaching
Target category	Frequency	%	Frequency	%	Frequency	%
Articulation	1	2	2	2	0	0
Breathing/airflow	1	2	0	0	0	0
Blend/balance	0	0	0	0	0	0
Diction/pronunciation	0	0	0	0	0	0
Dynamics	2	4	3	4	0	0
Embouchure	0	0	1	2	0	0
Ear training/sightreading	0	0	4	7	0	0
Improvising	0	0	0	0	0	0
Intonation/tone	1	2	0	0	1	1
Movement	0	0	0	0	4	3
Multiple	1	2	6	11	5	4
Other	1	2	0	0	7	5
Pitch accuracy	23	43	13	26	37	27
Posture/instrument carriage	0	0	0	0	3	2
Phrasing/word stress	2	4	0	0	1	1
Presentation	0	0	0	0	0	0
Rhythm accuracy	4	8	9	16	30	22
Reading/notating	0	0	7	13	1	1
Singing	0	0	0	0	6	4
Technical facility	0	0	3	5	19	14
Tempo	1	2	4	7	3	2
Tone quality/vocal placement	2	4	0	0	9	6
Unidentified target	14	26	3	5	10	7

Note. Frequency stands for frequency count.

Question #2—What are the frequencies and durations of specified teacher and student behaviors observed in rehearsal frames with two or more performance trials for the three early-career levels?

I selected 58 rehearsal frames which contained two or more performance trials for subsequent analysis of specified teacher and student behaviors. Table 4 reports frequency counts and percentages of instructional targets. The only observed targets in peer teaching rehearsal frames with two or more performance trials were pitch accuracy (60%) followed by multiple targets (30%) and breathing/airflow (10%). The most frequently observed targets in rehearsal frames with two or more performance trials in student teaching were rhythmic accuracy (43%), multiple targets (29%), and pitch accuracy (14%). The observed targets in novice teaching rehearsal frames with two or more performance trials were multiple targets (44%), followed by pitch accuracy (18%), rhythmic accuracy (15%), tone quality/vocal placement (12%), posture/instrument carriage (6%), and tone quality/vocal placement (6%).

Table 4.

Frequency and percentage of instructional targets for rehearsal frames with two or more performance trials (N = 58) across all levels.

Target category	Peer teaching		Student teaching		Novice teaching
Target category	Frequency	%	Frequency	%	Frequency	%
Articulation	0	0	0	0	0	0
Breathing/airflow	1	10	0	0	0	0
Blend/balance	0	0	0	0	0	0
Diction/pronunciation	0	0	0	0	0	0
Dynamics	0	0	1	7	0	0
Embouchure	0	0	0	0	0	0
Ear training/sightreading	0	0	0	0	0	0
Improvising	0	0	0	0	0	0
Intonation/tone	0	0	0	0	0	0
Movement	0	0	0	0	0	0
Multiple	3	30	4	29	15	44
Other	0	0	0	0	0	0
Pitch accuracy	6	60	2	14	6	18
Posture/instrument carriage	0	0	0	0	2	6
Phrasing/word stress	0	0	0	0	0	0
Presentation	0	0	0	0	0	0
Rhythm accuracy	0	0	6	43	5	15
Reading/notating	0	0	1	7	0	0
Singing	0	0	0	0	0	0
Technical facility	0	0	0	0	2	6
Tempo	0	0	0	0	0	0
Tone quality/vocal placement	0	0	0	0	4	12
Unidentified target	0	0	0	0	0	0

I compared data by level of teaching to reveal alterations in teaching behavior. Table 5 provides information regarding rate per minute (RPM), percentage, duration, and standard deviation. I summed the mean rates per minute and average standard deviation for each item in all rehearsal frames in each experience level. Teacher talk percentages were consistently near 50% regardless of the level. Over the period of study, teacher modeling increased almost by two times in novice teaching with a higher RPM. Student performance data indicated similar percentages in whole group performance across levels. The results percentages and combined RPM of episodes addressing small groups decreased with experience. Individual student performances were not addressed during peer teaching but were addressed at increasingly higher percentages during student teaching to novice teaching. On- and off-task student talking did not occur in peer teaching but did during student teaching and novice teaching.

Table 5.

Comparison of teacher behaviors and student behaviors across experience levels in relation to mean rate per minute, mean percentage, mean duration in seconds, and mean standard deviation.

	Peer teaching				Student teaching				Novice teaching
	RPM	%	Mean dur	Av. SD	RPM	%	Mean dur	Av. SD	RPM	%	Mean dur	Av. SD
Teacher behaviors
Teacher talk	7.21	49.27	00:04	2.20	4.87	51.86	00:07	5.00	6.87	46.18	00:04	3.29
Teacher modeling	2.23	12.84	00:03.5	1.12	1.00	10.96	00:06	1.49	3.45	24.44	00:04	1.56
Student behaviors
Whole group	1.44	21.23	00:06	1.57	1.44	21.57	00:09	2.66	2.00	23.58	00:06	2.29
Small group	2.49	14.94	00:04	0.21	1.00	11.07	00:07	0.86	0.46	06.82	00:02	0.48
Individual	0	0	0	0	0.11	00.01	00:08	0.26	1.32	07.78	00:01	0.71
On-task talking	0	0	0	0	0.73	00.03	00:07	0.29	1.27	04.48	00:01	0.48
Off-task talking	0	0	0	0	1.66	00.13	00:13	0.94	0.10	00.24	00:00:16	0.02

Note. RPM stands for rate per minute, Mean dur stands for mean duration, and Av. SD stands for average standard deviation.

Comparing the three teaching levels in relation to frequency counts for verbal teacher categories also indicated how participants adjusted behaviors with experience. Table 6 provides frequency counts and percentages of teacher verbal categories across experience levels. Directives were the highest observed category (peer teaching, 47%; student teaching, 46%; novice teaching, 38%) throughout all levels. Information statements were least used in student teaching (3%) and similarly utilized in peer and novice teaching. Questions had similarly low frequency in all levels. I observed off-task statements the most in student teaching (9%) but were rarely utilized in peer teaching and novice teaching (1%). Specific positive and negative feedback were lower observed categories among all three levels with a slight increase in each level. Nonspecific positive feedback decreased subtly with experience. Nonspecific negative feedback increased slightly in student teaching but was the lowest percentages among all levels. Positive modeling was one of the most frequently observed categories in peer teaching (15%) and novice teaching (11%). Concurrent performance model increased in student teaching (9%) and incrementally so in novice teaching (14%) and had low frequency in peer teaching (5%).

Table 6.

Frequency and percentage of teacher verbal categories across experience levels.

Behaviors	Peer teaching		Student teaching		Novice teaching
Behaviors	Frequency	%	Frequency	%	Frequency	%
Concurrent performance model	5	5.37	33	9.02	114	13.97
Directives	44	47.31	168	45.90	308	37.75
Information	6	6.45	10	2.73	58	07.11
Negative modeling	0	0.00	0	0.00	21	02.57
Nonspecific negative	1	1.07	12	3.28	13	01.59
Nonspecific positive	9	9.67	30	08.20	61	07.48
Off-task statements	1	1.07	33	09.02	12	01.47
Positive modeling	14	15.05	21	05.74	90	11.03
Questions	5	5.37	27	07.38	70	08.58
Specific negative	4	4.30	23	06.28	43	05.27
Specific positive	4	4.30	9	02.46	26	03.19

Note. Nonspecific positive, nonspecific negative, specific positive, and specific negative indicate types of feedback.

Research question #3—What are the ratings of teaching effectiveness at each level of teaching?

An inspection of TEEF ratings across levels revealed some trends. Observers rated all participants consistently high across levels in information and demonstrations and musical model, receiving mostly a 4 or 5 out of the 5-point Likert scale. Observers also consistently gave low feedback ratings across levels with participants receiving mostly a 2 or 3. Flow, instructional directives, teaching style, and overall rating were inconsistent and did not indicate specific trends. TEEF scores by item can be found in the Supplemental Materials in Tables 3 and 4. While this research question related to ratings only, observers’ comments gave strength and clarity to TEEF findings as well as the other two rehearsal frame related research questions. Therefore, some of the observer’s comments are included in the discussion section to provide further triangulation.

Discussion

Readers should interpret results with consideration due to the limited number of participants and the number of individual observations per participant. In relation to this, participant videos with different classes or portions of the academic year may have resulted in divergent results from the current study. However, reviewing trends across participants and levels can assist teacher trainers in a deeper understanding of preservice teacher skills compared to student and novice teachers and what they are lacking.

All levels evidenced a general lack of specificity as demonstrated in instructional targets and feedback categories. In addition, ratings on the TEEF regarding flow, instructional directives, and teaching style were either low or inconsistent. Instructional targets consistently observed for both rehearsal frames with single performance trials and multiple performances trials included pitch accuracy and rhythmic accuracy. More nuanced targets related to expression, blend/balance, phrasing/word stress, and improvisation were either not observed or observed sparingly.

Specificity improved subtly over time. Unidentified targets were the second most frequently observed instructional target during peer teaching but also one of the least frequently observed during student teaching and novice teaching. An example of this was participants asking their peers, class, or ensemble to repeat a portion of the piece or activity without giving a reason. Nonspecific positive feedback (i.e. good job) was one of the highest verbal categories observed in selected rehearsal frames during peer teaching. Multiple instructional targets were utilized more during rehearsal frames with multiple performance trials. Additionally, tone quality/vocal placement was slightly addressed during novice teaching during selected rehearsal frames which could be considered a more nuanced instructional target. Therefore, perhaps participants in novice teaching were attempting to approximate expertise.

Throughout all levels, participants used specific positive and negative feedback at moderate to low frequencies. This finding is similar to studies which compared expertise levels and those which focused on expert teachers (Goolsby, 1999). These results might suggest that early career teachers may desire to provide positive experiences for their students (hence nonspecific positive feedback), but do not have the experience and practice to utilize constructive and specific positive feedback statements. The summative evaluation results also indicated low use of overall feedback across levels. Evaluators commented on the lack of feedback, ignoring blatant problems, and providing nonspecific positive feedback. Comments from evaluators included, “Lots of praise, but not specific,” “Glaring wrong pitches with tenors and didn’t address it at all,” and “Didn’t address singing voice when kids were shouting during song.” The feedback trait findings are similar to studies focused on novice and ineffective teachers (Bergee, 2005; Bonds, 2015; Goolsby, 1997, 1999; Pike, 2014; Waymire, 2011). Therefore, it is important to require preservice teachers to write lesson plans and objectives which include specificity. Perhaps it is necessary for teacher trainers to encourage an ideal sound model in student’s minds while planning in order to understand the standard of musicianship in each lesson sequence. Guiding students with score analysis and brainstorming on pitfalls regarding selected repertoire would be helpful. Requiring students to give specific feedback statements related to objectives during peer teaching sequences might also assist. Having students observe their own teaching and requiring them to write down their feedback statements and feedback type would also provide clarity. It is also important for teacher trainers to remind preservice teachers that while the subject of music can be quite enjoyable, it is okay to give proper feedback to students so that they can improve and become independent musicians and thinkers. However, even with best laid plans and intentions from teacher trainers, it may be that novice teachers are not developmentally prepared for the next step of specificity (Berliner, 1988).

Directives were the most frequently observed teacher verbal category and information and demonstrations were consistently the highest ranked item across all levels, yet overall instructional directive scores were inconsistent on the TEEF. Evaluators wrote comments such as “No learning objectives or goals were mentioned to the students,” “I’m not sure what the goal of her lesson was,” and “It seems like the teacher is playing big chunks and hoping to find things that were wrong.” The high ratings in information and demonstrations coupled with the inconsistency of instructional directive scores across levels on the TEEF indicate that early career teachers of all levels may have acquired appropriate pedagogical knowledge but have difficulty explaining concepts to students. This aligns with Berliner’s theory of teaching expertise (1988) in novice and advanced beginner stages in which teachers in these levels lack understanding of environment and personal agency. Findings from several studies reinforce the lack of proper communication with students in early career teachers (Bergee, 2005; Goolsby, 1997, 1999; Pike, 2014). Providing preservice teachers learning objectives in lab teaching experiences related to using clear and succinct directives may assist in mindfulness. Perhaps requiring students to write down a list of pithy directives and information statements to include in lesson plans would also assist in awareness.

Flow (related to pacing) and teaching style were either marked low or indicated inconsistency across levels. Lessons observed that received lower ratings indicated lack of understanding of best teaching practices. For example, the majority of a 30-minute lesson by Participant A during novice teaching utilized rhythm sticks with play-along tracks. Participant B took 30 minutes out of a 45-minute lesson during novice teaching to take turns on a staff recognition game that did not connect to literacy or prior learning. They indicated to students that learning the notes on the staff was “good for their brains.” Participant E took an entire instructional period during novice teaching for students to create choral warm-ups.

The amount of modeling observed doubled during novice teaching. Positive modeling was the second most observed teacher category of teacher verbal and modeling behaviors during peer teaching and third in novice teaching. Concurrent performance model was the second most observed teacher category during student teaching and novice teaching. Many of the evaluators commented about the lack of competency-based education techniques during student teaching and novice teaching. Evaluators made statements such as, “Any person without a music degree could have accomplished what Participant A did in this lesson,” “Relied on demo and kids copying her too much—did they know the rhythms or were they just imitating her?,” “Did everything with students—give them more chances to be independent,” and “I don’t have confidence the kids know how to count rhythms. Lots of rote teaching going on.”

These comments are consistent with the observed high use of concurrent modeling in student teaching and novice teaching and may underscore why I did not observe participants using more nuanced instructional targets. This is consistent with the findings of Goolsby (1997, 1999), Waymire’s (2011) study of ineffective directors, and Bonds (2015) study of student teachers. It is also consistent with Henninger et al.’s (2006) study whose novice teachers utilized rote teaching and modeling with successful results. Henninger et al. indicated nuance in teaching styles of novice versus experienced teachers in that perhaps early career teachers desire quick fixes which modeling and rote teaching can provide. However, consistent usage of these techniques requires constant reliance on the teacher and do not provide student independence. Teacher trainers need to explain to preservice teachers that a broad range of teaching styles are necessary for students to experience long-term satisfaction and independence.

Implications for preservice training and early career educators

The resulting inquiry from the research questions indicated that experience can be beneficial in early career levels; however, emphasis should be placed on crafting those early experiences to foster growth. Specific requirements during peer teaching and student teaching may result in lack of growth in the next stage of novice teaching due to the artificial conditions given. Perhaps starting teaching experiences earlier in the teacher preparation process would contribute to growth occurring closer to the start of the professional teaching career. Growth may be easier to determine at larger gaps of time such as the completion of the novice period of teaching (5 years).

Data in peer teaching indicated higher occurrences of unidentified targets and nonspecific positive feedback. I observed no off- or on-task student talk, indicating a limitation of peer teaching settings. Teacher modeling doubled in novice teaching from peer teaching and student teaching. Concurrent performance modeling was the second-most observed teacher behavior in student teaching and novice teaching. More data are needed to fully determine the function of the modeling observed. When observing the novice teachers informally, however, it appeared that participants were either performing habitually with students or were doing so to over-guide students. However, this usage may be due to the overwhelming nature of the novice years of teaching and therefore, novice teachers automatically rely on their greatest skillset which may be their own performance and demonstration.

There are several studies and best practice articles indicating that preservice teachers require experience as part of the process of altering early career teaching behaviors (Bergee, 2005; Murnane & Phillips, 1981). It is essential for those in higher education to be persistent in assisting early career teachers because experience without guidance is less effective. Early career teachers need the tools to become self-aware of problematic behaviors. Multiple studies performed during preservice and student teaching indicate that accountability in time allocation and verbalizations can increase self-awareness of problematic behaviors (Goolsby, 1999; Killian & Dye, 2009; Worthy, 2005).

Novice teachers need expert mentor music teachers to assist with reflective practicum. While any mentor would be able to assist novice teachers, it would be more useful if the mentor teacher was in the same field. School systems need to allow specialty teachers to collaborate and work with each other to become a support system as this would alleviate a problem indicated via informal conversation by two participants. One solution school systems could use to address this issue is by allowing specialty teachers to attend quality professional development in their own field to assist in creating positive learning experiences for students. Districts that provide funding for additional training in particular methodologies may also help early career teachers with continuing to be self-aware and provide opportunities for self-accountability.

Concluding thoughts and considerations for further research

Findings from this study have raised several research possibilities. The sample size for this study was small and replicating the study with a larger pool of participants would provide interesting trends. Collecting data over a longer period would also provide interesting results. Concurrent performance modeling was prevalent in student teaching and novice teaching. Little data has been collected regarding this type of modeling. It would be of interest to observe teachers of various levels in relation to its usage.

In relation to the results in the current study, perhaps a best practice for teacher trainers might be to provide thorough and specific feedback related to expectations regarding problematic behaviors. I argue that teacher trainers also need to use a broad range of materials to assist teachers and not only utilize the traditional methods books for specific content areas. Providing typical method overviews versus specificity may unconsciously create the problematic issues observed in the current study. Perhaps a structured approach with a focus on self-awareness with plenty of practice teaching opportunities can be part of this process.

Supplemental Material

sj-docx-1-ijm-10.1177_02557614221087478 – Supplemental material for Measures of music teacher expertise in three early career levels in America

Supplemental material, sj-docx-1-ijm-10.1177_02557614221087478 for Measures of music teacher expertise in three early career levels in America by Alicia Canterbury in International Journal of Music Education

Footnotes

Declaration of conflicting interests

As mentioned in previous revisions and in my initial cover letter, Dr. Harry E. Price may be familiar with raw data in this study. Other than this conflict, I have no other conflicts of interest to disclose, financial or otherwise.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Alicia Canterbury

Supplemental material

Supplemental material for this article is available online.

References

Bergee

M. J.

(2005). An exploratory comparison of novice, intermediate, and expert orchestral conductors. International Journal of Music Education, 23(1), 23–36. https://doi.org/10.1177/0255761405050928

Berliner

D. C.

(1988). Implications of studies of expertise in pedagogy for teacher education and evaluation. In Pfleiderer

(Ed.), New directions for teacher assessment (pp. 39–68). Educational Testing Service.

Biddlecombe

T. T.

(2012). Increasing feedback specificity in the choral rehearsal: Assessing and enhancing feedback of teacher conductors through analysis and training (publication no. 3539529) [Doctoral dissertation, Florida State University]. ProQuest Dissertations and Theses Global.

Bonds

E. C.

(2015). An analysis of student teachers’ instruction in a beginning band setting (publication no. 3726949) [Doctoral dissertation, University of Mississippi]. ProQuest Dissertations and Theses Global.

Cavitt

M. E.

(2003). A descriptive analysis of error correction in instrumental music rehearsals. Journal of Research in Music Education, 51(3), 218–230. https://doi.org/10.2307/3345375

Chingos

M. M.

Peterson

P. E.

(2011). It’s easier to pick a good teacher than to train one: Familiar and new results on the correlates of teacher effectiveness. Economics of Education Review, 30(3), 449–465. https://doi.org/10.1016/j.econedurev.2010.12.010

Derby

S. E.

(2001). Rehearsal of repertoire in elementary, middle, and high school choirs: How teacher effect change in student performance (publication no. 3008313) [Doctoral dissertation, University of Texas at Austin]. ProQuest Dissertations and Theses Global.

Dreyfus

S. E.

Dreyfus

H. L.

(1980). A five-stage model of the mental activities involved in directed skill acquisition (no. ORC-80-2). University of California Berkeley Operations Research Center. https://vorakl.com/files/dreyfus/skill-acquisition-1980.pdf

Duke

R. A.

(1994). Bringing the art of rehearsing into focus: The rehearsal frame as a model for prescriptive analysis of rehearsal conducting. Journal of Band Research, 30(1), 78–95.

10.

Duke

R. A.

(2019). Scribe 5 for observation and assessment (beta version) [computer software]. University of Texas at Austin Learning & Behavior Resources. https://cml.music.utexas.edu/online-resources/scribe-5

11.

Goolsby

T. W.

(1997). Verbal instruction in instrumental rehearsals: A comparison of three career levels and preservice teachers. Journal of Research in Music Education, 45(1), 21–40. https://doi.org/10.2307/3345463

12.

Goolsby

T. W.

(1999). A comparison of expert and novice music teachers' preparing identical band compositions: An operational replication. Journal of Research in Music Education, 47(2), 174–187. https://doi.org/10.2307/3345722

13.

Hamann

D. L.

Lineburgh

Paul

(1998). Teaching effectiveness and social skill development. Journal of Research in Music Education, 46(1), 87–101. https://doi.org/10.2307/3345762

14.

Hendel

(1995). Behavioral characteristics and instructional patterns of selected music teachers. Journal of Research in Music Education, 43(3), 182–203. https://doi.org/10.2307/3345635

15.

Henninger

J. C.

Flowers

P. J.

Councill

K. H.

(2006). Pedagogical techniques and student outcomes in applied instrumental lessons taught by experienced and pre-service American music teachers. International Journal of Music Education, 24(1), 71–84. https://doi.org/10.1177/0255761406063108

16.

Hester

C. V.

(2013). Cooperating teacher perceptions of music student teacher preparedness for the elementary music classroom (publication no. 1091) [Doctoral dissertation, University of Mississippi]. Electronic Theses and Dissertations.

17.

Killian

J. N.

Dye

K. G.

(2009). Effects of learner-centered activities in preparation of music educators: Finding the teacher within. Journal of Music Teacher Education, 19(1), 9–24. https://doi.org/10.1177/1057083709343904

18.

Marsh

H. W.

(2007). Do university teachers become more effective with experience? A multilevel growth model of students’ evaluations of teaching over 13 years. Journal of Education & Psychology, 99(4), 775–790. https://doi.org/10.1037/0022-0663.99.4.775

19.

Murnane

R. J.

Phillips

B. R.

(1981). Learning by doing, vintage, and selection: Three pieces of the puzzle relating teaching experience and teaching performance. Economics of Education Review, 1(4), 453–465. https://doi.org/10.1016/0272-7757(81)90015-7

20.

Orman

E. K.

(2002). Comparison of the national standards for music education and elementary music specialists' use of class time. Journal of Research in Music Education, 50(2), 155–164. https://doi.org/10.2307/3345819

21.

Patterson

(2009). Analysis of instructional time use and preferred teaching strategies of three highly successful choral directors [Doctoral dissertation, Texas Tech University]. Texas Tech Electronic Theses and Dissertations. http://hdl.handle.net/2346/15567

22.

Pike

P. D.

(2014). The differences between novice and expert group-piano teaching strategies: A case study and comparison of beginning group piano classes. International Journal of Music Education, 32(2), 213–227. https://doi.org/10.1177/0255761413508065

23.

Price

H. E.

Hardin

(1990). Elementary school music teachers' comparative use of classroom time: Teachers who are and are not Orff-Schulwerk certified. Missouri Journal of Research in Music Education, 27, 30–49.

24.

Rivkin

S. G.

Hanushek

E. A.

Kain

J. F.

(2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458. https://doi.org/10.1111/j.1468-0262.2005.00584.x

25.

Schleuter

(1991). Student teachers’ preactive and postactive curricular thinking. Journal of Research in Music Education, 39(1), 46–63. https://doi.org/10.2307/3344608

26.

Siedentop

Eldar

(1989). Expertise, experience, and effectiveness. Journal of Teaching in Physical Education, 8(3), 254–260. https://doi.org/10.1123/jtpe.8.3.254

27.

Taylor

D. M.

(2006). Refining learned repertoire for percussion instruments in an elementary setting. Journal of Research in Music Education, 54(3), 231–243. https://doi.org/10.1177/002242940605400306

28.

Waymire

M. D.

(2011). Behavioral analysis of directors of high-performing versus low-performing high school bands (publication no. 3479034) [Doctoral dissertation, University of Mississippi]. ProQuest Dissertations and Theses Global.

29.

Worthy

M. D.

(2005). The effects of self-evaluation on the timing of teacher and student behaviors in lab rehearsals. Journal of Music Teacher Education, 15(1), 8–14. https://doi.org/10.1177/10570837050150010103

30.

Worthy

M. D.

Thompson

(2009). Observation and analysis of expert teaching in beginning band. Bulletin of the Council for Research in Music Education, 180, 29–41.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB