Vocal Sight-Reading Assessment

Abstract

This study investigated choral singers’ comfort level using computer technology for vocal sight-reading assessment. High school choral singers (N = 138) attending a summer music camp completed a computer-based sight-reading assessment and accompanying pre- and posttest surveys on their musical backgrounds and perceptions about technology. A large majority reported both frequent sight-reading instruction (74%) and individual assessment (70%) in their prior instruction. Whereas 94% of these students previously had been assessed live, only five students had been assessed through recording and one by computer. Preassessment opinions (69% favorable/neutral) shifted substantially (69% negative) after experiencing computer-based assessment. Inability to set the tempo was cited as the predominant reason for disliking the technology. Results indicate great potential for using technology in vocal sight-reading instruction and assessment. However, it is important that students’ introduction to it be intentional and thorough. Adequate practice and comfort must be established for students to trust and appreciate this new instructional tool.

Keywords

assessment choral music sight-reading sight-singing SmartMusic technology

The landscape of educational assessment is continually changing. Advancements, particularly in the mode of delivery, have occurred at an accelerated pace as computer access in the schools has also increased. Although “computer-assisted testing” was advocated as early as 1970 (Bunderson, Inouye, & Olsen, 1989, p. 2), its proliferation from that time to the present has made this type of assessment commonplace in many disciplines. It is incumbent on the choral music education field to develop in the assessment arena alongside other disciplines, both in testing content and in delivery system.

Vocal assessment, principally in sight-singing, has typically employed one of two assessment strategies—note-by-note scoring, in which credit is granted for each correct pitch or rhythm (Boyle & Lucas, 1990; Demorest & May, 1995; Henry & Demorest, 1994; Killian, 1991; Lucas, 1994), or evaluation by measure, in which credit is given for each measure in which no errors occur (Bowles, 1971; Larson, 1977; Scott, 1996). Henry (2001, 2009) developed a procedure for vocal sight-reading assessment by categorizing pitch and rhythm skills by difficulty level, then assessing only notes within a melody that are representative examples of each skill. During the development, Henry (2001) compared note-by-note scoring procedures with the targeted skill scoring, achieving a .96 correlation between scoring systems.

In each instance mentioned above, testing and scoring processes were conducted by human test administrator/scorers, either live or through recordings. Although there have been attempts to create computerized vocal assessments (Kolb, 1984; Lorek & Pembrook, 1989; Platte, 1981), none of these assessments gained widespread use, and in some cases they employed hardware that was obsolete by the time the study was concluded (Higgins, 1987). Since the mid-1980s when most of these studies were conducted, little has been achieved in developing or refining voice recognition technology for this type of educational purpose.

Historically, instrumental assessment has preceded developments in the vocal realm. The popularity of the Watkins–Farnum Performance Scale (Watkins & Farnum, 1962) and the Farnum String Scale (Farnum, 1969) lent preference to the measure scoring system in live and recorded instrumental performance assessment for many years. Instrumental instruction has also outstripped vocal instruction in the technological realm. Peters first investigated possibilities in computer-assisted instruction for instrumentalists in 1974. Continued attention to accurate instrument sampling allowed steady development of instrumental music software for education. In particular, the evolution of the web-based SmartMusic program (MakeMusic, 2011) has revolutionized the use of computers for instrumental assessment. Evolved from essentially a computerized accompaniment track program known as Vivace that debuted in 1994, its current form allows assessment for both repertoire performance and sight-reading. Its latest version now includes similar capabilities for vocal sight-reading.

These technological advancements also bring new options for classroom instruction and individual practice. However, vocal students not used to instruction or assessment via computer may be uncomfortable with the procedures or skeptical of the accuracy of computer-based assessment. Converting traditional assessment practices by human evaluators into computer-driven processes may cause the credibility of scores to be challenged, particularly when used for grading or audition purposes, or may result in an increase in performance anxiety.

The purpose of this study was to determine choral students’ comfort level with the use of computer-based voice recognition software for vocal sight-reading assessment. Research questions included the following.

Research Question 1: What is the sight-reading experience of high school choral students with respect to group instruction and individual assessment?

Research Question 2: How accurately do high school choral students assess their individual sight-reading skill?

Research Question 3: What previous experiences do high school choral singers have with computers in vocal or instrumental instruction and assessment?

Research Question 4: What level of anxiety do high school choral singers report for various modes of vocal sight-reading assessment?

Research Question 5: What are the preexposure impressions or expectations for a computer-administered and scored vocal sight-reading assessment?

Research Question 6: Do those expectations change after having experienced a computer-based assessment?

Method

High school choral singers (N = 138) attending a summer music camp completed surveys requesting information on their choral and instrumental background, their experiences with sight-reading instruction and assessment, their experiences with computers in music instruction, and their views on the use of technology for music instruction. On completion of the survey, each participant sang a three-melody individual sight-reading assessment, delivered and scored using a beta version of SmartMusic 2012, as well as a live scorer. Graduate-level and upper-level undergraduate choral music education majors who were on the camp staff served as live scorers. They received training in live sight-reading assessment prior to the start of the camp. Using procedures similar to those employed by Killian and Henry (2005), live scorers were asked to categorize each singer as beginning, intermediate, or advanced, based on their sight-reading performance. Singers completed a posttest survey on their anxiety levels, their perceptions of the computerized assessment, and the likelihood of their using this type of technology in the future.

As a part of the choral camp experience, participants selected two elective courses from the following list: diction, vocal health, audition strategies, majoring in music, conducting, composition, and sight-reading. Those classified as “beginning sight-readers” were required to take the sight-reading class as one of their elective choices. Although not mandatory, 47 participants classified as either “intermediate” or “advanced sight-readers” chose to enroll in a sight-reading class. Separate sections were held for the beginning participants and the intermediate/advanced sight-readers. The beginning sight-reader classes provided information about sight-reading fundamentals and included opportunity for supervised practice and peer teaching.

The intermediate/advanced sight-reader classes contained recommended practice and performance strategies for sight-reading based on findings from Killian and Henry (2005), in which beginning, intermediate, and advanced sight-readers were observed via video recording. These observations resulted in the identification of strategies for both practice and performance associated with successful sight-reading. Likewise, strategies for practice and performance were also identified with less successful sight-reading (see Figure 1 for a summary of these desirable and undesirable strategies). The intermediate/advanced classes also included an overview and tutorial for the SmartMusic program that had been used for the sight-reading assessment. Highlighted features included selection of available sight-reading materials and settings options, including the ability to adjust the tempo, and record keeping options. After receiving a systematic introduction to the software, and the opportunity to interact with the software in an instructional setting, these 47 participants were polled again about their opinion of computer-based sight-reading instruction and assessment.

Figure 1.

Practice and performance strategies given to singers in the intermediate/advanced sight-reading classes.

Results

Each participant (N = 138) sight-read three melodies. The computer scoring system calculated the percentage of pitches and rhythms performed correctly for each participant. Percentage scores ranged from 2% to 100%, with a mean score of 35.81%. In addition to the computer-based score, a live scorer categorized each participant as a beginning, intermediate, or advanced sight-reader, based on his or her success with designated pitch and rhythm skills. Beginning sight-readers (n = 47) obtained a mean computer score of 18.23%, with scores ranging from 2% to 60%. Intermediate sight-readers (n = 46) obtained a mean computer score of 33.09%, with scores ranging from 3% to 86%. Advanced sight-readers (n = 45) obtained a mean computer score of 56.96%, with scores ranging from 8% to 100%. Table 1 contains a demographic breakdown of each ability group, including grade level to be entered in the upcoming school year; selection for region-, area-, and state-level honor choirs; keyboard experience; and other instrumental training.

Table 1.

Demographic Information by Sight-Reading Level.

	Sight-reading level
Variable	Beginning (n = 47)	Intermediate (n = 46)	Advanced (n = 45)
Grade level
9th	9	3	4
10th	12	11	13
11th	5	17	14
12th	21	15	14
Honor choirs
All-region	23	31	38
All-area	3	7	15
All-state	2	3	10
Piano experience	29	27	36
Average years	2.95	3.11	4.33
Instrumental experience	9	15	13
Average years	3.00	3.20	2.31

Note. Because entering ninth graders had not yet had the opportunity to audition for any of the honor choirs reflected in this table, the possible number of honor choir participants was beginning = 38, intermediate = 44, and advanced = 40.

Participants reported the amount of sight-reading instruction they received within their regular choral rehearsals. All 138 participants reported practicing sight-reading during choir rehearsals; 102 participants reported practicing frequently, 33 reported practicing sometimes, and 3 reported practicing rarely. The proportion of participants in each response category was similar among each ability grouping—beginning, intermediate, and advanced. Participants also reported on the format for sight-reading instruction within the choral rehearsal. Large group instruction was by far the most common (n = 125), with small group instruction (n = 43) and peer teaching (n = 16) employed most often in addition to large group practice. No other format for sight-reading instruction during choir was reported.

Of the 138 participants, 97 reported having individual sight-reading assessment as a part of their choir experience, whereas 41 participants reported no individual assessment. A significant difference, χ²(2) = 9.65, p = .0082, was found between beginning, intermediate, and advanced sight-readers, with the higher the ability level, the greater the likelihood of individual assessment. Of the 97 participants who experienced individual sight-reading assessment, 91 identified live performance as a means of assessment, 5 of whom also reported being assessed through recording and 1 of whom also reported assessment with computer. Five participants identified recording as the sole means of assessment. No other means of assessment were identified.

When asked about practicing sight-reading on their own, participants responded as follows: frequently = 13, sometimes = 59, rarely = 45, and never = 21. The proportion of participants between ability groupings was similar for individual practice frequency. Of the participants who practiced sight-reading on their own, 52 reported using the sight-reading materials from their choir, 53 reported using sight-reading books other than those used in choir rehearsal, 6 reported using the computer, 2 replied “the piano,” and two replied “voice teacher.”

Participant responses concerning their opinion of their own sight-reading skill indicated a significant correlation between self-reported ability and the live scorer-assigned ability group, F(1, 136) = 22.11, p < .0001. Mean scores for a 5-point Likert-type rating scale were 3.06 for beginning, 3.30 for intermediate, and 3.80 for advanced groups. Only 10 out of 138 participants reported using SmartMusic software for vocal instruction or for testing. Four of the 37 participants with instrumental experience reported using it with instrumental music.

Participants were asked to complete a posttest survey, soliciting their opinions on computer-based assessment. They reported greater anxiety for computer-based testing than for testing live in front of a teacher (60% vs. 40%) or through recording (59% vs. 41%). Participants reported less anxiety with computer-based testing than for testing live in front of their peers in class in approximately reverse proportion to teacher or recording (42% vs. 58%).

Prior to the sight-reading assessment, participants were asked their opinions about the use of technology for practicing and assessing sight-reading in the choral classroom. Opinions were somewhat evenly distributed, with 51 positively disposed, 44 neutral, and 32 negatively disposed. Eleven participants chose not to respond. Positive responses varied quite a bit but included efficiency, convenience, individual help, creativity, accuracy, less pressure, and a general willingness to embrace new technology. Neutral responses primarily represented those with no experience and therefore no opinion. Negative responses focused primarily on a lack of trust in the accuracy of the technology, wanting a live individual listening to them, being a more difficult task, and being more stressful.

After experiencing computer-based assessment, equal numbers of participants reported having their opinion change and having their opinion confirmed, resulting in a significant shift, χ²(2) = 52.39, p < .0001, in the overall acceptance of computer-based music testing (see Table 2). When asked if they would consider using this program to practice sight-reading, 57 participants responded favorably, 81 participants responded unfavorably. Positive responses included “easy to use,” “makes me better,” “makes me keep a tempo,” “less nervous,” “use it on my own,” “more accurate,” “gives immediate/visual feedback,” “is more challenging,” and “can record my own progress.” Negative responses included “can’t set my own tempo,” “just don’t like it,” “concerned about accuracy,” “nervous/uncomfortable,” “confusing,” “I did bad/usually do better,” “I prefer a teacher,” “I prefer sight-reading books,” “it’s OK for band but not choir,” and “I don’t have access to it.” By far, the most frequent response (n = 43) was the inability to set their own tempo as the reason they did not like the program. Nine participants cited the enforced tempo as a positive.

Table 2.

Pre- and Posttest Opinions About Computer-Based Testing.

	Pretesting opinion	Posttesting opinion
Favorable	51	22
Neutral	44	21
Unfavorable	32	95

Of the 47 intermediate/advanced participants who voluntarily took part in the elective sight-reading classes, 10 had indicated a positive opinion about computer-based sight-reading, 6 had indicated a neutral opinion, and 31 had indicated a negative opinion. At the completion of the elective class, participants were again polled about their opinion of computer-based sight-reading. After this introductory experience, 46 of the 47 participants were favorably disposed, with 1 participant still holding a negative opinion of the software.

Discussion

Although somewhat common in the instrumental realm, the use of computer-based tools for vocalists is a relatively recent development. Surveys administered before and after a vocal sight-reading assessment yielded interesting insights into choral students’ attitudes and adaptability toward the use of technology for assessment.

Although computer-based scores ranged from 2% to 100%, an overlap in computer scores occurred within ability ratings assigned by live scorers. Live scorers were given the latitude to categorize participants based on the entire interaction during the testing process, including the practice time. It was anticipated that with the newness of the technology and the unfamiliar procedures, participants may not perform to their potential during the actual testing time. This proved to be the case, with many participants uncertain about when to begin singing and unaware of or unused to following a prescribed tempo. This resulted in considerably lower scores for many participants than they likely would have received if scored by a human judge, particularly in the case of the intermediate and advanced sight-readers. Thus the bottom range of scores for each group was not substantially different. However, it was important for the remainder of the survey to have participants accurately categorized by sight-reading ability.

Nearly 75% of participants reported frequent sight-reading practice within their choir rehearsals, with 23% reporting practicing sight-reading sometimes. Only 2% reported rarely practicing sight-reading, and none of the participants reported not ever practicing sight-reading in choir. These encouraging numbers indicate a significant commitment to sight-reading instruction on behalf of a large number of directors. The vast majority of practice is taking place through group instruction, with only 9% not citing that as a means of instruction. Small group or peer instruction was experienced by 36% of participants, most often in addition to large group instruction.

A clear majority (70%) of participants had experienced individual sight-reading assessment. Although the proportion of those receiving sight-reading instruction was similar between ability groups, the proportion of those being tested individually was not. Only 55% of beginners reported individual testing, whereas 72% of intermediate and 84% of advanced sight-readers reported individual testing. This verifies previous findings of the effectiveness of individual testing as an instructional tool for vocal sight-reading (Demorest, 1998; Killian & Henry, 2005). The overwhelming majority (94%) of those with individual testing experience reported being tested by a live scorer, with only one participant having any experience with computer-based sight-reading testing. These results demonstrate an opportunity to introduce computer-based testing as a means of assessment in choral classrooms. Additionally, because almost half of participants reported rarely or never practicing sight-reading on their own, this technology could help facilitate student practice outside of the choir rehearsal.

Self-reported sight-reading ability generally paralleled the live scorer ratings, both with the score ranges and with ability-rating means increasing from beginner to intermediate to advanced. This indicates a general self-awareness on the part of these participants, when it comes to their own musicianship levels.

Consistent with the data concerning instruction and testing in their choirs, participants reported very little experience with computer-based technology for vocal sight-reading or other vocal performance tasks, either for practice or for assessment. Thirty-seven participants reported having instrumental experience. Although the percentage of these participants with some level of experience with technology was slightly higher, it did not approach significance, particularly when considering all, but one of these participants also reported experience with the technology for vocal/choral tasks. This may be because these students have elected to forego their instrumental training to focus on choir, thus eliminating the opportunities to experience technology associated with advanced instrumental study. Another possible explanation could be that the perception of a comparatively greater use of technology within instrumental instruction is inaccurate.

Not surprisingly, participants reported greater anxiety levels for computer assessment than for assessment by a teacher and lesser anxiety levels than for assessment in front of their peers. Somewhat unexpectedly, a large number of participants reported greater anxiety levels for computer assessment than for assessment through recording. Causes for this anxiety may be found among responses provided about their opinions on using technology in the choral classroom.

Whereas preexposure opinions on use of technology were somewhat evenly distributed between favorable, neutral, and unfavorable, postexposure numbers shifted drastically to unfavorable (almost 70%). This indicates a negative first experience with the technology. Over half of those unwilling to consider using this type of technology in the future cited the inability to set their own tempo as the primary reason.

With live scoring, singers are typically asked to establish (and maintain) their own tempo for sight-reading, yet there is nothing that regulates their attempts to do so. And although they may receive a deduction in their score as a result of erratic tempo or rhythmic error, they can still proceed with the performance. With the computer-based sight-reading through SmartMusic, a click track and cursor provide an audible and visible prescribed tempo and cue to begin (although these features may be disabled by the user). During the initial encounter with the software, this was apparently the most disconcerting feature in the process. Although not at the same level of dissatisfaction, participants were also concerned with the accuracy of the program. Although there was no basis on which to make this judgment, the perception should be a concern for those interested in implementing this technology in their classroom.

Those who were favorably impressed with the technology and would consider using it in the future for their own sight-reading practice cited a variety of positive features. Interestingly, among the reasons was the fact that it enforced a strict tempo. What some saw as a deterrent, others saw as attractive. Those responding positively also felt that using the software would make them better sight-readers—a mind-set that may be related to those identifying a mandated tempo as a benefit. Again, in contradiction to the negative responders, several participants felt that it would be more accurate than other means. They also liked that it was easy to use, that they could use it on their own, and that it provided immediate feedback and could keep a record of their progress.

Perhaps the most useful revelation from this investigation came through the subsequent voluntary sight-reading class, in which the technology was systematically introduced to the participants outside of the testing arena. Although these participants were self-selected as those wanting supplemental sight-reading instruction, their opinion of the software mirrored that of the entire population, with greater numbers holding a negative opinion about the software than those with a positive or neutral opinion. After interacting with the software and becoming familiar with all of the features, including tempo control, available sight-reading materials, instant feedback, and the tracking of progress, participants had a drastic change in their responses. All but one of the participants viewed it favorably and were able to articulate benefits that perhaps had been previously overlooked or misunderstood. The lone holdout had an instrumental background and had formed an opinion about SmartMusic based on his experiences with it as an instrumentalist and was unwilling to reconsider his opinion regardless of any of the new capabilities. The most frequent comment by those who had changed their opinion after the class was that they wished they had been introduced to it in an instructional setting rather than in a high-pressure assessment situation. This is an important reminder for all those responsible for instruction. Although we may be excited about new and novel instructional or assessment devices, our students need to be fully acclimated to these new procedures before being held accountable for them.

With the advent of voice recognition technology, the time is ripe for investigating its effectiveness in educational applications, such as vocal/choral instruction and assessment. Research should focus on a variety of applications for group instruction, individual practice, and assessment. Future research should also explore the efficacy of presenting new technology systematically and the impact that a thoughtful and intentional introduction may have on student perceptions and values. A number of recent studies in the instrumental realm have investigated the effect of performance anxiety (Alexander, 2012; Hamann & Sobaje, 1983; Kokotsaki & Davidson, 2003; Osborne & Kenny, 2008; Smith, 2010; Wehr-Flowers, 2006). Likewise, future research should explore the role of anxiety in vocal sight-reading. With the goal of creating independent and musically literate vocalists, there is much to be explored in today’s world of technology and its potential for instruction in choral music.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Alexander

(2012). Fearless improvisation: A pilot study to analyze string students’ confidence, anxiety, and attitude toward learning improvisation. Update: Applications of Research in Music Education, 31(1), 25–33. doi:10.1177/8755123312457884

Bowles

(1971). The Belwin–Mills Singing Achievement Test. New York, NY: Belwin-Mills.

Boyle

Lucas

(1990). The effect of context on sightsinging. Bulletin of the Council for Research in Music Education, 106, 1–9. Retrieved from https://www-jstor-org.web.bisu.edu.cn/stable/40318858

Bunderson

Inouye

Olsen

(1989). The four generations of computerized educational measurement. In Linn

R. L.

(Ed.), Educational measurement (3rd ed., pp. 367–407). Washington, DC: American Council on Education.

Demorest

(1998). Improving sight-singing performance in the choral ensemble: The effect of individual testing. Journal of Research in Music Education, 46, 182–192. doi:10.2307/3345622

Demorest

May

. (1995). Sight-singing instruction in the choral ensemble: Factors related to individual performance. Journal of Research in Music Education, 43, 156–167. doi:10.2307/3345676

Farnum

(1969). Farnum String Scale: A performance scale for all string instruments. Winona, MN: Hal Leonard.

Hamann

D. L.

Sobaje

(1983). Anxiety and the college musician: A study of performance conditions and subject variables. Psychology of Music, 11, 37–50. doi:10.1177/0305735683111005

Henry

(2001, Fall). The development of a vocal sight-reading inventory. Bulletin of the Council for Research in Music Education, 150, 21–35. Retrieved from https://www-jstor-org.web.bisu.edu.cn/stable/40319097

10.

Henry

(2009, February). The development of a rhythm skills hierarchy for vocal sight-reading assessment. Research poster presented at the Texas Music Educators Association annual convention, San Antonio, TX.

11.

Henry

Demorest

(1994). Individual sight reading achievement in successful choral ensembles: A preliminary study. Update: Applications in Music Education, 13(1), 4–8.

12.

Killian

(1991). The relationship between sightsinging accuracy and error detection in junior high singers. Journal of Research in Music Education, 39, 216–224. doi:10.2307/3344721

13.

Killian

Henry

(2005). A comparison of successful and unsuccessful strategies in individual sight-singing preparation and performance. Journal of Research in Music Education, 53, 51–65. doi:10.1177/002242940505300105

14.

Kokotsaki

Davidson

J. W.

(2003). Investigating musical performance anxiety among music college singing students: A quantitative analysis. Music Education Research, 5, 45–59. doi:10.1080/1461380032000042899

15.

Kolb

(1984). A real-time microcomputer-assisted system for translating aural, monophonic tones into music notation as an aid in sight-singing. Bulletin of the Council for Research in Music Education, 93, 49–52.

16.

Larson

(1977). Relationships between melodic error detection, melodic dictation, and melodic sightsinging. Journal of Research in Music Education, 25, 264–271. doi:10.2307/3345267

17.

Lorek

Pembrook

(1989). Present and future applications of a microcomputer-based frequency analysis system. Psychomusicology, 8, 97–109. Retrieved from http://www.upei.ca/%257Emusicog/site_files/Volume%208,%20No%202.pdf

18.

Lucas

(1994). Contextual condition and sightsinging achievement of middle school choral students. Journal of Research in Music Education, 23, 23–34. doi:10.2307/3345700

19.

MakeMusic. (2011). SmartMusic 2012. Edina, MN: Author.

20.

Osborne

M. S.

Kenny

D. T.

(2008). The role of sensitizing experiences in music performance anxiety in adolescent musicians. Psychology of Music, 36, 447–462. doi:10.1177/0305735607086051

21.

Peters

(1974). Feasibility of computer-assisted instruction for instrumental music education (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign, IL. Retrieved from http://vufind.carli.illinois.edu/vf-uiu/Record/uiu_6139978

22.

Platte

(1981). The effects of a microcomputer-assisted instructional program on the ability of college choral ensemble members to sing melodic configurations at sight. Dissertation Abstracts International, 42, 04A.

23.

Scott

(1996). The construction of a holistic, criterion-reference sightsinging test for high school sopranos based on the voluntary national standards for music education (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign.

24.

Smith

(2010). Perspectives on improvising in beginning string pedagogy: A description of teacher anxiety, confidence, and attitude (Unpublished master’s thesis). Kent State University, OH.

25.

Watkins

Farnum

(1962). The Watkins–Farnum Performance Scale. Winona, MN: Hal Leonard Music.

26.

Wehr-Flowers

(2006). Differences between male and female students’ confidence, anxiety, and attitude toward learning jazz improvisation. Journal of Research in Music Education, 54, 337–349. doi:10.1177/002242940605400406