This investigation studied the effects of perceptual anchors on the dispersion and reliability of listener ratings of nasality.
Design
Listeners (N = 129) were assigned to one of six listening groups. Each group rated nasality independently for 100 speech samples on a seven-point scale that ranged from 1 = normal nasality to 7 = severe hypernasality. The anchors used were examples of a 1, 3, 4, 5, and/or 7 on the rating scale. These anchors were played selectively to group 2 (4), group 3 (1 and 7), group 4 (3 and 5), group 5 (1, 4, 7), and group 6 (7). Group 1 had no anchor.
Participants
Of the speakers, 95 were children followed by a craniofacial team and five were children without histories of speech disorders.
Main Outcome Measures
The outcome measures were 12,900 ratings of nasality on a seven-point scale.
Results
Q values showed that group 5, which was the only group to receive three anchors, had the lowest, or best, Q value (0.78), and group 1 (no anchor) had the highest, or worst, Q value (0.99). Across groups, the most reliable ratings were those at scale values 1 (Q = 0.46) and 7 (Q = 0.56). The least reliable ratings were at scale values 3 (Q = 1.01), 4 (Q = 1.03), and 5 (Q = 1.06).
Conclusions
Nasality rating reliability/dispersion was influenced by the presence and location of anchor stimuli. Consistent with absolute judgment theory, nasality ratings showed a strong end effect.
AltmanD.Practical Statistics for Medical Research.London: Chapman and Hall; 1991.
2.
AwanS., LawsonL.The effect of anchor modality on the reliability of vocal severity ratings.J Voice.2009; 23: 341–352.
3.
BettensK., WuytsF.L., Van LierdeK.M.Instrumental assessment of velopharyngeal function and resonance: a review.J Comm Disord.2014; 52: 170–183.
4.
BradfordL., BrooksA., SheltonR.Clinical judgment of hypernasality in cleft palate children.Cleft Palate J.1964; 6: 329–335.
5.
BrancampT., LewisK., WattersonT.The relationship between nasalance scores and nasality ratings obtained with equal appearing interval and direct magnitude estimation scaling methods.Cleft Palate Craniofac J.2010; 47: 631–637.
6.
CarneyP., ShermanJ.Severity of nasality in three selected speech tasks.J Speech Hear Res.1971; 14: 396–407.
7.
ChanK., YiuEM-L.A comparison of two perceptual voice evaluation training programs for naïve listeners.J Voice.2006; 20: 229–241.
8.
ChanK.M.K., YiuEM-L.The effect of anchors and training on the reliability of perceptual voice evaluation.J Speech Hear Res.2002; 45: 111–126.
9.
CounihanD., CullinanW.Reliability and dispersion of nasality ratings.Cleft Palate J.1970; 7; 261–270.
10.
CuddyL.L.Practice effects in the absolute judgment of pitch.J Acoustic Soc Am.1967; 43: 1069–1076.
11.
CuddyL.L., PinnJ., SimonsE.Anchor effects with biased probability of occurrence in absolute judgment of pitch.J Exp Psychol.1973; 100: 218–220.
12.
CullinanW., CounihanD.Some factors affecting the size of Q values for speech ratings.Percept Motor Skill.1968; 27: 531–536.
13.
CullinanW.L., PratherE.M., WilliamsD.E.Comparison of procedures for scaling severity of stuttering.J Speech Hear Res.1963; 6: 187–194.
14.
EadieT.L., Kapsner-SmithM.The effect of listener experience and anchors on judgments of dysphonia.J Speech Hear Res.2011; 54: 430–447.
15.
EriksenC.W., HakeH.W.Absolute judgments as a function of stimulus range and number of stimulus and response categories.J Exp Psychol.1955; 49: 323–332.
16.
EriksenC.W., HakeH.W.Anchor effects in absolute judgments.J Exp Psychol.1957; 53: 132–138.
17.
GalekK., WattersonT., LewisK.The effects of training on the reliability of listener judgments of nasality. Presented at the annual meeting of the American Cleft Palate and Craniofacial Association; April 2012; San Jose, CA.
18.
GarnerW.An informational analysis of absolute judgments of loudness.J Exp Psychol.1953; 46: 373–380.
19.
GerrattB., KreimanJ., Antonanzas-BarrosoN., BerkeG.Comparing internal and external standards in voice quality judgments.J Speech Hear Res.1993; 36: 14–20.
20.
HardinM., DemarkD.V., MorrisH.L., PayneM.M.Correspondence between nasalance scores and listener judgements of hypernasality and Hyponasality.Cleft Palate Craniofac J.1992; 29: 346–351.
McWilliamsB.J.Some factors in the intelligibility of cleft-palate speech.J Speech Hear Disord.1954; 19: 524–527.
31.
MollK.L.Objective measures of nasality.Cleft Palate J.1964; 35: 371–374.
32.
PratherE.Scaling defectiveness of articulation by magnitude-estimation.J Speech Hear Res.1960; 3: 380–392.
33.
ShermanD.The merits of backward playing of connected speech in the scaling of voice quality disorders.J Speech Hear Disord.1954; 19: 312–321.
34.
ShermanD., GoodwinF.Pitch level and nasality.J Speech Hear Disord.1954; 19: 423–428.
35.
ShermanD., SilvermanF.Three psychological scaling methods applied to language development.J Speech Hear Res.1968; 11: 837–841.
36.
ShermanD., SpriestersbachD.C., NollJ.D.Glottal stops in the speech of children with cleft palates.J Speech Hear Disord.1959; 24: 37–42.
37.
SilvermanF.Interpretation of mean Q values for sets of stimuli rated on a seven point equal appearing interval scale.Percept Motor Skill.1967; 24: 842.
SpriestersbachD.C.Assessing nasal quality in cleft palate speech of children.J Speech Hear Dis.1955; 20: 266–270.
40.
SpriestersbachD.C., PowersG.Nasality in isolated vowels and connected speech of cleft palate speakers.J Speech Hear Res.1959; 2: 40–45.
41.
TonerM., EmanuelF., ParkerD.Relationship of spectral noise levels to psychophysical scaling of vowel roughness.J Speech Hear Res.1990; 33: 238–244.
42.
Van HattumR.Articulation and nasality in cleft palate speakers.J Speech Hear Res.1958; 1: 383–387.
43.
VolkmannF.C., EngenT.Three types of anchoring effects in the absolute judgment of hue.J Exp Psychol.1961; 61: 7–17.
44.
WattersonT., HintonJ., McFarlaneS.Novel stimuli for obtaining nasalance measures from young children.Cleft Palate Craniofac J.1996; 33: 67–73.
45.
WattersonT., LewisK., AllordM., SulprizioS., O'NeillP.Effect of vowel type on reliability of nasality ratings.J Commun Disord.2007; 40: 503–512.
46.
WattersonT., LewisK.E., MurdockT., CorderoK.N.Reliability and validity of nasality ratings between a monolingual and bilingual listener for speech samples from English-Spanish speaking children.Folia Phoniatr Logo.2013; 65: 91–97.
47.
WattersonT., McFarlaneS., WrightD.The relationship between nasalance and nasality in children with cleft palate.J Commun Disord.1993; 26: 13–28.
48.
YoungM., DownsT.Testing the significance of the agreement among observers.J Speech Hear Res.1968; 11: 5–17.
49.
YunusovaY., WeismerG., KentR.D., RuscheN.M.Breath-group intelligibility in dysarthria: characteristics and underlying correlates.J Speech Hear Res.2005; 48: 1294–1310.
50.
ZraickR., LissJ.A comparison of equal appearing interval scaling and direct magnitude estimation of nasal voice quality.J Speech Lang Hear R.2000; 43: 979–988.
