The Cleft Audit Protocol for Speech—Augmented: A Validated and Reliable Measure for Auditing Cleft Speech

Abstract

Objectives

To develop an assessment tool for use in intercenter audit studies of cleft speech and to test its acceptability, validity, and reliability. The tool is to be used systematically to record and report speech outcomes, providing an indication of treatment needs and continuing burden of care.

Setting

Regional Cleft Center, U.K.

Methods

The Cleft Audit Protocol for Speech—Augmented (CAPS-A) was developed by three cleft speech experts who identified the key features required from existing assessment measures. Criterion validity was assessed by comparing the Cleft Audit Protocol for Speech—Augmented outcomes reported for 20 cases with clinical assessment results and other investigations. Intra- and interrater reliability were tested following the training of specialist speech and language therapists who used the Cleft Audit Protocol for Speech—Augmented on two occasions to assess 10 cases. The raters evaluated acceptability and ease of using a questionnaire.

Results

The mean percentage agreement for criterion validity in each section was 87% (range 70% to 100%). Both intra- and interexaminer reliability were rated as good/very good (Kappa 0.61 to 1.00) for seven sections and moderate (Kappa 0.41 to 0.60) for three sections. Raters reported that the Cleft Audit Protocol for Speech—Augmented was acceptable and easy to use with appropriate training.

Conclusion

An acceptable, valid, and reliable cleft speech audit tool has been developed based on a small sample. The Cleft Audit Protocol for Speech—Augmented is recommended for use in intercenter audit studies in the U.K. and Ireland and could be used in other English-speaking countries. In addition, it has wider applicability for use in reporting speech outcomes of surgical procedures.

Keywords

audit cleft palate outcome measure reliability speech validity

Get full access to this article

View all access options for this article.

References

Ainoda

, Okazaki

. Results of systematic speech sound monitoring in children with cleft palate. Folia Phoniatr Logop. 1993; 48: 286–291.

Bearn

Mildinhall S, Murphy T, Murray J, Sell D, Shaw W, Williams A, Sandy J. Cleft lip and palate care in the United Kingdom—the Clinical Standards Advisory Group (CSAG) Study. Part 4: outcome comparisons, training, and conclusions.

Cleft Palate Craniofac J. 2001; 38(1): 38–43.

Broder

H.L.

, Smith

F.B.

, Strauss

R.P.

. Habilitation of patients with clefts: parents and child ratings of satisfaction with appearance and speech. Cleft Palate Craniofac J. 1992; 29: 262–267.

Clarke

. A Study of Nasalance Scores for Normal English-Speaking Children From the London and Essex Regions. London: University College; 2001. BSc project.

Clinical Standards Advisory Group (CSAG). CSAG Report—Cleft Lip and Palate. London: HMSO; 1998.

Cobden

. The Influence of Age, Gender and Dialect on Nasalance Scores for Normal English-Speaking Children From the London and Essex Regions. London: University College; 2001. BSc project.

Dalston

R.M.

, Marsh

J.L.

, Vig

K.W.

, Witzel

M.A.

, Bumsted

R.M.

. Minimal standards for reporting the results of surgery on patients with cleft lip, cleft palate, or both: a proposal. Cleft Palate J. 1988; 25(1): 3–7.

D'Antonio

L.L.

, Scherer

N.J.

. The evaluation of speech disorders associated with clefting. In: Shprintzen

R.J.

, Bardach

, eds. Cleft Palate Speech Management. St. Louis: Mosby; 1995: 176–220.

Department of Health. Clinical Governance: Quality in the New NHS. London: HMSO; 1999.

10.

Eurocleft Speech Group: Grunwell

, Bronsted

, Henningsson

, Jansonius

, Karling

, Meijer

, Ording

, Sell

, Wyatt

, Vermeij-Zieverink

. A phonetic framework for the cross-linguistic analysis of cleft palate speech. Clin Linguist Phon. 1994; 8: 109–125.

11.

Eurocleft Speech Group: Grunwell

, Bronsted

, Henningsson

, Jansonius

, Karling

, Meijer

, Ording

, Sell

, Wyatt

, Vermeij-Zieverink

. A six-centre international study of treatment outcome in patients with clefts of the lip and palate. Scand J Plast Reconstr Hand Surg. 2000; 34: 31–41.

12.

Gooch

J.L.

, Hardin-Jones

, Chapman

K.L.

, Trost-Cardamone

J.E.

, Sussman

. Reliability of listener transcriptions of compensatory articulations. Cleft Palate Craniofac J. 2001; 38(1): 59–67.

13.

Harding

, Harland

, Razzell

. Cleft Audit Protocol for Speech (CAPS). Broomfield, Chelmsford, Essex: St. Andrew's Plastic Surgery Centre; 1997.

14.

Harland

. An extended audit of complete boney unilateral cleft lip and palate patients aged 5–6 and 8–11: speech outcomes. Presented at the annual meeting of the Craniofacial Society of Great Britain and Ireland; 1996; Egham, Surrey, U.K.

15.

Hayden

, Klimacka

. Interrater reliability in cleft palate speech assessment. J Clin Excel. 2000; 2: 169–173.

16.

Henningsson

, Hutters

. Perceptual assessment of cleft palate speech, with special reference to minimum standards for inter-centre comparisons of speech outcome. Presented at the Eighth International Congress on Cleft Palate and Related Craniofacial Anomalies; 1997; Singapore.

17.

Hirschberg

, Van Demark

D.R.

. A proposal for standardization of speech and hearing evaluations to assess velopharyngeal function. Folia Phoniatr Logop. 1997; 49: 158–167.

18.

Howard

S.J.

, Heselwood

B.C.

. Learning and teaching phonetic transcription for clinical purposes. Clin Linguist Phon. 2002; 16: 371–401.

19.

Jocelyn

L.J.

, Penko

M.A.

, Rode

H.L.

. Cognition, communication and hearing in young children with cleft lip and palate and in control children: a longitudinal study. Paediatrics. 1996; 97: 529–534.

20.

Kent

R.D.

, Weismer

, Kent

J.F.

, Vorperian

H.K.

, Duffy

J.R.

. Acoustic studies of dysarthric speech: methods, progress, and potential. J Commun Disord. 1999; 32: 141–180.

21.

Keuning

, Wieneke

G.H.

, Dejonckere

P.H.

. The intra-judge reliability of the perceptual rating of cleft palate speech before and after pharyngeal flap surgery: the effect of judges and speech samples. Cleft Palate J. 1999; 36: 328–333.

22.

Kreiman

, Gerratt

B.R.

, Kempster

G.B.

, Ermnan

, Berke

G.S.

. Perceptual evaluation of voice quality: review, tutorial, and a framework for future research. J Speech Hear Res. 1993; 36: 21–40.

23.

Kuehn

, Moller

. Speech and language issues in the cleft palate population: the state of the art. Cleft Palate Craniofac J. 2000; 37(4): 348.

24.

Kuehn

, Sell

, Trost-Cardamone

. Issues in universal reporting parameters for speech. Presented at the 59th annual meeting of the American Cleft Palate-Craniofacial Association; 2002; Seattle, Washington.

25.

Landis

, Koch

. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 159–174.

26.

Lohmander-Agerskov

, Olsson

. Methodology for perceptual assessment of speech in patients with cleft palate: a critical review of the literature. Cleft Palate Craniofac J. 2004; 41: 64–70.

27.

McComb

. Cleft lip and palate new directions for research. Cleft Palate J. 1989; 26: 145–147.

28.

McWilliams

B.J.

, Morris

H.L.

, Shelton

R.L.

. Cleft Palate Speech. Philadelphia: BC Decker; 1984.

29.

McWilliams

B.J.

, Morris

H.L.

, Shelton

R.L.

. Cleft Palate Speech. 2nd ed. Philadelphia: BC Decker; 1990.

30.

McWilliams

B.J.

, Philips

B.J.

. Velopharyngeal incompetence audio seminars in speech pathology. Philadelphia: WB Saunders; 1979.

31.

National Institute for Clinical Excellence. Available at: http://www.nice.org.uk. Accessed May 31, 2005.

32.

Razzell

R.E.

, Harding

. A review of cleft palate speech assessment methods in the U.K. Presented at International Association of Logopedics and Phoniatrics; 1995; Cairo, Egypt.

33.

Reilly

, Douglas

, Oates

. Evidenced Based Practice and Speech Pathology. London: Whurr; 2003.

34.

Sandy

, Williams

, Bearn

, Mildinhall

, Murphy

, Sell

, Murray

, Shaw

. Cleft lip and palate care in the United Kingdom—the Clinical Standards Advisory Group (CSAG) Study. Part 1: background and methodology. Cleft Palate Craniofac J. 2001; 38(1): 20–23.

35.

Sell

. Issues in perceptual speech analysis in cleft palate and related disorders: a review. Int J Lang Commun Disord. 2005; 40: 103–121.

36.

Sell

, Grunwell

, Mildinhall

, Murphy

, Cornish

, Bearn

, Shaw

, Murray

, Williams

, Sandy

. Cleft lip and palate care in the United Kingdom—the Clinical Standards Advisory Group (CSAG) Study. Part 3: speech outcomes. Cleft Palate Craniofac J. 2001; 38(1): 30–37.

37.

Sell

, Harding

, Grunwell

. A screening assessment of cleft palate speech: “GOS.SP.ASS.” (Great Ormond Street Speech Assessment). Eur J Disord Commun. 1994; 29: 1–15.

38.

Sell

, Harding

, Grunwell

. Revised GOS.SP.ASS(98): speech assessment for children with cleft palate and/or velopharyngeal dysfunction. Int J Lang Commun Disord. 1999; 34: 7–33.

39.

Shriberg

L.D.

, Kwiatkowski

, Hoffmann

. A procedure for phonetic transcription by consensus. J Speech Hear Res. 1984; 27: 456–465.

40.

Shriberg

L.D.

, Lof

G.L.

. Reliability studies in broad and narrow phonetic transcription. Clin Linguist Phon. 1991; 1: 171–189.

41.

Stoel-Gammon

. Transcribing the speech of young children. Journal article, CEU, exam questions, tables/charts. Top Lang Disord. 2001; 21(4): 12–21, 87, 89.

42.

Streiner

, Norman

. Health Measurement Scales. Oxford: Oxford University Press; 1989.

43.

Sweeney

. The Perceptual and Instrumental Assessment of Nasality and Nasal Airflow Errors Associated With Velopharyngeal Dysfunction. Dublin: Trinity College; 2000. Thesis.

44.

Sweeney

, Sell

, O'Regan

. Nasalance scores for normal-speaking Irish children. Cleft Palate Craniofac J. 2004; 41: 168–174.

45.

Whitehill

T.L.

. Assessing intelligibility in speakers with cleft palate: a critical review of the literature. Cleft Palate Craniofac J. 2002; 39(1): 50–58.

46.

Williams

, Bearn

, Mildinhall

, Murphy

, Sell

, Shaw

, Murray

, Sandy

. Cleft lip and palate care in the United Kingdom—the Clinical Standards Advisory Group (CSAG) Study. Part 2: dentofacial outcomes and patient satisfaction. Cleft Palate Craniofac J. 2001; 38(1): 24–29.

47.

Wirz

, Mackenzie Beck

. Assessment of voice quality: the Vocal Profiles Analysis Scheme. In: Wirz

., ed. Perceptual Approaches to Communication Disorders. London: Whurr; 1995: 39–55.

48.

Witzel

. Speech evaluation and treatment. Oral Maxillofac Surg Clin North Am. 1991; 3: 501–516.

49.

Wyatt

, Sell

, Russell

, Harding

, Harland

, Albery

. Cleft palate speech dissected: a review of current knowledge and analysis. Br J Plast Surg. 1996; 49: 143–149.

50.

Young

M.A.

. Observer agreement: cumulative effects of repeated ratings of speech sample and knowledge of group results. J Speech Hear Res. 1969; 12: 144–155.