A pilot study comparing the effectiveness of speech language therapy provided by telemedicine with conventional on-site therapy

Abstract

We compared the progress made by school children in speech language therapy provided through videoconferencing and conventional face-to-face speech language therapy. The children were treated in two groups. In the first group, 17 children received telemedicine treatment for 4 months, and then subsequently conventional therapy for 4 months. In the second group, 17 children received conventional treatment for 4 months and then subsequently telemedicine treatment for 4 months. The outcome measures were student progress, participant satisfaction and any interruptions to service delivery. Student progress reports indicated that the children made similar progress during the study whichever treatment method was used. There was no significant difference in GFTA-2 scores (Goldman-Fristoe Test of Articulation) between students in the two treatment groups. Satisfaction surveys indicated that the students and parents overwhelmingly supported the telemedicine service delivery model. During the study, a total of 148 of the 704 possible therapy sessions was not completed (21%); the pattern of cancellations was similar to cancellations in US public schools generally. Videoconferencing appears to be a promising method of delivering speech language therapy services to school children.

Introduction

Telemedicine has been used to provide access to speech language therapy services for underserved populations.^1–3 This is sometimes called telepractice. The reports of telepractice applications in educational settings⁴ have provided descriptions and satisfaction survey results, but have not included outcomes data. Two recent articles have reported the use of telehealth for the delivery of the Lidcombe programme of early stuttering intervention to young children who stutter and their families.^2,5 The results suggest that telehealth delivery is effective for preschool children. Sicotte et al. ⁶ found that children who stuttered improved their fluency using stuttering therapy provided by videoconferencing. A pilot study investigating the assessment of childhood speech disorders via videoconferencing found high levels of agreement in scoring for single word articulation, speech intelligibility and oral motor tasks between the online and the on-site speech language pathologists.⁷

We have conducted a pilot trial of speech language therapy services via videoconferencing for schools in rural Ohio.

Methods

The four school districts chosen to participate in the trial were small rural districts in the state of Ohio. At the time of the study, these districts had speech and language therapy services available. However, they had experienced previous shortages of services. A total of 38 students from the four school districts took part in the study. The students were aged 4–12 years. Thirteen of the 38 students were female. The students exhibited articulation, language and/or fluency disorders as either a primary handicapping condition or related to a learning disability. Data for six of the students are not included in the present report, because three students did not receive baseline testing, two students did not complete therapy, and one student was dismissed from speech and language services when her disability identification changed and speech and language therapy services were no longer deemed necessary.

Students were eligible to participate in the study if they had a current Individualized Education Plan (IEP) which included the provision of speech and/or language therapy services. Students with a diagnosis of autism, pervasive developmental disorder, severe cognitive deficit or severe emotional disturbance were excluded. Potential subjects were identified from the current caseloads of the speech language pathologists (SLPs) and consent obtained. Families were offered a $150 gift card for completing the research project. The study was approved by the appropriate ethics committee.

Intervention

The children were treated in two groups. In the first, Group A, 17 children received telemedicine treatment for 4 months, and then subsequently conventional therapy for 4 months. In the second, Group B, 17 children received conventional treatment for 4 months and then subsequently telemedicine treatment for 4 months. There was no washout period between treatments. Allocation of children to the two groups was random.

The services provided followed the goals and objectives listed on each child's IEP. The therapy sessions provided via telemedicine were mainly individual therapy sessions. The therapy sessions provided on-site were primarily group sessions for 2–4 students.

Therapy was provided by SLPs. Students were escorted to and from telemedicine therapy sessions by ‘e-helpers’. The e-helper remained during the therapy sessions to solve any technology problems and to provide adult supervision. The e-helpers also assisted by communicating with the remote SLP, receiving and sending faxes and mail related to telepractice, sending home paperwork associated with telepractice and maintaining a log of all therapy sessions. The e-helpers were trained in the use of the telepractice equipment, in basic troubleshooting strategies, in their responsibilities and in maintenance of confidentiality.

PC-based videoconferencing was used for telepractice at the remote sites. The students wore headphones and the e-helpers could also listen via headphones. At the SLP site, PC-based videoconferencing was also used. In addition, the SLP had a document camera. Videoconferencing took place via the educational network at a minimum bandwidth of 10 Mbit/s.

Outcome measures

The outcome measures were student progress, participant satisfaction and any interruptions to service delivery.

Student progress

Student progress was measured by progress reports, ratings from the National Outcomes Measurement System⁸ and the Goldman-Fristoe Test of Articulation.⁹

Progress reports were completed every three months by the SLP providing services at that time. The therapists rated student performance on each IEP objective using the following scale: Mastered, Making Adequate Progress, Making Inadequate Progress and Objective Not Initiated.

The National Outcomes Measurement System (NOMS)⁸ of the American Speech Language Hearing Association (ASHA) uses Functional Communication Measures (FCMs), which are a series of seven-point scales to assess functional change in communication and swallowing abilities over time. These scales are designed to reflect functional or daily life change in student performance. Students are rated on one or more of the scales depending on their particular communication impairment (e.g. spoken language comprehension, spoken language production, speech intelligibility, speech sound production, fluency). The NOMS scales were employed in a non-conventional manner in the present study. We obtained approval from ASHA to use the scales to rate the student's functional abilities at the start and end of treatment. In order to use the NOMS scales, the SLPs passed the ASHA inter-rater reliability test⁸ required for all potential users.

The Goldman-Fristoe Test of Articulation (GFTA)⁹ is commonly used in public schools to assess articulation and was administered by two of the authors at the beginning, middle and end of the project.

Satisfaction

Satisfaction surveys were distributed to students, their parents, and selected Hardin County educational staff and administration staff. Student, parent and staff surveys were distributed at the end of therapy. Principals, SLPs and e-helpers were surveyed at the completion of the study.

The survey was administered to the children with the help of one of the investigators. The student was shown the survey and instructions were provided orally and then repeated following each of the questions on the survey.

Parents were asked to complete a satisfaction survey when their child finished participating in the telepractice portion of the pilot project.

Teachers were asked to complete a satisfaction survey when students in their classrooms finished their period of telemedicine therapy.

The four Hardin County SLPs working in the districts involved in the study were asked to complete satisfaction surveys. These SLPs collaborated with the investigators to schedule the students for the project, obtain permission forms and solve any problems encountered with the start of the project.

The four e-helpers for the telepractice project were asked to complete satisfaction surveys.

The four principals of the four schools involved in the study were asked to complete a satisfaction survey.

Cancellations

The e-SLPs and e-helpers maintained a daily log which recorded all therapy sessions, audio and video quality and reasons for any cancellations that occurred.

Results

Student progress

The results for each service delivery method are summarised in Table 1. The progress reports indicated that the children made similar progress during the study whichever treatment method was used.

Table 1

Student progress report data summarized by service delivery method. The values represent the totals for both groups

	Telepractice	On-site
No. of students	36	36
Total number of IEP objectives	126	112
No. of objectives mastered	50	20
No. of objectives adequate progress	51	70
No. of objectives limited progress	2	4
No. of objectives no progress	0	0
No. of objectives not initiated	23	6
No. of objectives not scored	0	7
No. of students dismissed	3	2

The student progress reports after the first treatment period indicated that adequate progress or mastery was accomplished for 75% of the objectives (58/77 objectives for telepractice and 34/45 for on-site) in both service delivery models. There were no objectives for which no progress was reported. Note that 18/77 (23%) objectives were not initiated in the telepractice model as compared to 4/45 (8%) in the on-site model. This difference resulted from objectives being written in a hierarchical manner so that objective 1 had to be accomplished before beginning objective 2, and some objectives required data collection in the classroom which could not be accomplished by telemedicine. Also, an error in reporting in the on-site model meant that three objectives were not scored.

The student progress reports after the second treatment period indicated that mastery or adequate progress was accomplished for 88% of the objectives for telepractice (42/48) and 84% of the objectives for on-site (56/67). There were no objectives for which no progress was reported. The difference between the two groups in achieving mastery/adequate progress was significant (z = 2.0, P < 0.05). This may have resulted from fewer objectives being targeted in the telepractice model as compared to the on-site model during the second treatment period.

Complete NOMS data was not available for all students throughout the study. At the conclusion of the study, 9 of the 30 students (30%) had made no progress, 7/30 (23%) had improved by one level on their respective FCM and 14/30 (47%) had improved by multiple levels. We compared these changes with the K-12 Schools 2008 National NOMS Data Report for the FCMs of Intelligibility, Speech Sound Production and Spoken Language Production.¹⁰ The data for children in the study who were scored on the FCMs are summarised in Table 2, together with the results from the National NOMS report.¹⁰ Caution must be exercised in interpreting these results, due to the limited number of students for whom we were able to obtain NOMS data.

Table 2

Change in spoken language production, intelligibility and speech sound production FCMs for subjects in the present study vs. 2008 NOMS national report¹⁰

	Present study	NOMS National Report¹⁰
FCM D: intelligibility	(n = 8)
Percent of students who made no progress	38	29
Percent of students who improved one level	25	40
Percent of students who improved multiple levels	38	30
FCM G: speech sound production	(n = 14)
Percent of students who made no progress	29	21
Percent of students who improved one level	21	31
Percent of students who improved multiple levels	50	48
FCM I: spoken language production	(n = 7)
Percent of students who made no progress	29	37
Percent of students who improved one level	29	41
Percent of students who improved multiple levels	43	21

At the start of the study, there was no significant difference in GFTA-2 scores between students in the two treatment groups (t = 1.0, P = 0.16). The students were also compared after the first and second treatment periods. There was no significant difference between the groups after the first (t = 1.7, P = 0.06) or the second treatment period (t = 0.8, P = 0.21). This suggests that student performance with telepractice was similar to student performance with conventional treatment. A comparison of student performance in the two groups is shown in Table 3.

Table 3

Performance in the two groups as judged by the GFTA-2 scores⁹

	Group A		Group B
	Period 1 Telepractice (n = 11)	Period 2 On-site (n = 11)	Period 1 On-site (n = 11)	Period 2 Telepractice (n = 11)
No. of students whose standard score did not change	1	0	1	0
No. of students whose standard score increased	6	6	5	8
No. of students whose standard score decreased	4	5	5	3

Satisfaction

Survey results were available for 29 of the 38 students. Eight of them did not complete the study and the principal investigator failed to administer the survey to one of the students. Students in the telepractice project overwhelmingly expressed satisfaction with the service delivery model (see Table 4).

Table 4

Student satisfaction survey results. Values are percentages (n = 29)

	Yes	Don't know	No
1. I like coming here	93	7	0
2. I like having speech therapy with Ms Robin on the computer	86	10	3
3. Speech therapy helps me learn to talk better	83	17	0
4. I would like to do speech therapy using the computer again	83	14	3
5. I could see and hear Ms Robin during my therapy sessions	86	10	3

Twenty-two of a possible 33 parent surveys were returned. The results suggest overwhelming positive parent satisfaction with the telepractice service delivery model (see Table 5).

Table 5

Parent satisfaction survey results. Values are percentages (n = 22)

Rating:	0 do not know	2 below average	3 average	4 above average	5 very good
1. Delivery of speech language therapy services using the telepractice technology	9	0	9	41	41
2. Your child's overall therapy progress this year	0	0	5	59	36
3. The telepractice speech therapist's availability for communication with you regarding your child	9	0	18	27	45
4. Attitude of your child about receiving speech therapy services via telepractice	9	0	5	23	64
5. Your attitude about speech therapy services via telepractice	9	0	14	27	50
6. How do you think telepractice compares to face-to-face speech therapy?	9	5	18	41	27
7. What is the likelihood that you would recommend telepractice to other parents?	9	5	9	27	50

Fifteen of 27 possible staff surveys were returned (see Table 6). The results suggest that the teachers did not know about the components of the programme or student response/progress with the exception of the students' attitude towards telepractice, which they reported as ‘very good.’

Table 6

Staff satisfaction survey results. Values are percentages (n = 15)

Rating:	0 do not know	3 average	4 above average	5 very good
1. Delivery of speech language therapy services using the present technology	47	7	20	27
2. Student's overall therapy progress this semester	27	20	20	33
3. Speech therapist's availability for communication with you regarding this student(s)	7	3	40	3
4. Attitude of the students about receiving speech therapy services via telepractice	20	7	13	60
5. Attitude of the parents about speech therapy services via telepractice	67	0	0	33
6. Your attitude about speech therapy services via telepractice	13	33	13	40
7. How do you think telepractice compares to face-to-face speech therapy?	60	7	13	20
8. What is the likelihood that you would recommend telepractice to your colleagues?	40	13	20	27

The results from the four Hardin County SLPs suggest satisfaction with the programme (see Table 7).

Table 7

Hardin County SLPs satisfaction survey results. Values are percentages (n = 4)

Rating:	0 do not know	3 average	4 above average	5 very good
1. Delivery of speech language therapy services using the present technology	0	25	25	50
2. Telepractice speech therapist's availability for communication with you regarding this student(s)	0	25	75	0
3. Attitude of the students about receiving speech therapy services via telepractice	0	0	50	50
4. Attitude of the parents about speech therapy services via telepractice	75	25	0	0
5. Your attitude about speech therapy services via telepractice	0	0	75	25
6. How do you think telepractice compares to face-to-face speech therapy?	0	75	25	0
7. What is the likelihood that you would recommend telepractice to your colleagues?	0	0	10	0

All four e-helpers completed the survey. Overall results suggest satisfaction with the programme (see Table 8).

Table 8

E-helpers satisfaction survey results. Values are percentages (n = 4)

Rating:	0 do not know	3 average	4 above average	5 very good
1. Delivery of speech language therapy services using the present technology	0	0	75	25
2. Student's overall therapy progress in the telepractice project	50	25	25	0
3. Telepractice speech therapist's availability for communication with you regarding this student(s)	0	0	25	75
4. Attitude of the students about receiving speech therapy services via telepractice	0	0	50	50
5. Attitude of the parents about speech therapy services via telepractice	50	25	25	0
6. Your attitude about speech therapy services via telepractice	0	0	25	75
7. How do you think telepractice compares to face-to-face speech therapy?	50	0	50	0

All four principals completed the survey. Three of the four principals were satisfied with the project (see Table 9). The fourth principal received comments from the teaching staff regarding limited amounts of therapy provided via telepractice. Students successfully received the services identified on their individual IEPs.

Table 9

Principals satisfaction survey results. Values are percentages (n = 4)

Rating:	0 do not know	1 not very good	2 below average	3 average	4 above average	5 very good
1. Delivery of speech language therapy services using the present technology	50	0	0	0	25	25
2. Student's overall therapy progress this semester	25	25	0	0	25	25
3. Telepractice speech therapist's availability for communication with you regarding this student(s)	0	25	0	0	25	50
4. Attitude of the students about receiving speech therapy services via telepractice	0	0	25	0	0	75
5. Attitude of the parents about speech therapy services via telepractice	0	0	25	0	25	50
6. Attitude of the staff about speech therapy services via telepractice	0	25	0	25	25	25
7. Your attitude about speech therapy services via telepractice	0	0	25	0	25	50
8. How do you think telepractice compares to face-to-face speech therapy?	25	25	0	25	0	25
9. What is the likelihood that you would recommend telepractice to your colleagues?	25	25	0	0	0	50

Cancellations

The attendance data from the four school districts showed that there were 704 possible telepractice therapy sessions during the project period (see Table 10). Of the total number of possible sessions, 189 were missed. As 41 of the missed sessions were subsequently completed, the net total of missed sessions was 148 (21%). The pattern of cancellations was similar to cancellations in US public schools generally.

Table 10

Cancellation data for all districts

	Period 1	Period 2	Total
Total number of potential client contacts	289	415	704
Number of contacts cancelled – teacher unavailable	33	6	39
Number of contacts cancelled – bad weather	24	48	72
Number of contacts cancelled – student absences	12	16	28
Number of contacts cancelled – school activities	3	22	25
Number of contacts cancelled – site-based technical problems	0	7	7
Number of contacts cancelled – university-based technical problems	0	0	0
Number of contacts cancelled – SLP ill	18	0	18
Number of contacts rescheduled and completed	19	22	41

Discussion

The present study shows that videoconferencing of speech language therapy services was a reliable and effective method of service delivery. Student progress reports, ratings on the NOMS scales and results of standardized testing using the GFTA-2 all indicated that the students made similar progress using telepractice as they did with on-site therapy. Satisfaction surveys indicated that the students and their parents were overwhelmingly satisfied with telepractice. Speech language pathologists, e-helpers and school principals were generally satisfied with telepractice. Teacher satisfaction surveys indicated that they did not know enough about the project or the students' and parents' reactions. Daily logs indicated that telepractice sessions were rarely cancelled due to technology problems. Sessions were cancelled or missed for similar reasons that sessions were cancelled or missed in conventional therapy.

The perceived disadvantages of telemedicine for service delivery included the difficulty of collaborating with classroom teachers, of relating therapy to the classroom curriculum and of providing services in the typical classroom environment. Nonetheless, we believe that further research in using telemedicine for speech/language therapy services in schools is warranted. This is because: (a) telepractice therapy appears to be effective in accomplishing IEP goals and objectives; (b) failure to provide children with access to adequate speech/language therapy services could violate their rights to a free and appropriate public education; and (c) the telepractice delivery model can be implemented relatively easily.

The present study had certain limitations. First, it was an exploratory study which was conducted within the constraints of service delivery to students with communication impairments. Because of this, some independent variables were not controlled. For example, several different SLPs provided the assessment of the communication disorders, and developed the IEP goals and objectives for each student. Therapy sessions in the telepractice and conventional service delivery models differed in how services were provided, utilized different therapy materials and were delivered by five different SLPs. Progress reports were based on an individual clinician's data collection and clinical assessment of the student's progress. Finally, only a few subjects were recruited, resulting in reduced statistical power. A randomized controlled trial is required in the future.

Based on the results of the pilot study, videoconferencing appears to be an effective and reliable service delivery method for school age children who receive speech language therapy services in public schools. Thus telemedicine may be useful for delivering therapy to rural school districts which do not have adequate speech language therapy services available to them.

Footnotes

Acknowledgements

We are grateful to Dallas Johnson and Rod Gabel for assistance in the preparation of this article. We also thank the Hardin County Educational Service Center and Gail Peterson, Emily O'Brien, Celeste Hoverman, Mandy Bour and Molly Burnell. We thank the students and their parents for participating in the project and also the Upper Scioto Valley, Ridgemont, Riverdale and Hardin Northern School Districts. We also thank Steve Bostic, Mary Tipton and Aaron Near. The project was supported by a grant from the Ohio Department of Education.

References

American Speech-Language-Hearing Association. Speech-Language Pathologists Providing Clinical Services via Telepractice: Position Statement. See http://www.asha.org/policy (last checked 25 September 2009)

Lewis

, Packman

, Onslow

, Simpson

, Jones

. A phase II trial of telehealth delivery of the Lidcombe Program of Early Stuttering Intervention. Am J Speech Lang Pathol 2008;17:139–49

Polovoy

. Telepractice in schools helps address personnel shortages. ASHA Leader 2008;13:22–24

Forducey

. Speech telepractice program expands options for rural Oklahoma schools. ASHA Leader 2006;11:12–13

Wilson

, Onslow

, Lincoln

. Telehealth adaptation of the Lidcombe Program of Early Stuttering Intervention: five case studies. Am J Speech Lang Pathol 2004;13:81–93

Sicotte

, Lehoux

, Fortier-Blanc

, Leblanc

. Feasibility and outcome evaluation of a telemedicine application in speech-language pathology. J Telemed Telecare 2003;9:253–8

Waite

, Cahill

, Theodoros

, Busuttin

, Russell

. A pilot study of online assessment of childhood speech disorders. J Telemed Telecare 2006;12(Suppl. 3):92–94

American Speech-Language-Hearing Association. National Outcomes Measurement System (NOMS): K-12 Speech–Language Pathology User's Guide. Maryland: American Speech Language Hearing Association, 2003

Goldman

, Fristoe

. Goldman-Fristoe Test of Articulation-2. Minnesota: American Guidance Service Inc., 2002

10.

ASHA. National Outcomes Measurement System K-12 Schools 2008 National Data Report. Maryland: American Speech Language Hearing Association, 2008