Reliability of oral language and emergent literacy tele-assessments in autistic children: A preliminary study

Abstract

BACKGROUND:

The growing need and popularity of telepractice over conventional direct therapy by speech-language pathologists (SLPs) demand validation of oral language and literacy assessments via telepractice mode. Considering the limited research on the validation of standardised oral language and literacy tele-assessment in autistic children, it is vital to explore the feasibility of standardised oral language assessments in specific cultural and linguistic contexts.

OBJECTIVE:

To examine the reliability of standardised oral language and emergent literacy assessments delivered via telepractice in autistic children.

METHODS:

Emergent literacy and oral language skills of ten autistic children aged between 4 to 8 years were assessed using the Test of Emergent Literacy Assessment (TELA), and Assessment of Language Development (ALD) through in-person and tele-assessment with a gap of 15 days.

RESULTS:

The findings of the present study establish the reliability of standardised oral language and literacy tele-assessment in autistic children by demonstrating a high level of agreement between in-person and tele-assessment modes.

CONCLUSIONS:

Telepractice, may therefore, be a feasible and reliable mode of oral language and literacy assessment in autistic children, with adequate knowledge of the required special accommodation(s), training and strong collaboration with the facilitator(s).

Keywords

Tele-assessment reliability autistic children oral language literacy

1. Introduction

Autism is a neurodevelopmental disorder that is characterized by difficulties with social interaction, narrow interests, and repetitive behaviours [1]. Autism spectrum disorder (ASD) has been associated to numerous types of neurological and behavioral traits, such as motor coordination difficulties, attentional and emotional regulation difficulties, hyperactivity, and challenging behaviours [2]. Due to the range of difficulties exhibited, they experience difficulties learning to read and acquire early literacy skills [3]. Given the pervasive communication impairments in autistic children, early identification, diagnosis, and regular intervention services are essential to reduce the negative effect of such impairments on personal, academic and social skills and to promote communication development and educational achievement [4].

There is often reports on lack of facilities and services available for children with developmental disabilities in low- and middle-income countries (LMIC) [5]. The COVID-19 pandemic added an unexpected disruption to the already known barriers to access clinical services to autistic children in India [6]. There was a delay and interruption in the diagnosis and completion of assessment in autistic children due to a prolonged lockdown period [7]. Studies suggest that following COVID-19 lockdown, children with autism exhibit a significant decline in social, academic and communication skills globally [6, 8].

A rapid shift to telepractice is observed during pandemic in order to deliver diagnostic evaluation, assessment and intervention. Advantages of telepractice include easier access to speech and language services to clients from various cultural and linguistic diversity, speech and language services to clients from remote and rural areas, cost-effectiveness, ease of communication and collaboration among inter- or multi-disciplinary teams [9]. Before the onset of pandemic, there were only 12.9% speech language pathologists (SLPs) and audiologists indulged in tele practice and few research initiatives to study the speech-language services delivered via tele practice in India [10]. Studies pre-pandemic suggest the lack of evidence based and standardized resources for teletherapy and need for comprehensive training for professionals in India to deliver services through telepractice [10]. Survey on tele practice among Indian SLPs and audiologists’ points to the necessity to provide secured services, employ legal guidelines and policies for delivery through telepractice [10].

The feasibility and acceptability of telepractice in autistic children was explored prior to pandemic [11]. Systematic review on use of telepractice in autistic children and adults were found to be comparable to in-person services [12]. However, most of the research is centred around educating parents or teachers to assist in assessment and intervention, while there has been insufficient research on the direct involvement of children with telepractice services [11]. Studies indicate that SLPs have only limited knowledge and experience in conducting tele-assessments and intervention [13]. This is due to the varied dimensions involved in tele-assessment and in-person assessment. Standardised tele-assessment requires specific considerations due to the necessary change in administration procedures, when delivered over a screen through video conferencing software [14]. Tele-assessment platforms like Pearson Q global systems offer digitised versions of language assessment tools such as Clinical Evaluation of Language Fundamentals (CELF), Expressive Vocabulary Test (EVT) and Peabody Picture Vocabulary Test (PPVT) [15]. However, the cost of these tests are high, and hence not affordable by all clinicians and organisations [16]. During the pandemic, a “Letter of No Objection” was issued by test developers like Pearson and Pro-Ed, allowing the remote administration of test tools [17]. Researchers have focused on the evaluation of the validity, reliability and feasibility of online standardised language and literacy assessment tools, namely CELF in 5 to 9 year old children with language impairment [18], Receptive-Expressive Emergent Language Test; REELS in 0 to 6 year old children [19], Comprehensive Receptive and Expressive Vocabulary; CREV in 5 to 6.11 year old children with language impairement [20], PPVT in 4.3 to 12.9 year old children [21], and Test of Integrated Language and Literacy Skills; TILLS in 6 to 18.11 year old children with language impairment [22]. There is only limited research in standardized language and literacy assessments via telepractice in autistic children. A recent study in this regard, demonstrates higher correlation and agreement in standardised oral language assessment of 9 to 12 year old autistic children between in-person and tele-assessment conditions [11], while children’s responses were heterogenous and highly unique. In the Indian context, the only research which demonstrates equivalence and report high agreement in oral language scores between in-person and tele-assessment is reported in grade 1 children with and out language impairment [23]. The above studies reported that in-person assessment scores are comparable to tele-assessment scores. As tele- assessment normative for the standardized language tests are not available for all the tests, there is a need to evaluate the equivalency of scores obtained through onsite and remote administration [17].

Autistic children are generally perceived to be one of the most challenging groups to work via telepractice [24]. Hence, use of an e-helper or facilitator to support the examiner during tele-assessment as recommended by Pearson publishers, is preferred [25]. A facilitator during an assessment or intervention session could be any adult, caregiver or parent who can support the child with the technology and follow the instructions given by the clinician [26]. Adapting tele-assessment in autistic children poses few challenges. Behavioural challenges and difficulty comprehending the instruction could be one of the barriers to conduct tele-sessions with autistic children [24]. Standardized direct observation and skill assessment might be difficult for clinicians due to limited control over the home environment of the child [27]. Characteristics of the child and family such as high levels of activity that restrict child to stay in camera view, variable social skills, anxiety and poor parent-child interaction can make it difficult for clinicians to do a successful virtual behaviour observation [27]. Tele-assessment challenges demands the clinician to modify the administration procedures and rely more on supplemental materials and information from parents rather than direct observation [28]. In addition, technological difficulties such as poor internet connectivity impact tele-assessment negatively [28].

The overall development and evidence of telepractice is limited in LMIC due to various factors such as unavailability of sustainable models, insufficient economic resources, poor technological support and health policy preferences of each country. Hence, studies from high-income countries cannot be generalised to country like India, as they are conducted in different setting and health care systems [29]. Despite challenges faced by telepractice in India, it has the potential to overcome existing barriers such as non-availability of services and financial burden involved in an in-person healthcare setting. As discussed in the previous section, there is sparse research in establishing equivalency of formal standardized evaluation of language and literacy in tele-assessment for autistic children. Indian Speech and Hearing Association (ISHA) has proposed general guidelines encouraging clinicians to follow assessments via telepractice [30], yet the recommended procedures and standardized tools for tele-assessment are essentially lacking. Considering all these factors, exploring the feasibility and validity of standardised oral language assessment is vital. Hence, the current study aims to examine the reliability of standardised oral language and emergent literacy assessments delivered via telepractice in autistic children. We hypothesized that tele-assessments would be reliable for oral language and literacy skills in autistic children, based on previous research reports [11, 31].

2. Method

The study followed a cross-sectional research design. Prior permissions were obtained from the authors of the target tests such as Assessment of language development (ALD) and Test of Emergent Literacy Assessment (TELA) for the use and digitization. Ethical approval was obtained from Institutional Ethics Committee (No: 400/2020), and the study was registered under the Clinical Trial Registry of India (CTRI/2020/11/029417).

2.1 Participants

The participants were recruited from private speech and language therapy centers. Autistic children who fulfilled the following criteria were included in the study: a) children with mild to moderate ASD using Childhood Autism Rating Scale-2, scores ranging from 28 to 33.5 [32], b) autistic children having receptive and expressive language age above two years on ALD, c) autistic children having average and/or borderline intelligence based on a nonverbal cognition test: Seguin Form Board [33], and an adaptive functioning test: Vineland Social Maturity Scale [34], d) autistic children with predominant English use for communication at home, and e) autistic children having able to follow activities such as label pictures, point and comprehend simple directions. Information on English language exposure and proficiency was measured using Child Language Experience and Proficiency Questionnaire [35] with the help of parents and clinicians of autistic children. Children with a proficiency level of greater than 5 were recruited for the study. Autistic children having co-morbid conditions such as psychiatric disorder, genetic or chromosomal disorder, neurological conditions, such as fragile X syndrome or epilepsy, were excluded from the study. Furthermore, autistic children with poor cognitive skills and those who are not able to participate in preschool literacy activities, were also excluded from the study. Written informed consent was obtained from parents of autistic children to participate in the study.

2.2 Test material(s)/tools

Oral language skills of the autistic children were assessed using the ALD [36]. ALD is a language assessment tool in Indian English, to determine a child’s receptive and expressive language development from birth to 7.11 years. Children speaking multiple languages are typical in a multilingual nation like India, where English is the language of instruction for many in the school. Thus, in our study we used ALD to assess oral language abilities (in English) for autistic children having different native languages. TELA [37] was used to assess emergent literacy skills in autistic children. TELA is developed to evaluate emergent literacy skills in Indian English language learners between the age of 3 to 6 years with high test-retest (0.74 to 0.94) and inter-rater reliability (0.82 to 0.89). TELA evaluates code and meaning-related emergent literacy skills such as oral language, print knowledge and phonological processing. The subtests under each domain are as follows, a) oral language: vocabulary and story retell, b) concept about print: book handling skills, text discrimination and environmental print, c) alphabet knowledge: letter-sound knowledge, alphabetic principle and word recognition, and d) phonological awareness: phonological awareness including word awareness, rhyme awareness, syllable awareness, alliteration awareness and phoneme awareness, phonological short-term memory and phonological access to lexical storage (rapid automatized naming). All test images were scanned and inserted in a power point presentation to deliver pictures via a laptop for the tele-assessment of oral language and literacy skills.

2.3 Equipment

The presentation of stimuli from the examiner-end was controlled using a laptop (Hewlett-Packard-15s- du2071tu). Built-in laptop camera was used during tele-assessment for recording the child’s responses and examiner’s instructions. Headphones with inbuilt microphones were used by the examiner during tele-assessment (JBL Quantum 100 over ear wired headphone). All devices used at the child’s end for all participants were personal computer and/or laptops. LAN connection was used to connect the laptop to the internet. A portable hotspot dongle was utilised in place of the LAN network during power outages or unavailability. The video-conferencing software used for tele-assessments was ZOOM (ZOOM Video Communications, Inc., 2020). Internet connectivity was ensured both at the examiner- and the child-end with higher bandwidth. The minimum required upload/download speed of at least 3 MB was maintained for best connectivity and screen sharing [38].

2.4 Procedure

Both in-person and tele-assessments were conducted by the primary investigator (the first author). The primary investigator randomly assigned the order of two assessments for each participant. Prescribed guidelines and recommendations [39] were followed for standardised oral language and literacy tele-assessments. The two assessments, i.e., in-person as well as tele-assessment were conducted over a gap of 15 to 20 days for each child with ASD. This gap was provided between in-person and tele-assessment to eliminate the practice effect. The following sections provide a summary of procedures followed for both in-person and tele-assessment of language and literacy skills in autistic children. ALD was administered to determine the child’s expressive and receptive language skills. A brief break of 5–10 minutes was given in between the testing. In TELA, oral language assessment was administered at first followed by print knowledge and phonological processing. Since phonological assessment was demanding than other tests, a break of five minutes was given for each child before the starting of the assessment. The investigator recorded the entire session using the built-in audio-video recorder in Zoom videoconferencing software for tele-sessions, while an external mobile phone camera was used to record in-person assessment. Although technical difficulties occurred during the tele-assessment, sessions were not suspended or cancelled. Technical difficulties led to increased time for completion of the tele-assessment for a few children. Whenever technical issues such as audio lag occurred, facilitators encouraged the children for repetition of instructions from the examiner’s end and vice versa. Behavioral issues encountered during a few assessment sessions were handled by providing sensory breaks, movement activities, virtual games and a reward system in collaboration with the parent. Below we describe the settings and procedure for in-person and tele-assessment sessions, respectively.

2.4.1 In-person assessment

The primary investigator and the participant were in the same room within speech therapy clinics, during in-person assessment. The primary investigator conducted an in-person assessment in a room with fewer distractions. Each session lasted for an average duration of 40 to 60 minutes, including the breaks and practice trials. Maximum of two breaks were given during each session which lasted for 5 minutes each. During the break time, the clinician engaged with the child in play activities with toys or a game requiring movement. The entire assessment was completed on an average of three to four sessions. Oral language and literacy assessments were conducted according to the standard procedures recommended in the test manuals. The SLP scored the test according to the scoring guidelines given in the test manual. Whenever required, the SLP provided positive reinforcement to the child during testing.

2.4.2 Tele-assessment

Before carrying out the tele-assessment, the investigator completed several trial tests with a typically developing child to familiarise herself with tele-assessment procedures. The investigator delivered the tele-assessment from the department clinics, and the participants accessed it from their homes. During the assessment session, the child’s parent was also present in the room as facilitator. The SLP advised the facilitator to refrain from providing any prompt or support during the assessment. Parents were instructed to choose a quiet and distortion-free room to participate in the tele-assessment. The investigator initially checked and adjusted for the positioning of the child within the camera view, ensured sufficient lighting and minimum distraction during testing session(s). Each session lasted for an average duration of 40 to 60 minutes including the breaks and practice trials. Maximum of three breaks were given during each session which lasted for 5 to 7 minutes. During the break time, the clinician instructed the parent to take the child for a movement break. Child was also engaged in online games during the breaks based on the interest of the participant. On an average, the entire assessment was completed over six to seven sessions. The saved video recordings of tele-assessment sessions (both language and literacy) were scored separately by the first author and an independent SLP who is not part of the study, as per the test manual.

2.4.3 Training of the facilitator

The facilitator plays a vital role in the commencement of the tele-assessment of children [23]. In the present study we employed a model known as ‘professional-facilitator-patient’ to administer the tele-assessment of oral language and literacy skills [40]. In our study, parents assumed the role of a facilitator. Prior to the commencement of the standardised oral language and literacy assessment, the facilitator was given online training on following three significant skills, namely a) modification of the test environment, b) handling technology, and c) ways to engage the child virtually [23]. Modifying the test environment includes choosing a quiet space for assessment, sufficient lighting, minimal background noise and positioning the child within the camera view. Training on handling technology involves aspects such as a fully charged laptop/desktop, secure and stable internet connectivity, basic troubleshooting of the laptop, use of a headset with a microphone if the child allows and features of the video conferencing software (screen sharing, screen drawing, annotation, shared mouse control). To engage the child virtually during the assessment, the following considerations were made: complete the test over several meetings, boost the child’s interest by providing reinforcements whenever required, and consider providing movement activities or breaks and virtual games during the assessment. In addition, facilitators were instructed not to give any prompts or support and avoid the repetition of instructions during testing. Additional considerations during the tele-assessment included visual schedules to prepare the child for tele-assessment and a total body warm-up to get the child focused during the session.

2.4.4 Modification of the response elicitation methods

Certain adjustments were made for response elicitations during the tele-assessment sessions for receptive language and literacy tasks, such as ‘point to the ball’ and ‘point to the word’, ‘Teacher’, since pointing to the desired stimulus item was not possible during tele-assessment. With screen sharing of the stimuli, the shared mouse control allowed the child to touch the correct answer with the mouse pointer. Another method used was the annotation feature of the software to circle the correct response during screen sharing of the test stimuli.

2.5 Data analysis

Table 1
Participants characteristics

Participants	Age (years)	Gender	IQ	Autism severity	Total score (ALD)		Total score (TELA)
					In-person	Tele-assessment	In-person	Tele-assessment
1	7.2	Female	80	Mild to moderate	72	71	297	298
2	5.6	Male	81	Mild to moderate	61	61	265	265
3	6.4	Male	83	Mild to moderate	63	62	277	278
4	7.8	Male	84	Mild to moderate	81	81	421	420
5	7.7	Male	85	Mild to moderate	74	73	341	342
6	6.9	Male	87	Mild to moderate	76	76	402	402
7	7.1	Male	88	Mild to moderate	68	68	314	314
8	7.5	Male	89	Mild to moderate	68	68	301	300
9	6.7	Male	81	Mild to moderate	64	64	281	280
10	6.3	Male	80	Mild to moderate	62	62	264	263

Table 2

Emergent literacy and language measures comparison between in-person and tele-assessment in autistic children

Measure	In-person ( $n=$ 10)				Tele-assessment ( $n=$ 10)				$t$ (9)	$p$ value	Cohens $d$
	Mean	SD	Range	SE	Mean	SD	Range	SE
Overall EmLS	316	55.5	264–421	17.5	316	55.4	264–423	17.5	0.36	0.72 ${}^{\text{NS}}$	0.11
OL	127	27.3	99–167	8.64	127	27.3	99–167	8.65	0.0	1.00 ${}^{\text{NS}}$	0.0
PK	138	9.67	123–155	3.06	138	9.70	123–156	3.07	0.0	1.00 ${}^{\text{NS}}$	0.0
PP	52.0	23.6	25–90	7.46	51.9	23.4	25–90	7.42	0.55	0.59 ${}^{\text{NS}}$	0.17
Overall language score	68.9	6.69	61–81	2.12	68.6	6.67	61–81	2.11	1.96	0.08 ${}^{\text{NS}}$	0.62
RLS	35.3	3.53	31–42	1.12	35.1	3.35	31–42	1.06	1.50	0.16 ${}^{\text{NS}}$	0.47
ELS	33.6	3.41	30–39	1.08	33.5	3.50	30–39	1.11	0.55	0.59 ${}^{\text{NS}}$	0.17

EmLS: Emergent Literacy Skill, OL: Oral Language, PK: Print Knowledge, PP: Phonological Processing, RLS: Receptive Language Score, ELS: Expressive Language Score; NS: Not Significant at $p>$ 0.05.

Obtained data was scored as per the test manual and tabulated in excel sheet for analysis using SPSS software version 16 for windows. Parametric statistics were used to compare the scores from in-person and tele-assessment. A paired $t$ -test was used to compare oral language and literacy scores between in-person and tele-assessment sessions. Bland-Altman’s plot was used to evaluate the levels of agreement between scores of oral language and literacy acquired through in-person and tele-assessment. A 95% of agreement was computed for overall emergent literacy, receptive language and expressive language scores (average difference $\pm$ 1.96 standard deviations of the difference). Inter-rater reliability was determined for total language and literacy (TELA and ALD) tele-assessment from the scoring given by the primary investigator (SLP) and an independent SLP. Inter-rater reliability was also performed at the TELA and ALD subtest level. Based on a two-way mixed effect model, absolute agreement and average measurement, the intraclass correlation coefficient (ICC) was employed to evaluate the degree of reliability on language and literacy tele-assessment scores between the two SLPs. The degree of inter- and intra-rater reliability was assessed using the following criteria: ICC $<$ 0.50 $=$ poor reliability; 0.50–0.75 $=$ moderate reliability; $>$ 0.75 $=$ excellent reliability [41]. Furthermore, the audio-video recordings of tele-assessment of emergent literacy and oral language were examined by the primary investigator to observe the frequency of echo, audio-video delay, and blurring during tele-assessment at both ends.

3. Results

Present study aimed to examine the reliability of standardised oral language and emergent literacy assessments delivered via telepractice in autistic children. The study included ten children aged 4 to 8 years (Mean $=$ 6.96, S.D $=$ 0.61) with a diagnosis of ASD. Table 1 shows the participant characteristics in detail. Ten autistic children participated in in-person as well as tele-assessment sessions for emergent literacy and oral language measures. Table 2 presents the comparison of emergent literacy and oral language scores between the two modes of assessment (in-person and tele-assessment) using the paired $t$ -test.

Results of the test revealed no significant difference in overall emergent literacy scores between in-person (M $=$ 316, SD $=$ 55.5) and tele-assessment (M $=$ 316, SD $=$ 55.4); $t$ (9) $=$ 0.36, $p=$ 0.72). Further, subtests of emergent literacy assessment such as oral language, print knowledge and phonological processing also showed no significant difference between in-person and tele-assessment scores. On similar lines, comparison of overall language scores revealed no significant difference between in-person (M $=$ 68.9, SD $=$ 6.69) and tele-assessment (M $=$ 68.6, SD $=$ 6.67); $t$ (9) = 1.96, $p=$ 0.08). Subtests of language assessment such as receptive language and expressive language skills too showed no significant difference between in-person and tele-assessment scores.

Figure 1.

Bland-Altman plot depicting agreement between In-person and tele-assessment across overall and subtests scores of the Test of emergent literacy assessment (TELA).

Agreement between in-person and tele-assessment scores was computed using Bland-Altman’s plot. Figures 1 and 2 show Bland-Altman plots that provide visualization of mean differences and the limits of agreement (Mean $\pm$ 1.96 SD of the difference) of emergent literacy and oral language scores, obtained through in-person and tele-assessment modes, respectively. The solid lines in each plot depict the mean difference or bias between measures. For the majority of comparisons, the overall bias was close to 0. The dotted lines in each plot depict the limits of agreement. A visual observation of the plots indicate that all the data points were within the limits of agreement for the overall emergent literacy score ( $-$ 3.06 and $+$ 3.06; Fig. 1a) and overall language score ( $-$ 0.64 and 1.24; Fig. 2a). Also, most data points lie close to the mean difference, indicating less bias between the in-person and tele-assessment. Figure 1b, c and d, display the level of agreement on the Bland Altman plot for subtests of emergent literacy assessment. The limits of agreement for the subtests such as oral language, print knowledge, and phonological processing were ( $-$ 0.92, $+$ 0.92), ( $-$ 0.92, $+$ 0.92) and ( $-$ 1.01, $+$ 1.11), respectively. All the scores for the subtests of emergent literacy were within the limits of the agreement, except for two values each for oral language and print knowledge. Bland Altman plots for overall language score (as shown in Fig. 2), receptive language score and expressive language scores show that the values lie within the limits of agreements ( $-$ 0.64 to $+$ 1.24, $-$ 0.62 to $+$ 1.02 and $-$ 0.62 to $+$ 1.24).

Table 3

Inter-rater reliability for language and literacy tele-assessment scoring between two SLPs

Language and	Intra-class correlation	95% CI		$p$ value
literacy measures	coefficient (ICC)	Lower limit	Upper limit
Total TELA score	1.00	1.00	1.00	$<$ 0.001 ${}^{*}$
Oral language	1.00	1.00	1.00	$<$ 0.001 ${}^{*}$
Print knowledge	1.00	0.99	1.00	$<$ 0.001 ${}^{*}$
Phonological processing	1.00	–	1.00	$<$ 0.001 ${}^{*}$
Total ALD score	0.99	0.99	0.99	$<$ 0.001 ${}^{*}$
Receptive language	0.99	0.98	0.99	$<$ 0.001 ${}^{*}$
Expressive language	0.99	0.98	0.99	$<$ 0.001 ${}^{*}$

${}^{*}$ Significant at $p<$ 0.001.

Figure 2.

Bland-Altman plot depicting agreement between in-person and tele-assessment across overall and subtests scores of the Assessment of language development (ALD).

Table 4

Frequency of audio-video distortions during tele-assessment

Observed distortions	( $n=$ 11)	%
Video delay	4	36.3
Audio delay	3	27.2
Video blurring	2	18.1
Echo	2	18.1

Intra-class correlation (ICC) analysis indicated excellent reliability (0.99–1.00; 95% CI) when comparing total TELA and ALD tele-assessment scores by two SLPs. ICC analysis for TELA and ALD subtests also indicated excellent reliability (0.99–1.00; 95% CI). ICC results are presented in Table 3. The distortions noted in the audio-video recordings of tele-assessment are presented in Table 4. The most common distortion observed during tele-assessment was video delay ( $n=$ 4, 36.3%) followed by audio delay ( $n=$ 3, 27.2%), video blurring ( $n=$ 2, 18.1%), and echo ( $n=$ 2, 18.1%), respectively.

4. Discussion

The current study aimed to examine the reliability of standardised oral language and emergent literacy assessments delivered via telepractice in autistic children. While tele-assessment provides several benefits, such as a reduction in wait time, enhanced service efficiency and less expensive service delivery [21], its feasibility and reliability remains a concern. Results of the study indicate a high level of agreement between in-person and tele-assessment as well as good inter-rater reliability for tele-assessment of emergent literacy and oral language measures. The findings suggest that tele-assessment of language and literacy measures in autistic children are as reliable as in-person assessment. Our results are in line with the reports of Sutherland et al., (2019) on tele-language assessment in autistic children despite having a different methodology. Our results are also congruent with studies which demonstrate a high level of agreement for oral language and literacy assessment in children with communication disorders and reading and writing difficulties between in-person and tele-assessment [18, 22, 23, 42, 43]. The current study, thus, adds to the limited literature on reliability of conducting a tele-assessment of emergent literacy and oral language skills in autistic children. This is perhaps one of the first studies comparing emergent literacy measures between in-person and tele-assessment in autistic children.

SLPs perceive that paediatric cases, especially autistic children, are the most challenging group for tele-assessment and intervention [24]. Hence, special considerations or modifications has to be adapted while conducting tele-assessment of autistic children. Certain special accommodations, such as completion of the standardized test over several meetings, providing reinforcement to boost the child’s interest, providing virtual games, sensory breaks and movement breaks whenever required, and providing movement activities or breaks and virtual games during the assessment, could make it less challenging for SLPs [39]. The use of a visual schedule, which is a routine support for autistic children, could be implemented during a tele-assessment session to reduce anxiety and behavioural issues. In addition, modification of elicitation methods for receptive and expressive language tests such pointing to desired stimulus on screen share, clicking on the desired item and annotate function in zoom, play an important role in making tele-assessment reliable and feasible for use [20]. Employing special accommodations in tele-assessment is vital because, the successful implementation of tele-assessment depends on the child’s motivation, focus, and attention [23]. The implementation of the above accommodations, however, requires adequate preparation and extensive training of the facilitators.

We employed the ‘professional-facilitator-patient’ model, in the current study, which, eased the conduct of tele-assessment for autistic children [40]. Similar studies on oral language assessments employing telepractice in schools (during pre-pandemic time), report presence of a secondary clinician at the child’s site for better facilitation [18, 44]. It is also recommended to implement the ‘professional-facilitator-patient’ model, that could overcome a few of the challenges related to tele-assessments, such as technological issues, test environment modification, manage challenging behaviours, provide feedback to clinicians regarding child’s response, equipment failures and child-related factors. Training of the facilitator ensures an easy transition of the tele-assessment for autistic children [23]. The role of facilitator or a caregiver in tele-assessment of autistic children and the guidelines and practices to be followed during a tele-assessment should be explored in future research.

Lack of professional training among SLPs on telepractice has been reported by several studies in the past [45, 46, 47]. Taking into account the special accommodations and specific guidelines recommended during a standardized tele-language and literacy assessment, the present study affirms the need for formal training among Indian SLPs. Thus, incorporation of guidelines on tele-assessment and intervention into the speech language pathology curriculum is advocated [10, 24]. While there exists general guideline on telepractice for speech, and language pathology service by professional organizations, such as ISHA in India, present study points to the need for specific guidelines and tools for standardized tele-assessment to be formulated, for specific conditions, such as autism.

Considering the limited literature and challenges of telehealth services in a LMIC country like India, the present study explores the potential of telepractice to overcome challenges such as transportation, non-availability of services, lack of trained professionals to serve autistic children. Results of our study also offer a framework for future research on feasibility and administration of standardized language and literacy assessment in Indian autistic children. The study also highlights the role of caregiver or facilitator in tele-assessment of standardized literacy and language in autistic children. The consideration or modifications to tele-assessment in the present study do not serve as a conclusive guideline for conducting tele-assessment in autistic children. However, it could act as a foundation for clinicians and researchers seeking directions to direct their tele-assessment practice.

4.1 Strengths and limitations

Present study is one of the first attempts to compare emergent literacy measures between in-person and tele-assessment in autistic children. Finding from the present study confirm the feasibility and reliability of conducting tele-assessment for oral language as well as emergent literacy measures with the accommodation of special considerations in autistic children, who are considered as one of the most challenging groups to handle via telepractice. However, the entire spectrum of autistic children is not well represented in the present study due to the small sample size. Our finding, could therefore be considered as preliminary attempt that suggests the feasibility of tele-assessment of oral language and literacy in autistic children. Thus, replication of the study with a large sample of autistic children with varying severity, cognitive skills and language abilities is recommended. While, most of the tele-assessment studies include a remote SLP to assess the interrater reliability of test scores [11, 18, 23], it was difficult to employ another remote SLP at the child’s end for tele-assessment in our study during the pandemic. Future research may explore the feasibility and reliability of standardized language and literacy tele-assessment with in-person and online SLPs at both ends. Besides, the current study did not consider the perceptions and satisfaction of the facilitator or parent on tele-assessment. Hence, future studies on the tele-assessment of oral language and literacy in autistic children could evaluate the perceptions and satisfaction of parents or facilitators. Future research should also extend the feasibility of administration of standardised oral language and literacy assessment tools in various languages via telepractice which would aid the practicing SLPs.

5. Conclusion

The findings of the present study establish the feasibility and demonstrate reliability of standardized oral language and literacy tele-assessment in autistic children by reporting a high level of agreement between in-person and tele-assessment. Our study adds to the growing body of evidence that tele-assessment can be used for language and literacy evaluation in autistic children. Findings of the present study, therefore, provide direction for researchers and clinicians to guide them in validation of other standardized assessment via telepractice. Our findings further provide implications for practicing and prospective SLPs in telepractice for improving their knowledge base, imparting adequate training to the facilitator, and making use of special accommodations for the successful implementation of tele-assessment in autistic children.

Author contributions

HSM collected the data, analysed and drafted the manuscript. ST conceptualized, interpreted and discussed the results. ST also corrected and reviewed the manuscript draft. VG helped in analysis and interpretation of the data.

Ethical considerations

The authors declare that the study followed ethical guidelines for human research. Written informed consent was obtained from all participants. The study protocol was approved from the Institutional Ethics Committee on human research.

Footnotes

Acknowledgments

The authors would like to thank the participants for their time and support. The authors also acknowledge constructive comments and inputs from Dr. KS Prema towards this work.

Conflict of interest

The authors have no conflicts of interest to report.

References

Hodges

Fealko

Soares

. Autism spectrum disorder: definition, epidemiology, causes, and clinical evaluation. Transl Pediatr. 2020; 9(Suppl 1): S55-65. doi: 10.21037/tp.2019.09.09.

Bolton

Golding

Emond

Steer

. Autism spectrum disorder and autistic traits in the Avon Longitudinal Study of Parents and Children: precursors and early signs. J Am Acad Child Adolesc Psychiatry. 2012; 51(3): 249-260.e25. doi: 10.1016/j.jaac.2011.12.009.

Ramdoss

Mulloy

Lang

O’Reilly

Sigafoos

Lancioni

, et al. Use of computer-based interventions to improve literacy skills in students with autism spectrum disorders: A systematic review. Res Autism Spectr Disord. 2011; 5(4): 1306-1318. doi: 10.1016/j.rasd.2011.03.004.

Law

Garrett

Nye

. The efficacy of treatment for children with developmental speech and language delay/disorder: a meta-analysis. J Speech Lang Hear Res. 2004; 47(4): 924-943. doi: 10.1044/1092-4388(2004/069).

Smythe

Zuurmond

Tann

Gladstone

Kuper

. Early intervention for children with developmental disabilities in low and middle-income countries – the case for action. Int Health. 2021; 13(3): 222-231. doi: 10.1093/inthealth/ihaa044.

Kaur

Boobna

Kallingal

. Effect of COVID-19 lockdown on indian children with autism. Res Dev Disabil. 2022; 125: 104230. doi: 10.1016/j.ridd.2022.104230.

Menezes

Shwetha

D’Mello

. A Study on the Implications of COVID-19 Pandemic on Families with Autism Spectrum Disorder. Int J Heal Sci Pharm. 2021; 5(2): 28-41. doi: 10.47992/IJHSP.2581.6411.0072.

Levante

Petrocchi

Bianco

Castelli

Colombi

Keller

, et al. Psychological impact of covid-19 outbreak on families of children with autism spectrum disorder and typically developing peers: An online survey. Brain Sci. 2021; 11(6): 808. doi: 10.3390/brainsci11060808.

Theodoros

. Telerehabilitation for service delivery in speech-language pathology. J Telemed Telecare. 2008; 14(5): 221-224. doi: 10.1258/jtt.2007.007044.

10.

Mohan

Anjum

Rao

PKS

. A Survey of Telepractice in Speech-Language Pathology and Audiology in India. Int J Telerehabilitation. 2017; 9(2): 69-80. doi: 10.5195/ijt.2017.6233.

11.

Sutherland

Trembath

Hodge

Rose

Roberts

. Telehealth and autism: Are telehealth language assessments reliable and feasible for children with autism?. Int J Lang Commun Disord. 2019; 54(2): 281-291. doi: 10.1111/1460-6984.12440.

12.

Sutherland

Trembath

Roberts

. Telehealth and autism: A systematic search and review of the literature. Int J Speech Lang Pathol. 2018; 20(3): 324-336. doi: 10.1080/17549507.2018.1465123.

13.

Tucker

. Perspectives of speech-language pathologists on the use of telepractice in schools: the qualitative view. Int J telerehabilitation. 2012; 4(2): 47-60. doi: 10.5195/ijt.2012.6102.

14.

Hewitt

Rodgin

Loring

Pritchard

Jacobson

. Transitioning to telehealth neuropsychology service: Considerations across adult and pediatric care settings. Clin Neuropsychol. 2020; 34(7-8): 1335-1351. doi: 10.1080/13854046.2020.1811891.

15.

Kester

. Conducting Student Speech-Language Evaluations Via Telepractice. 2020 [cited 2023 Feb 9]; Available from: https://leader.pubs.asha.org/do/10.1044/leader.SCM.25062020.36/full/.

16.

Garrisi

King

Hillyer

Gaab

. General recommendations and guidelines for remote assessment of toddlers and children. OSF Preprint. 2020 [cited 2023 Feb 9]. Available from: https://osf.io/preprints/wg4ef/.

17.

Hamner

Salorio

Kalb

Jacobson

. Equivalency of In-Person Versus Remote Assessment: WISC-V and KTEA-3 Performance in Clinically Referred Children and Adolescents. J Int Neuropsychol Soc. 2022; 28(8): 835-844. doi: 10.1017/s1355617721001053.

18.

Waite

Theodoros

Russell

Cahill

. Internet-based telehealth assessment of language using the CELF-4. Lang Speech Hear Serv Sch. 2010; 41(4): 445-458. doi: 10.1044/0161-1461(2009/08-0131).

19.

Ciccia

Whitford

Krumm

McNeal

. Improving the access of young urban children to speech, language and hearing screening via telehealth. J Telemed Telecare. 2011; 17(5): 240-244. doi: 10.1258/jtt.2011.100810.

20.

Schmitt

Tambyraja

Thibodeaux

Filipkowski

. Feasibility of Assessing Expressive and Receptive Vocabulary via Telepractice for Early Elementary-Age Children with Language Impairment. Lang Speech Hear Serv Sch. 2022; 53(2): 445-453. doi: 10.1044/2021_LSHSS-21-00057.

21.

Eriks-Brophy

Quittenbaum

Anderson

Nelson

. Part of the problem or part of the solution? Communication assessments of Aboriginal children residing in remote communities using videoconferencing. Clin Linguist Phon. 2008; 22: 589-609. doi: 10.1080/02699200802221737.

22.

Nelson

Plante

. Evaluating the Equivalence of Telepractice and Traditional Administration of the Test of Integrated Language and Literacy Skills. Lang Speech Hear Serv Sch. 2022; 53(2): 376-390. doi: 10.1044/2022_LSHSS-21-00056.

23.

Raman

Nagarajan

Venkatesh

Monica

Ramkumar

Krumm

. School-based language screening among primary school children using telepractice: A feasibility study from India. Int J Speech Lang Pathol. 2019; 21(4): 425-434. doi: 10.1080/17549507.2018.1493142.

24.

Bhattarai

Sanghavi

Abhishek

. Challenges in Delivering Tele-Practice Services for Communication Disorders Among Audiologists and Speech Language Pathologists. Indian J Otolaryngol Head Neck Surg. 2022; 74: 4360-4365. doi: 10.1007/s12070-021-03032-7.

25.

Pearson Assessments. The Use of Facilitators in a Remote Test Administration. San Antonio: 2020 [cited 2023 February 9]; Available from: https://www.pearsonassessments.com/content/dam/school/global/clinical/us/assets/telepractice/facilitators-role-w-telepractice.pdf.

26.

Douglass

Lowman

Angadi

. Defining Roles and Responsibilities for School-Based Tele-Facilitators: Intraclass Correlation Coefficient (ICC) Ratings of Proposed Competencies. Int J Telerehabilitation. 2021; 13(1): e6351. doi: 10.5195/ijt.2021.6351.

27.

Kryszak

Albright

Fell

Butter

Kuhlthau

. Clinician Perspectives on Telehealth Assessment of Autism Spectrum Disorder During the COVID-19 Pandemic. J Autism Dev Disord. 2022; 52(12): 5083-98. doi: 10.1007/s10803-022-05435-z.

28.

Gibbs

Cai

Aldridge

Wong

. Autism assessment via telehealth during the Covid 19 pandemic: Experiences and perspectives of autistic adults, parents/carers and clinicians. Res Autism Spectr Disord. 2021; 88: 101859. doi: 10.1016/j.rasd.2021.101859.

29.

Tiwari

Kulkarni

Zhang

Khan

Zhang

. Utilization of telehealth services in low- and middle-income countries amid the COVID-19 pandemic: a narrative summary. Glob Health Action. 2023; 16(1): 2179163. doi: 10.1080/16549716.2023.2179163.

30.

Indian Speech and Hearing Association. Telepractice guidelines for audiology and speech, language pathology services in India. Mysore; 2020 [cited 2023 February 9]. Available from: https://www.ishaindia.org.in/downloads/TELEPRACTICE-GUIDELINES.pdf.

31.

Waite

Theodoros

Russell

Cahill

. Assessment of children’s literacy via an Internet-based telehealth system. Telemed J e-health Off J Am Telemed Assoc. 2010; 16(5): 564-75. doi: 10.1089/tmj.2009.0161.

32.

Schopler

Bourgondien ME Van Wellman

Love

. Childhood Autism Rating ScaleTM (CARSTM2). Western Psychological Services. 2nd Ed. 2nd Ed. Los Angeles, Second Edition Manual. Los Angeles: Western Psychological Services; 2010. Available from: https://www.wpspublish.com/cars-2-childhood-autism-rating-scale-second-edition.html.

33.

Seguin

. Seguin form board test. 2010 [cited 2023 Feb 9]; Available from: https://prasadpsycho.com/product/seguin-form-board-test-sfbt-3-to-15-years/.

34.

Bharathraj

Malin

. Mysore, India: Vineland Social Maturity Scale and Manual, Indian Adaptation. 1992 [cited 2023 Feb 9]; Available from: https://prasadpsycho.com/product/vineland-social-maturity-scale-vsms/.

35.

Marian

Blumenfeld

Kaushanskaya

. The Language Experience and Proficiency Questionnaire (LEAP-Q): assessing language profiles in bilinguals and multilinguals. J Speech Lang Hear Res. 2007; 50(4): 940-967. doi: 10.1044/1092-4388(2007/067).

36.

Lakkanna

Venkatesh

Bhat

. Assessment of language development. Bakersfield, CA: 2008 [cited 2023 Feb 9]; Available from: https://mup.manipal.edu/product/assessment-of-language-development/.

37.

Khurana

Prema

. Development of emergent literacy in Kannada speaking English language learners. Mysore (IND): University of Mysore; 2011 [cited 2023 Feb 9]. Available from: https://hdl-handle-net.web.bisu.edu.cn/10603/105118.

38.

American speech and hearing association. Telepractice. ASHA [cited 2023 Feb 03]. Available from: https://www.asha.org/practice-portal/professional-issues/telepractice/.

39.

Wood

Bhalloo

McCaig

Feraru

Molnar

. Towards development of guidelines for virtual administration of paediatric standardized language and literacy assessments: Considerations for clinicians and researchers. SAGE open Med. 2021; 9: 1-19. doi: 10.1177/20503121211050510.

40.

Shankar

Ramkumar

Kumar

. Understanding the implementation of telepractice in speech and language services using a mixed-methods approach. Wellcome Open Research. 2022; 7: 46. doi: 10.12688/wellcomeopenres.17622.2.

41.

Koo

. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016; 15(2): 155-163. doi: 10.1016/j.jcm.2016.02.01.

42.

Waite

Cahill

Theodoros

Busuttin

Russell

. A pilot study of online assessment of childhood speech disorders. J Telemed Telecare. 2006; 12(3): 92-94. doi: 10.1258/135763306779380048.

43.

Hodge

Sutherland

Jeng

Bale

Batta

Cambridge

, et al. Literacy Assessment Via Telepractice Is Comparable to Face-to-Face Assessment in Children with Reading Difficulties Living in Rural Australia. Telemed J e-health Off J Am Telemed Assoc. 2019; 25(4): 279-287. doi: 10.1089/tmj.2018.0049.

44.

Fairweather

Lincoln

Ramsden

. Speech-language pathology teletherapy in rural and remote educational settings: Decreasing service inequities. Int J Speech Lang Pathol. 2016; 18(6): 592-602. doi: 10.3109/17549507.2016.1143973.

45.

Hao

Zhang

Conner

Lee

. The Evolution of Telepractice Use during the COVID-19 Pandemic: Perspectives of Pediatric Speech-Language Pathologists. Int J Environ Res Public Health. 2021; 18(22): 12197. doi: 10.3390/ijerph182212197.

46.

Alghamdi

Aldhahir

Alqahtani

Siraj

Alsulayyim

Almojaibel

, et al. Healthcare Providers’ Perception and Barriers Concerning the Use of Telehealth Applications in Saudi Arabia: A Cross-Sectional Study. Healthc (Basel, Switzerland). 2022; 10(8): 1527. doi: 10.3390/healthcare10081527.

47.

Singh

Kamarudin

Sharma

. The Implementation of Telepractice by Malaysian Speech-Language Pathologists During the COVID-19 Pandemic. Commun Sci Disord. 2022; 27(1): 239-250. doi: 10.12963/csd.21848.

Reliability of oral language and emergent literacy tele-assessments in autistic children: A preliminary study

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSIONS:

Keywords

1. Introduction

2. Method

2.1 Participants

2.2 Test material(s)/tools

2.3 Equipment

2.4 Procedure

2.4.1 In-person assessment

2.4.2 Tele-assessment

2.4.3 Training of the facilitator

2.4.4 Modification of the response elicitation methods

2.5 Data analysis

Table 1 Participants characteristics

4.1 Strengths and limitations

5. Conclusion

Author contributions

Ethical considerations

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
Participants characteristics