Cognitive and language intervention in primary language impairment: Studying the effectiveness of working memory training and direct language intervention on expansion of grammar and working memory capacities

a Assessment

Due to the study design, we applied two series of measures: repeated measures and pre- and post-test measures. The repeated measures were used to track the changes in WM and language skills during the baseline and the intervention periods. The performances of every participant in selected tasks were measured eight times overall, at one-week intervals (three times at baseline and five times in the intervention period) during each phase. Throughout the intervention period, assessments were carried out at the end of each week and after 3 sessions of intervention, at the end of the third session. In addition to the repeated measures, some WM and morpho-syntactic language measures were used before and after each treatment phase to provide supplementary data for interpreting changes in the repeated measure tasks.

For tracking the changes of WM and grammatical skills throughout the baseline and intervention periods, participants were asked to complete five different tasks repeatedly: two WM tasks, two linguistic tasks and one control task. The repeated measures for WM were a non-word repetition test for Persian children (Soleymani et al., 2014) and backward digit span (BDS) subtests of the Persian version of WISC-IV (Abedi et al., 2013). The Persian non-word repetition test (NWR) consists of two counterpart checklists, where every list contains 25 one- to four-syllable non-words. It is difficult to learn such low probability non-word stimuli, especially for children with PLI; however, to reduce the probability of learning the test’s stimuli at repeated performances, the same checklist was not used in two consecutive sessions of measurement.

To track changes in the morpho-syntactic skills of the participants across the baseline and intervention periods, a picture description task and a sentence completion task were employed.

Fifteen pictures of different scenes, where a few children are carrying out different actions (selected from Shafiei and Koshkhabari-khamene, 2011) were administered to elicit language samples. The audiotaped children’s descriptions of the pictures were transcribed directly into the computer (as we listened to the audiotapes) based upon the conventions of the systematic analysis of language transcripts (described in the SALT programme documents), and all transcriptions were checked again while listening to the tapes. Unintelligible or unanalysable utterances (such as utterances that were broken off) were excluded from the analysis. Then, the mean length of utterances in words (MLU-w) and the percent of grammatically well-formed utterances (GU%) were calculated as indexes of grammatical development. The MLU was calculated by dividing the total numbers of words by the total numbers of utterances of the language sample (Marini et al., 2014). The GU% was calculated by dividing the total number of well-formed grammatical utterances produced within the language sample by the total number of utterances, multiplied by 100 (Marini et al., 2014). An utterance was measured as grammatical if all the obligatory arguments of the verb had been correctly inserted and there were no omissions or substitutions of free or bound morphemes. The reliability of the transcriptions was checked by re-transcribing 20% of the files by a trained speech-language pathology student. The agreement between transcribers was 94% (SD = 2%).

The other task for the repeated measurement of morpho-syntax was the sentence completion task. Sentence completion tasks measure the child’s ability to process inflectional morphology to produce well-structured sentences (Marini et al., 2014). Each stimulus of this task was made of a pair of model and target sentences. The children were asked to listen to the first sentence (the model) and then complete the unfinished second sentence (the target) by assigning the correct morphemes to the verb. Each correct answer was assigned 1 point, with a maximum score of 20. Please see Appendix 1 for more information about the method of constructing the stimuli for this task and determining the validity.

As in the study of Ebert and Kohnert (2009), a control repeated task, keeping the balance on one leg for as long as possible, was included within the design to specify general patterns of performance on tasks repeated over time. Although children with PLI show weakness in gross motor movements (to some extent), there is no evidence for significant effects from language or cognitive treatments on these skills (Ebert, 2014). Therefore, unlike other repeated measures, it was expected that the performance variability on this measure would be minimal across time points, with a nearly stable performance throughout the baseline and treatment phases. The timespan of maintaining balance on each leg was timed using a stopwatch, and the overall score on the balance task was calculated by averaging the balance time on the left and right legs.

Immediately prior to the intervention phases, the children received a series of tests to establish primary language and WM scores. The participants were also re-evaluated by these tests after the treatment phases to verify the effects of the treatment programmes. The Persian syntax comprehension test (Mohamadi et al., 2015) and the photographic expressive Persian grammar test (Haresabadi et al., 2016) were used to evaluate the effects of both training programmes on the participant’s receptive and expressive morpho-syntactic skills. The Persian syntax comprehension test is a picture selection test that encompasses 96 items and four pictures (one target picture and three distractor pictures) for each item and assesses the comprehension of 4- to 6-year-old children on 24 important syntactic structures of the Persian language. The photographic expressive Persian grammar test is a fast and easy to administration test that comprises 40 image items and is designed to evaluate the production of important morph-syntactic structures of the Persian language in 4- to 6-year-old children. Although both tests have been designed, validated and normed for appraising the receptive/expressive morpho-syntactic skills of 4- to 6-year-old Persian-speaking children, these two tests assess the most basic and important Persian syntactic structures that are extremely difficult for children with PLI to learn, even in older age groups. Therefore, considering this issue and due to the absence of formal tests for assessing Persian morpho-syntactic skills, we applied these tests as available valid tasks to evaluate the changes of morpho-syntactic proficiencies of the participants as the result of the treatment programmes.

The evaluation of WM memory skills in the pre- and post-treatment periods was based on the word list recall and backward word span subtests of the unpublished Persian working memory battery for 5- to 15-year-olds (Delfani et al., in press). The word list recall task tests the pSTM span. Children listened to lists of familiar monosyllabic words that varied in length (from 2 to 8 words), and a maximum of four lists were presented at each length. They were then asked to recall the words from each list in the correct serial position with full accuracy. Phoneme substitution errors were accepted when the errors were part of the child’s habitual articulation pattern. If the child correctly recalled at least two lists at any given length, longer lists of the words were subsequently offered. The task score was the sum of correct recalled lists. The structure of the backward word span task for evaluating the verbal working memory span was identical to the word list recall task, except that the words of this task were two-syllable words, and the participants were asked to recall the list of words in the reverse sequence. For delivering these two tasks, as well as non-word repetition and backward digit span tasks, the test stimuli were recorded on CD. Moreover, though the scoring was done during the test administration, to ensure the accuracy of scoring the children’s responses were also digitally audio-recorded and later re-scored. Additionally, 20% of the children’s recorded data were re-scored by a trained speech-language pathology student who was blind to the treatment assignment. In the case of disagreement between scorers due to the low intelligibility of the recorded voice, the original scoring was retained. Inter-scorer reliabilities were above 85% in all tasks.

The assessor was also the person who provided the intervention programmes; therefore, she was not blind to intervention allocation. However, to reduce the risk of bias, during the test administration the participants’ answers were merely scored (1 for correct and 0 for incorrect); calculating the final scores and entering them into the computer was postponed until after completion of the research phases and was carried out by an employee outside of the research team who was completely blind to the study design.

b Intervention programmes

The treatment programmes were both conducted on a fixed-time basis to offer the same opportunity for learning to all children across the two treatments. In the first phase of the intervention, all participants completed a no-treatment baseline period of three sessions, followed by a one week-long gap, and then 15 sessions (three 60-minute sessions per week for five weeks) of WM training. After completing the WM training phase and a break period of five weeks, the participants underwent the second phase of intervention, which, similar to the first phase, started with three no-treatment baseline sessions, followed by 15 sessions of direct language intervention. The treatment programmes were performed on an intense basis (180 minutes per week), and the two treatment programmes differed only in the content of the intervention (WM vs. language), while all other aspects of the intervention programmes were identical. The durations of the sessions where repeated measures were carried out (the last session in each week) were longer than the other sessions by approximately 15 minutes.

Individualized training for each participant in both intervention programmes was conducted by the first author, a speech-language pathologist with good clinical experience in working with children with PLI. The physical environment of the intervention was a quiet room with enough light, an appropriate temperature and away from distractions. To maintain motivation and participation of the participants, encouragement and positive reinforcements were provided. Details of the strategies, tasks and stimuli of both treatment programmes were fully explained in drafts of the WM and language treatment protocols and were then evaluated and verified by two experts within the field of speech and language pathology and psychology.

The first phase of intervention, WM training, was aimed to improve the capacities of pSTM and executive WM, especially grammatical-morphology and syntax, which are the targets of language evaluations, without directly treating language. The intervention programmes were implemented in two interactive computerized ways and involved repeated practice that required storage or concurrent processing and storage of stimuli. Table 2 shows the selected stimuli and training activities. A part of each intervention session was devoted to computerized training because it was attractive to children and led to more involvement in the training. However, computerized training could not cover all of the training goals. During the interactive training, important WM strategies (including rehearsal, use of concrete images to support the process of rehearsal, dual encoding between visual and verbal stimuli and mental imagery of presented words) were also used to approach the tasks, though the training was not designed to explicitly teach strategies. For each child, the starting point for training was in the difficulty level of tasks at which the child could properly complete 10 to 30% of the activities. The complexity levels of activities were increased when the participant was able to complete the tasks presented at a specific difficulty level without the help of the therapist in 75% of the cases. Increasing the task’s difficulty levels was done by adding a new item to the chain of stimuli, increasing the length of syllables/non-word stimuli that needed to be recalled or by adding visual or auditory distractors.

OPEN IN VIEWER

Table 2.

Working-memory (WM) training programme: Method of delivery, stimuli and activities.

Delivery	Time	Training stimuli	Training activities
Interactive	The beginning 40 minutes of each training session	- non-words - letters - nonsense syllables - digits a list of 50 very familiar, well-learned and imaginable words from animals, fruits or object categories	- words/non-words/speech sounds/digit lists matching - repeating the sequences of phonemes/syllables/digits/words/single-syllable non-words in forward or backwards order - repetition of one- to four-syllable non-words - combining the two one-syllabic non-words into a single non-word and repeat the chain of non-words that was made from the two-syllable non-words.
Computerized (auditory components of WM training software, Persian version of RoboMemo, a WM training program)	The last 20 minutes of each training session	- letters - digits - 12 very common and familiar words (involved in the 50-word list of interactive training stimuli)	- listening to a sequence of 2 to 9 words and selecting their image from among 12 images in forward or backwards order. - repeating a sequence of 2 to 9 digits/letters in forward or backwards order

Although a number of high-frequency words were used as training stimuli, the combinations of words or sentences (that are the basic elements for grammatical learning) were not the treatment stimuli. Sentences were avoided as much as possible, and methods of stimulating and facilitating language learning were not used. Short conversations between the therapist and the child for greeting, teaching how to execute the activities, and giving feedback were inevitable.

The latter phase of intervention, language intervention, was intended to enhance the morpho-syntactic skills of participants by a non-computer-based face-to-face intervention. The focus of intervention activities was on the areas of grammatical morphology and syntax, which according to the literature are impaired in elementary school-aged Persian children with PLI and are also more resistant to language therapy. Therefore, as many linguistic targets as possible (presented in Table 3) for each participant were addressed by a range of evidence-based effective techniques for presenting grammatical targets to children with PLI. The imitation, modeling and focused stimulation techniques were administered to teach the grammatical structures that were not present in the children’s speech; conversational recasting was also applied to facilitate the generalization of target grammatical structures to spontaneous speech. Common clinical tools, such as comic books and images of objects, functions, events and scenes, dolls, doll houses and toy forest animals were used to guide the trainings. Appendix 2 presents the list of the targets of language intervention for each participant along with a brief report of his/her characteristics and attitudes.

OPEN IN VIEWER

Table 3.

Focused language targets in the second phase of intervention.

Main goals	Addressed targets
Producing longer and more complex sentences	- elaboration of noun phrases:  • use of determiners  • use of adjectives to modify nouns - elaboration of verb phrases:  • use of prepositional phrases and relative clauses  • adding auxiliary verbs  • adding adverbs - teaching formation of compound sentences by adding and/or/but between independent sentences - teaching formation of complex sentences by embedding nominal or adverbial clauses
Helping children to speak in grammatically well-formed sentences by targeting the following:	- correct use of verb arguments (basic sentence components) - proper use of function words - subject–verb agreement - correct use of pronouns/tense-markers/question words - sentences with negative, passive and causative verbs - correct use of superlatives and comparatives

Regarding the suggestion of Ebbels (2014), for each child the grammatical structures that were produced in spontaneous speech but with errors in more than 50 percent, or the structures that were not produced but were understood by the child were a priority for intervention. The difficulty level of the activity was increased when at least 75% of the child’s production of targeted structure was accurate; for structures with lower accuracy in production, additional support was provided. Based on Eisenberg (2014), the proficiency level was adjusted to 80–90% production accuracy.

a Efficacy of WM training on WM and morpho-syntax skills

As shown in Figures 1 and 2 and Table 4 (based on PND and SMD scores), all participants showed clear evidence of improvement in working memory tasks, including NWR and BDS, during the WM training. There are notable differences between the levels of baseline and training for all participants in both tasks; there are no or very little overlaps between the scores of every participant in the baseline and training phase. Following the introduction of the training, the participants’ performances in NWR tasks were improved immediately and continued with an upward or variable trend line with a positive slope. Similar patterns of performance could be seen for BDS among the participants. However, the performance pattern trends of participants 8 and 9 were slightly different from the others. For these two participants, a sudden increase in performance could be seen after the onset of training, but the score remained constant throughout the training phase, with zero slope.

Although visual analysis of the data indicates notable variability among participants, the performances of all participants show a significant increase in the sentence completion task during WM training. Based on the charts in Figure 3 and effect size scores (Table 4), the performances of all participants in this task were effectively developed during the WM training phase.

It is difficult to explain the effectiveness of the WM training on extracted indexes from language samples (including MLU and GU%) among the participants. The efficacy of WM training on MLU enhancement is clear for all participants, except participants 2 and 7. Despite the diversity of the participants’ performances, the average MLU scores in the training phase are greater than those in the baseline phase for all participants. Similar to the other participants, participants 2 and 7 also showed this level enhancement pattern, but for participant 2 the difference between the levels of the baseline and training phases was not significant; while the training shows a trend towards improvement and the overlap between the training and baseline phases is small (PND = 80), the effectiveness of WM training for increasing the MLU is not clear because of the small change in level shown by the SMD (= 1.96). Participant 7 demonstrated an improved performance across the training period, with an upward trend. While the visual analysis and SMD scores relatively support the effectiveness of the training, the 3 points of overlap between the data from the training and baseline periods, which is also demonstrated by the PND score (= 60), raise some questions about the effectiveness of the training for this participant.

Regarding the GU% charts (Figure 5), seven participants (P1, P3, P6, P7, P8, P9, and P10) demonstrated a clear enhancement of grammatically well-formed utterances in their speech samples during WM training. While a large variability could be seen among these seven participants’ performances, for all participants the GU% increased suddenly or gradually following the introduction of WM training. The patterns of performance are variable, but the level of the training phase is obviously greater than the level of the baseline phase. The effect size scores (SMD ~ 2 or greater and PND > 70, Table 4) also support the visual analysis. However, the effectiveness of the training was not acceptable for participants 2 and 4 and is doubtful for participant 5. For participants 2 and 4, the average data for the training period are greater than those for the baseline, but the difference between them is not significant. The data from the training period shows high variability and has many overlaps with the baseline data. The effect size scores are in accordance with the visual analysis. For participant 5, a relatively slight increase in performance could be seen during the WM training period, and there is little data overlapped between the training and baseline phases (PND = 80); however, the insignificant difference between the levels of the training and baseline periods raise doubts about the clinical significance of the training (SMD = 1.64).

On the balance task, the performance patterns of all participants during the training phase were variable, and although there was high variability among the performance patterns of different subjects, both visual analysis (Figure 6) and the effect size scores (Table 4) do not support the effectiveness of the training on gross motor skills for all participants.

The participant scores for the repeated measures of WM and language (Figures 1–5) during the WM training and after the break period show that for some children the scores in the second baseline are lower than the scores obtained during the first treatment phase. This pattern of performance is visible for all participants in NWR, for all participants except participants 8 and 9 in BDS, for all participants except participants 8 and 10 in the sentence completion task, for all participants except participants 2, 5, 6 and 8 in MLU-w, and for all participants in GU%.

b Efficacy of language intervention on WM and morpho-syntactic skills

The effect size scores in Table 5 clearly show that the performances of six participants (P1, P3, P4, P6, P8 and P9) exceeded the clinical significance thresholds on the NWR task, although there are substantial differences between their overall task performances (Figure 1). Among the remaining four participants (P2, P5, P7 and P10), for participant 2 the presence of two points of overlap between the data from the intervention and baseline phases and the PND score (= 60) raise some uncertainty about the efficacy of the intervention. However, a variable pattern of performance in the first three sessions of intervention, followed by a sudden improvement and upward trend at the end of the intervention period resulted in a level during the intervention period that was greater than the level of the baseline period and gave an SMD = 3.57, which is well above the threshold of 2 for clinically significant change. The performance pattern of participant 10 is inconstant during the intervention phase, though the slope of the trend line is positive. Two points of overlap between the data of intervention phase and the data of baseline phase resulted in a PND = 60, which indicates a small effect size. Nevertheless, the average of the task performances during the intervention period is significantly above the participant’s performances in the baseline phase (SMD = 3.2). Therefore, both visual analysis and the SMD scores support the efficacy of the intervention for participants 2 and 10. For participants 5 and 7, the overall difference between the levels of the baseline and treatment phases is small, and there are some points of overlap in task performances between the intervention and baseline phases. The effect size scores (SMD = −.05, PND = 20 and SMD = .92, PND = 60 for participants 2 and 10, respectively) do not support the intervention efficacy.

OPEN IN VIEWER

Table 5.

Effect sizes of language intervention on repeated measures (WM, language and control tasks).

Participants	NWR		Backward digit span		Sentence completion task		MLU-w		Grammatical utterances (percentages)		Gross motor task
	PND	SMD	PND	SMD	PND	SMD	PND	SMD	PND	SMD	PND	SMD
P1	80	4.96	0	−1.86	80	6.81	100	5.61	100	3.35	20	0.3
P2	60	3.57	20	−0.11	100	8.89	40	1.8	80	2.17	40	.11
P3	100	4.6	60	2.3	100	11.3	60	1.83	40	1.68	0	−0.8
P4	100	4.04	20	0	100	7.73	20	0.96	80	9.46	0	−1.1
P5	20	−0.05	0	0.8	80	3.8	40	1.85	100	4.26	20	0.62
P6	100	2.4	0	0.57	100	5.2	100	8.02	80	10.72	20	−0.4
P7	60	0.92	0	0	100	9.23	100	4.32	80	1.67	20	0.8
P8	100	4.96	0	−1.2	80	2.4	0	−0.07	100	5.8	0	−0.2
P9	100	3.6	0	−0.4	100	10.8	0	0.33	60	0.77	0	−0.1
P10	60	3.23	0	−0.4	60	3.34	100	4.2	20	−0.46	0	0.17

Notes. MLU-w = mean length of utterances in words; NWR = non-word repetition; PND = non-overlapping data; SMD = standardized mean differences; WM = working memory.

Changes of participants’ performances in the second WM task, BDS, are presented in Figure 2. These charts show the small decrease in level for participants 1, 2, 8, 9, 10 and an unchanging level for participants 4 and 7 during the treatment phase. Only for three participants (P3, P5 and P6) was the level of the intervention phase above the level of baseline. The performances of participants 5 and 6 are nearly stable throughout the intervention period, and the level increase is not significant for either. Therefore, based on a visual analysis (Figure 2) and on the effect size scores (Table 5), it is clear that the language intervention did not result in performance improvements in BDS for nine of the participants. The performance of participant 3 was variable, with no detectable trend during the intervention period, in spite of an upward trend during the baseline period. Therefore, despite the increase in level, regarding the variable pattern of performance during the intervention and the PND score ( = 60), the intervention was not effective for this participant.

Both a visual analysis of Figure 3 and the effect size scores (Table 5) confirm the efficacy of the language intervention programme for improving the participants’ performances in the sentence completion task. Figure 3 shows that the performances of participants 1, 2 and 9 improved immediately after the introduction of intervention and continued with an upward trend. For participants 3, 4, 5, 6, 7 and 8, though the performance patterns were variable during the intervention phase, increases in performances began following the introduction of the intervention. Although in the first sessions of intervention the task performances of P10 have overlaps with performances in baseline period (PND = 60), the steady upward trend in performance during the intervention phase indicates that the intervention resulted in significant improvement.

Regarding Figure 4 and Table 5, the efficacy of language intervention for improving MLU is obvious for four participants (P1, P6, P7 and P10). The MLU of participant 1 increased immediately after the introduction of intervention and continued with a steady upward trend throughout the intervention. The differences between the levels of intervention and baseline phases are significant for P6, P7, and P10, though they show variable patterns of performance during the intervention phase (SMD = 8, 4.3 and 4.2, respectively). However, the intervention was not effective for increasing the MLU for the other six participants (P2, P3, P4, P5, P8 and P9). Participant 2 demonstrated a slow upward trend that did not result in a significant improvement in the MLU (SMD = 1.8). Furthermore, many overlaps can be seen between the MLU scores of the baseline and intervention phases (PND = 40). Participants 3, 4 and 5 demonstrated variable patterns of performance, with no significant improvement in the MLU during the intervention phase (SMD = 1.8, 0.96 and 1.8, respectively). These participants’ MLU scores during the intervention period had many overlaps with MLU scores in the baseline period (PND = 60, 20 and 40, respectively). The performances of participants 8 and 9 during the intervention phase were variable and altogether lower than the baseline performances. For participant 8, the effect size scores were SMD = −.07 and PND = 0. For P9, the presence of one very low data point during the baseline period resulted in positive, non-significant SMD of .3.

Based on a visual analysis of Figure 5 and the effect size scores (Table 5), it appears that the language intervention caused improvements in expressing grammatically well-formed utterances in seven participants (P1, P2, P4, P5, P6, P7, and P8). Participants 1 and 5 showed improved performances immediately after the introduction of the intervention, which continued with an upward trend. For participant 2, a sudden increase in the GU% occurred two weeks after the onset of intervention and continued with slow rate during the intervention period; the level is significantly above the level of the baseline period. Participants 4 and 6 showed highly variable performances during the intervention, which have no (P6) or little (P4) overlaps with the performances in the baseline period; their effect size scores (SMD = 9.46, PND = 80 and SMD = 10.7, PND = 100, respectively) demonstrate the effectiveness of treatment. The pattern of performance of P7 is variable; however, by excluding one very low data point, it seems that their performance follows an upward trend. Though the presence of one very low data in the intervention phase resulted in no significance SMD ( = 1.67), the PND effect size score ( = 80) and visual analysis relatively support the effectiveness of intervention for this child. The performance of participant 8 shows a sudden increase immediately after the introduction of the intervention, which was followed by a downward pattern; in spite of the presence of this visible downward trend, the intervention phase scores are still above the baseline phase scores, and no overlap between the data could be seen. The effect size scores (SMD = 5.8 and PND = 100) confirm the intervention efficacy for this participant. However, the performance patterns of the other three participants (P3, P9 and P10) during the intervention phase are not representative of language intervention efficacy in increasing grammatical correctness. Participant 3 shows a large variability and no significant increase in the level during the intervention phase. For participant 9, a very slow upward trend was replaced by a downward trend at the end of the intervention. While some increase in performance could be seen, this increase was not significant. The performance pattern of participant 10 during the intervention phase is variable and has many overlaps with baseline (PND = 20); the level during the intervention period is lower than the level of baseline and resulted in a negative SMD ( = −.46).

Finally, regarding Figure 6 and Table 5, all participants demonstrated a large variability in performance patterns, which does not support the language intervention efficacy on the balance task.