Abstract
This pilot study investigated the impact of an oral narrative intervention program implemented with 24 children who attended a College of Education on campus laboratory school for children with specific language learning difficulties. Oral narratives were elicited before and after treatment and underwent T-unit and story grammar component analyses. The 13 weeks of treatment involved a systematic approach to teaching story grammar components and story organization. Paired sample t-tests were conducted and a significant improvement in number of T-units and use of story grammar components was measured following intervention. These results lend preliminary support to the effectiveness of the narrative intervention implemented in a group setting.
Keywords
I Introduction
The ability to tell a story is an important skill. Many professionals believe that the ability to tell a story involves both higher-level language and cognitive abilities (Klecan-Aker and Gill, 2005; Paul et al., 1996; Westby, 1989) and requires a more abstract grasp of language structure than conversational discourse. Specifically, the concise syntactic style, complexity of sentence structures, rare and rich vocabulary, focus on unfamiliar and abstract topics, appreciation of story grammar, and decontextualization (distancing from immediate experience) increase the linguistic demands of the narrative genre (Roth, 2000; Silliman, 1989; Silliman and Wilkinson, 1991). Oral narratives are also considered a ‘bridge to literacy’ as they are closely related to children’s literacy development and school achievement (Boudreau, 2006; Elster, 1994; Hedberg and Westby, 1993; Mehta et al., 2005). Petersen et al. (2010) screened core curricula from seven randomly sampled states in the USA for narrative-related objectives and found that features of narrative macrostructure and microstructure were explicitly included as important content in all seven. Researchers have found that children who understand and produce the type of narratives required by the teacher participate more effectively in classroom discourse and also become more competent readers and writers (Au and Kawakami, 1985; Compton-Hall, 2002; Fang, 2000; Gee, 1989; Michaels, 1991). Lastly, because a significant amount of classroom instruction, assessment, and interaction occurs in narrative form, the ability to tell a coherent story can predict successful adaptation to school literacy (Feagans, 1982; McCabe and Rollins, 1994). Thus, children in primary through secondary grades need to be able to successfully understand and produce narratives.
The connection between oral narrative abilities and school success is strengthened when examining the narrative production limitations of children who have specific language learning difficulties (Feagans and Short, 1984; Kaderavek and Sulzby, 2000). Roth and Spekman (1986) demonstrated that school age (i.e. 8–13-year-old) children with specific language learning difficulties struggle with spontaneous story generation. This difficulty could be due to the fact that children with such impairments tend to have small vocabularies characterized by short, frequently-occurring words (Paul, 1995), high frequency use of nonspecific words (Wiig and Semel, 1984), fewer complex sentences, and less elaboration of noun phrases (Paul, 1995). They also have specific challenges with several aspects of spontaneous story generation that include problems with the use of story grammar components, inappropriate use of cohesive ties, and delayed developmental story levels (Feagans and Short, 1984; Klecan-Aker and Gill, 2005; Liles et al., 1995). Thus, many of these children struggle in the classroom and require intervention that targets comprehension and production of narrative stories to increase academic participation and overall success (Fey et al., 2003; Swanson et al., 2005). Narrative intervention could be used to help these students not only improve narrative skill, but also build a bridge between oral and literate language. A number of researchers have suggested that narrative ability is one of the best predictors of school success for children with specific language learning difficulties (Bishop and Edmundson, 1987; Kaderavek and Sulzby, 2000). Because of its importance, oral narration should be a major component of assessment and intervention programs for these children (Bliss and McCabe, 2008).
To that end, speech-language pathologists have been encouraged to assess a child’s ability to tell a story (Gillam, Pena and Miller, 1999). Speech-language pathologists must supplement information they routinely obtain from standardized tests of spoken language and conversational language sampling with information on curriculum-based assessment of children’s oral language skills, if they are to identify weaknesses in language ability (American Speech-Language-Hearing Association (ASHA), 2000). Narratives can be analyzed at both a macrostructure level and a microstructure level. The former analyses consider the overall organization and components within the story (Ukrainetz et al., 2005), and the latter focus on lexical and grammatical structures (e.g. T-units, words per subordinate clause, number of subordinate clauses per T-unit) within the narrative (Justice et al., 2006). Both can provide valuable information about students’ areas of difficulty in formulating a logical, coherent story. Once these analyses are complete, interventions can be designed to target macrostructure and microstructure elements to improve storytelling. Several techniques for teaching narrative skills to school-age children with specific language learning difficulties have been documented in the literature.
Researchers have investigated a variety of approaches to improve students’ use of narrative macrostructure. Ukrainetz (1998) used pictography, or picture writing, as a prewriting technique to improve the overall quality of written stories produced by students with writing difficulties. Specifically, this technique encourages students to generate simple sketches organized in a left-to-right, chronologically-based layout. This system of stick figures and arrows gives a visual base of ideas and events that provides memory and organizational support for story writing. In individual intervention case studies, pictography was observed (qualitatively) to improve the temporal organization and content focus in the story writing of several third and fourth graders (i.e. 8–10-year-olds) with specific language learning difficulties. In this study, data were available for single participants only, which somewhat limits our ability to generalize the treatment results.
In a group study, Montague et al. (1991) investigated the use of planning time and story grammar cue cards (e.g. a list of the story grammar components and their definitions) compared to no intervention and planning time alone with 60 junior high school students, 40 exhibiting specific language learning difficulties and 20 who were typically developing. They found that there were significant differences between the stories of the two groups on both quality and quantity of writing in the non-cueing conditions, but not when planning time and story grammar cueing were combined.
Klecan-Aker and colleagues (Klecan-Aker et al., 1997; Klecan-Aker and Gill, 2005) completed two small studies in which they looked at improving oral narrative ability in children with specific language learning difficulties. In both of these studies they found that teaching these children to structure the language in narrative form (e.g. first through understanding and then through including all relevant story grammar components in story generation) improved the developmental story level (e.g. narrative complexity) of the children’s stories. In the 2005 study, the authors also reported that as story organization improved, reading comprehension improved as well. However, there were limitations in both of these investigations. In the 1997 study, only seven participants were included in the investigation. In the 2005 study, there was only a single participant. Green et al. (2008) used the same approach as Klecan-Aker and colleagues in seven separate single participant applied behavior analysis (ABA) treatment research experiments conducted in the public schools. All seven participants, who were in grades 2 through 5 (i.e. 7–11-year-olds), increased their developmental story levels (six of the seven students increased two story developmental levels) following eight weeks of intervention. Again, a limitation of these studies was that they each contained only a single participant.
In summary, there are few available studies for improving the narrative macrostructure skills of primary and secondary school-age children, and most of these studies involved single participants only. So, while researchers have illustrated the importance of narrative ability and demonstrated a link between that ability and literacy success, information is still lacking about successful intervention practices to improve oral narrative ability.
A last, related issue that needs to be considered with respect to any type of language intervention is the concept of evidence-based practice among speech-language pathologists. Evidence-based practice (EBP) is:
the conscientious, explicit and judicious use of current best practice in making decisions about the care of individual patients … by integrating individual clinical expertise with the best available external clinical evidence from systematic research. (Sackett et al., 2000: 71)
There are many potential benefits of EBP, such as improving clinical services, linking research and practice, making clinicians more accountable, and reducing variation in service provision (Zippoli and Kennedy, 2005). Professionals in the field of communication disorders have recognized the importance of utilizing evidence-based practices for quite some time. ASHA has issued an official policy statement to the effect that audiologists and speech-language pathologists integrate the principles of evidence-based practice into the clinical decision-making process (ASHA, 2005). Yet, in the area of language disorders in primary and secondary school-age children, there is not a large body of clinical research evidence available (Cirrin and Gillam, 2008).
Therefore, as a result of the need to document the effectiveness of teaching narrative organization and to contribute to our evidence base in child language intervention, the purpose of this pilot group treatment study was to further examine the effectiveness of teaching narrative macrostructure (e.g. story grammar components and their organization) utilizing The expression connection: A structured approach to teaching storytelling to school-age children (Klecan-Aker and Brueggeman, 1991) to improve the oral narrative abilities of twenty four 6–9-year-old children with language learning disabilities.
II Method
1 Design
This study utilized a single-group pre-test/post-test design with narrative skills analyzed according to the above-described procedures before and after intervention. It was predicted that these scores (e.g. developmental story level) would increase following narrative treatment. A control group was not utilized in this pilot study. In future research, this will be added and more conclusions will be able to be drawn. This study does, however, represent a design that can be successfully completed in a school-based, classroom environment and can provide preliminary data as to the success of a treatment program.
2 Participants
The 24 participants were selected from an on-campus laboratory school for children with language learning disabilities run by the College of Education at the second author’s former university. Participants included 14 males and 10 females, ranging in age from 6 years, 3 months to 9 years, 6 months. The school does not have designated grades, only academic levels based on students’ achievement test scores. Given the school’s small size, parents requested that individual chronological age information not be shared such that a 9-year-old placed in a class with younger students would not feel judged by others. Participants selected for this investigation were from the school’s Levels I and II (roughly academically equivalent to first and second grade or to typically developing 6–8-year-olds).
The students demonstrated average or above average cognitive ability, with a range of intelligence test scores between 89 and 118. Each student demonstrated a gap between IQ scores and achievement scores. Additionally, per school report, 80% of all students who attend exhibit deficiencies in expressive language, language comprehension or both. Specific test scores for each participant were unavailable due to the school’s privacy policy, but the students who participated in the study were a part of that 80% and their teachers approved the narrative intervention as an appropriate and necessary objective. Informed parental consent was obtained for all of the students.
3 Materials and equipment
The expression connection (Klecan-Aker and Brueggeman, 1991) was developed for use as a criterion-referenced measure of narrative ability and as a highly structured story grammar instruction program for children who exhibit difficulty in organizing language in the narrative genre. The program consists of a narrative elicitation procedure including five model stories and accompanying stimulus pictures, a procedure for transcribing and analyzing these stories, and an intervention program. The treatment program consists of initial instruction in story grammar components and then the application of this information in multiple-choice and fill-in-the-blank story activities. Stories are scored for and activities organized according to developmental story levels, which are determined based on the number and type of story grammar components. For a complete description of story grammar components and each developmental story level, see Appendix 1.
a Narrative assessment
The oral narratives were elicited in a small, quiet room with the examiner and the child seated across from each other at a small table. All elicitations were audiotaped and the equipment consisted of a Magnavox tape recorder, a microphone and audiotapes. Telling a Level 5 story was modeled for each child, so that children experienced the type of information they were to provide when asked to tell a story. Specifically, the examiner randomly chose one of the five pictures contained in The expression connection (Klecan-Aker and Brueggeman, 1991) and told the matching story. For each elicitation, the participant was given the following instructions:
I am going to tell you a story. Listen to me carefully so that you will know what a story is. Then I’m going to ask you to look at a different picture and make up a story of your own. Are you ready?
After sharing the example story, the examiner then randomly chose a different picture from the five and gave the following instructions: ‘Now it’s time for you to tell a story. You may begin.’ The child was allotted as much time as was needed. Prompts that were occasionally used consisted only of ‘Can you tell me more?’. Once the child had completed the story, the tape recorder was turned off and the results were transcribed. A different picture stimulus from the program was utilized for each story elicitation. The same process was utilized to elicit the post-treatment narratives.
All narratives were transcribed orthographically with no capitals or periods (i.e. full stops), as if they were running speech. The transcription was triple spaced, leaving room for the analysis on the same form. After each story was transcribed, the story was replayed twice more to check for accuracy of the initial transcription. Any discrepancies that were noted were corrected before the analysis was begun. The oral stories were analyzed by the second author (a university professor and licensed speech-language pathologist) who was blind to the time of the elicitation (e.g. pre- or post-treatment) and who used the procedures specified in Klecan-Aker and Brueggeman (1991). Stories were first divided into T-units. A T-unit is the equivalent of a simple or complex sentence. As a result, a compound sentence would be equal to two or more T-units and a complex sentence would equal one T-unit and two or more clauses. Each T-unit was then assigned a story grammar component label (see Appendix 1). The labels were used to determine the developmental story level of the story. In addition to the total number of T-units, words per T-unit, words per clause and clauses per T-unit were also calculated. While these narrative microstructure elements were not targeted specifically in treatment, there was an interest in examining any changes that might occur if use of story macrostructure improved.
Intra-judge and inter-judge reliability scores were obtained for developmental story level (the primary dependent variable for examining change in story complexity) by having the examiner (the second author) analyze the narratives twice and by having another individual, a certified speech-language pathologist, analyze 25% of the narratives as well. The intra-judge reliability score was 100% and the inter-judge reliability score was 92.1% using a point-by-point percentage of agreement formula.
b Control measure (parts of speech)
Pre- and post-treatment data were collected on the participants’ ability to identify nouns, verbs, adjectives, and prepositions in sentences. Students were given a sheet with 20 sentences and asked to underline nouns in the first five sentences, verbs in the second five, adjectives in the third five sentences, and prepositions in the final five sentences. The parts of speech identification task was repeated after intervention was completed. While oral narration occurs more frequently in classroom interaction than identification of parts of speech, both the learning of story grammar components and the learning of nouns, verbs, adjectives, and prepositions typically require specific, focused instruction in children with learning disabilities. Given that story grammar components were being taught and parts of speech were not, it was predicted that the participants’ performance identifying nouns, verbs, adjectives, and prepositions would not change pre-and post-treatment. We conceptualized this as a quasi-control for developmental changes (as in a multiple baseline single subject design) to increase confidence that any change in narrative ability could be attributed to the treatment targeting it, but we note that there are flaws in this argument due to differences in the types of knowledge required for narration versus labeling parts of speech.
c Intervention programme
Treatment was implemented in the laboratory school classroom and continued for 13 weeks. For the treatment protocol and schedule outline, see Appendix 2. Small group intervention was provided for 30 minutes twice weekly in the classroom by speech-language pathology graduate students. The three graduate students were trained in the intervention procedure and were supervised 95–100% of the time by the second author to ensure treatment fidelity. The participants were randomly assigned to groups of eight, with one graduate clinician leading each group. The smaller group size allowed for more hands-on instruction and greater opportunities for student participation.
The first six sessions of therapy (three weeks) focused on teaching the underlying concepts of the three story grammar components that comprise a Level 3 story (e.g. initiating event, action, and consequence). It was decided to begin with Level 3 stories to make sure the children understood this story ‘core’ before adding internal responses and because Level 3 was one developmental story level higher than the participants’ initial mean developmental story level of 2.1. The initiating event was explained by saying that many stories have a problem, which is what the story is really about (e.g. Joe’s puppy ran away). Numerous examples were then given. The action was described as what happens in an attempt to solve the problem (e.g. Joe put ‘lost puppy’ signs up all over the neighborhood). A consequence was explained as a result of the action (e.g. The neighbors called to say they had found Joe’s puppy). Sometimes the problem was solved and sometimes another action was needed for resolution. The story examples were generated to represent experiences that were deemed appropriate given the age of the participants (e.g. forgetting lunch money, arguing with a friend, waking up not feeling well, etc.). Story grammar components were reviewed in the context of fun, engaging activities like ‘grab bag’, in which students pulled a card with a story grammar component written on it and labeled it or ‘add on’, in which students were given an initiating event and then added on the next two components as a group.
After three weeks of this story grammar instruction, the next three weeks involved completion of multiple choice and fill-in-the-blank story activities. The children were first presented with a series of multiple choice questions designed around Level 3 stories. This format consisted of four possible choices from which the students could select the appropriate story grammar component. The initial part of the story and the subsequent choices were read to the students and each child was instructed to choose which answer was most appropriate. When eight out of 10 stories activities were successfully completed by the group, the participants then completed the Level 3 fill-in-the-blank tasks. These provided a slight increase in difficulty as the students had to create versus choose a correct answer. In this format, each child was read the initial part of the story and was then asked to answer fill-in-the-blank questions about the action and ensuing consequence. Where appropriate, the clinicians recorded answers for students with literacy difficulties so writing challenges were not a deterrent to completion of the activity. When eight out of 10 stories were successfully completed for the group, the story grammar component of internal response was targeted. An additional component was necessary before moving to Level 4 multiple choice activities as these stories contain one more than a Level 3 story. Because internal responses often drive actions and consequences in a story, and because internal response statements are often not included in the stories of children with learning disabilities, this component was deemed a logical choice.
Thus, feelings were introduced during the seventh week of intervention and, during the next two weeks, seven feelings were introduced and taught. They included happy, mad, sad, scared, confused, embarrassed, and frustrated. Each feeling was explained to the class with an accompanying picture of a person displaying that emotion, and several age-appropriate examples were provided. Then the class was asked to generate situations in which they had experienced the feelings being discussed. When the students could all label and explain the target emotions, the participants completed Level 4 multiple-choice and fill-in-the-blank format activities. When the participants successfully completed eight out of 10 story activities in both formats, the story grammar component of setting was introduced.
During week 12, the participants were taught that setting statements reveal ‘who’, ‘when’, and ‘where’ in the story. Several examples of each part of the setting were provided to ensure that the participants were familiar with the concept (e.g. ‘One summer morning a boy named Jake, who lived on a farm in the country, woke up and got ready to do his chores.’). This step was followed by asking different members of the group to generate examples of each setting element. Once this step was successfully completed, settings became part of the story format. Level 5 multiple-choice and fill-in-the-blank formats were completed during the last week of treatment. Participants’ scores ranged from 5 to 10 stories completed correctly with a mean score of 9.0 for the multiple-choice format and a mean score of 7.2 on the fill-in-the-blank format.
III Results
1 T-units and clauses
On the pretreatment measure, the number of T-units for all participants for the pre-treatment stories ranged from 3 to 14 and the mean for number of T-units was 5.33 (SD: 2.23). On the post-treatment measure, the number of T-units for all participants ranged from 5 to 14 and the mean for number of T-units was 7.37 (SD: 1.74). A paired sample t-test (2-tailed) was used to compare the T-unit means pre- and post-treatment and found a significant difference (t(23) = −6.1, p < .001) between the two. An effect-size measure was also calculated to determine the extent to which treatment had a significant effect on the group’s mean number of T-units. Taking into account the correlation between the pre- and post-test scores, the Cohen’s d (M 1 – M 2 / σpooled) was considered large at .747 (equal to an r of .35), indicating a significant increase in mean number of T-units.
Additional pre-treatment means related to T-unit and clause structures were 8.5 words per T-unit, 7.3 words per clause and 1.15 clauses per T-unit. Following treatment, means for words per T-unit (7.9), words per clause (6.64), and clauses per T-unit (1.18) were not significantly different from the pre-treatment means (p > .05). Pre- and post-treatment T-unit data for all of the participants can be found in Table 1.
Participants’ (n = 24) pre- and post-treatment scores for number of T-units and developmental story level.
2 Developmental story level
The initial mean developmental story level for the 24 participants was 2.1 (SD: .65) with a range of 1–3. Immediately following intervention, the mean developmental story level was 4.33 (SD: .64), with a range of 3 to 5. A paired sample t-test (2-tailed) was used to compare these pre- and post-treatment means and found a significant difference (t(23) = 13.01, p < .001) between the two. Effect size was again calculated using Cohen’s d and was large (d = 2.68; r = .80), indicating marked improvement in developmental story level following treatment. Pre- and post-treatment developmental story level data for all of the participants can be found in Table 1.
3 Control measure (parts of speech)
Pre- and post-treatment group mean percentages for identification of parts of speech are detailed in Table 2. Based on visual inspection of the data, one measure remained exactly the same (prepositions), one increased by only two percentage points (nouns), and two actually decreased (verbs and adjectives), indicating little change in or decreased performance on this comparison skill during the treatment period.
Pre- and post-treatment group mean percentage correct for identification of parts of speech.
IV Discussion
This study provides preliminary evidence that treatment targeting oral narrative skills in a group setting, using an approach such as The expression connection (Klecan-Aker and Brueggeman, 1991), can have measurable effects on students’ storytelling performance. Two of the narrative analysis results support this conclusion. First, the mean number of T-units increased significantly and the effect size was large. As children learn about and then begin to use more story grammar components (which are assigned to the T-units), it is logical that they would also use more T-units in their stories to convey these ideas. Second, mean developmental story level increased significantly after completion of the intervention program and the effect size was large, indicating greater use of story grammar components and therefore greater overall narrative complexity. The other measures of narrative microstructure (e.g. words per T-unit, words per clause, and clauses per T-unit) did not change substantially as increasing sentence and clause length was not a skill specifically targeted in therapy. Additionally, the students’ ability to identify parts of speech did not improve during the treatment period. A control measure such as this one provides some reassurance that the changes seen in treatment are not a result of maturity or participation in the classroom curriculum.
From a practical standpoint, it is worth noting that the documented improvements in narrative skills may have been a result of an in-class, group intervention. Given that the caseloads of public school speech/language pathologists continue to increase and classroom instruction has become much more common, these findings provide encouragement that narrative improvement can be seen as a result of structured group narrative intervention.
In terms of external validity (the extent to which the findings may be generalized), our sample of 24 participants is adequate to suggest that children with language learning disabilities may benefit from the type of program described in this study. A serious limitation of this investigation, however, is inherent in the design. The instructional setting and the educational needs of the participants did not allow for use of a control group with randomized assignment to control and treatment conditions. So the results, although achieved in more typical clinical setting and showing a large effect on developmental story levels, can provide only a weak level of evidence with regard to the success of the narrative intervention. Additionally, it cannot be assumed that narrative ability would improve in children with other disabilities.
Future research could involve utilizing a control group for greater experimental control and examining not only the stories of students receiving narrative intervention, but also measuring potential changes in other related academic areas such as written language and reading comprehension. Correlations could also be calculated between these measures to further corroborate previous research findings that narrative skill is related to other academic abilities.
Finally, as speech-language pathologists who wish to increase our status as scientifically oriented professionals (Damico, 1988; Meline and Paradiso, 2003; Zippoli and Kennedy, 2005), we need to be accountable for the success of our treatment programs, regardless of the type of disorder. Empirical demonstration of effective clinical procedures and their underlying factors need to be addressed. There are challenges to doing this in school-based settings. In fact, in their review of the last 20 years of language intervention studies in search of those that met Level 1 (randomized clinical trials) and Level 2 (non-randomized comparison studies or multiple baseline single participant designs) evidence requirements, Cirrin and Gillam (2008) only found 21 studies that qualified. Additionally, many of those qualifying studies were conducted with preschool children. As we begin to attempt to bridge this gap between this 21-study evidence base and actual research that is being conducted with school-age children, we will need to explore how functional but less powerful studies like this one can be expanded upon. While practitioners are already obtaining baseline data, implementing treatment as a result of the data, and measuring progress in consistent and objective ways, they must now sort out how to create additional experimental control and also be ready to disseminate their findings. Because this can be a logistically challenging and time- consuming process, preliminary evidence about a treatment program can provide some assurance that the approach is worth the time and effort required for further study.
Footnotes
Appendix 1
Appendix 2
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
