Abstract
In order to meet the communication needs of individuals who use Augmentative and Alternative Communication (AAC) devices, communication partners are often responsible for the programming of the AAC devices. However, the prolonged learning time and operational demands of AAC devices are often barriers to an efficient use of time during a therapy session. We developed a prototype app for iPad, called Speech-to-Symbol that decreases the programming time by finding pictures stored in the app using speech-to-text technology. We compared our solution to the TalkTablet app during AAC sessions with children with different communication disabilities. The aims of this study were 1) to compare the time to program both apps, 2) to identify the type of vocabulary used, and 3) to assess the usability of the two apps. Results showed that the reduced operational demands of our Speech-to-Symbols app allow communication partners to expose children to a larger and more diverse vocabulary. In addition, the communication partners gave our Speech-to-Symbols app a higher usability rating. Implications for clinical and educational settings and directions for future research are also discussed.
Keywords
Introduction
Augmentative and Alternative Communication (AAC) is an area of clinical and educational practice that provides individuals with communication disabilities the opportunity to engage with society through the use of evidence-based communication strategies and devices that allow them to express their thoughts, needs, and ideas (Beukelman and Mirenda, 2013). Communication partners, such as family members, friends, teachers, and AAC professionals, are responsible for identifying unmet communication needs, supporting the interactions of individuals who use AAC, and programming and maintaining the AAC devices (Beukelman and Ray, 2010; Radici et al., 2022).
One of the strategies communication partners could use is language modeling, an evidenced-based practice (Lynch et al., 2018) that has been found to successfully enhance language learning and to promote vocabulary expansion (Cress and Marvin, 2003; Douglas et al., 2013; Kent-Walsh and McNaughton, 2005; Romski and Sevcik, 1996). While language modeling does not require communication partners to master programming, being able to program allows communication partners to model a more diverse and potentially relevant set of language structures and targets (Baxter et al., 2012). Previous research studies reported that communication partners often consider programming high-tech devices time consuming and difficult (Donato et al., 2014).
Communication partners need to anticipate which words or symbols will be needed in each conversation or learning activity. Considering the wide variety of topics occurring in natural conversations, selecting and programming words or symbols ahead of time is challenging and laborious (Light, 1997; Porter and Cafiero, 2009).
AAC just-in-time (JIT) technologies simplify the process of programming and accessing vocabulary by reducing the steps required to add or edit new content (Caron et al., 2016; Schlosser et al., 2016). Decreasing the time needed by communication partners to display images or symbols may lead to greater engagement (Caron et al., 2017).
The purpose of this research study was to compare a JIT-based app prototype, called Speech-to-Symbols, with TalkTablet, an app (Gus Communication Devices, 1996) already available in the market, during AAC sessions with children with complex communication needs. In particular, this study aimed 1) to compare the time to program both apps, 2) to identify the type of vocabulary used, and 3) to assess the usability of the two apps.
Method
Ethical approval was obtained before this research study started collecting any data. An informed consent was signed by the professionals and by the parents of the children involved in this study.
1 Participants
Four professionals with different backgrounds ranging from occupational therapy, special education and special pedagogy therapy, and with extensive experience working with children with communication needs and using AAC devices, participated in this study (Table 1). The AAC professionals were three females and one male with a mean age of 49 years old and a mean of 11 years (from 3 to 18) of experience in AAC. Inclusion criteria were: (a) AAC experience working with individuals with communication needs, (b) no prior knowledge of how to use the two apps under investigation, and (c) a willingness to commit to the research study with no monetary benefit. The four professionals were chosen from those with prior AAC experience to mitigate the need to teach basic AAC skills and to avoid any anxiety related to the use of a completely unknown technology.
Professionals demographic information.
Professionals demographic information.
Two children, one boy and one girl, with complex communication needs participated in the study. The boy was 5 years and 10 months old, and the girl was 5 years and 11 months old. The boy was diagnosed with Angelman Syndrome and the girl was diagnosed with the Phelan McDermid Syndrome. Inclusion criteria were: (a) familiarity with the use of AAC, and (b) ability to participate in a play activity for a minimum of 10 min.
2 Target vocabulary
In order to allow professionals to start engaging in a play activity (“play with bubbles”) with the child, ten target vocabulary words with a corresponding picture were selected and added to the two apps. To select the target vocabulary for the play activity, the four AAC professionals were asked to list the set of words they usually use the most when engaged with the activity “playing with bubbles”. The most frequent eight words of these lists were selected as the target vocabulary. The words “more” and “all done” were also added to the activity target vocabulary as appropriate words across activities (Caron et al., 2016). The activity target vocabulary included three verbs (“to pop”, “to blow” and “to get”), one noun (“bubbles”), two adjectives (“big”, “small”), two adverbs (“up”, “down”), and two across routines words (“more”, “all done”). The vocabulary was randomly inserted in a 5 × 3 grid on both apps. The symbols and their position on the grid were the same for the two apps.
3 Augmentative and alternative communication applications
We installed the two apps on an iPad Air2 with built-in cameras and speakers (Apple Inc., 2014). Both apps included a pre-stored communication grid board (5 × 3) with 15 cells, ten of which were filled with the target vocabulary selected (Figure 1). Both apps had a local database with a large repository of symbols.
Talktablet – App 1 (left) and speech to symbol – App 2 (right).
The TalkTablet app (App 1) offered communication support through grids of symbols. This app offered options for customization—such as adding symbols or images taken from the iPad library or the Internet, synthetized voice, or recording messages and texts—through a multi-step programming procedure. Among all the apps available on the App Store, TalkTablet was chosen because its layout and features were the most similar to the layout and features of the other AAC app used in this research study. To program a new symbol, the communication partner needed to go through a ten-step procedure to retrieve a symbol from the iPad library, or an 11-step procedure to retrieve a symbol from the Internet.
a Speech-to-symbols app
The Speech-to-Symbols app (App 2) offered communication support through grids of symbols. The app offered options for customization: when a symbol was not available on the local app database, the communication partner was able to add the symbol using a speech-to-text feature that retrieved symbols from the app database or the Internet, based on the text recognized. Both the synthetic voice and the text were automatically included in the database together with the corresponding image. To program a new symbol, the communication partner needed to go through a one-step procedure to retrieve a symbol from the app database and a three-step procedure to retrieve a symbol from the internet.
4 Measure
We compared the two apps based on: (a) the time spent by the professionals searching for a new image in the main grid, and (b) the type of vocabulary used during the activity. In order to evaluate the usability of the apps, we used the System Usability Scale (SUS) questionnaire (Brooke, 1996). The SUS includes ten 5-point Likert scale questions ranging from 1 (strongly disagree) to 5 (strongly agree). A SUS score above 68 would be considered above average and anything below 68 is below average. The SUS questionnaire was provided at the end of each of the eight sessions.
5 Procedure
The four professionals engaged in approximately 20 min of training per app before starting the intervention. The training consisted of using a step-by-step printed user guide that walked the professionals through the programming procedure of adding a new symbol for each app. One of the researchers read the guide with the professionals while performing the procedure on the app. The professionals were then asked to perform the programming procedure with the researcher three times, and they were encouraged to ask questions if something was not clear. Then, the professionals were asked to perform the programming procedure three times independently to ensure that performance standards were met and that all the participants knew how to program both apps. The professionals were free to use the printed user guide during the live session with the children. At the end of the training, all the participants were able to program three symbols without any help from the researcher on both apps.
Sessions were ten minutes in length and consisted of engaging the child with the activity “play with bubbles”. Each professional performed one session per child per app. Professionals were asked to use the AAC apps for ten minutes with the child and to program a minimum of one symbol during the interaction. In order to mitigate practice effects, data were collected at least two days apart for each AAC app. The sequence of the AAC apps was counterbalanced across professionals to account for sequence effects. The professionals were randomly assigned to a sequence of conditions (i.e., App 1 then App 2, or vice versa). All the sessions took place in a room with a small table and chairs. The bubbles, the iPad and the printed user guide were positioned on the table. Each professional was told to sit at the table and start playing with the child. To ensure equal time, the activity was monitored with a timer. Intervention sessions were also videotaped for reliability checks. A tripod with a camera was positioned in a corner of the room and pointed toward the table.
The activity, “play with bubbles”, was chosen to compare the AAC apps. This is a gender-neutral activity and was among the favorite play activities of the children involved in the study. The play activity consisted of making and popping bubbles. This activity allows for many communication opportunities between the professionals and the children. It allows for turn taking (e.g., who blows or pops the bubbles), as well as making choices (e.g., many bubbles, a few bubbles, big bubbles, small bubbles, blow bubbles up, blow bubbles down), comments (e.g., I like it, wet, good job, funny), and requests for repetition (e.g., more or all done).
6 Reliability
We calculated reliability for the number of symbols programmed in the apps. We randomly selected samples from both apps, professionals, and children that constituted at least 20% of the sessions. A graduate student in special education served as second coder after training in the coding procedures. We calculated the number of agreements divided by the number of agreements plus disagreements and then multiplied it by 100. The mean interrater reliability for the number of symbols programmed was 96.5% (range: 86%–100%). We also compared the overall time taken to program the symbols in both apps. The time calculated for programming App 1 showed 18 s of difference, whereas the time calculated for programming App 2 showed 1 s of difference.
Four professionals used both App 1 (TalkTablet) and App 2 (Speech-to-Symbols) with two children with communication needs for a total of 16 sessions. Professionals held eight sessions with App 1 and eight sessions with App 2. The symbols added in both AAC apps during interactions with the children were adjectives (13, 42%), verbs (10, 32%), nouns (7, 23%) and subjects (1, 3%). Table 2 shows how all the professionals combined performed while using the two apps.
App comparison.
App comparison.
Prior to the beginning of the study, we asked the professionals to list eight to ten words they predicted they would use the most when playing the bubble activity. They listed 38 words from which we used eight words as part of the target vocabulary. We removed 19 words due to duplication. At the end of the project, we noticed that six of the eleven remaining words were never used by the professionals during any session, while five of them were used during at least one of the sessions. During the sessions, the professionals needed 11 new words inside the grid to effectively engage with the children. None of these words were among the ones that the professionals had predicted they would use (Table 3).
Lists of words predicted and not predicted.
Among the eight sessions, the SUS questionnaire reported an average of 52.88 (SD = 24.173) for App 1 and 78.81 (SD = 3.195) for App 2.
AAC technologies provide solutions that increase the participation and engagement of individuals with a speech disability (Elsahar et al., 2019). For children with severe speech impairments, AAC can substantially improve communication (van der Meer et al., 2012). The Speech-to-Symbols app allowed the communication partners to access their programmed vocabulary faster due to the reduced number of steps require to add a new symbol. The number of symbols programmed showed that by using the Speech-to-Symbol app the communication partner was able to program a larger number of symbols. The reduced operational demands of App 2 allowed the professionals to expose children to a larger number of symbols. This is very important in educational settings, considering that literature studies reported that exposing children to new vocabulary both supports vocabulary expansion and children's language learning (Romski and Sevcik, 1996).
A previous study highlighted the importance of exploring the application of technology based intervention for building communication skills for persons with communicative deficit (Mohan et al., 2019). Other studies have shown that the operational demands of programming AAC devices poses a barrier to their effective use (Caron et al., 2016; Schlosser et al., 2016). The results of this study showed that App 2 allowed communication partners to program symbols faster when compared to App 1. Professionals took more than twice the time to program App 1 than App 2.
Considering the types of symbols programmed, our results showed that most of the symbols used by the professionals were verbs and adjectives. This result differs from previous research studies that reported that the vocabulary typically included in AAC communication boards or devices primarily consists of nouns (Adamson et al., 1992). A more extensive use of verbs and adjectives could help children with a communication disability to engage in natural conversations (Adamson et al., 1992). However, it is possible that the specific activity selected (“play with bubbles”) may have led to more use of verbs and adjectives rather than nouns.
Other research studies demonstrated that the AAC vocabulary is usually implemented outside of the context in which it is going to be used (Caron et al., 2016). However, considering the wide variety of topics occurring in natural conversations, it is difficult for communication partners to predict the required vocabulary prior to any given interaction (Porter and Cafiero, 2009). Our research study confirmed that there is a difference between the predicted required vocabulary and the vocabulary used during an interaction. Comparing the words used in the ACC apps during the interactions with the words listed by the professionals prior to the beginning of the interactions confirmed that predicting vocabulary in advance and out of context of an interaction may result in the selection of an inaccurate and limited vocabulary (Donato et al., 2014; Porter and Cafiero, 2009). Indeed, in looking at the words programmed within App 1 and App 2, not only did App 2 allow for a larger number of symbols to be used, but it also allowed for the most use of unpredicted vocabulary.
Based on the SUS questionnaire, the communication partners felt that App 2 was more usable than App 1. This result is not surprising considering the speech-to-text technology, available in App 2, allowed for quick access to previously stored symbols as well as new symbols that became relevant while communication partners engaged with the children.
There are several limitations in this research study. The sample size of children and professionals was small. Although the features of the two apps under investigation were similar, App 1 was not the only app currently available in the market. Therefore, we cannot rule out the possibility that another traditionally designed AAC app may have provided different outcomes.
Another limitation relates to the lengths of the experimental sessions, which were set to ten minutes. Although the duration of this session is consistent with other research studies comparing AAC apps (Caron et al., 2016), a longer session may have provided more communication opportunities and, therefore, both the number of symbols used, and the vocabulary implemented would have been larger. The sessions in our study were based on a single play activity. Communication partners were chosen from those with extensive AAC experience. For this reason, we cannot be sure how successful communication partners with less experience would be able to use the two apps.
This research study may indicate several possible directions for future studies. First, App 2 should be used by a larger number of communication partners, including parents, in multiple play and day-to-day activities. Second, considering that an AAC app with reduced operational demands exposes children to more vocabulary, future research studies should focus on verifying whether the vocabulary to which children are exposed during interactions is actually learned and then used appropriately in their communication. While the professionals were able to add symbols faster using App 2, the new vocabulary programmed could not be maintained and organized in a structure for longer-term learning benefits. Multiple communication partners could overuse the capability offered by App 2 and they may not devote enough time to preparing and organizing the session with the child with a disability.
Conclusion
We developed an AAC app with reduced operational demands to better meet the needs of communication partners for implementing vocabulary during interactions with children with disabilities. The results of this study demonstrated that the reduced operational demands of a Speech-to-Symbols app increases the number of symbols programmed during an interaction. Moreover, when compared to a traditionally designed app, it was easier to find different types of symbols on App 2 when the need for additional vocabulary arose unexpectedly during an interaction.
Our app was considered more usable by the professionals. This result reinforces the previous literature that reported that the design of AAC devices should meet the communication needs of both the communication partners and the individuals who use the AAC devices (Blackstone et al., 2007). Previous research has also reported that brief speech and language therapy training sessions can lead to increased use of some interaction strategies that help children's communication skills develop (McDonald et al., 2015).
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
