Repurposing a Smartwatch to Support Individuals With Autism Spectrum Disorder: Sensory and Operational Considerations

Abstract

Smartwatches may provide a natural, portable, and unobtrusive strategy by which to support directive-following for individuals with autism spectrum disorders (ASD). A mentor can send visual supports (e.g., photographs, videos, text) “just-in-time” (JIT) to a learner’s smartwatch. This may reduce the need for extraneous face-to-face intervention within social and educational settings and thereby preserve the naturalness of interactions for individuals with ASD. In this article, the following questions are explored: (1) Will some children with ASD with limited functional speech tolerate wearing a smartwatch after receiving only spoken instructions? (2) Can children with ASD who did not initially tolerate wearing a smartwatch increase their tolerance for wearing a smartwatch after they view video models within a visual schedule? and (3) Will children with ASD who tolerate wearing a smartwatch attend to, and follow, scene cue text messages received on the watch? Results provide preliminary evidence for the feasibility of using a smartwatch with children with ASD. Limitations of this study are addressed and implications for future research are posited.

Keywords

augmentative and alternative communication autism spectrum disorder directive-following sensory tolerance just-in-time wearable technology

Individuals with autism spectrum disorder (ASD) often demonstrate preferences for technology and visual supports (Mechling et al., 2006; Shane & Albert, 2008). Additionally, many of these visually based supports have been established as evidence-based practice (Wong et al., 2014). As such, special educators have increasingly repurposed technological devices (e.g., tablets, computers, phones) to provide visual supports to enhance the communication, learning, and independence of individuals with ASD (Odom et al., 2015; Virnes et al., 2015). Novel advances in general consumer-level technology, including wearable technology (e.g., Apple Watch®, Google Glass™), create new and potentially more discreet, naturalistic, and real-time strategies by which to support these individuals (Keshav et al., 2017; O’Brien et al., 2016; Schlosser et al., 2017; Torrado et al., 2017).

To date, clinicians have primarily repurposed technology to support expressive communication for individuals with ASD who have limited functional speech (Schlosser & Koul, 2015). Within the field of augmentative and alternative communication (AAC), individuals with ASD who use speech output technologies (e.g., computer and tablet applications that produce speech when activated) have demonstrated an increase in requesting and a reduction of challenging behaviors (Schlosser & Koul, 2015). In addition to expressive language deficits, some children with ASD demonstrate difficulty with receptive language skills (Rice et al., 2005). In contrast to expressive language interventions, however, there is limited research that focuses on the receptive language interventions for individuals with ASD (Muller & Brady, 2016; Sevcik, 2006). As such, it is essential that clinical researchers explore receptive language strategies that leverage the visual and technological preferences of this population.

New technological features (e.g., Bluetooth, high-speed Wi-Fi, global positioning system), the mobile technology revolution in AAC (McNaughton & Light, 2013; Shane et al., 2012), and an emerging literature related to “just-in-time” (JIT) supports as it pertains to AAC (O’Brien et al., 2017; Schlosser et al., 2016) provide a rationale for exploring the presentation of receptive language supports using mobile technology. O’Brien et al. (2017) defined the JIT construct as the “rapid creation and delivery of behavioral, organizational, and language supports including prompts, reminders, and rewards in a timely fashion” (p. 298). A mentor can provide JIT prompts in precisely the moment that they are needed to enhance the situational opportunity to learn, alter behavior, or aid comprehension. JIT intervention capitalizes on situated cognition, teachable moments, and reduced working memory demands (Schlosser et al., 2016). Some supports are considered to be JIT-based because of the timing of the support, while other JIT supports may involve the creation of the needed materials “on the fly” (Caron et al., 2016; Light et al., 2019).

While mentors may use a range of technology (e.g., tablets, phones) to provide JIT supports, recent advances in wearable technology (e.g., smartwatches) may provide an ideal and practical way to unobtrusively enhance the implementation of JIT prompts. For example, a classroom aide could send a visual message to a student’s smartwatch if the student is confused during a transition. Upon receiving the message that depicts the next activity, the student may better understand task expectations and subsequently complete the activity correctly. In this scenario, the aide subtly provides JIT visual support to enhance the student’s comprehension and independence. Previous research that involved holding up the watch display to participants supports the feasibility of using the relatively small display size of the Apple Watch to show visual supports (e.g., photographs, video clips) to children with ASD in a JIT manner (O’Brien et al., 2016; Schlosser et al., 2017). This research, however, did not require that the child wears the Apple Watch. The field is lacking research that examines the tolerance of students with ASD for wearing smartwatch technology, and its impacts on using this JIT strategy.

Wearable Technology and ASD

Given the sensory sensitivities of many individuals with ASD, understanding this population’s tolerance of wearable technology remains an important, yet relatively unstudied, question. Over 90% of children with ASD demonstrate atypical sensory responses (Leekam et al., 2007). In fact, hyper- and hyposensitivity to sensory input (e.g., sound, texture, taste, smell) are a diagnostic feature of ASD (American Psychological Association, 2013). Cascio et al. (2008) assessed sensitivity response to touch at the forearm in adults with ASD and found normal thresholds for the detection of touch and temperature, with increased sensitivity to vibration on the forearm. While similar research does not exist for children with ASD, these findings underscore the importance of understanding whether individuals with ASD will tolerate wearing wrist-based technology.

Researchers in the fields of affective computing, education, and sleep disorders have established a precedent for the use of wearable technology for individuals with ASD. Wearable sensors may be useful in understanding the relationship between an individual’s personal biomarkers and subsequent behaviors (el Kaliouby et al., 2006; Picard & Goodwin, 2008). Individuals with ASD and intellectual disabilities may also benefit from using watches to enhance the use of self-regulation strategies and on-task behaviors (Evmenova et al., 2018; Finn et al., 2015; Green et al., 2011; Torrado et al., 2017). Further, wearable devices with cameras may help individuals with ASD better understand social interactions (Madsen et al., 2008).

Medical professionals use wrist-based monitors (i.e., actigraphs) to better understand an individual’s sleeping patterns (Fawkes et al., 2015). Children with developmental disorders have demonstrated improved tolerance of actigraphs when they are placed on the ankle instead of the wrist and when professionals provide parent training (e.g., encouraging a clearly defined bedtime, use of a practice wristband; Fawkes et al., 2015; Souders et al., 2009). When considering the use of a smartwatch with children with ASD, alternative placement of the device is not an option. Use of a practice watch may be one strategy to teach children to tolerate the smartwatch. To our knowledge, despite the growing potential uses of wearable technology, a patient-centered protocol for teaching children with ASD to tolerate new forms of wearable technology does not exist.

Video modeling, an intervention considered to be evidence-based by the National Professional Development Center on ASD (Wong et al., 2015), may be a useful strategy for teaching children with ASD to wear and use new technology. Video models are video clips of novel behaviors that are used to teach the skills depicted by the videos. A series of related video models can be embedded within a visual schedule to provide a narrative framework (Shepley et al., 2018; Shepley et al., 2019; Spriggs et al., 2015). A visual schedule with embedded videos may be a useful strategy to teach individuals with ASD to tolerate, wear, and use novel technologies.

Application of Visual Supports to Wearable Technology

If an individual with ASD tolerates wearing a smartwatch, the successful implementation of JIT supports, as described here, requires that the individual also responds to, and follows, directives delivered to the watch. While the watch can provide auditory, visual, and audiovisual prompts, in this study, we focus exclusively on the use of visual JIT supports due to the documented benefits of visual supports for individuals with ASD (Shane et al., 2015). Shane et al. (2015) hypothesized that the visual strengths observed in this population may compensate for deficits in receptive language skills. Extant literature indicates that the use of visual supports (e.g., visual schedules, video models) enhances several skill areas including directive-following, transitions between activities, independence within activities, and a reduction of challenging behaviors for individuals with ASD (Lequia et al., 2012; National Autism Center, 2015).

Visual supports have also been used to enhance comprehension of language concepts embedded within spoken directives (Schlosser et al., 2013). Dynamic and static scene cues (i.e., short video clips or photographs, respectively, that depict the relevant concept in the corresponding directive) are an effective visual strategy for supporting directive-following in children with ASD, as compared to the provision of spoken directives alone (Remner et al., 2016; Schlosser et al., 2013). Conventionally, an instructor presents these visual supports on tablets or smartphones while face-to-face with the child. Wearable technology (e.g., smartwatches) may provide a less obtrusive and more naturalistic medium for providing JIT visual supports (e.g., scene cues) to children with ASD. Specifically, if JIT supports can be delivered to children with ASD in an unobtrusive, yet salient, manner (e.g., via text message to the wrist), the naturalness of the ongoing social or educational context may be preserved, while the requisite prompt is maintained.

To date, the study of wearable supports to enhance language comprehension of individuals with moderate to severe ASD is limited. Previous research has explored the viability of using the Apple Watch screen to present scene cues to individuals with ASD (O’Brien et al., 2016) and to individuals with ASD who also have an additional diagnosis of intellectual disability (Schlosser et al., 2017). These studies determined that the screen size of the Apple Watch was sufficient for the participants to view and follow directives portrayed in static and dynamic scene cues. Additionally, these studies demonstrated the effectiveness of a JIT approach to the delivery of scene cues. Due to the preliminary nature of these initial studies, however, the participants were not required to wear the Apple Watch or respond to text messages. Instead, the researcher presented the scene cues on the watch that she held in front of each child.

In the present study, we aim to extend previous findings by addressing the following research questions: (a) Will some children with ASD with limited functional speech tolerate wearing a smartwatch after receiving only spoken instructions? (b) Can children with ASD who did not initially tolerate wearing a smartwatch increase their tolerance for wearing a smartwatch after they view video models within a visual schedule? and (c) Will children with ASD who tolerate wearing the watch attend to, and follow, notifications of scene cue text messages received on the watch?

Method

This study was approved by the relevant institutional review board. Informed written consent was obtained from a parent of all participants included in the study.

Participants, Setting, and Researcher

Participants met the following inclusion criteria: (a) primary diagnosis of ASD (as indicated by medical records), (b) chronological ages between 6 and 16 years, (c) hearing and vision within normal limits (based on medical records and/or parent report), (d) strong interest in screen media (per observations and parent report), and (e) ability to complete screening activities with social reinforcement only (see screening procedures below). Participants were recruited following the completion of their biannual follow-up AAC evaluation based on meeting the inclusion criteria. A convenience sample was used.

Ten children met the inclusion criteria and participated in the screening. Eight children participated in the full study; two participants were excluded from the study due to experimental error. Participants ranged in age from 6.8 to 15.1, demonstrated limited functional speech, and benefited from using AAC strategies to support their expressive and receptive communication skills. One participant (Participant 8) had a diagnosis of intellectual disability secondary to ASD (see Table 1 for participant characteristics). The researcher sent scene cue directives to the five participants who successfully wore the watch (see Figure 1, a flowchart of procedures). No incentives were provided to the participants.

Table 1.

Participant Characteristics.

Participant	Diagnosis	CA	Sex	Communication Characteristics
Participant	Diagnosis	CA	Sex	Speech	Manual Signs	Aided Strategies
1	Autism spectrum disorder	14;4	F	Uses <50 single spoken words to request and label	Uses several manual signs (e.g., more, help)	Uses the GoTalk NOW speech generating application to communicate
2	Autism spectrum disorder	6;10	M	Uses several single words and spoken approximations to request, comment, and greet others	Uses a few manual signs (e.g., more, all done)	Uses TouchChat HD speech-generating app to communicate
3	Autism spectrum disorder	13;5	M	Spontaneously uses <50 spoken word approximations (limited intelligibility) to protest, request, comment, answer personal information questions, and direct	No manual signs reported or observed	Uses the NOVA chat SGD to communicate
4^a	Autism spectrum disorder	10;0	F	Spontaneously uses <50 spoken word approximations (limited intelligibility) to protest, request, comment, and direct	Uses physical communication to request and direct	Uses the TouchChat HD-AAC and WordPower speech generating application to communicate
5^a	Autism spectrum disorder	12;3	M	Uses <50 single spoken words to request and label	No manual signs reported or observed	Uses the NOVA chat SGD to communicate
6^a	Autism spectrum disorder	9;4	F	Uses <100 spoken single words and several two to three words phrases to protest, request, comment, and direct	No manual signs reported or observed	Uses the NOVA chat SGD to communicate
7^a	Autism spectrum disorder	6;10	M	Uses spoken sentences (3+ word phrases) to protest, request, comment, ask and answer questions, and direct	No manual signs reported or observed	Benefits from social narratives and video models to support comprehension of social situations
8^a	Autism and intellectual disability	15;1	M	Uses <20 single spoken words to protest, request, and direct	Uses several two to three words scripted phrases to request (e.g., “I want [x]”) and ask questions (e.g., “Where’s [x]?”)	Uses the NOVA chat SGD to communicate

Note. CA = chronological age; SGD = speech-generating device.

^a Also participated in the scene cue portion of the study.

Figure 1.

Flowchart depicting the phases of the experimental procedures and rationale for participant inclusion and exclusion.

The study was performed in a 12 × 10 square feet clinical treatment room at a pediatric ASD center. The room was equipped with a table and chairs. During the experimental procedure, the participating child sat at a table next to the researcher.

A licensed speech-language pathologist experienced with children with ASD served as the researcher. Additionally, a graduate student in speech-language pathology served as an independent observer. The independent observer stood behind the participant.

Design

A case series design (Carey & Boden, 2003; Kooistra et al., 2009) was used to identify participants with ASD who could tolerate wearing the Apple Watch.

Measures

The following measures were used: (a) tactile sensitivity survey (see Online Appendix A), (b) observations of watch wearing following spoken and video instruction, (c) observations during the acclimation period, and (d) observations during the scene cue portion of the study.

Tactile sensitivity survey

The researcher administered an 8-item parent survey related to each participant’s sensory sensitivity. This survey was used because the communication status of the participants (i.e., little or no functional speech) precluded the researchers from simply asking the participants whether they would tolerate wearing the watch. This yes–no response survey included four relevant questions from the tactile sensitivity portion of the Short Sensory Profile and Sensory Processing survey, which has been shown to demonstrate adequate internal validity and discriminate validity (Tomchek & Dunn, 2007). The most relevant questions to the task of wearing a watch were selected from the tactile sensitivity portion of the survey. Four questions about touch sensitivity and parent prediction of watch-wearing success were added to the survey to obtain more specific information about our study’s specific sensory task. Questions were randomized, such that yes and no answers indicated both sensory sensitivity and nonsensitivity responses.

For each child, the total number of “responses suggesting sensitivity” was calculated to determine a “sensitivity quotient.” This quotient was considered in relation to watch-wearing success for each child. The number of responses suggesting sensitivity (across all participants) was calculated for each question (e.g., for the question, “Does your child wear hats?” the response suggesting sensitivity issues is “No”). For each question, the fraction of responses suggesting sensitivity for children who did not tolerate the watch was calculated (i.e., sensitivity quotient). This quotient was used to identify questions that seem more “predictive” of smartwatch tolerance.

Observations of watch wearing following spoken and video instruction

The researcher made observations related to whether the participants wore the watch with (a) spoken instruction or, if spoken instruction was unsuccessful, (b) given video models within a visual schedule. A teaching strategy was deemed unsuccessful if the participant became agitated while wearing the watch or tried to remove the watch. If the participants did not demonstrate frustration-related behaviors, the acclimation period was initiated.

Observations during the acclimation period

To judge the success of the acclimation period, the researcher observed participants’ willingness to wear the watch and ability to engage in another activity while wearing the watch for 10 min. Participants were deemed to be successfully acclimated to wearing the watch if they (a) wore the watch for the 10-min acclimation period, (b) engaged in a motivating activity while wearing the watch, and (c) did not become agitated while wearing the watch. Participants were determined to be partially successful in wearing the watch if they tolerated the watch for part of the acclimation period but indicated that they wanted the watch removed before the end of the 10-min acclimation period.

Directive-following based on scene cues received on the smartwatch

Following a successful acclimation period, the participants who tolerated wearing the watch during the acclimation period (see Figure 1 for inclusion and exclusion information) received static scene cues on the Apple Watch (see Figure 2 for a sample scene cue of “Put the boy on the horse”). The experimenter used the Messages application on an iPad® to send five static scene cues to the Apple Watch and gave each participant 10 s to follow the directive depicted by the scene cue.

Figure 2.

Example static scene cue depicting the directive “Put the boy on the horse.”

A response was considered correct if the participant independently executed the depicted directive by positioning the agent (e.g., boy) in relation to the object (e.g., horse) as indicated by the prepositional phrase (e.g., on) within 10 s. If the child did not implement the directive correctly, the next scene cue was sent. If the participant did not respond or did not attend to the received message after 10 s, gestural and spoken prompting was used to show the participant the scene cue message on the watch (e.g., “Look at the watch, you do it”). If a child successfully implemented the directive within 10 s of the researcher providing a prompt, the response was considered “correct with support.” If a child implemented the directive incorrectly or still did not respond following a spoken prompt, the response was marked as incorrect and the next scene cue message was sent.

Interobserver agreement (IOA) data were collected for two of the five (40%) scene cue participants. IOA was calculated by counting the number of agreements divided by agreements plus disagreements, multiplied by 100. IOA was 100%.

Materials

Materials included (a) an iPad Air®, (b) video models embedded within a visual schedule on the Pictello application (see Online Appendix B), (c) the Apple Watch Sport (Model A 1554, 42 mm size, 1.65″ Ion-X glass retina display, 312 × 390 pixels resolution, and composite black), (d) scene cues sent via the Messages application, (e) photographs and objects for the screening task, and (f) figurines and objects required for the participants to carry out directives on the tabletop (i.e., grapes, toy table, cat, boy, rocking horse, small cardboard box).

The digital visual schedule included seven pages; each page contained a photograph or video depicting the steps required to wear the Apple Watch. There was a short text-based description of each step on every page, which was read aloud by a synthesized voice.

The five scene cues depicted the preposition “in” or “on” along with an object and a location (i.e., “put the grapes on the table,” “put the cat on the table,” “put the boy on the rocking horse,” “put the grapes in the box,” “put the rocking horse in the box”; see Figure 2 for an example). Preposition-based scene cues (in contrast to verb-based scene cues) were used because they previously resulted in greater success (O’Brien et al., 2016). Static scene cues were chosen instead of dynamic scene cues, as they have previously supported directive-following (Schlosser et al., 2013) and require fewer operational competencies relative to dynamic scene cues because they do not require the extra step of activating the video upon receipt.

Procedures

Screening tasks

Two brief screening tasks, previously used during the initial Apple Watch feasibility studies, were administered to determine inclusion (O’Brien et al., 2016; Schlosser et al., 2017). First, the researcher presented each participant with six photographs (one at a time) on an iPad. Each photograph depicted an object that was on the table in front of the child (in an array of 6 items). For each photograph, the researcher instructed the children to match the corresponding item to the iPad screen within 10 s of the researcher saying “Match (name of object).” In order to be included, children needed to match at least three of six objects to the respective photographs.

Next, the participants were given five spoken directives paired with the corresponding static scene cues on the iPad. Static scene cues are “photographic or pictorial visual scenes that portray relevant concepts and their relationships” (Schlosser et al., 2013, p. 132). The researcher presented the scene cue in conjunction with a spoken directive (e.g., “Have the boy kick the ball”). The participants were given 10 s in order to carry out each directive with the corresponding items on the table (in an array of six). In order to be included, children needed to complete at least three of five directives. Children were provided with intermittent nonspecific reinforcement (e.g., “Nice job”) to sustain attention and motivation to both screening tasks. All children passed the screening tasks and therefore enrolled as participants in the study. Per medical records, participants had previously demonstrated successful completion of scene cues during prior visits to the clinic.

Tactile Sensitivity Survey

Before presenting the Apple Watch to the participants, the researcher administered the Tactile Sensitivity Survey to a parent of each participant (see Online Appendix A). The survey was completed face-to-face. The researcher asked the questions orally and filled out the responses.

Instruction to wear the watch

The researcher showed the Apple Watch to the participants and instructed each child that he or she would wear the watch for 10 min. Initially, only spoken instruction was provided (i.e., “You are going to wear this watch for 10 min. Put out your arm. I will put the watch on your wrist.”). The researcher provided gestural cues and in-person models as needed to support participant understanding of the spoken instructions. If the participant tolerated wearing the watch, the acclimation period began. If the participant pushed the watch away or otherwise communicated that they did not want to wear the watch (e.g., via gestures, speech-generating device, vocalizations), the researcher put the watch on the table.

When spoken instruction alone was insufficient in supporting a child to wear the watch, the researcher presented a digital visual schedule with embedded video models on the iPad (see Online Appendix B). The researcher read the schedule with the participant and activated the videos that were embedded within the narrative. After reading the instructional schedule, the researcher placed the watch on the participant’s wrist. If the participant tolerated wearing the Apple Watch, the acclimation period began. If the child resisted wearing the watch a second time, the watch was removed and the study was discontinued for that participant.

Acclimation period

During the acclimation period, participants were asked to wear the watch while engaged in a motivating activity (e.g., playing on an iPad, playing with cars, watching videos) for 10 min. The researcher observed, and narratively noted, whether the participant demonstrated any discomfort associated with wearing the watch and whether the participant appeared to focus on another activity while wearing the watch. If a participant demonstrated discomfort while wearing the watch (e.g., attempted to remove the watch, became upset), the researcher removed the watch and the study was terminated for this participant.

Directive-following based on scene cues received on the smartwatch

For each of the five participants with a successful acclimation period, the researcher placed all of the toy figurines described above on the table in front of the participant. The researcher informed each participant that a “picture” (i.e., static scene cue) would appear on the watch and sent each of the five static scene cues with a haptic and auditory notification. For the first scene cue directive, the researcher provided gestural (i.e., pointing to the watch) and verbal prompts (e.g., “You have a picture,” “Look at your watch,” “You do it”) for all participants. For all subsequent directives (i.e., directives two through five), the researcher then provided least to most prompting. In other words, for directives two through five, the researcher provided the gestural and spoken prompts described above only if the participant did not independently attend to, and/or follow, the scene cue within 10 s of receiving the cue. There was a 3 s intertrial interval following each directive. Throughout the study, the researcher provided nonspecific intermittent feedback (e.g., “Nice work”) to sustain participation.

Results

Tactile Sensory Survey

Sensory-sensitive responses (i.e., responses suggesting tactile sensitivity) were summed for each participant to determine each individual’s sensory quotient (SQ). Individual participant SQ values are provided in Table 2. SQ values ranged from 2 to 6, with a higher score indicating increased sensory sensitivity (Mean = 3.25, SD = 1.75). When comparing each individual’s tolerance of the watch (as observed during the acclimation period) to their SQ, all participants who tolerated wearing the Apple Watch received an SQ within 1 SD of the mean. Two SQ scores fell more than 1 SD above the mean (Participants 2 and 3) each with an SQ of 6, and neither of these participants tolerated wearing the Apple Watch. One participant who did not tolerate the Apple Watch fell within 1 SD of the mean (Participant 1, SQ of 3).

Table 2.

Individual Participant Responses to Sensitivity Questionnaire and Actual Tolerance of Wearing the Watch.

Participant	Sensitivity Quotient	Tolerate Watch?
1	3/8 (37.5%)	No
2	6/8 (75.0%)	No
3	6/8 (75.0%)	No
4	2/8 (25.0%)	Yes
5	2/8 (25.0%)	Yes
6	3/8 (37.5%)	Yes
7	2/8 (25.0%)	Yes
8	2/8 (25.0%)	Yes

The number of responses suggesting sensitivity was summed across participants for each question to explore whether specific questions on the Tactile Sensitivity Survey were more indicative of difficulty wearing the watch than other questions (see Table 3). The number of responses suggesting sensitivity for each question ranged from 2/8 or 25% (lowest number of responses suggesting sensitivity to the items: “Does your child wear hats?” [response suggesting sensitivity: no], “Does your child look startled after a sudden touch?” [response suggesting sensitivity: yes], and “Do you think your child will wear a watch for 10 min?” [response suggesting sensitivity: no]) to 5/8 or 62.5% (greatest number of responses suggesting sensitivity to the item: “Do certain types of fabric seem to make your child uncomfortable?” [response suggesting sensitivity: yes]).

Table 3.

Results From Sensitivity Questionnaire.

Question	Sensitivity Response^a	% of Sensitivity Responses	% Predictive Value of Sensitive Responses
Does your child wear hats?	No	25.0	50.0 (1/2)^b
Does your child look startled after a sudden touch?	Yes	25.0	50.0 (1/2)^c
Do certain types of fabric seem to make your child uncomfortable?	Yes	62.5	40.0 (2/5)
Does your child show discomfort from clothing labels/tags?	Yes	50.0	50.0 (2/4)
Is your child especially ticklish?	Yes	37.5	33.3 (1/3)
Does your child wear a bracelet, watch, wrist ware, or ankle ware?	No	62.5	40.0 (2/5)
Do you think your child will wear a watch for 10 min?	No	25.0	100 (2/2)
Do you think your child will continue wearing the watch if it vibrates on their wrist?	No	37.5	100 (3/3)

^a Yes indicates potential sensory sensitivity. ^bOne of the three participants who did not wear hats did not wear the watch. ^cOne of the two participants who looked startled after a sudden touch did not wear the watch.

To explore the relevance of sensitivity responses for each item, we calculated the percentage of participants with a sensory sensitive response who also did not tolerate the watch to determine a potentially predictive value of the sensitive responses (see Table 3). Two questions had predictive values of 100%. For the question, “Do you think your child will wear a watch for 10 min?” two of two children with a sensory sensitivity response did not wear the watch (100% predictive value). For the question, “Do you think your child will continue wearing the watch if it vibrates on their wrist?” three of three children with a sensory sensitivity response did not wear the watch (100% predictive value).

Watch Wearing

Overall, five of the eight (62.5%) participants learned to wear the watch and tolerated the watch during the acclimation period.

Spoken instruction

Two of the eight participants (25%) tolerated the sensation of wearing the watch following spoken instruction alone. One participant (12.5%) became upset after the spoken instruction, and the study was discontinued due to frustration.

Visual instruction

Four of the eight participants (50%) learned to wear the watch after viewing video models within a visual schedule. One participant (12.5%) tolerated the watch for a short period of time after visual instruction only to remove the watch after 2 min of the acclimation period. Two participants (25%) did not tolerate the watch after spoken or visual instruction.

Acclimation Period

During the acclimation period, all five participants who successfully wore the watch engaged in a motivating activity that he or she chose (e.g., watching videos on an iPad, playing with a toy microphone). None of the participants who tolerated wearing the Apple Watch became distracted by the watch. The five participants who tolerated the watch did not express or indicate discomfort using speech, vocalizations, gestures, or a speech-generating device. Notably, three of these participants were reported to currently wear bands on their wrist (e.g., watch, name tag, bracelets). Of these children, two participants tolerated wearing the watch, and the third child demonstrated partial tolerance.

Directive-Following Based on Scene Cues Received on the Smartwatch

Each of the five participants who received scene cue messages followed five of five (100%) of the directives depicted within the scene cues sent to their watch for a summed total of 25 (100%) correct responses (see Figure 3). All participants were initially provided with gestural and spoken prompts to attend to the received message. All participants independently attended to at least one message without prompting to look at the watch. Participant 7 required only a single instructional trial to learn to look at the watch when a message was received. All participants continued to wear the watch for the entire scene cue task. The use of the haptic and auditory cues to signal the arrival of a message appeared to support increased attention to the received message and did not appear to reduce tolerance of watch wearing for these learners.

Figure 3.

Total number of correct responses to directives across participants by level of prompting.

Discussion

The results from this study, and previous research in which scene cues were shown in a JIT manner (O’Brien et al., 2017; Schlosser et al., 2016), provide proof of concept that a smartwatch is a viable option for supporting JIT directive-following for some children with ASD who present with little or no functional speech. Five of eight children in this study tolerated wearing the watch. Five of five children responded to the haptic and auditory cue and carried out the directives depicted by the visual supports.

This study provides preliminary evidence that some children with ASD who present with little or no functional speech tolerate the sensation of wearing the Apple Watch during a structured session. Outcomes from the Tactile Sensitivity Survey provide initial information regarding the profile of a child who may tolerate wearing the watch. Specifically, children with a higher sensitivity quotient (i.e., participants with SQ scores of 6 or higher, indicating increased sensory sensitivity) were noted to demonstrate a reduced tolerance of wearing the watch. Importantly, one participant with an average SQ did not tolerate the watch, suggesting that the survey should not be the only indicator used to assess potential of watch tolerance. As one may expect, parent perception of their child’s willingness to wear the watch (Item 7), and continue to wear the watch if it vibrated (Item 8), appeared to be the survey items that were most strongly related to a child’s tolerance of wearing the watch. Additional research with a larger sample size is necessary to determine whether the survey may have predictive value in the statistical sense.

The use of the Tactile Sensitivity Survey prior to introducing the Apple Watch may increase a mentor’s understanding of possible sensory aversion, which could be useful in assessing a child’s potential watch-wearing tolerance. For example, a child with limited sensory aversion may only require spoken instruction and a single opportunity to learn to wear the watch. In contrast, a child who demonstrates high sensory aversion on the survey, including parent prediction that the child will not tolerate wearing the watch, will likely require additional support (e.g., video models) before learning to wear the watch. Of course, we do not expect that the use of an Apple Watch will be an appropriate strategy for all learners with ASD; some children with ASD will likely not tolerate wearing the watch regardless of the teaching strategies used.

Two of the eight participants wore the smartwatch when given the spoken instruction alone. A digital visual schedule with embedded video models was noted to increase the number of participants who wore the watch from two to five of the eight participants. This is consistent with previous research that demonstrated the benefit of embedding video models into visual schedules to teach new skills (Spriggs et al., 2015). To our knowledge, this study is the first to attempt to outline a child-centered protocol to support tolerance of wearable technology. The use of video models within a visual schedule to support sensory tolerance of wearable technology may have implications for other wearable devices and medical fields, including that of actigraphy, a noninvasive method to monitoring human rest/activity cycles, and affective computing.

All of the participants who tolerated wearing the watch during the acclimation period (five of five) participated in other activities while wearing the watch. This suggests that some children with ASD can successfully wear the watch and simultaneously participate in other activities. Providing an entertaining activity as a distractor during initial acclimation to the watch may be a useful clinical strategy to build initial tolerance.

In addition to tolerating the sensation associated with wearing the watch, some children with ASD will tolerate the sensation of receiving haptic and auditory notifications that signify the receipt of a text message on the Apple Watch. The five participants who tolerated wearing the watch independently responded to the haptic and auditory prompts and followed static scene cues received as text messages. That is, the participants followed the necessary steps to view the text message, including to recognize that a message was received, to raise and tilt their arm toward their body, and to view the scene cue.

Additionally, and consistent with previous research, participants followed the directives that were depicted by the scene cues on the Apple Watch (O’Brien et al., 2016; Schlosser et al., 2017). While this study used directives containing prepositions, previous research has also shown that static scene cues (where the action is implicit, not explicit) depicting verbs also support directive-following in children with ASD (O’Brien et al., 2016; Schlosser et al., 2017). For example, a child who sees a static scene cue that depicts “make the doll push the car” does not simply match the objects to what is depicted in the photograph (e.g., lean the doll against the car), but rather the child will act on the object (e.g., use the doll to physically move the car). As such, we posit that children are not merely imitating or matching objects to the static scene cue but internalizing the depicted construct, extrapolating the conceptual information, and reconstructing (i.e., acting out) the concept with physical objects. This would suggest that scene cues are a visual medium through which to convey novel directives. This hypothesis should be further examined.

Although the researcher in this study sat next to the participants, the participants’ independent responses to the messages suggest that it may be feasible for children with ASD to receive and follow visual supports sent from afar. This would increase learner independence by increasing the amount of space between the learner and the mentor and by reducing what might be experienced as “hovering” of support personnel and reducing the obtrusive nature of the prompt.

Limitations

Limitations of this study include the small sample size, especially given the heterogeneity of the ASD population, and the introduction of the watch within a highly structured clinic setting. Due to the small sample size, the data reported from the Tactile Sensitivity Survey are descriptive and should not be interpreted in a normative or inferential manner.

Due to resource limitations, procedural fidelity data were not collected and IOA was only collected for 40% of the scene cue participants. Further, no social validity data were collected for this study. Social validation data would be useful in future studies, particularly intervention studies, to determine the acceptability of intervention procedures and perceived outcomes.

Additionally, we recognize that participants only experienced a single session exposure opportunity during a relatively short period of time. It is possible that a longer desensitization period may result in greater watch-wearing success or that increased exposure to the watch will reduce the novelty of the watch-wearing experience and thereby decrease a participant’s interest in wearing the watch. We propose future research that examines the generalized use of a smartwatch within school, home, and community-based settings. The authors acknowledge that the case series design used has limitations and therefore needs to be followed by appropriate single-case experimental designs or group experimental designs to establish a functional relationship between teaching the use of the Apple Watch and the outcomes targeted.

Technology-related limitations of using the Apple Watch include the requirement that the watch must be associated with, and paired to, an iPhone to function properly. Connections to the Apple Watch can be Wi-Fi dependent or cellular-based. If the connection is based on Wi-Fi connectivity, the sender and receiver need to be within a Wi-Fi-enabled location. Relatively, high-speed Wi-Fi is necessary for rapid message delivery because a message sent via “weaker” Wi-Fi may be delayed and therefore preclude its use in a JIT manner. To expand to environments that do not have Wi-Fi connectivity (e.g., a playground, bus stop, grocery store), a cellular connection is required.

Future Directions and Implications

The feasibility of using the Apple Watch to provide visual supports (e.g., scene cues) in a JIT manner potentially provides practitioners with a host of new options for using a JIT approach to support directive-following. For example, scene cues can be sent to a learner’s smartwatch to enhance directive-following, improve play skills, and facilitate transitions. As a next step, it would be important to determine whether support personnel, such as teacher aides, can recognize that a child needs support in the natural environment and then provide the necessary visual supports via text message on the fly from a distance. If the distance between the sender and the receiver were increased successfully, support personnel could capitalize on the potential of the Apple Watch to be an unobtrusive tool that increases independence.

Practitioners who are considering introducing a smartwatch to individuals with ASD may benefit from the structured protocol outlined in this investigation (a) present scene cues on a tablet to ensure comprehension of the task expectations associated with viewing a scene cue, (b) present scene cues on the smartwatch to ensure that images on the watch face are understandable, (c) use spoken instruction and/or a visual schedule with video models to teach how to wear the watch, (d) send static or dynamic scene cues via text message within a highly structured activity (e.g., tabletop activity), (e) provide initial prompting to view and follow the message and fade prompting over time, and (f) after a child achieves watch-wearing success, introduce cues within natural contexts (e.g., to support transitions at school or deliver functional directives). While an Apple Watch was used in this study, the above protocol may be applied to any smartwatch with the capacity to receive photographs via text message.

The authors also recognize that the cost of smartwatches may be prohibitive for some school teams and families. However, due to the rapid rate of changing technology, decreasing costs of consumer products, and an ever-increasing number of JIT-enabling features, it is expected that many of these limitations will be reduced over time. As these technological challenges are overcome and as new JIT-supporting features are developed, the potential benefits of using wearable technology to support children with ASD will only improve.

Supplemental Material

Supplemental Material, sj-pdf-1-jst-10.1177_0162643420904001 – Repurposing a Smartwatch to Support Individuals With Autism Spectrum Disorder: Sensory and Operational Considerations

Supplemental Material, sj-pdf-1-jst-10.1177_0162643420904001 for Repurposing a Smartwatch to Support Individuals With Autism Spectrum Disorder: Sensory and Operational Considerations by Amanda M. O’Brien, Ralf W. Schlosser, Christina Yu, Anna A. Allen and Howard C. Shane in Journal of Special Education Technology

Supplemental Material

Supplemental Material, sj-pdf-2-jst-10.1177_0162643420904001 – Repurposing a Smartwatch to Support Individuals With Autism Spectrum Disorder: Sensory and Operational Considerations

Supplemental Material, sj-pdf-2-jst-10.1177_0162643420904001 for Repurposing a Smartwatch to Support Individuals With Autism Spectrum Disorder: Sensory and Operational Considerations by Amanda M. O’Brien, Ralf W. Schlosser, Christina Yu, Anna A. Allen and Howard C. Shane in Journal of Special Education Technology

Footnotes

Authors’ Note

Amanda M. O’Brien is now at the Program of Speech and Hearing Bioscience and Technology, Harvard University.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Amanda M. O’Brien

Supplemental Material

Supplemental material for this article is available online.

References

American Psychological Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Carey

T. S.

Boden

S. D.

(2003). A critical guide to case series reports. Spine, 28, 1631–1634.

Caron

Light

Drager

(2016). Operational demands of AAC mobile technology applications on programming vocabulary and engagement during professional and child interactions. Augmentative and Alternative Communication, 32, 12–24. https://doi.org/10.3109/07434618.2015.1126636

Cascio

McGlone

Folger

Tannan

Baranek

Pelphrey

K. A.

Essick

(2008). Tactile perception in adults with autism: A multidimensional psychophysical study. Journal of Autism and Developmental Disorders, 38, 127–137.

el Kaliouby

Teeters

Picard

R. W.

(2006). An exploratory social-emotional prosthetic for autism spectrum disorders. In International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06) (pp. 2). IEEE.

Evmenova

A. S.

Graff

H. J.

Genaro Motti

Giwa-Lawal

Zheng

(2018). Designing a wearable technology intervention to support young adults with intellectual and developmental disabilities in inclusive postsecondary academic environments. Journal of Special Education Technology, 34, 92–105. https://doi.org/10.1177/0162643418795833

Fawkes

D. B.

Malow

B. A.

Weiss

S. K.

Reynolds

A. M.

Loh

Adkins

K. W.

Wofford

D. D.

Wyatt

A. D.

Goldman

S. E.

(2015). Conducting actigraphy research in children with neurodevelopmental disorders—A practical approach. Behavioral Sleep Medicine, 13, 181–196.

Finn

Ramasamy

Dukes

Scott

(2015). Using WatchMinder to increase the on-task behavior of students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45, 1408–1418.

Green

J. M.

Hughes

E. M.

Ryan

J. B.

(2011). The use of assistive technology to improve time management skills of a young adult with an intellectual disability. Journal of Special Education Technology, 26, 13–20.

10.

Keshav

N. U.

Salisbury

J. P.

Vahabzadeh

Sahin

N. T.

(2017). Social communication coaching smartglasses: Well tolerated in a diverse sample of children and adults with autism. JMIR mHealth and uHealth, 5, e140.

11.

Kooistra

Dijkman

Einhorn

T. A.

Bhandari

(2009). How to design a good case series design. The Journal of Bone and Joint Surgery, 91, 21–26.

12.

Leekam

S. R.

Nieto

Libby

S. J.

Wing

Gould

(2007). Describing the sensory abnormalities of children and adults with autism. Journal of Autism and Developmental Disorders, 37, 894–910.

13.

Lequia

Machalicek

Rispoli

(2012). Effects of activity schedules on challenging behaviors exhibited in children with autism spectrum disorders: A systematic review. Research in Autism Spectrum Disorders, 6, 480–492.

14.

Light

McNaughton

Caron

(2019). New and emerging AAC technology supports for children with complex communication needs and their communication partners: State of the science and future research directions. Augmentative and Alternative Communication, 35, 26–41. https://doi.org/10.1080/07434618.2018.1557251

15.

Madsen

el Kaliouby

Goodwin

Picard

(2008). Technology for just-in-time in-situ learning of facial affect for persons diagnosed with an autism spectrum disorder. Tenth ACM Conference on Computers and Accessibility (ASSETS), Halifax, Canada (pp. 19–26).

16.

McNaughton

Light

J. C.

(2013). The iPad and mobile technology revolution: Benefits and challenges for individuals who require augmentative and alternative communication. Augmentative and Alternative Communication, 29, 107–116.

17.

Mechling

L. C.

Gast

D. L.

Cronin

B. A.

(2006). The effects of presenting high-preference items, paired with choice, via computer video programming on task completion of students with autism. Focus on Autism and Other Developmental Disabilities, 21, 7–13.

18.

Muller

Brady

(2016). Assessing early receptive language skills in children with ASD. Perspectives of the ASHA Special Interest Groups, 1, 12–19.

19.

National Autism Center. (2015). Findings and conclusions: National standards project, phase 2. Author.

20.

O’Brien

Schlosser

R. W.

Allen

A. A.

Flynn

Costello

Shane

H. C.

(2017). Repurposing consumer products as a gateway to just-in-time communication. Seminars in Speech and Language, 38, 297–312.

21.

O’Brien

Schlosser

R. W.

Shane

H. C.

Abramson

Allen

Flynn

Dimery

(2016). Brief report: Just-in-time visual supports to children with autism via the Apple Watch®: A pilot feasibility study. Journal of Autism and Developmental Disorders, 46, 3818–3823.

22.

Odom

S. L.

Thompson

J. L.

Hedges

Boyd

B. A.

Dykstra

J. R.

Duda

M. A.

Szidon

K. L.

Smith

L. E.

Bord

(2015). Technology-aided interventions and instructions for adolescents with autism spectrum disorders. Journal of Autism and Developmental Disorders, 45, 3805–3819.

23.

Picard

Goodwin

(2008). Innovative technology: The future of personalized autism research and treatment. Autism Advocate, 1, 32–39.

24.

Remner

Baker

Karter

Kearns

Shane

(2016). Use of augmented input to improve understanding of spoken directives by children with moderate to severe autism spectrum disorder. eHearsay, 6, 4–10.

25.

Rice

M. L.

Warren

S. F.

Betz

S. K.

(2005). Language symptoms of developmental language disorders: An overview of autism, Down syndrome, fragile X, specific language impairment, and Williams syndrome. Applied Psycholinguistics, 26, 7–27

26.

Schlosser

R. W.

Koul

(2015). Speech output technologies in interventions for individuals with autism spectrum disorders: A scoping review. Augmentative and Alternative Communication, 31, 285–309.

27.

Schlosser

R. W.

Laubscher

Sorce

Koul

Flynn

Hotz

Abramson

Fadie

Shane

H. C.

(2013). Implementing directives that involve prepositions with children with autism: A comparison of spoken cues with two types of augmented input. Augmentative and Alternative Communication, 29, 132–145.

28.

Schlosser

R. W.

O’Brien

Abramson

Allen

Flynn

Shane

H. C.

(2017). Repurposing everyday technologies to provide just-in-time visual supports to children with intellectual disability and autism: A pilot feasibility study with the Apple Watch®. International Journal of Developmental Disabilities, 63, 221–227.

29.

Schlosser

R. W.

Shane

H. C.

Allen

A. A.

Abramson

Laubscher

Dimery

(2016). Just-in-time supports in augmentative and alternative communication. Journal of Physical and Developmental Disabilities, 28, 177–193.

30.

Sevcik

R. A.

(2006). Comprehension: An overlooked component in augmented language development. Disability and Rehabilitation, 28, 159–167.

31.

Shane

H. C.

Albert

P. D.

(2008). Electronic screen media for persons with autism spectrum disorders: Results of a survey. Journal of Autism and Developmental Disabilities, 38, 1499–1508.

32.

Shane

H. C.

Laubscher

Schlosser

R. W.

Fadie

H. L.

Sorce

J. F.

Abramson

J. S.

Flynn

Corley

(2015). Enhancing communication for individuals with autism: A guide to the Visual Immersion System™. Paul H Brookes Publishing.

33.

Shane

H. C.

Laubscher

Schlosser

R. W.

Flynn

Sorce

J. F.

Abramson

(2012). Applying technology to visually support language and communication in individuals with ASD. Journal of Autism and Developmental Disorders, 42, 1228–1235.

34.

Shepley

S. B.

Spriggs

A. D.

Samudre

Elliot

(2018). Increasing daily living independence using video activity schedules in middle school students with intellectual disability. Journal of Special Education Technology, 33, 71–82

35.

Shepley

S. B.

Spriggs

A. D.

Samudre

M. D.

Sartini

E. C.

(2019). Initiation and generalization of self-instructed video activity schedules for elementary students with intellectual disability. The Journal of Special Education, 53(1), 51–62.

36.

Souders

M. C.

Mason

T. B.

Valladares

Bucan

Levy

S. E.

Mandell

D. S.

Weaver

T. E.

Pinto-Martin

(2009). Sleep behaviors and sleep quality in children with autism spectrum disorders. Sleep, 32, 1566–1578.

37.

Spriggs

A. D.

Knight

Sherrow

(2015). Talking picture schedules: Embedding video models into visual activity schedules to increase independence for students with ASD. Journal of Autism and Developmental Disorders, 45, 3846–3861.

38.

Tomchek

S. D.

Dunn

(2007). Sensory processing in children with and without autism: A comparative study using the short sensory profile. American Journal of Occupational Therapy, 61, 190–200.

39.

Torrado

J. C.

Gomez

Montoro

(2017). Emotional self-regulation of individuals with autism spectrum disorders: Smartwatches for monitoring and interaction. Sensors, 17, 1359.

40.

Virnes

Kärnä

Vellonen

(2015). Review of research on children with autism spectrum disorder and the use of technology. Journal of Special Education Technology, 30, 13–27.

41.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

Brock

M. E.

Plavnick

J. B.

Fleury

V. P.

Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966.

42.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

Schultz

T. R.

(2014). Evidence-based practices for children, youth, and young adults with autism spectrum disorder. The University of North Carolina, Frank Porter Graham Child Development Institute, Autism Evidence-Based Practice Review Group.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB

0.12 MB