Adding to the Evidence Base: Effects of I-Connect in a Secondary Special Education Rural Classroom

Abstract

The purpose of this study was to examine the effects of a technology-based self-monitoring application, I-Connect, to enhance the on-task behavior of five secondary students (ages 15–16) with autism spectrum disorder, intellectual disability, and specific learning disability during Biology instruction in a rural special education classroom. We used an A-B-A-B withdrawal design with a generalization condition. The primary dependent variable was percent of intervals with on-task behavior as measured by momentary time sampling procedures. Results indicated overall higher levels of on-task behavior for all students when using I-Connect to monitor their behavior. However, there were also high percentages of overlapping data, and student satisfaction with the application was relatively low. Students used I-Connect in an employability seminar offered at their school as a means to generalize to a different setting. We provide implications for practice in rural settings and suggestions for future research related to I-Connect. We also provide recommendations for enhancing the social validity of technology-based self-monitoring for secondary students with disabilities.

Keywords

autism intellectual disability single subject design self-monitoring

Introduction

Secondary students with autism spectrum disorder (ASD), intellectual disability (ID), and other disabilities must be adequately prepared for life after high school. Longitudinal studies indicate individuals with ASD, including those with and without co-occurring ID, typically experience some of the most unsatisfactory postschool outcomes compared to individuals with other types of disabilities (Lipscomb et al., 2017; Liu et al., 2018; Roux et al., 2017). A successful transition to adulthood involves a variety of complex factors, one of which is fostering and promoting independence (Hume et al., 2014). However, secondary students with disabilities commonly receive support from school personnel (e.g., special education teachers, paraprofessionals) throughout their school day related to academic, organization, and social skills. As a result, overreliance on support from an adult is a common occurrence for secondary students with ASD and/or ID (Carter et al., 2009; Hollingshead & Barrio, 2019). Interventions and strategies focused on enhancing autonomy are of upmost importance to enhance adult outcomes for this population.

Self-management is a broad category of interventions that can promote independence in students with disabilities across different settings and behaviors (Reinecke et al., 2018). Self-management has been defined as “the personal application of behavior change tactics that produces a desired improvement in behavior” (Cooper et al., 2020, p. 683). Self-management is an established evidence-based practice for learners with ASD with a relatively large literature base supporting its use (Steinbrenner et al., 2020). Self-monitoring is perhaps the most common type of self-management strategy. Students who self-monitor their behavior are taught to distinguish between desired and non-desired behavior and record their performance (Cooper et al., 2020). Self-monitoring is commonly combined with other evidence-based practices (e.g., modeling, reinforcement, visual supports) and can be used as a type of individualized academic support in a variety of classroom settings. Importantly, secondary students with ASD, ID, and other disabilities need access to individualized academic supports that are age-appropriate and blend into their classroom environments (i.e., not stigmatizing or obtrusive).

I-Connect (Wills & Mason, 2014) is a technology-based self-monitoring application (app) that can be downloaded on portable devices such as Chromebooks, tablets, or smartphones. The I-Connect app includes visuals and audible prompts at selected intervals to support on-task behavior and engagement. When I-Connect is used in classroom settings, the teacher is typically the person who creates the I-Connect account for students and sets up the intervals, prompts, and goals. The interval length can be customizable to the student’s needs (e.g., 30 s to 45 min) and the visual prompt within the app can also be individualized such as, “Are you on task?” or “Did you ask a question?” The student then monitors their own behavior by responding to the question with a “yes” or “no.” The device can vibrate or chime according to the student’s preference when the question appears. In addition, the teacher and student can set a target goal for the student to meet (e.g., “I will be on-task 90% of my math class”). Teachers can provide reinforcement to students who accurately monitor their behavior using I-Connect and simultaneously measure the student’s behavior. For more information on I-Connect, readers are encouraged to visit https://iconnect.ku.edu/.

I-Connect to Improve On-Task Behavior in Classroom Settings

I-Connect has primarily been used in the academic literature to enhance on-task behavior of students with disabilities in various classroom settings. On-task behavior can promote academic engagement although the two are different outcome areas. Academic engagement is one of the six college and career readiness domains for secondary students with disabilities identified by Morningstar et al. (2017). On-task behavior commonly targeted by I-Connect studies can be classified as behavioral engagement and is typically measured through a behavioral approach. For example, three high school students (ages 15–17) with ASD, ID, and specific learning disability used I-Connect on tablets to improve their on-task behavior in general and self-contained special education settings (Clemons et al., 2016). The students received reinforcement if the recordings of their behavior accurately matched the recordings of researchers who were also observing their behavior. Results indicated a functional relation between improved on-task behavior and I-Connect was present for all three participants (Clemons et al., 2016). Similarly, four middle and high school students (ages 10–17) with ASD in a private day school used I-Connect on smartphones to reduce their disruptive behavior while also increasing their academic task-completion (Rosenbloom et al., 2019). A functional relational was present for these four participants, and social validity was high for both teachers and students who participated in the intervention. Two I-Connect studies have evaluated both on-task behavior and academic engagement by measuring some type of academic gain. Romans et al. (2020) evaluated the effects of I-Connect to increase academic accuracy in math and written expression in addition to on-task behavior for two high school students with ASD. Similarly, Beckman et al. (2019) measured on-task behavior and academic performance using permanent products for two upper elementary students with ASD. These studies demonstrate the interrelated nature of on-task behavior and academic engagement, that is students are more likely to achieve their academic goals when attending to learning tasks presented to them in the classroom.

Scheibel et al. (2022) conducted a meta-analysis to determine what contexts I-Connect has been used to improve the on-task behavior of K-12 students with disabilities. There were six studies identified in their search procedures with 14 students included across all studies. Students with ASD most commonly used I-Connect, but students with other disabilities such as attention deficit hyperactivity disorder (ADHD), ID, and specific learning disability were also represented in the literature. I-Connect has primarily been used to monitor on-task behavior during independent work in self-contained special education settings. However, students in general education settings have also used I-Connect, as well as other contexts such as small group instruction (Scheibel et al., 2022). Visual analysis of the included studies in the meta-analysis indicated an overall immediate and strong improvement of on-task behavior. The within-case parametric effect size calculated ranged from .17 to 2.23, which indicated changes in on-task behavior that ranged from 19%–834% (Scheibel et al., 2022). This meta-analysis and other I-Connect studies in K-12 settings (e.g., Beckman et al., 2019; Clemons et al., 2016; Romans et al., 2020; Rosenbloom et al., 2019) indicate I-Connect is likely to be effective in improving the on-task behavior of diverse students with disabilities in a variety of classroom contexts. Furthermore, there are two I-Connect studies that extend beyond K-12 education and focus on postsecondary outcomes for students with disabilities. A 30-year-old female with ASD who was competitively employed in an office setting used I-Connect to reduce her inappropriate vocalizations (Wills et al., 2019). Likewise, a 19-year-old male with ASD used I-Connect in a large, lecture-style class at a public university to increase his on-task behavior (Huffman et al., 2019). These studies focused on postsecondary outcomes demonstrate how technology-based self-monitoring such as I-Connect can be used to support the transition to adulthood for students with ASD and related disorders.

Rationale for Current Study

The current study adds to the existing I-Connect literature base to improve on-task behavior in several ways. First, the majority of participants in current I-Connect studies are males with ASD (Scheibel et al., 2022). Additional I-Connect replications for participants without ASD and female participants would be advantageous. Participants in the current study include two participants without an ASD diagnosis and one female participant. Second, I-Connect can be used on handheld technological devices with Wi-Fi access (e.g., Chromebooks, smartphones, tablets). Many schools provide such devices to students, and I-Connect can be incorporated into regular classroom routines. Less research has been conducted on the effects of I-Connect on school-issued devices with a classroom teacher serving as the intervention agent rather than a researcher. Third, although the Clemons et al. (2016) study was conducted in a rural, public high school, additional research examining technology-based interventions in rural special education classrooms is needed. Rural schools have unique needs related to integrating technology into learning activities and may face barriers related to acquisition and cost of technological devices and interventions (Sundeen & Sundeen, 2013).

Lastly, additional research is needed to examine technology-based interventions that promote independence and enhance postsecondary outcomes for secondary students with disabilities. Many public school districts in the United States, including those in rural areas, offer an occupational course of study (OCS) that focuses on academic and vocational skills to prepare students for adulthood. An OCS program is designed to meet the needs of students with disabilities who require accommodations, intensive instruction, and modifications to meet grade level standards. Academic skills commonly taught in OCS programs focus on core subjects such as language arts, math, science, social studies, and health. Developing science, technology, engineering, and mathematics (STEM) skills is particularly important for secondary students with disabilities given they have typically been underrepresented in STEM fields (Dunn et al., 2012; Lee, 2011). The teacher in the current study selected Biology instruction within the OCS program as the instructional time for students to monitor their behavior given high levels of off-task behavior observed prior to the study. Secondary students with disabilities face many barriers to accessing a STEM education (Dunn et al., 2012), and teachers must support students to overcome these barriers. Students enrolled in OCS programs in rural areas need access to individually designed interventions as they work towards their graduation requirements including a potential focus on STEM skills. I-Connect is a practical option to consider given its free to download and prior research indicates its efficacy as an individualized support during academic instruction (Clemons et al., 2016; Romans et al., 2020; Scheibel et al., 2022). Therefore, the purpose of this study was to evaluate the effects of I-Connect to increase on-task behavior of five secondary students with ASD, ID, and specific learning disability enrolled in an OCS program at a rural high school during Biology instruction. We were guided by the following research questions:

(1) To what extent does I-Connect increase the on-task behavior of secondary students with ASD, ID, and specific learning disability during Biology instruction?

(2) What is the social validity of the I-Connect app as a form of self-monitoring as reported by secondary students with ASD, ID, and specific learning disability who used the app?

Method

Participants

This study was approved by a university institutional review board prior to participant recruitment and data collection. We recruited students from a rural high school who were enrolled in the OCS program. Students met the following inclusion criteria: (a) diagnosis of ASD, ID, learning disability, or other related disorder as confirmed by the student’s individualized education program (IEP); (b) age 14 or older; (c) regularly exhibited off-task behaviors during academic instruction as confirmed by preliminary baseline observations and teacher report; and (d) willingness to use the I-Connect app to self-monitor their behavior. Five students, ages 15–16, with ASD, ID, and/or a learning disability participated in this study. The parents/guardians of all students provided signed consent, and students participated in an assent process with the first author. Parents/guardians and students also signed an acceptable use policy with expectations for the use of technology within the class setting. Alicia was a 15-year-old Hispanic female with mild ID. Caleb, Luke, and Paxton were 16-year-old White males with ASD. Lastly, Eric was a 16-year-old White male with a specific learning disability in reading comprehension and ADHD. All students were in special education settings for 64% of their school day as described on their IEPs. See Table 1 for additional participant information including current IEP goals and common off-task behavior observed.

Table 1.

Participant Demographics.

Pseudonym	Gender	Age	Disability category	Race	Current IEP goals	Common off-task behavior
Alicia	F	15	Mild ID	Hispanic	Solving one-step math problems and increasing reading comprehension skills	Engaging in conversation with other students during instruction
Caleb	M	16	ASD	White	Writing with fewer grammatical errors	Getting out of his seat during instruction and and blurting out
Eric	M	16	Learning disability and ADHD	White	Increasing reading comprehension and occupational skills	Putting his head down on his desk, falling asleep during class, and not completing assigned work tasks
Luke	M	16	ASD	White	Enhancing conversational skills and using expanded methods to solve math problems	Blurting out and not completing his assigned work tasks
Paxton	M	16	ASD	White	Increasing reading comprehension skills, completing one-step math problems, and fewer grammatical errors in writing	Getting out of his seat, gazing around the classroom, and not completing his assigned work tasks

Note. ADHD = attention deficit hyperactivity disorder; ASD = autism spectrum disorder; F = female; ID = intellectual disability; M = male.

Setting

The study was conducted in a rural high school (grades 9–12) with a population of 758 in a southeastern state of the United States. The high school was located outside the city limits of a rural city with a population of 11,352 in 2021 (U.S. Census Bureau). Twenty-seven percent of the school population was considered economically disadvantaged during the 2021-2022 academic year when the study occurred. Eighty-four percent of the school population participated in some type of career and technical education. Approximately 85% of the school population was White, 11% was Hispanic, 3% was multi-racial, less than 1% was Black, and less than 1% was Asian. All data collection sessions were conducted in the self-contained classroom used for the OCS program. The classroom was large and consisted of nine tables with two chairs at each table, a full kitchen, washer/dryer, SmartBoard, and whiteboard. The classroom also included 30 Chromebooks that were plugged in and stored on a cart. Twenty-five students were enrolled in the OCS program and received special education services focused on academic instruction, functional skills, and postsecondary job training. The typical class structure of the OCS program consisted of direct instruction, lecture, small group activities, group discussion, and written assignments. All behavioral observations for this study were conducted during the time allotted for academic instruction in the content area of Biology.

Intervention Agent

The first author served as the intervention agent who conducted all sessions throughout the study. He was a licensed special education teacher in the OCS program in which all participants were recruited. At the time of the study, the intervention agent was working towards his master’s degree in special education. He held a bachelor’s degree in special education and had taught for 17 years total. The intervention agent had prior experiences in middle and high school settings to teach both academic and functional skills. He had previously taught all participants for a minimum of five months and some participants for as long as two years.

Technology Used

The primary technology used in this study was the I-Connect app. The intervention agent and information technology (IT) personnel at the high school collaborated to download the I-Connect app on all Chromebooks that were in the OCS classroom. The HP Chromebooks were 2019 Dell versions with a 14” diagonal screen (Model 14-ca061dx). The IT personnel ensured the Chromebooks had wireless internet capabilities for I-Connect to function properly, and the school firewall protection program was installed. The intervention agent set the self-monitoring interval to 30 s for all participants. In addition, he programmed the prompt to read, “Are you on task?” for all students. When the students were using I-Connect to self-monitor, they placed the Chromebooks on their desks.

Experimental Design and Data Analysis

This study used an ABAB single-case withdrawal design (Gast et al., 2018) with a generalization condition. The withdrawal design is frequently used in applied behavioral research and entails a first baseline condition (A1) followed by implementation of the intervention (B1). Then, this process is replicated by withdrawal of the intervention for a second baseline condition (A2) followed by re-introduction of the intervention (B2) (Gast et al., 2018). There are three potential demonstrations of effect in an ABAB withdrawal design. We selected a withdrawal design because we predicted on-task behavior would be a reversible behavior; that is, we precited participants would return to their baseline levels of on-task behavior when not using I-Connect to self-monitor. We followed What Works Clearinghouse current recommendations for standards in single-case design research to Meet What Works Clearinghouse Standards Without Reservations (U. S. Department of Education, Institute of Education Sciences, 2022; University of Kansas, n.d.). Specifically, we included a minimum of two phases per condition. The initial baseline phase included a minimum of six data points, and all subsequent phases included a minimum of five data points. The intervention was introduced or withdrawn (i.e., phase change decisions) only when the data path in each prior phase was stable and minimum number of data points had been collected.

In the current study, we compared the baseline and intervention conditions for each participant to evaluate the potential effectiveness of I-Connect to improve on-task behavior. Using the process described by Barton et al. (2018), we conducted formative visual analysis by examining the level, trend, and variability/stability of adjacent phases. Specifically, we described the level of each phase by reporting the mean and range of on-task behavior. We also examined if there was an immediacy of change when the I-Connect app was implemented. We conducted summative visual analysis by comparing the level of the two baseline conditions (A1 and A2) and two intervention conditions (B1 and B2). We reported if a functional relation was observed for each participant and if confidence in the potential functional relation was high or low. Finally, we calculated percentage of non-overlapping data (PND; Scruggs & Mastropieri, 1998) as a supplementary tool of visual analysis to describe the overlap between baseline/withdrawal conditions versus intervention conditions.

Dependent Variable

The dependent variable was percent of on-task behavior during academic instruction. All behavior observations were 12 min in duration and occurred during the regular Biology lessons as part of the OCS program. On-task behavior was measured using momentary time sampling procedures. The intervention agent had an interval timer on his smartphone that buzzed at the end of the each 30-s interval. The intervention agent then marked on a paper data collection page if the student was on-task or off-task. On-task behavior was operationally defined as the participant working on the assigned Biology-related task, such as orienting body towards the teacher during lectures, reading the course textbook, writing activities, and/or typing on a Chromebook. To be scored on-task, participants also needed to remain in seat, raise their hand, and speak on topics related to the Biology lesson to other people in the classroom (e.g., teacher, paraprofessionals, classmates). Off-task behavior was operationally defined as not completing the requested Biology-related task. Examples of off-task behavior observed during baseline observations also included walking around the classroom, blurting out, and completing other tasks such as using the Chromebook to browse the internet. The paraprofessional in the OSC classrooms served the secondary data collector and completed the same procedures described above as the classroom teacher who served as the intervention agent.

Procedures

Baseline

The intervention agent had not yet downloaded I-Connect app on the Chromebooks during baseline sessions. Therefore, the participants did not have access to the I-Connect app while using their Chromebooks to complete Biology-related tasks as part of their regular classroom routine. The intervention agent measured on-task behavior using momentary time sampling procedures as described above. Students participated in the Biology lesson without self-monitoring their behavior in any way. The intervention agent and paraprofessional gave behavioral support to students as needed according to their regular classroom routines (e.g., verbal prompts, behavior specific praise, redirection).

I-Connect App Training

The intervention agent trained all participants how to monitor their behavior using I-Connect prior to the intervention condition. The training consisted of showing the participants how to log into the I-Connect app on their Chromebooks and practice self-monitoring their behavior. The intervention agent and participants discussed the difference between on-task and off-task behaviors. All participants had the opportunity to ask questions and receive feedback as needed. To prepare for the withdrawal condition, the intervention agent also explained that sometimes the participants would be using the I-Connect app and other times they would not. The training lasted approximately 30 min and occurred in a whole class format.

Intervention

During all intervention conditions, participants monitored their behavior during the Biology lesson using the I-Connect app on Chromebooks. The screen lit up and there was a chime from the Chromebook at each 30 s interval that read, “Are you on task?” Participants then selected “yes” with a green button or “no” with a red button. The intervention agent taught the Biology lesson as usual and provided no additional supports specific to self-monitoring.

Withdrawal

Withdrawal sessions were identical to baseline with the exception that I-Connect was downloaded on all Chromebooks. The participants were asked to not open the I-Connect app during the Biology lesson. If a participant asked to self-monitor, the intervention agent explained it was not available during this time. The students participated in the Biology lesson without self-monitoring their behavior.

Reintroduction of Intervention

After a minimum of five withdrawal sessions with stable responding, the intervention agent re-introduced the I-Connect app. The participants again used I-Connect on the Chromebooks to monitor their on-task behavior. All procedures in the second intervention condition were identical to the first intervention condition.

Generalization

As a means to measure generalization of self-monitoring in a novel setting beyond the OCS classroom, participants used I-Connect on the same Chromebooks at an employability seminar. The seminar was offered to all students at the school one time during the academic year. Participants used I-Connect to remain on-task while listening to the speaker share about strategies to be successful in the workforce. The intervention agent and/or paraprofessional served as the observers during the generalization condition. The duration of the behavioral observations was again 12 min. Finally, students were encouraged to continue using I-Connect in other classes upon completion of the study.

Interobserver Agreement

The intervention agent trained the paraprofessional in the OCS classroom to collect IOA data. The paraprofessional conducted IOA for a minimum of 30% of sessions across all conditions. We calculated interval-by-interval IOA using the following formula: [(Number of intervals agreed)/(Number of intervals agreed + Number of intervals disagreed) ✕100 = Percent of Agreement] (Cooper et al., 2020). The overall IOA for this study was 91% (range = 59–100%). The mean IOA for each condition across all participants was as follows: baseline 90% (range = 78–100%); first intervention condition 95% (range = 77–100%); withdrawal condition 85% (range = 59–100%); and second intervention condition 92% (range = 72–100%). Finally, IOA during the generalization condition was 96% (range = 94–100%).

Procedural Fidelity

The intervention agent and paraprofessional used a nine-step checklist to measure fidelity of implementation of the I-Connect app. The intervention agent completed the procedural fidelity checklist for 100% of the intervention sessions, and the paraprofessional for a minimum of 30% of intervention sessions. Across both intervention conditions of the study, the I-Connect app was implemented with 100% fidelity. There were no issues with Wi-Fi connectivity or app malfunction during the study. The intervention agent and paraprofessional also measured procedural fidelity during the baseline and withdrawal conditions with 100% fidelity to ensure the participants were not using the I-Connect app during these conditions.

Social Validity

All participants completed a researcher-created social validity questionnaire at the conclusion of the study. The questionnaire consisted of five Likert-style questions with anchors of strongly agree, agree, neutral, disagree, and strongly disagree. The questions focused on potential satisfaction and feasibility of using the I-Connect app. The questions asked if the participants would like to keep using a self-monitoring tool in other settings and if self-monitoring was helpful. Participants also answered an open-ended question to share their overall experience of using I-Connect as a means to improve their on-task behavior during Biology.

Results

Figure 1 through 5 provide the graphed data for each participant to demonstrate their percent of intervals with on-task behavior. Table 3 provides the mean and ranges across baseline/withdrawal conditions versus intervention conditions and PND for the study.

Figure 1.

Percent of intervals with on-task behavior for Alicia.

Figure 2.

Percent of intervals with on-task behavior for Caleb.

Figure 3.

Percent of intervals with on-task behavior for Luke.

Figure 4.

Percent of intervals with on-task behavior for Paxton.

Figure 5.

Percent of intervals with on-task behavior for Eric.

Alicia

Visual analysis of Alicia’s graphed data indicates some variability in each condition. During the baseline condition (A1), Alicia’s level of on-task behavior was 69% (range = 54–83%). There was no immediacy of changed observed from the A1 to B1 conditions. Alicia’s level of on-task behavior for the first intervention condition (B1) was 82% (range = 71–92%). There was a definite immediacy of change in the withdrawal condition from B1 to A2. Alicia’s level of on-task behavior during the withdrawal condition (A2) was 82% (range = 71–96%). There was no immediacy of change observed from the A2 to B2 conditions. Alicia’s level of of-task behavior during the return to intervention condition (B2) was 93% (range = 83–100%). Alicia’s on-task behavior during the generalization condition was relatively high at 88% and 96% respectively.

Given the lack of an immediacy of change when the intervention was introduced and large amount of overlapping data, the confidence in a function relation is low. However, summative visual analysis indicates slight improvement in Alicia’s on-task behavior when using I-Connect.

Caleb

Visual analysis of Caleb’s graphed data indicated clear changes in level for each adjacent condition. Caleb’s level of on-task behavior during the baseline condition (A1) was 40% (range = 25–58%). There was no immediacy of change from the A1 to B1 conditions. However, the level of on-task behavior during the first intervention condition (B1) was improved at 65% (range = 38–79%). There was a significant decreased in level when I-Connect was withdrawn during the B1 condition as Caleb’s mean on-task behavior was 28% (range = 13–50%). During the return to intervention condition (B2), Caleb’s mean on-task behavior was 62% (range = 42–75%). Caleb’s on-task behavior during the generalization sessions was 79% and 67% respectively. Given the clear changes in level when using I-Connect, confidence in a functional relation is high for Caleb.

Luke

Visual analysis of Luke’s graphed data indicated variability in most conditions but clear changes in level when using I-Connect. Luke’s level of on-task behavior during baseline (A1) was 60% (range = 33–83%). There was a drastic immediacy of change when I-Connect was first introduced. Luke’s level of on-task behavior during the initial intervention condition (B1) was 82% (range = 75–88%). Next, the level of on-task behavior during the withdrawal condition was 64% (range = 50–75%). Then, the intervention agent reintroduced I-Connect for five sessions. During the return to intervention (A2), Luke’s mean on-task behavior was 92% (range = 79–96%). Luke was on-task for 88% and 100% of the generalization sessions at the employability seminar. Given the clear changes in level and immediacy of changes observed, confidence in a functional relation is high for Luke.

Paxton

Paxton’s graphed data indicated clear changes in level when using I-Connect. Paxton’s level of on-task behavior during baseline (A1) was 56% (range = 25–75%). There was a drastic immediacy of change when I-Connect was first introduced, and Paxton’s level of on-task behavior during the first intervention condition (B1) was 91% (range = 79–100%). Then, the withdrawal condition occurred across five sessions, and the level was 73% (range = 50–88%) for the withdrawal condition (A2). Lastly, during the return to intervention, Paxton’s level of on-task behavior was 88% (range = 63–100%). Paxton’s generalized his on-task behavior during the employability seminar as indicated by being on-task 88% for both of the generalization sessions. Confidence in a functional relation is high for Paxton, particularly given the immediacy of changes between each adjacent condition.

Eric

Visual analysis of Eric’s graphed data indicated variability in each condition and slight changes in level between each condition. Eric’s level of on-task behavior during the initial baseline condition (A1) was 65% (range = 46–92%). Then, the implementation of I-Connect occurred for six sessions during the first intervention condition (B1) with level of on-task behavior as 90% (range = 75–100%). Next, the withdrawal condition (A2) was implemented across five sessions, and Eric’s level of on-task behavior was 78% (range = 67–100%). Finally, the intervention agent reintroduced I-Connect for five sessions. During the return to intervention condition (B2), Eric’s level of on-task behavior was 78% (range = 58–96%). Eric generalized his on-task behavior during the employability seminar as indicated by being on task 83% and 88% respectively. Confidence in a functional relation is high for Eric although the improvement in his on-task behavior is slight given the large amount of overlapping data.

Social Validity

See Table 2 for an overview of the social validity questionnaire results. All participants strongly disagreed, disagreed, or were neutral that they liked using I-Connect. However, the participants reported higher satisfaction with I-Connect helping them to stay on task and finding it easy to use. None of the participants reported a desire to continue using a self-monitoring tool although the majority agreed self-monitoring helped them in their classes. The open-ended responses were similar to the Likert-style questionnaire results in that participants reported they liked how I-Connect helped them focus but disliked other aspects of the app (e.g., noise, hitting the button, duration of intervals). For example, Alicia provided the following open-ended response: “What I liked about it was that when I was not paying attention it stayed me focused. What I didn’t like about it was that it got annoying with the sound.”

Table 2.

Participant Social Validity Questionnaire Results.

Questionnaire Statements	Alicia	Caleb	Luke	Paxton	Eric	Statement mean
1. I like using I-Connect.	2	1	3	2	2	2
2. I-Connect helped me stay on task.	4	5	5	4	2	4
3. I-Connect was easy to use.	4	2	5	4	4	3.8
4. I would like to continue using a self-monitoring tool.	2	2	3	2	1	2
5. I think self-monitoring helped me in my classes.	4	4	4	3	2	3.4

Open-Ended Responses
Alicia: “What I liked about it was that when I was not paying attention it stayed me focused. What I didn’t like about it was that it got annoying with the sound.”
Caleb: “The one I like is helping me stay on task, and one I dislike was using a self-monitoring tool.”
Luke: “I like how it helped me. I dislike how I couldn’t change the noise it made.”
Paxton: “I like how I-Connect helps you focus. I don’t like how I-Connect you hit that button say yes or no.”
Eric: “I dislike how it’s every 30 seconds.”

Note. 1 = Strongly disagree; 2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly agree.

Discussion

The purpose of this study was to examine the effects of I-Connect on the on-task behavior of five high school students with ASD, ID, or learning disability in a rural special education classroom. The participants used I-Connect during Biology instruction within their OSC program on their school-issued Chromebooks. All five participants increased their overall percentage of on-task behavior across the study when using I-Connect (see Table 3). However, the social validity and satisfaction of using I-Connect was relatively low for all participants. In addition, there was considerable overlap for all participants between baseline/withdrawal conditions (A1 and A2) and intervention conditions (B1 and B2). The PND calculations served as a supplementary tool of visual analysis (Barton et al., 2018) to demonstrate the large amount of overlapping data between baseline/withdrawal and intervention conditions. These findings have implications for research and practice related to technology-based self-monitoring such as I-Connect in rural settings.

Table 3.

Mean and Ranges Across Conditions and Percentage of Non-overlapping Data.

Participant	Mean and range of baseline and withdrawal conditions (A1 and A2)	Mean and range of intervention conditions (B1 and B2)	PND
Alicia	75% (range = 54–96%)	87% (range = 71–100%)	2/23 = 9
Caleb	34% (range = 13–58%)	64% (range = 38–79%)	8/24 = 33
Luke	62% (range = 33–83%)	86% (range = 75–96%)	8/24 = 33
Paxton	63% (range = 25–88%)	90% (range = 63–90%)	5/23 = 22
Eric	71% (range = 46–100%)	84% (range = 58–100%)	2/23 = 9

PND = percentage of non-overlapping data.

The results of this study extend the I-Connect literature base in several ways. First, two participants had different disability diagnoses beyond ASD. The majority of I-Connect studies have included at least one participant with ASD (e.g., Clemons et al., 2016; Romans et al., 2020; Rosenbloom et al., 2019), but the app is not specific to learners with ASD. Rather, I-Connect can be used by for any learner who selects a specific goal and reports a desire to monitor their behavior to meet that goal. Second, all participants were receiving special education services at a rural high school in an OCS program. I-Connect studies have occurred in elementary, middle, and high school settings but not previously an OCS program focused on postschool outcomes. Clemons et al. (2016) also conducted an I-Connect study in a rural, public high school and found similar findings. Third, participants monitored their behavior during Biology instruction which reflects the importance of STEM skills for secondary students with disabilities.

Participants in two other I-Connect studies (Beckman et al., 2019; Romans et al., 2020) monitored their behavior during math instruction. One participant in the Clemons et al. (2016) study used I-Connect during his general education Ecology class, which is similar to Biology in the current study. Accordingly, findings from this study add to the I-Connect literature base to enhance on-task behavior during some type of STEM instruction.

Although results indicated increased levels of on-task behavior for all participants when using I-Connect, the satisfaction with the app was relatively low. Participants reported that although I-Connect helped them stay on-task, they did not desire to continue self-monitoring after the conclusion of the study. Some participants reported they disliked the sound reminders that were part of the prompts provided through the app. The duration of the intervals was also a concern to some participants who reported 30 s was too quick to self-monitor. Given the increased importance of social validity in the field of special education, these relatively low social validity results were concerning. As described by Ledford et al. (2023), researchers must evaluate socially significant dependent variables and natural intervention agents should be used whenever possible. This study focused on improving on-task behavior during Biology instruction but no additional dependent variable measurements were taken (e.g., permanent products on Biology assessments). Although a natural intervention agent was used, social validity could have been enhanced by incorporating more participant choices and preferences (Ledford et al., 2016). For example, the I-Connect app allows for different types of notifications and interval durations. Perhaps participants would have reported higher satisfaction with the app if they had been included more in the decision-making process of the intervention procedures. Several articles and guidelines have been published related to involving autistic participants in research in a meaningful way (e.g., Fletcher-Watson et al., 2019; Gowen et al., 2019). It is important to note the current study was not participatory research; however, incorporating participants’ preferences into study procedures in applied research is advantageous to potentially enhance the social validity of the intervention.

Lastly, this study demonstrates the importance of technological access for secondary students with disabilities in rural settings. The U.S. Department of Education reported that 94% of public schools in the United States provided digital devices, such as laptops or tablets, to students during the 2022-2023 school year. The Chromebooks used in this study were stored within the OCS classroom, and students accessed them throughout their school day for a variety of learning activities. The I-Connect app was downloaded on the Chromebooks and embedded into regular classroom procedures. However, the classroom teacher did need assistance from IT personnel to ensure the I-Connect app was functioning properly throughout the study. Factors impacting special education teachers’ use of technology include both access to technology and self-efficacy with using the technology (Siyam, 2019). Mere access to technology is not sufficient; special education teachers must perceive the technology to be feasible and easy to use with students (Siyam, 2019). Rural special education teachers such as the teacher in this study typically have unique technology-related needs compared to teachers in urban settings (Almeida et al., 2016; Sundeen & Sundeen, 2013). Furthermore, students in rural areas experience less access to the internet (National Center for Education Statistics, 2023). Collaboration between the IT personnel and teacher in this study demonstrates how rural special education teachers may seek additional technology-related support to best meet the needs of their students. This study also demonstrates the key role of school-issued handheld technological devices for many students in the United States who may not have access to devices at home.

Limitations and Suggestions for Future Research

One primary limitation of this study was a lack of racial and ethnic diversity given four of the five participants were 16-year-old White males. Alicia was the only participant who was different from this demographic as she was a 15-year-old Hispanic female. The lack of diversity of participants was primarily due to the demographics of the school and larger geographic area where the study occurred. In addition, the participants had similar cognitive abilities. A more diverse group of participants on a variety of characteristics (e.g., race/ethnicity, gender, intellectual functioning) would have extended these findings. Future researchers should consider more rigorous recruitment methods across different schools and geographical areas. A second limitation was the potential for a reactivity effect (Cooper et al., 2020) because participants knew they were being observed throughout the study. The classroom teacher served the dual role of intervention agent and primary data collector, and the paraprofessional served as the secondary data collector. All participants had existing rapport with the classroom teacher and paraprofessional which may have altered their on-task behavior in either therapeutic or non-therapeutic trends. Future researchers may consider data collection methods other than in-vivo behavioral observations by known observers (e.g., video recordings, naïve or unknown observers). A third limitation was the use of a one time, researcher-created social validity questionnaire at the conclusion of the study. Although researcher-created social validity measures are quite common, future researchers should consider objective social validity measures such as normative comparisons and evidence of maintained behavior change over time (Ledford et al., 2016). Published social validity measures such as the Intervention Rating Profile-15 (Witt & Elliott, 1985) should also be considered. Measuring social validity before, during, and after intervention implementation may reveal more nuanced information from participants.

Implications for Practice

Results of this study have implications for rural special education teachers who desire to implement technology-based self-monitoring. This study contributes to the I-Connect literature base and general effectiveness of the app to improve on-task behavior of students with disabilities. Accordingly, I-Connect is a viable option for rural special education teachers to support students who display high levels of off-task behavior during academic instruction. However, given the relatively low social validity reported by participants, teachers may consider implementing more student choice and preference related to how the students engage with the app, as well as providing reinforcement to students who accurately self-monitor their behavior. Other I-Connect studies have used reinforcement with promising findings. For example, students in the Clemons et al. (2016) study received reinforcement if they accurately monitored their behavior for a minimum of 80% of observed intervals and were also on-task for a minimum of 80% of the intervals. Teachers could provide reinforcement to students for using I-Connect to increase motivation and make the self-monitoring experience rewarding. A more systematic process for incorporating choices of students could entail asking their preferred duration of monitoring intervals and mode of notification. Teachers can also encourage students to try different settings to determine which they prefer. I-Connect can be used on different devices beyond Chromebooks, such as iPads and smartphones. If students are successful in using I-Connect in classroom settings on school-issued devices, teachers may support them in downloading and using the app on their personal devices (if available). Finally, the participants in the study generalized their on-task behavior to the employability seminar. Teachers can support students to self-monitor their behavior in postschool settings such as the community and job sites as a component of transition planning.

Conclusion

Primary findings from this study were consistent with the I-Connect literature base; that is, I-Connect was effective to improve on-task behavior of secondary students with diverse disabilities during academic instruction. However, social validity of the I-Connect app was relatively lower compared to other published I-Connect studies. Increased emphasis on social validity on any type of intervention in applied research is critical. Secondary students with disabilities participating in technology-based interventions such as I-Connect are likely to have preferences related to how they want to engage with the app, device, and overall user experience.

Both teachers and researchers should incorporate these important preferences into implementation procedures for technology-based self-monitoring interventions.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Human Subjects Statement

This study received IRB approval from UNC Charlotte prior to data collection and participant recruitment.

ORCID iD

Leslie Ann Bross

Author Biographies

Jason L. Kester, MEd, NBCT has 20 years of experience as a licensed special education teacher in middle and high school settings. He specializes in preparing students with a variety of disabilities for the transition to adulthood by teaching occupational and everyday life skills. He earned his master's degree in special education from UNC Charlotte in 2022.

Leslie Ann Bross, PhD, BCBA-D is an assistant professor of special education at UNC Charlotte and director of the Graduate Certificate Program in ASD. Her research focuses on improving postschool outcomes for youth with ASD related to competitive employment, postsecondary education, and community integration.

References

Almeida

C. M.

Jameson

J. M.

Riesen

McDonnell

(2016). Urban and rural preservice special education teachers' computer use and perceptions of self-efficacy. Rural Special Education Quarterly, 35(3), 12–19. https://doi.org/10.1177/875687051603500303

Barton

E. E.

Lloyd

B. P.

Spriggs

A. D.

Gast

D. L.

(2018). Visual analysis of graphic data. In Single case research methodology: Applications in special education and behavioral sciences (3rd ed., pp. 179–214). Routledge.

Beckman

Mason

B. A.

Wills

H. P.

Garrison-Kane

Huffman

(2019). Improving behavioral and academic outcomes for students with autism spectrum disorder: Testing an app-based self-monitoring intervention. Education & Treatment of Children, 42(2), 225–244. https://www-jstor-org-443.web.bisu.edu.cn/stable/26623055

Carter

E. W.

O’Rourke

Sisco

Pelsue

(2009). Knowledge, responsibilities, and training needs of paraprofessionals in elementary and secondary schools. Remedial and Special Education, 30(6), 344–359. https://doi.org/10.1177/0741932508324399

Clemons

L. L.

Mason

B. A.

Garrison-Kane

L. G.

Wills

H. P.

(2016). Self-monitoring for high school students with disabilities: A cross-categorical investigation of I-connect. Journal of Positive Behavior Interventions, 18(3), 145–155. https://doi.org/10.1177/1098300715596134

Cooper

J. O.

Heron

T. E.

Heward

W. L.

(2020). Applied behavior analysis. Pearson High.

Dunn

Rabren

K. S.

Taylor

S. L.

Dotson

C. K.

(2012). Assisting students with high-incidence disabilities to pursue careers in science, technology, engineering, and mathematics. Intervention in School and Clinic, 48(1), 47–54. https://doi.org/10.1177/1053451212443151

Fletcher-Watson

Adams

Brook

Charman

Crane

Cusack

Leekam

Milton

Parr

J. R.

Pellicano

(2019). Making the future together: Shaping autism research through meaningful participation. Autism, 23(4), 943–953. https://doi.org/10.1177/1362361318786721

Gast

D. L.

Ledford

J. R.

Severini

K. E.

(2018). Withdrawal and reversal designs. In Single case research methodology: Applications in special education and behavioral sciences (3rd ed., pp. 215–238). Routledge.

10.

Gowen

Taylor

Bleazard

Greenstein

Baimbridge

Poole

(2019). Guidelines for conducting research studies with the autism community. Autism Policy & Practice, 2(1), 29–45. 32226635.

11.

Hollingshead

Barrio

(2019). Working with paraprofessionals. In Shurr

Jimenez

Bouck

(Eds.), Educating students with intellectual disability and autism spectrum disorder: Book 2, Foundational concepts for leadership and collaboration (pp. 47–72). Council for Exceptional Children. https://exceptionalchildren.org/store/books/educating-students-intellectual-disability-and-autism-spectrum-disorder-book-2

12.

Huffman

J. M.

Bross

L. A.

Watson

E. K.

Wills

H. P.

Mason

R. A.

(2019). Preliminary investigation of a self-monitoring application for a postsecondary student with autism. Advances in Neurodevelopmental Disorders, 3, 423–433. https://doi.org/10.1007/s41252-019-00124-y

13.

Hume

Boyd

B. A.

Hamm

J. V.

Kucharczyk

(2014). Supporting independence in adolescents on the autism spectrum. Remedial and Special Education, 35(2), 102–113. https://doi.org/10.1177/0741932513514617

14.

Ledford

J. R.

Hall

Conder

Lane

J. D.

(2016). Research for young children with autism spectrum disorders: Evidence of social and ecological validity. Topics in Early Childhood Special Education, 35(4), 223–233. https://doi.org/10.1177/0271121415585956

15.

Ledford

J. R.

Lambert

J. M.

Pustejovsky

J. E.

Zimmerman

K. N.

Hollins

Barton

E. E.

(2023). Single-case-design research in special education: Next-generation guidelines and considerations. Exceptional Children, 89(4), 379–396. https://doi.org/10.1177/00144029221137656

16.

Lee

(2011) A comparison of postsecondary science, technology, engineering, and mathematics (STEM) enrollment for students with and without disabilities (pp. 72–82). Career Development for Exceptional Individuals.

17.

Lipscomb

Haimson

Liu

A. Y.

Burghardt

Johnson

D. R.

Thurlow

M. L.

(2017). Preparing for life after high school: The characteristics and experiences of youth in special education. Findings from the National Longitudinal Transition Study 2012. Vol. 2: Comparisons across disability groups: Full report (NCEE 2017-4018). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.

18.

Liu

A. Y.

Lacoe

Lipscomb

Haimson

Johnson

D. R.

Thurlow

M. L.

(2018). Preparing for life after high school: The characteristics and experiences of youth in special education. Findings from the National Longitudinal Transition Study 2012. Vol. 3: Comparisons over time: Full report (NCEE 2018 – 4007). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.

19.

Morningstar

M. E.

Lombardi

Fowler

C. H.

Test

D. W.

(2017). A college and career readiness framework for secondary students with disabilities. Career Development and Transition for Exceptional Individuals, 40(2), 79–91. https://doi.org/10.1177/2165143415589926

20.

National Center for Education Statistics (2023). Rural students’ access to the Internet. Condition of education. U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/programs/coe/indicator/lfc/internet-access-students-rural?tid=1000

21.

Reinecke

D. R.

Krokowski

Newman

(2018). Self-management for building independence: Research and future directions. International Journal of Educational Research, 87, 119–126. https://doi.org/10.1016/j.ijer.2016.08.006

22.

Romans

S. K.

Wills

H. P.

Huffman

J. M.

Garrison-Kane

(2020). The effect of web-based self-monitoring to increase on-task behavior and academic accuracy of high school students with autism. Preventing School Failure: Alternative Education for Children and Youth, 64(3), 249–260. https://doi.org/10.1080/1045988X.2020.1732282

23.

Rosenbloom

Wills

H. P.

Mason

Huffman

J. M.

Mason

B. A.

(2019). The effects of a technology-based self-monitoring intervention on on-task, disruptive, and task-completion behaviors for adolescents with autism. Journal of Autism and Developmental Disorders, 49(12), 5047–5062. https://doi.org/10.1007/s10803-019-04209-4

24.

Roux

A. M.

Rast

J. E.

Anderson

K. A.

Shattuck

P. T.

(2017). National autism indicators report: Developmental disability services and outcomes in adulthood. Life course outcomes program, A. J. Drexel Autism Institute, Drexel University.

25.

Scheibel

Zimmerman

K. N.

Wills

H. P.

(2022). Increasing on-task behavior using technology-based self-monitoring: A meta-analysis of I-connect. Journal of Special Education Technology, 38(2), 146–160. https://doi.org/10.1177/01626434221085554

26.

Scruggs

T. E.

Mastropieri

M. A.

(1998). Summarizing single-subject research: Issues and applications. Behavior Modification, 22(3), 221–242. https://doi.org/10.1177/01454455980223001

27.

Siyam

(2019). Factors impacting special education teachers’ acceptance and actual use of technology. Education and Information Technologies, 24, 2035–2057. https://doi.org/10.1007/s10639-018-09859-y

28.

Steinbrenner

J. R.

Hume

Odom

S. L.

Morin

K. L.

Nowell

S. W.

Tomaszewski

Szendrey

McIntyre

N. S.

Yücesoy-Özcan

Savage

M. N.

(2020). Evidence based practices for children, youth, and young adults with autism. National Clearinghouse on autism evidence and practice review team. UNC Frank Porter Graham Child Development Institute.

29.

Sundeen

T. H.

Sundeen

D. M.

(2013). Instructional technology for rural schools: Access and acquisition. Rural Special Education Quarterly, 32(2), 8–14. https://doi.org/10.1177/875687051303200203

30.

University of Kansas . (n.d.) Juniper gardens children’s project. I-connect. University of Kansas. https://iconnect.ku.edu/

31.

US Census Bureau . Quick Facts. https://www.census.gov/quickfacts/fact/table/lincolntoncitynorthcarolina/BZA110220

32.

U. S. Department of Education, Institute of Education Sciences . (2022). What Works Clearinghouse procedures and standards handbook. U. S. Department of Education, Institute of Education Sciences. https://ies.ed.gov/ncee/wwc/handbooks

33.

Wills

H. P.

Mason

B. A.

(2014). Implementation of a self-monitoring application to improve on-task behavior: A high school pilot study. Journal of Behavioral Education, 23, 421–434. https://doi.org/10.1007/s10864-014-9204-x

34.

Wills

H. P.

Mason

Huffman

J. M.

Heitzman-Powell

(2019). Implementing self-monitoring to reduce inappropriate vocalizations of an adult with autism in the workplace. Research in Autism Spectrum Disorders, 58, 9–18. https://doi.org/10.1016/j.rasd.2018.11.007

35.

Witt

J. C.

Elliott

S. N.

(1985). Acceptability of classroom intervention strategies. In Kratochwill

T. R.

(Ed.), Advances in School Psychology (Vol. 4, pp. 251–288). Erlbaum.