The Experiences of Parents of and Students With Visual Impairments With 4to24,a Transition Application

Abstract

Introduction: This article describes the field test of 4to24, a transition application (app) for parents of and students with visual impairments. In this stage of the development process, researchers evaluated the app's usability and gained feedback on app use and experience from end users. Methods: Participants, including 27 parents of and 19 students with visual impairments, used 4to24 for 6 months. They completed a baseline survey, a survey about the account setup process, and three additional surveys at 2-month intervals (i.e., 2, 4, and 6 months). Results: Participants found the app account setup process quick and easy to complete. Parents and students used the app more frequently during the first 2 months, and then app usage decreased over time. Participants’ average ratings for app enjoyability, ease of use, understandability of information, and relevance of information were generally high and consistent across time points. Participants found 4to24 useful to track activities to help students prepare for college and work, and most planned to continue using it after the study. Discussion: Findings indicate that the app was usable, enjoyable, easy to use, and helpful. The decline in app usage over the 6 months may signify that participants lost interest, were impacted by the COVID-19 pandemic, or spent more time working on activities within the modules. Students rated app enjoyability and relevance higher than parents, which may relate to the language and tone of the student-focused modules. Introductory topics may have been unnecessary for some parents, which could have contributed to their lower relevance ratings. Implications for Practitioners: Service providers can inform parents and students about 4to24 and its potential benefits. 4to24 can encourage conversations about college and work, provide resources and activity suggestions, and serve as a resource for parents who are not connected or knowledgeable about services.

Keywords

visual impairment transition employment

Employment rates for people with visual impairments in the United States continue to lag behind the general population, despite positive gains in recent years (McDonnall & Sui, 2019). In 2019, the employment rate for working-age adults with visual impairments was 46.2% compared to 78.6% for the general population (U.S. Census Bureau, 2020). Youths with visual impairments also had lower employment rates than youths without disabilities during and after high school (McDonnall, 2010b; Sanford et al., 2011). Researchers have investigated predictors of employment for people with visual impairments, and four systematic literature reviews provide a synthesis of three decades of such research (Goertz et al., 2010; Lund & Cmar, 2019a, 2019b, 2020). The strongest employment predictors for adults and youths in this body of research were education level and previous work experience.

Preparation for postsecondary education and employment requires numerous concepts, skills, and experiences, which children begin developing at a young age. Studies focusing on postschool outcomes for youths with visual impairments have documented associations between employment and other factors, including expanded core curriculum (ECC) skills and parental support (Cmar, 2015; McDonnall, 2010a, 2011; McDonnall & Crudden, 2009; Wolffe & Kelly, 2011; Zhou et al., 2013). In a qualitative study, service providers identified parental involvement as a key factor in transitioning to employment for youths with visual impairments (Crudden, 2012).

To facilitate their transition from school to work, many students with disabilities receive individualized services authorized by the Individuals with Disabilities Education Act (2004) and the Rehabilitation Act of 1973, as amended by the Workforce Innovation and Opportunity Act (2016); however, federally mandated transition services do not usually begin until students reach high school. Children with congenital visual impairments need systematic instruction in all ECC areas to develop many of the concepts and skills that children without visual impairments learn through incidental visual observation (Hatlen, 1996; Sapp & Hatlen, 2010). The provision of ECC instruction to children with visual impairments starting in early childhood can promote the development of the prerequisite skills needed to navigate the transition process in the future (Allman & Lewis, 2014; Nagle, 2001; Wolffe, 2007; Zabelski, 2007). Parents have a critical role in facilitating transition activities for their child (Zabelski, 2007) and may benefit from relevant, user-friendly information supporting their child's independence and preparation for future employment.

Mobile telephones have become ubiquitous, with high rates of mobile telephone usage documented among people with and without visual impairments (Griffin-Shirley et al., 2017; Locke et al., 2020; Perrin, 2021). People use mobile applications (i.e., apps) for various reasons, including educational and informational purposes (DeForte et al., 2020; Kim & Xie, 2017; Madrigal-Cadavid et al., 2020; Virani et al., 2019). Providing information to students and parents via a mobile app can have several advantages over books, checklists, and other static resources, including (a) customization and user engagement (Kim et al., 2016), (b) portability (Kortum & Sorber, 2015), and (c) integration of active reminders (McDonald et al., 2011).

Two important considerations for mobile app development are usability and end-user feedback. Usability refers to “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (International Organisation for Standardisation, 2018, Introduction section). The benefits of evaluating usability at multiple points in the app development process have been documented in the literature (Beatty et al., 2018; Schnall et al., 2016). Research has also indicated that obtaining end-user feedback at various points in the development process can be instrumental in creating a usable, useful, and relevant app (Schnall et al., 2016).

We developed the 4to24 app through a project funded by a grant from the National Institute on Disability, Independent Living, and Rehabilitation Research. The 4to24 app is a resource for parents of and students with visual impairments that provides timely and relevant information to support the student's future success. It spans the ages of 4–24 years to empower families of students with visual impairments to start building skills and experiences early to facilitate their child's future transition to employment. Students can create an app account beginning at age 16. 4to24 sends informational modules to users over time based on the student's age, grade level, and skill level in different topic areas, such as technology, social skills, travel skills, independent living, and postsecondary education. The app encourages user engagement by sending push-notifications when new modules become available. Each module includes background information, links to topic-specific resources, and multiple suggested activities, which often involve collaborating with educators and other service providers. Modules cover the student's educational years and into their early career to support the transition from education to work.

Our team used an iterative, multistage process to develop the app over 5 years, beginning in late 2015. See Antonelli et al. (2021) for an overview of the development process and a timeline depicting the various stages. Early stages included (a) obtaining input from stakeholders and (b) developing and validating app content (Antonelli et al., 2021). Information about technical development and initial usability testing with parents and students is available in Antonelli et al. (in press). The final stage of the development process involved conducting a 6-month field test with parents and students. The purpose of this study was to describe parents’ and students’ perceptions of and experiences with the app during the field test. This study had two primary aims:
Evaluate the usability of the final app design with end users.

Obtain feedback from end users on app use and experience.

Method

Inclusion Criteria

For inclusion in the field test, participants needed to have internet service that they accessed at least weekly and have experience using smartphone or computer apps. Student participants had to be ages 16–24 years and have a visual impairment for which they were eligible to receive special education services or accommodations. Parents had to have a 4- to 24-year-old child who was eligible to receive special education, early intervention, or accommodations for visual impairment and had no severe learning or cognitive disability. Figure 1 provides more information about the flow of participants through the screening process.

Figure 1.
Flowchart of Participants.

Procedure

The study was reviewed and approved by the institutional review board at Mississippi State University. Beginning in September 2019, participants were recruited nationwide through an online participant registry, social media, educational institutions, partner organizations, consumer organizations, and listservs. Interested participants completed a prescreening survey to determine eligibility. They also reported the type of device they would use to access the app and rated their proficiency with mobile devices, apps, and computers. Eligible adults provided informed consent, then completed a baseline survey. Parents of eligible minors provided parental permission, then minors provided assent and completed the baseline survey. Participants reported their demographic information in the baseline survey, and parents provided relevant background information about their child with a visual impairment, including age and grade level.

Participants who completed the baseline survey received an invitation link by email to download the app. They had the option of using the app on an iOS device, Android device, or computer via the 4to24 website. After receiving the invitation link, participants downloaded the app, installed it onto their devices, created an account, and responded to a short survey about the account setup process. Participants then had the opportunity to use the app on their own for 6 months between January 2020 and August 2020. During that time, they completed three surveys at 2-month intervals (i.e., 2, 4, and 6 months after installing the app) to assess their usage and perceptions of the app. Participants completed all consent forms and surveys electronically through an accessible web-based survey platform. They received a $35 electronic gift card after completing the 6-month survey.

Participants

As shown in Figure 1, we excluded participants from our sample if they only completed the baseline survey or account setup survey or never used the app, resulting in 46 participants: 27 parents and 19 students. Most were female (n = 34, 73.9%) and from the South (n = 16, 34.8%). However, 32.6% (n = 15) were from the Midwest, 23.9% (n = 11) from the West, and 8.7% (n = 4) from the Northeast. Participants represented 23 states overall. Most parent participants were between 25 and 44 years of age (n = 18, 66.7%) and had some postsecondary education (n = 24, 88.9%) ranging from some college to a graduate degree. Their children ranged in age from 4 to 20 years (M = 11.15, SD = 3.94), most were in preschool to 11th grade, and none were deaf or hard of hearing. Student participants’ ages were between 16 and 22 years (M = 19.32, SD = 1.70). Most student participants were college freshmen, sophomores, or juniors (n = 11, 57.9%); six were high school juniors or seniors (31.6%); and two (10.5%) had some college credits but were not currently in school. No student participants were deaf or hard of hearing. Table 1 presents the devices participants used and their perceived proficiency in using mobile devices, apps, and computers.

Table 1.
Devices Used and Perceived Proficiency Levels.

Variable Parents (n = 27) Students (n = 19)

n % n %

Device

iPhone 21 77.8 14 73.7

Android phone 5 18.5 5 26.3

Computer 1 3.7 0 0.0

Proficiency level

Competent 10 37.0 2 10.5

Advanced 13 48.1 6 31.6

Expert 3 11.1 10 52.6

Not reported 1 3.7 1 5.3

Measures

Usability

In the account setup survey, participants rated how easy it was to set up their app account on a 1 to 10 scale (1 = very difficult, 10 = very easy). They also reported how long it took them (in minutes) to complete the account setup process. Participants completed the System Usability Scale (SUS) twice during the study: after setting up their account and after using the app for 6 months. The SUS is a brief quantitative measure of perceived usability (Brooke, 1996) wherein participants rated their level of agreement for 10 items (e.g., “I felt very confident using the system”) on a 5-point Likert-type scale (1 = strongly disagree, 5 = strongly agree). Previous research conducted with various types of products supports the validity and reliability of the SUS (Bangor et al., 2008; Sauro, 2011). We generated total SUS scores for each participant using calculations provided by Brooke (1996). Possible values for total scores range from 0 to 100, with higher scores indicating more positive responses. The average SUS score for more than 5,000 participants in hundreds of studies was 68 (Sauro & Lewis, 2012).

App Use and Experience

In the 2-month, 4-month, and 6-month surveys, participants provided information about their app usage over the previous 2 months: (a) how often they used the app, (b) how many modules they read, and (c) how many times they completed at least one activity suggested in a module. They rated the following aspects of the app on a 1 to 10 scale at each of the three time points: (a) enjoyability (1 = not very enjoyable, 10 = very enjoyable), (b) ease of use (1 = very difficult to use, 10 = very easy to use), (c) understandability of the information (1 = very hard to understand, 10 = very easy to understand), and (d) relevance of the information (1 = not very relevant, 10 = very relevant). In each survey, participants indicated whether they received any information through the app in the past 2 months that was particularly helpful or unhelpful. Participants who reported receiving helpful or unhelpful information had the opportunity to explain their responses via open-ended text entry fields. In the 6-month survey, participants rated the app's usefulness for keeping track of activities they (or their child) can do to prepare for work or college on a 1 to 10 scale (1 = not very useful, 10 = very useful). They also indicated whether they planned to use the app after it became available.

Data Analysis

We utilized SAS 9.4 to generate descriptive statistics for each variable for the whole sample and by group (i.e., parents and students). To evaluate usability, we computed means and standard deviations for the account setup variables and SUS scores. We calculated frequencies to investigate how often participants used the app during the 2-month, 4-month, and 6-month surveys. We reported medians and interquartile ranges for the number of modules read because this variable was positively skewed. To examine participants’ perceptions of the app, we generated means and standard deviations for the enjoyability, ease of use, understandability, and relevance ratings for each time point and presented them graphically. We used means and standard deviations to describe the app's usefulness for college and employment preparation and generated frequencies for planned app use. Three researchers independently coded open-ended responses to the queries about helpful and unhelpful information and then compared codes. Any discrepancies were discussed and resolved. We summated the codes across time points and reported the raw numbers and representative quotes.

Results

Usability

Overall, participants thought the app account setup process was easy to complete (M = 9.20, SD = 1.14, range 5–10). Students rated the setup process slightly more favorably (M = 9.60, SD = 0.63, range 8–10) than parents (M = 8.96, SD = 1.31, range 5–10). Participants completed the setup process in 3–30 min (M = 8.05, SD = 5.04). Parents took 3–20 min to complete the process (M = 8.28, SD = 3.60), whereas students took 3–30 min (M = 7.67, SD = 6.95). Table 2 presents SUS scores overall and by a group for the account setup and 6-month surveys.

Table 2.
System Usability Scale Scores.

Group Account setup survey 6-month survey

n M SD Range n M SD Range

Overall 40 84.75 13.50 50–100 43 75.87 18.47 40–100

Parents 25 82.90 13.88 50–100 24 74.90 18.92 40–100

Students 15 87.83 12.71 40–100 19 77.11 18.32 45–100

App Use and Experience

Table 3 shows participants’ app use and modules read over time. In the 2-month survey, most participants reported using the app more than once a month. A downward trend in app use was evident in the 4- and 6- month surveys, particularly among parents. Across time points, the median number of modules read ranged from 3 to 5 for parents and from 5 to 7 for students. On average, participants completed at least one suggested activity for 62.6%, 62.2%, and 63.9% of the modules they read at 2, 4, and 6 months, respectively.

Table 3.
App Use and Modules Read.

Group App use^a Modules read

More than once a month Once a month or less

n % n % Mdn IQR

Overall

2-month survey 20 83.3 4 16.7 5 4–7

4-month survey 18 62.1 11 37.9 4 2–6

6-month survey 18 41.9 25 58.1 4 1–7

Parents

2-month survey 10 83.3 2 16.7 5 4–6

4-month survey 8 50.0 8 50.0 3 1–4

6-month survey 8 33.3 16 66.7 4 1–6

Students

2-month survey 10 83.3 2 16.7 7 5–21

4-month survey 10 76.9 3 23.1 6 4–15

6-month survey 10 52.6 9 47.4 5 0–10

Note. Mdn = median; IQR = interquartile range.

a
One parent did not answer this question in the 2-month survey.

Figures 2–5 provide participants’ average enjoyability, ease of use, understandability of information, and relevance of information ratings after 2, 4, and 6 months of app use. For these variables, average ratings for all participants ranged from 6.64 to 8.49 across time points. Average ratings ranged from 5.38 to 8.42 for parents and from 7.85 to 9.08 for students.

Figure 2.
Average Enjoyability Ratings after 2, 4, and 6 Months of App Use. Note. Error Bars Represent Standard Deviations.

Figure 3.
Average Ease of Use Ratings after 2, 4, and 6 Months of App Use. Note. Error Bars Represent Standard Deviations.

Figure 4.
Average Understandability Ratings after 2, 4, and 6 Months of App Use. Note. Error Bars Represent Standard Deviations.

Figure 5.
Average Relevance Ratings after 2, 4, and 6 Months of App Use. Note. Error Bars Represent Standard Deviations.

Some participants described information they received through the app that was particularly helpful or unhelpful. Across time points, 12 parents (44.4%) and 10 students (52.6%) commented on helpful information. Five participants, all students, found information related to income taxes helpful. For example, one student said,
I had gotten a push notification for the module on filing taxes, coincidentally shortly after I applied for a job. The information in that module was really helpful, and potentially more relevant than ever for me at this point.

Another topic four participants found helpful was independent living. One parent mentioned,
I have a better understanding of how to help my daughter become more independent as she ages. She can now order and pay for her own meals in a restaurant and is so happy about her being able to be independent when she goes to a restaurant with her friends from her youth group.

Four participants found information about jobs and vocational rehabilitation (VR) helpful. One student commented, “Learning about VR, in particular, was very helpful as I continue my college career and find employment.” Three participants reported that information about self-determination was helpful. As one parent mentioned, “Self-determination and goal setting. It was good for him to see another resource encouraging this.”

Across time points, eight parents (29.6%) and one student (5.3%) commented on unhelpful information they received through the app. Six participants indicated that the information was not helpful because it was not relevant to them or they had already done the suggested activities. Some participants provided broader comments. For example, one parent said, “Most of it is info we already had from VI [visual impairment] teachers.” Other participants were more specific. For example, one parent reported,
The IEP [individualized education program] module is not helpful, as my kids do not have IEPs. I suppose it was helpful in that [it] said my kids should be advocating for themselves but doing it within the context of an IEP meeting is not at all relevant to us.

App usefulness ratings in the 6-month survey ranged from 1 to 10. Overall, parents and students found the app moderately useful for employment and college preparation (M = 6.84, SD = 2.91). Students thought the app was slightly more useful (M = 7.53, SD = 2.41) than parents (M = 6.29, SD = 3.20). Most participants (88.4%, n = 38) planned to use the app when it became available: 83.3% (n = 20) of parents and 94.7% (n = 18) of students.

Discussion

This article focused on the final stage in the development of 4to24, a mobile app that supports the preparation of 4- to 24-year-old students with visual impairments for postsecondary education and employment. We conducted a field test with parents and students and examined their app usage, perceptions of its usability, and experiences with the app over 6 months. Most participants indicated that the app was usable, enjoyable, and useful and that it provided understandable and relevant information. When comparing students’ and parents’ perceptions of the app, students provided more favorable ratings in several domains.

Participants’ positive ratings on the ease of the account setup process and quick setup times may reflect the team's efforts to streamline the process and fix the bugs identified during the previous project phases (Antonelli et al., in press). Students were slightly faster at setting up their accounts than parents. This finding may be a product of students being savvier with app technology, as indicated by their higher perceived technology proficiency ratings.

At both time points, app SUS scores were above average (Sauro, 2011; Sauro & Lewis, 2012), indicating that participants felt that the app was user-friendly and intuitive. The SUS scores from the account setup survey demonstrated an upward trend for students compared to scores from our initial usability sessions with similar tasks (Antonelli et al., in press), perhaps due to the improvements made to the account setup process in response to user feedback. Logistical differences between the initial usability sessions and the field test could have also influenced SUS scores since initial sessions were moderated by researchers and involved only brief interactions with the app. Furthermore, the 6-month survey SUS scores are unique because users experienced different and prolonged interactions with the app compared to the other SUS scores that focused on the account setup process.

Parents’ and students’ app use and engagement decreased over time. This finding was unsurprising, considering that only 6.5% of users worldwide continued using apps after 1 month in 2020 (Ceci, 2021). The system for delivering new modules may explain the slight decrease in the number of modules participants read after the 2-month survey. Upon setting up their account, users received introductory modules on various topics, including orientation and mobility specialists, low vision exams, and IEPs. After users completed those modules, the number of new modules in their queue decreased while becoming more individualized based on their interactions with the app. Accordingly, users may have spent more time working on activities within modules as the field test progressed, delaying the completion of those modules and the delivery of new ones. The field test coincided with the COVID-19 pandemic's onset, which could have also influenced participants’ interactions with the app. For example, participants may have focused on immediate needs impacted by the pandemic (Rosenblum et al., 2020) rather than the app. Additionally, social distancing measures and school closures likely impacted participants’ ability to complete the many suggested activities within the app that required interactions in school and community settings. Other potential reasons for the decrease in app usage and modules read are that the app's novelty wore off, and participants decided the app was not useful for them (Kim et al., 2016).

Participants’ ratings of the app's enjoyability, ease of use, understandability, and relevance were generally high throughout the 6-month field test, and some participants reported receiving modules that were very relevant to their current situation. Several components of our iterative app development process may have contributed to these findings. First, input from end users and other key stakeholders obtained throughout the development process informed the final app design. Second, accessibility was a central part of the app's design from the start. Third, the modules underwent several rounds of editing to make the information understandable, user-friendly, and readable on mobile devices. Finally, our focus group findings indicated that receiving relevant content was important to end users (Antonelli et al., 2021). The concept of relevance guided several subsequent design decisions related to the benchmarking and module-delivery systems.

Students rated some aspects of the app higher than parents, particularly enjoyability and relevance. One element that may relate to students’ app enjoyability ratings is the language and tone of the student-focused modules. Although team members wrote all modules to be reader-friendly, they intentionally wrote the student modules to be engaging and fun and used current, informal vernacular compared to the parent modules, which may be more academic-sounding. Accordingly, the parent modules may benefit from revisions to their language and tone to make them more engaging. Several students mentioned the module on filing income taxes as the most helpful and relevant, demonstrating the app's intended purpose of providing valuable and applicable information at the right time.

Several factors may explain the parents’ lower relevance ratings. Many introductory modules that parents received after setting up their account covered services for school-age children with visual impairments. Few students in our sample received those modules because they had already graduated from high school. A group of expert reviewers validated the app content (see Antonelli et al., 2021). They marked very few modules as irrelevant, and most of the introductory modules originated from their recommendations, which may indicate a discrepancy between what the validators and our end users considered relevant.

Furthermore, parents’ characteristics, knowledge, previous experiences, and their children's characteristics likely influenced their perceptions of relevance. For example, the modules covering low vision exams and individualized education program (IEP) meetings did not apply to the parents in our sample whose child was blind or did not have an IEP. Parents who were very informed about services may have rated the app low in relevance—and perhaps even disengaged from the app—based on the introductory modules. The 4to24 app contains over 400 modules covering many topics beyond the introductory content. Because the field test was limited to 6 months, the participants could not experience the vastness and depth of the information available to them.

Limitations

Several limitations of the field test are pertinent to contemplate when interpreting these findings. One limitation is the small sample of participants. Because field test recruitment began several months before implementation, some eligible people did not enroll, potentially because they lost interest or forgot about the study. Another limitation is not every participant completed every survey, further restricting our sample size. The COVID-19 pandemic's overlap with the field test may have impacted the lack of participation and participants’ nonresponse to surveys. Because we did not collect data on participants’ race, ethnicity, level of visual impairment, and parents’ disabilities, we do not fully understand the diversity of our sample. Furthermore, we did not collect information about participants’ mode of accessing information (e.g., audio, visual, or braille display). A final limitation is the use of participants’ self-reported data on app usage rather than detailed data analytics from the app.

Implications and Future Directions

This 5-year project culminated in the public release of the 4to24 app, which is free from the Apple App Store, Google Play store, and the app website (https://4to24.org/). The target audiences for the app include parents of 4- to 24-year-old students with visual impairments and 16- to 24-year-old students with visual impairments. The app contains a curated collection of informational modules written specifically for these audiences, with input from stakeholders and expert validators. Practitioners can identify parents and students who may benefit from the app, inform them about it, and explain how to access it. People who have used other apps should have little trouble setting up their accounts and using 4to24, but novice users may benefit from some initial guidance.

The 4to24 app does not provide a curriculum for parents or lesson plans for service providers and is not a substitute for the specialized services provided by orientation and mobility specialists, teachers of students with visual impairments, and other professionals. Rather, the 4to24 app can complement and reinforce the instruction that professionals provide in several ways. First, it can facilitate conversation between service providers, parents, and students and promote discussion between parents and their children. Second, service providers can recommend that parents and students use the app to explore resources and obtain suggestions for activities to work on at home and in the community. Third, teachers can encourage students to use the app to practice their technology skills while gaining knowledge about college and work.

Although some parents are well-informed about and connected to services, others are not as connected or knowledgeable about services for their children with visual impairments. The app can inform less-connected parents about resources, services, and ways to support their child in becoming an independent adult. These parents may benefit the most from the app; however, intentional marketing efforts, such as targeted advertisements, may be needed to reach them.

Our findings indicate several factors to consider for future iterations of this app and development of similar apps. Adding achievements, badges, or other gamification elements for all user groups may increase engagement, satisfaction, and continued usage (Bitrián et al., 2021). Providing more options for tailoring the content to individual users may improve perceptions of relevance; however, too much customization could make the setup process lengthy and cumbersome. Finally, conducting a multiyear longitudinal study with a larger sample and user focus groups would extend our findings and provide further insight into users’ perceptions of the 4to24 app and its potential benefits. Using data from the app in future studies would provide more precise metrics on user engagement and facilitate further evaluation of usage trends.

Variable	Parents (n = 27)	Students (n = 19)
Device
iPhone	21	77.8	14	73.7
Android phone	5	18.5	5	26.3
Computer	1	3.7	0	0.0
Proficiency level
Competent	10	37.0	2	10.5
Advanced	13	48.1	6	31.6
Expert	3	11.1	10	52.6
Not reported	1	3.7	1	5.3

Group	Account setup survey	6-month survey
Overall	40	84.75	13.50	50–100	43	75.87	18.47	40–100
Parents	25	82.90	13.88	50–100	24	74.90	18.92	40–100
Students	15	87.83	12.71	40–100	19	77.11	18.32	45–100

Group	App use^a	Modules read
Overall
2-month survey	20	83.3	4	16.7	5	4–7
4-month survey	18	62.1	11	37.9	4	2–6
6-month survey	18	41.9	25	58.1	4	1–7
Parents
2-month survey	10	83.3	2	16.7	5	4–6
4-month survey	8	50.0	8	50.0	3	1–4
6-month survey	8	33.3	16	66.7	4	1–6
Students
2-month survey	10	83.3	2	16.7	7	5–21
4-month survey	10	76.9	3	23.1	6	4–15
6-month survey	10	52.6	9	47.4	5	0–10

Footnotes

Authors’ Note

The contents of this report were developed under a grant from the U.S. Department of Health and Human Services, NIDILRR grant 90RT5040. However, these contents do not necessarily represent the policy of the Department of Health and Human Services and should not indicate endorsement by the Federal Government.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the U.S. Department of Health and Human Services (grant number NIDILRR/90RT5040).

ORCID iDs

Anne Steverson

Jennifer L. Cmar

Karla Antonelli

References

Allman

C. B.

Lewis

(2014). ECC essentials: Teaching the expanded core curriculum to students with visual impairments. AFB Press.

Antonelli

Cmar

J. L.

Steverson

(2021a). Development of 4to24, a transition application for parents of students with visual impairments. Journal of Visual Impairment & Blindness, 115(6), 493–505. https://doi.org/10.1177/0145482X211059190

Antonelli

Steverson

Cmar

J. L.

(in press). Usability of 4to24, a transition application for parents of and students with visual impairments. Journal of Visual Impairment & Blindness.

Bangor

Kortum

P. T.

Miller

J. T.

(2008). An empirical evaluation of the system usability scale. International Journal of Human–Computer Interaction, 24(6), 574–594. https://doi.org/10.1080/10447310802205776

Beatty

A. L.

Magnusson

S. L.

Fortney

J. C.

Sayre

G. G.

Whooley

M. A.

(2018). VA Fitheart, a mobile app for cardiac rehabilitation: Usability study. JMIR Human Factors, 5(1), e3. https://doi.org/10.2196/humanfactors.8017

Bitrián

Buil

Catalán

(2021). Enhancing user engagement: The role of gamification in mobile apps. Journal of Business Research, 132, 170–185. https://doi.org/10.1016/j.jbusres.2021.04.028

Brooke

(1996). SUS - A quick and dirty usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. L. McClelland (Eds.), Usability evaluation in industry (pp. 189–194). Taylor & Francis.

Ceci

(2021). Mobile app engagement and retention rates worldwide from July 2019 to September 2020. Statista. https://www.statista.com/statistics/1264722/mobile-apps-global-engagement-and-retention-rates/.

Cmar

J. L.

(2015). Orientation and mobility skills and outcome expectations as predictors of employment for young adults with visual impairments. Journal of Visual Impairment & Blindness, 109(2), 95–106. https://doi.org/10.1177/0145482×1510900205

10.

Crudden

(2012). Transition to employment for students with visual impairments: Components for success. Journal of Visual Impairment & Blindness, 106(7), 389–399. https://doi.org/10.1177/0145482 × 1210600702

11.

DeForte

Sezgin

Huefner

Lucius

Luna

Satyapriya

A. A.

Malhotra

(2020). Usability of a mobile app for improving literacy in children with hearing impairment: Focus group study. JMIR Human Factors, 7(2), Article e16310. https://doi.org/10.2196/16310

12.

Goertz

Y. H. H.

van Lierop

B. A. G.

Houkes

Nijhuis

F. J. N.

(2010). Factors related to the employment of visually impaired persons: A systematic literature review. Journal of Visual Impairment & Blindness, 104(7), 404–418. https://doi.org/10.1177/0145482×1010400704

13.

Griffin-Shirley

Banda

D. R.

Ajuwon

P. M.

Cheon

Lee

Park

H. R.

Lyngdoh

S. N.

(2017). A survey on the use of mobile applications for people who are visually impaired. Journal of Visual Impairment & Blindness, 111(4), 307–323. https://doi.org/10.1177/0145482×1711100402

14.

Hatlen

(1996). The core curriculum for blind and visually impaired students, including those with additional disabilities. RE:view, 28(1), 25–32.

15.

Individuals with Disabilities Education Act, 20 U.S.C., 1401 § 3 (2004).

16.

International Organisation for Standardisation (2018). Ergonomics of human–system interaction—Part 11: Usability: Definitions and concepts (ISO 9241-11:2018). https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:en

17.

Kim

Xie

(2017). Health literacy in the eHealth era: A systematic review of the literature. Patient Education and Counseling, 100(6), 1073–1082. https://doi.org/10.1016/j.pec.2017.01.015

18.

Kim

Baek

T. H.

Kim

Y.-K.

Yoo

(2016). Factors affecting stickiness and word of mouth in mobile applications. Journal of Research in Interactive Marketing, 10(3), 177–192. https://doi.org/10.1108/JRIM-06-2015-0046

19.

Kortum

Sorber

(2015). Measuring the usability of mobile applications for phones and tablets. International Journal of Human-Computer Interaction, 31(8), 518–529. https://doi.org/10.1080/10447318.2015.1064658

20.

Locke

Ellis

Kent

Mcrae

Peaty

(2020). Smartphones and equal access for people who are blind or have low vision. https://www.curtin.edu.au.

21.

Lund

E. M.

Cmar

J. L.

(2019a). A systematic review of factors related to employment outcomes in adults with visual impairments. Journal of Visual Impairment & Blindness, 113(6), 493–517. https://doi.org/10.1177/0145482X19885211

22.

Lund

E. M.

Cmar

J. L.

(2019b). Factors related to employment outcomes in vocational rehabilitation consumers with visual impairments: A systematic review. Journal of Visual Impairment & Blindness, 113(6), 518–537. https://doi.org/10.1177/0145482X19885277

23.

Lund

E. M.

Cmar

J. L.

(2020). A systematic review of factors related to employment in transition-age youth with visual impairments. Rehabilitation Psychology, 65(2), 122–136. https://doi.org/10.1037/rep0000303

24.

Madrigal-Cadavid

Amariles

Pino-Marín

Granados

Giraldo

(2020). Design and development of a mobile app of drug information for people with visual impairment. Research in Social and Administrative Pharmacy, 16(1), 62–67. https://doi.org/10.1016/j.sapharm.2019.02.013

25.

McDonald

Haslam

Yates

Gurr

Leeder

Sayers

(2011). Google calendar: A memory aid to compensate for prospective memory deficits following acquired brain injury. Neuropsychological Rehabilitation, 21(6), 784–807. https://doi.org/10.1080/09602011.2011.598405

26.

McDonnall

M. C.

(2010a). Factors predicting post-high school employment for young adults with visual impairments. Rehabilitation Counseling Bulletin, 54(1), 36–45. https://doi.org/10.1177/0034355210373806

27.

McDonnall

M. C.

(2010b). The employment and post-secondary educational status of transition-age youths with visual impairments. Journal of Visual Impairment & Blindness, 104(5), 298–303. https://doi.org/10.1177/0145482X1010400505

28.

McDonnall

M. C.

(2011). Predictors of employment for youths with visual impairments: Findings from the second national longitudinal transition study. Journal of Visual Impairment & Blindness, 105(8), 453–466. https://doi.org/10.1177/0145482X1110500802

29.

McDonnall

M. C.

Crudden

(2009). Factors affecting the successful employment of transition-age youths with visual impairments. Journal of Visual Impairment & Blindness, 103(6), 329–341. https://doi.org/10.1177/0145482 × 0910300603

30.

McDonnall

M. C.

Sui

(2019). Employment and unemployment rates of people who are blind or visually impaired: Estimates from multiple sources. Journal of Visual Impairment & Blindness, 113(6), 481–492. https://doi.org/10.1177/0145482X19887620

31.

Nagle

K. M.

(2001). Transition to employment and community life for youths with visual impairments: Current status and future directions. Journal of Visual Impairment & Blindness, 95(12), 725–738. https://doi.org/10.1177/0145482X0109501203

32.

Perrin

(2021). Mobile technology and home broadband 2021. Pew Research Center. https://www.pewresearch.org/internet/2021/06/03/mobile-technology-and-home-broadband-2021/

33.

Rosenblum

L. P.

Herzberg

T. S.

Wild

Botsford

K. D.

Fast

Kaiser

J. T.

Cook

L. K.

Hicks

M. A. C.

DeGrant

J. N.

McBride

C. R.

(2020). Access and engagement: Examining the impact of COVID-19 on students birth-21 with visual impairments, their families, and professionals in the United States and Canada. American Foundation for the Blind. https://www.afb.org/research-and-initiatives/education/covid19-education-research

34.

Sanford

Newman

Wagner

Cameto

Knokey

A.-M.

Shaver

(2011). The post-high school outcomes of young adults with disabilities up to 6 years after high school: Key findings from the National Longitudinal Transition Study-2 (NLTS2) (NCSER 2011-3004). SRI International.

35.

Sapp

Hatlen

(2010). The expanded core curriculum: Where we have been, where we are going, and how we can get there. Journal of Visual Impairment & Blindness, 104(6), 338–348. https://doi.org/10.1177/0145482X1010400604

36.

Sauro

(2011). Measuring usability with the system usability scale (SUS). https://measuringu.com/sus/

37.

Sauro

Lewis

J. R.

(2012). Quantifying the user experience: Practical statistics for user research. Elsevier.

38.

Schnall

Rojas

Bakken

Brown

Carballo-Dieguez

Carry

Gelaude

Mosley

J. P.

Travers

(2016). A user-centered model for designing consumer mobile health (mHealth) applications (apps). Journal of Biomedical Informatics, 60, 243–251. https://doi.org/10.1016/j.jbi.2016.02.002

39.

U.S. Census Bureau. (2020). 2019 American community survey 1-year estimates: Employment status by disability status and type, Table B18120. https://data.census.gov/cedsci/table?q=B18120&tid=ACSDT1Y2019.B18120

40.

Virani

Duffett-Leger

Letourneau

(2019). Parenting apps review: In search of good quality apps. MHealth, 5, Article 44. https://doi.org/10.21037/mhealth.2019.08.10

41.

Wolffe

K. E.

(2007). Transition: Planning for the world beyond school. In LaVenture

(Ed.), A parents’ guide to special education for children with visual impairments (pp. 245–290). AFB Press.

42.

Wolffe

K. E.

Kelly

S. M.

(2011). Instruction in areas of the expanded core curriculum linked to transition outcomes for students with visual impairments. Journal of Visual Impairment & Blindness, 105(6), 340–349. https://doi.org/10.1177/0145482×1110500605

43.

Workforce Innovation and Opportunity Act of 2014, 29 U.S.C. Sec 3101 (2016).

44.

Zabelski

(2007). Start at the beginning: The importance of early intervention. In S. LaVenture (Ed.), A parents’ guide to special education for children with visual impairments (pp. 37–58). AFB Press.

45.

Zhou

Smith

D. W.

Parker

A. T.

Griffin-Shirley

(2013). The relationship between perceived computer competence and the employment outcomes of transition-aged youths with visual impairments. Journal of Visual Impairment & Blindness, 107(1), 43–53. https://doi.org/10.1177/0145482×1310700104

Group	Account setup survey				6-month survey
Group	n	M	SD	Range	n	M	SD	Range
Overall	40	84.75	13.50	50–100	43	75.87	18.47	40–100
Parents	25	82.90	13.88	50–100	24	74.90	18.92	40–100
Students	15	87.83	12.71	40–100	19	77.11	18.32	45–100

Group	App use^a				Modules read
	More than once a month		Once a month or less		Modules read
	n	%	n	%	Mdn	IQR
Overall
2-month survey	20	83.3	4	16.7	5	4–7
4-month survey	18	62.1	11	37.9	4	2–6
6-month survey	18	41.9	25	58.1	4	1–7
Parents
2-month survey	10	83.3	2	16.7	5	4–6
4-month survey	8	50.0	8	50.0	3	1–4
6-month survey	8	33.3	16	66.7	4	1–6
Students
2-month survey	10	83.3	2	16.7	7	5–21
4-month survey	10	76.9	3	23.1	6	4–15
6-month survey	10	52.6	9	47.4	5	0–10