Electronic Assistive Technology to Support Memory Function After Traumatic Brain Injury: A Systematic Review of Efficacy and User Perspectives

Search strategy

Potentially eligible articles were identified through title and abstract searches of the following six databases: MEDLINE, Excerpta Medica database (Embase), PsychINFO, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Scopus, and Cochrane Library. In line with the PICO framework, ²⁷ three sets of search terms were used in each database, which related to the population (e.g., TBI, brain injury, head injury, head trauma), intervention (e.g., AT, technology, memory aid, digital, smartphone), and outcome (e.g., memory, remind, remember, cognition; see Supplementary Table S1). Comparator terms (e.g., usual care, wait-list) were not used in the search because designs relevant to this review included those without a control or comparator condition, for example, pre-post group studies.

Searches were restricted to articles on human subjects, written in or translated into English, published in peer-reviewed journals, and with full-text available. Databases were initially searched from their inception to November 26, 2021, with an updated search performed on June 13, 2022. In addition, backward and forward citation searches were conducted on all eligible articles.

Study selection

The screening of articles was facilitated by the web-based software platform Covidence. ²⁸ Following initial screening of articles based on title and abstract, two authors (JM and TO) independently assessed the eligibility of peer-reviewed full-text articles according to the following criteria:

The sample was comprised of adults (minimum age 18 years) with TBI. In samples with mixed causes of brain injury, at least 50% of participants had a TBI diagnosis. Where etiology was not specified, authors were contacted for further information (note: attempts were made to contact authors of four articles; one responded).

Studies were excluded if individuals with TBI were not responsible for using the AT for memory-related tasks themselves (e.g., technology was solely under clinician or caregiver control for both entering reminders and retrieving information) or if the technology was used by clinicians to deliver therapy. ²⁹ Similarly, studies focused on evaluating different approaches to training, ³⁰ rather than the efficacy of AT for supporting memory-related functioning, were excluded. Studies evaluating the efficacy of micro-prompting technology ³¹ to support step-by-step completion of specific tasks were not eligible. Studies were also ineligible if they outlined the conceptual development or design of AT without data to support efficacy. Further exclusion criteria included case studies without sufficient data (e.g., <3 baseline data points), group studies not reporting the statistical significance of pre-post changes or between-group differences, and studies reporting on outcomes not relevant to memory functioning. Conference abstracts, commentaries, letters, editorials, review articles, qualitative studies, study protocols, and non-peer-reviewed material were also excluded from the review. Any disagreements regarding eligibility of articles were resolved through discussion with a third author (CS) to reach consensus.

Data from eligible studies were extracted by one author (JM) and checked with another author (TO). Data extracted included: author(s) and year of publication, sample characteristics, study design, AT features and functions, the AT intervention (e.g., duration/frequency/intensity) and support with use of devices, outcome measures, statistical analytic approaches, and findings related to the review questions.

Quality assessment

Studies were initially classified according to the class levels used in reviews of evidence-based cognitive rehabilitation. ¹² Class I studies were well designed, prospective, RCTs. Within this category, Class Ia studies consisted of prospective designs with quasi-randomized assignment to treatment conditions (e.g., allocation to alternating conditions or crossover designs). Class II studies included prospective, non-randomized cohort studies; retrospective, non-randomized case-control studies; or clinical series with well-designed controls such as multiple-baseline or alternating treatments SCED, as defined by Tate and Perdices. ³² Class III studies referred to clinical series without concurrent controls (single-arm, pre-post studies), or single-case methodology without multiple baselines or experimental control (i.e., AB designs).

Class I studies were further appraised by two authors (JM and TO) using the PEDro-P ²² and PEDro+ ³³ scale. According to authors of the PEDro scale, scores of 6–8 constitute “good” methodological quality, whereas scores of 9–10 constitute “excellent” quality. In line with a review of digital health by Avramovic and co-workers, ³⁴ we included the two additional items developed by Cherney and colleagues ³³ to assess treatment replicability and treatment fidelity (PEDro+). The 11-item PEDro scale assesses: eligibility criteria (not included in score), random allocation, allocation concealment, similarity of intervention groups, blinding (of subject, therapists, and assessor), retention of > 85% of participants, intention-to-treat analysis, between-group statistical comparisons, and point and variability measures. ²² The additional PEDro+ items relate to treatment replicability or providing a sufficient description of treatment procedures to enable clinicians or researchers to accurately reproduce the treatment, and treatment fidelity, or reporting of methods used to examine adherence to the treatment protocol. ³³

Class II and III single-case studies were further appraised using the 15 items of the risk of bias in N-of-1 Trials (ROBiN-T) Scale. ^32,35 Seven items relating to internal validity include: design with control, randomization, sampling of behavior, blinding of patient/therapist, blinding of assessors, interrater reliability, and treatment adherence. The eight external validity and interpretation items include: baseline characteristics, setting, dependent variable, independent variable, raw data record, data analysis, replication, and generalization. As per guidelines by Tate and Perdices ³² and by Tate and colleagues, ³⁵ a 3-point scale (0, 1, 2) was used to rate each item. Any disagreements were resolved through discussion to reach consensus.

Study characteristics

The initial search yielded 4801 articles, which reduced to 2774 once duplicates were removed. A further 2721 articles were excluded by the title and abstract, leaving 53 articles to be screened as full texts. There was 94% agreement between two authors concerning the inclusion or exclusion of the 53 studies. Initial disagreement regarding eligibility of three studies was resolved through discussion with a third author. This process identified 19 articles eligible for review (see Fig. 1). The list of 34 excluded articles and reasons for exclusion are summarized in Supplementary Table S2. Citation searching resulted in a review of 7 further articles screened as full texts, none of which were eligible. Two articles by Wright and associates ^36,37 represent an original study and subsequent replication study of the same AT intervention with different samples. A summary of the study samples, designs, AT, interventions, primary outcomes, and key findings is presented in Table 1.

OPEN IN VIEWER

FIG. 1.

PRISMA flow diagram ²⁶ and search outcomes. AT, assistive technology; SCED, single-case experimental design; TBI, traumatic brain injury.

Quality of methodology

There were 2 Class I studies, ^38,39 2 Class Ia studies, ^17,40 5 Class II studies ^{41 –45} and 10 Class III studies ^{36,37,46 –53} (see Table 1). In the quality appraisal of the 4 Class I studies (see Table 2), there was substantial interrater agreement concerning PEDro-P and PEDro+ scores (κ = 0.88, p < 0.001), with disagreement resolved through discussion to yield final ratings. PEDro-P and PEDro+ scores ranged from 4 to 8. The comparator conditions for the Class I studies involved wait-list control, ³⁸ usual care or practice, ^39,40 and paper-based aids. ¹⁷ In an RCT with multiple crossover design by Dowds and colleagues, ¹⁷ four intervention conditions were evaluated, including usual approach or memory aids (baselines week 1 and 5), and three active conditions of a paper-based organizer and two types of personal digital assistant (PDA) in randomized order between baselines. All four studies reported eligibility criteria, random allocation, > 85% retention, between-groups analysis, and point and variability measures. Concealed allocation and assessor blinding were only conducted by Lannin and co-workers ³⁹ and Ishida and associates. ³⁸ None of the RCTs evaluated treatment fidelity in terms of delivery of the AT intervention.

OPEN IN VIEWER

Table 2.

Quality of Methodology Criteria Ratings of Class I Studies on the PEDro-P ²² and PEDro+ ³³ Scales

In the quality appraisal of the eight single-case studies (see Table 3), interrater agreement was also high (κ = 0.85, p < 0.001), with disagreement resolved through discussion to yield final ratings. The mean ROBiN-T rating for internal validity was 2.75/14 (range: 0–6) and mean external validity rating was 7.5/16 (range: 6–11). The total mean rating was 10.5/30 (range: 6–13). Most single-case studies (> 75%) scored 1–2 for sampling of behavior, baseline characteristics, raw data record, independent and dependent variables, and data analysis. Most studies did not use blinding; conduct interrater agreement; or examine treatment adherence, replication, or generalization. Of the five single-case studies employing a design with control, two used a multiple baseline across participants design with reversal (i.e., ABA or ABAB), ^43,45 one used an alternating treatments design (ABACA) comparing two types of AT, ⁴⁴ one employed a reversal design, ⁴² and one employed a multiple baseline across target behaviors design. ⁴¹

OPEN IN VIEWER

Table 3.

Quality of Methodology Criteria Ratings on the Risk of Bias in N-of-1 Trials (RoBiNT) Scale ^32,35

Sample characteristics

A total of 311 participants with brain injury were involved across 19 studies (M = 16, range: 1–78). Four studies included only individuals with severe TBI, ^42,44,46,50 whereas four involved participants with moderate to severe TBI ^47,51 or mild to severe TBI. ^41,43 Injury severity was not specified for the TBI-only sample employed by Dowds and colleagues. ¹⁷ The remaining studies included participants with mixed ABI etiology (> 50% sustaining TBI), with stroke as the main secondary cause. Studies involved participants in the chronic phase of recovery (> 2 years), apart from three studies, ^42,46,48 which predominantly recruited participants in the first year post-injury.

Studies typically measured severity of memory impairment using objective indices (e.g., Rivermead Behavioral Memory Test), although this was not always an eligibility criterion for participation, with some studies including participants who performed in the normal range on memory testing. ^41,52 The Class I studies by Ishida and associates ³⁸ and Lannin and co-workers ³⁹ only included participants who demonstrated impairment on objective memory tests, whereas other studies recruited participants with subjective memory complaints or those demonstrating functional difficulties.

Types of AT

Most studies (74%) used commercially available devices and software that were not designed specifically for people with brain injury. Devices included PDA, pocket computer and alarm, tablet, pager, smartphone, wristwatch, voice recorder or voice organizer, and television. Participants' own mobile phone was used in three studies. ^41,46,48 Software included alarm, calendar, agenda, notebook, and to-do-list applications, some of which were linked to text or SMS functions. Two purpose-designed systems included NeuroPage, a portable alphanumeric pager, ⁴⁵ and television-assisted prompting, a set-top device and computer connected to participants' home televisions. ⁴⁰ Scheduling and reminding software were purpose-designed for a pocket computer, ^36,37 tablet, ³⁸ and pager. ⁴²

Focus of AT interventions

Table 4 summarizes participants' involvement or independence in using the AT and evidence of efficacy of each intervention. Two studies by Wright and associates ^36,37 focused on use of the AT (pocket computer) for the encoding phase only, with participants taught to make new entries and edit entries as needed. Nine studies involved the researchers externally programming reminders into the AT device, with the interventions focused on participants' use of devices to retrieve intentions and execute memory tasks. ^{40,42 –46,48,51,52} In three studies, ^17,41,47 participants were cued and supervised to enter task reminders into the AT, with the evaluation focusing on their ability to retrieve and execute tasks. Five studies evaluated participants' more independent use of AT to support encoding, retrieval, and execution phases, ^{38,39,49,50,53} although none of these five studies verified that reminders were entered in the AT device.

Memory-related outcomes

Most studies (63%) evaluated the efficacy of the AT intervention using at least one objective behavioral outcome that could be verified by researchers. As summarized in Table 4, these outcomes included frequency of use of the AT or entries made, ^36,37 tested recall of information from reminders, ^48,51 entries made in a daily planner, ⁴² and completion of experimental tasks devised by researchers (e.g., posting a postcard, sending an e-mail or text message, calling a voice mail). ^{17,40,47,49,52} Ishida and associates ³⁸ used standardized tests of memory functioning as a secondary outcome. In addition to experimental tasks, Dowds and colleagues ¹⁷ devised personalized tasks in which participants were required to contact researchers to report on task completion, although task completion was not verified. Caregiver or residential care staff reports of task completion were used in several studies, ^{38,39,41,44,45,46} whereas self-reports of improvement in functioning was the sole approach used by Gentry and associates. ⁵⁰ Lannin and co-workers ³⁹ used Goal Attainment Scaling to assess achievement of personal memory goals.

User perspectives on the utility of AT devices

Nine studies (47%) assessed user perceptions of feasibility, acceptability, and/or utility of the AT and features. ^{36 –38,40,41,46,48,51,52} This included obtaining quantitative ratings of “user friendliness,” ⁵² ease of use, usefulness, frequency of use, and preference for keeping different styles of AT or likelihood of continuing to use or purchasing in the future. ^{36 –38,40,51,52} Lemoncello and colleagues ⁴⁰ additionally asked participants questions (Yes/No) regarding whether or not they found the AT helpful for completing tasks, if it impacted their daily function, and if they would continue to use or recommend it to others. Some studies interviewed participants to obtain feedback regarding their satisfaction and experiences of use with the AT (e.g., problems encountered) and suggested modifications. ^40,41,48,51 Dowds and colleagues ¹⁷ recorded which PDA device participants chose to retain at the end of the study, but without exploring reasons for the choice. Lannin and co-workers ³⁹ recorded whether caregiver support was needed for participants to learn or remember to use the AT and reported the proportion of participants still using the device at 8-week follow-up; however, user perceptions were not obtained.

AT interventions for encoding of memory intentions

Two Class III studies by Wright and associates ^36,37 evaluated individuals' ability (n = 12) to independently use a pocket computer with a purpose-designed interface for diary, alarm, notebook, and to-do list functions. Two different styles of pocket computers were examined, one with and one without a physical keyboard, with training for each provided prior to use. Participants were given each computer for 2 months (in counter-balanced order) with a 1-month break between use. The key outcomes were use of pocket computer functions and reminders set, as indicated by a computer file log, and perceived ease of use and usefulness. In the first study, ³⁶ all participants demonstrated successful use of both computers and 83% rated these as useful. The diary function was rated as significantly more useful than the notebook function. There were no differences in number of entries made, rating of usability, or preference for keeping each computer style. High-frequency users made a similar number of entries across each computer style, whereas low-frequency users made fewer entries in the one without the physical keyboard.

The second study ³⁷ sought to replicate the first study in a different sample (n = 12) using the same AT and intervention procedures, but also added a to-do-list and game functions to the interface of both pocket computer styles. Consistent with the original study, all participants demonstrated successful use of the memory aid functions and most (83%) reported these as very useful. The diary function was rated as more useful than the notebook and to-do-list functions. There was no significant difference in frequency of use of the two different computer styles. Number of diary entries was significantly higher for individuals with greater prior use of memory aids.

AT interventions with pre-programmed reminders

One Class Ia, ⁴⁰ four Class II, ^{42 –45} and four Class III studies ^46,48,51,52 evaluated participants' ability to carry out memory-related tasks that had been pre-programed by a researcher, clinician, or family member.

Using a randomized crossover design (Class Ia), Lemoncello and colleagues ⁴⁰ evaluated utility of Television Assisted Prompting (TAP) in 23 participants with ABI. Participants were randomly allocated to the TAP or “typical practice” conditions, with the latter entailing individuals' use of their usual memory strategies. Participants experienced two 2-week periods of both the TAP and typical practice conditions over an 8-week intervention period. TAP provided audiovisual reminders on the person's home television at scheduled times. The system was devised to allow prompts to be entered from any computer, with the television turning on automatically to deliver prompts at pre-scheduled times. Reminders remained on the screen for up to 1 h (with audio prompts) until the user confirms task completion or elects to refuse or postpone the task (using buttons on the device). Researchers entered the prompts based on individual preferences for the content and timing of delivery. Outcomes included task completion on preferred and non-preferred tasks selected by participants, with completion recorded using a home log, and two experimental tasks devised by researchers to enable objective verification.

Overall, the TAP condition was associated with significantly higher task completion (72%) as compared with the control condition (43%). All participants perceived that TAP helped them to complete tasks, whereas 72% indicated they would continue to use a system like TAP. Strengths of the study included participant retention (92%), use of an objective index of task completion, and treatment replicability. Weaknesses included lack of allocation concealment, blinding, intention-to-treat analysis, and treatment fidelity (PEDro-P = 5; PEDro+ = 6).

A Class II study by Stapleton and co-workers ⁴³ employed an ABAB multiple baseline-across-participants design to investigate use of reminder functions on mobile phones for five individuals with TBI. Individuals were randomly allocated to baseline length (2, 3, or 4 weeks of daily tallies) followed by a 7-week intervention, 2-week baseline, and 2-week second intervention phase. Participants and/or caregivers recorded daily completion of target behaviors. For the intervention periods, participants were trained to use new mobile phones that were programmed by researchers to deliver timely reminders relevant to individuals' memory goals. The analysis indicated a significant increase in target behaviors between the baseline and intervention phases for Participants 1 (51–95%) and 5 (58–88%), who also demonstrated learning of some routines in the second baseline. Target behaviors did not increase significantly for Participants 2, 3, and 4, who demonstrated more severe memory and executive impairment and required 24-h care. User perceptions were not assessed. Key strengths related to the design control, randomization, and behavior sampling. Weaknesses included lack of examination of interrater agreement, replication, and generalization (SCED: 13/30).

Kirsch and associates ⁴² employed an ABA single-case design (Class II) to evaluate the efficacy of an “in-house” alphanumeric paging system for a male with severe TBI. The paging system was designed to cue him to use a daily planner to record information related to speech therapy sessions for evening discussions with a family member. During the baseline (32 days) he received reminders from his therapist at the start of the session to record information in his daily planner at the end of the session. During the 24-day intervention the therapist additionally scheduled prompts from the pager to occur 5 min prior to the end of session to remind the participant to write in his daily planner. Overall, daily planner entries increased from 22% to 94% during the intervention period and decreased to 49% when the pager was withdrawn. User perceptions were not assessed. In addition to lack of examination of interrater agreement, replication, and generalization, there was considerable within-phase variability that was not examined or controlled for using statistical methods (SCED: 9/30).

Wilson and colleagues ⁴⁵ used an ABA group design (Class II) with 15 participants to evaluate the use of NeuroPage, a portable paging system. Target behaviors were selected in collaboration with participants and their caregivers. Memory failures were recorded by caregivers over a 2–6 week baseline that varied for each participant due to the nature of their memory problems and stability of baseline data. During the 12-week intervention period, the NeuroPage service programmed delivered timely reminders. In the 3-week post-treatment phase, the pager was withdrawn. An odds-ratio test accounting for varying success rates for each target behavior indicated that all participants significantly benefited from the pager. Task success increased on average from 37% during the baseline to 85% during treatment. Mean success in the post-treatment phase (74%) indicated some ongoing benefits in terms of learning routines. User perceptions of AT were not assessed. Key strengths included the systematic data analysis and replication, whereas weaknesses included lack of randomization, insufficient details of the intervention and approach to training, and lack of examination of interrater agreement (SCED: 9/30).

Using an ABACA design, a Class II study by Van Hulle and Hux ⁴⁴ evaluated the utility of two types of AT, a wristwatch alarm with vibration function and a digital voice recorder and alarm system, for supporting three individuals with severe TBI to take medication. During each 2-week baseline (A) phase, participants received written reminders (medication schedules). The first treatment (B) phase entailed the use of one AT device along with written reminders. Data collection ceased when participants achieved 100% independence with medication across multiple days. Participant 1 improved from 56% to 75% success during baseline and treatment (wristwatch alarm) and improved to 100% in the second baseline with the watch withdrawn. Participant 2 did not achieve 100% success across any of the phases, including the two AT treatment phases, and continued to require verbal reminders from staff. A third participant achieved 100% success during his initial baseline period and therefore AT was not introduced. User perceptions were not assessed. Key strengths included the lengthy baseline with specified criterion for success, whereas the lack of randomization to order of AT, generalization, replication, and details of training were weaknesses (SCED: 11/30).

A Class III study by Hart and associates ⁵¹ evaluated the efficacy of a portable voice organizer for supporting individuals' (n = 10) recall of therapy goals and plans. Six goal statements were generated by clinicians, with three randomly selected to be recorded on a voice organizer. Goals included broad statements regarding reasons for treatment and more focused personal goals and were recorded by clinicians. Participants selected three times each day to listen to recorded goals, as prompted by an alarm. They wore the voice organizer attached to a belt or purse strap. Recall for therapy goals was tested under two conditions: free recall (no cues) and cued recall (key words for goals provided). Following the 1-week intervention, participants demonstrated significantly better recall of recorded goals than unrecorded goals in both free recall and cued recall conditions. Compliance to listening to the recording was generally high, and 6 out of 10 participants perceived improvement in their memory for therapy goals. Nevertheless, two participants expressed frustration regarding the alarm or functions to play the recorded message.

Following similar procedures to and associates, ⁵¹ the Class III study by Culley and Evans ⁴⁸ tested the efficacy of SMS text messaging for improving recall of rehabilitation goals. Eleven participants were sent SMS texts on their personal mobile phones that related to three randomly selected personal rehabilitation goals out of six, three times each day for 14 days. There were no differences in free or cued recall (cue words given for goal) for these goals at baseline. However, at 7 and 14 days both free recall and cued recall was significantly higher for the goals participants received text messages for as compared with the goals without reminders. The magnitude of difference was greater from baseline to 7 days than from 7 days to 14 days, suggesting that the text messages had the greatest benefit for learning goals in the first 7 days. All participants provided positive feedback regarding the benefits of receiving text messages for supporting their memory, including improved alertness, motivation, and ability to remember and carry out goals.

Baldwin and Powell ⁴⁶ employed an AB design (Class III) to evaluate the efficacy of using Google Calendar on a personal mobile phone for improving task completion. A man with severe TBI and his wife selected the type and timing of prompts (five spaced alerts per event), whereas the researcher entered the events externally. The man's wife kept a log of task completion for three target behaviors and three control behaviors for the 6-week baseline (Weeks 1–6) and 6-week intervention (Weeks 8–13). Week 7 was a training week in which the man was taught to respond to task alerts. Using non-overlap of all pairs, the analysis showed a 90% reduction in forgetting for target memory behaviors, whereas no change was evident for two of the three control behaviors. A comparison of pre- and post-intervention ratings on self-report measures indicated improvements in the man's subjective memory and an increase in personal control and treatment control beliefs, whereas inappropriate beliefs and threat appraisals about use of memory aids decreased. Key strengths of the study included the 6-week baseline, in-depth sampling of behavior, and assessing generalization. Weaknesses related to the AB design, randomization, replication, or examination of interrater agreement or treatment adherence (SCED = 13/30).

A Class III study by Thöne-Otto and Walther ⁵² examined the utility of two memory aids, a palm organizer and a mobile phone with agenda function. They employed an ABAC design with group-level data (n = 12). During each 2-week phase, participants were required to perform 20 experimental memory tasks. The order of memory aids was counterbalanced, and participants received 5 h of training per device. Based on progress during training, six participants were classified as slow learners and six as fast learners. Overall, improvement in memory performance was not significant for either device. Only fast learners spontaneously used the devices and were more likely to report the desire to continue using the device, particularly the palm organizer. Various disadvantages related to the number of menu steps, interface (key and touch panel size), and alarm were reported.

AT interventions with cued encoding and independent retrieval and execution

A Class Ia study ¹⁷ and two single-case studies (Class II, ⁴¹ Class III ⁴⁷ ) evaluated AT interventions in which participants were cued and supervised to enter reminders into a device. Dowds and colleagues ¹⁷ examined the efficacy of automated reminders from two PDA-type devices in 36 people with TBI with self-reported memory problems. The palmtop-sized computers were a Palm device and a Microsoft device. A randomized multiple crossover design was employed over an 8-week period with four conditions: baseline, pen and paper memory, aid and two PDAs. Memory aid condition changed each week in a randomized order, with training occurring just prior to use. Participants were supported to enter reminders into each memory aid when receiving their weekly assignments. The main outcome was timely task completion based on researcher-assigned tasks (daily calls to an answering machine or office) and time-related personalized tasks selected in collaboration with participants. Use of either PDA was associated with significantly higher task completion rates as compared with baseline (27%) and paper memory aids (26%). The Palm PDA was associated with significantly higher completion rates (56%) than the Microsoft PDA (38%), which authors attributed in part to the louder alarm for the Palm device. Most participants (75%) chose to retain the Palm PDA after the study, although user perceptions were not assessed. Strengths included high retention (92%) and use of an objective index of task completion. Weaknesses included lack of allocation concealment, blinding, intention-to-treat analysis, treatment fidelity, and replicability (PEDro-P = 4; PEDro+ = 4).

A Class II study by Cruz and colleagues ⁴¹ evaluated the efficacy of Google Calendar with text messages on personal mobile phones for improving prospective memory in two participants with a mild or severe TBI. Using a multiple baseline across target behaviors design, the onset of reminders for each activity was randomly assigned (minimum baseline of 5 data points). Reminders were entered by participants during rehabilitation sessions with therapist support. The main outcome was activity completion as recorded by the participant or caregiver. Non-overlap of all pairs analysis for Participant 1 indicated significant improvement for one out of three tasks, along with generalization to a new activity. Participant 2 demonstrated significantly improved performance on one out of two tasks. User feedback indicated that the content of reminders and nature of the activity influenced effectiveness of the prompts, and that additional strategies were often required. Key strengths related to the randomization of reminder onset, length of baseline, and data analysis. Weaknesses included lack of examination of interrater agreement, replication, and generalization (SCED: 13/30).

Bos and associates ⁴⁷ evaluated the use of a smartphone relative to baseline and memory notebook conditions (AB₁B₂; n = 2) or relative to baseline only (AB; n = 5). Participants were randomly allocated to a baseline length of 3–7 weeks. Training for each memory aid was provided prior to the 8-week intervention period. Researchers cued participants to enter reminders for memory tasks during weekly appointments. Tasks entailed sending text messages with specified content and posting a postcard. For individuals receiving both memory aids, task performance improved between baseline and notebook use, but additionally improved during the smartphone condition for two out of three memory tasks. For individuals receiving the smartphone after baseline, results were mixed. Three participants showed clear improvement in task performance on two or three of the tasks, one participant showed better performance on the texting task and a decline on the postcard task, and one participant showed a decline in performance across all tasks when using the smartphone relative to baseline. The mixed findings were attributed to participants' variable motivation and attitude toward using smartphones, severity of memory impairment, and lack of personalization of memory tasks. However, user perceptions were not assessed. Strengths included randomization of baseline length, in-depth data record, and objective indices of task performance, whereas weaknesses included lack of experimental control, replication, and generalization (SCED: 12/30).

Interventions promoting independent use of AT to support memory

Two Class I studies ^38,39 and three Class III studies ^49,50,53 examined the efficacy of AT used independently by participants to enter information (encoding), retrieve, and execute tasks.

Lannin and co-workers ³⁹ conducted an RCT to evaluate the efficacy of a PDA for achieving memory and organization goals relative to standard occupational therapy without electronic memory aids. Forty-two individuals with ABI who demonstrated objective memory deficits were recruited. Individuals allocated to the PDA condition chose between a Windows or Macintosh device, both of which had alarm, calendar, address book, and camera features. Training and support to use the PDA was conducted during occupational therapy sessions over 8 weeks (mean therapy = 7 h; range = 6–21 h) and entailed five modules focused on: choosing personally meaningful activities to increase independence with caregiver input, self-awareness training, training in basic skills to use the PDA, learning to use the PDA to schedule and act on reminders, and generalization of strategy use outside of therapy. Individuals in the PDA condition demonstrated significantly greater achievement of memory goals and reduction in caregiver reports of forgetting. There were no significant differences in participants' subjective memory functioning. Caregiver support to use or remember to use the PDA was required for 19%, and nearly one-third required further support from caregivers between therapy sessions.

At the 8-week follow-up assessment, 67% of those who received the intervention reported that they continued to use the PDA daily, whereas 14% had ceased using the PDA. User perceptions or reasons for discontinued use were not assessed. Strengths of the study included allocation concealment, blinded assessment, participant retention (100%), and intention-to-treat analysis (PEDro-P = 8; PEDro+ = 8). Limitations included the lack of evaluation of treatment fidelity and varying intensity of therapy hours involved in the intervention, which impacts treatment replicability.

A Class I study by Ishida and associates ³⁸ used a wait-list control design to examine the efficacy of tablet computer software for improving memory function in 78 participants with ABI. The intervention entailed an initial 2-week training period during which therapists taught participants how to operate a 7-inch tablet PC, with practice to ensure they could use event scheduling, reminders, memo notes, and timer functions. The schedule function included a checklist to confirm actions were completed. Participants used the tablet independently over 3 months to manage their daily schedule, with additional support from therapists as needed. There were no significant effects of intervention for self -and caregiver-reported memory function (primary outcome), caregiver burden, or psychosocial functioning. However, the intervention was associated with better performance on an objective memory test (Rivermead Behavioral Memory Test) and improved general self-efficacy relative to the wait-list condition. Subjective reports indicated that the daily schedule with voice reminder and diary functions were most frequently used (35–41%); however, 78% reported the tablet was moderately difficult or difficult to use, mainly due to the size (portability) and managing message sounds. Strengths included allocation concealment, high retention (98%), and assessor blinding. Weaknesses included the lack of intention-to-treat analysis, analysis of treatment fidelity, and details to support treatment replicability (PEDro-P = 7; PEDro+ = 7).

A Class III study by Gentry and associates ⁵⁰ evaluated the effectiveness of PDAs for improving occupational performance in 23 individuals with severe TBI. The training was conducted in each participant's home during three to six 90-min sessions. No information was provided regarding PDA selection or allocation. Participants were given a PDA to use independently over 8 weeks. Approximately one-third contacted researchers during the intervention for support to use the PDA. A comparison of pre- and post-intervention functioning indicated significant improvement in self-perceived occupational performance on personally relevant tasks (e.g., remembering appointments, time and task management, medication management) and self-rated cognitive independence, mobility, and occupations. User perceptions were not assessed. Key limitations of this study related to the sole reliance on self-reported functioning and lack of experimental control.

A Class III study by Evald ⁴⁹ evaluated the effectiveness of a smartphone combined with Internet-based calendar for improving memory performance in 13 individuals with TBI. Participants received an initial individual session focused on phone and Internet set-up and synchronization followed by five group sessions in which they learned to use the AT to enter appointments, contacts, and memory assignments through smartphone applications. Self-reported memory function and daily achievement of target behaviors (daily memory log, e-mail reporting task) was assessed at pre-intervention, post-intervention, and 2-month follow-up assessment. Overall, there was no significant improvement in performance on the assigned memory tasks at post-intervention or follow-up; however, participants reported significantly fewer memory problems at both time-points relative to pre-intervention. Most participants (77%) continued to use the smartphone for memory compensation at 2-month follow-up, although user perceptions were not assessed.

A Class III study by Waldron and colleagues ⁵³ evaluated the efficacy of an intervention for teaching five participants with ABI (60% TBI) to program a PDA to provide reminders to complete seven personally relevant memory tasks. All tasks could be objectively verified as completed. An AB quasi-experimental design with usual memory aids only (3-week baseline) and PDA strategy phases was employed. Participants were trained to use the PDA in three weekly 90-min group sessions. Individual and group level analysis indicated that rates of task completion significantly improved between baseline (M = 59%) and PDA strategy phases (M = 90%). User perceptions of the AT were not assessed. Strengths included replication of findings and objective indices of task performance, whereas weaknesses included lack of experimental control, randomization, interrater agreement, and generalization (SCED: 6/30).

Limitations

Several limitations of this review need to be considered when interpreting the evidence of efficacy of electronic AT interventions and factors influencing uptake. First, the findings are largely based on Class II or Class III studies, which, according to guidelines on evidence-based cognitive rehabilitation, ¹² indicate “possible effectiveness.” Of the four Class I studies, none examined treatment fidelity and only two ^38,39 employed blinding of assessors. Although it is challenging to blind participants and therapists in behavioral interventions, in trials involving comparison of two or more active interventions, the hypotheses regarding intervention efficacy could be concealed from participants and therapists to minimize potential bias. Notably, only one RCT in the present review used an active control condition, which involved a paper-based organizer. ¹⁷ Nonetheless, there was evidence for the effectiveness of AT interventions for supporting memory-related function as a practice standard, based on at least one well-designed Class I study ³⁹ with additional support from two Class Ia, ^17,40 two Class II, ^44,45 and seven Class III ^{36,48 –53} studies.

Second, evidence in this review was combined for a wide range of technologies. Although the focus was on the phase of memory supported, it is important to acknowledge that individuals' experience and use of AT most likely varied for different devices. For example, in several studies participants used their own smartphone, ^41,46,48 television, ⁴⁰ or computer, ⁵⁰ which are likely to have been used for other various purposes. In other studies, devices were supplied for the purpose of the study and were only used for reminding. Hence, participants' prior use of and familiarity with the device may have impacted the effectiveness of the intervention. Many studies attempted to address the lack of familiarity through training ^{17,38,39, 41,43,46,47,49 –53} or supporting the use of AT throughout the trials, ^39,50 but their effect on outcomes is unclear. The influence of familiarity and prior use of AT devices and features on intervention outcomes requires further research.

Third, there was considerable variability in the memory-related outcomes assessed across the studies (see Table 4). Seven studies employed self-report or caregiver-reported outcomes that were not verified by researchers. More subjective indices may be more prone to bias, which is not addressed by the PEDro-P ²² or PEDro+ ³³ scales and is accounted for by one item on the RoBiNT scale. ^32,35 In the measurement of memory-related functioning, it can be challenging to employ ecological tasks that are meaningful to participants while minimizing potential bias. Reliance on participants with memory impairment to record their own task success or memory failures has questionable reliability and such actions during the baseline period could inadvertently support memory and reduce the chances of detecting an intervention effect. Some studies aimed to personalize performance-based tasks according to the nature of participants' typical memory failures or memory goals. ^48,51,53 For example, Waldron and colleagues ⁵³ asked individuals with TBI and family members to describe typical memory failures experienced and devised seven related ecological tasks for which outcomes could be objectively verified.

Fourth, long-term follow-up and investigation of factors influencing ongoing use of AT beyond the intervention was a major gap in the studies reviewed. In the studies that provided novel technology to participants, the device was typically withdrawn at the end of the intervention. As an exception, two studies, by Lannin and co-workers ³⁹ and Dowds and colleagues, ¹⁷ gave participants a PDA to keep at the end of the study; however, only Lannin and co-workers examined continued use at 8-week follow-up, based on self-report. In the three studies in which participants used their own smartphones, ^41,46,48 none examined long-term maintenance or ongoing use of the AT functions taught during the intervention.

Finally, due to the primary focus on evidence of efficacy, user perspectives of utility reflect only those gained following the AT intervention, rather than during earlier design or piloting phases. As noted by Aldrich, ⁶⁰ it is optimal to request feedback from potential users at frequent intervals rather than solely at the end of a trial period of use. Aldrich further highlighted likely benefits of direct observation of individuals engaging with the AT to gauge their reactions and immediate feedback, rather than relying on individuals' recall of experiences in a later interview. For example, Wallace and colleagues ⁶¹ reported user-centered development of an external memory aid (EyeRemember), in which ongoing feedback was sought from people with brain injury, their caregivers, and clinicians through focus groups, sit-by demonstrations, and prototype testing.

Future research directions

In future research, it is recommended that well-designed clinical trials focus on comparing the efficacy of AT interventions providing varied levels of support with encoding (e.g., self-initiated, cued, and pre-programmed reminders) and factors moderating outcomes (e.g., severity of cognitive impairment, caregiver support). For studies evaluating individuals' independent use of AT, it is optimal to verify device use (i.e., reminders entered) and task completion using objective indices. Further research is also needed to support the preliminary framework (Fig. 2) of general and phase-specific factors influencing utility and uptake of AT for memory in practice. Notably, recent experimental research by Jamieson and colleagues ⁶² compared the effect of narrow-deep (i.e., more screens with less information displayed per screen) and broad-shallow (i.e., fewer screens but more information displayed per screen) user interfaces when teaching people with ABI to use a custom designed reminding application (ApplTree ⁵⁸ ). Participants made more errors and omitted more information when using the broad-shallow interface as compared with the narrow-deep one. However, preferences for user interface varied and influenced perceptions of ease of use.

Understanding user preferences for the type of AT device and design features is integral to patient-centered care and is best facilitated through co-design with intended technology users. ^61,63 Research also shows that the approach used to train individuals to use AT for memory can influence learning outcomes, although there are mixed findings concerning the relative efficacy of systematic instruction, errorless learning, and trial-and-error learning. ^30,64 Outcomes of training may vary according to participants' preferred learning style and severity of cognitive impairment. ⁶⁵ Evaluating the efficacy of training approaches and the influence of user characteristics remains an important focus for future research. Given the lack of studies evaluating long-term use of AT, there is also a need for research to investigate long-term uptake or sustained use of AT following interventions and factors influencing long-term use.