Assessment of Aphasia Across the International Classification of Functioning,Disability and Health Using an iPad-Based Application

Abstract

Background:

Access2Aphasia™ is an iPad™-based aphasia assessment application that enables real-time audiovisual communication between people with aphasia (PWA) and speech-language pathologists (SLPs), and the use of supported conversation techniques. This study aimed to establish the reliability of aphasia assessment across the International Classification of Functioning, Disability and Health (ICF) using Access2Aphasia, and compare it with face-to-face (FTF) assessment. Consumer perspectives of Access2Aphasia were also examined.

Materials and Methods:

Thirty PWA were randomized into two conditions: online-led and FTF assessment. Participants in the online-led group were assessed remotely using Access2Aphasia™ in their own homes, while an FTF SLP scored silently simultaneously. Participants in the FTF group were assessed FTF using standard administration materials. Assessment included two subtests of the Psycholinguistic Assessment of Language Processing Activities (PALPA) and the Assessment of Living with Aphasia (ALA) to allow for outcomes to be captured across the ICF domains. Consumer perspectives on Access2Aphasia were obtained from both PWA and research SLPs in the online-led group.

Results:

Kappa statistics indicated moderate to almost perfect agreement between online and FTF SLPs (k = 0.71–1.00). Intrarater and interrater reliability was excellent (ICC = 0.99–1.00) and equivalent for the online-led and FTF conditions. Both PWA and research SLPs in the online-led group reported being satisfied with the experience overall, with suggestions provided by research SLPs to improve Access2Aphasia.

Conclusion:

This study supports the provision of iPad-based aphasia assessments across the ICF in the online environment, with comparable reliability to FTF assessments. Future research is warranted to support the development of iPad-based aphasia assessment and treatment as an alternative mode of service delivery to PWA.

Introduction

Equitable access to services is an issue for people with aphasia (PWA), particularly those with significant physical and communication disability.¹ For some PWA, the degree of physical, cognitive, and emotional effort required to attend a face-to-face (FTF) clinical session may negatively impact on their capacity to benefit from the intervention.¹ Other barriers to services include the need for carer assistance, transport, and cost.²

Telehealth has been previously found to be a viable mode of service delivery in aphasia assessment^3

–7 and may be able to mitigate some of these challenges. Overall, these aphasia studies demonstrated comparability between remotely delivered assessment using personal computers and FTF assessment, indicating that aphasia assessment may be reliably provided via telehealth. While preliminary findings have been encouraging, these studies were conducted within the artificial environment of a remote laboratory or clinic, rather than evaluating people in their own homes and, therefore, may lack ecological validity. In addition, not all of these studies included participants with severe aphasia, who may potentially have the greatest problem with technology access. For instance, in a study where participants with severe aphasia were included, it was found that the ability to assess naming and paraphasia (the phenomenon where PWA use the wrong words or use words in wrong or senseless combinations) through telehealth was impeded.⁴ Apart from focusing on participants with predominantly mild–moderate aphasia, studies have been limited to the investigation of impairment-focused assessments (e.g., with tasks such as naming of pictured nouns and following of one-step instructions). According to the World Health Organization (WHO), the International Classification of Functioning, Disability and Health (ICF) framework should be taken into consideration in assessment and outcome measurement.⁸ The ICF defines health in terms of Body Structure and Function, Activities and Participation, and Personal and Environmental Factors. This means that aphasia assessments should not only encompass tests of language abilities (“Body Function”) but also take into account other domains, including the person's ability to participate in daily activities and relationships, and the influence of personal and environmental factors. Of relevance also is how PWA view their lives across the ICF domains. The question therefore remains as to whether those with severe communication difficulties following stroke can be assessed adequately using telehealth technology without access to supported conversation techniques,⁹ particularly when they are excluded from research.

Access to supported conversation techniques is crucial to include people with severe aphasia in telehealth research and services. This is because people with severe aphasia may experience a multitude of language deficits, including difficulties understanding and generating spoken and/or written words and sentences. Supported conversation includes the use of pictographic support, written words, or drawings using a collaborative whiteboard to facilitate the interaction.⁹ These techniques are commonly used in FTF assessments by speech-language pathologists (SLPs) to enable the person to convey their message, and it is indeed critical for SLPs to evaluate how well a person can use these strategies to get their message across. Traditional videoconferencing technologies do not typically permit this type of multimodal communication, particularly in relation to the use of drawing to enhance communication.

Tablet technology has the potential to bridge this gap and tablets have been increasingly incorporated into aphasia management. However, the usage has been largely for communication support (e.g., Lingraphica SmallTalk™ apps) and increasing intensity of treatment through home practice, for example, using impairment-based treatment apps such as Constant Therapy.¹⁰ The evidence for the use of tablet apps as a means for telehealth assessment and treatment of aphasia remains lacking, particularly in people with severe aphasia.

Recently, researchers in Korea developed a mobile aphasia screening test (MAST) in the form of an iPad™ (Apple, Cupertino, CA) application to screen for aphasia poststroke.¹¹ Based on the Korean version of the Frenchay Aphasia Screening Test¹² (K-FAST), the MAST incorporates voice-guided directions during the test, audiorecording of patients' verbal responses, and touch patterns that are stored and sent to a central database. Verbal expression was scored by listening to the audiorecordings and auditory comprehension was automatically scored by the app system based on the touch patterns. The MAST and the conventional paper version (K-FAST) were given to patients with (n = 30) and without (n = 30) poststroke aphasia. PWA were also given the Korean Western Aphasia Battery¹³ (K-WAB). The MAST was found to have a significant correlation with the K-FAST as well as the K-WAB, providing support for its validity and reliability for detecting aphasia after stroke. Tablet technology can thus be used as an alternative to existing aphasia screening tests for people after stroke.

Going beyond aphasia screening, we previously investigated the usability of a novel iPad application, Access2Aphasia™, which allows the use of supported conversation techniques with PWA,¹⁴ including those with severe aphasia, thereby potentially improving access to services. In addition, Access2Aphasia enables simultaneous videoconferencing, which allows the use of techniques such as varying intonation and gestures to facilitate communication. This integration of features in Access2Aphasia enables impairment-, activity-, and participation-oriented assessments to be investigated via telehealth. As mentioned previously, the assessments that have been reported to date have been impairment oriented.^4,5 Activity- and participation-oriented assessments are also necessary so that real-life outcomes may be captured and communicated to stakeholders such as clients, families, and policy makers in a meaningful way.¹⁵

Therefore, the primary aim of this study was to determine the reliability of telehealth aphasia assessment based on measures that examine aphasia across the ICF spectrum within the home environment using Access2Aphasia. It was hypothesized that telehealth aphasia assessment can be as accurate and reliable as an FTF aphasia assessment (i.e., telehealth aphasia assessment will yield similar scores to FTF assessment for all outcome measures, and interrater and intrarater reliability for the telehealth assessment will be as high as for FTF assessment). The secondary aim of the study was to evaluate consumer perspectives of telehealth aphasia assessment in the home environment using Access2Aphasia.

Materials and Methods

Participants

Our study received ethical clearance from the National Healthcare Group, Singapore. All participants provided informed consent. Thirty participants who were at least 6 months poststroke onset were recruited from local rehabilitation facilities and speech pathology clinics. All participants had been diagnosed by an SLP to have acquired aphasia, which ranged from mild to severe as profiled using the Singapore Aphasia Test,¹⁶ a locally developed language assessment, and the Boston Diagnostic Severity Rating Scale.¹⁷ Individuals with comorbidities such as hemiparesis and visual field loss were included in the study. Information on education, language background, and technology experience was collected through a questionnaire.

Participants were excluded from the study if they had a previous history of speech and/or language disturbance before the current language disorder to ensure that assessment findings were due to the impact of the aphasia and not another prior communication disorder (e.g., stuttering), which may also influence the person's activity, participation levels, and self-esteem and invalidate the Assessment of Living with Aphasia (ALA), an activity and participatory measure.¹⁸ In addition, individuals were excluded if they had a known history of dementia, other major psychiatric illness, and/or significant uncorrected visual/hearing impairment. Individuals with significant visual perceptual problems such as agnosia were also excluded.

Four qualified SLPs, who were not part of the study team, but experienced in the assessment and treatment of communication disorders, participated in the online assessment of the participants, while one SLP (YEG) conducted the FTF assessments. All SLPs had undergone training in the use of the Access2Aphasia as well as in the administration of the ALA.

Measures

The ALA¹⁹ and the Spoken Word–Picture Matching and Spoken Word Naming subtests of the Psycholinguistic Assessments of Language Processing in Aphasia (PALPA)²⁰ were used to assess each participant. The ALA was selected to examine the participant's perception of their lives living with aphasia across the ICF domains of impairment, activity, participation, and environmental and personal factors, while the PALPA subtests provided information about the participant's aphasia at the impairment level. The ALA fills a gap in aphasia assessment that has not been adequately addressed as SLPs have typically focused on impairment-level language difficulties.^21,22 Our previous survey found PALPA subtests to be widely used by Singapore speech therapists, while internationally, the Spoken Word–Picture Matching and Spoken Word Naming subtests are reportedly the most frequently used PALPA subtests by speech therapists.²³ Therefore, we chose these two assessments to assess the reliability of the Access2Aphasia as they represented a spectrum of assessment tasks and response formats across the ICF, spanning from impairment to participation. For the ALA, PWA were asked questions in a conversational manner and then asked to respond through a 9-point Likert scale. The selected ratings were then computed to give domain and total scores. The domains, which are closely adapted from the ICF, are as follows: Language Impairment/Aphasia (score range: 0–20), Participation (score range: 0–64), Personal factors (score range: 0–16), Environmental factors (score range: 0–44), and a “moving on with life/wall” question (score range: 0–4). The ALA total score has a range of 0–148 and is considered an estimate of “aphasia-related quality of life.” Provision of context and consistency checks were required by the SLPs to ensure that the questions were understood by PWA (i.e., dynamic supported conversation was required even under remote assessment conditions). For the PALPA, scores were obtained based on touch selection of PWA for the Spoken Word–Picture Matching subtest and verbal output for the Spoken Word Naming subtest for both telehealth and FTF aphasia assessments. In the Spoken Word–Picture Matching subtest, the SLP read out a target word and PWA selected the corresponding picture out of five options, while the Spoken Word Naming subtest required PWA to name the picture displayed. Each PALPA subtest contained 40 items, which were scored correct/incorrect and had a score range of 0–40.

Access2Aphasia application

Access2Aphasia is a novel customized system with two apps, one for the SLP, and the other for PWA, which communicates through a custom server for secure communication (see Fig. 1 for the system architecture). The app consists of integrated videoconferencing (using a third-party software development kit ooVoo (www.oovoo.com), a collaborative whiteboard, and the ability to present preloaded flashcards and record responses into a MySQL database. It is only used for research purposes at this stage.

Fig. 1.

Access1Aphasia^™ system architecture.

Figure 2 shows the differing SLP and PWA views of the ALA stimulus, while Figures 3 and 4 show differing views of the PALPA stimulus. Figure 5 shows the full SLP view during test administration.

Fig. 2.

ALA test—Rating scale selection The green bar on the rating scale indicates the PWA's selection, which shows up as an orange numbered rating on the SLP view. The green triangle indicates pointing by the SLP when explaining the use of the rating scale and/or checking responses. The SLP may also click on the number ratings on behalf of the PWA. Additional features on the SLP view include arrows to move back and forth between questions if needed, the ability to enter comments in the Comment Box, confirmation of answers to save responses to the database on the server, and resetting the selected response. ALA, Assessment of Living with Aphasia; PWA, people with aphasia; SLP, speech-language pathologist. Color images available online at www.liebertpub.com/tmj

Fig. 3.

PALPA spoken word–picture matching subtest. In this subtest, the SLP reads out the target word (displayed in English, Chinese, and Hanyu Pinyin). PWA then tap the corresponding picture. PWA selection is indicated with a rectangle on both sides. The app automatically maps the PWA selection to a response type and displays it. In this example, “sock” for target word “hat” is categorized as Distant Related Distractor. A “No Response” option is also available if the PWA does not provide an answer. The PWA view is kept simple with picture stimulus. PALPA, Psycholinguistic Assessments of Language Processing in Aphasia.

Fig. 4.

PALPA picture naming subtest. In this subtest, the PWA is required to name the picture displayed. The SLP is able to categorize the initial response type, as well as provide optional information about cues that the PWA is responsive to. Additional functions such as entering comments in the Comment Box, confirming the answer to save the response to the database on the server, and resetting the selected answer are similar to the ones in the ALA Test shown in Figure 2.

Fig. 5.

SLP view of app on tablet, including ALA stimulus, videochat interface, and collaborative whiteboard. SLP, speech-language pathologist.

Procedure

To minimize test bias, participants were randomized to either an online-led or FTF assessment group. Stratified random sampling was used to ensure equitable representation of aphasia severity and English/Mandarin speakers in each group.

Online-led group

Each participant was seated at a table with adequate lighting in the living room or dining room of their own home and was assessed remotely by an online SLP who was located in an office 12–40 km away. No headphones or microphones were used during the sessions. Background noise within the participants' homes was minimized if possible (e.g., radios were turned off), but not controlled outside the participants' homes (e.g., passing trucks or neighbors). Both the participant and the SLP had access to an iPad and were simultaneously logged into Access2Aphasia application through the Internet. Internet access was enabled at the participants' home using a mobile modem (Huawei Mobile Wifi E5377) that supports LTE cat4 and has download speeds of up to 150 Mbps. However, actual bandwidth was dependent on the state of network and environment of the individual locations.

Apart from having the stimulus pictures displayed on the tablet rather than on paper and having the assessment conducted online, administration procedures were not modified. To enable comparison of online and FTF conditions, an FTF SLP was seated beside the participant, who scored the ALA and PALPA simultaneously on paper while observing the assessment, online SLP, and participant's responses on the iPad silently and blinded to the telehealth SLP's scoring. Video and sound recordings of the process were made to enable playback for reliability purposes.

All participants and SLPs in this group completed the adapted satisfaction questionnaires⁵ after the assessment with regard to the use of Access2Aphasia, which included questions on overall satisfaction, audio and video quality, level of comfort (PWA) or ease of use (SLP), their views on remote assessment compared to FTF assessment, as well as whether they would participate again. The surveys were presented in an aphasia-friendly format for PWA (Appendices 1 and 2). For participants with severe aphasia, the satisfaction surveys were completed with assistance from the FTF SLP.

FTF group

As described above, in the FTF environment, each participant underwent the same assessment protocol in their own home by the FTF SLP. The assessments were conducted based on test administration instructions using the paper stimulus materials from the ALA and the PALPA. Video and sound recordings of the assessments were made to enable playback for evaluation of reliability.

Data analyses

An independent t-test was used to rule out potential group differences on age, aphasia severity, and time since onset, while chi-square tests were used to compare gender, education level, and computer use pre- and poststroke. For the online-led group, kappa coefficients were calculated to determine the strength of agreement between online and FTF assessment for the PALPA subtest scores and the ALA domain and total scores (i.e., scores by the online clinician and the FTF clinician during the online session were entered into the kappa).

Stored video recordings for four randomly selected participants in each group were rated by three SLPs to determine interrater and intrarater reliability for each condition. Intraclass correlations (ICC) were used to determine interrater and intrarater reliability for the online and FTF assessment. The following criteria for strength of agreement were used to interpret the kappa and intraclass coefficients: <0.40 = minimal agreement; 0.40–0.59 = weak agreement; 0.60–0.79 = moderate agreement; 0.80–0.90 = strong agreement; and >0.90 = almost perfect agreement.²⁴

Responses to the participant satisfaction questionnaires and the SLP satisfaction questionnaires were analyzed descriptively.

Results

Thirty participants (23 males, 7 females) ranging in age from 35 to 79 years (mean 61.5 years, SD 12.03 years) were recruited to the study. All the participants had aphasia poststroke with a mean time postonset of 22.2 months (range 6–79 months). There was no significant difference between the online-led group (n = 15) and FTF group (n = 15) in terms of age, time since onset, aphasia severity, gender, dominant language, education level, and computer use prestroke and poststroke (Table 1).

Table 1.

Participant Characteristics

CHARACTERISTIC	ONLINE (N = 15)	FACE-TO-FACE (N = 15)	STATISTICAL TEST
	Mean (SD)		t-test	df	p
Age in years	60.9 (11.1)	62.0 (13.2)	−0.24	28	0.18
Time since onset (months)	26.4 (21.0)	18.04 (13.5)	1.29	28	0.21
BDAE Aphasia Severity (0–5)^a	2.80 (0.94)	2.53 (0.99)	0.76	28	0.46
Count (%)
Mild	5 (33.3)	5 (33.3)
Moderate	5 (33.3)	6 (40.0)
Severe	5 (33.3)	4 (26.7)
	Count (%)		Chi-square	df	p
Gender			1.68	1	0.20
Male	10 (66.7)	13 (86.7)
Female	5 (33.3)	2 (13.3)
Dominant language			0.00	1	1.00
English	8 (53.3)	8 (53.3)
Mandarin	7 (46.7)	7 (46.7)
Education level			3.51	3	0.32
Primary	5 (33.3)	4 (26.7)
Secondary	6 (40)	4 (26.7)
Pre-University/diploma	4 (26.7)	4 (26.7)
University	0 (0)	3 (20.0)
Computer use before stroke			1.37	1	0.71
Yes	9 (60)	8 (53.3)
No	6 (40)	7 (46.7)
Computer use after stroke			0.68	1	0.41
Yes	5 (33.3)	3 (20)
No	10 (66.7)	12 (80)

Boston Diagnostic Aphasia Examination (BDAE) Aphasia Severity: 0 = no usable speech or auditory comprehension; 5 = minimal discernible speech handicaps; patient may have subjective difficulties, which are not apparent to listener.

Strength of agreement between online and FTF assessment for the PALPA subtest scores and the ALA scores are presented in Table 2. Agreement of the online and FTF assessment ranged from moderate for PALPA Picture Naming (k = 0.701, 95% CI 0.47–0.94) to almost perfect for the rest of the scores (k = 0.927–1.00).²⁴

Table 2.

Strength of Agreement Between Online and FTF SLPs for the PALPA Subtests and the ALA

ASSESSMENT	KAPPA (K)	SE
PALPA
Spoken word–picture matching	1.00	0.00
Spoken picture naming	0.701	0.12
ALA
Aphasia domain	1.00	0.00
Participation domain	0.927	0.07
Environment domain	1.00	0.00
Personal domain	1.00	0.00
Total	0.928	0.069

ALA, Assessment of Living with Aphasia; FTF, face-to-face; PALPA, Psycholinguistic Assessments of Language Processing in Aphasia; SE, standard error; SLP, speech-language pathologist.

Item by item analysis of the ALA revealed 99.9% agreement on all questions administered. The less than perfect agreement for the participation domain and ALA total score was due to the FTF SLP missing out on scoring one item (out of 675 items) due to reported environmental distraction.

Item by item analysis of the PALPA Picture Naming subtest revealed 98.7% agreement on all questions (592/600 items) administered. Closer inspection of the PALPA Picture Naming subtest also revealed occasional qualitative differences in the responses recorded by the FTF SLP and the online SLP, even when the same scores were assigned. For instance, for the item “eye,” the FTF SLP recorded the patient's response as “pen,” while this was recorded as “hand” by the online SLP.

Reliability analyses for the online-led and FTF conditions revealed comparable intrarater and interrater reliability (Table 3). Intrarater reliability was excellent for all scores across conditions. Interrater reliability for the Spoken Picture Naming subtest was marginally lower for the online-led group at ICC of 0.99 (0.95–1.00), but excellent for all the other scores.

Table 3.

Intrarater and Interrater Reliability Online Versus Remote Conditions

	INTRARATER RELIABILITY (ICC, 95% CL)		INTERRATER RELIABILITY (ICC, 95% CL)
ASSESSMENT	ONLINE	FTF	ONLINE	FTF
PALPA
Spoken word–picture matching	1.00 (1.00, 1.00)	1.00 (0.98–1.00)	1.00 (1.00, 1.00)	1.00 (0.99, 1.00)
Spoken picture naming	1.00 (0.99–1.00)	1.00 (0.99–1.00)	0.99 (0.95–1.00)	1.00 (1.00, 1.00)
ALA
Aphasia domain	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
Participation domain	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
Environment domain	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
Personal domain	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
Total	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)

ALA, Assessment of Living with Aphasia; CL, confidence level; FTF, face-to-face; ICC, Intraclass correlations; PALPA, Psycholinguistic Assessments of Language Processing in Aphasia.

All participants in the online-led group completed the satisfaction questionnaires. Overall satisfaction with the online assessment was high, with 100% of participants indicating that they were at least satisfied with the session, while 53.3% of the group was more than satisfied or very satisfied with the experience (Table 4). Audio and visual qualities were rated as good or excellent by 73.7% and 86.7% of participants, respectively. In addition, 86.7% of participants reported being comfortable or happy with the online assessment, would participate again, and rated the online process as potentially more convenient than FTF sessions. The entire cohort (100%) was confident with the results obtained, and 73.3% of participants felt they would be equally satisfied if the service was delivered online or FTF.

Table 4.

Participant Satisfaction with Online Assessment

QUESTIONS	QUESTIONNAIRE RESPONSES
	Not at all satisfied	Less than satisfied	Satisfied	More than satisfied	Very satisfied
Overall satisfaction (%)	0	0	46.7	20	33.3
	Poor	Inadequate	Adequate	Good	Excellent
Audio quality (%)	0	0	26.7	53.3	20
Visual quality (%)	0	0	13.3	46.7	40
	Disliked session	Uneasy	Neutral	Comfortable	Very happy
Level of comfort (%)	0	6.7	6.7	60	26.7
	Yes	No
Would participate again (%)	86.7	13.3
More convenient (%)	86.7	13.3
Confident with the results (%)	100	0
Equally satisfied if service provided online (%)	73.3	26.7

Satisfaction questionnaires were also completed by the four research SLPs who conducted the online assessments and results are presented in Table 5. All SLPs indicated being at least satisfied with Access2Aphasia, found it easy (50%) or very easy (50%) to use, and found the audio and visual quality to be adequate (25%) or good (75%). In addition, all SLPS indicated that they would incorporate apps like Access2Aphasia into their clinical practice given the opportunity, would participate again, agreed that it was more convenient than FTF sessions, and were confident with the results. The majority of them (75%) also felt that they would be equally satisfied with providing online services. Comments and suggestions were provided to improve app stability and ease of use for clinicians (e.g., through provision of a pointer to assist with explanations). Feedback was also given to include therapy and data analytics in the future development of Access2Aphasia.

Table 5.

Therapist Satisfaction with Online Assessment

QUESTIONS	QUESTIONNAIRE RESPONSES
	Not at all satisfied	Less than satisfied	Satisfied	More than satisfied	Very satisfied
Overall satisfaction (%)	0	0	75	25	0
	Poor	Inadequate	Adequate	Good	Excellent
Audio quality (%)	0	0	25	75	0
Visual quality (%)	0	0	25	75	0
	Not at all easy	Not so easy	Easy	Very easy	Extremely easy
Ease of use of Access2Aphasia™ (%)	0	0	50	50	0
		Yes	No
Given the opportunity, would include apps that enable remote assessment like Access2Aphasia in practice (%)		100	0
Would participate again (%)		100	0
More convenient (%)		100	0
Confident with the results (%)		100	0
Equally satisfied if service provided online (%)		75	25

Discussion

This study was designed to determine the reliability of assessing aphasia across two ICF domains using the Internet. Our findings revealed that it was possible to assess aphasia across the ICF spectrum in the online environment comparable to an FTF assessment. Consumer perspectives of telehealth aphasia assessment in the home environment using Access2Aphasia were also evaluated, with positive ratings provided by both PWA and SLPs.

In the online-led group, moderate to almost perfect agreement was found between online and FTF SLPs. Equivalent intrarater and interrater reliability was also found between the online-led and FTF conditions, providing further support for the use of Access2Aphasia in remote conditions for the online assessment of aphasia. While necessary for the purpose of this study, it is acknowledged that the presence of an FTF SLP in the online-led condition is not typical of a telehealth scenario and further research using Access2Aphasia for aphasia assessment is required.

An important aspect of aphasia assessment that was incorporated into Access2Aphasia was the inclusion of the ALA. The assessment comprised quality of life-related questions discussed within a conversational context and was facilitated by the features of Access2Aphasia, as research SLPs could support conversation through using writing or drawing on the collaborative whiteboard, providing gestural and facial cues through videoconferencing, and referring to pictographs simultaneously. This enabled PWA to self-report on domains related to their aphasia (aphasia, participation, and environmental and personal domains) using a combination of rating scales and qualitative responses. The high level of exact agreement (99.9%) found between the online and FTF SLPs suggests that aphasia assessment beyond the impairment level can confidently occur in the online environment as in an FTF context. In addition, since supported conversation techniques⁹ play an important part in enabling communication with PWA and are frequently used in FTF aphasia therapy, Access2Aphasia potentially provides a platform for SLPs to conduct therapy in online conditions. Additional research using Access2Aphasia to implement aphasia therapy sessions online is required to support this.

While kappa calculations indicated moderate agreement on the PALPA Picture Naming subtest in the online-led group, this estimate of agreement may have been lowered excessively due to the assumptions the Kappa makes about rater independence.²⁴ In our study where the SLPs are well trained and little guessing was expected, percent agreement (98.7%) may be accepted to determine interrater agreement between online and FTF SLPs.²⁴ Nevertheless, this was lower compared to the rest of the scores, and item by item analyses revealed qualitative differences between online and FTF SLPs. This was attributed to varying Internet bandwidths across locations resulting in inconsistent visual and audio quality and is consistent with difficulties reported in previous research.^4,5 To address this problem, audio recording of the PWA's responses, which can be played back on a Web portal, will be enabled on later versions of the app.

PWA satisfaction with the online assessment was high in terms of their level of satisfaction and comfort, audio and visual quality, and choice of online assessment as an alternative to FTF assessment. These positive ratings are consistent with previously published research in telehealth aphasia assessment.⁵ It was particularly encouraging that all the participants were confident with the results obtained, and most of them (86.7%) reported being comfortable with the online assessment, would participate again, and rated the online process as potentially more convenient than FTF sessions. These responses provide support for the usability of Access2Aphasia, although caution should be taken that these high levels may be partially due to the novelty factor of online assessments. In addition, slightly fewer participants (73.3%) reported that they would be equally satisfied if the service was delivered online or FTF. One PWA indicated that while online sessions were more convenient, it should not replace FTF sessions entirely. Therefore, while telehealth has its benefits, the appropriateness of its application at different stages of the rehabilitation process and client needs need to be considered. Perhaps online sessions need to be interspersed with FTF sessions to support rapport building and maintain the therapeutic relationship.^25,26 Further research into what influences patient acceptance of aphasia assessment using telehealth is indicated,²⁷ although this may change over time with an increasing adoption of mobile technology in health, retail, and government services.²⁸

The usability of Access2Aphasia was also supported by feedback from SLP users. While overall positive ratings of the app were provided, feedback was also obtained to facilitate improvement of the app. For instance, app stability, that is, the ability of the app to progress seamlessly from one question to the next, was cited as an area that needed improvement as this resulted in disruption of the assessment at times. Negative experiences of technology use such as these have been found to be a potential barrier to telehealth implementation by healthcare workers.²⁹ In order for online aphasia assessment to be comparable with FTF assessments and suitable for clinical utility, improvements have been made to the app to improve its reliability. In addition, ease of using the app may be improved by incorporation of data analytics. For example, enabling assessment scores to be charted across time may help in tracking of outcomes. Further iterations of the app are underway and will continue with usage by more PWA and SLPs to address these usability issues and potential barriers to telehealth adoption.²⁹

Our study suggests that it is possible to reliably evaluate aphasia online using Access2Aphasia. PWA, particularly severe aphasia, have often been systematically excluded from research and interventions.³⁰ In this study, PWA of various severities were able to use Access2Aphasia successfully, with varying degrees of support by the online SLP. During the study, no training was provided, and participants were left to use the app on their own. However, it is possible that other PWA, particularly those with concomitant cognitive deficits, may require more training to use the app independently. They may also require the presence of a caregiver to assist them. However, this may not differ greatly from the need of a caregiver being present in FTF sessions.

Conclusion

In summary, this study has provided preliminary evidence for the reliability of assessing aphasia across the ICF online through an iPad-based application, Access2Aphasia. While consumer satisfaction was high, progression of telehealth services from “trial” status to routine health service must be supported by an improved research into patients' satisfaction with telehealth, as well as rectification of usability issues. Further research is warranted with larger sample sizes of PWA and SLPs to advance the development of iPad-based aphasia assessment and treatment as an alternative mode of service delivery to PWA.

Footnotes

Acknowledgments

This article is based on a doctoral research being undertaken at The University of Sydney by the first author under the supervision of the second and third authors. The first author was supported by the Academic Medicine Development Award from the National University Hospital, Singapore. This study was supported by the Saw Swee Hock School of Public Health Telehealth Innovation Research Programme (THOR) Fund. Permission was obtained from the publishers of the Psycholinguistic Assessment of Language Processing Activities (PALPA) and the Assessment of Living with Aphasia (ALA) to include the assessments on the tablet application for research purposes and for reproduction of pictures for use in this article. We thank the participants with aphasia and research speech pathologists for their time and input in this project.

Disclosure Statement

No competing financial interests exist.

Appendix 1. Participant Satisfaction with Online Assessment-PWA

Adapted from Theodoros et al. (2008)

Please select “Yes/No” for the following statements

Appendix 2. Access2Aphasia ™ App User Satisfaction Survey

References

Theodoros

. A new era in speech-language pathology practice: Innovation and diversification. Int J Speech-Language Path, 2012; 14:189–199.

Guo

, Togher

, Power

. Speech pathology services for people with aphasia: What is the current practice in Singapore?. Disabil Rehabil, 2014; 36:691–704.

Palsbo

. Equivalence of functional communication assessment in speech pathology using videoconferencing. J Telemed Telecare, 2007; 13:40–43.

Hill

, Theodoros

, Russell

, Ward

, Wootton

. The effects of aphasia severity on the ability to assess language disorders via telerehabilitation. Aphasiology, 2009; 23:627–642.

Theodoros

, Hill

, Russell

, Ward

, Wootton

. Assessing acquired language disorders in adults via the Internet. Telemed J E Health, 2008; 14:552–559.

Georgeadis

, Brennan

, Barker

, Baron

. Telerehabilitation and its effect on story retelling by adults with neurogenic communication disorders. Aphasiology, 2004; 18:639–652.

Hall

, Boisvert

, Steele

. Telepractice in the assessment and treatment of individuals with aphasia: A systematic review. Int J Telerehabil, 2013; 5:27–38.

World Health Organization.. International classification for functioning, disability and health (ICF). Geneva, Switzerland: World Health Organization, 2001.

Kagan

. Supported conversation for adults with aphasia: Methods and resources for training conversation partners. Aphasiology, 1998; 12:816–830.

10.

Kiran

, Roches

, Balachandran

, Ascenso

. Development of an impairment-based individualized treatment workflow using an iPad-based software platform. Semin Speech Lang, 2014; 35:38–50.

11.

Choi

Y-H

, Park

, Ahn

K-H

, Son

Y-J

, Paik

N-J

. A telescreening tool to detect aphasia in patients with stroke. Telemed J E Health, 2015; 21:729–734.

12.

Enderby

, Wood

, Wade

, Hewer

. The Frenchay Aphasia Screening Test: A short, simple test for aphasia appropriate for non-specialists. Int Rehabil Med, 1987; 8:166–170.

13.

Kertesz

. Western aphasia battery—Revised. San Antonio, TX: Pearson, 2006.

14.

Guo

, Togher

, Power

, Hutomo

, Tay

, Yen

S-C

, Koh

. Telehealth aphasia assessment in Singapore: Development of an iPad-based application enabling supported conversation techniques. In Aphasia Symposium of Australia. Perth, Australia, 2014.

15.

Kagan

, Simmons-Mackie

, Rowland

, Huijbregts

, Shumway

, McEwen

, Threats

, Sharp

. Counting what counts: A framework for capturing real-life outcomes of aphasia intervention. Aphasiology, 2007; 22:258–280.

16.

Lim

VPC

, Tay

DDY

, Douglas

. Singapore aphasia test. Singapore: Singhealth Publishing, 2009.

17.

Goodglass

, Kaplan

, Barresi

. Boston Diagnostic Aphasia Examination—Third Edition (BDAE–3). Baltimore, MD: Lippincott Williams & Wilkins, 2001.

18.

Kagan

, Simmons-Mackie

, Victor

, Carling-Rowland

, Hoch

, Huijbregts

, et al. Assessment for living with aphasia (ALA), 2nd ed. Toronto, Canada: Aphasia Institute, 2013.

19.

Kagan

, Simmons-Mackie

, Victor

, Carling-Rowland

, Hoch

, Huijbregts

, Streiner

, Mok

. Assessment for living with aphasia (ALA). Toronto, Canada: Aphasia Institute, 2010.

20.

Kay

, Lesser

, Coltheart

. PALPA: Psycholinguistic assessments of language processing in aphasia. Hove: Psychology Press, 1992.

21.

Guo

, Togher

, Power

. Speech pathology services for people with aphasia: What is the current practice in Singapore?. Disabil Rehabil, 2014; 36:691–704.

22.

Rose

, Ferguson

, Power

, Togher

, Worrall

. Aphasia rehabilitation in Australia: Current practices, challenges and future directions. Int J Speech Lang Pathol, 2014; 16:169–180.

23.

Bate

, Kay

, Code

, Haslam

, Hallowell

. Eighteen years on: What next for the PALPA?. Int J Speech Lang Pathol, 2010; 12:190–202.

24.

McHugh

. Interrater reliability: The kappa statistic. Biochem Med, 2012; 22:276–282.

25.

Brewster

, Mountain

, Wessels

, Kelly

, Hawley

. Factors affecting front line staff acceptance of telehealth technologies: A mixed-method systematic review. J Adv Nurs, 2014; 70:21–33.

26.

Walsh

. Small talk is “big talk” in clinical discourse: Appreciating the value of conversation in SLP clinical interactions. Top Lang Disord, 2007; 27:24–36.

27.

Williams

, May

, Esmail

. Limitations of patient satisfaction studies in telehealthcare: A systematic review of the literature. Telemed J E Health, 2001; 7:293–316.

28.

Zarmpou

, Saprikis

, Markos

. Vlachopoulou, M. Modeling users' acceptance of mobile services. Elect Commerce Res, 2012; 12:225–248.

29.

Taylor

, Coates

, Brewster

, Mountain

, Wessels

, Hawley

. Examining the use of telehealth in community nursing: Identifying the factors affecting frontline staff acceptance and telehealth adoption. J Adv Nursing, 2015; 71:326–337.

30.

Pound

. Reciprocity, resources, and relationships: New discourses in healthcare, personal, and social relationships. Int J Speech Lang Pathol, 2011; 13:197–206.