A Telescreening Tool to Detect Aphasia in Patients with Stroke

Abstract

Background: Early identification of patients with stroke-induced aphasia is essential because it is a significant disability affecting daily life and is linked to poor functional outcome after stroke. However, most patients with stroke are unable to undergo aphasia evaluation and detection and therefore remain undiagnosed. The purpose of this study is to develop a valid, reliable mobile aphasia screening test (MAST) for patients in remote locations. Materials and Methods: To accomplish this, we enrolled patients with (n=30) and without (n=30) stroke-induced aphasia. A MAST, which adopted the Korean version of the shortened version of the Frenchay Aphasia Screening Test (K-FAST), was designed as an iPad^® (Apple, Cupertino, CA) application. To validate the MAST, we compared its performance with that of the Korean version of the Western Aphasia Battery (K-WAB) and conventional shortened FAST paper version (K-FAST). We analyzed interrater and internal reliability, using Cronbach's alpha coefficient, and assessed the diagnostic sensitivity, specificity, and power. Results: There was significant correlation between K-FAST and MAST (intraclass correlation coefficient [ICC]=0.995, p<0.001). MAST also had a high correlation with K-WAB (ICC=0.752, p<0.001). Interrater reliability was very high (ICC=0.999, p<0.001). The test had high sensitivity (90.0%) and specificity (73.3%) with an accuracy of 0.930 (95% confidence interval=0.853–1.000). The MAST is a valid and reliable tool for detecting aphasia in patients with stroke. Conclusions: This telescreening test may overcome the limitations of test administration and may be a convenient and cost-effective alternative to the existing aphasia screening tests for patients with stroke.

Introduction

About 24–30% of patients experience various types of aphasia (i.e., global, Broca's, Wernicke's, transcortical motor/sensory, conduction, anomic aphasia) following stroke, and the degree of recovery varies among patients.^1,2 Aphasia is not only an important disability affecting daily life but also a key prognostic factor for functional outcome after stroke.³ Treatment of aphasia is effective in promoting the recovery of functional communication, receptive language, and expressive language.⁴ Therefore, identifying a candidate for treatment of aphasia is a requisite.

Many aphasia assessment tools have been introduced to date. However, most of them involve long testing times, and their administration requires a speech–language therapist. Moreover, many screening tools need to be administered face to face by a health-related professional, if not by a specialized speech–language therapist. Patients who have had a stroke usually have motor impairments due to weakness, imbalance, or poor general condition and have difficulty visiting a clinic or an institute because of their disabilities. Many patients who live in rural or urban locations with inadequate transportation cannot be identified and assessed and therefore remain undiagnosed. To circumvent these obstacles, a screening tool that can be used by nonspecialists, such as caregivers, and administered anywhere is required. Therefore, a readily available, simple, and quick screening tool is necessary to identify patients with potential language problems. As a result, there has been an increasing demand for information and communication technologies in the field of speech and language rehabilitation.⁵

At present, the Frenchay Aphasia Screening Test (FAST) is the most widely used, and thoroughly evaluated screening test for aphasia after stroke.^6,7 It is brief and easy to administer and has good reliability in identifying patients with language problems, even when used by a nonspecialist.⁷ It is suitable for use by general practitioners, junior medical staff, and other nonspecialists.^8,9 The FAST is made up of four subscales: comprehension, verbal expression, reading, and writing. To reduce administration time, the shortened version of FAST, with only the comprehension and expression sections, can be administered. The classification sensitivity of the shortened FAST is reported to be similar to that of the complete version.⁸ A significant inverse relationship between FAST score and age has been reported.⁹ The stratified cutoffs and normative data are available for both the complete and shortened version of the FAST for two age groups (≤64 years and ≥65 years).¹⁰ To administer the FAST, a double-sided stimulus card with attached reading cards, pencil, paper, and a stopwatch are required. Thus, developing a mobile version of the FAST is the need of the hour; a mobile version would not require these tools, and the test could be administered anywhere and anytime without time and space restraints.

We developed a mobile aphasia screening test (MAST) for patients with stroke, with an emphasis on cost-effectiveness, portability, and ease of use. We accomplished this by modifying the Korean language version of the FAST (K-FAST) as a mobile version. The items of the MAST are identical to those of the shortened version of K-FAST. In this present study, we report the validity and reliability of the MAST.

Materials and Methods

Participants

Our Institutional Review Board approved this study (approval number B-1206/160-005). We obtained informed consent from each participant, and we followed the procedures in accordance with institutional guidelines. We recruited stroke patients with (n=30; test group) and without (n=30; control group) aphasia as diagnosed and confirmed by a trained physiatrist. Patients with stroke without aphasia were age-matched with those with aphasia.

The presence of visual field defects, visual neglect or inattention, illiteracy, deafness, poor concentration, or confusion negatively affects the specificity of FAST.^8,11,12 Therefore, we excluded patients with such issues in this study.

Telescreening System Design

The MAST uses a client-server model in which all data are stored in a central server. The iPad^® (Apple, Cupertino, CA) mobile application was designed and implemented in order to improve the ease of use of the model. We developed a Web portal for service providers, such as therapists and assessors, with accessibility to and management of the data. The client program and server system communicated over the network with Hypertext Transfer Protocol Secure in order to ensure secure communication. The central server consisted of several layers of supportive servers in order to support efficient development of the mobile and Web applications. The supportive servers are as follows: the open application programming interface, which acts as a gateway between the mobile application and the central storage; the controller layer, which provides a Web service for providers; the data access layer, which provides queries to access the underlying database; and the database server in the persistence layer. We implemented all the server programs on a Java platform (Fig. 1).

Fig. 1.

System architecture of the telescreening system for aphasia in stroke patients. API, application programming interface; HTTPS, Hypertext Transfer Protocol Secure.

We designed the iPad mobile application, which includes a MAST, to be user-friendly and easy to use by using big letters, by having minimum user options, and by providing step-by-step voice-guided directions. Patients who had no experience using any mobile device were educated to answer through their finger movement on the touch screen. The patients' records, such as voice recording and the touch pattern on each item, were then automatically sent to, and stored in, the central database through the open application programming interface (Fig. 2).

Fig. 2.

iPad application for the aphasia screening test and the Web portal for scoring.

Test Administration

A research assistant (RA1), who is not a speech–language therapist, administered the MAST. A well-trained speech–language therapist (ST1) scored the patient's response on a K-FAST tool. To assess the comprehension function, we asked the subjects to respond by choosing the appropriate item from among the many displayed on the screen, using their fingers. This test had 10 questions with the pictures of river scenes and shapes. The system automatically scored the responses, and the scores were saved in a Web portal. To evaluate their verbal expression, we asked the subjects to describe the picture of a river scene and to name as many animals as possible, in 1 min. Two assessors (a well-trained speech–language therapist [ST2] and a research assistant [RA2]) scored the verbal expression section by listening to the patient's recording stored in the Web portal. A total combined score of the K-FAST and MAST is 20. In addition, the Korean version of the Western Aphasia Battery (K-WAB) was administered to patients with aphasia to obtain the receiver operating characteristics curve. Our team chose the K-WAB because it is a validated method that is accepted as the gold standard and is capable of reliably discriminating between individuals with or without aphasia.

Statistical Analysis

We assessed the concordance of the K-FAST and MAST by calculating the intraclass correlation coefficient (ICC) from the score of each item and the total scores. We determined the correlation between the MAST and the K-WAB by Pearson's correlation coefficient. Only patients with a diagnosis of aphasia were evaluated by the K-WAB.

Interrater reliability was determined using the Pearson correlation coefficient and ICC, to evaluate the consistency of measurements on the same patient by different raters and to determine the extent to which the raters are interchangeable. To evaluate external validity, we assessed sensitivity and specificity using cutoff values from Pyun et al.¹⁰ The diagnostic accuracy power was analyzed through the receiver operating characteristics curve, and the area under the curve with 95% confidence limit was measured.

We calculated Cronbach's alpha coefficient, a measure of reliability based on internal correlation of the items on the scale. In addition, we calculated the correlations between the score obtained on individual items (item–item correlations) and between the scores obtained on each individual item and the total score (item–total correlations).

Results

We enrolled 30 patients with stroke and aphasia (25 males and 5 females; mean age, 53.67±15.27 years [range, 21–77 years]) for this study. They had had either an ischemic (n=21) or a hemorrhagic (n=9) stroke. The lesion location was either to the left (n=26) or bilateral (n=4). The onset-to-assessment intervals for the patients with stroke and aphasia ranged from 2 to 2,736 days, with a mean of 413.7 days. The average Aphasia Quotient (AQ) of the K-WAB was 39.5±24.93% (range, 3–97%). We also recruited 30 stroke patients without aphasia (22 males and 8 females; mean age, 53.90±14.54 years [range, 21–79 years]) for the control group. They had had either an ischemic (n=20) or a hemorrhagic (n=10) stroke. The lesion location was on the left (n=11), on the right (n=18), or bilateral (n=1). The onset-to-assessment intervals for the patients without aphasia ranged from 3 to 8,128 days, with a mean of 925.3 days. The age (t=0.765), sex (t=0.064), type of stroke (t=0.204), and stroke location (t=0.794) were comparable between the two groups.

Equivalence of the K-FAST and MAST

The comparison of the K-FAST and MAST in the sample of 60 patients with stroke showed that the two versions were strictly equivalent with an ICC total score of 0.995 (p<0.001). The ICC scores for each comprehensive function assessed were 0.956 (p<0.001) for the river scene and 0.980 (p<0.001) for shapes. Likewise, tests on verbal expression had an ICC score of 0.997 (p<0.001) for the “Description test” and 1.000 for the “Name test.” Not one of the patients diagnosed as “aphasic” in one test was “nonaphasic” in the other test.

Concurrent Validity Assessed by Comparing the Scores With the K-WAB

The correlation between the total score of the MAST and the AQ of the K-WAB was 0.752 (p<0.001) (Table 1). Given that the correlation between the total score of the K-FAST and the AQ of the K-WAB was 0.737 (p<0.001), the MAST could measure most of the language function assessed by the K-WAB. With regard to individual items (river scene, comprehension and description of shapes, fluency, and naming of names), the item–item correlation score from the MAST was also similar to the item–item correlation between the scores from the corresponding section of the K-FAST and K-WAB tests. Therefore, MAST could predict the result from the K-WAB as well as K-FAST.

Table 1.

Concurrent Validity of the Mobile Aphasia Screening Test with the Korean Version of the Western Aphasia Battery (n=30)

	TOTAL SCORE	RIVER SCENE	SHAPES	DESCRIBE	NAME
AQ	0.752^a	0.782^a	0.546^b	0.610^a	0.641^a
Auditory comprehension	0.477^b	0.645^a	0.232^d	0.403^c	0.358^c
Fluency	0.648^a	0.725^a	0.421^c	0.514^b	0.571^a
Naming	0.649^a	0.673^a	0.394^c	0.616^a	0.521^b

Pearson's correlation coefficients are shown.

p<0.001, ^b p<0.01, ^c p<0.1, ^dno statistical significance

AQ, Aphasia Quotient.

Interrater Reliability

Interrater reliability of the MAST for all 60 of the stroke patients was near perfect: total score, 11.68±5.95 versus 11.62±5.90 (r=0.997, p<0.001) and ICC=0.999 (p<0.001) (Table 2).

Table 2.

Interrater Reliability

	COMPREHENSION		VERBAL EXPRESSION
	RIVER SCENE	SHAPES	DESCRIBE	NAME	TOTAL SCORE
r	0.977^a	0.997^a	0.997^a	0.984^a	0.997^a
ICC	0.988^a	0.998^a	0.999^a	0.992^a	0.999^a

p<0.001.

ICC, intraclass correlation coefficient; r, Pearson correlation coefficient.

Diagnostic Sensitivity and Specificity

Taking the diagnosis by an experienced physiatrist as the gold standard and using the cutoff points for K-FAST from Pyun et al.¹⁰ (16 for ≤64 years and 14 for ≥65 years), the sensitivity (detection rate of abnormal values in aphasic patients) of both MAST and K-FAST was 90.0%. The specificity of the test was 73.3% for MAST and 86.7% for K-FAST. In the aphasic group, the number of patients who had abnormal/normal values was the same for MAST and K-FAST (27 versus 3), and in the nonaphasic group, the distribution was 8 versus 22 and 4 versus 26 for MAST and K-FAST, respectively. We evaluated the sensitivity and specificity of detecting aphasic patients by means of a receiver operating characteristics curve analysis (Fig. 3). The area under the curve with 95% confidence limit was 0.930 (0.853–1.000) for MAST. Compared with K-FAST, the diagnostic accuracy was not inferior.

Fig. 3.

Receiver operating characteristics (ROC) curve showing the sensitivity (true-positive rate) and 1 – specificity (false-positive rate) of the mobile aphasia screening test (mobile version of the shortened Korean version of the Frenchay Aphasia Screening Test [sK-FAST]) versus the conventional sK-FAST.

Internal Reliability

The internal consistency of the 15 items in MAST was good with a Cronbach alpha of 0.88 (p<0.001). Table 3 shows the correlation between the scores obtained on each individual item (item–item correlations) and between the scores obtained on each individual item and the total score (item–total correlations).

Table 3.

Item–Item Correlations and Item–Total Correlations

	RIVER SCENE	SHAPES	DESCRIBE	NAME
River scene
Shapes	0.602^a
Describe	0.537^b	0.662^a
Name	0.674^a	0.809^a	0.603^a
Total score	0.812^a	0.890^a	0.839^a	0.888^a

p<0.001, ^b p<0.01.

Discussion

We have developed and validated a mobile language screening test for patients with stroke. The MAST has good internal validity, correlates well with the gold standard K-WAB, shows very high interrater reliability, and is quick to complete. We used the K-WAB to examine the validity of assessments because many clinicians consider it the gold standard for assessment. The present study showed that the MAST could predict the result from the K-WAB as much as from the K-FAST. It is important that even a nonspecialist can administer the MAST. It showed excellent sensitivity and specificity for aphasia, thus identifying patients warranting a thorough evaluation and treatment with a speech–language therapist.

The use of a telescreening test can solve the problems faced by remotely situated patients, such as transportation, time, cost, caregiver availability, and stroke impairment. Users only need to download the application on a mobile device and use it briefly with ease. The conventional FAST requires a double-sided stimulus card with attached reading cards, pencil, paper, and a stopwatch or a watch with a second's hand. Therefore, it is usually available in a clinical setting only. However, the mobile language screening test does not require any special equipment or stimulus cards and is available anywhere. We found that administration of the MAST was feasible and applicable to stroke patients. All participants easily adapted in using this iPad application regardless of age, after a brief introduction. Although most subjects had no experience in using a mobile device before, no one complained of difficulty in using it.

The demands of identification of candidate patients for aphasia treatment could be mitigated by using information and communication technologies. Use of mobile device–based decision-making in screening, diagnosis, and the planning of treatment is rare. Furthermore, the quality of the mobile device–based or PC-based applications that are currently being developed remains poor.⁵ The present study included an innovative approach for developing a ubiquitous tool for an aphasia screening test along with the performance of a validation and reliability analysis.

Previous studies have evaluated the efficacy and feasibility of aphasia assessment methods only in a single subject or only in aphasia groups.^{13
15} The present study is the first to report evidence showing that a telehealth service using a mobile device can be used to differentiate between patients with aphasia who are candidates for intensive speech therapy and patients without definite language problems.

Although the correlation between the conventional test and the mobile one was quite high, it was not 100%. The disagreement between the two tests could be due to the following reasons. The iPad used a resistive screen with a touch input. Therefore, in some cases when the subjects touched the device twice instead of once, or when our subjects responded early before the item–tasks presentation was completed by the system, the responses were considered wrong. In addition, the device was not sensitive enough to recognize subjects' finger touch on the screen. Therefore, even though our subjects responded accurately, the application perceived it as a wrong response. The automatic scoring system in the program workflow did not allow the subjects to change their response. In order for the MAST to be on par with the conventional test to be perfect for clinical utility, these usability errors must be rectified.

Conclusions

We propose a novel and validated approach to screen patients with stroke for aphasia. It is a brief, efficient, user-friendly, and cost-effective alternative for the already available aphasia screening tests. This mobile screening test can be used ubiquitously to identify stroke patients with aphasia in remote locations.

Footnotes

Acknowledgments

The KT-SNUBH Collaborative Research Fund supported this work.

Disclosure Statement

No competing financial interests exist.

References

Engelter

, Gostynski

, Papa

, et al. Epidemiology of aphasia attributable to first ischemic stroke: Incidence, severity, fluency, etiology, and thrombolysis. Stroke, 2006; 37:1379–1384.

Laska

, Hellblom

, Murray

, et al. Aphasia in acute stroke and relation to outcome. J Intern Med, 2001; 249:413–422.

Gialanella

. Aphasia assessment and functional outcome prediction in patients with aphasia after stroke. J Neurol, 2011; 258:343–349.

Brady

, Kelly

, Godwin

, Enderby

. Speech and language therapy for aphasia following stroke. Cochrane Database Syst Rev, 2012; 5:CD000425.

Bates

, Gawande

. Improving safety with information technology. N Engl J Med, 2003; 348:2526–2534.

Salter

, Jutai

, Foley

, et al. Identification of aphasia post stroke: A review of screening assessment tools. Brain Injury, 2006; 20:559–568.

J-W

, Pyun

S-B

, Lee

, et al. Reliability and validity analyses of the Korean version of Frenchay Aphasia Screening Test in brain-damaged patients. Korean J Commun Disord, 2009; 14:46–57.

Enderby

, Wood

, Wade

, Hewer

. The Frenchay Aphasia Screening Test: A short, simple test for aphasia appropriate for non-specialists. Int Rehabil Med, 1987; 8:166–170.

O'Neill

, Cheadle

, Wyatt

, et al. The value of the Frenchay Aphasia Screening Test in screening for dysphasia: Better than the clinician?. Clin Rehabil, 1990; 4:123–128.

10.

Pyun

, Hwang

, Ha

, et al. Standardization of Korean version of Frenchay Aphasia Screening Test in normal adults. J Korean Acad Rehabil Med, 2009; 33:436–440.

11.

Al-Khawaja

, Wade

, Collin

. Bedside screening for aphasia: A comparison of two methods. J Neurol, 1996; 243:201–204.

12.

Gibson

, MacLennan

, Gray

, Pentland

. Evaluation of a comprehensive assessment battery for stroke patients. Int J Rehabil Res, 1991; 14:93–100.

13.

Theodoros

, Hill

, Russell

, et al. Assessing acquired language disorders in adults via the Internet. Telemed J E Health., 2008; 14:552–559.

14.

Hill

, Theodoros

, Russell

, et al. The effects of aphasia severity on the ability to assess language disorders via telerehabilitation. Aphasiology, 2009; 23:627–642.

15.

Palsbo

. Equivalence of functional communication assessment in speech pathology using videoconferencing. J Telemed Telecare, 2007; 13:40–43.