Reliability,feasibility and satisfaction of telemedicine evaluations for cervical dystonia

Abstract

Introduction

Telemedicine is used successfully for evaluating patients with neurologic diseases, but has not been tested in cervical dystonia (CD). CD is uniquely suited for telemedicine as the scales validated to assess its severity rely only on visual inspection. The study sought to determine reliability, feasibility and satisfaction of telemedicine visits for evaluating CD.

Methods

Patients 18 years and older with a diagnosis of CD and scheduled for botulinum toxin (BoNT) injections were recruited, with a total of 46 enrolled. Dystonia severity was evaluated using the Toronto Western Spasmodic Torticollis Rating Scale (TWSTRS) motor severity subscale. Three total evaluations took place: an initial telemedicine evaluation on the day prior to a scheduled BoNT injection; an in-person evaluation in clinic immediately before injections; and a follow-up telemedicine visit 4–6 weeks after injection with subsequent completion, by both participants and the clinician, of satisfaction questionnaires. Agreement between telemedicine and in-person TWSTRS data was calculated using intra-class correlation coefficients (ICC) and kappa statistics where appropriate. Feasibility was determined by the percent of patients completing all three visits, and satisfaction with telemedicine visits was determined based on answers to satisfaction questionnaires.

Results

There was excellent agreement between visit types for the TWSTRS motor severity summary score (κ = 0.890; 95th CI 0.713; 0.949). Only two individual TWSTRS items failed to meet the threshold for moderate agreement. Feasibility and satisfaction were high.

Discussion

Telemedicine is reliable and feasible in the evaluation of CD. Some CD patients would prefer telemedicine visits. Participants and the clinician were satisfied with telemedicine visits.

Keywords

Telemedicine cervical dystonia botulinum toxin

Introduction

Cervical dystonia (CD) is defined by abnormal, sustained involuntary contraction of the muscles of the head, neck and shoulder(s). It is characterized by abnormal postures of the involved muscles, and is often accompanied by tremor.¹ The gold standard treatment for CD is injection of botulinum toxin (BoNT) into the affected muscles. Despite its efficacy, many patients who receive BoNT are lost to follow up.²

BoNT injections are typically given at 12-week dosing intervals. Clinical evaluations at these injection visits rely heavily on patient recollection of their responses to treatment and are likely unreliable. Adjustments to dosing and injection scheme based on such reports are difficult, and may contribute to limited patient satisfaction.^3,4 We hypothesize that more frequent assessments, particularly when the medication is at peak efficacy, would be helpful for clinicians and might also improve patient satisfaction with BoNT therapy. One way to facilitate these assessments is by using telemedicine – a tool that has already demonstrated efficacy in patients with neurologic disease.^5–8 Remote patient evaluation holds the potential to allow patients with CD and their treating physicians to conveniently discuss symptoms and response to treatment, and also to discuss potential alterations to injection schemes and schedules.⁹

We conducted a prospective study investigating whether telemedicine evaluation of motor symptom severity in a cohort of subjects with CD is reliable, feasible and satisfactory for both subjects and clinicians. These issues have not been studied in CD, and this disorder is uniquely suited for web-based visits as the scales validated to assess motor severity rely exclusively on visual examination.¹⁰

Methods

The study was submitted to, and approved by the Rush University Medical Center Institutional Review Board. Consecutive patients with diagnoses of ‘cervical dystonia’ and/or ‘spasmodic torticollis’, who were scheduled in the Rush University Movement Disorders Center in Chicago, Illinois for BoNT injections, were identified in the electronic medical record and recruited for participation between June 1, 2017 and February 1, 2018. Inclusion criteria included: at least 18 years old, fluency in English, possession of a compatible computer or mobile device and access to high speed Internet. Exclusion criteria included: suspected tardive and other secondary dystonic symptoms.

An initial telemedicine visit was scheduled within 24 h prior to the participants’ next scheduled BoNT injection visit. In preparation for this computer-based visit, participants completed a standardised ‘telemedicine training’, consisting of verification of demographic data and testing of computer or mobile devices for web-camera and microphone compatibility with the required software. Participants could not log into the telemedicine visit before this training was completed successfully. The rating clinician (A.F.) was notified via electronic staff message once the participant had logged in, allowing initiation of the visit. Once the telemedicine connection had been established, A.F. assessed symptom severity via the Toronto Western Spasmodic Torticollis Rating Scale (TWSTRS) motor severity subscale, and stored this data in the electronic medical record.¹¹ This numeric scale measures the severity of the different postures seen in CD as well as their duration and associated symptoms, with higher scores indicating increased severity. The order of symptom evaluation was standardised for every participant as follows: rotation direction and severity, laterocollis direction and severity, presence of anterocollis or retrocollis, anterocollis or retrocollis severity, presence/absence and direction of lateral shift, presence or absence of sagittal shift, duration of symptoms, effect of sensory trick, presence or absence of shoulder elevation and/or anterior displacement, severity of shoulder elevation and/or anterior displacement, range of motion without use of a sensory trick, and duration of time during which the participant could keep his/her head in the neutral position without use of a sensory trick. The individual item scores are then added together for a total motor severity subscore.

The next day participants were seen in-person in the movement disorders clinic before undergoing their scheduled BoNT injections. The TWSTRS motor severity subscore was again rated by A.F. using the same examination protocol as above and stored in the medical record. After this in-person assessment, participants received their BoNT injections according to usual practice.

A second computer-based visit was scheduled 4–6 weeks after the injections, so as to re-connect with participants when the medication would be at or near peak efficacy. Participants were again required to complete identical telemedicine training prior to this visit. During this visit, the participants were rated again by A.F. using the TWSTRS motor severity subscale via the standardised protocol mentioned previously. After completion of this second telemedicine visit, the participant and the rater (A.F.) completed satisfaction questionnaires with specific questions about their telemedicine experiences. The questions asked of the participants and the rater (A.F.) related to domains of perceived security of the telemedicine visits, ease of setting up the visits, quality of care of the visits and relative satisfaction compared with in-person visits. Answers were given on a Likert scale of 1–10, with scores of 1 indicating absolute disagreement and scores of 10 indicating complete agreement (see Appendix 1 for questionnaires; see Figure 1 for study visit schedule).

Figure 1.

Diagram of study visit schedule.

Sample size estimates for the intra-class correlation coefficient (ICC) were calculated using the ICC sample size as previously described by Zou.¹² There is excellent rater agreement within the TWSTRS motor severity subscale as previously demonstrated (ICC = 0.75).¹¹ For this study, a sample size of 50 subjects was targeted to afford sufficient power (1– β ≥0.80) to detect an ICC as low as 0.60 given a two-tailed significance level (alpha) of 0.05. This level of intra-rater agreement is considered in the range between good (ICC ≥0.60) and excellent (ICC ≥0.75).¹³

Reliability testing

Intra-rater agreement between the telemedicine and in-person ratings was assessed via two different statistical approaches. The weighted kappa statistic was used to evaluate agreement of the categorically scaled individual TWSTRS items.¹⁴ Reliability of individual item scores was considered acceptable if the weighted kappa value exceeded the score for moderate agreement (≥0.41).¹³ Agreement for the TWSTRS motor severity summary subscore was assessed via intra-class correlation coefficient (ICC, two-way mixed effect model, absolute agreement of measure).¹⁵ Intra-rater agreement for the TWSTRS summary score was considered acceptable if ICC values exceeded the ‘good’ score of ≥0.60.¹³

Feasibility testing

Feasibility was defined in two ways. The first was by calculation of the percentage of scheduled participants who successfully completed the ‘telemedicine training’. Successful completion was defined as obtaining the ability to log into the telemedicine visit and make a video connection with the rating clinician (A.F.). The second was by calculating the percentage of participants who completed all study visits – the first telemedicine visit, the in-person visit, and the second telemedicine visit – within an allotted timeframe.

Satisfaction assessments

Answers to the satisfaction questionnaires were tallied to determine participant and clinician (A.F.) satisfaction. As responses were given on a 1–10 Likert scale, scores six and above were considered affirmative responses and scores four and below were considered negative responses. Scores of five were considered neutral.

Results

The 143 patients (98 female and 45 male) who met the inclusion criteria were contacted for potential study enrollment. Of these, 49 participants consented to participate in the study, but only 46 completed an initial telemedicine visit. The mean TWSTRS motor severity summary subscore for participants before the initial telemedicine visit was 14.77 ± 4.130. The mean age of participants was 57.57 ± 11.99; 30 participants (65.2%) were female (see Table 1).

Table 1.

Demographics of recruited subjects.

	Subjects (n = 46)	Non-participants (n = 97)	p-value
Gender, n (%)
Male	16 (34.8%)	29 (29.9%)	0.560
Female	30 (65.2%)	68 (70.1%)
Mean age (SD)	57.57 (11.99)	59.48 (12.45)	0.326
Race, n (%)
Caucasian	44 (96%)	72 (74%)	0.001
African American	1 (2.3%)	5 (5.2%)
Asian	1 (2.3%)	0
Other	0	20 (20.6%)
Mean baseline TWSTRS motor severity score (SD)	14.77 (4.13)	N/A

Of the 97 recruits who did not participate, 68 (70.1%) were female and 29 were male, 61 (62.9%) did not respond to recruitment phone calls, 25 (27%) declined to participate without providing a reason, 8 (8.2%) did not have an appropriate device on which to complete the telemedicine visit, and 3 (3.2%) scheduled but did not attend their initial telemedicine visit (see Figure 2).

Figure 2.

Recruitment flow chart.

There were no significant differences when comparing participants to non-participants in terms of age or sex. The majority of contacted recruits were Caucasian, and Caucasians were more represented among participants compared with non-participants (96% vs. 74%; p = 0.001) (see Table 1). None of the participants had any prior experience with telemedicine.

Reliability

The ICC calculated for the TWSTRS motor severity summary score (baseline telemedicine vs. baseline in-person) was 0.890, indicating excellent intra-rater agreement. Of the individual TWSTRS items, laterocollis direction met the threshold for near perfect agreement (κ = 0.813) and several other items met the threshold for substantial agreement; these included rotation direction (κ = 0.753), rotation severity (κ = 0.661), presence of lateral shift (κ = 0.647), duration (κ = 0.732), and presence of sensory trick (κ = 0.795) (Table 2).

Table 2.

Agreement data.

Level of agreement	Parameter	Agreement value	Lower and Upper 95th Confidence Interval
Excellent	TWSTRS motor severity summary score	ICC = 0.890	0.713; 0.949
Almost perfect	Laterocollis direction	κ = 0.813	0.670; 0.956
Substantial	Rotation direction	κ = 0.753	0.578; 0.929
	Rotation severity	κ = 0.661	0.398; 0.924
	Presence of lateral shift	κ = 0.647	0.426; 0.868
	Duration	κ = 0.732	0.582; 0.881
	Sensory trick	κ = 0.795	0.607;0.983
Moderate	Laterocollis severity	κ = 0.592	0.383; 0.800
	Presence of anterocollis	κ = 0.572	0.309; 0.835
	Anterocollis severity	κ = 0.596	0.375; 0.817
	Presence of retrocollis	κ = 0.469	0.193; 0.745
	Presence of sagittal shift	κ = 0.465	0.183; 0.747
	Presence of shoulder elevation	κ = 0.528	0.338; 0.718
Fair	Retrocollis severity	κ = 0.404	0.164; 0.645
Fair	Range of motion	κ = 0.366	0.151; 0.581

Feasibility

Of the 49 recruits who agreed to participate and scheduled telemedicine visits, 46 successfully completed the initial telemedicine visit, indicating a training success rate of 93.8%; 44 of the 49 (89.8%) completed all three visits within the prescribed 4–6 week period; 3 participants declined follow-up participation and 2 participants were lost to follow up despite multiple attempts to connect via telephone.

Satisfaction

Setting up for the telemedicine was ‘understandable and easy’ for 73.5% of participants. Almost half (49%) of participants agreed that the quality of clinical care received during telemedicine was equal to that of an in-person visit, while 36.7% indicated that the quality of clinical care via telemedicine was better than that of a standard in-person visit. None of the participants had any personal safety concerns related to telemedicine, and almost all (94%) felt that their privacy was maintained during the visit (the remaining 6% felt neutral as to the level of their privacy). Most participants (75.5%) would favor additional telemedicine appointments, including 69% who indicated they would favor telemedicine visits over traditional in-person appointments in the future whenever feasible. Further, 75% of respondents indicated they were either ‘very likely’ or ‘likely’ to recommend telemedicine to another patient with the same or a similar health condition. Technical difficulties, including trouble downloading appropriate software, poor Internet speed and trouble using mobile devices, were the most frequently encountered issues contributing to decreased participant satisfaction.

The rating clinician (A.F.) reported that setting up the telemedicine visit was understandable and easy in 95% of cases with the overall quality of the telemedicine visit being equal to that of an in-person visit in 90% of cases. From the perspective of the rating clinician, the participants’ safety and privacy was maintained in all visits, and communication with participants was good at all visits. The rater was able to visualize the participants at all visits, and was able to hear them well in all but one visit. There were no visits in which the clinician felt that his clinical decisions would have been different had the participant been seen in-person. The rating clinician was pleased with the outcome of 97% of telemedicine visits; the one exception was related to poor Internet connection and lack of sound, though all verbal communication required for this visit was completed via telephone.

Discussion

This study investigated use of telemedicine in the evaluation of CD and demonstrates the feasibility of evaluating severity of CD remotely. The data provide critical information on patient compliance and ease of TWSTRS assessment by the physician, with direct applicability to assessing the need for outcomes after treatment. The TWSTRS scale was selected as the rating scale of choice because it is a validated scale for the assessment of CD, has excellent inter-rater reliability,¹⁰ and it does not require the physician to lay hands on the patient – making it an ideal rating scale for a computer-based visit. Telemedicine was well received by this cohort of CD patients and was implemented easily by the physician. The reliability of the telemedicine assessment with the in-person evaluation suggests that it is an effective means to assess motor severity in CD, though the same rater did all three assessments. All but two of the individual TWSTRS motor severity subscale items met the threshold for at least moderate agreement. These two exceptions were retrocollis severity and range of motion. We suggest that these two items are better assessed by a clinician who can rotate more freely around the patient rather than having the patient change his or her position, as is the case in the telemedicine visit.

The high percentage of participants who successfully completed the telemedicine training and the high rate of compliance with all three study visits indicate that the telemedicine evaluation of CD severity is feasible. Both the participants and the rating clinician were highly satisfied with the telemedicine visits. Some of the participants indicated that they felt they received higher quality care during the telemedicine visit when compared with an in-person visit, and would favor telemedicine visits over in-person visits whenever possible. The rating clinician did not feel clinical decision-making was compromised by the computer-based visits, and also felt that the quality of care provided during the telemedicine visits was as good as that provided in the standard clinic-based visits.

The findings of this study may be generalised to the greater CD population because this cohort had similar demographic and severity measures to previously published CD cohorts.^16–19 One of the major obstacles to participation in the study was inaccessibility of computer-based technology for many recruits. Similarly, there may be discomfort on the part of recruits in transmitting health information over the Internet, despite the encryption technology used. A strength of this study was that it successfully demonstrated the feasibility of and satisfaction with a new technology that allowed the participants and clinician to have face-to-face interaction via telemedicine following a therapeutic intervention. Utilization of telemedicine may lead to more efficient and optimal therapeutic approaches without losing patients for concerns of travel or inconvenience.

The major limitation of this study is that a single rater completed the TWSTRS ratings; thus, inter-rater reliability between the two visit types visits cannot be established. Therefore, further studies using multiple raters are needed to confirm our findings. Further, the fact that the initial telemedicine visit and the in-person assessment were done on consecutive days confounds the findings as the single rather likely remembered the patient and his/her evaluation on the previous day. Another limitation was the unfamiliarity of many participants with this type of computer technology, causing minor difficulties in setting up their devices for the visits. The quality of video and Internet varied significantly between participants and was dependent on the devices used, and quality of the Internet connection. Finally, although the study sample was estimated to be 50 participants, the high level of agreement achieved for most of the TWSTRS motor item scores and severity summary score suggests that the 46 participants was sufficient to confirm agreement and reliability.

In terms of future directions, telemedicine appears to be a very promising modality to reduce patient alienation and increase access to specialty neurologic care. Its use in CD and other neurologic disorders should be further studied so as to further integrate this valuable treatment modality into patient care.

Footnotes

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: AF has no relevant disclosures. GTS has provided consulting and advisory board membership with honoraria from Acadia, Pharmaceuticals, Adamas Pharmaceuticals, Inc., Biogen, Inc., Ceregene, Inc., CHDI Management, Inc., Cleveland Clinic Foundation, Ingenix Pharmaceutical Services (i3 Research), MedGenesis Therapeutix, Inc., Neurocrine Biosciences, Inc., Pfizer, Inc., Tools-4-Patients, Ultragenyx, Inc., and Sunshine Care Foundation and received grants and research funding from National Institutes of Health, Department of Defense, Michael J. Fox Foundation for Parkinson’s Research, Dystonia Coalition, CHDI, Cleveland Clinic Foundation, International Parkinson and Movement Disorder Society, and CBD Solutions, honoraria from International Parkinson and Movement Disorder Society, American Academy of Neurology, Michael J. Fox Foundation for Parkinson’s Research, Food and Drug Administration, National Institutes of Health, Alzheimer's Association, and a salary from the Rush University Medical Center GP has received consulting fees from Huron consulting, Krog Partners, US WorldMeds, and Allergan, and funding from Parkinson Disease Foundation and National Institute of Neurological Disorders and Stroke. CLC serves on the editorial board of Clinical Neuropharmacology and Sleep Medicine, receives research support from the NIH, Dystonia Medical Research Foundation, Merz Pharmaceutical, Revance Therapeutic, Retrophin, and Acorda Therapeutic, compensation/honoraria for services as a consultant or an advisory committee member from Acorda Therapeutics, Allergan, Inc., Lundbeck Ltd, Medtronic Inc., Merz Pharmaceuticals, Acadia Pharmaceuticals, Jazz Pharmaceuticals, Neurocrine Biosciences Inc., Revance Therapeutic, and Sunovion, royalties from Cambridge and Wolters Kluwer, and research support from the Parkinson’s Disease Foundation.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Appendix 1

References

Steeves

Day

Dykeman

, et al. The prevalence of primary dystonia: a systematic review and meta-analysis. Mov Disord 2012; 27: 1789–1796.

Werle

Takeda

Zonta

, et al. The physical, social and emotional aspects are the most affected in the quality of life of the patients with cervical dystonia. Arq Neuropsiquiatr 2014; 72: 405–410.

Comella

Bhatia

An international survey of patients with cervical dystonia.

J Neurol 2015; 262: 837–848.

Sethi

Rodriguez

Olayinka

Satisfaction with botulinum toxin treatment: a cross-sectional survey of patients with cervical dystonia.

J Med Econ 2012; 15: 419–423.

Goetz

Stebbins

Wolff

, et al. Testing objective measures of motor impairment in early Parkinson's disease: Feasibility study of an at-home testing device. Mov Disord 2009; 24: 551–556.

Jitkritsadakul

Rajalingam

Toenjes

, et al. Tele-health for patients with deep brain stimulation: The experience of the Ontario Telemedicine Network. Mov Disord 2017; 33: 491–492.

Zhang

Zeljic

, et al. A remote and wireless deep brain stimulation programming system. Neuromodulation 2016; 19: 437–439.

Schwab

Vatankhah

Kukla

, et al. Long-term outcome after thrombolysis in telemedical stroke care. Neurology 2007; 69: 898–903.

Dressler

Saberi

FA.

Safety of botulinum toxin short interval therapy using incobotulinumtoxin A. J Neural Transm (Vienna) 2016; 124: 437–440.

10.

Albanese

Sorbo

Comella

, et al. Dystonia rating scales: critique and recommendations. Mov Disord 2013; 28: 874–883.

11.

Comella

Perlmutter

Jinnah

, et al. Reliability of the Severity subscale of the revised Toronto Spasmodic Torticollis Rating Scale (TWSTRS-2) (S15.001). Neurology 2015; 84.

12.

Zou

GY.

Sample size formulas for estimating intraclass correlation coefficients with precision and assurance.

Stat Med 2012; 31: 3972–3981.

13.

Cicchetti

DV.

Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.

Psychol Assess 1994; 6: 284–290.

14.

Cohen

Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit.

Psychol Bull 1968; 70: 213–220.

15.

Landis

Koch

GG.

The measurement of observer agreement for categorical data.

Biometrics 1977; 33: 159–174.

16.

Boyce

Canning

Mahant

, et al. The Toronto Western Spasmodic Torticollis Rating Scale: reliability in neurologists and physiotherapists. Parkinsonism Relat Disord 2012; 18: 635–637.

17.

Comella

Jankovic

Truong

Hanschmann

Grafe

, Group USXCDS. Efficacy and safety of incobotulinumtoxinA (NT 201, XEOMIN(R), botulinum neurotoxin type A, without accessory proteins) in patients with cervical dystonia. J Neurol Sci 2011; 308: 103–109.

18.

Tarsy

Comparison of clinical rating scales in treatment of cervical dystonia with botulinum toxin.

Mov Disord 1997; 12: 100–102.

19.

Jankovic

Adler

Charles

, et al. Primary results from the cervical dystonia patient registry for observation of onabotulinumtoxina efficacy (CD PROBE). J Neurol Sci 2015; 349: 84–93.