Reliability of telemedicine evaluation for EDSS functional systems in multiple sclerosis

Abstract

Introduction: There was an increase in telemedicine during the COVID-19 pandemic to follow patients with multiple sclerosis (MS). However, there is scarce data if online evaluations can cover important information assessed during in-clinic appointments, especially the Expanded Disability Status Score (EDSS). This study aims to develop a remote evaluation tool for EDSS functional systems and compare the performance with face-to-face evaluations. Methods: This was a single-center study that included all MS patients followed up at outpatient clinics of Hospital São Lucas Pontifícia Universidade Católica do Rio Grande do Sul, between April and August 2022. Initially, patients were routinely in-clinic evaluated by one trained neurologist for EDSS. After, patients were evaluated remotely without any information about the in-clinic EDSS results. We used a standardized interview with an interactive video platform to evaluate EDSS functional systems by telemedicine. Results: Forty-nine participants completed the two steps. Intra-class coefficient was 0.97 (95% CI: 0.95–0.98), concordance for EDSS below 4.0 was 0.87 (95% CI: 0.77–0.93) and ≥4.0 was 0.97 (95% CI: 0.89–0.99). There was perfect agreement in the final EDSS in 71.4% of the online and in-clinic evaluations. In the multivariate analysis, the visual (beta = 0.453; p = 0.003) and pyramidal (beta = 0.403; p = 0.009) systems contributed significantly to the difference in the final EDSS. Conclusion: The telemedicine tool created in this study can detect changes in functional systems with reliable results compared to in-clinic EDSS assessment. Telemedicine evaluations may reduce the number of in-clinic visits and the disease burden for patients with MS.

Keywords

Multiple sclerosis telemedicine disability evaluation digital health assessment neurologic examination

Introduction

Multiple sclerosis (MS) is a chronic, immune-mediated demyelinating disease that affects the central nervous system (CNS).^1,2 The disease can lead to the accumulation of neurological symptoms or disabilities. To detect the progression of these disabilities, neurologists assess data such as annualized relapse rates, occurrence of progressive symptoms, comparative evolution of demyelinating CNS lesions in the MRI and changes in the Expanded Disability Status Score (EDSS).^3,4

However, in the wake of the COVID-19 pandemic, in-clinic medical appointments decreased significantly, particularly for MS patients using disease-modifying treatments, due to potential immunosuppression of some of these drugs and the risk of viral contamination. As a result, telemedicine evaluations increased in frequency.^5–8 However, to properly evaluate patients with MS by telemedicine, it is essential to assess information that can detect changes in the EDSS functional systems.

While some digital tools have already been developed to evaluate EDSS, most focus on generating a final scale score without providing a detailed assessment of the functional systems.^9–16 Additionally, most of these tools are self-applied and reported directly by the patient, which can generate a significant bias depending on the patient's interpretation of the questions. Therefore, the development of a reliable and standardized tool for the evaluation of functional systems is necessary for telemedicine evaluations that are likely to continue despite the end of COVID-19 pandemic.

To address this need, we developed a neurologist-guided tool for digital application of EDSS, which focuses on each functional system. We compared its performance with the gold standard in-clinic EDSS, performed by trained and certified neurologists.

Methods

Participants

The study included patients diagnosed with relapsing-remitting and progressive forms of MS who were seen at the outpatient clinics of the Neuroimmunology Department at Hospital São Lucas of Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS). Participants were required to meet the revised 2017 McDonald diagnostic criteria for MS,¹⁷ be at least 18 years old, have Portuguese as their native language, have access to a device with an internet connection, camera and audio system, and have a family member or caregiver available to assist during the online assessment. Participants had to be digitally evaluated within 60 days of their in-clinic appointment and could not have experienced a clinical relapse or new symptom secondary to MS between evaluations, lost interest in the study or experienced instability in their internet connection or technology issues.

Procedures

To assess the agreement between the examiners for the digital tool, a pilot study was conducted on ten patients. The patients were selected from those who visited the outpatient clinics between April to August 2022 and were diagnosed with MS. Two evaluators, C.E.M. and P.O.P., were responsible for administering the online assessment.

The patients who agreed to participate provided written informed consent, and data collection and patient assessment occurred in two steps:

The first step was an in-clinic evaluation that occurred on the day and time scheduled for the patient's regular follow-up visit. Patients were invited to participate in the study and provided written informed consent if they agreed to participate. During the in-clinic evaluation, patients were assessed by one of the four officially trained neurologists (C.E.M., P.O.P., G.R.P. and D.L.M.C.) using the Neurostatus EDSS, which is considered the gold standard physical examination of neurological functional systems. Additionally, the evaluation involved collecting socio-demographic and clinical data such as age, sex, education level, type of MS, disease duration, health service (public or private) and previous experience with telemedicine, both by video and phone calls.

The second step of the study involved an online evaluation that was scheduled after a telephone contact with the patient. The online evaluation was conducted at a time when the patient was accompanied by a family member or caregiver. The patient received all the necessary information for the online assessment via email, including access to the platform used (Zoom - www.zoom.us - Copyright ^®2022 Zoom Video Communications, Inc) and guidelines for installation. One of the two previously trained neurologists (C.E.M. or P.O.P.) guided the patient through the application of the digital tool, which was developed by the authors for the analysis of neurological functional systems of EDSS (Appendix 1). During the online evaluation, the examiner was available to clarify doubts if the patient or their companion encountered any difficulties in understanding the requested commands. The role of the caregiver or family member was limited to helping guide the camera to promote a good image view for the examiner, as well as helping the patient move between one command and another, and testing sensitivity. After the online assessment was completed, the examiner asked questions about the patient's comfort and satisfaction with the assessment process. It is important to note that patients were evaluated by different neurologists in the first and second steps, and the data collected in one step were not available to the examiner until the patient had completed participation in the second step.

Ethical approvals

This study was approved by the PUCRS Research Ethics Committee (CAAE – 52507321.0.0000.5336). All participants provided written informed consent before any study procedure.

Statistical analyses

Quantitative variables were described by mean, standard deviation, median and minimum and maximum values. Categorical variables were presented as counts and percentages. Kappa coefficient was used to assess agreement of the digital tool between examiners. Pearson's coefficient was used to measure correlation, and the intra-class coefficient (ICC) and concordance correlation coefficient (CCC) were used to measure agreement between EDSS results. Additionally, a multivariable linear regression model was used to explore the impact of selected functional systems on the difference between EDSS results. Findings with p values below 0.05 were deemed statistically significant. Data was analyzed using the R (version 4.1.3) and IBM-SPSS (version 25.0).

Results

Patients diagnosed with MS who visited the Neuroimmunology outpatient clinics of Hospital São Lucas of PUCRS between April and August 2022 were invited to participate in the study. About 76 patients were invited to participate in the study. Out of the 58 patients who agreed to participate, the final sample consisted of 49 participants, as illustrated in Figure 1. Nine patients were excluded during the data collection. Eight of them reported to lost interest in participating after the in-clinic evaluation, and one patient reported technical issues with instability of internet connection.

Figure 1.

Eligible patient selection flow chart.

The majority of the patients were female (63.3%) with a mean age of 37.7 years, ranging from 18 to 64 years. More than half of participants (61.2%) had completed secondary education, with 34.7% reported undergraduate school education. The average education was 13.5 years. Nearly all patients (93.9%) had relapsing-remitting MS, with the remaining diagnosed with secondary progressive MS. Less than half of the participants (44.9%) reported previous experience with telemedicine, and 79.6% used their cell phones for the online assessment. All online evaluations were performed indoors with adequate lighting, using high-speed internet. Other clinical and socio-demographic data can be found in Table 1.

Table 1.

Clinical and socio-demographic characteristics of the participants.

	N (%)	Mean	Median	SD	Min	Max
Age (years)		37.7	36	10.6	18	64
Female sex	31 (63.3)
Male sex	18 (36.7)
Education level (years)		13.5	12	3.5	10	24
Primary education	2 (4.1)
Secondary education	30 (61.2)
Undergraduate school education	17 (34.7)
Relapsing-remitting MS	46 (93.9)
Secondary progressive MS	3 (6.1)
Disease duration (years)		5.7	3	5.3	0	22
Public health service	32 (65.3)
Private health service	17 (34.7)
Previous telemedicine experience	22 (44.9)
Technology device used
Smartphone	39 (79.6)
Computer/laptop	10 (20.4)

SD: standard deviation; RRMS: relapsing-remitting multiple sclerosis; SPMS: secondarily progressive multiple sclerosis.

Online EDSS results between examiners showed a Kappa coefficient of 0.85 (95% CI: 0.63–1.0). As the primary outcome, concordance analyses were conducted between the in-clinic and online EDSS evaluations using the digital tool. The mean final EDSS score was 2.7 for both evaluations, with a minimum in-clinic EDSS score of 0.0 and a maximum of 7.0. The online EDSS score ranged from a minimum of 1.0 to a maximum of 7.0. The mean time between in-clinic and online assessments was 26 days, with a minimum of 2 days and a maximum of 58 days (Table 2). Patients with EDSS scores less than or equal to 3.0, between 3.5 and 5.5, and greater than or equal to 6.0 comprised 73.4%, 18.4% and 8.2% of the patients based on the in-clinic scores.

Table 2.

Final score of in-clinic and online EDSS.

	Mean	Median	SD	Min	Max
In-clinic EDSS	2.7	2.5	1.5	0	7.0
Online EDSS	2.7	2.0	1.6	1.0	7.0
Interval between in-clinic and online EDSS (days)	25.9	23	18.1	2	58

SD: standard deviation; EDSS: Expanded Disability Status Score.

The Pearson's correlation showed a positive correlation (r = 0.975) between the final in-clinic and online EDSS scores. However, there was a variation of up to 1.0 point in patients with EDSS below 4.0 and up to 0.5 point for patients with EDSS 4.0 or higher when comparing the scores of online EDSS and in-clinic EDSS (Figure 2). The ICC was 0.97 (95% CI: 0.95–0.98), indicating excellent agreement between the two evaluation methods. However, the concordance was lower in patients with low EDSS. The CCC for EDSS lower than 4.0 was 0.87 (95% CI: 0.77–0.93), whereas it was 0.97 (95% CI: 0.89–0.99) for EDSS greater than or equal to 4.0.

Figure 2.

Agreement between in-clinic and online EDSS.

The agreement between online EDSS and in-clinic EDSS, as well as each functional system, is presented in Table 3. A perfect agreement was observed in the final EDSS in 71.4% of the sample, with an agreement for a variation of up to 0.5 point in 91.8% and for a variation of 1.0 point in 100% of the cases. The mean difference between the online and in-clinic EDSS was −0.061 (95% CI: −1.0 to 1.0), and the mean differences for each functional system are depicted in Figure 3.

Figure 3.

Mean differences of each functional system (online – in-clinic).

Table 3.

The agreement of online EDSS with in-clinic EDSS and of each functional system.

	Perfect agreementn (%)	Agreement with 0.5 pointn (%)	Agreement with 1.0 pointn (%)
Online EDSS – in-clinic EDSS	35 (71.4)	45 (91.8)	49 (100)
Visual	43 (87.8)	43 (87.8)	49 (100)
Brainstem	43 (87.8)	43 (87.8)	49 (100)
Pyramidal	25 (51)	25 (51)	47 (95.9)
Cerebellar	32 (66.7)	32 (66.7)	45 (93.8)
Sensory	37 (75.5)	37 (75.5)	49 (100)
Bowel/bladder	43 (87.8)	43 (87.8)	49 (100)
Cerebral	44 (89.8)	44 (89.8)	49 (100)
Ambulation	43 (87.8)	43 (87.8)	49 (100)

EDSS: Expanded Disability Status Score.

To explore the contribution of each functional system to the difference in final EDSS, a multivariable linear regression model was fitted to the data. The dependent variable was the difference between the final online EDSS and final in-clinic EDSS. In the beta coefficient analysis, the visual (beta = 0.453; p = 0.003) and pyramidal (beta = 0.403; p = 0.009) functional systems significantly contributed to increase the final EDSS difference. Table 4 shows the mean difference of each functional system, as well as the final EDSS and the corresponding beta coefficients.

Table 4.

Mean difference of global and functional system score found in the regression model.

	Online EDSS	In-clinic EDSS	Differences (Online – in-clinic)	Beta coefficient	p value
Total	2.724	2.786	−0.061
Visual	0.816	0.857	−0.041	0.453	0.003
Brainstem	0.878	0.918	−0.041	0.223	0.150
Pyramidal	1.122	1.531	−0.395	0.403	0.009
Cerebellar	0.816	1.126	0.062	0.163	0.266
Sensory	0.796	0.878	−0.083	0.226	0.133
Bowel/bladder	0.796	0.796	0.000	0.144	0.355
Cerebral	0.816	0.796	0.020	−0.054	0.694
Ambulation	1.122	1.122	0.000	0.088	0.533

EDSS: Expanded Disability Status Score.

Bold represents as statistical significant (p value < 0.005).

Comparison of the difference in the final EDSS with the evaluated socio-demographic and clinical variables revealed that age and duration of disease did not show any significant results (p = 0.483, 0.499, respectively). Although not significant, patients with higher educational attainment (at least 12 years of study or completed high school) had a slightly smaller difference between final online and in-clinic EDSS compared to those with lower educational level (p = 0.112). The interval between evaluations also showed borderline significance (p = 0.083), with participants who had more than 14 days between the two assessments showing a smaller mean difference. Previous experience with telemedicine, technology device used and healthcare service utilized did not show any significant results (p = 0.504, 0.575 and 0.644, respectively). In the end, participants were asked to provide feedback on their satisfaction and comfort levels during the digital tool assessment. The results showed that approximately 94% of the patients were satisfied with the proposed evaluation, while only one patient reported discomfort during a specific command (i.e., examination of the visual functional system). Furthermore, all participants indicated that they would be willing to undergo the same online evaluation again if necessary and available.

Discussion

This cross-sectional study aimed to assess the level of agreement between a digital tool designed for the evaluation of the functional systems of the EDSS and the standard in-clinic EDSS, in patients with MS. The findings of this study indicate that remote assessment of MS patients by a trained neurologist using a digital tool can achieve a good level of correlation and agreement with the traditional in-clinic evaluation.

Few published studies have approached each functional system in detail with a neurologist as a guide for remote EDSS evaluation through a digital tool. Most of these previous studies have developed self-reported tools to obtain an estimation of the EDSS.^{9,12,15,16,18} Remarkably, a low level of agreement between in-clinic EDSS and online EDSS was identified in previously published studies that aimed to create a self-reported tool for EDSS evaluation. These studies reported Kappa values ranging from 0.13 to 0.73, which is considerably lower than the Kappa value (0.85) and ICC (0.97) observed in the present study. The higher agreement observed in our study may be attributed to the fact that a specialist-guided tool was used, which focused on each functional system rather than just the final EDSS score. Therefore, it is crucial to individually measure the functional systems in detail when monitoring patients with MS, particularly those with lower scores, to detect subtle disease progressions.

In a previously published study, Bove et al.¹¹ developed a remote tool to assess functional systems via video with the assistance of a neurologist. Patients were sent kits containing tools used in neurological physical examination, such as tuning fork, standardized Snellen chart and devices for sensory examination. The mean final EDSS of 41 patients was 2.0, like in our study. However, the mean difference between in-clinic and online EDSS was 0.32 (CI 95%), which was a favourable index, albeit higher than the present study. In this same analysis, the Kappa and Pearson coefficients were performed, showing an agreement of 0.72 and correlation of 0.89. Both results were superior in our study (0.85 and 0.97, respectively). Furthermore, in a multivariate analysis, the authors identified the pyramidal, cerebellar, sensory and bowel/bladder systems as contributing significantly to the difference between final EDSS. In comparison, our study showed a significant association between the functional visual system and the variation in final EDSS. This difference may be attributed to the fact that the patients in the first study received a standardized Snellen chart for home visual assessment, whereas the digital chart's size may have varied according to the technological device used by the patients in the current study. In Toscano et al.’s¹⁹ study, 40 patients with mild disabilities were assessed both in-clinic and remotely, with appointments guided by neurologists. The Kappa coefficient for the final EDSS was 0.30, indicating a lower agreement, particularly for the pyramidal, visual, brainstem, bowel/bladder and sensory functional systems.

The pyramidal functional system comprises the evaluation of strength, reflexes and muscle tone, which can be challenging to assess without a trained examiner present. In the study by Kane et al.,²⁰ the EDSS was compared between a general physician or a nurse, both trained in neurological physical examination of MS patients, remotely guided by an expert neurologist in neuroimmunology, and the EDSS performed synchronously online by the same neurologist. The inter-rater reliability index was satisfactory (Cronbach's coefficient ranging from 0.96 to 0.99), with the cerebellar and brainstem functional systems showing the least consistency. Therefore, having a professional trained in neurological physical examination present during neurologist-guided remote assessments could improve the accuracy of the pyramidal functional system.

Two other studies, Lechner-Scott et al.¹⁴ and Syed et al.,²¹ developed tools to assess patients’ EDSS through a telephone call, guided by neurologists. The examiner would conclude the final score based on individual questions asked to each patient. Both studies, similar to our present study, showed greater agreement in final EDSS for patients with higher scores. This data suggests that there is a greater influence of the ambulation functional system in these patients, which facilitate the determination of the final score.

In the multivariate analysis, the present study suggested a trend towards an association between patients with a higher educational level and a smaller mean difference in the final EDSS. This could be explained by the fact that patients with a higher education level may have a better understanding of the technical commands required to use the online platform and may have greater adherence to the remote assessment process. Regarding the type of technological device used by the patients, there was no significant association with the final EDSS agreement. This finding indicates that the proposed online assessment tool can be used on both cell phones and computers, which can facilitate its widespread use, given the high availability of cell phones as a current technological device.

When evaluating patients’ satisfaction with the method proposed in the present study, all patients reported that they would undergo the same evaluation again if given the opportunity for medical follow-up. With the emergence of the COVID-19 pandemic, studies have been conducted to assess patients’ satisfaction with digital strategies such as telemedicine. In Altmann et al.'s²² study, patients with MS assessed showed great satisfaction with remote appointments, both by telephone and by video, when compared to in-clinic evaluations. Chen et al.²³ found that patients with MS used telemedicine more frequently after the pandemic than a control group without MS and reported a higher level of satisfaction. This underscores the likelihood that remote assessments of MS patients will continue beyond the pandemic, especially given the physical limitations faced by many patients and the potential to increase healthcare access. In another recent study,²⁴ 97.1% of MS patients reported that they would recommend telemedicine to others due to its convenience, similarity to in-clinic appointments, and cost and time savings.

The present study has some limitations that should be mentioned. Firstly, the randomly selected sample consisted mostly of patients with EDSS scores less than 4.0, which may have created a gap in the assessment of patients with higher scores. Therefore, the results of this study cannot be extended to patients with higher EDSS scores. Additionally, the EDSS is an operator-dependent scale with subjective evaluation steps, which may cause inter-examiner variation. Another limitation is the difficulty in remotely reproducing some specific parts of the scale, such as the evaluation of strength, deep reflexes, vibration and proprioceptive sensibilities, which essentially require a face-to-face physician. Furthermore, the order in which the evaluation was performed could have influenced the patients’ answers remotely assessed, as they were initially evaluated in person and later online. Small fluctuations in symptoms may occur in patients with MS in short time intervals, without characterizing a new relapse of the disease, which may have influenced the data collected in the interval between evaluations. Lastly, this study did not standardize the type and model of smartphones and computed used, which may also have generated variations in the results.

The findings of this preliminary study suggest that the proposed telemedicine tool can effectively detect disabilities and functional system alterations in patients with MS when compared to in-clinic EDSS assessments. While most functional systems showed good agreement between online and in-clinic evaluations, improvements are needed for assessing the pyramidal and visual systems. Overall, the sample reported a high level of satisfaction with the proposed tool and indicated willingness to undergo the same assessment again. In conclusion, this study highlights the importance of maintaining this structured and neurologist-guided approach to EDSS assessment in remote evaluations, as it is becoming a crucial part of MS patient's care after the COVID-19 pandemic.

Supplemental Material

sj-docx-1-jtt-10.1177_1357633X231207903 - Supplemental material for Reliability of telemedicine evaluation for EDSS functional systems in multiple sclerosis

Supplemental material, sj-docx-1-jtt-10.1177_1357633X231207903 for Reliability of telemedicine evaluation for EDSS functional systems in multiple sclerosis by Cássia E Marin, Paula de O Pinto, Giordani R dos Passos, Daissy LM Cuervo, Mario B Wagner, Jefferson Becker and Douglas K Sato in Journal of Telemedicine and Telecare

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Cássia E Marin

Supplemental material

Supplemental material for this article is available online.

References

Nylander

Hafler

. Multiple sclerosis. J Clin Invest 2012; 122: 1180–1188.

Multiple Sclerosis International Federation. Atlas of MS 2013: mapping multiple sclerosis around the world. MSIF.org https://www.msif.org/ wp- content/uploads/2014/09/Atlas-of-MS.pdf (2013).

Díaz

Zarco

Rivera

. Highly active multiple sclerosis: an update. Mult Scl Rel Dis 2019; 30: 215–224.

Marques

dos Passos

Mendes

, et al. Brazilian Consensus for the treatment of multiple sclerosis: Brazilian Academy of Neurology and Brazilian Committee on Treatment and Research in multiple sclerosis. Arq Neuropsiquiatr 2018; 76: 539–554.

Contreras

Metzger

Beane

, et al. Telemedicine: patient-provider clinical engagement during the COVID-19 pandemic and beyond. J Gastrointest Surg 2020; 24: 1692–1697.

Ganapathy

. Telemedicine and neurological practice in the COVID-19 era. Neurol India 2020; 68: 555–559.

Mills

Savage

Lieder

, et al.

Telemedicine and the COVID-19 pandemic: are we ready to go live?

Adv Skin Wound Care 2020; 33: 410–417.

Caetano

Silva

Guedes

ACCM

, et al. Challenges and opportunities for telehealth during the COVID-19 pandemic: ideas on spaces and initiatives in the Brazilian context. Cad Saúde Publica 2020; 36: e00088920.

Kaufmann

Salmen

Barin

, et al. Development and validation of the self-reported disability status scale (SRDSS) to estimate EDSS-categories. Mult Scler Rel Dis 2020; 42: 102148.

10.

Collins

CDE

Ivry

Bowen

, et al. A comparative analysis of patient-reported expanded disability Status scale tools. Mult Scler 2016; 22: 1349–1358.

11.

Bove

Bevan

Crabtree

. Toward a low-cost, in-home, telemedicine-enabled assessment of disability in multiple sclerosis. Mult Scler 2019; 25: 1526–1534.

12.

Bowen

Gibbons

Gianas

, et al. Self-administered Expanded Disability Status Scale with functional system scores correlates well with a physician-administered test. Mult Scler 2001; 7: 201–206.

13.

Goodin

. A questionnaire to assess neurological impairment in multiple sclerosis. Mult Scler 1998; 4: 444–451.

14.

Lechner-Scott

Kappos

Hofman

, et al.

Can the Expanded Disability Status Scale be assessed by telephone?

Mult Scler 2003; 9: 154–159.

15.

Leddy

Hadavi

McCarren

, et al. Validating a novel web-based method to capture disease progression outcomes in multiple sclerosis. J Neurol 2013; 260: 2505–2510.

16.

Romeo

Rowles

Schleimer

, et al. An electronic, unsupervised patient-reported Expanded Disability Status Scale for multiple sclerosis. Mult Scler 2020; 27: 1432–1441.

17.

Thompson

Banwell

Barkhof

, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol 2018; 17: 162–173.

18.

Cheng

Hays

Myers

, et al. Factors related to agrément between self-reported and conventional Expanded Disability Status Scale (EDSS) scores. Mult Scler 2001; 7: 405–410.

19.

Toscano

Patti

Chisari

, et al. Reliability of televisits for patients with mild relapsing-remitting multiple sclerosis in the COVID-19 era. Neurol Sci 2022; 43: 2253–2261.

20.

Kane

Bever

Ehrmantraut

, et al. Teleneurology in patients with multiple sclerosis: EDSS ratings derived remotely and from hands-on examination. J Telemed Telecare 2008; 14: 190–194.

21.

Syed

Shah

Awan

, et al. Telephone validation of an Urdu translated version of the Extended Disability Severity Scale in multiple sclerosis patients. Mult Scler Rel Disord 2021; 48: 102684.

22.

Altmann

Leutmezer

Ponleitner

, et al. Remote visits for people with multiple sclerosis during the COVID-19 pandemic in Austria: The TELE MS randomized controlled trial. Digit Health 2022; 8: 20552076221112154.

23.

Chen

Goverover

Botticello

, et al. Healthcare disruptions and use of telehealth services among persons with multiple sclerosis during the COVID-19 pandemic. Arch Phys Med Rehabil 2022; 103: 1379–1386.

24.

Robb

Hyland

Goodman

. Comparison of telemedicine versus in-person visits for persons with multiple sclerosis: a randomized crossover study of feasibility, cost, and satisfaction. Mult Scler Relat Disord 2019; 36: 101258.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.88 MB