Physiotherapy approach in stroke rehabilitation: Development and testing of a survey of current practice

Abstract

BACKGROUND AND PURPOSE:

Physiotherapy is a key discipline in stroke rehabilitation. Physiotherapists sometimes select interventions following personal preference rather than a scientific rational. Data on physiotherapy approaches used in stroke rehabilitation may help policy makers and educators to plan education strategies and implement efficient clinical practices, thus improving rehabilitation effectiveness. We aimed to develop and test a questionnaire designed to survey physiotherapy interventions utilized in stroke rehabilitation.

METHODS:

We used a multistep questionnaire development method (literature review and synthesis; questionnaire drafting; expert validation; cognitive interviewing). Afterwards the survey proceeded to reliability testing; the outcomes of interest were completion time, estimated comprehensiveness of the lists of interventions, and intraclass correlation coefficient (ICC).

RESULTS AND DISCUSSION:

Retrieved survey tools, their items, and a taxonomy were useful for the questionnaire development. We interviewed four experts; changes were made to the survey following their suggestions. Thirteen physiotherapists participated in cognitive interviewing and further changes were made. Thirty-five raters participated in the test-retest study. Most participants considered the list of physiotherapy interventions and assistive device exhaustive. Median compilation time and time interval between the two compilations were 7 minutes and 21 days. The observed ICC was 0.844 (95% confidence interval, 0.829/0.857). Our method provided a valid and reliable questionnaire, however further methodological considerations of sampling methods and contact delivery modes are needed.

CONCLUSIONS:

If adequately implemented, the questionnaire can provide information about interventions utilized in stroke rehabilitation practice by physiotherapists. Data eventually acquired could be useful for planning educational strategies and implementing effective clinical practices.

Keywords

Physiotherapy stroke survey intervention types approach

1 Introduction

Physiotherapy plays a key role in stroke rehabilitation and presents a variety of approaches and techniques [1]. The growing body of knowledge on the physical and neurophysiological principles underlying movement control has led to the development of an array of treatment strategies such as neurodevelopmental approach, motor learning, constraint induced movement therapy, functional electrical stimulation, or robotics, just to name a few [2]. Debate about their relative effectiveness is ongoing. There is evidence that some physiotherapy interventions (e.g., intensive high repetitive task-oriented and task-specific training) promote recovery after stroke [2, 3]; however, none of these have demonstrated greater efficiency than any others in the recovery of function and mobility [1].

Professional practice affects the process, outcomes, and cost effectiveness of rehabilitation services [4]. The approach to stroke physiotherapy is usually multifaceted [3]. Physiotherapists could select interventions following personal preference rather than a scientific rationale [1]; it may result in administering a less effective therapy and minor client outcomes [5]. Data on physiotherapy approaches and interventions used in stroke rehabilitation may be evaluated by policy makers and educators [6]; an understanding of practitioner characteristics is needed to inform education strategies and implement efficient clinical practices, thus improving rehabilitation effectiveness [5].

Surveys have been conducted on the topic [6 –10]; however, previous questionnaires are now dated, the methodology adopted to develop them was underreported, and no surveys were tested for reliability. It would be useful if a dedicated tool conceived and tested according to current methods for the development of questionnaires was available.

The purpose of this study was to develop and test a questionnaire to survey approaches and interventions utilized in stroke rehabilitation practice by physiotherapists.

2 Methods

The reporting of this study conforms to the CHERRIES Checklist [11] and the GRRAS Guidelines [12]. The study was approved by the local Ethics Committee. A mixed multistep questionnaire development method was used [13] (Fig. 1).

Fig. 1

Questionnaire development flow chart.

2.1 Literature review

Two researchers (LC, FF) conducted the review and synthesized the literature. To identify survey tools or items that could be used or adapted, we investigated the questionnaires used in similar surveys. We searched for publications reporting taxonomies, classifications, and lists of physiotherapy interventions for stroke rehabilitation. An electronic search was conducted of MEDLINE, the National Rehabilitation Information Center, and Google Scholar up to November 2015. We used the following combination of keywords: (physical therapy modalities OR physical therapy OR physiotherapy OR rehabilitation) AND (intervention OR treatment OR approach OR principle) AND stroke AND ((surveys and questionnaires OR survey OR questionnaire) OR (classification OR taxonomy OR list OR inventory)). The reference lists of the retrieved articles were manually searched for further relevant publications.

2.2 Synthesizing the literature

Content areas and items emerging from the questionnaires retrieved in the literature review were highlighted and synthesized. Available lists of physiotherapy interventions were assessed for comprehensiveness and usefulness.

2.3 Drafting content areas and items

Based on the literature review findings, a list of potential questions was compiled and placed in draft survey format. We avoided double-barrelled or negatively worded items and developed construct-specific anchors [13]. When in doubt or unfamiliar with the content of an intervention, a description was sought. For this purpose, the intervention was eventually searched for in Google Scholar, in combination with other keywords (e.g., motor control AND (physiotherapy OR exercise OR approach)). A paper version of the survey was generated at the end of this phase.

2.4 Expert validation

This initial survey was presented to four experts in physiotherapy and stroke rehabilitation from the local University School of Physiotherapy, working in management, research, clinical practice, and education fields. After examining the survey, each expert was interviewed face-to-face about the clarity, item relevance, language appropriateness, and missing facets/aspects of its content. A version of the survey was then drafted in accordance with the experts’ opinions and adapted to a web-based format.

2.5 Cognitive interviewing

To determine how potential respondents would interpret the items, cognitive interviews were conducted using the retrospective probing procedure and the immediate retrospective approach [13, 14]. An interview protocol was set up.

The invitation to be interviewed was addressed to physiotherapists providing inpatient or outpatient rehabilitation to individuals with stroke in the network of Azienda USL Toscana centro clinics. To be interviewed, candidates had to meet the inclusion criteria designed for the test-retest study. All interviews were conducted by the same researcher and audio recorded. After a brief introduction, the physiotherapist was invited to complete the survey independently. The aims, procedures, and content of the cognitive interview were then explained in detail to the respondent, who provided oral consent to be interviewed. While browsing the survey, a series of probe queries designed to elicit specific information about the questions, response options, and instructions of the questionnaire was administered. If discrepancies in the interpretation of some items were evident, suggestions for rewording were sought. At the survey’s completion, the interviewer asked questions about the survey completion experience; the respondent was invited to express any other comments. Records were analyzed independently by two researchers (LC, FF); disagreements were solved by consensus. After the third interview, the researchers started to make changes to the questionnaire with consideration of the criticisms that emerged. Any revised version of the survey was then tested in subsequent interviews. The interviewing continued until no new discrepancies arose. The final version of the survey was obtained at the end of this phase.

2.6 Testing

The survey was named “Survey of physiotherapy approaches to stroke rehabilitation.” The areas surveyed are described later. A convenience sample of physiotherapists was considered for enrolment. To participate, each had to have treated at least 10 individuals referred for stroke rehabilitation in the last 12 months prior enrolment. By completing the questionnaire, the individual declared being a licensed physiotherapist and consented to participate in the reliability study. Data were managed only by authorized personnel.

A closed survey was developed and administered using the free online survey software tool Surveyexpression (https://www.surveyexpression.com). Eligible respondents were recruited among physiotherapists working in the national health system or registered on the Italian Association of Physiotherapists website or present in the contact lists of the study researchers. An e-mail with the description of the study and an invitation to disseminate the survey to colleagues was sent to physiotherapy managers. The same information was posted on two Facebook groups and spread via personal contacts. Those interested in participating were requested to send an email to a dedicated address and were therefore included in the candidate responders list. Apart from e-mail addresses, we were unaware of any other information of the potential participants, including whether they were involved in stroke rehabilitation. No incentives or compensation was offered for participation.

The invitation to complete the questionnaire was sent to all the candidate responders. The retest invitation was sent between 14 and 21 days after the eventual first compilation. Personal reminders were sent to the non-responders. Responses entered in the survey tool were transferred into a standard spreadsheet. Data were extracted by one investigator (LC or SK) and subsequently double-checked by another (FF). Disagreements were resolved by consensus.

Data were collected between February and July 2018 except for pilot data, which were obtained between November 2015 and February 2016. Items per page ranged from three to 32 and were distributed across five pages. Answering all the questions on one page was mandatory before moving on to the next page. Respondents could review and change their answers before submission. The answers were matched to the participants via e-mail addresses. The usability and technical functionality were tested in a pilot with nine participants; the observed intraclass correlation coefficient (ICC) was 0.840, with a 95% confidence interval (CI) of 0.810 to 0.867.

The required sample size was calculated using the method described by Walter et al. [15]. The ICC value observed in the pilot (0.840) was adopted as desirable, with the lowest acceptable ICC of 0.650. The sample size estimate indicated that 33 participants were required (α= 0.05, 80% power).

Only questionnaires with complete answers were considered. Sample characteristics were analysed using descriptive statistics. The participation rate was calculated as the ratio between the number of respondents and the number of individual invitations sent. Box-and-whisker plots were used to detect outliers. Test-retest reliability of the questionnaire was assessed using the ICC_2.1 for absolute agreement. To calculate the ICC, we considered only the answers related to the frequency with which the various physiotherapy approaches and interventions were used in clinical practice (items coded 01 to 46; see Supplemental File). An ICC value more than.75 indicated good reliability [16].

The statistical analyses were performed using IBM SPSS Statistics for Windows (version 20.0; IBM Corp, Armonk, NY, USA).

3 Results

3.1 Literature review

We found 14 studies, published from 1992 to 2012, investigating the types of approach and combination of interventions used by physiotherapists in stroke rehabilitation [4 , 17– 24].

DeJong et al. [20] proposed a conceptual framework aimed at categorizing physiotherapy interventions; functional activities were considered the key classification dimension, whole body systems (neuromuscular, musculoskeletal, cardiopulmonary, and cognitive/perceptual/sensory) were considered the secondary dimension; the combination of the two dimensions could capture the multidimensionality of physiotherapy clinical practice and represent an efficient way to classify interventions. A similar scheme was used to collect data in one of the retrieved studies [21].

In their systematic review, Pollock et al. [1] classified physiotherapy approaches using systematic categorization of the intervention components described in a randomized controlled trial’s sample. The method led to the identification of 27 components, which were a posteriori assigned to seven categories defined on the basis of the taxonomy described by DeJong et al. [20]. In other systematic reviews, the authors reached consensus on intervention categories [3] or outlined the main physiotherapy approaches used in stroke rehabilitation [2].

3.2 Synthesizing the literature

Many studies analyzed the characteristics of the physiotherapists providing care to individuals with stroke and focused on education and experience in stroke rehabilitation. Some studies focused on settings [21, 22], assistive devices [17], factors influencing the choice of treatment and attitudes toward new methods [7 –9], sources of conflict between physiotherapists and other healthcare professionals [4], and beliefs about stroke rehabilitation [19].

The taxonomy of stroke rehabilitation interventions proposed by DeJong et al. [20] was the only available.

3.3 Drafting content areas and items

The first page of the survey contained details on the study purposes, options for withdrawal, and data processing methods. Acknowledgments and contact information were provided at the end of the letter.

The questionnaire was structured into two sections. The first collected information about respondent characteristics (age; sex; graduation year; other academical degrees; clinical experience; and eventual involvement in the education, management, or research fields), number of individuals with stroke treated in the previous 12 months, phase of the disease of those treated, and the main work setting. The response options were mostly closed-ended, nominal, or ordinal. The second section investigated the frequency with which physiotherapy approaches and interventions were used in clinical practice. It was divided into two parts, functional activities and approaches, based on the DeJong et al. [20] framework. The functional activities list was faithfully adopted. As for the approaches, we adopted a modified version of the scheme provided in the form. The Guide to Physical Therapist Practice [25], the Clinician’s Handbook of Evidence-Based Review of Stroke Rehabilitation [26], and published systematic reviews [1 –3] were consulted to complete the list of interventions and check for its comprehensiveness and coherence. We limited questions on aids and orthoses to those most frequently used in clinical practice. The respondents could add other possible interventions to the end of the list. The frequency of application in clinical practice of any single intervention/approach listed was recorded with an end-labelled numeric rating scale [27] ranging from 0 (never) to 6 (always).

3.4 Expert validation

The expert consultations led to changes. Information on physiotherapists’ characteristics (e.g. type of contract and company, province of practice) was added. Questions about the respondent’s familiarity with the scientific literature on stroke rehabilitation (e.g., frequency of articles read, perceived importance of scientific literature) were included. Details on the delivery of the intervention (duration of the single session, number of sessions, and total duration) were also requested. Some response options were changed from an ordinal to a continuous scale (e.g. age, years of clinical experience).

Problems regarding the list of functional activities emerged during the expert validation, including a lack of clarity (e.g. gait vs. advanced gait), difficulty to recognize as distinct or different (e.g. sitting, transfers, sit-to-stand), and absence of functional activities involving the upper limbs [20]. Based on these criticisms, we reconsidered this section.

Since the questionnaire was conceived and tested in Italy, on experts’ advice we included the Perfetti’s Cognitive Sensory Motor Training [28] and the Grimaldi’s technique [29] among the neuromuscular interventions; the two approaches were developed in Italy and are locally used in clinical practice. Following suggestion, we merged robotics, electrostimulation, and biofeedback in the neuromuscular interventions category and split the assistive devices section into two subcategories (choice/training/manufacturing and use within therapy). The experts agreed on the list of aids and orthoses we selected. To facilitate each respondent’s answers [30], the questions were grouped into grids; at the end of the questionnaire, we asked how the respondent would rate the exhaustiveness of our lists (5-point Likert-type scale).

3.5 Cognitive interviewing

Thirteen physiotherapists (8 women) from ten clinics participated in the cognitive interview; duration averaged 22 min. The subjects interviewed were 25– 57 years old (median, 44 years) and their clinical experience in stroke rehabilitation was 1– 34 years (median, 20 years). Following suggestions, questions related to training of functional activities were revised and proposed to distinguish between functional training in basic and instrumental activities of daily living and functional training oriented to work or community/leisure activities. We added space to allow for any other changes that the respondent might want to make to the intervention and device lists. At the end of the questionnaire, space was left to allow the respondent to report whether there were disagreements or information that had been overlooked or omitted as well as for any other comment.

At the end of the cognitive interviewing phase, the questionnaire was ready to be tested for reliability. The original online survey is available on request from authors.

3.6 Testing

Invitations were sent to 322 physiotherapists; 65 returned the questionnaire after the first invitation (participation rate, 20.2%). Of them, 37 responded to the retest invitation. The box-plot related to time interval between the two compilations showed two outliers, which were then excluded. Therefore, 35 participants were included in the reliability analysis (Fig. 2).

Fig. 2

Reliability study flow chart.

The raters’ characteristics are presented in Table 1 and Fig. 3. Median completion time and time interval between the two compilations were 7 minutes and 21 days, respectively (Table 2).

Table 1

Characteristics of the participants

Variable	n = 35
Age-years*		42	(33.0– 49.0; 26 to 61)
Male		11	(31.4, 18.6/47.9)
Employed in a public company		24	(68.6, 52.0/81.5)
Open-ended/permanent contract		28	(80.0, 64.1/89.9)
Graduation year < 2000		17	(48.6, 32.9/64.4)
Academic degrees other than a		14	(40.0, 25.6/56.4)
Bachelor’s in physiotherapy
Clinical expertise in stroke		15	(7.5– 25.0; 1 to 35)
rehabilitation-years^*
Involved in education		19	(54.3, 38.2/69.5)
Involved in management		3	(8.6, 2.9/22.3)
Involved in research		9	(25.7, 14.1/42.1)
Number of individuals treated	10– 20	23	(65.7, 49.1/79.1)
in the last 12 months	≥21– 30	5	(14.3, 6.3/29.4)
	≥31– 40	2	(5.7, 1.6/18.6)
	≥41	5	(14.3, 6.3/29.4)
Main setting	Outpatient rehabilitation-day hospital	20	(57.1, 40.9/72.0)
	Inpatient rehabilitation-acute	12	(34.3, 20.8/50.9)
	hospital-stroke unit
	Home based-residential home	3	(8.6, 2.9/22.3)
Case-mix with at least 50% of the	< 15 day	7	(20.0, 10.0/35.9)
individuals in a specific	15 days to 3 months	10	(28.6, 16.3/45.1)
phase of the disease	> 3 to 6 months	5	(14.3, 6.3/29.4)
> 6 months	6	(17.1, 8.1/32.7)

Data are absolute frequencies (percentages, 95% Confidence Interval) except for ^*= median (1st– 3rd quartile; minimum to maximum).

Fig. 3

Frequencies of observed response options related to the estimated importance and amount of reading of scientific literature/data are percentages. Error bars represent the 95% confidence interval.

Table 2

Minutes needed for survey completion, and days between compilations

Variable	n = 35
First completion duration-minutes	7 (6.0– 12.0; 4 to 64)
Second completion duration-minutes	7 (4.5– 8.5; 3 to 13)
Time interval between the two compilations-days	21 (19.0– 26.0; 14 to 38)

Data are median (1st– 3rd quartile; minimum to maximum).

According to the first compilation, the interventions/approaches used frequently (median value, 5 or 6) by the participants were functional training in basic activities of daily living, balance and postural exercises, motor learning, mobilization and stretching, individual and family/caregiver education, and choice of assistive devices and training in their use. Raters mostly considered exhaustive the approaches/interventions list (82.8%; 95% CI, 67– 92) and the assistive devices list (94.3%; 95% CI, 81– 98) (Fig. 4). Action observation training and the neurodynamic approach were mentioned, each from one rater, as desired integrations to the interventions list. Patient lifter was mentioned as a desired integration to the assistive devices list from two raters; one of them added a stair lifter, anti-decubitus cushion and mattress, nursing bed, and bed rails. The observed ICC was 0.844 (95% CI, 0.829 to 0.857).

Fig. 4

Frequencies of observed response options related to the estimated exhaustiveness of lists/data are percentages. Error bars represent the 95% confidence interval.

4 Discussion

Construct and items of the retrieved survey tools investigating the clinical practice of physiotherapists could be used or adapted. None of the retrieved taxonomies, classifications, or lists of interventions could be considered a gold standard. We deemed the conceptual framework of DeJong et al. [20] as a valid base for the development of our questionnaire. Based on the literature review findings, we drafted a questionnaire collecting information about respondent characteristics and practices. Modifications and additions to our tool were made according to interviews with experts and physiotherapists. Achieving the desired sample size, 35 raters were included in the analysis. The observed questionnaire completion time was satisfactory, and the time lag between the two completions was appropriate. The lists of interventions/approaches and assistive devices were judged as exhaustive. The ICC obtained was good and had a narrow 95% CI.

4.1 Limitations

Because we felt that the construct of interest was already well conceptualized and described, we did not deem it necessary to conduct a preliminary exploratory qualitative research on the topic (e.g. focus groups). Although expert consultation and cognitive interviewing provided unprompted information useful for adapting the construct to the population of interest, focus groups may have anticipated the problems we encountered in developing the questionnaire. Moreover, all individuals participated in the present research on a voluntary basis, and volunteers might slightly differ from the general population of physiotherapists engaged in stroke rehabilitation [31].

In our study, we observed a low participation rate. It should be considered that we were unaware if the physiotherapists we invited were involved in stroke rehabilitation; moreover, some of the invitation e-mails may have been intercepted by spam-blocking tools [32]. These factors could have influenced the response rate. However, we achieved the desired sample size, and as our objective was to test the reliability of the survey form, we believe that the actual participation rate did not affect the study results.

4.2 Interpretation

Compared to similar surveys published in the last 10 years [6 , 24], in addition to demographic and background information, our questionnaire focused primarily on the frequency with which approaches and interventions were used. We used a rigorous qualitative and quantitative development method [13]. We clearly described the target population and context of use of the survey tool; cognitive interviewing was conducted according to a protocol. Two researchers were involved in the analysis, and the tool was adapted and re-tested when necessary. An adequate number of participants was involved in the qualitative and quantitative aspects of the research.

When strengths and limitations of the study are carefully balanced, we feel that integrating pre-existing surveys, the DeJong et al. [20] framework, and qualitative research (literature review, item generation, and expert and cognitive interviews) provide the foundation to build an updated, comprehensive, easy to complete, valid, and reliable questionnaire.

Further methodological considerations are needed before an eventual survey is conducted. Sampling methods and contact delivery modes, although adequate for this development study, must be reviewed to obtain a representative sample [32]. To prevent coverage- and sampling- error, it would be necessary to identify who provides stroke rehabilitation in a specific area and provide face-to-face presentation meetings of the survey. A combined hard copy/web-based approach could be proposed according to the preferences of candidate responders [32]. Other factors such as the support of authority figures, to declare the time needed to complete the survey and, in case of web-based surveys, to send a low number of personalized reminders messages, may be helpful [33, 34].

5 Conclusion

Here we used rigorous qualitative and quantitative methodologies to support the reliability and validity of the survey tool we developed. If adequately implemented, the questionnaire can provide information about approaches and interventions utilized in contemporary stroke rehabilitation practice by physiotherapists. Data eventually acquired could be useful for planning educational strategies and implementing effective clinical practices.

Footnotes

The supplementary appendix is available in the electronic version of this article: .

References

Pollock

, Baer

, Campbell

, et al. Physical rehabilitation approaches for the recovery of function and mobility following stroke. Cochrane Database Syst Rev. 2014;4:CD001920. doi: 10.1002/14651858.CD001920.pub3

Langhorne

, Bernhardt

, Kwakkel

. Stroke rehabilitation. Lancet. 2011;377(9778):1693–702.

Veerbeek

, van Wegen

, van Peppen

, van der Wees

, Hendriks

, Rietberg

, Kwakkel

. What is the evidence for physical therapy poststroke? A systematic review and meta-analysis. PLoS One. 2014;9(2):e87987. doi: 10.1371/journal.pone.0087987

Davidson

, Waters

. Physiotherapists working with stroke patients: a national survey. Physiotherapy. 2000;86(2):69–80.

Salbach

, Jaglal

, Korner-Bitensky

, Rappolt

, Davis

. Practitioner and organizational barriers to evidence-based practice of physical therapists for people with stroke. Phys Ther. 2007;87(10):1284–303.

Natarajan

, Oelschlager

, Agah

, Pohl

, Ahmad

, Liu

. Current clinical practices in stroke rehabilitation: regional pilot survey. J Rehabil Res Dev. 2008;45(6):841–9.

Nilsson

, Nordholm

. Physical therapy in stroke rehabilitation: bases for Swedish physiotherapists’ choice of treatment. Physiother Theory Pract. 1992;8(1):49–55.

Carr

, Mungovan

, Shepherd

, Dean

, Nordholm

. Physiotherapy in stroke rehabilitation: bases for Australian physiotherapists’ choice of treatment. Physiother Theory Pract. 1994;10(4):201–9.

Ogiwara

. Physiotherapy in stroke rehabilitation: a comparison of bases for treatment between Japan and Sweden. J Phys Ther Sci. 1997;9(2):63–9.

10.

Khan

, Vijesh

, Rahool

, Radha

, Sukumaran

, Kurupath

. Physiotherapy practice in stroke rehabilitation: a cross-sectional survey of physiotherapists in the state of Kerala, India. Top Stroke Rehabil. 2012;19(5):405–10.

11.

Eysenbach

. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3):e34.

12.

Kottner

, Audigé

, Brorson

, et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96–106.

13.

Artino

, La Rochelle

, Dezee

, Gehlbach

. Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach. 2014;36(6):463–74.

14.

Beatty

, Willis

. Research Synthesis: The practice of cognitive interviewing. Public Opin Q. 2007;71(2):287–311.

15.

Walter

, Eliasziw

, Donner

. Sample size and optimal designs for reliability studies. Stat Med. 1998;17(1):101–10.

16.

Portney

, Watkins

. Foundations of clinical research: applications to practice, 3rd ed. Upper Saddle River (NJ): Pearson Education; 2009.

17.

Sackley

, Lincoln

. Physiotherapy treatment for stroke patients: a survey of current practice. Physiother Theory Pract. 1996;12(2):87–96.

18.

Ballinger

, Ashburn

, Low

, Roderick

. Unpacking the black box of therapy – a pilot study to describe occupational therapy and physiotherapy interventions for people with stroke. Clin Rehabil. 1999;13(4):301–9.

19.

Lennon

, Baxter

, Ashburn

. Physiotherapy based on the Bobath concept in stroke rehabilitation: a survey within the UK. Disabil Rehabil. 2001;23(6):254–62.

20.

DeJong

, Horn

, Gassaway

, Slavin

, Dijkers

. Toward a taxonomy of rehabilitation interventions: Using an inductive approach to examine the “black box” of rehabilitation. Arch Phys Med Rehabil. 2004;85(4):678–86.

21.

Jette

, Latham

, Smout

, Gassaway

, Slavin

, Horn

. Physical therapy interventions for patients with stroke in inpatient rehabilitation facilities. Phys Ther. 2005;85(3):238–48.

22.

Hale

, Piggot

. Exploring the content of physiotherapeutic home-based stroke rehabilitation in New Zealand. Arch Phys Med Rehabil. 2005;86(10):1933–40.

23.

De Wit

, Putman

, Lincoln

, Baert

, et al.

Stroke rehabilitation in Europe: what do physiotherapists and occupational therapists actually do?

Stroke. 2006;37(6):1483–9.

24.

Tyson

, Connell

, Lennon

, Busse

. What treatment packages do UK physiotherapists use to treat postural control and mobility problems after stroke? Disabil Rehabil. 2009;31(18):1494–500.

25.

American Physical Therapy Association. Guide to physical therapy practice. 2nd ed. Alexandra (VA): APTA; 2001.

26.

Evidence-Based Review of Stroke Rehabilitation. Clinician’s Handbook, motor rehabilitation. http://www.ebrsr.com/clinician-handbook Accessed 18 March 2020.

27.

Moors

, Kieruj

, Vermunt

. The effect of labeling and numbering of response scales on the likelihood of response bias. Sociol Methodol. 2014;44(1):369–99.

28.

Chanubol

, Wongphaet

, Chavanich

, Werner

, Hesse

, Bardeleben

, Merholz

. A randomized controlled trial of Cognitive Sensory Motor Training Therapy on the recovery of arm function in acute stroke patients. Clin Rehabil. 2012;26(12):1096–104.

29.

Crippa

, Cardini

, Pellegatta

, Manzoni

, Cattaneo

, Marazzini

. Effects of sudden, passive muscle shortening according to Grimaldi’s method on patients suffering from multiple sclerosis: a randomized controlled trial. Neurorehabil Neural Repair. 2004;18(1):47–52.

30.

Roßmann

, Gummer

, Silber

. Mitigating satisficing in cognitively demanding grid questions: evidence from two web-based experiments. J Surv Stat Methodol. 2017;2017. pii: smx020, https://doi.org/10.1093/jssam/smx020

31.

Sedgwick

. Bias in observational study designs: cross sectional studies. BMJ.h. 2015;350:h1286. doi: 10.1136/bmj.h1286

32.

Fan

, Yan

. Factors affecting response rates of the web survey: A systematic review. Comput Human Behav. 2010;26(2):132–9.

33.

Saleh

, Bista

. Examining factors impacting online survey response rates in educational research: Perceptions of graduate students. JMDE. 2017;13(29):63–74.

34.

Sánchez-Fernández

, Muñoz-Leiva

, Montoro-Ríos

. Improving retention rate and response quality in web-based surveys. Comput Human Behav. 2012;28:507–14.