Effectiveness and cost-effectiveness of an optimized process of providing assistive technology for impaired upper extremity function: Protocol of a prospective,quasi-experimental non-randomized study (OMARM)

Abstract

BACKGROUND:

Impaired upper extremity function due to muscle paresis or paralysis has a major impact on independent living and quality of life (QoL). Assistive technology (AT) for upper extremity function (i.e. dynamic arm supports and robotic arms) can increase a client’s independence. Previous studies revealed that clients often use AT not to their full potential, due to suboptimal provision of these devices in usual care.

OBJECTIVE:

To optimize the process of providing AT for impaired upper extremity function and to evaluate its (cost-) effectiveness compared with care as usual.

METHODS:

Development of a protocol to guide the AT provision process in an optimized way according to generic Dutch guidelines; a quasi-experimental study with non-randomized, consecutive inclusion of a control group ( $n=$ 48) receiving care as usual and of an intervention group (optimized provision process) ( $n=$ 48); and a cost-effectiveness and cost-utility analysis from societal perspective will be performed. The primary outcome is clients’ satisfaction with the AT and related services, measured with the Quebec User Evaluation of Satisfaction with AT (Dutch version; D-QUEST). Secondary outcomes comprise complaints of the upper extremity, restrictions in activities, QoL, medical consumption and societal cost. Measurements are taken at baseline and at 3, 6 and 9 months follow-up.

Keywords

Efficiency cost-effectiveness impaired upper extremity function dynamic arm supports robotic arms provision of assistive technology devices user satisfaction

1. Background

Different neurological diseases, traumatic injuries and other health conditions can lead to impaired upper extremity function due to muscle paresis or paralysis, with severe consequences for independent daily functioning [1, 2], and impacting largely on autonomy, self-esteem and quality of life (QoL) [3, 4]. No aggregated prevalence data on clients with impaired upper extremity function exist; however, considering merely the prevalence of certain neurological diseases, the target population is substantial. The prevalence data, expressed as number per 100,000 inhabitants, of neuromuscular diseases (1,145) [5] (including amyotrophic lateral sclerosis (ALS) (9–11) [6]), multiple sclerosis (MS) (183) [7], stroke (338.8) [8] and cervical spinal cord injury (48.6) [9] would add up to about 332,000 clients in the Netherlands. While a large fraction is not in need of assistive technology (AT) devices for enhancement of upper extremity function (i.e. dynamic arm supports and robotic arms), the estimated number of 150 provisions per year, based on unpublished data from a Dutch supplier and a health insurance company, appears to be relatively small [10, 11].

According to the ISO 9999 standard [12], dynamic arm supports are “arm supports to permit manual activities”. In this study, dynamic arm supports are operationally defined as “AT that facilitates arm function during activities of daily living” [13]. They “allow people with weak arms to move more freely. They consist of a trough on hinged brackets which attaches to the wheelchair and support the arm in a range of positions for activities such as eating, typing, writing etc.” [14]. Robotic arms are referred to as robotic manipulators and described as “powered products, controlled by the user with e.g. a joystick, to replace arm, hand and finger functions, to reach, grasp and move objects in space” [12].

In 2020, a variety of 14 dynamic arm supports and robotic arms were commercially available from three different suppliers in The Netherlands. Dutch health insurances reimburse dynamic arm supports and robotic arms, which are relatively expensive devices, under the Health Care Insurance Act (Section 1.4 AT devices, Articles 2.6 and 2.12). In The Netherlands, annually more than 2 million euro is spent on these devices [15]. These direct costs are based on the relatively small proportion of clients receiving such a device. In terms of cost-effectiveness, the direct costs need to be compared against societal impact, e.g. consumption of care and paid productivity; not receiving (appropriate) dynamic arm supports or robotic arms may lead to waste of resources through abandoned devices, overconsumption of care and underrepresentation of the target population in paid productivity. The use of an appropriate dynamic arm support or robotic arm is expected to lead to improved user satisfaction with the AT, a higher QoL, less care consumption and reduced productivity loss [2, 16, 17].

Internationally, little is known about the (cost-)effectiveness of dynamic arm supports and robotic arms. A decrease in formal and informal care was reported in a study with seven JACO ${}^{\text{\textregistered}}$ robotic arm users in Canada [18], who also indicated that JACO ${}^{\text{\textregistered}}$ enabled working or studying. Additionally, decreased need for assistance i.e. during self-care activities was reported for various robotic arms in clients with various diagnoses [19] and clients with Duchenne muscular dystrophy [20]. The only study that structurally assessed the effect of the provision of a robotic arm on care in clients with various diagnoses showed that time spent by informal caregivers decreased in five of the ten new users. Time spent by formal caregivers did not decrease in seven of the ten users, but did not increase either [19]. With regard to the effects of dynamic arm supports, a previous study in users with various diagnoses showed great variation in effects. In half of the cases, clients experienced a moderate benefit on the ability to perform ADL with the selected dynamic arm support, but it was concluded that more added value seemed possible [13].

Various studies have shown that dynamic arm supports and robotic arms are not used to their full potential [2, 13, 16]. A number of bottlenecks have been identified in the process of providing dynamic arm supports and robotic arms, which is currently not standardised. These include:

1.
Not all clients who might benefit from the use of AT know about their existence or how they could initiate the provision process. Care professionals do not have sufficient knowledge about the (range of) AT devices, and are not always aware of all products’ features, functions, and functionalities.
2.
The cooperation between client, supplier and care professional is hampered, and the match between AT device, client and environment is often not adequate. Due to the current effectuation of the Dutch legislation and regulations, suppliers and care professionals are limited in their ability to provide the most suitable device based on the International Classification of Functioning, Disability and Health (ICF), inter alia, as finding and funding a suitable AT device is hindered by different reimbursement schemes.
3.
The delivery period takes quite (and sometimes in the case of rapidly progressive diseases too) long.
4.
Clients’ needs and abilities change over time, which is hard to anticipate and adjusting devices is only possible to a limited extent. Professionals are not facilitated to realise a trial period or to select and deliver another device if evaluation shows that the current is insufficiently effective.
5.
The amount of (additional) training and aftercare as well as required inter-professional cooperation is restricted.
6.
Health insurance companies that reimburse a dynamic arm support or robotic arm oblige suppliers to evaluate the provision. However, evaluations provide only limited insight into the added value of the AT [2, 13, 16].

In 2018, two stakeholder meetings with clients, occupational therapists, suppliers and researchers confirmed these bottlenecks. Additionally, these stakeholders identified further bottlenecks in the way dynamic arm supports and robotic arms are financed leading to difficulties in getting devices reimbursed, and a lack of knowledge regarding the devices’ (cost-)effectiveness. It was concluded that changes are required on micro-, meso- and macro level, including regulations and working processes [21].

These findings are in line with the problems described by the Netherlands Patients Federation regarding the provision of AT in general [22]. In international scientific literature, hardly any information is available about bottlenecks in the selection and provision of dynamic arm supports and robotic arms. Kumar reports difficulties clients using dynamic arms supports and robotic arms encounter in daily life situations, e.g. barriers in mobility due to increased width of the wheelchair. Such difficulties could have been tackled by carefully selecting devices [23].

In order to determine the effectiveness of AT devices, it is essential that the provision process runs smoothly, resulting in an optimal match between the individual clients’ needs and abilities, the physical and social context, and AT devices [24]. Not optimally providing AT means that resources are not (optimally) used. Since clients’ problems are not solved, they are not able to function optimally, experience a diminished QoL and may unnecessarily utilise care or search for alternative solutions [13]. Optimizing the provision process as a central part of this study is important, as a well-performed provision process that takes into account all relevant factors for the successful provision of this type of AT devices is a prerequisite for determining the effectiveness of the devices themselves [25]. If an optimal match is not realised and there is no proper instruction and training, resulting in no or suboptimal use, no optimal effect of the device can be achieved for the client in question. The objective of this study is to OptiMize the provision process of AT devices for impaired upper extremity (ARM and hand) function and to evaluate its (cost-)effectiveness compared with care as usual (OMARM study).

The following hypotheses will be tested: an optimized provision process leads to more user satisfaction with the AT device and the related service delivery, fewer complaints of the upper extremity, less difficulties in performing meaningful activities of daily living, improved participation and a better QoL, and less societal consumption when compared with usual care. Moreover, an optimized provision process is expected to be cost-effective compared with usual care.
2. Methods

2.1 Study design

The study is registered at the Dutch trial register with number NL8087 and is designed as a prospective, quasi-experimental study with non-randomized, consecutive inclusion of a control group in year 1 and of an intervention group in year 2. Participants in the control group will receive a dynamic arm support or robotic arm through the common, non-standardized provision process (care as usual), whereas the AT provision of participants in the intervention group will be conducted according to the standardized protocol of the optimized process, which will be developed in co-creation with all stakeholders in the first year of the study [26, 27, 28]. Cost-effectiveness is assessed by questionnaires, which are sent before provision (baseline), and 3, 6 and 9 months after provision. For an overview of the study design, see Fig. 1.

Figure 1.

Flow diagram of the OMARM study. DAS $=$ dynamic arm support, RoA $=$ robotic arm.

The study will be performed in five phases:

Phase 1:

Baseline measurement and effect measurements in the control group (receiving care as usual), paralleled by the development of a protocol for the optimized provision process of AT devices for upper extremity function.

Phase 2:

Implementation of this protocol in the care process of participating centres and suppliers.

Phase 3:

Baseline measurement and effect measurements in the intervention group (receiving care according to the newly developed protocol).

Phase 4:

Data-analysis and economic analysis to ascertain (cost-)effectiveness of the optimized provision process compared with care as usual.

Phase 5:

National implementation of the protocol when the optimized provision process is (cost-)effective.

2.2 Study population

The study population consists of adult clients with impaired upper extremity function due to muscle paresis or paralysis (e.g. neuromuscular disease, stroke, MS, ALS, or cervical spinal cord injury). Inclusion criteria are: 1) aged 18 years or older; 2) considering a dynamic arm support or robotic arm to support limitations in upper extremity function, defined as trying out a dynamic arm support or robotic arm with one of the Dutch suppliers; and 3) falling under Dutch legislation with respect to care and the provision of AT devices. Exclusion criteria are: 1) experiencing limitations in hand, but not in arm function and 2) not being able to complete a Dutch-language questionnaire or to respond to the questions orally. All adult clients who meet the inclusion criteria and receive care in one of the participating centres or supplier companies (Amsterdam UMC location Academic Medical Center, Radboudumc, Siza, Assistive Innovations, Focal Meditech, and Sarkow) in the period from December 2019 to March 2021 (control group) and April 2021 to March 2022 (intervention group) are eligible for participation in the study, even if they ultimately do not obtain or use such a device.

2.3 Sample size calculation

A scoping review was carried out on studies in which user satisfaction with AT devices and the associated services was measured as an outcome using the (Dutch version of the) Quebec User Evaluation of Satisfaction with assistive Technology ((D-)QUEST). From this review, studies on different types of devices were evaluated [29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40], specifically devices that compensate or replace the upper extremity function [41, 42, 43, 44, 45, 46, 47]. The reviewed studies showed averages and standard deviations from 4.3 ( $\pm$ 0.45) to 3.28 ( $\pm$ 0.82) for device satisfaction and from 4.21 ( $\pm$ 0.57) to 4.6 ( $\pm$ 0.70) for service satisfaction if only upper extremity related devices were considered, and averages and standard deviations from 3.9 ( $\pm$ 0.15) to 3.25 ( $\pm$ 1.04) (device) and 3.6 ( $\pm$ 0.18) to 4.1 ( $\pm$ 1.10) (service) concerning other devices. Based on an expected change in the primary outcome measure of 0.5 and assuming a standard deviation on the average value of 0.7 [43, 46], a power of 80%, and a significance level of alpha $=$ 0.05 in the case of a two-sided independent $t$ -test, each group must consist of 40 participants. Taking into account an expected loss to follow-up of 20%, 96 participants are needed: 48 in the control group and 48 in the intervention group [48].

2.4 Procedures

2.4.1 Recruitment and consent

Physicians, occupational therapists and suppliers’ advisors will ask all eligible clients for their interest to participate and provide brief oral information about the study, in an attempt to prevent volunteer or referral bias. After confirmation of their interest, clients will receive the comprehensive patient information letter and can ask additional questions. Subsequently, after one week of reflection time, they can definitively register with a researcher and sign an informed consent form.

2.4.2 Data collection

After obtaining informed consent, for each participant a record will be created in Castor (Castor EDC, Amsterdam, The Netherlands) [49]. Participants in the control and intervention group will be asked to fill in an online questionnaire (sent through Castor) regarding clients’ satisfaction with the AT and related services, complaints of the upper extremity, restrictions in activities, QoL, medical consumption and societal cost four times during the study period (at baseline, and 3, 6 and 9 months after provision of the device); each questionnaire takes about 30 to 45 minutes. Further, participants will fill in forms for monitoring the provision process, taking about two minutes each.

2.4.3 Care as usual

The process of providing AT for impaired upper extremity function within care as usual is described in van der Heide et al. [50], based on data of 16 persons who were retrospectively interviewed on their service delivery process and participated in a focus group session with advisors of a supplier. Important aspects are summarized below; a comprehensive overview of all steps with more detailed information is presented in van der Heide et al. [13, 50].

Devices are mainly provided to clients living at home and to a lesser extent to people in residential care facilities or to support work processes in the context of the Employee Insurance Agency.

The following people are generally involved in the provision process: clients, possibly family members, advisors of AT suppliers, care professionals such as occupational therapists and rehabilitation physicians. Together with clients, the latter play a major role in the first three steps of the process as detailed below. In the subsequent steps, the role of healthcare professionals decreases when clients contact an AT supplier’s advisor.

Step 3b: Step 1:
Identifying a problem. Clients signal problems related to impaired upper extremity function and limitations in the performance of activities of daily living and take different routes (i.e. contact an occupational therapist or suppliers directly).
Step 2:
Request for care. According to general Dutch guidelines for the provision of AT [27, 28] this includes an anamnesis to identify impairment, disabilities, participation restrictions, contextual factors, and prognosis. In current practice, this information might be collected by occupational therapists, but is not shared with suppliers’ advisors with whom a try-out is arranged, so that comparable information is collected (again) by AT suppliers’ advisors during step 4.
Step 3a:
Type of solution. Professionals, mostly occupational therapists and rehabilitation physicians, and clients jointly decide whether there is potential benefit in using an AT device for a specific goal related to impaired upper extremity function. In practice, goal setting is repeated by suppliers’ advisors, guided by a short protocol. This is necessary, since not every healthcare professional is well informed about the potential of dynamic arm supports and robotic arms. Conversely, once it has been decided that clients can benefit from such a device, other available options, such as eating devices, are only considered to a limited extent [13, 50].
Step 3b:
Programme of requirements. Step 3 and 4 are often carried out simultaneously in practice. The requirements are determined by suppliers’ advisors, primarily by addressing their tacit knowledge and guided by a protocol. Furthermore, Dutch health insurance policy imposes the rule that the most simple and adequate solution is procured. Additionally, client’s health insurance company does not always procure each type of dynamic arm support without financial consequences for the client.
Step 4:
Selecting, trying out and deciding. This is mainly done while trying out different types of devices (e.g. electronic, passive, robotic arm) in clients’ homes. Selecting an appropriate device is part of a documented protocol based on the ICF [50] and ISO9999 [12], and partly on tacit knowledge. Usually, a fit takes 30 to 60 minutes in the presence of a supplier’s advisor and an occupational therapist. During this time clients can experience what it feels like to use the AT devices and often the activities that clients would like to have supported are tried out, sometimes simulated. The final choice for a device is made in consultation between client, supplier and care professional present. However, part of the clients feel that they have had too little say in the final choice.

During the selection and testing of devices, clients receive a lot of information and might experience fatigue, so that it is not always clear afterwards which choice they have made and why. Trying out a device for a short period in the home environment is currently not possible, due to limited financial resources.
Step 5:
Delivering. In general, a device is delivered between two and five months after a choice has been made. A brief instruction is given by the supplier and a manual is supplied.
Step 6:
Using. In case of problems, clients should contact the supplier to find a solution. Sometimes clients solve problems themselves, sometimes they keep an AT device that does not function optimally. Training in use is not a structural part of the provision process, yet.
Step 7:
Evaluation. Structural evaluation is not a standard part of the current provision process.

A summary of the bottlenecks in the provision process of AT devices for impaired upper extremity function [50] is provided in Fig. 2.

Figure 2.
Summary of aspects that could be optimized in the provision process of dynamic arm supports (based on van der Heide et al. [50]).

2.4.4 Intervention (optimized process)

In the intervention group, a new working method explicated in a module for the provision of AT for impaired upper extremity function will be applied. This module will follow all steps of the generic Dutch quality framework for the provision of AT [26].

Step 7: Step 1:
Identifying a problem. The module will provide tools to create awareness for AT to support limited upper extremity function and information about persons to contact for advice and provision.
Step 2:
Request for care. The module will include, among other things, an overview of assessments to identify clients’ problems that might be solved by AT for upper extremity function.
Step 3:
Plan of care. Based on insight into possible solution directions (e.g. dedicated devices, assistance dogs, and environmental control devices), it will be determined whether a dynamic arm support or robotic arm is a possible solution. An overview of currently available products will be established and this information will be made accessible for all stakeholders.
Step 4:
Selecting, trying out and deciding. Functionalities and properties of the different arm supports and robotic arms will be made transparent in a standardised manner. A best practice will support the process of selecting, trying out and deciding. The module will provide insight into factors to be considered when selecting an appropriate device (product characteristics in relation to activities, daily use and the environment in which the client operates).
Step 5:
Delivering. Clients will be provided with clear instructions about (the use of) their device, e.g. by using film material.
Step 6:
Using. Involved healthcare professionals (occupational therapists/physiotherapists) will be given a greater role in training and encouraging the use of the device. The module will contain agreements on follow-up and maintenance when problems arise.
Step 7:
Evaluation. This step will comprise a structural evaluation of the effect of the AT provision in relation to the plan of care and treatment objective, making use of existing instruments and methodologies as much as possible. The module will be in line with current Dutch legislation and regulations.

Table 1
Measurement instruments OMARM study

Theoretical concept Measurement instrument Items/content Answer/response options Base-line Follow-up at 3, 6 and 9 months

ICF – health condition Intake Diagnosis incl. disease progression Multiple choice x

ICF – personal factors Intake Age, gender, highest level of education, marital status etc. Input field and multiple choice x

ICF – environmental factors D-QUEST [54, 55] Generic tool for all populations and types of AT devices 8 questions about characteristics of AT devices, 4 questions about the associated services Ordinal 5-point scale (not satisfied at all – very satisfied) total score 12–60, ratio scale x

ICF – environmental factors KWAZO [57] 7 questions about accessibility, information/clarity, coordination, expertise, efficiency and having a voice Ordinal 3-point scale (insufficient, sufficient or good), total score 7-21, ratio scale x

ICF – body functions DASH- DLV [58] 5 items on symptoms/complaints upper extremity Ordinal 5-point scale (no problem at all – impossible; no restrictions at all – impossible; no pain – extreme pain; completely disagree – completely agree)total score 0–100, ratio scale x x

ICF – activities DASH-DLV [58] 25 items on restrictions in activities

ICF – activities IPPA [60] 1–7 self-defined items on problems in performing activities of daily living that are important to the individual client Ordinal 5-point scale relevance (not relevant at all – very relevant) and perceived difficulty (no difficulty at all – too much difficulty to perform the activity) difference score between baseline and follow-up x (base-line) x (follow-up)

ICF – participation iMTA PCQ [63] See below (Economic evaluation) x x

QoL EQ-5D-5L [61] 5 items about mobility, self-care, usual activities, pain/discomfort, and anxiety and depression & 1 items on health Ordinal 5-point scale and a 0–100 Visual Analogue Scale x x

Economic evaluation iMTA MCQ [62] Type and number of appointments with health care professionals in the last 3 months, type & number of received treatment, support or other social and health care interventions Input fields x x

Economic evaluation iMTA PCQ [63] Time spent with (unpaid) work, absenteeism and support of informal caregivers for unpaid work/productivity during the last 4 weeks Input fields x x

D-QUEST $=$ Dutch version of the Quebec User Evaluation of Satisfaction with assistive Technology; KWAZO $=$ Kwaliteit van zorg; tevredenheid over de verstrekking van een hulpmiddel (Quality of care; satisfaction with the provision of Assistive Technology); DASH-DLV $=$ Disabilities of the Arm, Shoulder and Hand $-$ Dutch Language Version; IPPA $=$ Individually Prioritised Problem Assessment; EQ-5D-5L $=$ 5-dimension 5-level EuroQol (the 5-level EQ-5D version); iMTA PCQ $=$ Productivity Cost Questionnaire; iMTA MCQ $=$ Medical Consumption Questionnaire.

2.5 Study variables (primary and secondary outcomes)

Theoretical concept	Measurement instrument	Items/content	Answer/response options	Base-line	Follow-up at 3, 6 and 9 months
ICF – health condition	Intake	Diagnosis incl. disease progression	Multiple choice	x
ICF – personal factors	Intake	Age, gender, highest level of education, marital status etc.	Input field and multiple choice	x
ICF – environmental factors	D-QUEST [54, 55]	Generic tool for all populations and types of AT devices 8 questions about characteristics of AT devices, 4 questions about the associated services	Ordinal 5-point scale (not satisfied at all – very satisfied) total score 12–60, ratio scale		x
ICF – environmental factors	KWAZO [57]	7 questions about accessibility, information/clarity, coordination, expertise, efficiency and having a voice	Ordinal 3-point scale (insufficient, sufficient or good), total score 7-21, ratio scale		x
ICF – body functions	DASH- DLV [58]	5 items on symptoms/complaints upper extremity	Ordinal 5-point scale (no problem at all – impossible; no restrictions at all – impossible; no pain – extreme pain; completely disagree – completely agree)total score 0–100, ratio scale	x	x
ICF – activities	DASH-DLV [58]	25 items on restrictions in activities
ICF – activities	IPPA [60]	1–7 self-defined items on problems in performing activities of daily living that are important to the individual client	Ordinal 5-point scale relevance (not relevant at all – very relevant) and perceived difficulty (no difficulty at all – too much difficulty to perform the activity) difference score between baseline and follow-up	x (base-line)	x (follow-up)
ICF – participation	iMTA PCQ [63]	See below (Economic evaluation)		x	x
QoL	EQ-5D-5L [61]	5 items about mobility, self-care, usual activities, pain/discomfort, and anxiety and depression & 1 items on health	Ordinal 5-point scale and a 0–100 Visual Analogue Scale	x	x
Economic evaluation	iMTA MCQ [62]	Type and number of appointments with health care professionals in the last 3 months, type & number of received treatment, support or other social and health care interventions	Input fields	x	x
Economic evaluation	iMTA PCQ [63]	Time spent with (unpaid) work, absenteeism and support of informal caregivers for unpaid work/productivity during the last 4 weeks	Input fields	x	x

In selecting appropriate outcome measurements for a cost-effectiveness analysis of AT devices, the different components of the ICF (functions, activities and participation, and external factors) [26, 51] and the different dimensions of QoL were taken into account [52]. These were operationalised as 1) satisfaction with the AT device and the related services, 2) experiencing fewer problems in upper extremity functions and functioning and in performing daily activities that are important for the individual client, 3) increased participation and QoL and 4) less dependence on (informal) care [24, 53] (see overview in Table 1).

2.5.1 Primary outcome measure

Clients’ satisfaction with the device and the associated services will be measured using the D-QUEST [54, 55]. The D-QUEST is a Patient Reported Experience Measure (PREM), at the ICF level ‘environmental factors’. The D-QUEST comprises eight questions about certain characteristics of AT devices and four questions about the associated services on an ordinal 5-point scale (from ‘not satisfied at all’ to ‘very satisfied’). The instrument leads to two sub-scores (device satisfaction and service satisfaction) and a total score (range 12 to 60). In most studies, the ordinal variables are converted to numbers (1 to 5) and then used as variables at ratio level, with averages and standard deviations. The D-QUEST is a generic tool for all populations and all types of AT devices. Validity and reliability of the (D-)QUEST have been qualified as “good” in clients using different AT, including orthoses [56].

2.5.2 Secondary outcome measures

Clients’ satisfaction with the process of device delivery will be measured with the “Kwaliteit van zorg; tevredenheid over de verstrekking van een hulpmiddel [Quality of care; satisfaction with the provision of Assistive Technology]” (KWAZO) [57]. The KWAZO comprises seven questions concerning accessibility, information/clarity, coordination, expertise, efficiency and having a voice, which can be answered by ‘insufficient’, ‘sufficient’ or ‘good’. Its psychometric properties were considered “decent”, with a high user-friendliness in clients using a variety of AT [57].

Restrictions in upper extremity activities and complaints will be measured with the Disabilities of the Arm, Shoulder and Hand - Dutch Language Version (DASH-DLV) [58]. The DASH comprises 30 items on the ICF level ‘body functions and activities’, concerning the degree of complaints or impairments in the complete upper extremity during the previous week. Each item is answered on a 5-point-scale (from ‘no problem at all’ to ‘impossible’). Based on assessment of its psychometric quality in clients with different one-sided upper limb disorders, the DASH-DLV has been qualified as “a valid and reliable instrument”, with excellent internal consistency, satisfactory test-retest reliability, and a concurrent validity of 81% compared with the COPM [59].

The extent to which the AT device contributes to reducing or eliminating problems in performing activities of daily living important to the individual client, will be measured by the Individually Prioritised Problem Assessment (IPPA) [60]. The IPPA is a Patient Reported Outcome Measure (PROM) at the ICF level ‘activities’. During the baseline measurement, participants can list up to seven activities and rate their relevance and perceived difficulties while performing them, each on a 5-point scale (from ‘not important at all’ to ‘very important’, and from ‘not difficult at all’ to ‘too difficult to perform the activity’). During follow-up, participants will rate the perceived difficulties again. The IPPA has been characterized as a user-friendly and valid instrument [28].

QoL will be measured using the 5-dimension 5-level EuroQol (EQ-5D-5L) [61]. The questionnaire contains five questions about mobility, self-care, usual activities, pain/discomfort, and anxiety and depression scored on a 5-point scale indicating the degree of problems with each dimension and one question about experienced health scored on a 0-100 visual analogue scale. The EQ-5D-5L provides a profile of QoL of which utilities between 0 and 1 will be derived using Dutch tariffs. Using the utilities, the quality-adjusted life years (QALY) will be calculated by correcting the life years for the quality of these life years.

Participants’ medical consumption, productivity and societal costs will be measured with the iMTA Medical Consumption Questionnaire (MCQ) and Productivity Cost Questionnaire (PCQ) [62, 63], which have been tailored for the purpose of this study. In the iMTA MCQ, participants specify the number of appointments made with various healthcare and related professionals in the last three months, as well as the type and number of received treatments, support or other social and health care interventions. The iMTA PCQ ascertains the time spent with (unpaid) work and absenteeism, and addresses the ICF participation level. The instrument is largely based on formerly validated questions and was evaluated as feasible and understandable [64]. Its psychometric properties have been assessed as “good” in clients with musculoskeletal disorders [65].

2.6 Monitoring the provision process

To objectify the differences between care as usual and intervention, the provision process will be monitored. Participants will receive forms with questions on what date, with which care provider(s), and for what purpose (e.g., fitting, evaluation) an appointment took place, which tools (assessments, checklists, etc.) were used, how long the appointment lasted and where (e.g. rehabilitation centre, free practice, at the participant’s home) it took place. Participants can add any further comments and are asked to fill in one form each time they contact professionals during the AT provision process. The evaluation forms can be sent to the researchers by stamped return envelopes.

2.7 Data management

Data will be collected via and stored in Castor, a secure electronic data capture system [49]. Anonymised data and metadata will be made findable and available to third parties for reuse via Figshare, including the creation of a Zuyd DOI.

2.8 Statistical analysis

Demographic and background data will be displayed using descriptive statistics. If matching is not possible, propensity score matching will be applied using logistic regression analysis (with confounders as a dependent variable and control/intervention group as independent variables). Subsequently, an ANOVA repeated measures will be applied (with Post-hoc Bonferroni) when the entire dataset is available, based on the “intention to treat” principle. SPSS version 26 will be used for the statistical analyses.

2.9 Economic evaluation

The economic evaluation will be carried out in accordance with Dutch guidelines [66] and is a combination of a cost effectiveness analysis (CEA) and a cost utility analysis (CUA) from a societal perspective with a time horizon equal to the follow-up of the impact study of 9 months. In the CEA, the outcome of interest is satisfaction with the device and related services as assessed with the D-QUEST. Primary outcome measure for the CUA is QALY gained as measured with the EQ-5D-5L.

Costs will be calculated bottom-up. The valuation of the costs will be based on the updated Dutch manual for health care cost research [67], containing standardised cost prices.

As cost data are often skewed, non-parametric bootstrapping will be done using Microsoft Excel (Microsoft Corporation, Washington, USA) to determine statistical differences in costs between the intervention and control group. To be able to make a statement about the efficiency of the new protocol, the Incremental Cost-Effectiveness Ratio (ICER) is calculated. The ICER combines the difference in costs and effects of two treatment alternatives into one measure using the following formula:

ICER $=$ (cost of intervention group – cost of control group) / (effect in intervention group – effect in control group), where effect is expressed as improvement in D-QUEST for the CEA and QALY gained for the CUA. The uncertainty surrounding the ICER due to sampling variation will be analysed using non-parametric bootstrapping. The resulting 95% confidence interval will then be plotted in a cost-effectiveness plane. This is a graph showing the incremental cost on the Y-axis and the incremental effect on the x-axis. To help policymakers in the Netherlands, the maximum cost of an intervention in relation to health gains, expressed in QALYs (Willingness to Pay (WTP)), has been determined. In the Netherlands, society is prepared to pay a maximum of €80,000 per QALY per year, depending on the severity of the disease. The results of the current research will be expressed in a Cost-Effectiveness Acceptability Curve (CEAC) presenting the probability of the new provision process of AT being more cost-effective in terms of satisfaction and QALYs gained compared with the current/usual provision process of AT for different WTP thresholds. In addition, a number of one-way and multi-component sensitivity analyses will be performed on important cost parameters to investigate the robustness of the results.

2.10 Dissemination policy

The protocol will be submitted to the National Health Care Institute. It will be digitalized and published via Vilans, the national Centre of Expertise for Long-term Care in the Netherlands. The results and recommendations will be reported to health insurers via Zorgverzekeraars Nederland (ZN), the umbrella organization of eleven health insurers in the Netherlands. In addition, the results of the study will be published in the form of scientific articles for the Dutch associations of Occupational Therapy (Ergotherapie Nederland (EN)) and the Dutch Society of Rehabilitation Medicine (DSRM) (Nederlandse Vereniging van Revalidatieartsen (VRA)), and international scientific journals. An information report for patient associations and dissemination via social media and presentations will also be arranged.

After development, all parties involved will issue a news report on the new protocol. A presentation on the protocol will be submitted for the annual conferences of EN and VRA. Furthermore, a postgraduate course will be developed which will be provided as a pilot together with the suppliers to healthcare professionals of the healthcare centres involved.

3. Discussion

The objective of this study is to optimize the provision process of AT devices for impaired upper extremity function and to evaluate its (cost-)effectiveness compared with care as usual in patients with muscle paresis or paralysis due to a variety of (neuro-muscular) disorders. In the context of efficiency research, the process of providing AT devices has rarely been studied before. This study on AT devices for upper extremity function could be seen as an example with transferable and generalizable results for other types of complex AT [21]. The effectiveness of an AT device cannot be examined irrespective of clients’ personal characteristics, their limitations, desired and necessary activities and participation and their social and physical context [24]. The complexity of the provision process and in particular the matching of client and device implicates that many different factors influence satisfaction with the process and device. Moreover, the study deals with a very diverse target group. The (existing) diversity of the target group and the broad applicability of the protocol (high external validity) have been taken into account by focusing on the problem in terms of functioning (ICF-based) instead of focusing on certain diagnoses or a clinical measure for upper extremity function. The high external validity will increase the ability to transfer outcomes of the study to daily practice after implementation of the protocol at the end of the study. The risk of confounders and different forms of possible bias are taken into account.

The risk of information bias is considered small as data is collected prospectively and will be equal in the control group and intervention group. Furthermore, by separating the control and intervention groups in time, contamination bias is counteracted. In addition, the involvement of professionals (therapists and suppliers) is more likely to be equal and consistent for the participants in the intervention and control group, as the same professionals are involved in both groups. The inclusion of participants takes place at a comparable time of the year, so that seasonal effects are equally distributed over both groups. In order to gain insight into the possible presence of selection bias, basic characteristics of the participants are mapped.

Changes in the health care system may affect the provision of AT devices, resulting in no longer comparable situations for the control group and intervention group. However, changes in legislation and regulations at macro level are not expected from the National Health Care Institute within the project period (personal communication). Another source of bias could be that healthcare providers spend more time on provision in the intervention group, leading to more service satisfaction. Being able to take more time to make a good selection (fitting), and to provide instruction and training, however, these are essential elements of an optimized provision process and part of the intervention, which is also included in the cost-effectiveness analysis.

The client-centered selection of the outcome measures is a strength of the study. Client satisfaction is decisive for the use of a device and the prevention of abandonment. The subjective experience of the client is taken into account by means of the D-QUEST, which includes a question about the perceived effectiveness of the device. In addition, a selection of (user-centered) assessments is included, and the IPPA specifically addresses the question whether the AT device leads to fewer problems for users during the performance of activities that are important to them. The selected assessments are recommended by experts in the field of AT provision and economic analysis [68].

To conclude, various measures have been taken to adequately address the methodological challenges of the study described. Through an optimized provision process of AT devices for upper extremity function, implemented into daily practice by useful tools, and insights into its (cost-)effectiveness, the results of the OMARM study may contribute to the quality of care and service delivery for (potential) dynamic arm support and robotic arm users.

Author contributions

CONCEPTION: Uta R. Roentgen, Loek A. van der Heide, Ingrid E.H. Kremer, Huub Creemers, Merel A. Brehm, Jan T. Groothuis, Edith A.V. Hagedoren, Ramon Daniëls and Silvia M.A.A. Evers

PERFORMANCE OF WORK: Uta R. Roentgen, Loek A. van der Heide, Ingrid E.H. Kremer and Edith A.V. Hagedoren

PREPARATION OF THE MANUSCRIPT: Uta R. Roentgen, Loek A. van der Heide, Ingrid E.H. Kremer and Edith A.V. Hagedoren

REVISION FOR IMPORTANT INTELLECTUAL CONTENT: Huub Creemers, Merel A. Brehm, Jan T. Groothuis, Ramon Daniëls and Silvia M.A.A. Evers

SUPERVISION: Ramon Daniëls and Silvia M.A.A. Evers

Ethical considerations

The study protocol has been approved by the Medical Research Ethics Committee (MREC Z), protocol number METCZ20190108 on August 26 ${}^{\text{th}}$ , 2019. The study will be carried out in accordance with the principles of the Helsinki Declaration (2013) Good Clinical Practice (GCP), the Medical Research with People Act (WMO) and other guidelines, laws and regulations [69].

Footnotes

Acknowledgments

The authors would like to thank Romy Schenninck and Linda van den Bedem (Siza), Yolanda van den Elzen (Radboud umc), Mattanja Hoenderdos, Sarah el Markhous, Paul Coops (Dwarslaesie NL), Marit Dohmen, Corne van den Burg and Marieke Spreeuwenberg for their valuable contribution to the design of the OMARM study.

Several authors of this publication are members of the Netherlands Neuromuscular Center (NL-NMD) and the European Reference Network for rare neuromuscular diseases EURO-NMD.

The study is funded by ZonMw, the Dutch organisation for health care research and innovation, project number 853001107.

Conflict of interest

The OMARM project has been made possible by ZonMw, the Dutch organisation that stimulates health research and innovation, grant number 853001107. The authors have no conflicts of interest to report.

References

Winter

Schepelmann

Spottke

, et al. Health-related quality of life in ALS, myasthenia gravis and facioscapulohumeral muscular dystrophy. J Neurol. 2010; 257(9): 1473-1481.

Graham

Rose

Grunfeld

Kyle

Weinman

. A systematic review of quality of life in adults with muscle disease. J Neurol. 2011; 258(9): 1581-1592.

Janssen

Bergsma

Geurts

, de Groot

. Patterns of decline in upper limb function of boys and men with DMD: An international survey. J Neurol. 2014; 261(7): 1269-1288.

Benito-León

Manuel Morales

Rivera-Navarro

Mitchell

. A review about the impact of multiple sclerosis on health-related quality of life. Disabil Rehabil. 2003; 25(23): 1291-1303.

Spierziektencentrum [Centre for muscle diseases] [homepage on the Internet]. [cited 2020 Dec 5]. Available from: https://www.spierziektencentrum.nl/scholing/fellows/#:∼:text=Neuromusculaire%20ziekten%20(spierziekten)%20zijn%20aandoeningen,dan%20600%20verschillende%20ziektebeelden%20beschreven.

ALS Centrum Nederland [ALS Centre The Netherlands]. [homepage on the Internet]. [cited 2020 Dec 5]. Available from: https://www.als-centrum.nl/kennisplatform/epidemiologie-van-als/.

Hersenstichting [Brain Foundation]. [homepage on the Internet]. [cited 2020 Dec 5]. Available from: https://www.hersenstichting.nl/hersenaandoeningen; 2020.

SAFE Stroke Alliance for Europe. [homepage on the Internet]. [cited 2020 Dec 5]. Available from: https://www.safestroke.eu/wp-content/uploads/2017/12/SAFE_STROKE_NETHERLANDS.pdf

Nederlandse Vereniging van Revalidatieartsen [Dutch Association of Rehabilitation Physicians]. [homepage on the Internet]. Behandelkader Dwarslaesie [Framework for the treatment of spinal cord injury]. [cited 2020 Dec 5]. Available from: https://revalidatiegeneeskunde.nl/sites/default/files/attachments/Kwaliteit/Behandelkaders/behandelkader_dwarslaesie_-_2019-04-12_def.pdf.

10.

Volksgezondheid en zorg info [Public health and health care information] [homepage on the Internet]. [cited 2020 Dec 5]. Available from: https://www.volksgezondheidenzorg.info.

11.

Wyndaele

. Incidence, prevalence and epidemiology of spinal cord injury: What learns a worldwide literature survey? Spinal Cord. 2006; 44(9): 523-529.

12.

ISO 9999: 2016 standard.

13.

Van der Heide

. Dynamic arm supports; matching user needs and preferences with technology. Dissertation. Maastricht: Maastricht University; 2017.

14.

Assistive Technology Australia [homepage on the Internet]. Assistive Technology Guide - Upper Limb Accessories [cited 2020 Dec 5]. Available from: https://www.at-aust.org/items/2225.

15.

Zorginstituut Nederland. Gipdatabank [Database on AT devices] [homepage on the Internet]. Aantal gebruikers 2012–2016 hulpmiddelencategorie R: Hulpmiddelen t.b.v. arm-hand-vingerfunctie. 2018 [Number of users 2012–2016 AT product category R: AT devices for upper extremity function] [cited 2020 Dec 5]. Available from: https://www.gipdatabank.nl/databank?infotype=h&label=00-totaal&tabel=B_01-basis& geg=gebr&item=R.

16.

Van der Heide

Gelderblom

de Witte

. Effects and effectiveness of dynamic arm supports: A technical review. Am J Phys Med Rehabil. 2015; 94(1): 44-62.

17.

Janssen

Bergsma

Geurts

de Groot

. Patterns of decline in upper limb function of boys and men with DMD: An international survey. J Neurol. 2014; 261(7): 1269-1288.

18.

Beaudoin

Lettre

Routhier

Archambault

Lemay

Gélinas

. Long-term use of the JACO robotic arm: A case series, Disabil Rehabil Assist Technol. 2019; 14: 3, 267-275.

19.

De Witte

Gelderblom

van Soest

Dijcks

Goossens

Tilli

van’t Hoofd

van der Pijl

Wessels

Rutten-van Molken

MPMH

. MANUS: Een helpende hand. Een verkennende studie naar doelgroepen, indicatiecriteria, gebruik en aspecten van kosten-effectiviteit van de MANUS robotarm [MANUS: A helping hand. An exploratory study into target groups, indication criteria, use and aspects of cost-effectiveness of the MANUS robotic arm]. Hoensbroek, The Netherlands: IRv, Kenniscentrum voor Revalidatie en Handicap; 2000.

20.

Bach

Zeelenberg

Winter

. Wheelchair-mounted robot manipulators: Long term use by patients with Duchenne muscular dystrophy. Am J Phys Med Rehabil. 1990; 69: 55-59.

21.

Van der Heide

Roentgen

Wauben

Daniëls

. Rapport ontwikkeling en verstrekking van complexe hulpmiddelen (ZonMw) (2018, niet gepubliceerd). [Report development and provision of complex AT devices (unpublished)].

22.

NPCF. Hulpmiddelen: Weinig keuzevrijheid en lang wachten. [AT devices: Little choice and a long wait]. Utrecht: Patiëntenfederatie NPCF; 2015.

23.

Kumar

Phillips

. Use of mobile arm supports by people with neuromuscular conditions. JRRD. 2013; 50(1): 61-70.

24.

Gelderblom

de Witte

. The assessment of assistive technology outcomes, effects and costs. Technol Disabil. 2002; 14(3): 91.

25.

Wessels

. Ask the user. User perspective in the assessment of assistive technology. Dissertation. Maastricht: Universiteit Maastricht; 2004.

26.

Bestuurlijk Overleg Hulpmiddelen

VWS

. Generiek kwaliteitskader hulpmiddelenzorg. [Administrative Consultation on Assistive Technology Devices VWS. Generic quality framework for Assistive Technology care.] Den Haag; 2017. https://www.zorginzicht.nl/binaries/content/assets/zorginzicht/kwaliteitsinstrumenten/Generiek+Kwaliteitskader+Hulpmiddelenzorg.pdf.

27.

Nictiz. Procesbeschrijving Hulpmiddelenzorg. [Process description assistive technology care]. Den Haag; 2009.

28.

CG-Raad. Verslag van het Project Opstellen Richtlijnen voor Functiegerichte Aanspraak hulpmiddelen. RiFA. Fase 1 [Project Report Drafting Guidelines for the ICF-based provision of AT devices. RiFA. Phase 1]. Utrecht: CG-Raad; 2010.

29.

Wessels

de Witte

. Reliability and validity of the Dutch version of QUEST 2.0 with users of various types of assistive devices. Disabil Rehabil. 2003; 25(6): 267-272.

30.

Gasq

Acket

Caussé

Cantagrel

Combe

Cintas

Arné-Bes

. Validation of an helicoidal versus standard ankle-foot orthosis for patients with unilateral drop foot. Abstr Annals Phys Rehabil Med. 2018; 61S: E435–e557.

31.

Kozlowski

Fabian

Lad

Delgado

. Feasibility and Safety of a Powered Exoskeleton for Assisted Walking for Persons With Multiple Sclerosis: A Single-Group Preliminary Study. Arch Phys Med Rehabil. 2017; 98: 1300-1307.

32.

Magnusson

Ahlström

. Patients’ Satisfaction with Lower-limb Prosthetic and Orthotic Devices and Service delivery in Sierra Leone and Malawi. BMC HSR. 2017; 17: 102.

33.

Vincent

Routhier

Martel

Mottard

Dumont

Côté

Cloutier

. Field testing of two electronic mobility aid devices for persons who are deaf-blind. Disabil Rehabil Assist Technol. 2014; 9(5): 414-420.

34.

Caligari

Godi

Guglielmetti

Franchignoni

Nardone

. Eye tracking communication devices in Amyotrophic Lateral Sclerosis: Impact on disability and quality of life. Amyotroph Lateral Scler Frontotemporal Degener. 2013; 14: 546-552.

35.

Karmarkar

Collins

Kelleher

Cooper

. Satisfaction related to wheelchair use in older adults in both nursing homes and community dwelling. Disabil Rehabil Assist Technol. 2009; 4(5): 337-343.

36.

Hill

Goldstein

Gartner

Brooks

. Daily utility and satisfaction with rollators among persons with chronic obstructive pulmonary disease. Arch Phys Med Rehabil. 2008; 89: 1108-1113.

37.

Bergström

Samuelsson

. Evaluation of manual wheelchairs by individuals with spinal cord injuries. Disabil Rehabil Assist Technol. 2006; 1(3): 175-182.

38.

Goodacre

Turner

. An investigation of the effectiveness of the quebec user evaluation of satisfaction with assistive technology via a postal survey. BJOT. 2005; 68(2): 9.

39.

Chiu

CWY

Man

DWK

. The effect of training older adults with stroke to use home-based assistive devices. OTJR. 2004; 24(3): 113-120.

40.

Jedeloo

de Witte

Linssen

BAJ

Schrijvers

AJP

. Satisfaction with and use of assistive devices and services for outdoor mobility. Technol Disabil. 2000; 3: 173-181.

41.

Zuniga

. 3D Printed Antibacterial Prostheses. Appl Sci. 2018; 8: 1651.

42.

Joseph

Constant

Rickloff

Mezzio

Valdes

. A survey of client experiences with orthotics using the QUEST 2.0. Journal of Hand Ther. 2018; 31: 538e543.

43.

Eriksson

Jylli

Villard

Kroksmark

Bartonek

. Health-related quality of life and orthosis use in a Swedish population with arthrogryposis. Prosth Ortho Int. 2018; 42(4): 402-409.

44.

Pani

Piga

Barabino

, et al. Home tele-rehabilitation for rheumatic patients: Impact and satisfaction of care analysis. J Telemed Telecare. 2016; 23(2): 292-300.

45.

Hussain

Spagnoletti

Salvietti

Prattichizzo

. Toward wearable supernumerary robotic fingers to compensate missing grasping abilities in hemiparetic upper limb. Int J Robotics Res. 2017; 36(13-14): 1414-1436.

46.

Golea-Vasluian

van Wijk

Dijkstra

Reinders-Messelink

van der Sluis

. Adaptive devices in young people with upper limb reduction deficiencies: Use and satisfaction. J Rehabil Med. 2015; 47: 346-355.

47.

Chen

Teng

Lou

Lin

Chen

Yeung

. User satisfaction with orthotic devices and service in taiwan. PLoS ONE. 2014; 9(10): E110661.

48.

Lenth

. Java Applets for Power and Sample Size [Computer software]. 2006-2009. Available from: http://www.stat.uiowa.edu/∼rlenth/Power.

49.

Castor

EDC

. https://www.castoredc.com/.

50.

Van der Heide

Roentgen

van der Pijl

de Witte

. How could the service delivery process of dynamic arm supports be optimized? Technol Disabil. 2017; 29(3): 101-108.

51.

WHO. Towards a common language for functioning, disability and health: ICF. The International Classification of Functioning, Disability and Health. Geneva: WHO; 2002.

52.

Fuhrer

. Assessing the efficacy, effectiveness, and cost-effectiveness of assistive technology interventions for enhancing mobility. Disabil Rehabil Assist Technol. 2007; 2(3): 149-158.

53.

Andrich

Caracciolo

. Analysing the cost of individual Assistive Technology programmes. Disabil Rehabil Assist Technol. 2007; 2(4): 207-234.

54.

Demers

Weiss-Lambrou

Ska

. Development of the quebec user evaluation of satisfaction with assistive technology (QUEST). Assist Technol. 1996; 8(1): 3-13.

55.

Demers

Monette

Lapierre

Arnold

Wolfson

. Reliability, validity, and applicability of the Quebec User Evaluation of Satisfaction with assistive Technology (QUEST 2.0) for adults with multiple sclerosis. Disabil Rehabil. 2002; 24(1-3): 21-30.

56.

Bettoni

Ferriero

Bakhsh

Bravini

Massazza

Franchignoni

. A systematic review of questionnaires to assess patient satisfaction with limb orthoses. Prosthet Orthot Int. 2016; 40(2): 158-169.

57.

Dijcks

Wessels

de Vlieger

Post

MWM

. KWAZO, a new instrument to assess the quality of service delivery in AT provision. Disabil Rehabil. 2006; 28(15): 909-914.

58.

Palmen

van der Meijden

Nelissen

Köke

AJA

. De betrouwbaarheid en validiteit van de NL vertaling van de Disabilities of the Arm, Shoulder and Hand questionnaire (DASH). [The reliability and validity of the Dutch translation of the DASH]. Ned tijd fysio. 2004; 114(2). 30-35.

59.

Veehof

Sleegers

EJA

van Veldhoven

NHMJ

Schuurman

van Meeteren

NLU

. Psychometric qualities of the Dutch language version of the Disabilities of the Arm, Shoulder, and Hand questionnaire (DASH-DLV). J Hand Ther. 2002; 15(4): 347-354.

60.

Wessels

Persson

Lorentsen

Andrich

Ferrario

Oortwijn

van Beekum

Brodin

de Witte

. IPPA: Individually prioritised problem assessment. Technol Disabil. 2002; 14: 141-145.

61.

EuroQol Group. EQ-5D. Rotterdam: EuroQol Research Foundation; 2018.

62.

Institute for Medical Technology Assessment. Medical Consumption Questionnaire. Rotterdam: Erasmus Universiteit. www.imta.nl iMTA Productivity and Health Research Group. Handleiding iMTA Medical Cost Questionnaire (iMCQ). Rotterdam: IMTA, Erasmus Universiteit Rotterdam; 2018.

63.

Institute for Medical Technology Assessment. Productivity Cost Questionnaire. Rotterdam: Erasmus Universiteit. www.imta.nl iMTA Productivity and Health Research Group. Handleiding iMTA Productivity Cost Questionnaire (iPCQ). Rotterdam: IMTA, Erasmus Universiteit; 2018.

64.

Bouwmans

Krol

Severens

Koopmanschap

Brouwer

Hakkaart-van Roijen

. The IMTA Productivity Cost Questionnaire: A standardized instrument for measuring and valuing health-related productivity losses. Value Health 2015; 18: 753-758.

65.

Munk

Storheim

Smastuen

Grotle

. Measuring productivity costs in patients with musculoskeletal disorders: Measurement properties of the institute for medical technology assessment productivity cost questionnaire. Value Health 2019; 22: 1410-1416.

66.

Zorginstituut

. Richtlijn voor het uitvoeren van economische evaluaties [Guideline for carrying out economic evaluations]. Diemen: ZiNL; 2016.

67.

Hakkaart-van Roijen

van der Linden

Bouwmans

Kanters

Tan

. Kostenhandleiding: Methodologie van kostenonderzoek [Cost manual: Methodology of cost evaluations]. Diemen: ZiNL; 2016.

68.

Andrich

Caracciolo

. Analysing the cost of individual assistive technology programmes. Disabil Rehabil Assist Technol. 2007; 2(4): 207-234.

69.

Centrale Commissie Mensgebonden Onderzoek [Central Committee Research Involving Humans]. [homepage on the Internet]. [cited 2020 Dec 5]. Available from: https://www.ccmo.nl/.