IMBA-ICF linking by integrating consensus methods: How group consensus of experts can contribute to in-depth linking of instruments to the ICF

Abstract

BACKGROUND:

A frame of reference is needed to increase the comparability of vocational rehabilitation assessment instruments and the interpretation of their results. The International Classification of Functioning, Disability and Health (ICF) is a relevant framework, and when linking rules are used, items from existing assessment instruments can be linked to the appropriate categories as described in the ICF.

OBJECTIVE:

To develop an adapted linking methodology in which experts are involved by means of the application of consensus methods and to transfer this result in a step-by-step set of guidelines, supporting researchers and professionals, linking complex instruments to the ICF.

METHODS:

The main researcher developed the initial linking of the Integration von Menschen mit Behinderungen in die Arbeitswelt (IMBA) to the ICF by rigorously applying the refined ICF linking rules. To validate this linking, the Delphi and nominal group technique was integrated through different steps, and experts were involved in the process. The method section describes the linking process chronologically with focuses on the used approach, the involvement of experts, and the processing of the output.

RESULTS:

The results are presented in a 7-step set of guidelines describing the chronological process from the initial to the validated linking. These guidelines describe the core elements in the application of the linking rules and consensus methods in a manual for researchers who are interested in linking complex instruments to the ICF by involving experts.

CONCLUSIONS:

The Delphi and nominal group technique can be successfully integrated in the linking process, making it possible to involve experts in linking complex instruments to the ICF. A homogeneous composition of the expert panel in terms of knowledge, a heterogeneous composition in terms of setting, a rigorous and repeated application of the linking rules, and structured processing of the output are essential to achieve a valid linking.

Keywords

Delphi technique nominal group technique linking rules international classification of functioning disability and health return to work

1 Introduction

Return to Work (RTW) has become a policy priority from an individual and societal perspective, since research results have shown a strong association among work, health, and well-being [1]. The societal cost of work disability is still high, with Organization for Economic Co-operation and Development (OECD) countries spending on average 2% of their Global Domestic Product (GDP) on disability benefits [2]. However, the practice of RTW is facing important challenges: the need for a conceptual framework and common language among stakeholders; the assessment and documentation of individuals’ work abilities; the documentation of job demands; and the illustration of the balance between individuals’ abilities and the work demands, taking into account work-related contextual factors [1 –4]. The Belgian Centre of Knowledge in Work incapacity (CKWC) of the National Institute for Health and Disability Insurance (NIHDI) is also facing these challenges in Belgium’s national policies. The CKWC supports, among other things, research that provides insights into (1) the prevention of work disability, (2) the RTW process, and (3) the provision of adapted support for people who are confronted with work disabilities. Responding to the need for a conceptual framework and common language, The CKWC promotes the International Classification of Functioning, Disability, and Health (ICF), since this was developed to provide a reference framework and to standardize communication among patients, healthcare providers, policymakers, and researchers. The extensive set of ICF categories makes it possible to describe individuals’ functional (dis)abilities [5 –7]. In the context of RTW and disability evaluation, there is increasing agreement internationally to use ICF as the reference framework [8 –10]. However, a complete set of Vocational Rehabilitation (VR) concepts has neither been defined nor covered by the ICF (e.g. work capacity, content of work, work tasks, and work organization) [11, 12]. Therefore, the CKWC is looking for specific work-related assessments and instruments that are relevant in this context. The “Integration of Menschen mit Behinderungen in die Arbeitswelt” (IMBA) or the “Instrument for Specialists in Job Rehabilitation and Integration,” is specifically a work-related documentation tool that was developed in 1996 on behalf of the Ministry of Labor and Social Affairs of the German federal government. Since then, the Institute for Quality assurance in Prevention and Rehabilitation (IQPR) has continued the development. The IMBA is based on profile comparison and specifically designed to support vocational rehabilitation, vocational integration, and the prevention of work disability. Using a standardized set of 70 function-, activity-, and context-related main items, both patients’ capacity profiles and each job’s requirement profile are created. These profiles are based on the same scoring system, using a combination of an ordinal scale ranging from 0 to 5 (the extent of abilities or requirements) and a dichotomous scale (presence or absence of an ability or requirement). Therefore the interaction between a job’s demands and an individual’s capacity becomes clear, and the competences to return to work after a trauma or disease are visualized. Consequently the opportunities for RTW and the necessary support can be determined (e.g. job training, adaptation of tasks and/or adaptation of the working environment) [13 –15]. Since IMBA contributes to the challenges of assessing work capacity, work demands, and the balance between them, the CKWC formulated the question to link the IMBA to the ICF. This linking had never been performed before. This paper discusses the linking methodology in which experts were involved through the integration of consensus methods.

1.1 The ICF linking rules

A common frame of reference is needed to increase the comparability of instruments and the interpretation of their results. Health information is collected in various ways. For instance, information about the same domain (e.g., work ability) can be collected from different perspectives and through different types of assessments. Each instrument uses a different approach, a different unit of measurement, and a different scale to map its construct. Because the ICF is an internationally accepted frame of reference and serves as a common language within health care, the interest and importance of linking instruments to the ICF has grown in recent years, but it still requires a reliable linking system [16 –18]. Linking rules provide a clear roadmap to link items from existing assessment instruments to the appropriate categories described in the ICF.

ICF linking rules (Table 1) were updated in 2016 [18], building on previously published linking rules from 2002 and 2005 [16, 17], and consisted of ten rules, which need to be followed chronologically. The linking rules were refined on the basis of experience in research and practice, with the aim of improving the transparency of the documentation of the linking process [18]. ICF linking rules have been developed for the content comparison of instruments to the ICF [19]. This development makes it possible to compare, interpret, and integrate information and results from different instruments in practice and in research. Recent linking studies have described the ICF linking rules as providing guidance in the selection of the most suitable instrument or outcome measure based on a conceptual match with the context, the research question, and the intervention goal [20, 21]. The linking results also indicate which constructs are missing and which outcome measures may additionally be relevant [21]. The possibilities of ICF linking extend further than just to content comparison. Starting from an ICF linking, the extent to which different instruments cover certain ICF core sets can be examined. The study of Maritz et al. [22] provides an example in which the Spinal Cord Injury Model System Database is compared with the ICF core set for spinal cord injuries. When researchers are looking for outcome measures concerning one particular core set, they know through this type of research which instruments can provide the most complete coverage of a specific core set. The ICF linking rules also make it possible to develop ICF-based questionnaires, such as the recently developed “Cochin Scleroderma ICF-65 questionnaire,” which assesses the activities and participation of patients with systemic sclerosis [23].

1.2 The use of consensus methods

Consensus methods aim to extract professionals’ “collective knowledge” and to contribute to the development of consensus in complex research questions where there is insufficient empirical evidence. These methods are widely used in various domains, including in healthcare research. Frequently used methods are the Delphi technique and the nominal group technique (NGT). Common features are a structured interaction and an iterative process, in which there is anonymity, controlled feedback, and a static group response [24].

The nominal group technique (NGT) is a structured method that involves a group of experts physically meeting to discuss and to integrate opinions. This technique consists of five steps: 1) generating ideas, 2) collecting ideas, 3) discussing ideas, 4) voting on the ideas, and 5) integrating and publishing the results [25 –28]. The core element of the NGT is the involvement of experts from a specific professional field in structured face-to-face meetings, which ensures the collection of first-hand information and makes results relevant to the professional field. Other main features are that the NGT is time and money efficient and requires little preparation from the participants. A balanced participation and collaboration of group members is facilitated, and results and consensus become clear during the meeting so as to ensure participant satisfaction [26].

The Delphi technique uses a structured process involving several rounds to help the participants reach a consensus. The process steps are usually the following: (1) identifying a research problem, (2) completing a literature search, (3) developing a questionnaire of statements, (4) conducting anonymous iterative questionnaire rounds, (5) providing individual and/or group feedback between rounds, and (6) summarizing the findings [27 , 30]. The literature is ambiguous when it comes to the strength of the agreement rate that should be accepted as consensus. Loughlin and Moore recommended an agreement rate of 51%, Sumsion 70%, and Green et al. 80% [31 –33]. A major advantage of this method is the relative ease of involving different experts without the need to be physically together; consequently, this method is efficient and inexpensive. In addition, anonymity eliminates domineering influences among group members because the participants do not know each other’s answers. A disadvantage is that this method requires a high level of dependency on the participants’ commitment throughout the iterative rounds, and there is no possibility for discussion and idea generation.

The Delphi and the nominal group techniques are often combined, such as in the development of therapy or education programs [34, 35], but the reporting quality varies greatly and is generally poor [36, 37]. In order to inform best practice, a rigorous application is crucial as well as is a detailed description of the methods used and an argumentation for the choice of certain methods. To improve the quality of research integrating consensus methods, clear guidelines in the application of consensus methods are needed. This includes, among other things, the following elements: (1) how criteria are determined for the selection of experts, (2) a determined good size for the expert panel, (3) how many rounds are necessary, (4) which agreement rate is best, and (5) which analysis techniques are best applied in processing the results [38].

1.3 The rationale behind the integration of the ICF linking rules and consensus methods in the IMBA-ICF linking

The authors of the linking rules [18] recommended carrying out the linking with two independent researchers. When ambiguity occurs, a third researcher should be consulted to decide on the most appropriate linking. IMBA is an instrument that has only recently been integrated into the field of RTW in Belgium. It is a comprehensive documentation tool that requires training and certification. In Belgium, the group of experts who are both trained in IMBA and in the ICF and who are familiar with the ICF linking rules is limited. Given the limited number of experts; their full-time employment as occupational physicians, occupational therapists, physiotherapists or academics; and the expansiveness of IMBA (70 main items) at the time of the study, it was not feasible for the experts to be engaged as independent researchers. As a result, the linking could not be performed with two independent researchers and a third who could be consulted in cases of ambiguity. Therefore, within the IMBA-ICF linking study, establishing a profound yet efficient linking methodology became a challenge, notably in the need to involve experts trained in IMBA and/or ICF in the linking process to validate the linking. This led to the development of an adapted linking methodology in which consensus methods were integrated.

2 Aims

The aims of this paper are to present an adapted linking methodology in which experts are involved by means of the application of consensus methods and to describe a step-by-step set of guidelines, supporting researchers and professionals, linking complex instruments to the ICF.

3 Methods

In this section, the development process of the adapted linking methodology in which consensus methods are integrated will be described. First, the application of the ICF linking rules will be described in the development of the initial IMBA-ICF linking. Since they are thoroughly and scientifically substantiated [16 –18], there is no need to deviate from these rules. Second, the integrated use of consensus methods will be described in order to validate the initial IMBA-ICF linking.

3.1 Developing the initial IMBA-ICF linking: Applying the ICF linking rules

3.1.1 Linking rule 1: Acquiring a good level of knowledge of the ICF

Clearly, profound knowledge of the concepts, definitions, and structure of the ICF is an important requirement for applying the ICF linking rules (Linking rule 1, Table 1). In the IMBA-ICF linking case, a thorough preparation phase was therefore implemented. The main researcher (the first author, a PhD researcher in health sciences) studied the ICF both in the basic and in the master training of occupational therapy. Experience in using the model was therefore present. The ICF e-learning tool [39] was consulted before the start of the linking process. This tool clearly illustrates the hierarchy and structure of ICF codes. Even for those who are experienced in using the ICF, it is recommended to use this tool prior to a linking study. The main researcher also studied the instrument to be linked (IMBA in this case) by theoretical training and using the instrument in practice (certified IMBA profiler).

Table 1
Refined ICF linking rules

1. Acquire a good level of knowledge of the conceptual and taxonomical fundamentals of the ICF, as well as of the chapters, domains, and categories of the detailed classification, including definitions, before starting to link the meaningful concepts to the ICF categories.

2. Identify the purpose of the information to be linked by answering the questions: “What is this piece of information about?” and “What is this item about?” The answer to these questions will help to identify the main concept(s) most relevant to be linked to the ICF.

3. Identify any additional concepts contained in the piece of information in addition to the main concept(s) already identified in the previous step.

4. Identify and document the perspective taken on within a certain piece of information when linking to the ICF. Consider the most frequently used perspectives when collecting health or health-related information.

5. Identify and document the categorization of the response options. Consider the most frequently used approaches.

Note: This rule applies only to instruments, questionnaires, assessments, and tests that contain response options.

6. Link all meaningful concepts, the most relevant and any additional ones, to the most precise ICF category.

7. Use “other specified” (8) or “unspecified” (9) ICF categories as appropriate.

At the end of the chapter and at the end of each embedded set of third- or fourth- level ICF categories, there are categories with the final code number “8” for “other specified” and “9” for “unspecified.” Number 8 is used when the concept is not contained within any of the other specific categories at the respective level of a chapter. The additional information is documented after the ICF code. Number 9 is used when the concept to be linked fits within a given chapter but there is not sufficient information at hand to assign it to a specific ICF category.

8. If the information provided by the meaningful concept is not sufficient for deciding about the most precise ICF category, assign the concept to nd (not definable). Concepts referring to health in general, physical health or mental (emotional) health in general, are assigned nd-gh, nd-ph or nd-mh (not definable-general health, not definable-physical health, not definable-mental health, respectively), as well as to disability in general (nd-dis), functioning (nd-func), or a child’s development (nd-dev).

9. If the meaningful concept is not contained in the ICF but is clearly a personal factor as defined in the ICF, assign the meaningful concept to pf (personal factors).

10. If the meaningful concept is not contained in the ICF, assign this meaningful concept to nc (not covered).

Further specifications: Meaningful concepts referring to the diagnosis of a health condition are assigned to nc-hc (not covered-health condition). Meaningful concepts referring to quality of life in general are assigned to nc-qol (not covered-quality of life).

3.1.2 Linking rules 2,3, and 4: Identifying main concepts, additional concepts, and perspective

In the next step, the content of items is analyzed, and decisions are made on the information to be linked to the ICF. This leads to the identification of main and additional concepts (Linking rules 2 & 3, Table 1). When the information needing to be linked is identified, the perspective is documented to make clear the purpose for which this information is collected (Linking rule 4, Table 1).

In the IMBA-ICF linking, the main researcher executed the application of these three rules through a structural analysis of the concepts, whereby the relevant information from each of the 70 IMBA main items was documented and divided into main concepts and additional concepts. Studied information and decisions were registered in an extensive linking table that described the following elements: IMBA items and definitions, the adopted perspective in information, and the identified main and additional concepts (Table 2).

Table 2
Extensive linking table (fragment)

IMBA items ICF categories

Number + Name Definition ICF code Name Definiton Annotation

Body posture 1. Sitting This is the ability or the requirement to assume and maintain a body posture such that the buttocks rest upon a seating arrangement. d4103 Sitting Getting into and out of a seated position and changing body position from sitting down to any other position, such as standing up or lying down. To assume

AND d4153 Maintaining a sitting position Staying in a seated position, on a seat or on the floor, for some time such as when sitting at a desk or a table. To maintain

2. Standing This is the ability or the requirement to assume and maintain an upright posture with outstretched legs. d4104 Standing Getting into and out of a standing position or changing body position from standing to any other position, such as lying or sitting down. To assume

AND d4154 Maintaining a standing position Staying in a standing position for some time as required, such as when standing in a queue. To maintain

	IMBA items	ICF categories
Body posture	1. Sitting	This is the ability or the requirement to assume and maintain a body posture such that the buttocks rest upon a seating arrangement.	d4103	Sitting	Getting into and out of a seated position and changing body position from sitting down to any other position, such as standing up or lying down.	To assume
			AND d4153	Maintaining a sitting position	Staying in a seated position, on a seat or on the floor, for some time such as when sitting at a desk or a table.	To maintain
	2. Standing	This is the ability or the requirement to assume and maintain an upright posture with outstretched legs.	d4104	Standing	Getting into and out of a standing position or changing body position from standing to any other position, such as lying or sitting down.	To assume
			AND d4154	Maintaining a standing position	Staying in a standing position for some time as required, such as when standing in a queue.	To maintain

3.1.3 Linking rule 5: Identify and document the categorization of the response options

When the linked instrument contains response options, the approach in the categorization of these response options has to be identified and documented (Linking rule 5, Table 1). Since IMBA uses a scoring scale but does not use response options, this rule was not relevant in the IMBA-ICF linking.

3.1.4 Linking rules 6–10: Linking all meaningful concepts to the most specific ICF category

When concepts and perspectives are identified, this information needs to be linked to the most specific ICF category (Linking rule 6, Table 1). In some cases, linking to a specific ICF code is not possible. Rules 7 to 10 (Table 1) demonstrate what the other options are in the linking process. Regarding the application of these rules, the ICF linking decision tree [18] (Fig. 1) supports the reasoning of whether certain concepts are part of the ICF and, if so, which components and chapters these belong to. Complementarily, the ICF browser [40] shows the structure and hierarchy of ICF codes in a well-arranged way, and the search engine allows users to search for specific concepts and shows in which ICF codes and associated definitions the concepts can be found. Therefore, this tool can be useful in linking the meaningful concepts to the most relevant and specific ICF category at the second, third, and fourth levels.

Fig. 1

Linking decision tree.

Rules 6 through 10 were rigorously implemented in the IMBA-ICF linking. The main researcher in the IMBA-ICF linking case started with the initial linking. Every main and additional concept of the 70 IMBA items was linked to the most specific ICF category. The extensive linking table (Table 2) was completed with the following elements: the ICF category of main concepts, the ICF category of additional concepts, and an annotation. The outcome of this process was the initial linking.

3.2 Validating the initial linking: Integration of consensus methods

Since the main researcher was the only one involved in the initial linking (because of the expertise on both the IMBA and the ICF), the linking had yet to be validated. From this phase in the study, consensus methods were integrated to involve experts in order to validate the initial linking.

3.2.1 Sampling the expert panel

An expert panel of key informants was compiled using a combination of purposive sampling techniques: heterogeneous and homogeneous [41, 42]. The researchers opted for a homogeneous group in terms of expertise. Theoretical knowledge about the IMBA and/or the ICF was a requirement. In addition, a heterogeneous group was sampled in terms of setting in which the participants were using the IMBA and/or the ICF. Experts were recruited through different channels: a group of Flemish certified IMBA profilers known by the professional association for occupational therapy, NIHDI, and the Vocational Rehabilitation Service GTB. When addressing potential experts, the main researcher planned a meeting to explain the project and to argue why these individuals would be an important partner in the research project. In this way the participants experienced project ownership, which is important to improving commitment and to increasing the response rate during the research procedures [38].

This resulted in an expert panel of eight key informants, in which the experts shared a common knowledge (homogeneous) about a clearly defined content (IMBA and ICF) but were employed in different settings (heterogeneous). Five occupational therapists, two physical therapists, and one occupational physician participated. The following settings were represented within this group: occupational medicine, vocational rehabilitation, policy, academics, and organizations specialized in job placement and vocational re-integration. All the experts received an informed consent and gave permission to integrate their opinions in the results and this study received ethical approval from the Committee for Medical Ethics, Ghent UniversityHospital.

3.2.2 First feedback round: Integrating the Delphi technique

The Delphi technique was chosen deliberately during the first feedback round since there was no need to physically bring the experts together, anonymity was presented, and the process was not influenced by the possible dominating participation of any one expert. Through the first feedback round, the implementation of linking rules 6 to 10 was actually repeated in a time-efficient way with the involvement of several experts and was offered in a structured way, which made specific knowledge of linking rules unnecessary for participants to be able to contribute to the linking. In the IMBA-ICF linking case, a 70% agreement rate was predetermined to be efficient while still being strict.

All the experts (8) received a semi-structured questionnaire by e-mail in which they could document their opinions about the proposed linking. In cases of disagreement, the experts were given the opportunity to formulate new proposals (ICF categories). To complete the questionnaire, some necessary attachments were sent to the experts by e-mail:

Explanatory notes explaining the approach of the linking process, the linking rules, and the application of the ICF browser;

Summarized linking tables (Table 3) where the experts were easily able to consult the IMBA and ICF items and definitions;

Extensive linking tables (Table 2) for when experts wanted to consult the reasoning process.

Table 3
Summarized linking table (fragment)

Name of instrument: Integration von Menschen mit Behinderungen in die Arbeitswelt (IMBA)

Body posture

IMBA item number + name Verbatim health information (e.g. wording of item of instruction) (1) Perspective adopted in information (2) Main concept: What is this information about? (3) Additional concepts contained in information (4) ICF code main concept (6) Name ICF code (7) ICF code additional concepts (8) Name ICF code (9) Annotation (10)

1. Sitting This is the ability or the requirement to assume and maintain a body posture such that the buttocks rest upon a seating arrangement. Descriptive Assume and maintain a sitting body posture / d4103 Sitting / / To assume

AND d4154 Maintaining a sitting position To maintain

2. Standing This is the ability or the requirement to assume and maintain an upright posture with outstretched legs. Descriptive Assume and maintain a standing body posture / d4104 Standing / / To assume

AND d4154 Maintaining a standing position To maintain

Name of instrument: Integration von Menschen mit Behinderungen in die Arbeitswelt (IMBA)
Body posture
IMBA item number + name	Verbatim health information (e.g. wording of item of instruction) (1)	Perspective adopted in information (2)	Main concept: What is this information about? (3)	Additional concepts contained in information (4)	ICF code main concept (6)	Name ICF code (7)	ICF code additional concepts (8)	Name ICF code (9)	Annotation (10)
1. Sitting	This is the ability or the requirement to assume and maintain a body posture such that the buttocks rest upon a seating arrangement.	Descriptive	Assume and maintain a sitting body posture	/	d4103	Sitting	/	/	To assume
					AND d4154	Maintaining a sitting position			To maintain
2. Standing	This is the ability or the requirement to assume and maintain an upright posture with outstretched legs.	Descriptive	Assume and maintain a standing body posture	/	d4104	Standing	/	/	To assume
					AND d4154	Maintaining a standing position			To maintain

The experts were given two months to study the initial linking and to return their responses by e-mail. In the IMBA-ICF linking, all eight experts responded to the questionnaire. The results of the feedback rounds were described in an audit trail in which consensus, feedback, and new proposals were registered. The results of the first round of feedback and the adjustments in the linking output were registered in a chronological audit trail (Table 4).

Table 4

Chronological audit trail (fragment)

IMBA item 11: Arm movements
Initial linking	ICF code	Name ICF code
	d445 WITH	Hand and arm use
	b7300	Power of isolated muscles and muscle groups
Feedback round 1	Agreement rate	Consensus
	83%	Yes
Expert committee meeting	Adjustments/ suggestions	Consensus
	/	Yes
Feedback round 2	Adjustments/Suggestions	Consensus
	Addition of code b7401 (Necessary to reach consistency with the decision that was made during the expert committee meeting to add this code to items 10 and 11)	Yes
Expert meeting	/	Yes

3.2.3 Expert committee meeting: Integrating the nominal group technique

After processing the first feedback round, an expert committee meeting was organized to discuss the items for which no consensus was reached (20 items). The nominal group technique was used to ensure a structured and efficient approach while giving experts the opportunity to formulate their opinions from their different, relevant perspectives to the professional field. One expert dropped out, but this did not influence the settings nor the represented disciplines. The items for which consensus was not reached in the first feedback round and the new proposals were documented in pre-structured guidelines to facilitate the discussion in the expert committee meeting (Table 5). Two small groups of experts (3–4) were formed, and two moderators were assigned to lead the respective discussions. The moderators were experienced in applying the linking rules.

Table 5
Pre-structured guideline expert committee (fragment)

IMBA item 5: Bent/over stooped

Initial linking ICF code Name Definiton Annotation

d4105 Bending Tilting the back downwards or to the side at the torso, such as in bowing or reaching down for an object.

AND d4158 Maintaining a body position, other specified / Bent over (up to 30°) / Stooped (over 30°)

Feedback •d4105: not applicable. The subject of the IMBA item is specifically “maintain” the posture. Bending in the ICF is more about a short bending movement.

•d4158: correct reasoning.

•ICF is too general, too vague for work situations.

New proposals •Only retain d4158.

•Reference to a specific annotation to describe “other specified”?

•By analogy with the previous ICF items, maintaining the bending posture should be included in the annotation, even though the duration of maintaining that posture is not stated in either description.

Subject of discussion •Omit d4105 “bending”? What about the concept “assume a body posture”?

•Retain d4158 “maintaining a body position, other specified” and compensate for the vague description of ICF with an annotation that specifies the bending body posture.

IMBA item 5: Bent/over stooped
Initial linking	ICF code	Name	Definiton	Annotation
	d4105	Bending	Tilting the back downwards or to the side at the torso, such as in bowing or reaching down for an object.
	AND d4158	Maintaining a body position, other specified	/	Bent over (up to 30°) / Stooped (over 30°)
Feedback	•d4105: not applicable. The subject of the IMBA item is specifically “maintain” the posture. Bending in the ICF is more about a short bending movement.
	•d4158: correct reasoning.
	•ICF is too general, too vague for work situations.
New proposals	•Only retain d4158.
	•Reference to a specific annotation to describe “other specified”?
	•By analogy with the previous ICF items, maintaining the bending posture should be included in the annotation, even though the duration of maintaining that posture is not stated in either description.
Subject of discussion	•Omit d4105 “bending”? What about the concept “assume a body posture”?
	•Retain d4158 “maintaining a body position, other specified” and compensate for the vague description of ICF with an annotation that specifies the bending body posture.

In the IMBA-ICF linking, the group remained close to the structure suggested in the literature on NGT [25 –28]. Based on the pre-structured guidelines, the following steps were takenchronologically:

Generating ideas: The moderator explained the feedback and the newly acquired proposals form Feedback Round 1.

Collecting and discussing ideas: The experts were asked to think about these proposals and to discuss as a group whether they could agree with one of the new proposals or if they would rather formulate another proposal (collecting and discussing ideas). The experts were given 15 minutes per item. The moderators had the responsibility of letting every participant argue their proposal.

Voting on the ideas: The two moderators formulated the decision that was made in their respective groups. If consensus was reached, the process continued for the next item. If there was no consensus between the two groups, the possibility was initially offered to find a consensus between the two groups of experts through discussion, but the time was limited to 15 minutes. If no consensus could be reached in that period, then there was a vote wherein a 70% agreement-rate was applied.

The discussions during the expert committee meetings were recorded and stored as backup. Arguments and consensus were registered in an audit trail. The results of the expert committee meetings and the adjustments to the linking output were registered in a chronological audit trail (Table 4).

3.2.4 Second feedback round: Integrating the Delphi technique

A second feedback round was held to question the inconsistencies in the IMBA-ICF linking that arose from decisions made during the expert committee meeting. The methodology used was analogous to the first feedback round. The experts received a short questionnaire with five questions. The seven remaining experts documented their opinions about the proposed adjustments to achieve more consistency in the linking. Results of this second feedback round were again registered in the chronological audit trail (Table 4).

3.2.5 Expert meeting: Final validation

An expert meeting was organized with two IMBA experts from the Institute for Quality Assurance in Prevention and Rehabilitation (IQPR), the institution responsible for the development of IMBA. These experts closely followed the theoretical developments of IMBA and also had practical experience in the use of IMBA. The IQPR experts examined the results of the linking in order to refine the output. During the expert meeting, decisions were made on items for which consensus had not yet been reached. These experts also received an informed consent and agreed to include their opinions in the research project. Adjustments were again registered in the chronological audit trail (Table 4), which resulted in the definitive results of the IMBA-ICF linking.

4 Results

The refined ICF linking rules [18] were strictly followed, and consensus methods were integrated in the ICF linking process. This resulted in the adapted seven-step linking methodology, which needs to be followed chronologically, and experts must be involved as the core element. The results are described in step-by-step guidelines starting from a flowchart (Fig. 2), in which each step is explained with main guidelines and key points in order to support researchers and healthcare professionals linking complex instruments to the ICF.

Fig. 2

Flowchart linking methodology.

4.1 Step 1: The preparation phase

Acquire profound knowledge of ICF concepts, definitions, and structure. It is recommended to use the ICF e-learning tool [39] to support this objective (Linking rule 1, Table 1).

Acquire profound knowledge of the instrument to be linked. Thoroughly review the available literature and manuals.

4.2 Step 2: Sampling the expert panel

Compile an expert panel that is heterogeneous in terms of setting: Address health professionals from various domains so that different perspectives are involved in the linking study (e.g. IMBA-ICF linking case: occupational medicine, vocational rehabilitation, organizations specialized in job placement and vocational re-integration, policy, and academia).

Compile an expert panel that is homogeneous in knowledge: Specifically ask for the available knowledge and experience in ICF and or the instrument that will be linked.

Plan a meeting to explain the project to the experts and to argue why they are important partners in the research project.

4.3 Step 3: Initial linking

Follow the ICF linking rules strictly and document the reasoning process accurately.

For each item, ask yourself, “What is this information about?” and identify the main and any additional concepts (Linking rules 2 & 3, Table 1).

For each item, ask yourself: “What is the purpose for which this information is collected?” and identify the perspective (Linking rule 4, Table 1).

If applicable, identify the categorization of response options and the most frequently used approach (Linking rule 5, Table 1).

Link the information (main and additional concepts) to the most specific ICF category. This is the rigorous implementation of linking rules 6 through 10 (Table 1). The linking decision tree can support this objective (Fig. 1).

In an extensive linking table, register studied information and decisions made (Table 2).

Structure this information in summarized linking tables (Table 3) in which only the items, the linked ICF categories, and the definitions are represented.

The preliminary results will be available at the end of Step 3.

4.4 Step 4: Feedback round 1

Prepare a semi-structured questionnaire with the preliminary results of the initial linking, send it by e-mail to the experts, and allow one to two months to give the experts the opportunity to

indicate whether they agree or disagree with the proposed linking to the ICF, and

formulate new proposals in cases of disagreement.

Send explanatory notes to explain the approach of the linking process, the linking rules, and the application of the ICF browser.

Send the summarized and extensive tables as attachments. These are necessary for the experts to consult while completing the questionnaire.

Apply a 70% agreement rate.

Document the adjustments made during this step relative to the initial linking in a chronological audit trail (Table 4).

4.5 Step 5: Expert committee meeting

Structure the items that did not reach consensus in the first round of feedback in a schedule (Table 5). This document will guide the moderator and the experts through the discussion.

Depending on the group size, compile two (or more) small groups of experts (3–4). Ensure that different settings are represented and assign a moderator to each group.

Let the moderator explain the feedback and new proposals for the linking per item (generating ideas).

Let the experts think about these proposals and discuss their opinions. In case of disagreement, the moderator needs to stimulate the group into formulating a new proposal (collecting and discussing ideas). Limit the time allotment based on the number of items needing to be discussed (15 minutes).

Have the moderators of the small groups formulate the decision made, either to agree or disagree, in their respective groups.

In case of disagreement, have the moderators offer the experts the possibility of reaching consensus through discussion, but limit the time to 15 minutes. If no consensus can be reached, let the experts vote and apply a 70% agreement rate.

Document the adjustments made during this step relative to the initial linking and the output of the first feedback round in a chronological audit trail (Table 4).

4.6 Step 6: Feedback round 2

Pay attention to inconsistencies that may arise from decisions made during the expert committee meeting. Because decisions made at that time may have an impact on items of which the linking was approved during previous steps, it is possible that the linking of these items might also need to be refined and/or adjusted.

For the items where inconsistency is found, formulate proposals/adjustments to achieve the most appropriate ICF linking, and take into account any arguments from the expert committee meeting.

Integrate these proposals in a semi-structured questionnaire and give experts the opportunity to

indicate whether they agree or disagree with the proposals, and

formulate a new proposal in cases of disagreement.

Apply a 70% agreement rate.

At the end of this step, make sure the predefined results are available.

4.7 Step 7: Expert meeting

If possible, organize a meeting with researchers who are directly involved in the development of the instrument being linked to the ICF.

Let the researchers examine the predefined results and refine where necessary.

Document the adjustments made during this step relative to the predefined results in a chronological audit trail (Table 4).

At the end of this step, make the definitive results available.

5 Discussion

Due to the extensiveness of IMBA and the specific training required, the number of researchers who have both the in-depth IMBA knowledge and the skills and resources to apply the linking procedure was very limited. Because these challenges were encountered in the IMBA-ICF linking project, an adapted linking methodology integrating the Delphi and nominal group technique was developed to achieve a valid linking. Integration of consensus methods makes it possible for the main researcher to start with the initial linking and guide the further linking process by involving experts to validate the work in an efficient manner and in a culture of collaboration.

A step-by-step process involving various experts requires good preparation, structured documentation, and a presentation of the results. It is important to communicate preliminary results with experts and to register the progress in a way that makes the decision-making process clear for the researchers, the experts, and future users who want to consult the linking after the research project. With the establishment of detailed guidelines of the IMBA-ICF linking, with reference to methods for data collection (and examples), the results contribute to the reporting quality when applying the consensus methods. This way, the output can support fellow researchers and health professionals, linking complex instruments to the ICF.

5.1 The added value of integration the ICF linking rules and consensus methods in the IMBA-ICF linking process

An in-depth linking can be established when researchers apply the suggested methodology because linking rules are rigorously applied. The advantage of involving experts with different backgrounds is that they can contribute from their perspective to the linking of instruments to the ICF. In this way the linking process is not only carried out from the perspective of two or three independent researchers. The involvement of experts throughout the linking process and the facilitation of ownership by applying the consensus methods, creates support for the output, and this facilitates the implementation of the linking results in practice.

Both the Delphi and the nominal group techniques have an added value in the linking process due to these techniques’ specific properties. Using the Delphi technique can enable the initial selection of (un)approved items to be established by the involvement of experts. All that is needed is a well-prepared initial linking and a questionnaire in which feedback can be indicated. The strictness depends on the agreement rate that is applied. The higher the agreement rate, the less quickly the linking of a certain item will be approved, the more steps this item will go through, the stronger the linking output will be. The NGT protocol’s structured but adaptive design was experienced as a strong feature when used by the expert committee. The protocol structured the discussions but varied in the degree of consensus, the time available, and the composition of the expert panel. It was possible to adjust certain steps in the protocol. For example in this case, the silent generation of ideas was replaced by a round-robin in which all participants took turns speaking briefly to express their ideas. In addition, this technique gave the experts the opportunity to formulate their opinions from their perspective, creating support for the linking output at the end of the process.

When consensus methods are used, the experts of the research group from the instrument being linked (IMBA in this case) can also be involved in the linking process. This facilitates co-creation, which is beneficial in a linking study, considering that knowledge and expertise about the instrument being linked to the ICF is necessary to answer questions during the linking process e.g. “what is this item about?” and “from what perspective is this collected information?” Substantive knowledge about the instrument being linked to the ICF is just as important as a good level of knowledge about the ICF. Since the experts of the IMBA research group were involved, it was possible for the main researcher to obtain training and certification in IMBA, and all the necessary documentation became available and accessible.

5.2 Study strengths and limitations

The adapted linking methodology made it possible for the participants to share valuable insights for both IMBA and ICF, because different experts thoroughly and repeatedly analyzed the concepts of both instruments. The involvement of five Occupational Therapists out of eight original members of the expert panel is pertinent since the scope of this study is Return to Work. The IMBA experts confirmed that the predefined linking that was achieved by involving the experts was very well founded and was in line with both the content and the perspective of the IMBA concepts. No remarkable adjustments were made in the final step as it was just a refinement to achieve the final linking output. We can therefore conclude that content validity was strengthened by integrating consensus methods and involving experts in the linking process. A limitation of the suggested methodology remains that there was a deviation from the ICF linking rules [1 –3] in that it is proposed that the linking be carried out by at least two independent researchers and independence was not present in the methodology described above because experts started from an initial linking that was established by a single researcher. However, by the repeated application of the linking rules throughout the steps, in combination with the final validation by the original research group, a valid linking was achieved. The involvement of the original research group of the instrument that was being linked and the completion of various steps in which the linking rules were strictly and repeatedly applied is very important. The application of a homogeneous sampling technique was necessary to compose an expert panel that was homogeneous in terms of knowledge and expertise. Consequently this strengthens the linking’s content validity.

6 Conclusion

The following recommendations for practice and research are proposed:

In case of linking items from complex instruments to the ICF, the Delphi and nominal group technique proved to be very suitable to integrate in the linking process and to involve experts efficiently and collaboratively.

The developed step-by-step guidelines provide an overview of the application of the adapted linking methodology in practice. The following factors are important to be applied:

In-depth knowledge of both ICF and the instrument that is being linked.

Collaboration with the research group of the instrument being linked.

Integration of the Delphi and nominal group technique to efficiently validate the initial linking.

The rigorous and repeated application of the linking ICF rules through different steps.

The application of the homogeneous sampling technique in terms of knowledge, and a heterogeneous sampling technique in setting and background.

A structured method for data collection.

Ethical approval

This study involving experts was positively evaluated by the Committee for Medical Ethics, Ghent University Hospital (Belgian registration number: B670201734562, date of ethical approval and positive advice: 02/01/2018).

Informed consent

Informed consent was obtained from all experts involved in this study.

Conflict of interest

The authors declare that they have no conflict of interest.

Footnotes

Acknowledgments

The authors have no acknowledgements.

Funding

The authors report no funding.

References

Waddell

, Burton

. Is Work Good for Your Health and Well-Being? London: TSO 2006.

OECD Sickness, Disability and Work: Breaking the Barriers. OECD Publishing; 2010.

Heerkens

, Engels

, Kuiper

, Van der Gulden

, Oostendorp

. The use of the ICF to describe work related factors influencing the health of employees. Disability and Rehabilitation. 2004;6(17):1060–6.

van Dijk

FJH

, van Dormolen

, Kompier

MAJ

. Herwaardering model belasting-belastbaarheid. Tijdschrift voor Sociale Gezondheidszorg. 1990;68(1):3–10.

World Health Organization (WHO). ICF: International Classification of Functioning, Disability qnd Health. Geneva World Health Organization (WHO); 2001.

World Health Organization (WHO). Towards a Common Language for Functioning, Disability and Health: ICF. Geneva: World Health Organization (WHO); 2002.

World Health Organization (WHO). How to Use the ICF: A Practical Manual for Using the International Classification of Functioning, Disability and Health (ICF). Exposure draft for comment. Geneva: World Health Organization (WHO); 2013.

Anner

, Schwegler

, Kunz

, Trezzini

, de Boer

. Evaluation of work disability and the international classification of functioning, disability and health: What to expect and what not. BMC Public Health. 2012;12(1):470.

Escorpizo

, Stucki

. Disability evaluation, social security, and the international classification of functioning, disability and health. The time is now. Journal of Occupational and Environmental Medicine. 2013;55(6):644–51.

10.

Reneman

, Soer

, Gross

. Developing research on performance-based functional work assessment: Report on the first international functional capacity evaluation research meeting. Journal of Occupational Rehabilitation. 2013;23:513–5.

11.

Heerkens

, de Brouwer

, Engels

, van der Gulden

, Kant

. Elaboration of the contextual factors of the ICF for Occupational Health Care. Work. 2017;57:187–204.

12.

Finger

, De Bie

, Selb

, Escorpizo

. An examination of concepts in vocational rehabilitation that could not be linked to the ICF based on an analysis of secondary data. Work. 2016;53(4):775–92.

13.

German Ministry of Labour and Social Affairs (BMAS). IMBA introduc-tion. 2000. Available from: https://www.imba.de/docu–ments/einfuehrungenglisch.pdf.

14.

Kersting

, Kaiser

. IMBA als Baustein der Qualitätssicherung in der beruflichen Rehabilitation. Förderung der Integration von Menschen mit behinderungen in die Arbeitswelt. Köln: IQPR. 2002. Available from: https://www.iqpr.de/iqprweb/public/dokumente/forschung/publikationen/imba_baustein.pdf.

15.

Schian

, Weinmann

, Wieland

. IMBA –Integration von Menschen mit Behinderungen in die Arbeitswelt, Loseblattsammlung. Bonn: Bundesministerium für Arbeit und Sozialordnung. 1996.

16.

Cieza

, Geyh

, Chatterji

, Kostanjsek

, Ustün

, Stucki

. ICF linking rules: An update based on lessons learned. Journal of Rehabilitation Medicine. 2005;37:212–8.

17.

Cieza

, Brockow

, Ewert

, Amman

, Kollerits

, Chatterji

, et al. Linking health-status measurements to the international classification of functioning, disability and health. Journal of Rehabilitation Medicine.. 2002;34:205–10.

18.

Cieza

, Fayed

, Bickenbach

, Prodinger

. Refinements of the ICF linking rules to strengthen their potential for establishing comparability of health information. Disability and Rehabilitation. 2016;41(5):574–83.

19.

Stucki

, Kostanjsek

, Üstün

, Cieza

. ICF-based classification and measurement of functioning. European Journal of Physical Rehabilitation Medicine. 2008;44(3):315–27.

20.

Ballert

, Hopfe

, Kus

, Mader

, Prodinger

. Using the refined ICF linking rules to compare the content of existing instruments and assessments: A systematic review and exemplary analysis of instruments measuring participation. Disability and Rehabilitation. 2019;41(5):584–600, DOI: 10.1080/09638288.2016.1198433.

21.

, MacDermid

, Rosenbaum

. A narrative review and content analysis of functional and quality of life measures used to evaluate the outcome after TSA: An ICF linking application. BMC Musculoskeletal Disorders. 2020;21(228).

22.

Maritz

, Pongpipatpaiboon

, Melvin

, Graves

, Prodinger

. Content comparison of the spinal cord injury model system database to the ICF generic sets and core sets for spinal cord injury. Spinal Cord. 2019;57:1023–30.

23.

Papelard

, Daste

, Alami

, Sanchez

, Roren

, Segretin

, et al. Construction of an ICF core set and ICF-based questionnaire assessing activities and participation in patients with systemic sclerosis. Rheumatology. 2019;58:2260–72.

24.

Jones

, Hunter

. Consensus methods for medical and health services research. BMJ. 1995;311:376–80.

25.

Potter

, Gordon

, Hamer

. The Nominal Group Technique: A useful consensus methodology in physiotherapy research. New Zeeland Journal of Physiotherapy. 2004;32(2):70–5.

26.

Harvey

, Holmes

. Nominal group technique: An effective method for obtaining group consensus. International Journal of Nursing Practice. 2012;18:188–94.

27.

McMillan

, King

, Tully

. How to use the nominal group and Delphi techniques. International Journal of Clinical Pharmacy. 2016;38:655–62.

28.

Center for Disease Control and Prevention (CDC). Gaining consensus among stakeholders through the nominal group technique. CDC. 2018;7. Available from: https://www.cdc.gov/healthyyouth/evalua–tion/pdf/brief7.pdf.

29.

Hasson

, Keeney

, McKenna

. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing. 2000;32(4):1008–15.

30.

McPherson

, Reese

, Wendler

. Methodology Update Delphi Studies. Nursing Research. 2018;67(5):404–10.

31.

Loughlin

, Moore

. Using Delphi to achieve congruent objectives and activities in a pediatrics department. Journal of Med Education. 1979;54:101–6.

32.

Sumsion

. The Delphi technique: An adaptive research tool. British Journal of Occupational Therapy. 1998;61(4):153–6.

33.

Green

, Jones

, Hughes

, Williams

. Applying the Delphi technique in a study of GP’s information requirements. Health & Social Care in the Community. 1999;7(3):198–205.

34.

Safran-Norton

, Sullivan

, Irrgang

, Kerman

, Benne

, Calabrese

, et al. A consensus-based process identifying physical therapy and exercise treatments for patients with degenerative meniscal tears and knee OA: The TeMPO physical therapy interventions and home exercise program. BMC Musculoskeletal Disorders. 2019;20:514.

35.

Paz-Pascual

, Artieta-Pinedo

, Grandes

, ema

. GrouConsensus on priorities in maternal education: results of Delphi and nominal group technique approaches. BMC Pregnancy Childbirth. 2019;19:264.

36.

Humphrey-Murto

, Varpio

, Wood

, Gonsalves

, Ufholz

, Mascioli

, et al. The use of the Delphi and other consensus group methods in medical education research: A review. Academic Medicine. 2017;92(10):1491–8.

37.

Foth

, Efstathiou

, Vanderspank-Wright

, Ufholz

, Dütterhorn

, Zimansky

, et al. The use of Delphi and nominal group technique in nursing education: A review. International Journal of Nursing Studies. 2016;60:112–20.

38.

Keeney

, Hasson

, McKenna

. Consulting the oracle: Ten lessons from using the Delphi technique in nursing research. Journal of Advanced Nursing. 2006;53(2):205–12.

39.

World Health Organization (WHO). ICF eLearning Tool. 2020 [cited 28 Jan 2021]; Available from: https://icf.ideaday.de/en/index.html.

40.

World Health Organization (WHO)ICF Browser. 2020 [cited 28 Jan 2021]; Available from: https://apps.who.int/classifications/icfbrowser/.

41.

Suri

. Purposeful sampling in qualitative research synthesis. Quality Research Journal. 2011;11(2):63–75.

42.

Patton

. Qualitative Research and Evaluation Methods: Integrating Research and Practice. Thousand Oaks: SAGE Publications; 2015.

1.	Acquire a good level of knowledge of the conceptual and taxonomical fundamentals of the ICF, as well as of the chapters, domains, and categories of the detailed classification, including definitions, before starting to link the meaningful concepts to the ICF categories.
2.	Identify the purpose of the information to be linked by answering the questions: “What is this piece of information about?” and “What is this item about?” The answer to these questions will help to identify the main concept(s) most relevant to be linked to the ICF.
3.	Identify any additional concepts contained in the piece of information in addition to the main concept(s) already identified in the previous step.
4.	Identify and document the perspective taken on within a certain piece of information when linking to the ICF. Consider the most frequently used perspectives when collecting health or health-related information.
5.	Identify and document the categorization of the response options. Consider the most frequently used approaches.
	Note: This rule applies only to instruments, questionnaires, assessments, and tests that contain response options.
6.	Link all meaningful concepts, the most relevant and any additional ones, to the most precise ICF category.
7.	Use “other specified” (8) or “unspecified” (9) ICF categories as appropriate.
	At the end of the chapter and at the end of each embedded set of third- or fourth- level ICF categories, there are categories with the final code number “8” for “other specified” and “9” for “unspecified.” Number 8 is used when the concept is not contained within any of the other specific categories at the respective level of a chapter. The additional information is documented after the ICF code. Number 9 is used when the concept to be linked fits within a given chapter but there is not sufficient information at hand to assign it to a specific ICF category.
8.	If the information provided by the meaningful concept is not sufficient for deciding about the most precise ICF category, assign the concept to nd (not definable). Concepts referring to health in general, physical health or mental (emotional) health in general, are assigned nd-gh, nd-ph or nd-mh (not definable-general health, not definable-physical health, not definable-mental health, respectively), as well as to disability in general (nd-dis), functioning (nd-func), or a child’s development (nd-dev).
9.	If the meaningful concept is not contained in the ICF but is clearly a personal factor as defined in the ICF, assign the meaningful concept to pf (personal factors).
10.	If the meaningful concept is not contained in the ICF, assign this meaningful concept to nc (not covered).
	Further specifications: Meaningful concepts referring to the diagnosis of a health condition are assigned to nc-hc (not covered-health condition). Meaningful concepts referring to quality of life in general are assigned to nc-qol (not covered-quality of life).

	IMBA items		ICF categories
	Number + Name	Definition	ICF code	Name	Definiton	Annotation
Body posture	1. Sitting	This is the ability or the requirement to assume and maintain a body posture such that the buttocks rest upon a seating arrangement.	d4103	Sitting	Getting into and out of a seated position and changing body position from sitting down to any other position, such as standing up or lying down.	To assume
			AND d4153	Maintaining a sitting position	Staying in a seated position, on a seat or on the floor, for some time such as when sitting at a desk or a table.	To maintain
	2. Standing	This is the ability or the requirement to assume and maintain an upright posture with outstretched legs.	d4104	Standing	Getting into and out of a standing position or changing body position from standing to any other position, such as lying or sitting down.	To assume
			AND d4154	Maintaining a standing position	Staying in a standing position for some time as required, such as when standing in a queue.	To maintain