Abstract
Objective
Our aim was to specify the requirements of an architecture to serve as the foundation for standardized reporting of health information and to provide an exemplary application of this architecture.
Methods
The World Health Organization’s International Classification of Functioning, Disability and Health (ICF) served as the conceptual framework. Methods to establish content comparability were the ICF Linking Rules. The Rasch measurement model, as a special case of additive conjoint measurement, which satisfies the required criteria for fundamental measurement, allowed for the development of a common metric foundation for measurement unit conversion. Secondary analysis of data from the North Yorkshire Survey was used to illustrate these methods. Patients completed three instruments and the items were linked to the ICF. The Rasch measurement model was applied, first to each scale, and then to items across scales which were linked to a common domain.
Results
Based on the linking of items to the ICF, the majority of items were grouped into two domains, Mobility and Self-care. Analysis of the individual scales and of items linked to a common domain across scales satisfied the requirements of the Rasch measurement model. The measurement unit conversion between items from the three instruments linked to the Mobility and Self-care domains, respectively, was demonstrated.
Conclusions
The realization of an ICF-based architecture for information on patients’ functioning enables harmonization of health information while allowing clinicians and researchers to continue using their existing instruments. This architecture will facilitate access to comprehensive and consistently reported health information to serve as the foundation for informed decision-making.
Keywords
Introduction
Information on health care should have value for the clinical encounter, the allocation of resources and payment of services and the development and monitoring of policies at national and international levels.1,2 Information systems are required that build upon a common conceptual understanding to facilitate harmonization.
In 2001, the World Health Organization (WHO) published the International Classification of Functioning, Disability and Health (ICF). 3 Information about people’s functioning is widely collected, including investigations, clinical observations and through Patient Reported Outcome Measures (PROMs). However, this information is often not comparable within and between health conditions, across people’s life span and across settings.4–6 Thus, a standardized reporting system would facilitate harmonization by capitalizing on the utility of existing health information, making it available in a consistent and accessible manner.
The need for such an architecture is compliant with the International Organization for Standardization’s (ISO) eHealth architecture roadmap 7 which aims to facilitate a seamless and meaningful interchange of health information to be used for any decision making in health systems. In this context, architecture refers to the components of an information system, their functions and inter-relationships as well as the principles guiding their design and development.
Our aim was to develop such an architecture. The objectives were to specify the requirements of this architecture, outline the methods for linking existing health information to this architecture and provide an exemplary application of this architecture for standardized reporting of health information.
Methods and Results
Requirements of an architecture for standardized reporting of health information
Two key components are required to develop such an architecture: a common conceptual framework to facilitate content comparability and a measurement framework to enable metric equivalence of existing information. For determining the structure of components and their inter-relationship, the ICF is internationally accepted so can serve as the common conceptual framework. The classification is divided into functioning and disability, and contextual factors, with each having two components (Figure 1). ICF categories are the unit of the classification and are structured hierarchically into chapters within each component. Not only has the ISO recognized that an architecture for harmonizing existing information is needed
8
but has also emphasized that the ICF is well suited to serve as a conceptual foundation.
9
Structure of International Classification of Functioning, Disability and Health (ICF).
ICF categories represent constructs to be measured. While the ICF does not specify how to measure these elements, existing health information – irrespective of the methods with which it was collected – can be linked to the ICF. This generic scheme therefore has considerable potential for standardizing existing health information into a common reporting format. Nevertheless, it is worth to keep in mind that classifications, such as the ICF, are necessary but not sufficient for the development of measurement systems. 10 The latter are concerned with the reliable and valid assignment of values to a construct and its attributes.
A measurement framework which builds upon the assumption of fundamental measurement requires that the numbers assigned to a construct can be conjoint or amalgamated 11 and be independent of the particular test or items of an instrument that are used, also referred to as specific objectivity. 12 So, just as with an instrument to measure height, the distances between adjacent levels of the attribute being measured remain the same and equal, irrespective of the amount of the attribute processed; it works in the same way (invariant) irrespective of group membership (e.g. gender); the measurement unit is independent of the original calibration sample and instruments measuring different levels of the attribute can be co-calibrated to make a longer measure (just as adding two rulers together). Consequently, to achieve metric equivalence, scores from different instruments with different response options (and perhaps different ranges) need to be aligned on a common metric whereby the poles of the metric have the same meaning for all instruments (e.g. 0 = No problem to 100 = Complete problem). Once these requirements are met, invariant measurement is achieved. 13
A particular challenge for systems meeting these two requirements is inclusivity – that irrespective of local resources, clinicians and researchers have access to a standardized reporting architecture. For example, for any PROM, the raw score needs to be a sufficient statistic (i.e. the raw score contains all the information needed to identify the extent of an individual’s problem on a given attribute). 14 In turn, the raw score should be able to be translated, via access to a transformation table, into the units of the ICF reporting architecture. For a clinician or researcher, this implies that he or she can simply generate the raw score from the instruments and tests used in routine practice and obtain a standardized score which enables comparability with other instruments and tests, without the necessity of any further statistical operation.
Methods for linking existing health information to the architecture
The methods to realize the requirements of the conceptual and measurement framework are the ICF Linking Rules and the Rasch measurement model, respectively. The ICF Linking Rules are a method to guide the process of identifying the meaningful concept of any source of information to be subsequently linked to the most appropriate ICF category. The ICF category best representing the meaning of the concept is selected by a standardized procedure. This approach has been widely applied to compare existing sources of information based on the ICF. 15
The Rasch measurement model constitutes a probabilistic form of additive conjoint measurement, enabling invariant measurement with the raw score as a sufficient statistic. 11 Applying the Rasch model is an iterative process to test the model’s assumptions of invariance, unidimensionality, local independency of items and, in case of identified problems, the application of remedies to resolve these problems as detailed in Appendix 1.16,17 If the scale satisfies the assumptions of the Rasch model, a person’s ability is measured on an interval scale with equal units at any point of the continuum. The logit scale, which results from Rasch analysis, can be easily transformed into a more meaningful and user-friendly scale (e.g. ranging from 0 to 100) by a simple linear transformation. Preferably, full (sub-)scales from existing PROMs can be linked to the ICF reference metric to enable a meaningful cross-walk between existing instruments and the ICF.
Exemplary application of the architecture
To demonstrate how routinely collected information can be translated into the proposed architecture, a secondary analysis of the data collected within the North Yorkshire Survey in England is described. 18
People with hip and knee problems at risk for joint arthroplasty aged 19 and over participated in the survey. Two random samples, a development (N = 388) and validation (N = 342) sample, were drawn from 1115 people. These sample sizes were suitable for Rasch analysis to provide accurate estimates of item and person locations. 19 In both samples, 45% were aged below 65 years. Similar proportions of men were in the development and validation sample (35% and 38%).
Three instruments were used: the Physical Function sub-scale of the Short Form 36 (SF-36), a generic instrument to assess health status; 20 the Stanford Health Assessment Questionnaire (HAQ), a disability index originally developed as a generic scale and now widely used in the field of rheumatology; 21 and the Lequesne Index of Severity of Osteoarthritis in the Hip (LEQ), a health condition specific instrument. 22 The first two columns of Appendix 2 provide an overview of the items and response options of each instrument.
First, the existing ICF Linking Rules were applied to link the instruments to the ICF. Second, the Rasch measurement model was applied to test the metric properties of each individual scale and to establish metric equivalence of the (sub-)scales of instruments linked to a given domain. (A domain is any meaningful aggregation of ICF categories and, could for example, include chapters or components.) Combining items into domains was guided mainly by the results of the ICF linking. The ordering of the response options of items was aligned so that the lowest score indicated lowest severity and the highest score the greatest severity. As items from different instruments with different response options were co-calibrated on the same metric, a partial credit parameterisation of the Rasch model was required. 23 Rasch analysis was conducted with RUMM2030. 24
All items were linked to the most precise ICF category as presented in the fourth column of Appendix 2. For instance, item 1 of the HAQ, Dress yourself, was linked to the second level category d540 Dressing; or the item a from the Physical Function subscale of the SF-36, Vigorous activities, was linked to the component level d, Activities & Participation. Based on these ICF linkings, the items were grouped into the domain of d4 Mobility and d5 Self-care which resulted in good fit to the Rasch measurement model for the SF-36 Physical Function sub-scale and the HAQ. Certain items of the LEQ contained more than one concept (e.g. remaining standing increases pain and pain when walking). This analysis shows that items combining concepts related to Pain and Maintaining a body position fitted well together from a psychometric perspective, and items integrating Pain and Changing a body position fitted with other mobility items. Some items included both concepts, Pain and Mobility. Different combinations guided by the ICF links were tested to reach the best fit to the Rasch model. Furthermore, separating the items LEQ31-33 into a separate group from the other Mobility items improved the fit to the Rasch measurement model (as shown in Appendix 3). Once the fit of the individual scales to the Rasch measurement model was achieved, Rasch analysis was conducted separately for Mobility and Self-care to establish metric equivalence within each of these domains; fit to the Rasch model was achieved for both domains. All findings of the Rasch analysis are detailed in Appendix 3. Figure 2 illustrates the common construct measured by the three (sub-)scales and the equating of its scores. The conversion from the raw scores to the standardized reference metrics for Mobility and Self-care is provided in Appendix 4.
Illustration of the metric equivalence of the (sub-)scales under study (SF-36 – Physical Function Sub-scale, HAQ, LEQ) mapping on to the common metric for Mobility and Self-care. It is worth mentioning that the endpoints of the common metric are beyond any endpoint of a single scale indicating that having the best score on one scale is still not as good as having the best scores across all scales.
Discussion
We have outlined the requirements and methods of establishing an architecture for harmonizing existing health information as operationalized through functioning. We have built upon the ICF to serve as a conceptual framework and reference classification to establish content comparability of constructs. Using the ICF for harmonizing health information facilitates health services to be compliant with international standards for quality management (ISO). Furthermore, harmonizing existing health information in this way allows clinicians and researchers to continue using their existing instruments and methods for data collection, with no need for imposing new instruments on health services and systems unless there are other reasons for doing so.
To illustrate the methods, we used three PROMs. The requirements and methods, however, are applicable for any mode of data collection. La Porta et al., 25 for instance, have applied the Rasch measurement model to harmonize existing clinical tests for measuring balance. Integrating information derived from various modes of administration, such as clinical tests and patient- or proxy-reported outcomes, becomes particularly important for the monitoring of health over time across the continuum of care. Chang et al. 26 highlighted that ‘linking patient-reported health with physiological markers of disease provide not just unique information in patient care, but also help to determine the severity of disease and monitor the trajectory of illness’.
It is worth mentioning that the Rasch model belongs to the family of Modern Test Theory. There are other models within item response theory, but the Rasch measurement model has specific a priori criteria, including invariance and specific objectivity, consistent with fundamental measurement, whereas other item response theory models apply a statistical modelling approach to identify a model that best describes the variance of observed data. 27 Therefore, the Rasch measurement model provides a metric consistent with fundamental measurement principals for monitoring the health of individuals over time. 28 The properties of the Rasch measurement model apply within a specific context. Establishing invariance in other diagnostic groups, institutionalized persons and other countries is required. Variance or differential item functioning, such as by health condition or cultural differences, can be dealt with by creating group-specific items within the Rasch measurement model. 29
Further application of these principles and method is needed in multicentre, international studies to establish harmonization of existing health information across various health conditions and different countries, languages and cultures. The ICF Research Branch, a cooperation partner within the WHO Collaborating Centre for the Family of International Classification in Germany (at DIMDI) hosts an international collaborative project which expands and promotes the methods outlined in this paper (http://icf-research-branch.org/icf-info). Previous work of the ICF Research Branch, such as ICF linking studies, the development of ICF Core Sets (short-lists of ICF categories most relevant to be consistently reported for a specific health condition), or the ICF Generic Set (seven ICF categories most relevant to be reported across the general and clinical populations) and ICF Rehabilitation Set (30 ICF categories most relevant to describe functioning across various clinical sub-populations), serves as the foundation for this collaborative project.
In conclusion, WHO’s conceptualization and classification of health in the ICF and the Rasch measurement model are promising for building a rigorous architecture for the development and implementation for standardized reporting of health information. The methods for establishing content comparability and metric equivalence between existing instruments support the establishment of comprehensive and consistently reported information. Developing an architecture for standardized reporting that builds upon existing health information, regardless of its origin, method and mode of collection has potential to support inclusivity and benefit meta-analyses, thus strengthen evidence-informed decision making.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.