Abstract
Introduction
The number of patients needing care who suffer from chronic obstructive pulmonary disease (COPD) is expected to increase in the future. The consequences thereof will increase the socio-economic burden for both patients and society. Telehealthcare technologies have shown potential in reducing hospitalisation-related costs and in improving health-related quality of life (HRQOL) for some COPD patients, but not all. The aim of this study was to investigate the potential of predictive algorithms for helping the general practitioner to stratify telehealthcare for COPD patients in a way that maximises HRQOL and minimises COPD-related costs.
Methods
Data from 553 COPD patients based in the North Denmark Region were analysed and used as predictors for four multiple linear regression models. The models were trained and evaluated for their abilities to predict individual patient’s future health- and cost-related developments, with and without telehealthcare.
Results
The average root-mean-square error (RMSE) of the health and cost models was 5.265 HRQOL scores and US dollars (US$)5430.49, respectively. The accuracy regarding the polarity of the predicted changes ranged from 61–65% for the health models and 74–75% for the cost models. While differences in the magnitude of predictions with and without telehealthcare were statistically significant (p < 0.01), the polarity of predictions was similar across models in 82.05% of all cases.
Discussion
Our results indicate that it may be possible to predict the magnitude and polarity of a COPD patient’s future health- and cost-related developments with and without telehealthcare. Predictive algorithms may provide a useful decision support tool in stratifying telehealthcare for COPD patients.
Keywords
Introduction
Chronic obstructive pulmonary disease (COPD) is an umbrella term used to describe chronic lung diseases causing airflow limitations in the lungs. 1 Approximately 64 million people suffer from COPD worldwide, and the disease is a major cause of morbidity and mortality.2–4 Despite a decreasing trend in mortality, 5 COPD caused 3.1 million deaths in 2012, making it the third most common cause of death that year.2,6 In Denmark, COPD affects 430,000 people and is the direct cause of 3,500 deaths and 23,000 hospitalisations annually.7–9 The large number of deaths and hospitalisations puts substantial socio-economic pressure on both patients and healthcare systems.7,10,11 In fact, currently, 10% (approximately US$440 m) of the total annual Danish healthcare budget for citizens over 40 years is related to the care and treatment of COPD. 8 The prevalence of and socio-economic pressure exerted by COPD is expected to increase as a result of mortality reduction related to improved treatment and a generally increasing incidence of disease.7,10,11
Home support technologies have been investigated as a way to address the expected increase in costs generated by COPD 10 and the increasing need for home care services. 12 According to a review by Paré et al., 10 nine of 10 recent studies have reported a statistically significant decrease in the number of hospitalisations when applying home telehealthcare compared with conventional care. Several studies have also reported a decrease in total COPD-related costs. 10 Despite these positive effects, not all COPD patients seem to benefit from telehealthcare, which is why an in-depth screening of patients should be conducted before offering such a service. 6 To our knowledge, no existing studies have addressed the issue of stratifying telehealthcare for COPD patients.
The aim of this present study was to investigate whether it is possible to use predictive algorithms to help stratify telehealthcare for COPD patients in a way that maximises health-related quality of life (HRQOL) and minimises the hospitalisation costs of the individual patient.
Methods
Data
The data for this study were obtained from 1,225 COPD patients included in the TeleCare North trial, which was a large-scale, cluster-randomised telehealthcare randomised controlled trial (RCT) conducted from 2012–2015 as a cross-sectional collaboration between Aalborg University and general practitioners (GPs), as well as the North Denmark Region and its 11 municipalities.13–15
Characteristics of the patients.
BMI: body mass index; HRQOL: health-related quality of life; US$: US dollars; FEV1: forced expiratory volume in one second; FVC: forced vital capacity; GOLD: Global Initiative for Chronic Obstructive Lung Disease.
Only patients with costs in 2013; bmedian and interquartile range (IQR).
The data was extracted from the TeleCare North trial database and the Danish National Patient Register (DNPR) based on patient social security number. Each patient had a set of 39 predictors, of which 11 were physiological measurements, 24 were extracted from the questionnaire, and four were extracted from the DNPR.
The physiological parameters consisted of systolic, diastolic and mean arterial blood pressure, pulse, body mass index (BMI), spirometry measures (percentage of expected forced expiratory volume in one second (FEV1 (%)), FEV1 measured in litres (FEV1L (l)), percentage of expected forced vital capacity (FVC (%)), FVC measured in litres (FVCL (l)), FEV1/FVC), and oxygen saturation.
The questionnaire was divided into two sections: (a) the standardised European Quality of Life-5 Dimensions questionnaire (EQ-5D), 17 which consists of medical- and COPD-related questions, such as questions concerning the presence of comorbidities (e.g. diabetes mellitus type 2, cardiovascular disease, mental illness, cancer and disease affecting muscles, bones or joints), years since COPD diagnosis and smoker/non-smoker status; and (b) the Short Form-36 Health Survey (SF-36) questionnaire,18,19 which is a well-documented and validated patient-reported survey consisting of 36 questions that are computed into eight subscales; vitality, physical functioning, bodily pain, general health perceptions, physical role functioning, emotional role functioning, social role functioning, and mental health. A mental and a physical component summary (MCS and PCS) score are computed from the eight subscales. Henceforth, the notion of HRQOL equals the average of MCS and PCS of each patient on a scale from 0–100. The lower the score, the more disability is present. Data from the DNPR provided information on the number and total cost of hospitalisations during 2013 and 2014.
Preprocessing
Baseline BMI data were not available for all patients because height information was not reported at baseline for 18.01% of the patients. Heights reported at the 12-month follow-up were imported, and any height values still missing were subsequently imputed using single imputation. Approximately 34% of the spirometry parameters were missing. These were computed using pre-existing parameters. 20 In cases in which the GP did not report any oxygen saturation measurements prior to the study, the mean of the first 10 home measurements for the individual intervention patient was calculated and used as a baseline substitute. Oxygen saturation was removed as a predictor for the control patients due to a large proportion (80.22%) of missing values.
Patients who did not answer the 12-month follow-up questionnaire (intervention group: n = 126, control group: n = 164) and patients who had more than four missing predictors (intervention group: n = 49, control group: n = 183) were excluded. Patients with excessively large hospitalisation costs (intervention group: n = 9, control group: n = 10) were also excluded due to high-cost treatments not related to COPD. Patients who performed less than 10 home measurements (intervention group: n = 131, control group: n = 0) were excluded due to an inadequate use of the intervention. A total of 553 patients (intervention group: n = 263, control group: n = 290) were included in this study (see Table 1). Any missing values from the remaining patients were imputed using single imputation, as the proportion of missing values amounted to only 0.97%.
Regression models
To predict COPD patient HRQOL and hospitalisation cost changes with and without telehealthcare, four multiple linear regression models were developed (see Figure 1). The predictors were used to train the models to predict health and cost changes with telehealthcare for the patients in the intervention group, whereas the same were used to train the models to predict these changes without telehealthcare for the control group. Only patients with hospitalisation costs throughout the baseline year (2013) and the intervention period (2014) were included in the development and evaluation of the cost models.
Illustration of the development of the four multiple linear regression models and an example of how their outputs may be implemented in a decision support tool. COPD: chronic obstructive pulmonary disease; QOL: quality of life.
Each model was trained to fit the data to a criterion variable using the 12 predictors with the largest correlation with the criterion variable. All possible combinations of the 12 predictors (n = 4,095) for each of the four models were trained on 70%, validated on 15%, and tested on the remaining 15% of the data using five-fold cross-validation. Based on the validation process, the best-performing models were chosen, and the performance metrics from the test process were reported. The performances of the models were evaluated based on the spread of the differences between predicted and criterion values (residuals). For each of the four types of predictions, the model resulting in the lowest residual spread was chosen.
The agreement of the models (HI vs HC and CI vs CC) regarding the polarity of the predicted changes was assessed by applying each model to the counterpart’s test data (e.g. using HI to predict changes based on the test data of HC). Thereby, the predictions of the two health and the two cost models were applied to the same sets of patients, which allowed for direct comparison. The degree of agreement was calculated, and statistical analysis was used to assess differences in predictions.
Statistical analysis
The data sets were exported into SPSS (IBM SPSS Statistics). The Shapiro-Wilk test revealed non-normal distributions in the predicted values of each of the four models. Therefore, the medians of the predictions of the health models (HI and HC) and the cost models (CI and CC) were compared using the paired-samples sign test.
Results
The predictors of the four multiple linear regression models.
FEV1: forced expiratory volume in one second; FVC: forced vital capacity; MCS: mental component summary.
Overview of the performance of the four models. All metrics are provided as the mean and standard deviation (SD) of a five-fold cross-validation. The accuracies presented represent the models ability to correctly predict an increase or decrease.
RMSE: root-mean-square error; SD: standard deviation; R2: coefficient of determination.
HRQOL scores; bUS dollars.
The accuracies represented the models ability to predict the correct polarity of the change from baseline to the end of the study (i.e. an increase or decrease in HRQOL or cost). The accuracies of HI, CI, HC, and CC were 61.25 ± 0.07%, 74.44 ± 0.11%, 64.62 ± 0.03%, and 75.03 ± 0.05%, respectively (see Table 3). The average overall probability that a predicted increase actually was an increase (positive predictive value (PPV)) was 65.02 ± 20.00%, and the average overall probability that a predicted decrease was actually a decrease (negative predictive value (NPV)) was 72.36 ± 10.12%. In general, the cost models performed better than the health models.
By applying the intervention models (HI and CI) to the data from the corresponding control models (HC and CC, respectively), a comparable measure of model effects was obtained. Statistically significant differences in medians were found when comparing predictions of HI and HC based on the same data (p < 0.01). CI and CC revealed statistically significant differences in medians when applying CC to the CI test data (p < 0.01), but the reverse was not true (p = 0.171). The average median absolute differences of the predictions of HI and HC on the counterpart’s test data was 1.12 (interquartile range (IQR): 1.51) and 1.37 (IQR: 1.54) HRQOL scores, respectively. For CI and CC, these metrics were US$1605.40 (IQR: US$1901.85) and US$1874.49 (IQR: US$1990.03), respectively. In 76.87% of all cases, the HI and HC predict HRQOL changes with the same polarity, while CI and CC predict cost changes with the same polarity in 87.23% of all cases.
Discussion
The aim of this study was to investigate whether the prediction of future health- and cost-related changes for a specific COPD patient, with and without telehealthcare, was possible. Polarity accuracies ranging from 61.25 ± 0.07% to 75.03 ± 0.05% indicate that it is possible to predict the direction in which a COPD patient’s future HRQOL and hospitalisation cost will develop. However, RMSEs of 5.27 ± 0.17 HRQOL scores and US$5430.49 ± 393.53 for the health and cost models, respectively, suggest that it is difficult to predict the exact magnitudes of the changes. When applying both health and cost models to the same patients, the models predict changes in the same direction in 76.87% and 87.23% of the cases, respectively. The health models predict changes with statistically significant different median values when applied to the test data of both HI and HC, while the cost models only predict changes with statistically significant different median values when applied to the test data of CC.
The spread of predictions was 2.63 ± 0.47 and 2.19 ± 0.33 HRQOL scores for HI and HC, respectively, which was markedly lower than the spread of the criterion values (see Table 3). For CI and CC, the spreads were US$7733.08 ± 2268.87 and US$8188.78 ± 3578.47, respectively, which were also lower than the spread of the criterion values. The lower spread in the predicted values compared with the observed values indicated a higher density, suggesting that some degree of predictive potential is present. This is in line with all the models having accuracies and both positive and negative predictive values well above 50.00%.
As expected, the health models contained predictors concerning MCS, PCS, health status and limitations, whereas the cost models contained predictors concerning total hospitalisation days and cost in 2013. The predictors used within the two health and two cost models differ by some degree (see Table 2), which is interesting as the cluster-randomised study design 14 implies that the physiological and demographic properties of the groups are homogenous. An interesting difference between the models with and without telehealthcare was the overall type of their respective predictors. HI and CI consisted mostly of predictors directly related to either mental or physical health, whereas HC and CC, to a larger degree, consisted of socially related predictors, such as social function, educational level, number of people in the household and civil status. This indicates that social factors affect HRQOL and hospitalisation cost to a larger degree for patients who did not receive the telehealthcare intervention.
FEV1 has also been found to predict changes in HRQOL, 21 which is in line with the inclusion of FEV1 in HI and possibly the FEV1/FVC-ratio in HC. A systematic review regarding the risk factors of hospitalisation and readmission found that previous hospitalisations, low FEV1, comorbidity, impaired health status and gender were risk factors for hospitalisation. Living alone, length of hospitalisation and physical activity were risk factors for readmission.22,23 This may explain the inclusion of hospitalisation cost and number of hospitalisation days in 2013, presence of disease in muscles, bones or joints, gender, FVC, number of people in the household and civil status as predictors in the two cost models.
The models were developed based on predictors from 553 COPD patients. The set of predictors for each model was chosen based on the correlation with the corresponding criterion value, and the feature combination resulting in the lowest residual spread was chosen. Other methods, including both forward and backward elimination of predictors, did not improve model performance despite the use of different inclusion and exclusion criteria.
A reassessment of the exclusion criteria would most likely result in a larger patient population, e.g. imputing missing information for patients with more than four missing predictors and following an intention-to-treat approach would result in a population of 916 patients; an increase of 65.64%. The use of other types of models, e.g. non-linear regression models, may also improve model performance and allow a clear distinction between the changes with and without telehealthcare.
When applied to the same data set, the two health and two cost models predicted changes with the same polarity in 76.87% and 87.23% of all cases, respectively. In combination with the statistically significant differences between the medians of both of the two pairs of models, these results indicate that the HRQOL and hospitalisation cost developments are predictable and that there are differences in developments with and without telehealthcare. However, the large prediction variability does complicate the distinction of changes with and without telehealthcare.
There are several limitations to our study. The high loss-to-follow-up might have been prevented by revising the exclusion criteria. The linear regression approach used in this study provides a tool to predict the direction in which a patient will develop; however, more complex approaches (e.g. non-linear regression models) might provide more accurate predictions of the magnitude of the changes.
In conclusion, our findings indicate that it is possible to predict whether the future HRQOL score and cost for a COPD patient will increase or decrease within the following year, with or without telehealthcare. Furthermore, our results suggest that an optimisation of the models and predictor selection may permit clear distinctions between developments with and without telehealthcare, and thereby provide useful decision support to GPs in stratifying telehealthcare for COPD patients. Therefore, we suggest that more research in this area should be conducted.
Footnotes
Acknowledgements
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
