Abstract
Background
Increased use of telemedicine in the healthcare system is a political goal in Denmark. Although the number of hospital patients using interventions such as the video consultation has increased in recent years only a small proportion of the outpatient and inpatient visits involve telemedicine. The TELEMED database (https://telemedicine.cimt.dk/) has been launched at the Center for Innovative Medical Technology in Denmark to ensure that hospital managers and healthcare professionals have access to information about telemedicine services and their effectiveness. This article describes the development and the content of the TELEMED database.
Methods
A structured literature search was made in the PubMed Database for randomised controlled trials or observational studies with a control group that investigated the effect of telemedicine interventions for hospital patients. Data were extracted from each article on the clinical effectiveness, patient perceptions, economic effects and implementation challenges. As the database should only provide inspiration to healthcare professionals regarding possibilities for use of telemedicine, the risk of bias in the studies was not assessed.
Results
The literature search resulted in 2825 hits. Based on full text assessment, 331 articles were included for data extraction and assessment. These articles present telemedicine services used in 22 different medical specialities. Forty-eight percent of the studies found a positive, statistically significant clinical effect, while 47% showed no statistically significant difference. In 48% of the studies, patients’ experiences were examined and of these 68% found positive patient experiences. Fifty-four percent of the articles included information on the economic effects and, of these, 51% found reduction in healthcare utilization. In the majority of studies between two and four types of implementation challenges were found.
Keywords
Introduction
Increased use of telemedicine and other digital health technologies is a political goal in Denmark and many other countries as a key strategy for handling the demographic challenges to the healthcare system. This is described, for example, in ‘Strategy for digital health 2018–2022’ from the Danish government. 1
The Association of Danish Regions, which is financing the Danish hospitals based on tax collected by the government, has a similar goal regarding the use of telemedicine, described in the strategy for the future healthcare system. 2 The Danish hospitals are part of the public health insurance and services are free and without co-payment for the citizens. The current financial agreement between the government and the Association of Danish Regions 3 also states a need for greater use of virtual contacts between hospitals and patients, and positive economic incentives to increase the use of virtual contacts as an alternative to physical outpatient visits.
The COVID-19 pandemic has further increased the focus on the possible benefits of telemedicine. The use of video consultations with patients has increased in Danish hospitals and primary care in line with international efforts to use telemedicine to improve patients’ access to care. In its COVID-19 health system response monitoring report the World Health Organization (WHO) notes that digital health tools play a central role in countries responses to COVID-19. 4
The political goal to increase the use of telemedicine appears to reflect a greater demand in the Danish population for digital communication with the healthcare system. The Association of Danish Patients completed an interview study in May 2020 with a random sample of 1460 Danes and found that 55% of the respondents in the study would like more of the contacts with the healthcare system to be digital. 5 Among the respondents with practical experiences with telemedicine, 75% expressed a positive attitude towards the use of telemedical communication.
Despite this widespread interest, telemedicine – narrowly defined as the delivery of healthcare services over a distance between hospital staff and patients by use of information and communication technology (ICT) – is used in only a small proportion of the total number of outpatient and inpatient visits in countries like Denmark. 6 The Ministry for Health in Denmark has estimated on the basis of the national reimbursement system that, in 2017, the proportion of patients using some kind of virtual communication as part of their hospital treatment in Denmark was 14%, but most of these activities were comprised of e-mails or telephone communications. 7 Thus, only a small proportion of patients are using telemedicine.
A major challenge for hospitals and clinical departments, that wish to implement new telemedicine services, is how to obtain an overview of the available technologies and their effects and impact. This is difficult partly because of the increasing number of available digital technologies and the number of new effectiveness studies. A simple search in the PubMed database using the search string ((telemedicine (MeSH - Medical Subject Headings)) OR (telemedicine) OR (mHealth) OR (eHealth) OR (telehealth) OR (mobile health) OR (home monitoring)) revealed a huge increase in the number of telemedicine studies per year from 964 articles in 2000 to 10,304 articles in 2020. A recent review on the evolution of telemedicine and digital health reported a similar increase in the number of studies of telemedicine and other digital health technologies. 8 In addition, many studies focus on the clinical effect of the telemedicine service and do not describe its impact on patient perception, economic consequences or implementation challenges, as demonstrated in an example of a review of the area. 6
In view of these challenges, a new database – the hospital telemedicine TELEMED database – was established at the Center for Innovative Medical Technology (CIMT) in Odense, Denmark. The aim was to provide relevant information to hospital managers and healthcare professionals on evidence-based telemedicine services and the expected consequences of their implementation. The long-term objective of the database is to inspire hospital managers and healthcare professionals and provide a basis for selecting and implementing evidence-based telemedicine services in hospitals in Denmark and other countries.
The aim of this article is to describe the development and the content of the TELEMED database, including assessment of the effects of the telemedicine technologies.
Methods
The description of the approach taken in the development of the database is presented below including the following steps: search and eligibility criteria, selection of articles, data extraction and assessment of clinical and other effects.
Search and eligibility criteria
Telemedicine was defined as the delivery of healthcare services over a distance between hospital staff and patients by use of information and communication technologies. Thus, telemedicine includes synchronous or asynchronous communication between healthcare professionals and patients through the use of applications (apps), homepages, video, text messages, telephone, home monitoring of patients health or submission of patient-reported outcomes (PROs) data from the patient’s home to the hospital.
A structured literature search was made in the PubMed database on 13 August 2019 with the objective to identify randomised controlled trials or observational studies with both intervention group and control group on the effect of telemedicine services for hospital patients. The search combining search terms for telemedicine, the hospital setting and the relevant research designs. Following adjustments from a reverse record check, the following search string was used: ((telemedicine(MeSH)) OR (telemedicine) OR (mHealth) OR (eHealth) OR (telehealth) OR (‘mobile health’) OR (‘home monitoring’)) AND ((hospitals(MeSH)) OR (hospital)) AND ((RCT) OR (controlled trial) OR (randomised trial) OR (randomised trial) OR (control group) OR (control groups) OR (controls) OR (control) OR (comparison group) OR (comparison groups) OR (comparisons) OR (comparison)). In addition, the PubMed-filter for species was set to ‘Humans’, publications were restricted to the last 10 years (from 13 August 2009–13 August 2019) and the search was limited to publications in English, German, Danish, Norwegian or Swedish. ePublications were included in the database.
Selection of articles
All abstracts identified in the search were exported to Covidence (https://www.covidence.org/) for sorting and review of articles. Two consultants independently assessed of the first 100 articles and after discussing the results, defined the final inclusion and exclusion criteria. The inclusion criteria were further refined to studies including patients with somatic disease (excluding studies focusing on patients with mental diseases), hospital telemedicine studies with two-way communication between patients and healthcare professionals, randomised controlled trials or observational studies with a control group and studies published in a peer-reviewed journal. Conference proceedings and reviews were excluded. Articles deemed ‘relevant', ‘dubious' or ‘unknown’ were reviewed in full text. Selection of articles in full text was made by a group of five staff members in health technology assessment (HTA) with a PhD or master's level degree. If a reviewer was uncertain about the relevance of an article, the relevance was assessed by consensus in the group.
Data extraction
The data extraction was designed to reflect the domains included in HTA of telemedicine and digital health technologies to ensure the relevance of the TELEMED database for decisions on selection and implementation of new telemedicine services.9,10 The following general study characteristics were extracted from all articles where possible:
Source: author, year of publication, PubMed ID. Setting: country, type and number of hospitals involved in the study. Study design: randomised controlled trial or observational study. Follow-up: duration of follow-up per patient. Patient group: the number of patients in the total sample, in the intervention group and the control group. Inclusion criteria: criteria described in the article. Type of technology: description of the information and communication technology including devices etc. Intervention: description of the intervention including information about the objective, the patients’ use of the service and the duration of the intervention measured in weeks or months. Clinical effects: whether or not statistically significant differences were found on primary and secondary outcomes. Types of outcome measures were also described. Patient safety: description of incidents or adverse effects. Patient experiences: information on the patients’ use of and experience with the telemedicine service. Staff experiences: information on staff experiences and time used by the staff as part of the telemedicine services. Costs and organization: information on investments, use of hospital staff, statistically significant differences in healthcare utilization between the intervention and control group. Organizational changes as part of the telemedicine service were taken into consideration.
To ensure similar data extraction by the five reviewers, a template was developed with guidance on the specific information needed, see Supplementary Material File 1.
Assessment of the effects of the telemedicine services
To make the information on the effects of the telemedicine services more accessible, visual illustration of the effects was developed using a system of traffic lights. This visualization was inspired by the color-coding used in the risk of bias assessments for studies in a Cochrane review. 11
In the TELEMED database, the different traffic lights indicate the following for each telemedicine study:
Clinical outcomes: Green indicates a statistically significant improvement (p<0.05) in the primary clinical outcome for the intervention group compared to the control group. Yellow indicates that no statistically significant difference in the primary outcome was found and red indicates a statistically significant deterioration in the primary clinical outcome. If a study included several primary outcomes, green indicates that a statistically significant improvement was found in at least one outcome and no effect was found on other outcomes. If both short- and long-term clinical outcomes were described in the same article, the information on long-term outcomes was used as basis for the assessment. Patient perceptions: Green indicates that more than 50% of patients expressed satisfaction with the service or that more than 50% of patients in the intervention group used the telemedicine service. Yellow indicates mixed results for patient perceptions, e.g. more than 50% of patients expressed satisfaction in the the intervention group, but less than 50% of patients in the intervention group used the service. Red indicates that less than 50% of patients expressed satisfaction with the service or that less than 50% of patients in the intervention group used the telemedicine service. Economic effects: Green indicates a statistically significant reduction (p<0.05) in the patients’ healthcare utilization for the intervention group compared to the control group. Yellow indicates no statistically significant change in healthcare utilization, and red indicates a statistically significant increase in the patients’ healthcare utilization. If a study reports several measures of healthcare utilization, the green colour indicates a statistically significant reduction in at least one measure and no effect on other measures. Similarly, yellow indicates both statistically significant reductions and increases in healthcare utilization measures, and red indicates that at least one measure of healthcare utilization was statistically significantly increased in the intervention group and other measures did not show significant changes.
The grey colour was used if the article did not include information on effects. Additional information was extracted on the challenges related to the practical implementation of the telemedicine services described in each article. These challenges were divided as follows:
This telemedicine service requires collaboration between healthcare organizations or sectors (e.g. between hospital and municipality). This telemedicine service requires procurement of new information technology (IT) devices (e.g. devices for measuring blood pressure that patients cannot be expected to have in advance). This telemedicine service requires development of new IT solutions (e.g. Web page or app). This telemedicine service requires training and education of healthcare professionals (e.g. training in the use of IT devices beyond what is expected from the particular group of professionals). This telemedicine services requires additional time for healthcare professionals (e.g. time for additional video consultations). This telemedicine service requires integration with the local electronic patient record system (e.g. to ensure that planned hospital visits are included in a new patient app).
The results from the articles were used directly as reported to generate the visual illustration in the database. As the database should only inspire and provide information about the potential use of telemedicine, we did not assess the risk of bias in the studies or address problems in the statistical analysis presented in the articles
Two authors (IWS, TK) assessed the reliability of the effects of the telemedicine services and the color-coding of the articles. Intra-rater reliability were tested by comparing independent assessment of the effects found in 47 articles. The results were discussed and used to identify challenges in the assessment and to ensure agreement between the two reviewers. Statistical analysis of agreement between the two reviewers was made by estimation of Kappa statistics.
Results
Study selection
The structured search identified 2825 articles. The review of title and abstract led to 2254 articles being removed and 571 articles were eligible for full-text screening. Eventually, 331 full-text articles fulfilled the eligibility criteria. A flow chart of the process is provide in Figure 1.
The flowchart illustrating the process of identifying, screening and including articles.
Assessment of the outcomes of telemedicine services by medical speciality
The articles included studies of patients from 22 medical specialities, as shown in Table 1. As some studies included several different patient groups the total number of studies in the table is higher than the number of articles. Similar to previous reviews of telemedicine such as Eze et al., 6 the largest number of telemedicine studies were found in cardiology, respiratory medicine (e.g. patients with Chronic obstructive pulmonary disease (COPD)) and endocrinology (e.g. patients with diabetes), but studies in gynaecology and obstetrics, oncology and paediatrics were also frequent.
Distribution of articles in the hospital telemedicine TELEMED database on medical specialities and information on clinical effect, patient perception and economic effects.
Regarding the clinical outcomes, 164 studies (49%) reported a statistically significant improvement in the primary outcomes for the patients using telemedicine compared to the control group of patients. In contrast, 161 studies (48%) showed no statistical differences. Only eight studies (3%) found a statistically significant deterioration in the primary clinical outcomes. Regarding patient perception, 164 studies (49%) included information about patient experiences and 119 studies (35%) found a positive patient experience based on the definition used in the database. Thus, 73% of the studies that included information on patient experience found a positive result.
Regarding patients’ healthcare utilization, 179 studies (53%) included this information. Of these, 93 studies (28%) reported that healthcare utilization or some type of healthcare resource use was statistically significantly reduced in the telemedicine group. This means 52% of the 197 studies indicated that telemedicine reduces healthcare utilization in general or reduced some type of healthcare utilization.
Most studies reported between two and four types of implementation challenges, generally that the telemedicine services required new ICT equipment or devices as well as education or training for the healthcare professionals.
Search results from the database
The TELEMED database is available at https://telemedicine.cimt.dk/. Searching the database for telemedicine studies can be limited to specific medical specialities, countries, technologies, clinical effects, patient experiences and economic effects.
Most of the studies involved more than one type of technology. The most frequently used is home monitoring (175 articles), phone call (111 articles), Web pages (92 articles) and video consultation (81 articles).
The visual presentation of a search for evidence-based telemedicine studies in the TELEMED database is illustrated in Figure 2 with an example of a telemedicine service for patients with diabetes using home monitoring. 12 This study of a telemedicine service for self-monitoring of Hemoglobin A1c (HbA1c) and other clinical outcomes by patients with poorly controlled, insulin-requiring diabetes reported no statistically significant impact on the primary clinical outcome or the patients’ healthcare utilization. The article also described implementation challenges in the form of new equipment and a need for training of healthcare professionals. Figure 2 further shows how to how to interpret the icons, the traffic lights and links to the extracted data and to PubMed.
Example of the results of a search in the hospital telemedicine TELEMED database.
Inter-rater reliability in assessing the effects of telemedicine
The two raters assessed a subsample of 47 articles within nine medical specialities (Table 2). The agreement between the two reviewers was 89% for clinical outcomes, 85% for patient experiences, 93% for the economic outcomes and 40% for the implementation challenges. The Kappa value for concordance between the two rates with regard to the outcomes described by use of traffic lights in the database was 0.8423 (standard error (SE) = 0.1304) for the clinical outcomes, 0.7726 (SE = 0.1026) for patient perceptions and 0.8809 (SE = 0.1393) for economic outcomes.
Proportion of studies with similar assessment of effects by two independent reviewers in different medical specialities.
Discussion
The TELEMED database was made available on 10 August 2020 at https://telemedicine.cimt.dk/. It holds information on the content and the effects of telemedicine services from 331 articles published from 13 August 2009–13 August 2019. The articles reported on hospital patients from 22 medical specialities and investigated technologies such as home monitoring, phone calls, webpages, video consultation and patient apps.
The objective of the database is to provide easy access to information on evidence-based telemedicine services to help inspire hospital managers and healthcare professionals to implement telemedicine services. Therefore, the effects of the services are visualised using a traffic light system to enable users to easily interpret the information on clinical effects, patient experiences, economic effects and implementation challenges. Thus, the database goes further than other databases with respect to the assessment of the consequences of implementation of telemedicine technologies. The TELEMED database reflects many examples of the domains included in HTA of telemedicine and digital health technologies9,10 although it can be criticised for not including all relevant domains, e.g. the ethical and legal aspects of telemedicine included in some HTA models. 9
The inter-rater comparison for 47 articles assessed by two reviewers independently (Table 2) shows that assessment of the effects was subjective, especially with regards to implementation challenges, where agreement was only 40%. This was largely due to the lack of detailed information on the organization and implementation aspects of the telemedicine services. The CONSORT-EHEALTH guideline 13 for reporting eHealth and mHealth services has addressed this problem by adding new items to the CONSORT guideline: (a) clarify the level of human involvement, including timing and frequency of the support (item x), and (b) describe any co-interventions including training/support (item xii). If the CONSORT-EHEALTH guideline is used in future publications of telemedicine studies, fewer problems would be expected with assessment of the implementation challenges. It is also noted that representing the clinical impact of a telemedicine service with just one traffic light is a simplification of complex information. The description of the primary outcomes and long-term effects in the articles was used as the the basis for the traffic lights, but there is a degree of uncertainty and subjectivity in these assessments.
In addition, using a green traffic light to describe the patients’ perception of telemedicine services where more than 50% of patients expressed satisfaction or more than 50% of patients used the telemedicine service, can be criticised because satisfaction and use of telemedicine are two different aspects of the patients view. However, this combination of different information was made to include information on patient perception of the telemedicine services for as many studies as possible in the database.
Other databases of telemedicine technologies are available, such as the the Digital Health Care Directory of telehealth, mHealth platforms and tools, which is produced by the American Telemedicine Association and Consumer Technology Association. 14 It provides short descriptions and links to industry homepages with information from each company about their technology. Specific searches for telemedicine, telemental health, diabetes care services, etc. can be made from either a patient or provider perspective. The database was established to help patients or providers to identify digital health solutions during the COVID-19 pandemic. In contrast to the TELEMED database, the Digital Health Care Directory does not include information about clinical effects, patient perception, economic effects or implementation challenges but instead refers to company information.
A clear limitation of the TELEMED database is that only telemedicine services used for somatic hospital patients are included. In addition, only studies of telemedicine services with communication between healthcare professionals and patients are included. This means that many services are not included e.g. for patients with mental health issues, primary care services and studies of communication between healthcare professionals at different hospitals. Another limitation of the database relates to the chosen search terms. Inclusion of additional search terms may have helped us identify additional studies relevant to the database. In the selection of articles for the database, this was an important reason for the difference in the number of studies identified by the PubMed search and the final number of studies included. Further funding will be sought for the database to ensure regular update, expansion of the focus of the database and inclusion of studies from additional databases.
It is important to note that the TELEMED database should not be the only information source for decisions about implementing new telemedicine services. Hospitals and clinical departments can use the database to identify interesting evidence-based telemedicine services for groups of patients similar to their own. However, the relevance of the technology and the service must be considered. If, for example, a telemedicine service includes a patient app similar to apps already used in hospital, then the service may be easy to adapt to the local setting. The likely effects of the technology must also be assessed and this should include additional searches for relevant scientific studies e.g. in other databases and an assessment of the risk of bias in the studies found. A possible criterion could be that only technologies with (a) a positive or no clinical impact (b) a positive patient experience and (c) a limited number of implementation challenges should be considered for implementation in the local setting. If such a telemedicine service can be found in the database, additional detail should be collected by reviewing the full-text article, assessing the risk of bias in the study, and reading reviews of similar services for comparable patient groups. Finally, the local hospital IT department should be involved to ensure interoperability with existing IT systems.
Conclusions and recommendations
The TELEMED database (see https://telemedicine.cimt.dk/) is a free and easily accessible database that includes information about the content and the effects of telemedicine services extracted from articles describing patient studies from 22 medical specialities. The objective of the database is to inspire hospital managers and healthcare professionals to implement telemedicine services by providing an overview of existing evidence-based technologies with information on clinical effects, patient experiences, economic effects, and implementation challenges. It is intended to apply for additional funding to ensure that regular updates and expansion of the focus of the database will occur.
Supplemental Material
sj-pdf-1-jtt-10.1177_1357633X211015585 - Supplemental material for The hospital telemedicine TELEMED database: Providing information on evidence-based telemedicine services to hospital managers and healthcare professionals
Supplemental material, sj-pdf-1-jtt-10.1177_1357633X211015585 for The hospital telemedicine TELEMED database: Providing information on evidence-based telemedicine services to hospital managers and healthcare professionals by Kristian Kidholm, Ida W Svendsen, Knud Yderstræde, Anne M Ølholm, Kathrine Rayce and Tue Kjølhede in Journal of Telemedicine and Telecare
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
