Abstract
Introduction
Telemedicine is rapidly becoming a standard means of delivering many healthcare services. 1,2 Much of the research on telemedicine has focused on two central questions. The first is the question of whether particular health services can be delivered via telemedicine as reliably and effectively as is possible using traditional in-person delivery systems. 1,3,4 A second theme has been the question of whether and which health services can be delivered with equal or greater cost-effectiveness using telemedicine compared with traditional systems. 3,5,6
There is a growing recognition that wide varieties of programs, settings, provider types, patient populations, and payment mechanisms exist within the bounds of what is called “telemedicine,” and many of these characteristics can have significant impacts on both effectiveness and costs associated with healthcare delivery via telemedicine. 5,7 Costs are especially subject to variation according to how technological components of a program are allocated and priced or as equipment and connectivity costs vary over time and between regions. Because of these complexities, “telemedicine” may not exist in any generic form, making general evaluations of its costs and effectiveness idiosyncratic and unreliable at best and keeping specific evaluations of innovative but difficult-to-generalize programs of continuing interest to researchers. Furthermore, it is likely that facilities adopting telemedicine services do so for a variety of reasons, only some of which have to do with research evidence or societal costs. 8,9 Evaluations of telemedicine programs from the perspective of the implementing organization may help elucidate sustainability issues and important benefits that can justify program expansion.
In 2009, the Indiana Rural Health Association received grant funding to increase access to mental healthcare in rural parts of Indiana through forming “peer-to-peer” telemedicine networks among rural mental health service providers in Indiana. It was believed that these peer-to-peer networks would allow rural clinics, hospitals, and other providers to work together in innovative ways to improve access to care and find viable ways to sustain their telemedicine programs.
The Otis R. Bowen Center for Human Services, a private not-for-profit comprehensive community mental health center, was among the first organizations to implement such a program. The Bowen Center serves a five-county area in northeast Indiana with a staff of approximately 500 spread among nine sites, eight of which are located in rural mental health provider shortage areas. Driving time between sites can exceed 2 h.
The Bowen Center implemented telemental health clinics to support its own advanced practice nurses (APNs) at two of its most rural locations. These providers were already traveling to multiple clinic sites on a weekly basis, providing medication evaluations and medication management follow-up services. After implementation of the telemedicine program, these APNs continued to provide the same services via live interactive two-way video at two sites while continuing to travel to four other sites to provide the same services in person. Psychiatrists and other APNs provided these same services in person at other Bowen Center sites. This situation provided a naturalistic quasi-experimental factorial design in which multiple comparisons across several factors could be examined. This program is explained in detail in an initial evaluation presented elsewhere. 10
The financial sustainability of this program, from the perspective of the Bowen Center, was predicated on the ability of these clinicians to maintain the same level of quality while seeing as many patients as possible during the times they were scheduled at each site. Long drives combined with high rates of “no-shows” made in-person clinics at these sites financially prohibitive. The use of telemedicine to staff these clinics allowed for the elimination of driving time and increased flexibility of scheduling. In an effort to further increase sustainability, the Bowen Center added open-scheduled or “walk-in” appointments during telemedicine hours. This system was intended to make follow-up appointments more convenient for clients and less likely to be missed.
A 13-month analysis of data from this program found significant increases in efficiency relative to the standard medication management clinics offered in other Bowen Center locations. Wait times to first appointments were cut nearly in half, and similar reductions were shown in time to first follow-up. 10 This initial analysis left open the possibility that differences in access were the result of either program novelty or the existence of “new” previously unavailable clinical hours during the startup period. A follow-up evaluation was conducted at 24 months to replicate the previous findings and explore the results further.
Materials and Methods
Evaluation Constructs
Methods of the evaluation were explained in detail previously 10 and will be only summarized here. The telemental health program was evaluated using three elements drawn from a four-component model that included access, quality, outcomes, and costs. Clinical outcome was an initial measure targeted for analysis, but a routine clinical outcome measure had not been implemented in time for the evaluation.
Access was operationalized as “time to first appointment,” with shorter times indicating better access. Quality was operationalized as the mean time between appointments, assuming that reliable and timely follow-up would be associated with higher service quality for medication management services. Follow-up medication management services after an initial evaluation were generally targeted for as close to 30 days as possible, suggesting that 30 days would be an expected lower limit for this measure.
Costs can be measured in multiple ways and from a variety of perspectives. 11 For this study, cost was conceptualized from the perspective of the provider and calculated in terms of revenue generation because sustainability of the telemedicine program was the primary focus. Direct costs for both traditional and telemedicine clinics were fixed (clinicians and support staff), covered by grant funds (telemedicine equipment), or included in normal operating costs (network connectivity). Because of these similarities, relative billing revenue became the main financial differentiator of the two service delivery methods. This “relative revenue” construct was calculated as the percentage of scheduled clinician time converted to billable time. Operationalizing financial impact in this way had the additional benefit of focusing the results on program sustainability in terms applicable to the host organization.
Walk-In Services
The Bowen Center allowed open-scheduled “walk-in” services during the telemedicine clinics in an effort to increase efficiency and sustainability of the telemedicine program. Patients were referred to a walk-in clinic if they missed two consecutive scheduled appointments. When referring to a walk-in clinic, staff would give patients a target date for a follow-up but not a specific time. Patients could come on that date or any subsequent date during “open clinic” hours. Those who came were seen between other appointments or after scheduled patients were finished.
The potential confounds in this design were addressed by using clinicians and sites as their own controls when possible, comparing telemedicine clinics with in-person clinics staffed by the same APNs and comparing telemedicine sites with other sites that offered the same services using traditional methods.
Data Collection
Data were collected retrospectively from the existing electronic medical record and administrative systems. Patient identifiers were removed prior to analysis to maintain confidentiality, and data were handled using appropriate security measures. Data for all medication management services provided at all Bowen Center sites were used to generate comparisons among sites, providers, and service delivery methods. Services were also examined within individual clients to allow comparisons of the time between follow-up sessions. All statistical analyses were done using SPSS software (SPSS, Inc., Chicago, IL).
Results
In total, 350 patients had a telemedicine visit during the 24 months reported here. Demographic variables for the group that received telemedicine and the entire population served by the Bowen Center are shown in Table 1. The telemedicine sample is about 4.7 years older on average than the general population served at the Bowen Center but is otherwise similar.
Demographic Variables
ADHD, attention deficit hyperactivity disorder.
Of the 350 clients receiving telemedicine services over 2 years, over half (n=179) received only telemedicine services for medication management during that time. Of the 171 clients who had both telemedicine and traditional services for medication management, 70 (41%) had a telemedicine visit first and subsequently received traditional services, whereas 101 (59%) had traditional services first and then had subsequent telemedicine services. The average number of telemedicine visits per client was 2.61 (standard deviation [SD] 2.16) for those who had only telemedicine visits and 2.71 (SD 4.53) for those who had both telemedicine and traditional services. Patients who had only traditional services averaged 5.31 visits (SD 5.14) over the same 2-year period.
Access—days to First Appointment
Two analyses were done to compare the number of days between the first request for services and the date of first actual service. In the first analysis, days to first service was examined for the two nurse practitioners who provided both telemedicine and traditional encounters at multiple sites (n=10,483 encounters at six sites), comparing wait times for telemedicine services with those for traditional services. Days to first service was 19.1 (SD 21.0) for telemedicine services compared with 33.1 (SD 28.4) for traditional services (F 1,10,481=400.95, p<0.001), indicating that telemedicine appointments were available significantly sooner than traditional appointments for these two providers, regardless of site.
The second analysis focused only on two sites where both traditional and telemedicine medication management services were available from a range of providers. Three APNs (one traditional and two via telemedicine) and three psychiatrists (all traditional) provided services at these sites. At these two sites (n=17,337 services from six providers), telemedicine services for medication management were available in an average of 19.1 days (SD 21.0) compared with 37.2 days (SD 29.0) for traditional services (F 1,17,335=615.0, p<0.001). Again, telemedicine services were provided significantly sooner than traditional services, even when limiting the analysis to sites where both types of services were available.
Quality—days Between Medication Appointments
Average days to follow-up was compared for all patients who received at least one telemedicine visit (n=350). This group had all received an initial telemedicine service at some point during the year and most went on to receive subsequent services of either type. Patients with only one visit were excluded. For these clients who had multiple medication management visits, telemedicine follow-up services were available in less than half the wait time of traditional services, on average in 33.5 days (SD 26.2) compared with 77.6 days (SD 64.3) for traditional follow up services (F 1,342=77.8, p<0.001).
Service Efficiency—Percentage Converted to Billable Hours
To evaluate changes in service efficiency and sustainability, the amount of time scheduled for direct services was compared with the amount of billable time actually provided in direct services for all medication services provided by the two nurse practitioners at all sites they served. Efficiency was calculated as the percentage of scheduled direct service time converted to billable services within each service modality (traditional versus telemedicine). Medication services are traditionally overbooked in these clinics to compensate for the high rate of nonattendance. This was done at both traditional and telemedicine sites.
Analysis revealed that scheduled time converted to billable time was about 20–30 percentage points higher for telemedicine clinics than for traditional, in-person clinics (Table 2), with a difference of 20–30 percentage points between the two clinicians. Further examination of the data was conducted to determine the contribution of walk-in patients to the overall number of patients seen. This revealed that walk-in visits accounted for 20% of the encounters in telemedicine clinics overall but only 6% in traditional clinics, suggesting that most of the difference in service efficiency between telemedicine and traditional services was attributable to greater numbers of walk-in services provided during telemedicine clinics.
Service Efficiency as Measured by Percentage of Scheduled Minutes Converted to Billable Minutes
Percentages higher than 100% indicate clinics that ran longer than scheduled because of overbooking and walk-in visits.
Advanced practice nurse (APN) 1 left the Bowen Center after Year 1.
Telemedicine percentages are higher than in-person for both APN 1 and APN 2, years pooled: F 1,95=7.58, p<0.01 and F 1,384=34.24, p<0.001, respectively.
Discussion
Results of this follow-up analysis suggest that the telemedicine clinics implemented at the Bowen Center retained the characteristics seen in the initial 13-month evaluation. Specifically, they provided significantly more rapid initial access and consistently shorter wait times for routine follow-up than traditional in-person services. This pattern held across sites and clinicians and was even seen within individual clinicians who used both traditional and telemedicine services. Service efficiency also remained higher with telemedicine, with the APN who remained for both years continuing to show about 20 percentage points more of her scheduled clinical time converted to billable time during the second year. Volume due to “walk-in” patients remained approximately the same for both years in the telemedicine clinics (20%) but was significantly lower for traditional clinics (6% average across sites).
Some of these results may be due to hidden systematic differences between the conditions compared. The second year of data included only one APN doing telemedicine at the sites studied. Results from this single practitioner were consistent across the 2 years, suggesting a stable effect, but the generalizability of the results is still difficult to determine without further study. In addition, the study included multiple clinic sites that might have had their own idiosyncrasies in scheduling that could have had a disproportionate impact on the results. Finally, differences in practice style and patient panels between the psychiatrists and nurse practitioners could have influenced the results in unknown ways.
It is interesting that although all of these weaknesses were understood by Bowen Center management, their positive evaluation of the financial benefits of the telemedicine clinics was consistent and unshakeable. By their understanding, other factors that may have influenced the findings were considered either constants or irrelevant to the sustainability of the program. The greater accessibility and improved billing efficiency were enough to convince them to continue and even expand the program at their expense. Further exploration of the data and informal interviews of staff suggested that the increased efficiency for the telemedicine clinics was due in part to increased popularity of these clinics and in part to a more efficient use of time on the part of both clinicians and patients during the video encounters.
These observations suggest that the use of telemedicine opens up possibilities for new service design and delivery methods (in addition to removing the barriers associated with distance) in multiple ways. New service designs may introduce synergies of effectiveness not available in traditional services. Unfortunately, such new and potentially synergistic services may not be directly comparable to the services they replace or extend. In this study, the telemedicine clinics had several potential advantages (novelty, sites that were historically underserved) that were not measured but that traditional clinics were unlikely to have shared. These factors make results observed more complex and more difficult to interpret accurately. Despite these interpretive difficulties, however, the evidence here suggests that telemedicine clinics can take advantage of potential synergies to increase service access, quality, and efficiency and can do so past the time when any initial sense of novelty has worn off.
Footnotes
Acknowledgments
The authors wish to thank Ms. Kathy Cook and Mr. Matt Serricchio for their help in collecting the data used in this evaluation. We would also like to thank Mr. Kurt Carlson and the staff of the Otis R. Bowen Center for Human Services for their help in bringing this project to fruition. This work was partially funded by grant G22RH24745 from the Office for the Advancement of Telehealth, Health Resources and Services Administration.
Disclosure Statement
No competing financial interests exist.