Abstract
Introduction
In the digital age, telemedicine provides an effective way for health care professionals to deliver care beyond the constraints of the traditional in-person visits. By increasing access to health providers, telemedicine is advancing the health care system toward patient-oriented, value-based care. The benefits of telemedicine to patients include increased adherence to doctor's orders, savings in cost, faster and increased access to treatment, and improved patient experience. 1 In the case of pediatrics, telemedicine offers the added benefit of reducing school absences. 2 A Canadian study on attitudes toward telemedicine in pediatrics reported that a majority of families support the use of videoconferencing to communicate with physicians. For providers, telemedicine offers enhanced communication among caregivers and increased capacity for delivering high-quality care. 3 Seeing this need and array of benefits, hospitals are increasingly adding telemedicine programs to the portfolio of their services. In pediatrics, these have spanned general and specialty care, including feeding disorders, 4 surgery follow-up care, 5 intensive critical care, 6 and childhood obesity. 7 However, there are few large comparative studies that focus on direct-to-patient telehealth services in pediatric populations, and still fewer that are subspecialty oriented.
Another factor favoring pediatric telemedicine is the rising number of children coupled with a shortage and the continuing decline of pediatricians in the United States. 8 With the number of children in the United States projected to reach 77.9 million by 2050 (up from 73.9 million in 2016), there is a growing need for fostering more efficient pediatric care services through programs such as virtual visits. 9
The children's hospital in this study (hereinafter termed CHN) is a world leader in pediatric care and a pioneer in telemedicine. Its virtual visit program started in late 2016 and focuses on patients who require follow-up visits after an outpatient clinic or surgery. Currently established in 11 specialties, the program is growing monthly in terms of the total number of clinicians in a variety of subspecialties. This study focuses on the satisfaction and efficiency of completed virtual visits in cardiovascular (CV) surgery, urology, and ophthalmology.
Several papers in the field have noted the need for larger, prospective studies, since small sample sizes make it difficult to make generalizations across pediatric telemedicine. In 2016, the Wisconsin Institute for Surgical Outcomes Research conducted a large review of studies in telemedical postsurgical care; of the 21 reviewed studies, almost all had too small of sample sizes to effectively detect significant differences. 10 The vast majority of them had sample sizes <100 patients, and the largest had 519. As an example, the yearlong pilot study in pediatric urology at Arkansas Children's Hospital only involved 61 patients 11 ; another study that reported on patient and provider attitudes toward postdischarge videoconferencing for patients with congenital heart disease involved only 27 families. 12 The study presented in this article comprises data from 814 distinct telehealth visits and 14 in-person visits, and provides a more comprehensive view into modern pediatric telehealth. Furthermore, this research is unique in its comparison of three pediatric surgical subspecialties in addition to three scheduling methods. To the best of authors' knowledge, scheduling methods for pediatric telehealth visits have not yet been studied and reported in the literature, and this is the first article to do so.
Materials and Methods
Virtual Visit Data Collection
Data were analyzed retrospectively from 814 virtual follow-up visits in the CV surgery (80%), urology (12%), and ophthalmology (7%) departments of CHN. The visits took place from November 2016 to January 2018. Surveys were administered online through Redcap immediately after each visit, where patients (or their parents) and providers were asked questions regarding the visit and to note any technical difficulties they encountered. The wait time and visit duration data were collected through SBR Health. In addition to data analysis, interviews with clinicians provided qualitative descriptions of how the virtual visit program operates.
The virtual clinics were performed by a variety of provider levels: physician assistants (PAs), medical doctors (MDs), and registered nurses (RNs). This study comprises visits conducted by two PAs in CV surgery, two MDs and four RNs in urology, and two MDs in ophthalmology (Table 1).
Breakdown of Visits by Department and Provider Type
MD, medical doctor; PA, physician assistant; RN, registered nurse.
The three departments involved in this study schedule virtual visits differently: CV surgery performs full virtual clinics, with virtual visits scheduled for an entire day with no in-person visits in between. Urology uses block scheduling, with a morning block (7:30–9:00 am) and an afternoon block (4:00–5:45 pm) reserved for telemedicine visits. Ophthalmology schedules virtual clinics intermittently, so that telemedicine visits are interleaved with in-person visits.
Technology Used for Virtual Visits
The virtual visit software is an add-on to the videoconferencing software Vidyo, providing additional features such as scheduling, multiparty phone calls, and virtual waiting rooms. 13 Providers use their hospital-administered computers or personal devices, primarily Windows desktops/laptops and Mac desktops/laptops. Some departments occasionally use iPads or iPhone/Android phones. In this study, no significant technological difficulties were reported, in spite of the variety of hardware and software used.
In-Person Visit Data Collection
Data from in-person visits were collected from CV surgery and urology clinics in April and May 2018. Patient wait time was calculated as the time elapsed between check-in and the physician entering the examination room. Visit duration was defined as the amount of time that the physician spends in the examination room with the patient. Data on visit satisfaction were collected through Press Ganey surveys.
Statistical Analysis
Python's scipy implementation of the Kruskal test was done to calculate significance levels. 14 Values of p < 0.05 were considered statistically significant. Variables were summarized using mean and/or median as appropriate.
Results
Post-Visit Satisfaction Results
Reported patient and provider satisfaction was consistently very high. In the Redcap survey administered after the virtual visit, patients and providers were asked “On a scale from 1 to 10 (where 1 is the worst possible experience and 10 is the best possible experience), how would you rate the virtual visit experience overall?” The average patient rating was 9.5, and the average provider rating was 9.4 (Table 2 and Fig. 1). These ratings are slightly higher than those for in-person visits gathered through Press Ganey patient surveys (where patients were asked to report “Satisfaction with care received during the visit”), which were 9.4 in urology and 9.2 in ophthalmology.

Overall and technology satisfaction ratings for virtual visits, by patients and providers.
Survey Satisfaction Ratings for Virtual Visits
Values shown are means, medians with interquartile ranges, and sample size.
As for the technology, patients and providers were asked “On a scale from 1 to 10 where 10 is the best possible experience and 1 is the worst possible experience, how would you rate the use of technology (video, voice quality, connection) during the visit?” Patient and provider satisfaction rates were 9.1 and 9.3, respectively (Table 2 and Fig. 1).
In the survey, 36 patients and 111 providers elaborated on a technical issue they experienced (Table 3). In some cases, the doctor could see and hear the patient, but the patient could not see the doctor. Providers experiencing audio problems sometimes supplemented with a phone call. Difficulties were often mitigated by the patient propping up the phone on a stationary surface, moving closer to the router or to a better lit area, switching from WiFi connection to mobile (3G/4G/LTE), or downloading the software and preparing an account ahead of time. In CV surgery, the clinical coordinator noted that using a phone was more effective than laptops. In total, there were <20% of patients or providers experiencing technical difficulties, and all of them were solved in real time. In the future, the number of technical difficulties can be minimized by adequately preparing the patient ahead of time with these pieces of advice.
Common Technical Difficulties Encountered by Patients and Providers Who Responded to the Question “If You Experienced a Technical Issue, Please Provide Details About the Issue”
Providers were also asked “How effective was the virtual visit in delivering the same care you provide during an in-person visit?” Of the 803 providers who responded to this question, 749 (93%) responded with “very effective,” and only 16 (2%) responded with “very ineffective” or “somewhat ineffective” (Fig. 2).

Provider ratings for care effectiveness of virtual visits compared with in-person visits.
Patients were asked “How likely are you to schedule another virtual visit in the future?” Of the 209 patients who answered, 182 (87%) responded with “very likely,” and 8 (4%) responded with “very unlikely” or “somewhat unlikely.”
In addition, patients were asked to check whether they received the following benefits from their visit: Convenience: less time away from work/school; ability to include other family members as part of the visit. Improved access: ability to complete visit remotely from any location. Efficiency: not going to the hospital in person (commute, parking, registration, check-in/out, and waiting time). Quality care: care delivery that meets your needs (environment, good level of engagement and interactions, provides for privacy).
Of 210 patients surveyed, each of these benefits was experienced by a majority of them. The most widely experienced benefit was convenience, with 178 (85%) of patients reporting increased convenience. In addition, 126 (60%) of patients experienced improved access, and 157 (75%) reported increased efficiency. However, only 107 (51%) reported quality care as a benefit; this number is surprising taking into consideration the near-perfect satisfaction ratings and the 93% of providers who believed that they provided “very effective” care compared with an in-person visit. However, it is unclear whether the patients interpreted this question as comparative to in-person visit, in which checking the box would indicate that they received higher quality care than they would have in an in-person visit. This finding may also indicate a disparity between the perceived effectiveness of a visit from a patient and provider perspective. Additional research on clinical outcomes and more targeted survey questions will be useful in drawing further conclusions.
Analysis of Visit Duration
The average televisit duration was 9.9 min in CV surgery, 9.1 min in urology, and 17.4 min in ophthalmology follow-ups (Kruskal H = 49.3, p < 2e-11). However, there were significant differences in time spent per visit between the different providers within each department: ophthalmology (Kruskal H = 11.3, p < 0.0008), CV surgery (Kruskal H = 18.3, p < 1.9e-5), and urology (Kruskal H = 11.6, p < 0.05). Furthermore, the CV surgery virtual visits were conducted by PAs exclusively, and ophthalmology visits were conducted by MDs exclusively; only urology had variation in provider levels, possibly stemming from the differences in the timing of their visits. In the first 2 days postsurgery, RNs check for infections and hematomas, whereas MDs focus on the longer term result.
In the urology department, 2 MDs and four RNs performed 49 and 51 encounters, respectively. The mean visit duration was 9.4 min for the MD visits and 8.9 min for RN visits (Kruskal H = 7.1, p < 0.008). There was no significant variation in visit duration among the MDs in urology (Kruskal H = 2.4, p > 0.12) and the RNs in urology (Kruskal H = 5.4, p > 0.14). These results suggest that visits are slightly longer with MDs than with RNs, at least in the urology department. For urology in-person visits, data were collected on 14 patients seen by a single physician. The average face-to-face visit duration was 10 min, comparable with the 9.1-min average for virtual visits (Table 4).
Differences in Visit Duration Across Provider Levels in Urology
p < 0.008.
Due to difficulties in gathering live data, a comparison was done against figures reported in the past studies. The average live clinic duration time for urology (10 min) was below the national average: a retrospective study on adult primary care visits reported an average visit duration of 20.8 min in 2005 15 ; another reported a range of 13.4–19.3 min, 16 and yet another policy report gave a figure between 15 and 20 min for primary care physician/specialist time spent per visit when no procedures were performed. 17 The average urology virtual visit duration (9.1 min) was on par with the average CHN urology visit duration.
Analysis of Wait Time
Upon analyzing the average provider wait time for virtual visits, 23 outliers (above the 97% percentile) out of 814 visits were removed from consideration. Of the 791 remaining virtual visits, the mean visit duration was 3.8 min and the median was 2 min (Table 5).
Wait Times and Visit Durations by Department for Virtual Visits
While the average time spent in a virtual visit is comparable with the time spent in an in-person visit, the wait time is drastically reduced. This allows patients to spend less time in transport and waiting. Virtual and in-person visits comprise the same procedures; however, vital sign collection was omitted in virtual visits.
In urology, the average wait time for in-person visits (time between check-in and the physician entering the examination room) was 23 min, ranging from 6 to 79 min. An alternate definition of wait time is the time elapsed between the scheduled appointment time and the physician entering the room; in-person visits averaged 20 min for this adjusted wait time, and ranged from −10 to 106 min, where a negative wait time indicates patients seen before their scheduled time. Extremely long wait times were due to unexpected findings and delays in receiving hospital test results. Compared with an average in-person wait time of 23 min, the average wait time for urology virtual visits was only 6.8 min.
For CV surgery in-person visits, data were only collected on one patient whose parents required a translator. Due to limited examination room space, CV surgery is currently transitioning to a full virtual postoperative program; live clinics will soon be completely phased out, except for the cases in which physical examination is needed. The family waited 7 min in the examination room for the translator and spent 14 min in the examination room with the PA. In the future, integrating translators into the virtual process will streamline the process for non-English speakers.
A comparison against previously cited figures was also done for wait times: Healthcare Finance reported a national average of 18.5 min 18 ; a study on outpatient urology clinics reported a wait time of 14.8 min for patients who arrived on time. 19 The average virtual visit wait time is lower than the average in-person wait time as well as the reported numbers in urology and CV surgery, suggesting that transitioning to virtual visits can increase efficiency while maintaining direct physician-to-patient care. A summary of visit durations and wait times can be found in Table 5.
Impact of Scheduling Methods on Average Visit Wait Time
The CV surgery department, which only schedules telemedicine visits in full-day blocks, reported the lowest wait time (mean 2.9 min), whereas ophthalmology, with telemedicine visits scheduled intermittently, has the highest wait time (mean 9.8 min). Urology, which schedules telemedicine visits in two uninterrupted blocks, has a smaller wait time than ophthalmology but a longer wait time than CV (p < 1.1e-26). These results indicate that reserving exclusive blocks of time for telemedicine visits reduces average wait time, and increases efficiency for both the provider and the patient (Table 5). Scheduling telemedicine visits in succession reduces time spent setting up and taking down equipment, as well as transitions between in-person and telemedicine visits.
Discussion
In multiple studies, telemedicine has been shown to be a cost- and time-saving tool. 10,20 –24 Looking at postsurgical patients who have been discharged and are typically in a stable condition, these lower risk situations make excellent cases for provision of virtual care. With postoperative care as the fourth leading reason for hospital visits, 25 virtual follow-up visits offer convenience and comfort while sustaining patient and provider satisfaction. The postvisit survey data suggest very high satisfaction with the virtual visit program, which is consistent with the past studies. 7,11,26,27 In addition, virtual visits allow patients the same amount of face-to-face time with a physician, while significantly cutting down on waiting time. Opting for virtual visits offers convenience for patients and parents, reduces the risk of contracting an infection from a hospital, and may lead to ecological benefits due to reduced transport. 28 However, not all visits can be conducted virtually. There are special cases where an in-person visit is required. Having demonstrated significant discrepancies in wait time between different scheduling methods, it is recommended to set aside blocks of uninterrupted time for telemedicine visits to maximize efficiency.
Telemedicine can be extended to solve the problem of scheduling postsurgical appointments. Long appointment wait times are correlated with low hospital ratings, and even lead to patients cancelling appointments or switching providers. 29 In major U.S. cities, appointment wait times have increased by 30% since 2014; the current national average is 24 days (52 days in Boston). 30,31 Efficient telemedicine programs allow providers to see more patients per day, decreasing not only time spent in the waiting room but also appointment wait times. 32,33
Many studies have focused on the applications of telemedicine for rural areas or areas that lack specialists. 4,5,7,24,27,34 Yet, this work suggests that telemedicine also benefits large academic hospitals in urban settings such as CHN, a top-ranked pediatric hospital in a city with an incredible health care network. 35 In this study, patients called in from as far as Peru and South Korea to receive specialty care, demonstrating the global value of telemedicine in providing child- and family-centered care.
Limitations
The sample size of the in-person visit group was relatively small (n = 14) and limited to urology, which limits statistical efficiency. This limitation was due to hospital regulations and the nature of the CV surgery unit, which was phasing out in-person follow-up visits at the time of data collection. Where possible, results were compared with national averages for in-person visits. Although recall bias was minimized by administering surveys immediately after the visits, another potential source of bias is the use of different survey methods for telemedicine and in-person visits.
Footnotes
Acknowledgments
The authors are grateful to Caitlin Schumann, Heather Meyers, Gordon Massey, Dr. Ankoor Shah, Dr. Michael Kurtz, Jacqueline O'Brien, and Barbara Rhodes for their valuable inputs, information, and other help provided to them.
Disclosure Statement
No competing financial interests exist.
