Abstract
Objective:
Pediatric dermatology appointment wait times often exceed several months. We evaluated the usability, acceptability, and clinical impact of a store-and-forward teledermatology mobile application (app) linking families with pediatric dermatologists.
Methods:
Parents of children age 6 weeks to 17 years or individuals 18–21 years old were invited (by e-mail or referral) to participate in this single group, prospective study. Within the app, users photographed the skin condition, answered questions, and submitted their case for review. One pediatric dermatologist viewed cases, diagnosed conditions, and provided instructions and prescriptions. User surveys immediately following app use and 1 week later, supplemented by electronic logs, assessed usability, acceptability, and impact.
Results:
One hundred ninety-seven parents and one adolescent submitted cases within 39 days of invitation. App users were more likely to be white than those in the population invited (67% vs. 34%, p < 0.001) and their children were slightly younger (mean 7.3 vs. 9.0 years, p < 0.001). A majority, 83% found the app easy to use, 97% felt that submitting a case took “the right amount of time,” 87% were satisfied, and 93% would use the app again. Prescription receipt was associated with increased app satisfaction (p = 0.008). The median user received a response in 2.8 h (interquartile range 1.1–6.4). Had the app been unavailable, 44% reported that they would have waited for primary care, 32% for a dermatology appointment, and 7% would have gone to an urgent care clinic.
Conclusions:
A mobile health app allowing families to directly consult a pediatric dermatologist was usable, acceptable, and expedited care.
Introduction
In Crossing the Quality Chasm, the Institute of Medicine (IOM) argued that broad changes were needed to ensure that healthcare be more effective, patient centered, timely, efficient, and equitable. 1 Transitioning care from time-consuming office visits to virtual consultations offers one strategy to achieve IOM goals. This realization has sparked a growing interest in telehealth in dermatology, psychiatry, and other subspecialties. Given the growth in smartphone use with rates of adoption exceeding 60% even among traditionally underserved groups, 2 –4 mobile health approaches may help a broad range of families. However, despite the promise of telehealth and evidence of cost-effectiveness, 5 –8 further evaluation is needed to support the optimal integration of virtual consultation into pediatric care.
Dermatology represents an ideal paradigm to study telehealth implementation in pediatrics. For decades, physicians have used photography to document and track skin conditions. 9 Teledermatology was established as a form of telemedicine in the 1990s, 10 and mobile devices have facilitated teledermatology for adults. 11 –20 Teledermatology may be delivered in either a videoconference format or through store-and-forward methods, in which a patient or referring provider takes digital images and transmits them to the dermatologist for later review. Diagnoses achieved by both methods highly correlate with in-person assessment. 12 –16,21 –27 However, store-and-forward teledermatology has been more commonly used by pediatric dermatologists 28,29 and may be more readily used due to the efficiency of asynchronous communication. 27
Although teledermatology has been used to evaluate and monitor childhood skin conditions in specific settings, 30 –33 no prior study has evaluated the use of a direct-to-consumer mobile application (app) to facilitate routine dermatologic consultation for children. We conducted this study to address this knowledge gap by evaluating a store-and-forward app enabling these consultations. This article describes the results of the app pilot implementation led by a dermatology practice at an academic medical center affiliated with a primary care network. The aim of the pilot study was to describe the usability, acceptability, and preliminary impact of the app. We hypothesized that the app would be easily used and the approach would be acceptable and clinically useful.
Materials and Methods
Study Design
We conducted a single-group, prospective evaluation of the implementation of a direct-to-consumer store-and-forward teledermatology app.
Settings and Study Population
This study was conducted within The Children's Hospital of Philadelphia (CHOP) Dermatology Division and three primary care sites, purposely sampled to include varied demographic groups. 34 E-mail invitations to use the iOS (iPhone/iPad) app were sent to parents of children aged 6 weeks to 17 years and individuals aged 18–21 years receiving primary care at study sites, who had enrolled in CHOP's online patient portal, regardless of whether they had a skin complaint. In addition, dermatology practice schedulers and primary care staff referred eligible families to the app.
An “app user” was anyone using the app to submit a skin complaint (“case”) during the pilot period. In-app screening questions restricted usage to Pennsylvania residents (due to physician licensing requirements) with no prior CHOP dermatology visit related to the submitted case (with one exception). For each user, we evaluated data for the first case submitted. Deidentified data on app use was available for all app users.
Following the submission of a case, all app users were invited to complete two voluntary online surveys, 35 one immediately following app use and a second 5–7 days later, about their experience and satisfaction with the app. Participants completing both surveys were offered a $20 gift card. The Institutional Review Board (IRB) at CHOP approved this study. All participating app users completing surveys provided informed consent. The IRB waived the requirement of child assent.
Intervention
The app (Supplementary Appendices S1–S4; Supplementary Data are available online at
Outcome Measures
Participant characteristics
Data gathered through electronic app use logs (for all users) and postuse surveys provided participant characteristics, including patient sex and age as well as respondent race (white vs. non-white), ethnicity (Hispanic vs. non-Hispanic), and education (college graduate vs. less than college). Additional descriptives included app referral source (e-mail vs. clinician), prior use of a mobile device to look up health information, and prior use of electronic communication with doctors.
Usability
Usability was assessed through app usage data captured electronically. Information was collected regarding the number of users who received an invitation e-mail, downloaded the app, and submitted a case. In addition, we recorded the time to dermatologist accepting the case within the app, time to providing a diagnosis, and whether a user viewed the diagnosis.
Postusage surveys also collected information on app usability, including user perceptions of the time required to use the app, ease of navigating the app's different screens, and ease of taking clear photographs.
Acceptability
User acceptability of the intervention was evaluated through the two postusage surveys. Three outcome variables were assessed: willingness to use the app again, willingness to recommend the app to others, and overall satisfaction.
Clinical utility
We described the number of users who received a diagnosis and a prescription, as well as the range of diagnoses. We then assessed user perceptions of utility using data from the follow-up survey delivered 5–7 days following initial app use. By this time, participants had submitted a case and had the opportunity to receive a diagnosis. Measures of clinical utility included user opinion on the quality of advice and clarity of instructions, and whether families scheduled and received a follow-up visit. In addition, we asked participants how they would have addressed the skin complaint had the app not been available.
Analysis
Electronic data on app use were merged with user survey data. Participant demographic characteristics and outcome measures were described using means and standard deviations for normally distributed continuous variables, medians and interquartile ranges (IQRs) for skewed continuous variables and frequencies, and percentages for categorical variables. Multivariable models assessed the association of time to clinician response (hours), ease of taking photographs, and receipt of a prescription with overall satisfaction, adjusting for prior mobile device use, the only variable significantly associated with satisfaction in bivariate analyses. All analyses were implemented in Stata Version 13 (StataCorp, College Station, TX). Odds ratios (ORs) and 95% confidence intervals (CIs) were obtained and p-values <0.05 were considered significant. We then used marginal standardization to calculate the adjusted predicted proportion of users satisfied for each exposure. Responses to open-ended questions were tabulated and grouped by themes.
Results
Usability
Enrollment and study population
Figure 1 demonstrates the recruitment process for app enrollment. From 22,878 potential users who received an invitation e-mail, 1,520 (6.6%) downloaded the app and 203 cases were submitted within 39 days of the initial invitation; as some individuals used the app more than once, 198 unique users completed a case. Among these 198 users, nearly all submitted a case for their child, 52.5% of patients were male, and the median patient age was 6.0 years (IQR 3.0–11.0) (Table 1). Of all users, 135 (68.2%) users completed the first survey and 119 (60.1%) completed both surveys. There were no differences in patient gender, patient age, or referral source between those completing a survey and other users (all p > 0.2).
Enrollment flow.
Demographics of Mobile Teledermatology Application Users
Basic demographic information was collected from all users of the application.
Additional information collected through the initial postusage survey.
app, Application.
Among users who completed a survey, the majority were white (66.7%) and non-Hispanic/Latino (91.9%). The majority were highly educated (85.9% college degree) with previous experience using a mobile device to look up health information and using the Internet to communicate with their doctor. Of note, survey respondents were more likely to be white (66.7% vs. 33.7%, p < 0.001) and patients were younger (mean 7.3 vs. 9.0 years, p < 0.001) than the overall population invited to download the app.
Process and perceptions of app use
The median time for a dermatologist to open and accept a case (from time of submission) was 2.2 h (IQR 0.9–5.4) and the median total case turnaround time (time for the dermatologist to accept the case plus time to send diagnosis to patient) was 2.8 h (IQR 1.1–6.4).
Nearly all (98.5%) survey participants found the app easy to use (Table 2). On the initial app survey, 97.0% of users thought the amount of time to use the app was “just right.” 90.4% were easily able to navigate the different screens in the app, and 86.7% of respondents considered it easy to take clear photographs.
Participant Perceptions of Application Usability, Acceptability, and Clinical Utility
Information collected through the initial postusage survey.
Additional information collected in the follow-up survey.
Information collected from all users of the application.
Some users submitted a case that was nonspecific and did not receive a diagnosis.
The app automatically created a single push notification when a message and/or diagnosis/management plan was returned to the user by the dermatologist. Notably, no e-mail, text message, or icon notification was sent to users. With this workflow, 17.1% of those receiving a diagnosis did not view their diagnosis, a problem often requiring telephone follow-up.
Acceptability
Approximately 90% of users surveyed agreed that they would be willing to use the app again (Table 2). Almost 90% would recommend the app to friends and family and 86.6% reported that they were satisfied. 88.2% of users said that the doctor responded faster than or as expected. Adjusting for covariates, parents who reported receiving a prescription for their child were more likely to be satisfied with the app (OR 3.2 [95% CI: 1.4–7.7]), more likely to recommend it to friends and family (OR 2.7 [95% CI: 1.0–7.3]), and more likely to use the app again (OR 3.3 [95% CI: 1.2–9.1]). Standardized for covariates, the proportion of users reporting receiving a prescription who were satisfied with the app was 81.7% compared to 59.2% among those not reporting receiving a prescription. Neither reported ease of taking a photograph nor time to receive a diagnosis was associated with any of the satisfaction outcomes (all p > 0.3).
Clinical Utility
Of the 198 cases submitted, 193 were diagnosable through the app (97.7%). Eczema was most commonly diagnosed (Table 2). Half (47.9%) of patients cared for through the app reported receiving a prescription. Respondents found the app clinically useful with 78.1% reporting that all concerns were addressed. The majority of users (89.1%) reported receiving instructions from a dermatologist through the app, with nearly all (82.4%) reporting that care instructions were clear. If the app had not been available, 75.6% of users would have waited for a primary care or dermatology appointment, while 6.7% would have visited an urgent care or retail walk-in clinic. Although all users were offered a visit, 32 (16%) chose to schedule in-person visits (14 of whom needed follow-up per the dermatologist) and 26 completed visits.
Qualitative Results
Responses to open-ended questions revealed four primary benefits of the app: convenience (endorsed by 54.0% of respondents), ease of use (49.6%), speed of response (29.6%), and clinical utility (15.6%) (Table 3).
Qualitative Responses (Presented Verbatim)
Survey respondents were also asked to provide suggestions for app improvement. Suggestions included allowing additional conversation with the doctor both inside and outside the app (17.4%), improving the visibility of notifications received through the app (14.5%), and enabling users to add multiple children to the app (5.3%).
Discussion
Although studies have evaluated the use of mobile phone-based teledermatology as part of routine care for adults, 14,15,17,19,20 such studies are lacking in pediatrics. We found that an app that allowed families to directly consult a pediatric dermatologist was usable, acceptable, and positively impacted clinical care. Compared to the population invited to download the app, survey respondents were more likely to be white. The majority were highly educated with previous experience using the Internet to communicate with their doctor. A pediatric dermatologist rapidly evaluated submitted cases and provided diagnoses, and nearly all users were satisfied. Results also indicate that receipt of a prescription was strongly associated with satisfaction measures. Users lauded the app's convenience, ease of use, speed of response, and utility.
Store-and-forward interventions implemented in prior pediatric studies have used systems in which clinicians take photographs and upload/e-mail them to dermatologists 23,32,33,38,39 or parents send pictures via e-mail/hard copy. 30,31 We deployed a direct-to-consumer mobile app that collected the relevant medical history and images, enabling parents to submit cases using only their smartphone. One of the key determinants of mobile app adoption is usability. 40 The mobile app used in this pilot proved usable, with almost all users reporting the app was easy to use. In addition, although this was a pilot evaluation, the turn-around time for the dermatologist to accept the case and send a diagnosis to the patient (median 2.8 h) demonstrated an improvement over previous store-and-forward teledermatology interventions for adult patients (range 14 h to 4 days) 17,20,41 and was far shorter than typical dermatology office visit wait times. Despite high overall usability, the app's notification system did not alert all users when a diagnosis and management plan were ready. As a result, 17% of users were unaware of the dermatologist's diagnosis. This finding underscores the need for more effective notification, a process that might include a combination of text messages, e-mails, and icon notifications. No matter which system is used, our findings indicate that users should be educated upfront as to when and how a response will arrive.
The workflow developed for the mobile app described in this article addressed an important limitation of many prior teledermatology interventions: lack of documentation in patients' existing EHRs. 33,42,43 Documentation in the medical record is an important aspect of the medical home that supports care coordination between primary care and specialists. 44 Currently, integration of teledermatology consultations into the medical record is not standard, especially for commercially available, direct-to-patient teledermatology apps. 45,46 In our study, the dermatologist documented diagnoses and recommendations made through the app in the EHR. However, the system was not seamlessly integrated and the documentation was only possible because dermatology and primary care were within the same health system using the same EHR. Given the complexity in creating interfaces between apps and emerging standards to integrate m-health tools with EHRs, 47,48 further work is needed to ensure a unified and comprehensive medical record. As health information exchanges become more common, it may be possible to link a single dermatology app to multiple EHRs.
Overall, app users reported high levels of satisfaction. We also found that 82% of users were satisfied when they received a prescription versus 59% who did not. In contrast, neither time to receive a diagnosis nor ease of taking photographs, which we considered the most difficult aspect of app use, was associated with satisfaction. Although prescriptions may increase satisfaction by leading patients to feel an issue is being resolved and those children receiving prescriptions may have had more serious skin conditions, a potential concern for commercially available, direct-to-consumer teledermatology apps is that patients will receive non-evidence-based prescriptions to bolster satisfaction 49 or will not be fully informed of potential adverse effects. 45 Evidence exists that for certain conditions, care through commercial telehealth vendors may increase inappropriate antibiotic use. 50,51 To minimize medication overuse, techniques such as suggesting actions parents can take to reduce symptoms without medication and explicitly ruling out the need for a prescription, may reduce the risk of overprescribing, while satisfying families. 52 Determining how such communication techniques may best be integrated into teledermatology, and telemedicine more broadly, is an area warranting additional investigation.
Telemedicine has been widely acknowledged as a potential solution to disparities in access to healthcare. 36,53 –55 Ongoing evaluation and research is critical to determine the impact of these interventions on disparities. Our finding that this mobile app was used primarily by white, highly educated users who had previously used the Internet to communicate with their doctor suggests that additional efforts may be needed to ensure that telemedicine apps promote equity, not disparities. 56 Although the app and consultation were available at no cost in this pilot, the need for an iOS device may have been a barrier to universal use. 57 In a previous study of an EHR-linked patient portal for families with asthma, we found that a diverse group of families used the portal. 58 However, in that study, families were enrolled directly by the research team and given training on portal use. It is possible that more direct encouragement or training by providers/office staff also may be needed to encourage families from traditionally underserved groups to use teledermatology apps.
This study had several limitations. First, we only collected survey data from individuals who submitted a case. As a result, we lack information on how the app might have better served others. Second, the app was provided free of charge, which may limit generalizability of results. Surveys of participants in prior studies indicated users would be willing to pay for these services, 14,41 although one study indicated patients preferred the cost to be less compared with an in-person visit. 41 In addition, a single, highly motivated dermatologist provided consultation for app users, which may have contributed to short response times. Finally, the app was only available to iOS users and was implemented within one hospital system with a modest size population. Results may differ in a larger scale implementation.
Conclusions
An app that allowed families to directly consult a pediatric dermatologist was usable, acceptable, and expedited care for children and adolescents with dermatological complaints. Such approaches may improve access to care for families, although the finding that the app was used primarily by white, highly educated parents indicates the need to ensure these strategies do not exacerbate disparities.
Footnotes
Acknowledgments
We thank Dr. Robert W. Grundmeier and Ariel Rodriguez for providing data and/or contextual information useful in preparing this article. We also thank the CHOP Office of Entrepreneurship and Innovation, especially Paul Dehel and Kamilah Weems, for their support in facilitating the development of the teledermatology mobile app. This study was funded by a Chair's Initiative grant from The CHOP. The mobile app tested was developed by CHOP's Office of Entrepreneurship and Innovation.
Disclosure Statement
Dr. Fiks is the coinventor of the Care Assistant decision support software. He holds no patent, no licensing agreement exists, and he has earned no income from the intervention. Dr. Fiks's research team was supported by an independent research grant from Pfizer for work unrelated to this study. Dr. Fiks received no salary support. O.S.J. worked as a Research Student for the Scientific Advisory Committee on Entrepreneurship and Innovation at the CHOP, the office that funded the development of the Telederm app, but she has not received any income from the app. She holds no patent and no licensing agreement, and has earned no income from the invention. P.F. is the Vice President of the CHOP Office for Entrepreneurship and Innovation. He holds no patent, no licensing agreement exists, and he has earned no income from the telederm app. Dr. Winston is the Chair of the Scientific Advisory Committee on Entrepreneurship and Innovation at The CHOP, the Office that funded Telederm app development, but she has not received any income from the app. She holds no patent and no licensing agreement, and has earned no income from the invention. Dr. McMahon is a pediatric dermatologist and the inventor of the Teledermatology app used in this pilot study. He has not received income from this invention; however, per the institutional patent policy, he would be entitled to a share of royalties if it were to be commercialized. All other authors have no competing financial interests.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
