Evaluation Criteria for Mobile Teledermatology Applications and Comparison of Major Mobile Teledermatology Applications

Abstract

Background: Mobile teledermatology applications have enabled increased patient access to dermatologic care. For groups interested in starting a mobile teledermatology program, selection of the appropriate application can be challenging. Having pretested evaluation criteria allows for efficient, systematic assessment of mobile teledermatology applications and identification of features important for comparison. The primary aim of this study is to determine a framework for evaluation of mobile teledermatology applications and to compare two major mobile teledermatology applications available in the United States using the proposed criteria. Materials and Methods: We incorporated previous teledermatology application evaluation criteria and developed new evaluation criteria to reflect matters specific to the mobile platform. Through a systematic search, we identified two publicly available mobile teledermatology applications in the United States and applied the evaluation criteria. Results: The 13-point evaluation criteria encompass three major domains: (1) technical specifications, (2) user experience and workflow, and (3) integration and scalability. The evaluation criteria provided an effective way of assessing the two mobile teledermatology applications. Both AccessDerm version 1.0 (Vignet Corp., McLean, VA) and ClickMedix version 1.3 (ClickMedix LLC, Rockville, MD) were capable of managing consultations. These applications adopted different approaches to balancing image quality versus data transmission, managing follow-up patients, and enabling dialogue between providers. Conclusions: Mobile teledermatology provides convenient and scalable means of providing specialty care. The creation of mobile application evaluation criteria offers a useful guide for assessing future mobile applications.

Introduction

Teledermatology entails the use of communication technology to provide dermatologic care. Implementation of teledermatology has been successful in improving access to dermatologic care and patient outcomes, while maintaining diagnostic accuracy and concordance, compared with conventional in-person dermatology.^1

–6 As mobile technology continues to mature, clinicians have found novel applications of teledermatology through the mobile platform.^7,8 Studies have shown mobile teledermatology to be valuable in providing more immediate connection between patients and providers, more convenient upload and transmission of images and clinical data, and potentially more rapid response time because of immediate availability of the clinical information.^9

–12

There are relatively few comprehensive, commercially available mobile teledermatology applications. For groups interested in starting a mobile teledermatology program, selection of the appropriate application can be challenging. Having pretested evaluation criteria allows for efficient, systematic assessment of mobile teledermatology applications and identification of features important for comparison. Although applications evolve constantly, criteria that are based on clinical relevance, efficiency, and usability will continue to be important for assessing future applications.

The primary aims of this study are to determine evaluation criteria of mobile teledermatology applications and to compare the major mobile teledermatology applications in the United States. We anticipate that the results from this study will help users determine important parameters in choosing the appropriate mobile application for clinical use.

Materials and Methods

Selection of Mobile Teledermatology Applications

We conducted a search of the iOS (Apple Inc., Cupertino, CA) and Android (Google Inc., Mountain View, CA) application markets in the United States as of September 7, 2012 to identify potential mobile teledermatology applications. Applications with a niche focus (e.g., moles only) or that did not allow for consultations between physicians were excluded. This yielded two major, commercially available mobile teledermatology applications: AccessDerm (version 1.0; Vignet Corp., McLean, VA) and ClickMedix (version 1.3; ClickMedix LLC, Rockville, MD). Both applications meet the American Telemedicine Association standard for teledermatology applications.¹³

Evaluation Process

For each application, we developed and submitted 10 identical cases. We applied a set of a priori pretested criteria that were originally developed for comparison of teledermatology applications on non-mobile platforms.¹⁴ In brief, these criteria were established through a query of the American Telemedicine Association Teledermatology Special Interest Group and the American Academy of Dermatology Telemedicine Task Force.¹⁴ After seven iterations, we developed new evaluation criteria to reflect matters specific to the mobile platform. The evaluations were performed by a multidisciplinary group of dermatologists, information technicians, and potential users. In-depth interviews with the developers were conducted to gain further insight into application capabilities.

Results

Evaluation Criteria for Mobile Teledermatology Applications

The 13-point evaluation criteria encompass three major domains: (1) technical specifications, (2) user experience and workflow, and (3) integration and scalability (Table 1).

Table 1.

Evaluation Criteria for Mobile Teledermatology Applications

EVALUATION CRITERIA	DESCRIPTION
Technical specifications
Supported mobile devices and desktop capability	Types of mobile devices that support the application; the ability to display and manage information on a computer Web application
Information security	Data-level, application-level, network-level, and host-level security; HIPAA compliance
Storage and backup	Local storage versus cloud-based storage; availability of backup systems
Images	Resolution, file size, and number of images that can be uploaded per case to the mobile device; compressibility of images
User experience and workflow
User interface	User-friendliness, accessibility, and intuitiveness of the application
Work process from referring provider	Customizability of workflow; relevance of queried clinical information; ease of data entry; integration of image capture and editing features; autodetection of image quality; transmission of images and clinical history; ability to direct cases to either a specific consultant or to a pool of consultants
Work process from consulting dermatologist	Customizability of workflow; customizability of response templates; ease of data entry; transmission of recommendations
Care of follow-up patients	Ability to identify follow-up patients and provide longitudinal care; ability to view previous images and recommendations
Communication among providers	Communication between referring providers and dermatologists; communication among dermatologists for second opinion; communication between dermatology trainees and faculty
Integration and scalability
Electronic health record integration	Integration of existing electronic health records systems and applications
Billing integration	Ability to capture and/or process billing information directly to a third-party payer
Scalability	Ability to adapt to high consult volume and multiple users
Cost-effectiveness	Consider set-up and user costs in relationship to application capabilities

HIPAA, Health Insurance Portability and Accountability Act.

Technical specifications entail the types of mobile platforms supported by the application and how the application handles information security, transmission of media, and data storage. These factors are especially important in mobile applications, given the need for multiplatform communication and the concern for information security over a telecommunication network.

User experience and workflow cover the user-friendliness of the application interface as well as how the application facilitates communication between referring and consultant providers. In addition, how the application addresses follow-up patient care is also considered. These user experience and workflow features will be central to provider satisfaction.

Integration and scalability address the degree to which the mobile application can communicate with or be integrated into existing electronic medical record and billing systems. In addition, responsiveness of the application to accommodate significant increases in consult volume is relevant. Finally, cost-effectiveness of the mobile application relative to its capabilities is important to consider.

Application of Mobile Teledermatology Criteria to Existing Applications

We summarize the results of our evaluation in Table 2. In this section we highlight the aspects of our evaluation most likely to be relevant for healthcare professionals when selecting a mobile teledermatology application.

Table 2.

Summary of Feature Comparison Between Major Mobile Teledermatology Applications Using Evaluation Criteria

EVALUATION CRITERIA	ACCESSDERM VERSION 1.0	CLICKMEDIX VERSION 1.3
Technical specifications
Supported mobile devices and desktop capability	iOS 4.1 or above (iPhone and iPad), Android OS 2.2 or above (phone and tablet), Windows Phone 7. Application is accessible on the Web.	iOS 3.0 or above (iPhone and iPad), Android OS 2.1 or above (phone and tablet). Application is accessible on the Web.
Information security	Multilayer implementation of security with user authentication, access control, host-level antivirus and network control. Data are encrypted. Auto-logout occurs after 15 min of inactivity on Web application. HIPAA-compliant across broadband spectrum	Uses multifactor authentication and encrypts the data during on-device data entry, data transmission, post-data transmission, and database encryption when data are at rest. Auto-logout occurs after 30 min of inactivity on Web application. HIPAA-compliant across broadband spectrum
Storage and backup	No data are stored locally once case is submitted. Data are stored in servers in a DC6 data center and replicated for disaster or outage scenarios.	No data are stored locally once case is submitted. Data are stored on private virtual servers and replicated for disaster or outage scenarios.
Images	Image size is compressed to 2 megabytes prior to upload. Application can allow for images from 1 megapixel to 5 megapixels. User may submit four images. The compression rate and number of images attached may be customized on request. Current deployment does not support DICOM-compliant images.	Recommend using camera phones with 5 megapixels. Image resolution is based on phone camera capability and not limited by application. User may submit up to 20 images (iPhone) or unlimited images (Android). The number of images attached may be customized on request. DICOM integration available through third-party application.
User experience and workflow
User interface	Streamlined interface with ability to send SMS, push notifications, and e-mails for case alerts and user updates	Clean intuitive workflow with customizable options
Work process from referring provider	Patient clinical history is submitted via a preset template. Template uses free text boxes and toggle buttons. Images can be captured directly to application or uploaded from phone. Cases are submitted to a pool of consultants.	Patient clinical history is submitted via a customizable template. Default template composed of free text boxes and radio buttons. Images can be captured directly to application or uploaded from phone. Images can be tagged to body location. Cases are submitted to a single consultant or a pool of consultants.
Work process from consulting dermatologist	Claimed cases are locked and cannot be viewed by other providers. Preset response template offers two free text boxes for diagnosis and treatment.	Claimed cases are locked but can still be viewed by other providers. Preset response template offers several free text boxes and option to indicate photo quality. Response forms can be customized to include additional fields.
Care of follow-up patients	Can view previously submitted images and recommendations under case history	Can use previously assigned patient ID number to create follow-up case. Search function allows user to query all fields to view previous cases.
Communication among providers	Referring provider and consultant may add comments to a case to communicate in a “chat” format. Cases may not be forwarded to other consultant providers. Resident physicians may accept cases and forward to attending physicians for review.	Referring provider and consultant may add comments to a case. Cases may be forwarded to other consultant providers who can add comments. Resident physicians may accept cases and forward to attending physicians for review.
Integration and scalability
Electronic health record integration	Compatible with Greenway and Nextgen electronic health records using Web application	Compatible with Phyaura, Practice Fusion, and eClinicalWorks electronic health record systems using Web application
Billing integration	Billing function requires customization	Integration with electronic health records provides billing function. Can print billing documents directly from application
Scalability	Scalable. Workflow can be customized to suit practice. Web-based administrator portal enables account management, case management, and creation of groups.	Scalable. Workflow can be customized to suit practice. Web-based administrator portal enables account management, case management, and creation of groups.

DICOM, Digital Imaging and Communications in Medicine; HIPAA, Health Insurance Portability and Accountability Act; SMS, short message service.

The AccessDerm version 1.0 application's strengths lie in accessibility and ease of use. The application is supported on multiple platforms, improving accessibility. The application also supports push notifications and e-mail to facilitate communication between referring providers and consultant dermatologists. The user interface is simple and streamlined, which makes it easier for providers to learn how to use the application (Fig. 1).

Fig. 1.

(A) Home screen of AccessDerm version 1.0 from the primary care provider's perspective. (B) Active drop-down free text box after selecting “Yes” toggle button.

The ClickMedix version 1.3 application offers several notable unique features in addition to its intuitive workflow and layout (Fig. 2A). The ability to tag a picture to a body diagram allows for easy labeling of patient images (Fig. 2B). The built-in camera tutorial and image feedback field give active instruction and feedback for referring providers to improve upon image acquisition skills. Lastly, the Web interface offers built-in analytic tools that allow for a very comprehensive and informative method for viewing data.

Fig. 2.

(A) ClickMedix version 1.3 Summary screen before submitting case. (B) Ability to tag pictures to body location.

Discussion

Teledermatology continues to be an active and growing field of practice in the United States.^15,16 The use of mobile applications in teledermatology will expand healthcare access and possibly provide improved patient outcomes.^17
–19 Therefore, the selection of a secure, user-friendly, and robust mobile teledermatology application is essential.

The mobile teledermatology application evaluation criteria presented here offer an efficient and systematic approach to the selection process. Together, these criteria address technology factors, provider priorities, and workflow and communication preferences. Individual criteria within these categories offer an infrastructure for further assessment of mobile teledermatology applications.

A potential limitation to mobile teledermatology evaluation criteria is the pace of technological advancement. As future applications are developed, new technology may enable features that are not specifically addressed in our evaluation criteria. A challenge in the evaluation was to maintain stable versions of each application, as software updates can frequently change application capabilities.

This evaluation revealed several important areas of further development in mobile applications for teledermatology: (1) increased integration with existing systems of electronic medical records, (2) availability of built-in reimbursement and billing capability, and (3) mechanisms for easy recall of previous consults on the same patient for longitudinal follow-up. As mobile technology continues to progress at a rapid pace, comparative analysis and user feedback will lead to further improvements in the field of mobile teledermatology technology.

Footnotes

Acknowledgments

The authors thank developers Dave Klein and Praduman Jain of Vignet Corp. and Ting Shih of ClickMedix, LLC for communicating with the research team regarding the capabilities and functionalities of their applications and allowing us to trial demo versions of their platforms.

Disclosure Statement

No competing financial interests exist.

References

Pak

, Edison

, Whited

. Teledermatology: A user's guide. New York: Cambridge Press, 2008.

Warshaw

, Hillman

, Greer

et al. Teledermatology for diagnosis and management of skin conditions: A systematic review. J Am Acad Dermatol, 2011; 64:759–772.

Hsiao

, Oh

. The impact of store-and-forward teledermatology on skin cancer diagnosis and treatment. J Am Acad Dermatol, 2008; 59:260–267.

Armstrong

, Kwong

, Ledo

et al. Practice models and challenges in teledermatology: A study of collective experiences from teledermatologists. PloS One, 2011; 6:e28687.

Edison

, Ward

, Dyer

et al. Diagnosis, diagnostic confidence, and management concordance in live-interactive and store-and-forward teledermatology compared to in-person examination. Telemed J E Health, 2008; 14:889–895.

Barnard

, Goldyne

. Evaluation of an asynchronous teleconsultation system for diagnosis of skin cancer and other skin diseases. Telemed J E Health, 2000; 6:379–384.

Lamel

, Haldeman

, Ely

et al. Application of mobile teledermatology for skin cancer screening. J Am Acad Dermatol, 2012; 67:576–581.

Frühauf

, Hofman-Wellenhof

, Kovarik

et al. Mobile teledermatology in sub-Saharan Africa: A useful tool in supporting health workers in low-resource centres. Acta Derm Venereol, 2013; 93:122–123.

Hayn

, Koller

, Hofmann-Wellenhof

et al. Mobile phone-based teledermatologic compliance management—Preliminary results of the TELECOMP study. Stud Health Technol Inform, 2009; 150:468–472.

10.

Massone

, Hofmann-Wellenhof

, Ahlgrimm-Siess

et al. Melanoma screening with cellular phones. PloS One, 2007; 2:e483.

11.

Tran

, Ayad

, Weinberg

et al. Mobile teledermatology in the developing world: implications of a feasibility study on 30 Egyptian patients with common skin diseases. J Am Acad Dermatol, 2011; 64:302–309.

12.

Azfar

, Weinberg

, Cavric

et al. HIV-positive patients in Botswana state that mobile teledermatology is an acceptable method for receiving dermatology care. J Telemed Telecare, 2011; 17:338–340.

13.

Krupinski

, Burdick

, Pak

et al. American Telemedicine Association's practice guidelines for teledermatology. Telemed J E Health, 2008; 14:289–302.

14.

Armstrong

, Sanders

, Farbstein

et al. Evaluation and comparison of store-and-forward teledermatology applications. Telemed J E Health, 2010; 16:424–438.

15.

Massone

, Wurm

, Hofmann-Wellenhof

, Soyer

. Teledermatology: An update. Semin Cutan Med Surg, 2008; 27:101–105.

16.

Armstrong

, Wu

, Kovarik

et al. State of teledermatology programs in the United States. J Am Acad Dermatol, 2012; 67:939–944.

17.

Arsand

, Tatara

, Østengen

, Hartvigsen

. Mobile phone-based self-management tools for type 2 diabetes: The Few Touch application. J Diabetes Sci Technol, 2010; 4:328–336.

18.

Braun

, Vecchietti

, Thomas

et al. Telemedical wound care using a new generation of mobile telephones: A feasibility study. Arch Dermatol, 2005; 141:254–258.

19.

Kroemer

, Fruhauf

, Campbell

et al. Mobile teledermatology for skin tumour screening: Diagnostic accuracy of clinical and dermoscopic image tele-evaluation using cellular phones. Br J Dermatol, 2011; 164:973–979.