Eleven years of teledermoscopy in the Netherlands: A retrospective quality and performance analysis of 18,738 consultations

Abstract

Introduction

Teledermoscopy supports and enhances the early detection of skin lesions by general practitioners in primary practice through remote consultation with dermatologists. Teledermoscopy has been a regular health service in Dutch primary care since February 2009. Teledermoscopy quality and performance outcomes on Dutch general practitioner primary care 11 years after its implementation were assessed.

Methods

Dutch primary care teledermoscopy consultation data was retrospectively analysed on timestamps and responses on non-mandatory evaluation questions posed to the general practitioner and teledermatologist during the teledermoscopy consultation process. Anonymized data (February 2009–February 2020) was extracted from a Dutch teledermoscopy service database. The timestamps and evaluation questions data were subject to the teledermoscopy quality and performance outcomes. A limited cost evaluation was performed.

Results

A total of 18,738 teledermoscopy consultations were sent by 1341 general practitioners (February 2009–February 2020). For 3908 (31.9%) teledermoscopy consultations, the general practitioner requested second opinion advice which led to 712 (18.2%) extra teledermoscopy referrals of patients who would not have been referred without teledermoscopy, including skin cancer teledermoscopy diagnoses. The general practitioner followed the teledermatologists’ advice on patient referral for 8813 (88.5%) patients, reported 97.3% of the teledermoscopy consultations as helpful and 95.1% as instructive, referred 68.0% less patients with teledermoscopy availability, referred overall 59.4% less patients, and needed 5.4 minutes (median) for sending a teledermoscopy consultation. Teledermatologist's median answer and response time was 2 minutes and 2.4 hours, respectively. The estimated cost reduction was €144.18 ($164.65) (50.2%) per teledermoscopy patient.

Discussion

Teledermoscopy is a useful service in general practitioner practice for requesting dermatologist advice in primary care settings to support the detection of skin lesion at an early stage and at lower costs. Teledermoscopy could also decrease the burden of secondary dermatology care since general practitioners reported that they did not refer the majority of patients to a dermatologist after the teledermoscopy consultation compared to their initial referral decision. General practitioners reported the teledermoscopy system as helpful and instructive which could contribute to enhancement of their dermatological knowledge.

Keywords

Telemedicine teledermoscopy dermatology quality of healthcare efficiency

Introduction

Teledermoscopy is a possible approach to support general practitioners (GPs) in the early detection of skin cancer and to distinguish skin cancer from benign lesions by consulting a remote dermatologist based on patient history and dermatoscopic images supplied by the GP. Currently, skin cancer is among 52% of all reported cancers the most common type of cancer in the Netherlands. Further, the Netherlands has the highest skin cancer incidence in Europe.^1–4 Skin cancer was diagnosed in 82,800 patients in the Netherlands in 2021.⁵ Out of all secondary care disciplines, dermatology secondary care received the most patient referrals from GP primary care.⁶ Moreover, the number of patients suspected of having skin cancer is expected to rise further in the coming years.^7,8 Thereby resulting in an ever-increasing burden upon the Dutch healthcare system.^9,10 Teledermoscopy has been a field of interest since the early 1990s.¹¹ Since then, many studies have been published and presented teledermoscopy as a promising tool to overcome geographic distance and limited dermatologists available, thereby improving access to dermatology care resulting in an earlier detection of skin cancer.^12–14 Additional teledermoscopy advantages include an increase in the GPs’ knowledge of skin cancer and a reduction in waiting time to visit a dermatologist compared to conventional dermatology.^7,15–18

Even though teledermoscopy is presented as a promising service, numerous barriers have been mentioned in the literature, still resulting in a low acceptance of teledermoscopy in clinical practice. An example of such a barrier is the concern about diagnostic accuracy.^19,20 Despite these barriers, more rapid uptake of teledermoscopy was seen during the COVID-19 pandemic to prevent physical contact with patients but continue care.²¹

In the Netherlands, teledermoscopy is implemented in primary care to support GPs in diagnosing, treating and managing skin lesions. Teledermoscopy has been reimbursed and integrated into the Dutch healthcare system as a nationwide, regular health service since February 2009. Thereafter, no long-term assessment of outcomes on the intended use of teledermoscopy by GPs, its effect on GP's referral policy, GP's valuation of teledermoscopy usefulness, cost evaluation and GP's and teledermatologist’s time investments in performing teledermoscopy consultations in the Dutch general setting has been performed. The purpose of this study is to provide an overview of 11 years of the use of teledermoscopy by GPs in general practice by evaluating defined teledermoscopy quality and performance outcomes.

Methods

Teledermoscopy in Dutch primary care

In the Netherlands, GPs function as a gatekeeper by which patients must visit their GP before they can be referred to secondary care. GPs who require advice from a remote dermatologist can send a teledermoscopy consultation based on their own selection of patients with a suspicious skin lesion.

Ksyos²² is one of the largest store-and-forward teledermoscopy providers in the Netherlands. In the Ksyos teledermoscopy consultation system, the GP adds a maximum of four (dermatoscopic and overview) pictures and provides additional patient data (e.g. findings of a physical examination, anamneses, medication, (differential) diagnosis). The GP can add additional questions to the teledermatologist in a free text field. Patients provided oral consent to the GP to send a teledermoscopy consultation to a teledermatologist. The teledermatologist has to answer the consultation within two working days by providing the diagnosis which is automatically mapped on the International Statistical Classification of Diseases and Related Health Problems, 10^th revision (ICD-10) classification ²³ (providing a diagnosis is mandatory since July 2015) and advise on an in-person referral, medication, treatment and management of the patient in free text. The teledermatologist could also respond that a diagnosis could not be provided. Ksyos does not have access to an eventual histopathology diagnosis to compare with the diagnosis provided by the teledermatologist.

Upon sending and closing the teledermoscopy consultation, embedded non-mandatory evaluation questions Q1 and Q3–Q5 are posed to the GP by a pop-up in the Ksyos teledermoscopy consultation system (Figure 1; Q1–Q5). The mandatory evaluation question Q2 is embedded in the teledermoscopy form for the teledermatologist. Answer options for Q1 and Q3 include ‘Yes’ or ‘No’, for Q2 ‘Yes’, ‘No’ or ‘Not applicable’ (NA), for Q4 and Q5 ‘Yes’, ‘No’, ‘Some’. The ‘NA’ answer option (Q2) was intended for teledermatologists who did not want to explicitly respond with ‘Yes’ or ‘No’ on Q2 . These evaluation questions and answer options were designed by Ksyos and implemented since the start of the teledermoscopy service and could not be adapted for research purposes only.

Figure 1.

Teledermoscopy (TDsc) flow with evaluation questions (Q) posed to general practitioners (GPs) and teledermatologists (TD) during the TDsc consultation process. Reponses used for assessing teledermoscopy quality and performance outcomes are numbered.

GPs signed a contract with Ksyos by which they permitted to evaluate the teledermoscopy quality and performance by evaluation questions. Ethical approval was therefore not needed to evaluate the teledermoscopy quality and performance outcomes.

Study design

Using an anonymized extract of the nationwide Ksyos teledermoscopy service database, retrospective teledermoscopy consultations data from routine clinical practice performed between February 2009 and February 2020 (the first sent teledermoscopy consultation until data extraction) in the Netherlands were analysed. Data also included timestamps of creating and sending a teledermoscopy consultation by the GP to the teledermatologist, and the time needed by the teledermatologist to send a response to the GP. The fixed non-mandatory evaluation questions implemented in the Ksyos system were also subject to the teledermoscopy quality and performance outcomes. GPs and teledermatologists who performed these teledermoscopy consultations differed in their years of experience in the field of (tele)dermatoscopy.

Because GPs use a variety of technological tools (types of digital camera, smartphone and digital dermatoscope), the specific tool type was not considered in the data analysis. Consultations with missing timestamps, responses on evaluation questions or which were responded with ‘NA’ by the teledermatologist (Q2) were excluded from the analyses.

Teledermoscopy quality and performance outcomes

Teledermoscopy quality and performance outcomes were based on the five evaluations questions (Figure 1. Q; question) and various timestamps recorded by the teledermoscopy system (e.g. times that GPs created, send and teledermatologists answered a teledermoscopy consultation). An overview and detailed descriptions of teledermoscopy quality and performance outcomes and evaluation questions are given in Table 1.

Table 1.

Quality and performance outcomes with their description and used evaluation questions.

Quality outcomes	Description	Evaluation questions
1. Second opinion	The percentage of teledermoscopy consultations initiated by GPs to request teledermatologists’ advice for patients that would (without availability of teledermoscopy) not have been referred by the GP to dermatologic care.	Q1 = No
2. Extra teledermoscopy referrals	The percentage of patients referred to the dermatologist of which the GP indicated that he would not have physically referred these patients without teledermoscopy.	Q1 = No AND Q3 = Yes
3. GP following teledermatologists’ advice on patient referral.	Agreement percentage between the GP and teledermatologist on patient referral after the teledermoscopy consultation. The numerator is the total number of consultations for which Q2 and Q3 had the same response. The denominator is the total number of consultations with Yes or No as a response on Q2.	(Q2 = Q3) / (Q2 = Yes OR Q2 = No)
4. GP valuation of teledermoscopy	The percentage of consultations for which the GP indicated to have been helped/supported.	Q4 = Yes, Some
4. GP valuation of teledermoscopy	The percentage of consultations for which the GP indicated to have gained (dermatologic) knowledge.	Q5 = Yes, Some
Performance outcomes	Description	Evaluation questions
5. Teledermoscopy referral of patients otherwise referred	The percentage of patients who would have been referred to the dermatologist if teledermoscopy would not have been available, but who are not referred after a teledermoscopy consultation as reported by the GP.	Q1 = Yes AND Q3 = No
6. Overall referral in teledermoscopy	The numerator is the total number of patients when Q3 is answered with ‘Yes’. And the denominator includes the total number of patients when Q1 is answered with ‘Yes’.	((Q3 = Yes) / (Q1 = Yes))*100%
7. Time needed of GPs to create and send a teledermoscopy consultation, and teledermatologists’ answering and response time (time between sending and receiving answer)	The time (minutes, seconds) needed by GPs to create and send a teledermoscopy consultation (median, IQR), and teledermatologists’ answering (median, IQR) and response time (median, mean, IQR).	Timestamps were retrieved from the teleconsultation system (after July 2011).

GP: general practitioner.

Teledermoscopy quality outcomes were defined as the percentage of (1) second opinion request, (2) resulting in extra teledermoscopy referrals, (3) GPs following teledermatologist advice on patient referral and (4) GP valuation of teledermoscopy. Teledermoscopy performance outcomes were defined as (5) the teledermoscopy referral percentage of patients otherwise referred, (6) the overall teledermoscopy referral percentage and (7) the time needed of GPs to create and send a teledermoscopy consultation, and teledermatologists’ answering and response time (time between sending and receiving answer).

Healthcare perspective costs of an average teledermoscopy patient were assessed for all teledermoscopy consultations with responses on evaluation question Q3. The healthcare perspective costs only included the teledermoscopy consultation costs and weighted mean outpatient conventional healthcare costs. No other costs were included (such as patient travel time and costs). We compared the teledermoscopy costs to the weighted mean outpatient conventional healthcare costs for teledermoscopy patients who were either referred (Q3 = Yes) or not (Q3 = No) after the teledermoscopy consultation (Formula 1). Since 2009, Ksyos declared teledermoscopy consultation costs at the patients’ healthcare insurance company. These costs covered a dermatologist’ compensation, plus all technical, operational and legal overhead costs, for example, maintenance and innovation of the Ksyos platform, training of GPs, etc. The weighted mean outpatient conventional healthcare costs were derived from the Dutch Healthcare Authority and based on the weighted average price of the Diagnosis Treatment Combination.²⁴ This was done for 12 (85%) of the most reported ICD-10 code diagnoses in the Ksyos database. These costs were based on prices from 2020 price levels (European currency, Euro €). Euro values were multiplied by 1.142, which was the mean exchange rate in 2020, to convert to US dollars.²⁵

The intended purpose of the ‘NA’ answer options has not been evaluated before. Instead of just treating these responses as missing data, we therefore performed a sub-analysis to gain insight if the current use for providing the ‘NA’ response (Q2) by teledermatologists still matches the intended purpose. A questionnaire was sent by email to thirty random teledermatologists who had been involved in teledermoscopy consultations. Responses on this questionnaire were anonymous.

The time needed by teledermatologists to answer the teledermoscopy consultation requested by the GP was measured in time intervals on working days (8.30 am to 5.30 pm) of a 5-day working week, excluding national holidays and weekends. These timestamps were stored since July 2011. For the assessment of the third performance outcome, the time needed by GPs to create and send a teledermoscopy consultation was analysed for consultations completed within one working day (9 hours) and consultations that were created on a working day but send the day after.

Experience of GPs and teledermatologists with teledermoscopy was taken into account in analysing the time GPs and teledermatologists needed to perform a teledermoscopy consultation. The experience was defined as the number of teledermoscopy consultations previously performed by a GP or teledermatologist. A linear regression model was used to assess the impact of GP and teledermatologist experience on the time GPs and teledermatologists had needed to create and send or answer a teledermoscopy consultation respectively (with p < 0.05 as significant).

Results

A total of 18,738 teledermoscopy consultations were performed from the first teledermoscopy consultation on 26 February 2009 to the last data extraction on 5 February 2020. These consultations were sent by 1341 GPs working in 953 practices and assessed by 158 teledermatologists. Results regarding the teledermoscopy quality and performance outcomes are summarized in Figure 1 and Table 2.

Table 2.

Overview of the outcomes on the teledermoscopy quality and performance outcomes.

Teledermoscopy quality outcomes
Of the teledermoscopy consultations:
1.	31.9% was extra requested by GP due to the availability of teledermoscopy, to get dermatological advice.
2.	18.2% led to in-person referrals of patients otherwise not referred by the GP, these included teledermoscopy diagnosis of skin cancer.
3.	GPs follow the teledermatologists’ advice on patient referral in 88.5% of all teledermoscopy consultations.
4.	97.3% was reported as helpful by the GP and 95.1% was considered instructive.
Teledermoscopy performance outcomes
Of the teledermoscopy consultations:
5.	68.0% was not referred after the teledermoscopy consultation, while the GP indicated to refer these patients if teledermoscopy was not available.
6.	Overall 59.4% of all patients were not referred after the teledermoscopy consultation.
7.	The GP needed 5.4 minutes to create and send a teledermoscopy consultation, the teledermatologist needed 2.0 minutes to fill in the teledermoscopy consultation (median times), the teledermoscopy response time was 2.4 hours (median).

GP: general practitioner.

Quality outcomes

Of the 18,738 assessed teledermoscopy consultations, 12,241 had responses on both Q1 and Q3, and 3908 (31.9%) out of these 12,241 teledermoscopy consultations were considered as a second opinion request by the GP (Table 3) (quality outcome 1; Q1 = No).

Table 3.

Responses on GP evaluation questions Q1, Q3, Q4 and Q5 for consultations with responses on all evaluation questions.

	Number of consultations (%)
Teledermoscopy referral percentage (Q1 = Yes)	8333 (68.1)
Referred (Q3 = Yes)	2668 (32.0)
Not referred (Q3 = No)	5665 (68.0)
Requesting dermatologic advice (Q1 = No)	3908 (31.9)
Referred (Q3 = Yes)	712 (18.2)
Not referred (Q3 = No)	3196 (81.8)
Helpful response (Q4 = Yes, Some)	11,901 (97.3)
Instructive (Q5 = Yes, Some)	11,632 (95.1)

GP: general practitioner.

Of the 3908 patients that would not have been referred by the GP without teledermoscopy, 712 (18.2%) patients were physically referred to a dermatologist after the teledermoscopy consultation (quality outcome 2; Q1 = No AND Q3 = Yes) (Table 3). These referrals were defined as extra teledermoscopy referrals. The teledermatologist provided an ICD-10 code for 458 of these consultations since July 2015, including skin cancer teledermoscopy diagnoses.

Three consultations with missing responses and 4107 (33.6%) consultations for which the teledermatologist responded ‘NA’ (Q2) were excluded from the analysis. The GP followed up on the teledermatologists’ advice on patient referral (quality outcome 3; (Q2 = Q3) / (Q2 = Yes OR Q2 = No) (Table 4) for 88.5% (N = 8313) of the 12,241 teledermoscopy consultations. Eight out of 30 teledermatologists with years of teledermoscopy expertise in assessing teledermoscopy consultations participated in the email questionnaire inquiring them on the reasons why teledermatologists could have responded ‘NA’ on Q2 instead of ‘No’ or ‘Yes’ in the system. Teledermatologists reported that they preferred the ‘NA’ response when they would like to write their advice on referral management of the patient in free text. They preferred to leave the final referral decision up to the GP, for example, when skin lesion photos were of poor quality.

Table 4.

Follow-up of teledermatologists’ advice on patient referral (Q2) by GPs (Q3).

Q3 (GP)	N	Q2 (TD)	N
Yes	2668	Yes	1439
		No	431
		NA^a	797
		Missing^a	1
No	5665	Yes	219
		No	3271
		NA^a	2174
		Missing^a	1
Yes	712	Yes	439
		No	110
		NA^a	163
No	3196	Yes	171
		No	2051
		NA^a	973
		Missing^a	1
Total	12,241

Response is excluded from the analysis.

GP: general practitioner.

Teledermoscopy consultations with missing responses for Q4 and Q5 were excluded from the analysis of quality outcome 4 (N = 6506). GPs reported that they and their patients were helped by the teledermatologists response (Q4 = Yes, Some) for 11,901 (97.3%) teledermoscopy consultations. For 11,632 (95.1%) consultations, GPs considered the teledermoscopy consultations as instructive (Q5 = Yes, Some) as they had gained (dermatologic) knowledge.

Performance outcomes

For 8333 (68.1%) consultations, GPs had the intention to prevent an in-person referral of which 5665 (68.0%) patients actually were not referred after teledermoscopy consultation as reported by the GP (performance outcome 5).

Concerning the overall patient referral percentage, GPs indicated that without teledermoscopy they would have referred all 8333 patients (Q1 = Yes), while GPs reported that they actually referred 3380 patients after the teledermoscopy consultation (Q3 = Yes) regardless of their response on Q1 (performance outcome 6). These numbers were compared ((Q3 = Yes) / (Q1 = Yes))*100%) and overall, 40.6% of the patients were referred after the teledermoscopy consultation. And thus, 59.4% of the patients were not referred after the teledermoscopy consultation as reported by the GPs.

Ksyos declared an average of €64 ($73.08) for a teledermoscopy consultation at a patient's health insurance company. The weighted mean outpatient healthcare costs in conventional care were €287.43 ($328.25). In total, 12,269 teledermoscopy consultations had responses on Q3 with 3383 Q3 = Yes and 8886 Q3 = No responses. After the teledermoscopy consultation, 72.4% of all patients was not referred (Q3 = No) and with only costs for the teledermoscopy consultation (€64, $73.08). The remaining 27.6% of the patients was referred (Q3 = Yes) after the teledermoscopy consultation with additional (weighted mean) outpatient healthcare costs. This resulted in an average cost estimation of €143.25 ($163.59) for a teledermoscopy patient (Formula 1). This reflects a cost saving of €144.18 ($164.65; 50.2%) per teledermoscopy patient in comparison with conventional healthcare costs.

The time GPs and teledermatologists needed for creating and sending and answering a teledermoscopy consultation respectively, was assessed for 18,619 teledermoscopy consultations with timestamps (performance outcome 7). GPs created and sent 94.8% of the teledermoscopy consultations within one working day (9 hours) with a median time of 5.4 minutes (IQR 6.9, N = 17,660). For 25.3% of the consultations, GPs needed more time than the standard time of 10 minutes for a GP consultation, with 12.4% of the consultations taking longer than 20 minutes. For 962 teledermoscopy consultations (5.2%), GPs needed more than 8 hours to create and send a teledermoscopy consultation. Teledermatologists took a median time of 2.0 minutes (IQR 2.6) to fill in the teledermoscopy consultation form and 75% of all teledermoscopy consultation forms were filled in within 4 minutes.

The median response time of teledermatologists was 2.4 hours (mean: 6.7 hours; SD: 22.9 hours; IQR: 6.7 hours; Table 5) which decreased over the years. Teledermatologists answered nearly a third of the consultations within one working hour, 80.7% within one working day and 90% of all teledermoscopy consultations within a maximum of two working days (Table 2).

Table 5.

Response times of teledermatologists' in working hours by year.

Year	Consultations (N)	Mean response time (hr)	IQR (hr)	Median response time (hr)
2011	80	8.4	7.5	4.8
2012	602	6.6	7.0	2.6
2013	1360	6.0	5.7	2.5
2014	2894	5.6	5.7	2.1
2015	2152	6.7	6.4	2.3
2016	2515	8.0	6.0	2.2
2017	2400	8.0	7.6	2.8
2018	3333	6.4	6.1	2.8
2019	3005	6.1	6.9	2.8
Total	18,341	6.7	6.7	2.4

hr: hour; IQR: inter quartile range.

Linear regression modelling indicated that GPs and teledermatologists with more experience in performing a teledermoscopy consultation needed significantly less time to perform a teledermoscopy consultation with each consecutive consultation reducing the needed time by 0.5% (p < 0.01) and 0.05% (p < 0.01), respectively.

Discussion

This study showed that the availability of teledermoscopy in the GPs’ practices is a useful service for requesting dermatologist advice in order to resolve any doubts from the GPs and support them in the diagnosis and management of a patient's abnormal lesions.

As a result of the send second opinion request (quality outcome 1), patients were afterwards referred to dermatology care (quality outcome 2). For some of these cases, teledermatologists provided a teledermoscopy diagnosis for skin cancer. This suggests that these patients suspected of skin cancer had better access to secondary care, and a patient and doctor’s delay might be prevented due to teledermoscopy so that patients are treated as soon as possible. Early detection and adequate treatment of patients with skin cancer are key to decrease mortality, especially for melanoma.¹⁵

For the majority of teledermoscopy consultations, the GP followed the teledermatologists’ advice on patient referral (quality outcome 3). Interestingly, teledermatologists reported ‘NA’ as their response on patient referral for a third of teledermoscopy consultations. It seems that teledermatologists prefer to write their advice on referral management in free text for some cases. Teledermatologists are cautious in their decision making and just want to fulfil an advisory role concerning the referral and treatment decisions of patients with skin lesions. The ‘NA’ response option is thus still used as intended. We believe that these findings were very insightful.

In addition, almost all GPs said that the teledermatologists’ responses were helpful and instructive (quality outcome 4). The majority of GPs followed the teledermatologists’ advise on patient referral, possibly indicating that they were satisfied with the reaction and advice. Hence, GPs could be provided with more dermatological knowledge by teledermoscopy.²⁶

Our study reported that GPs would have referred the majority of patients if teledermoscopy was not available (performance outcome 5 and 6). This reflects that the teledermoscopy consultation provided the GP with enough information to no longer consider a referral of his patient to a dermatologist necessary. This has also been evaluated in another Dutch teledermoscopy study with a data set ranging from 2009 to 2016.²⁷ This study reported a comparable outcome that 69.4% of the physical referrals were prevented by teledermoscopy. The implementation of the Ksyos teledermoscopy service during a Belgium pilot study resulted in 71% reduction of the referral rate of patients with skin lesions,²⁸ but this study was based on data of only 54 teleconsultations. Overall, these findings could suggest that despite the differences in teledermoscopy services platforms and healthcare settings, teledermoscopy seems to prevent most physical referrals of patients over the last years and is therefore a valuable service to healthcare professionals and their patients.

We performed a limited cost evaluation and estimated a cost reduction of 50.2% for a teledermoscopy patient compared to the conventional healthcare costs. Despite this limited evaluation, these outcomes of teledermoscopy are promising for future (Dutch) healthcare cost savings. More extensive costs analysis were performed in other studies.²⁹ Surprisingly, teledermoscopy skin cancer referrals were more expensive compared to conventional care in these studies, but the time to establish a diagnosis was significantly reduced and therefore the costs were deemed worthwhile.³⁰ Although generalizability of economic analysis is limited, it was concluded by a systematic review from 2020 that teledermoscopy can be cost-effective in certain circumstances such as when the number of physical referrals can indeed be reduced.³¹

Finally, timestamps were evaluated (performance outcome 7). GPs needed on average 5.4 minutes to create and send a teledermoscopy consultation, while a regular consultation at the GP is scheduled for ten minutes.³² Some GPs spend more time on a teledermoscopy consultation than on a standard GP consultation. Yet, the time GPs and teledermatologists needed to send or answer a teledermoscopy consultation respectively decreased when they became more experienced in teledermoscopy. It is important to note that GPs could save the teledermoscopy consultation as a draft for an unlimited time, which we could not take into account in the analyses. We expect that GPs could have sent some teledermoscopy consultations faster if they would have continued their workflow by sending the teledermoscopy consultation immediately to a teledermatologist for advice. Teledermatologists answered 80% of the teledermoscopy consultations within one working day (9 hours) with a median response time of 2.4 hours (mean: 6.7 hours). These fast responses contribute to early reassurances of both the patients and GPs. This finding is especially relevant for patients with suspected malignancies in the context of the average patient waiting time of 2.8 weeks in the Netherlands for a dermatology outpatient clinic visit, with maximum waiting times for a patient referral of four weeks.³³ Reported teledermoscopy response times in our study were longer than the teledermatology response times by Börve et al.¹⁷ who reported a median response time of 1.8 h. Kips et al.²⁸ reported a median response time of 17 h. However, this comparison of results should be interpreted with caution, since both study populations were much smaller and study settings were completely different from our study.

Strengths and limitations

So far, retrospective evaluation studies of teledermoscopy quality and performance outcomes are limited. By including over 18,000 teledermoscopy consultations sent by GPs from primary care practices in the Netherlands, this is, to our knowledge, the largest retrospective nationwide teledermoscopy evaluation study ever performed. All data on consultations and evaluation questions was part of the nationwide Ksyos teledermoscopy system flow. GPs and teledermatologists were unaware of this retrospective study at the moment of sending and responding on teledermoscopy consultation requests by which study outcomes reflect the actual general practice of teledermoscopy use in the Netherlands.

Limitations of this study include that we were not able to report on patient’s histopathology diagnoses due to health record interoperability and privacy limitations. Therefore, we could not confirm the teledermoscopy diagnoses for extra and prevented physical referrals. The systematic reviews of Finnane et al.³⁴ and Warshaw et al.³⁵ both concluded that the management accuracy between in-person and teledermoscopy assessment is comparable by which both groups of patients as often received the accurate treatment. Due to privacy limitations, we could also not inspect the free text entries provided by the GPs and teledermatologists in the teledermoscopy consultation on further questions and information they provided.

Second, not all teledermoscopy consultations had responses on the evaluation questions needed to quantify the teledermoscopy quality and performance outcomes, nor could we validate the GP-reported outcomes on patient referrals. As a result, we are unable to provide the actual reasons for which GPs reported not to refer the patient after the teledermoscopy consultation in contrast to the advice of the teledermatologist to physically refer the patient.

Third, the cost evaluation was very limited and should be interpreted with caution as we only included the teledermoscopy consultation and weighted outpatient healthcare costs. Not all primary (e.g. GP reimbursement, costs of primary care consultation) and secondary healthcare costs (e.g. patient travel costs) were taken into account. And we did not correct for the different dermatology consultation rates per year. The weighted outpatient healthcare costs are thus a rough estimation of the actual costs since not all healthcare expenditures were included in this analysis. However, we believe that this cost estimation shows promising results for implemented teledermoscopy services used in real general practice.

Future recommendations

We observed that the majority of GPs followed the teledermatologists’ advice on patient referral. However, we did not assess for which teledermoscopy diagnosis groups the GP changed his referral decision. Therefore, we will assess the GP's decision on patient referral before and after the teledermoscopy consultation per group of (pre-)malignant and benign teledermoscopy diagnosis provided by teledermatologists.

Also, there are currently no validated indicators for evaluating teledermoscopy systems. Interviews could first investigate the need for such indicators. If such a need exists, then focus groups could assess in-depth the domains for which indicators concerning quality and performance are desirable. Carefully designed indicators could help in objectively evaluating the progress in quality and performance of teledermoscopy systems, as well as comparing the outcomes of different teledermoscopy systems. The embedded evaluation questions in the Ksyos teledermoscopy system may serve as a starting point for developing such indicators.

Conclusion

Based on 11 years of data, this study indicates that the availability of teledermoscopy in the GPs’ practices is a useful tool for requesting dermatologist advice to resolve doubts by GPs about the diagnosis and management concerning a patient's suspicious lesion. Teledermoscopy supported GPs to uncover new cases of patients in need of secondary dermatology care. GPs often follow the teledermatologists’ advice on patient referral and reported that the TDs’ responses were helpful and instructive. Furthermore, a majority of patients for whom the GPs requested a teledermoscopy consultation was not physically referred to a dermatologist after the teledermoscopy consultation. Hence, teledermoscopy is a useful tool to support primary care with dermatologist expertise at lower costs.

Supplemental Material

sj-docx-1-jtt-10.1177_1357633X221122113 - Supplemental material for Eleven years of teledermoscopy in the Netherlands: A retrospective quality and performance analysis of 18,738 consultations

Supplemental material, sj-docx-1-jtt-10.1177_1357633X221122113 for Eleven years of teledermoscopy in the Netherlands: A retrospective quality and performance analysis of 18,738 consultations by F van Sinderen, E Tensen, RAB Lansink, MWM Jaspers, and LWP Peute in Journal of Telemedicine and Telecare

Footnotes

Acknowledgements

The authors would like to thank Dr D.C.K.S. Tio and Dr J.P. van der Heijden for their review of the manuscript.

Declaration of conflicting interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: E.T. and F.V.S are employed (part-time) by Ksyos. The remaining authors state no conflicts of interest.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Disclosure statement

Femke van Sinderen MSc and Esmée Tensen MSc are PhD researchers at the Amsterdam UMC and are (part-time) employed by Ksyos. Prof. Dr Monique Jaspers, Dr Linda Peute and Rick Lansink MSc have nothing to disclose.

ORCID iDs

Femke van Sinderen

Esmée Tensen

Supplemental material

Supplemental material for this article is available online.

References

KWF Kankerbestrijding. Huidkanker [Internet]. [cited 10 April 2017]. Available from: https://www.kwf.nl/kanker/kwf-en-huidkanker/Pages/default.aspx (accessed 11 May 2022)

KWF Kankerbestrijding. Huidkanker [Internet]. [cited 10 April 2017]. Available from: https://www.kwf.nl/SiteCollectionDocuments/brochure-Huidkanker.pdf

Sacchetto

Zanetti

Comber

, et al. Trends in incidence of thick, thin and in situ melanoma in Europe. Eur J Cancer 2018; 92: 108–118.

Huidkanker IKNL [Internet]. [cited 11 May 2022]. Available from: https://iknl.nl/kankersoorten/huidkanker

KWF Kankerbestrijding. Huidkanker [Internet]. [cited 11 May 2022 Available from: https://www.kwf.nl/kanker/huidkanker

Nivel. Cijfers huisartsen - Verwijzingen naar de tweede lijn [Internet]. [cited 11 May 2022]. Available from: https://www.nivel.nl/nl/nivel-zorgregistraties-eerste-lijn/verwijzingen-naar-de-tweede-lijn

Gorgels

WJMJ

. De rol van de huisarts in de zorg voor huidkankerpatiënten. Bijblijven 2017; 33: 73–83.

Koelink

CJL

Kollen

Groenhof

, et al. Skin lesions suspected of malignancy: An increasing burden on general practice. BMC Fam Pract 2014; 15: 29.

Howe

Lake

Shen

. Method to assess identifiability in electronic data files. Am J Epidemiol 2007; 165: 597–601.

10.

de Vries

van de Poll-Franse

Louwman

, et al. Predictions of skin cancer incidence in the Netherlands up to 2015. Br J Dermatol 2005; 152: 481–488.

11.

Peracca

Jackson

Weinstock

, et al. Implementation of teledermatology: Theory and practice. Curr Derm Rep 2019; 8: 35–45.

12.

Edison

Dyer

Whited

, et al. Practice gaps—the barriers and the promise of teledermatology. Arch Dermatol 2012; 148: 650–651.

13.

Armstrong

Kwong

Ledo

, et al. Practice models and challenges in teledermatology: A study of collective experiences from teledermatologists. PloS one 2011; 6: e28687.

14.

Dovigi

Kwok

EYL

English

. A framework-driven systematic review of the barriers and facilitators to teledermatology implementation. Curr Dermatol Rep 2020; 9: 353–361.

15.

Morton

Downie

Auld

, et al. Community photo-triage for skin cancer referrals: An aid to service delivery. Clin Exp Dermatology Clin Dermatology 2011; 36: 248–254.

16.

Tensen

van Sinderen

Witkamp

, et al. The Value of Teledermoscopy to the Expertise of General Practitioners Diagnosing Skin Disorders Based on ICD-10 Coding. In: MedInfo. 2019. pp. 834–838.

17.

Börve

Dahlén Gyllencreutz

Terstappen

, et al. Smartphone teledermoscopy referrals: A novel process for improved triage of skin cancer patients. Acta Derm Venereol 2015; 95: 186–190.

18.

Giavina-Bianchi

Santos

Cordioli

. Teledermatology reduces dermatology referrals and improves access to specialists. EClinicalMedicine 2020; 100641: 29–30.

19.

Coates

Kvedar

Granstein

. Teledermatology: From historical perspective to emerging techniques of the modern era: Part I: History, rationale, and current practice. J Am Acad Dermatol 2015; 72: 563–574.

20.

Harvey

Valenta

Simpson

, et al. Utilization of outpatient telehealth services in parity and nonparity states 2010–2015. Telemedicine and e-Health 2019; 25: 132–136.

21.

Lowe

Atwan

Mills

. Teledermoscopy as a community based diagnostic test in the era of COVID-19? Clin Exp Dermatol 2021; 46: 173–174.

22.

Ksyos [Internet]. [cited 11 May 2022]. Available from: www.ksyos.nl.

23.

World Health Organization. Classifications [Internet]. [cited 11 May 2022]. Available from: https://icd.who.int/browse10/2019/en

24.

DIS open data [Internet]. [cited 22 April 2020]. Available from: https://www.opendisdata.nl/downloads

25.

European Central Bank [Internet]. [cited 15 July 2022]. Available from: https://www.ecb.europa.eu/stats/policy_and_exchange_rates/euro_reference_exchange_rates/html/eurofxref-graph-usd.en.html

26.

Ahmadi

Prickaerts

Smeets

JGE

, et al. Current approach of skin lesions suspected of malignancy in general practice in the Netherlands: A quantitative overview. J Eur Acad Dermatology Venereol 2018; 32: 236–241.

27.

Tensen

van der Heijden

Jaspers

MWM

, et al. Teledermatologie. Bijblijven 2017; 33: 101–113.

28.

Kips

Lambert

Ongenae

, et al. Teledermatology in Belgium: A pilot study. Acta Clin Belg 2020; 75: 116–122. Epub 2019 Jan 8.

29.

Lopez-Villegas

Bautista-Mesa

Baena-Lopez

, et al. Economic impact and cost savings of teledermatology units compared to conventional monitoring at hospitals in southern Spain. J Telemed Telecare. 2022; 28(6): 436–444.

30.

Snoswell

Caffery

Whitty

, et al. Cost-effectiveness of skin cancer referral and consultation using teledermoscopy in Australia. JAMA Dermatol 2018; 154: 694–700.

31.

Snoswell

Caffery

. Current economic evidence for teledermoscopy. Curr Derm Rep 2020; 9: 131–135.

32.

Nivel. De werkbelasting van huisartsen - Nederland in internationaal perspectief [Internet]. [cited 15 January 2022]. Available from: https://www.nivel.nl/sites/default/files/bestanden/rapport_werkbelasting_huisartsen.pdf

33.

Ministry of Healthcare, Welfare and Sports. Ziekenhuiszorg [Internet] [cited: 18 August 2021]. Available from: https://www.volksgezondheidenzorg.info/onderwerp/ziekenhuiszorg/cijfers-context/wachttijden#node-wachttijden-behandeling-2021

34.

Finnane

Dallest

Janda

, et al. Teledermatology for the diagnosis and management of skin cancer: A systematic review. JAMA Dermatol 2017; 153: 319–327.

35.

Warshaw

Greer

Hillman

, et al. Teledermatology for diagnosis and management of skin conditions: A systematic review. J Am Acad Dermatol. 2011; 64(4): 759–772.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB