Creation and Validation of a Spanish Questionnaire for Evaluating Pediatricians’ Satisfaction with Teledermatology

Abstract

Introduction:

In 2021, we introduced a program to promote the use of teledermatology among pediatricians. In the present study, we created and validated a satisfaction questionnaire to assess pediatricians’ perceptions of teledermatology. We used this questionnaire to evaluate the efficacy of the program.

Methods:

First, a provisional questionnaire, based on questionnaires available in the literature, was drafted. A group of experts evaluated the clarity, coherence, relevance and sufficiency of the questions, and we calculated a content validity index (CVI). Only questions with a CVI above 0.78 were acceptable. Based on these results and on the experts’ comments, we revised the questionnaire and then sent it to a population of pediatricians. We used their responses to validate the revised questionnaire using statistical methods. A Cronbach’s alpha above 0.7 indicated adequate internal consistency, and an intraclass correlation coefficient above 0.75 indicated adequate reproducibility. We used the Varimax method to measure construct validity.

Results:

We clarified and reformulated some questions from the provisional questionnaire based on the experts’ comments. All questions had a CVI above 0.78, so no other changes were needed. Although the statistical validation showed suboptimal construct validity, the revised questionnaire had good internal consistency and reliability, and high content validity index.

Discussion:

The validated questionnaire is a robust tool for assessing pediatricians’ satisfaction with teledermatology.

Introduction

Teledermatology enables dermatologists to evaluate images of skin lesions remotely.¹ In the field of pediatrics, skin problems account for 30% of primary care visits,² but children can wait up to 4 or even 6 months to see a dermatologist due to high care demand.³ Promoting teledermatology could help to reduce wait lists, ease the care burden of emergency departments, and improve interprofessional communication.

In asynchronous teledermatology, dermatologists evaluate stored images and clinical data of each patient to establish a diagnosis and therapeutic plan. This modality provides children with rapid access to high-quality dermatological care⁴ and enables remote management of 63% of skin lesions with no need for an in-person visit.⁵ Pediatric dermatologists favor the use of teledermatology in their routine practice.⁶ In a 2015 study by Forgel et al., 56% of dermatologists considered teledermatology an important component of dermatological consultations.⁷ In general, these professionals are satisfied with teledermatology.⁸

In 2009, Alicante General University Hospital (HGUA) developed an asynchronous teledermatology platform with its own resources. Through this system, dermatologists could manage cases via telemedicine or schedule and prioritize in-person visits or surgical treatment as needed, depending on the individual condition. HGUA expanded access to the platform to all primary care pediatricians (PCPs) in 2015 and has continued to develop the system over the years, facilitating communication between PCPs and specialists.

In 2020, another research team at the HGUA analyzed the use of teledermatology by pediatricians and found that only 10% of consultations had taken place through the system, which suggested it was being underused. We decided to intervene to optimize the use of teledermatology, establishing a series of actions to improve the availability and infrastructure of the service, as well as a personalized training program for pediatric professionals who had access to the tool. To determine the effectiveness of this intervention, it was necessary to evaluate the tool.

Evaluations of teledermatology systems should cover user satisfaction, educational capacity, and economic impact.⁹ Few published studies have evaluated user satisfaction with teledermatology, and the questionnaires used in these studies have important methodological limitations. For this reason, we developed and validated a questionnaire for evaluating the experience of professionals in the HGUA with teledermatology, based on existing questionnaires in the literature.¹⁰

Methods

We created and validated a questionnaire aimed at the 45 PCPs who have access to teledermatology in the 12 public health care centers within the HGUA catchment area. The aim of the questionnaire was to evaluate PCPs’ experience and satisfaction with the tool. With the results, we can evaluate the impact of the intervention implemented in 2021.

The validation process was granted ethics committee approval. We established a target population comprising all pediatricians in the province of Alicante and the Region of Murcia. This population has similar working conditions to the professionals who will receive the final questionnaire. We calculated the sample size following the rule of 5–10 individuals per questionnaire item, with the aim of collecting at least 200 to 300 responses for validation.¹¹

Our study involved the following steps: 1)

Development of a provisional questionnaire: Two dermatologists and one PCP adapted questionnaires published in the literature to our population. The provisional version of our questionnaire comprised 15 questions with Likert response options, grouped into four dimensions, as well as questions about the respondent’s sex, age, position, professional category, and the use of teledermatology. We designed the questionnaire to be self-administered through Google Forms.

Revision of the questionnaire: After a group of survey experts had evaluated the provisional version, we made all necessary changes to produce a revised version according to their opinions. After that, the expert evaluated this revised version of the questionnaire for clarity, coherence, and relevance of each question, according to the definitions established in Table 1. They also evaluated whether each dimension had sufficient questions. We used a template to ensure consistency (Table 2). Each item received a score from 1 to 4, and there was a space for additional comments. To decide whether a question was suitable for the questionnaire, we used the content validity index for each item (I-CVI), referred to as clarity, coherence, relevance, and sufficiency, defined as the ratio between the number of experts that provided a score of 3 or more to that item, divided by the total number of participating experts. For a question to be included in the final questionnaire, the I-CVI for every condition had to be greater than 0.78. We also calculated the CVI for the whole questionnaire (S-CVI), defined as the mean value of every I-CVI calculated. To ensure adequate content validity, S-CVI had to be greater than 0.9.¹²

Validation of the questionnaire: after revising the questionnaire and proving every item had an I-CVI greater than 0.78, we sent it to the target population (on two occasions, in March and April 2021, to guarantee reproducibility) and performed a statistical analysis of the responses. Two weeks after sending the questionnaires, we sent a reminder to all potential respondents to increase the response rate.

Table 1.

Definition of Clarity, Coherence, Relevance, and Sufficiency for the Questionnaire Evaluation

ELEMENT	DEFINITION	SCORE
ELEMENT	DEFINITION	1	2	3	4
Clarity	Whether the question is easy to understand (i.e., has adequate structure and meaning)	The question is unclear	The question requires several modifications or one very large modification related to the use of appropriate words or the order of words	A very specific modification in some terms of the question is required	The question is clear, with adequate structure and meaning
Coherence	The logical link between the question and the dimension it is measuring	There is no logical link between the question and the dimension	There is a tangential link between the question and the dimension	There is a moderate link between the question and the dimension	There is a clear logical link between the question and the dimension
Relevance	The importance of the question (i.e., whether it should be included)	Removing the question would have no effect on the measurement of the dimension	The question has some relevance, but another question may be measuring the same aspect	The question is quite important	The question is very important and should be included
Sufficiency	Whether the number of questions in a dimension is sufficient to measure said dimension	There are insufficient questions to measure the dimension	The questions measure some aspects of the dimension, but not the whole dimension	Some items should be added for a complete evaluation of the dimension	There are sufficient questions to measure the dimension

Table 2.

Template for the Expert Evaluation of the Provisional Questionnaire

We collected the data in an Excel spreadsheet using Google Forms, and we used SPSS 21.0.0.0 to analyze the data. The statistical analysis included the following measures: 1)

Internal consistency of each item in the same dimension. We calculated Cronbach’s alpha coefficient; items that scored over 0.7 were considered adequate.

Reproducibility of the questionnaire, or temporal stability in the responses to questionnaires sent at different times (March and April 2021). We considered that after 1 month, respondents were likely to have forgotten their responses, but significant changes that could alter the responses (e.g., pediatric staff changes) were unlikely to have occurred. For this measure, we calculated the intraclass correlation coefficient, considering a value greater than 0.75 indicative of adequate reproducibility.

Construct validity, which required a statistically significant Bartlett test (p < 0.05) and a Kaiser-Meyer-Olkin (KMO) index greater than 0.7. After confirming that these requirements were met, we performed a principal component analysis to study the shared and common variance of each variable and a factorial analysis, following the Varimax method and creating a rotated component matrix. The criterion for including an item in each domain of our questionnaire was a factor greater than 0.5. In this way, we determined whether the domains of our questionnaire (and their components) coincided with the components and domains detected in the factorial analysis.

After conducting the statistical analysis and making the necessary modifications to the revised questionnaire, we created the final validated questionnaire for professionals.

Results

EVALUATION AND REVISION OF THE PROVISIONAL QUESTIONNAIRE

Twenty experts evaluated the provisional questionnaire. Based on the experts’ comments, we modified the wording and response options of some questions to improve comprehensibility and thus elicit more reliable responses. We also made the following changes. 1)

We added a question about knowledge of teledermatology to the initial demographic data section to filter out people who had insufficient knowledge to provide valid responses.

We added a question about the role of teledermatology in reducing wait times: “Do you believe teledermatology speeds up the dermatological assessment of your patients?” (Question 7 of the revised questionnaire).

We detected two questions related to problems with using teledermatology during consultations that transmitted a similar idea (questions 10 and 11 of the provisional questionnaire); we combined them into a single question (question 11 of the revised questionnaire).

We split question 12 (“Are you uncomfortable with having to use your personal mobile phone to take photographs of your patient?”) into two questions (12 and 13 of the revised questionnaire).

We created a new final open question where respondents could indicate areas of improvement in teledermatology.

Table 3 compares the provisional and revised versions of the questionnaire.

Table 3.

Comparison Between the Provisional Questionnaire and the Revised Version Submitted for Validation (Green: new Question; Orange: merged Questions)

We then calculated the CVI for each question (I-CVI) and for the whole questionnaire (Table 4). All questions had an I-CVI greater than 0.78 for clarity, coherence, relevance, and sufficiency, and an S-CVI greater than 0.9 for those attributes.

Table 4.

I-IVC Values of the Experts’ Scores for Clarity, Coherence, Relevance, and Sufficiency of the Questions in the Questionnaire

I-IVC VALUES FOR THE QUESTIONS OF THE TEST
QUESTION NO.	CLARITY	COHERENCE	RELEVANCE	SUFFICIENCY
1	0.95	1	1	0.95
2	0.9	0.95	0.9
3	0.9	0.95	0.9
4	0.95	0.95	0.9
5	0.9	0.9	0.85	1
6	0.95	0.9	1
7	0.85	0.9	0.85
8	0.85	1	0.95
9	0.95	0.95	0.95
10	0.8	0.85	0.9
11	0.85	1	0.95	0.95
12	0.95	0.95	1
13	0.95	1	0.95
14	0.95	1	1
15	0.95	1	0.95	0.95
16	0.85	0.95	0.9	0.95
WHOLE QUESTIONNAIRE (S-IVC)	0.91	0.95	0.93	0.96

STATISTICAL VALIDATION OF THE REVISED QUESTIONNAIRE

We sent the revised questionnaire to our target population (Table 3), and 95 PCPs responded. All the responses were included in the validation process. The analysis of internal consistency showed a Cronbach’s alpha greater than 0.7 in the domains of “Quality” (0.765) and “Utility” (0.832). However, the index was 0.522 for “Limitations” and 0.672 for “Satisfaction”. Removing question 13 of the provisional version (“Do you believe that having a camera for the exclusive use of teledermatology would facilitate the use of said tool?”) increased the index for “Limitations” to 0.709. Cronbach’s alpha for the whole questionnaire was 0.890, or 0.911 without question 13 (Table 5).

Table 5.

Validation of Internal Consistency: Cronbach’s Alpha Coefficients

DOMAIN	CRONBACH’S ALPHA	MEAN SCORE	NO. OF QUESTIONS	SURPLUS QUESTION	CORRECTED CRONBACH’S ALPHA
Quality	0.765	7.11	4	Q2.3 (0.316)	0.822
Usefulness	0.832	9.38	6	—	—
Limitations	0.552	8.97	4	Q4.4 (0.016)	0.709
Satisfaction	0.672	2.88	2	—	—
Total	0.890	27.86	16	Q4.4 (–0.081)	0.911

Bold values in Table 5 correspond with Cronbach’s alpha coefficients over 0,7.

We measured the reproducibility or reliability of the questionnaire by sending it out a second time. We received 34 completed questionnaires. The intraclass correlation coefficient was 0.875 (95% confidence interval 0.753–0.939), confirming the temporal stability of the responses (Fig. 1).

Fig. 1.

Scatter graph of the questionnaire responses from the two time points.

The KMO test produced an index of 0.846, and the result of the Bartlett test was statistically significant (p < 0.001); therefore, we were able to conduct the factorial analysis to determine construct validity. The questions with the lowest communality indices, obtained through principal component analysis (Table 6), were “How would you rate the health care received by patients via teledermatology compared with health care received during an in-person dermatology consultation?” (0.463) and “Do you consider that teledermatology can help you improve your knowledge of dermatology?” (0.478).

Table 6.

Communality Indices Calculated for Each of the Questions through Principal Component Analysis

QUESTIONS	COMMUNALITY INDEX
1. How would you rate the performance of the teledermatology application in your hospital?	0.894
2. How would you rate the health care received by patients via teledermatology?	0.823
3. How would you rate the health care received by patients via teledermatology compared with health care received during an in-person dermatology consultation?	0.463
4. How would you rate the overall quality of teledermatology in your healthcare area (functioning health care provided capacity and speed of resolution)?	0.810
5. Do you consider teledermatology a useful tool?	0.628
6. Do you think teledermatology can improve your patient’s condition?	0.600
7. Do you believe teledermatology speeds up the dermatological assessment of your patients?	0.737
8. Do you think the diagnoses and plans issued by the dermatologist via teledermatology are reliable?	0.521
9. Do you consider that teledermatology can help you improve your knowledge of dermatology?	0.478
10. Do you think teledermatology improves the relationship between pediatricians and dermatologists?	0.722
11. Do you think the extra time spent carrying out a consultation via teledermatology is worth it?	0.708
12. Do you think taking photographs of patients with your personal mobile phone could make it more difficult to use teledermatology?	0.684
13. Do you think using a personal mobile phone to take photographs of your patients violates their privacy?	0.603
14. Do you think having a camera for exclusive use in teledermatology would facilitate the use of said tool?	0.799
15. Are you generally satisfied with teledermatology?	0.774
16. Will you continue using teledermatology in the future?	0.652

The Varimax method detected four components with an eigenvalue greater than 1, which explained 68.1% of the variance in the questionnaire (Fig. 2). Four was also the number of domains in the provisional questionnaire.

Fig. 2.

Screen plot obtained using the Varimax method for detecting the questionnaire domains.

To include the questionnaire items in each of the components, we studied the data in a rotation matrix (Table 7). For an item to be included in a component, it had to have a factor greater than 0.5. After ordering the items in the four components using this criterion, we found some discordance with respect to the domains defined before validation.

Table 7.

Rotation Matrix Obtained Though the Varimax Method for Determining the Components of Each of the Questionnaire Domains

QUESTIONS	COMPONENTS
QUESTIONS	1	2	3	4
1. How would you rate the performance of the teledermatology application in your hospital?	0.256	0.896	0.044	–0.156
2. How would you rate the health care received by patients via teledermatology?	0.380	0.755	0.188	0.270
3. How would you rate the health care received by patients via teledermatology compared with health care received during an in-person dermatology consultation?	−0.013	0.250	0.547	0.317
4. How would you rate the overall quality of teledermatology in your healthcare area (functioning health care provided capacity and speed of resolution)?	0.187	0.707	0.468	0.236
5. Do you consider teledermatology a useful tool?	0.733	0.218	0.201	–0.059
6. Do you think teledermatology can improve your patient’s condition?	0.741	0.165	–0.108	–0.105
7. Do you believe teledermatology speeds up the dermatological assessment of your patients?	0.555	0.216	0.415	0.459
8. Do you think the diagnoses and plans issued by the dermatologist via teledermatology are reliable?	0.378	0.389	0.446	0.166
9. Do you consider that teledermatology can help you improve your knowledge of dermatology?	0.527	0.128	0.428	–0.024
10. Do you think teledermatology improves the relationship between pediatricians and dermatologists?	0.688	0.271	0.308	0.283
11. Do you think the extra time spent carrying out a consultation via teledermatology is worth it?	0.647	0.319	0.403	0.159
12. Do you think taking photographs of patients with your personal mobile phone could make it more difficult to use teledermatology?	0.171	0.062	0.803	–0.076
13. Do you think using a personal mobile phone to take photographs of your patients violates their privacy?	0.201	0.145	0.728	–0.108
14. Do you think having a camera for exclusive use in teledermatology would facilitate the use of said tool?	–0.030	0.037	0.090	–0.888
15. Are you generally satisfied with teledermatology?	0.525	0.623	0.267	–0.198
16. Will you continue using teledermatology in the future?	0.666	0.252	0.162	0.344

Bold values correspond with factors over 0,5.

Component 1 explained 44.46% of the variance and included 5 of 6 items from the “Utility” domain, as well as two items from the “Satisfaction” domain. Component 1 also included the question “Do you think the extra time spent carrying out a teledermatology consultation is worth it?” which belonged to the “Limitations” domain.

Component 2 explained 8.87% of the variance and included 3 of 4 questions from the “Quality” domain.

Component 3 explained 8.17% of the variance and included 2 of 4 questions from the “Limitations” domain, as well as the question “How would you rate the health care received by patients via teledermatology compared with health care received during an in-person dermatology consultation?” from the “Quality” domain.

Component 4, which explained 6.6% of the variance, included no items.

Two items had a factor below 0.5 in all components; these were the questions, “Do you think the diagnoses and plans issued by the dermatologist via teledermatology are reliable?” And “Do you believe that having a camera for the exclusive use of teledermatology would facilitate the use of said tool?”

FINAL VALIDATED QUESTIONNAIRE

With the results obtained in the validation process and considering that Cronbach’s alpha was above 0.7 for the questionnaire as a whole, we decided to keep all the questions of the revised questionnaire, with only minor variations in the wording. Although the construct validity was suboptimal, the questionnaire showed good internal consistency, good reliability, and good content validity.

The result of our study was a validated questionnaire (Table 8) suitable for virtual dissemination through Google Forms.

Table 8.

Final Validated Questionnaire (Spanish and English Version)

FINAL QUESTIONNAIRE (SPANISH VERSION VALIDATED)	FINAL QUESTIONNAIRE (ENGLISH VERSION TRANSLATED)^a
Número de identificación (Corresponde a los 4 últimos dígitos de su DNI + Letra)	Identification number (Last 4 digits of national ID number + letter):
Sexo: □ Hombre □ Mujer Edad: Centro de trabajo: Especialidad y cargo (Por ejemplo: residente de pediatría. adjunto de pediatría. enfermería. etc.):	Sex: □ Male □ Female Age: Workplace: Specialty and position (e.g., pediatrics resident. pediatrician, and nurse):
Conoces la Teledermatología? □ Sí □ No Si la respuesta ha sido positiva puede continuar con el cuestionario	Do you have experience with teledermatology? □ Yes □ No If you answered YES. you can continue the questionnaire
¿Utilizas la Teledermatología en tu práctica habitual? □ Sí □ No	Do you use teledermatology in your usual practice? □ Yes □ Never
En caso NEGATIVO. indica el motivo por el que no la utilizas (señalar tantos como sean necesarios): □ Desconocía la existencia de la Teledermatología en nuestro departamento. □ La aplicación de la Teledermatología me resulta difícil de manejar. □ No dispongo de tiempo suficiente para su utilización. □ No dispongo de medios para la obtención de imágenes. □ Considero que los pacientes deben ser evaluados de modo preseencial □ Otros (especificar):	If you answered NEVER. please indicate why you do not use it: (select all applicable responses) □ I did not know teledermatology was available in our area. □ I find the teledermatology application difficult to understand and use. □ I do not have enough time during consultations to use it. □ I have no way to obtain images. □ I think patients should be evaluated in person. □ Other (specify):
En caso POSITIVO. ¿con qué frecuencia la utilizas?: □ Varias veces por semana □ Varias veces al mes □ Varias veces al año □ Esporádicamente	If you answered YES. how often do you use it? □ Several times a week □ Several times a month □ Several times a year □ Sporadically
1. ¿Cómo calificarías el funcionamiento de la aplicación de laTeledermatología de tu hospital? □ Muy bueno □ Bueno □ Regular □Malo □ Lo desconozco	1. How would you rate the performance of the teledermatology application in your hospital? □ Very good □ Good □ Fair □ Poor □ Don’t know
2. ¿Cómo calificarías la atención médica recibida por el paciente víaTeledermatología? □ Muy buena □ Buena □ Regular □ Mala	2. How would you rate the health care received by patients via teledermatology? □ Very good □ Good □ Fair □ Poor
3. ¿Cómo evaluarías la atención médica recibida por el paciente víaTeledermatología en comparación con la recibida en la consulta presencial dedermatología? □ Mejor □ Igual □ Peor	3. How would you rate the health care received by patients via teledermatology compared with health care received during an in-person dermatology consultation? □ Better □ Same □ Worse
4. ¿Cómo evaluarías la calidad general de la Teledermatología en su área desalud (funcionamiento. atención médica proporcionada capacidad y rapidez enla resolución)? □ Muy buena □ Buena □ Regular □ Mala□ Lo desconozco	4. How would you rate the overall quality of teledermatology in your healthcare area (performance. health care provided capacity and speed of resolution)? □ Very good □ Good □ Fair □ Poor □ Don’t know
5. ¿Consideras la Teledermatología una herramienta útil? □ Sí. es muy útil □ Sí. es bastante útil □ No. es poco útil □ No. no es nada útil	5. Do you consider teledermatology a useful tool? □ Yes. very useful □ Yes. quite useful □ No. it is not very useful □ No. it is not at all useful
6. ¿Crees que la Teledermatología puede mejorar la patología de tus pacientes? □ Sí. puede mejorarla en gran medida □ Sí. puede mejorarla bastante □ No. no la mejora □ No. la empeora	6. Do you think teledermatology can improve your patients’ condition? □ Yes. it can greatly improve their condition □ Yes. it can improve their condition somewhat □ No. it does not improve their condition □ No. it makes their condition worse
7. ¿Crees que la Teledermatología agiliza la valoración dermatológica de tuspacientes? □ Sí. la agiliza en gran medida □ Sí. la agiliza bastante □ Sí. pero solo un poco □ No. no la agiliza	7. Do you believe teledermatology speeds up the dermatological assessment of your patients? □ Yes. greatly □ Yes. somewhat □ Yes. but only a little □ No
8. ¿Crees que el diagnóstico y plan emitido por el dermatólogo víaTeledermatología es fiable? □ Sí. es muy fiable □ Sí. es bastante fiable □ No. no es fiable □ Depende del dermatólogo	8. Do you think the diagnoses and plans issued by the dermatologist via teledermatology are reliable? □ Yes. very reliable □ Yes. quite reliable □ No. they are not reliable □ It depends on the dermatologist
9. ¿Consideras que con la Teledermatología puede mejorar tus conocimientosen Dermatología? □ Sí. es una herramienta muy docente □ Sí. en cierta medida □ No. no es una herramienta docente	9. Do you consider that teledermatology can help you improve your knowledge of dermatology? □ Yes. it is a very educational tool □ Yes. to some extent □ No. it is not an educational tool
10. ¿Crees que la Teledermatología mejora la relación entre pediatras ydermatólogos? □ Sí. la mejora en gran medida □ Sí. la mejora bastante □ No. no produce modificaciones □ No. la empeora	10. Do you think teledermatology improves the relationship between pediatricians and dermatologists? □ Yes. greatly □ Yes. quite a lot □ No. it does not change the relationship □ No. it worsens the relationship
11. ¿Crees que el tiempo adicional destinado a efectuar una consulta víaTeledermatología le compensa? □ Sí. compensa mucho □ Sí. compensa bastante □ Sí. pero en escasa medida □ No. no compensa	11. Do you think the extra time spent carrying out a consultation via teledermatology is worth it? □ Yes. very worth it □ Yes. quite worth it □ Yes. but to a small extent □ No. it’s not worth it
12. ¿Crees que fotografiar a los pacientes con su teléfono móvil personal lesupone una dificultad añadida al uso de la Teledermatología? □ No. no lo dificulta □ Sí. lo dificulta un poco □ Sí. lo dificulta bastante □ Sí. lo dificulta en gran medida	12. Do you think taking photographs of patients with your personal mobile phone could make it more difficult to use teledermatology? □ No □ Yes. a little more □ Yes. quite a lot more □ Yes. much more
13. ¿Crees que el uso del teléfono móvil personal para fotografiar a tuspacientes vulnera la privacidad de los mismos? □ No. no la vulnera □ Sí. la vulnera un poco □ Sí. la vulnera bastante □ Sí. la vulnera en gran medida	13. Do you think using a personal mobile phone to take photographs of your patients violates their privacy? □ No □ Yes. a little □ Yes. quite a lot □ Yes. greatly
14. ¿Crees que disponer de una cámara fotográfica destinada exclusivamentea la Teledermatología facilitaría el uso de dicha herramienta? □ Sí. lo facilitaría en gran medida □ Sí. lo facilitaría bastante □ No. no lo facilitaría □ No. lo dificultaría	14. Do you think having a camera for exclusive use in teledermatology would facilitate the use of said tool? □ Yes. it would make it much easier □ Yes. it would make it quite a lot easier □ No. it wouldn’t make it easier □ No. it would make it harder
15. ¿Estás satisfecho en general con la Teledermatología? □ Sí. muy satisfecho □ Sí. bastante satisfecho □ No. no estoy satisfecho □ No soy usuario de la Teledermatología	15. Are you generally satisfied with teledermatology? □ Yes. very satisfied □ Yes. quite satisfied □ No. I am not satisfied □ I do not use teledermatology
16. ¿Seguirías o empezarías a utilizar la Teledermatología en el futuro? □ Sí. por supuesto □ Sí. pero introduciendo mejoras □ No. no me convence	16. Will you continue using teledermatology in the future? □ Yes. of course □ Yes. but with improvements No. I’m not convinced
A continuación. puedes indicar qué aspectos mejorarías de laTeledermatología:	Please indicate any aspects of teledermatology you would improve:

The English version is a translation of the original Spanish questionnaire. Please consider that the translation of the items into english does not replace a transcultural adaptation of the instrument.

Discussion

When we consulted the literature, we were unable to find a suitable Spanish-language questionnaire for evaluating pediatricians’ satisfaction with teledermatology. For this reason, we decided to create our own questionnaire through a process of consensus and with reference to previously published questionnaires.^8,10,13,14 After a group of experts had evaluated our provisional version, we made pertinent changes to the first version of the questionnaire. Then, we submitted the revised version for statistical validation; once checked, every item had an I-CVI greater than 0.78 for clarity, coherence, relevance, and sufficiency.

The two main aspects of the validation of a questionnaire are reliability and validity.¹⁵ Reliability was assessed by the internal consistency, with a Cronbach’s alpha coefficient above 0.70 for the whole questionnaire (0.890), as well as for most of its dimensions, and by the reproducibility, as the target population’s responses were stable across two time points separated by 1 month, reaching an intraclass correlation index of 0.875. Therefore, we can consider our questionnaire reliable with results comparable to the study published by Vidal i Abadall in 2018, who obtained a Cronbach’s alpha coefficient of 0.84 and an intraclass correlation index of 0.93.¹⁰

To assess validity, we studied construct validity. The principal component analysis revealed two questions with a lower communality index (questions 3 and 9 of the revised questionnaire), probably because they had fewer response options than the other questions (three instead of four). Furthermore, the Varimax method and the rotated component matrix detected four components to match our four domains. However, the items in each component were different from our initial grouping, which means the construct validity was suboptimal. In addition, we studied content validity with S-CVI, reaching values for clarity, coherence, relevance, and sufficiency greater than 0.9. That is why our questionnaire is considered valid, with excellent content validity, though construct validity was poor. This discordance could be explained because of the low answer rate despite the correct diffusion of the questionnaire.

Although the question “Do you think having a camera for exclusive use in teledermatology would facilitate the use of said tool?” had a factor of less than 0.5 in the rotated component matrix, and although eliminating this question increased the Cronbach’s alpha coefficient of the dimension and of the whole questionnaire, we decided to keep it in the validated version, as the question had an adequate I-CVI and the responses were very useful for guiding the intervention. For this reason, and in view of the questionnaire’s validity and reliability, we made no changes to the revised version. Such a situation was approached similarly by Zhou et al.¹⁶

The final validated questionnaire has an initial section for collecting demographic information and for determining professionals’ experience with teledermatology to filter out those with insufficient knowledge to complete the questionnaire (Table 8). After this initial section, there are a further 16 questions, adapted and reformulated from the 15 questions in the provisional questionnaire: we added one question to the “Usefulness” domain, and we regrouped questions from the “Limitations” domain (Table 3).

Our study has the following limitations: 1)

The virtual dissemination of the questionnaire and the lack of incentives may have reduced the response rate.

Although there were sufficient respondents to validate the questionnaire, as some studies do,¹⁶ other studies have achieved larger samples.¹⁰

The validated questionnaire focuses on pediatric asynchronous teledermatology, so it is not applicable to other professionals who use teledermatology or to professionals who use other modalities of telemedicine, including synchronous teledermatology.

Our questionnaire evaluates the use of personal mobile phones to take pictures in the teledermatology process, so organizations using other devices may remove these questions referring to the use of mobile phones in teledermatology.

In conclusion, our results show that our questionnaire is a robust tool for evaluating pediatricians’ satisfaction with teledermatology.

Footnotes

Disclosure Statement

The authors declare that there is no conflict of interest.

Funding Information

The author(s) received no financial support for the research, authorship, or publication of this article.

References

Carnicero

, Fernández

, (CEPAL) . Manual De Salud Electrónica Para Directivos De Servicios Y Sistemas De Salud. CEPAL: Chile; 2012.

Prindaville

, Antaya

, Siegfried

. Pediatric dermatology: Past, present, and future. Pediatr Dermatol, 2015; 32(1):1–12; doi: 10.1111/pde.12675

Stephens

, Murthy

, McMahon

. Wait times, health care touchpoints, and nonattendance in an academic pediatric dermatology clinic. Pediatr Dermatol, 2019; 36(6):893–897; doi: 10.1111/pde.13943

Gehris

, Herman

EIX

. Pediatric teledermatology: A review. Curr Derm Rep, 2020; 9(2):114–122.

Duan

, Lambert

, Hight

, et al. Comparison of pediatric dermatology conditions across telehealth and in-person visits during the COVID-19 pandemic. J Drugs Dermatol, 2022; 21(11):1260–1263; doi: 10.36849/JDD.6843

Kourosh

, Schneider

, Hawryluk

, et al. Combining teledermatology with nonphysician members of the health care team to address access and compliance barriers in pediatric atopic dermatitis: A needs assessment. J Am Acad Dermatol, 2020; 83(1):237–239; doi: 10.1016/j.jaad.2019.12.012

Fogel

, Teng

. Pediatric teledermatology: A survey of usage, perspectives, and practice. Pediatr Dermatol, 2015; 32(3):363–368; doi: 10.1111/pde.12533

Kohn

, Pickett

, Day

, et al. When is synchronous telehealth acceptable for pediatric dermatology? Pediatr Dermatol, 2022; 39(2):236–242; doi: 10.1111/pde.14919

Gatica

, Aragón-Caqueo

, Aedo

, et al. Teledermatology evaluation and feedback systems: A tool for improving care. Healthcare (Basel), 2023; 11(11):1626; doi: 10.3390/healthcare11111626

10.

Vidal I , Alaball

. Evaluating the impact of asynchronous telemedicine in the Catalan central region. PhD Thesis, Universitat de Vic: Spain; 2018. Available from: http://dspace.uvic.cat/bitstream/handle/10854/5626/tesdoc_a2018_vidal_alaball_josep_evaluating_impact.pdf?sequence=1&isAllowed=y

11.

Roco Videla

, Hernández Orellana

, Silva González

. Cuál es el tamaño muestral adecuado para validar un cuestionario? Nutr Hosp, 2021; 38(4):877–878; doi: 10.20960/nh.03633

12.

Polit

, Beck

. The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health, 2006; 29(5):489–497; doi: 10.1002/nur.20147

13.

Fluhr

, Gueguen

, Legoupil

, et al. Teledermatology in times of COVID-19 confinement: Comparing patients’ and physicians’ satisfaction by the standardized brest teledermatology questionnaire. Dermatology, 2021; 237(2):1–6; doi: 10.1159/000514029

14.

Pahalyants

, Murphy

, Gunasekera

, et al. Evaluation of electronic consults for outpatient pediatric patients with dermatologic complaints. Pediatr Dermatol, 2021; 38(5):1210–1218; doi: 10.1111/pde.14719

15.

Boparai

, Singh

, Kathuria

. How to design and validate a questionnaire: A guide. Curr Clin Pharmacol, 2018; 13(4):210–215; doi: 10.2174/1574884713666180807151328

16.

Zhou

, Bao

, Setiawan

IMA

, et al. The mHealth App Usability Questionnaire (MAUQ): Development and validation study. JMIR Mhealth Uhealth, 2019; 7(4):e11500; doi: 10.2196/11500