Teledentistry: A Tool to Promote Continuing Education Actions on Oral Medicine for Primary Healthcare Professionals

Abstract

Background: Difficulties in diagnosis of oral mucosal lesions are a significant cause of delayed oral cancer diagnosis, and this difficulty may be due to gaps in knowledge. This study evaluated the diagnostic skills of primary healthcare professionals regarding oral cancer and presented them with an e-learning course. Materials and Methods: Forty-seven primary healthcare professionals (32 dentists and 15 nondentists) enrolled in a 24-h course on oral medicine delivered through an e-learning platform. A test, based on 33 clinical images of oral lesions, was used to evaluate the diagnostic skills of participants. The participants were requested to classify each lesion as benign, potentially malignant, or malignant as well as to inform their clinical impression. Three specialists also took the test as the gold standard. Results: Twenty-seven participants completed the test. Nondentists and dentists showed a comparable sensitivity of 68.8 ± 11.1 and 63.7 ± 15.8, respectively. Specialists performed somewhat better; however, the difference was not statistically significant (81.0% ± 4.1%, p = 0.16). Dentists and specialists (70.0% ± 16.6% and 95.5% ± 3.1%, respectively) showed higher specificity than nondentists (39.3 ± 20.6, p < 0.01). Nondentists had a higher number of unanswered questions (p < 0.01) for classification and clinical impression (50.0% ±45.1% and 72.0% ± 25.0%, respectively) than dentists (5.7% ±11.9% and 19.8% ± 20%, respectively). Both dentists and nondentists had low attendance in the course (44.57% ± 37.38% and 26.53% ± 26.53%, respectively, p = 0.26). To the best of our knowledge, this is the first study to assess the diagnostic skills of public health workers belonging to different professional categories. Conclusion: Both dentists and nondentists have a fairly good capacity for discriminating the nature of oral lesions. Early squamous cell carcinoma is the most challenging situation and remains an issue to be addressed.

Introduction

Oral cancer is an important, ongoing public health problem, considering its morbidity and mortality.¹ In Brazil, oral cancer represents the fifth most common type of malignant tumor. The estimated number of new cases in 2016 is 11,140 cases in men and 4,350 in women, accounting for 5.2% and 2.1%, respectively, of all types of cancer; this excludes nonmelanoma skin tumors.² In the past few decades, the mortality rate associated with oral cancer has remained stable.³ These data may be attributed, at least in part, to a high percentage of cases being detected in advanced stages.^4,5 Brazil has one of the lowest survival rates for oral cancer,⁶ particularly in the south region.⁷

To face the challenges described above, Brazil established a universal public healthcare system more than 25 years ago: the Unified Health System (SUS), which focuses on the principle of primary healthcare.⁸ At present, SUS has 24,468 oral health teams responsible for covering ∼77 million people (38% of the population). Despite many improvements such as better access to dental services, SUS faces systemic problems, including the fragmentation of its service network.⁹

Considering that the oral cavity is easily accessible on examination, earlier detection of oral cancer or potentially malignant disorders is certainly feasible.¹⁰ However, dentists usually face difficulties in detecting, diagnosing, and managing oral lesions.¹¹ This difficulty has been attributed to few hours of lectures and lack of training in oral medicine during undergraduate education.^12,13 As expected, the majority of these professionals desire more training on oral cancer detection and prevention.¹⁰

This fragmented service network opens up an opportunity for telemedicine/telehealth initiatives. As defined by the Institute of Medicine, telemedicine is “… the use of electronic information and communications technologies to provide and support health care when distance separates the participants.”¹⁴ Teledentistry refers to the inclusion of the fields of oral healthcare and education to this definition, supported by telecommunication technology.¹⁵

Taking this idea into consideration, the Brazilian Ministry of Health created nine telehealth working groups in 2007. These groups work in collaboration with public universities such as Universidade Federal do Rio Grande do Sul (UFRGS). In 2010, this pilot project was expanded to become a full-fledged government program called Telessaúde Brasil Redes.

At present, TelessaúdeRS/UFRGS (www.ufrgs.br/telessauders) is a large research program aimed at designing and evaluating systemic solutions to overcome the challenge of fragmentation in SUS, with a special focus on improving healthcare practice. To ensure the achievement of its goals, Telessaúde/UFRGS (TSRS) has developed professional consultation (store-and-forward and real-time consultation), telediagnosis, telemedicine and teledentistry solutions and technologies, support for implementation of IT solutions, and tele-education activities.

Few studies have assessed the diagnostic skills of dentists for oral cancer and precancer identification. Based on a set of pictures obtained from clinical cases, Seoane et al.¹⁶ showed that dentists working in Spanish primary healthcare services had 61.4% sensibility and 86.5% specificity for oral cancer and potentially malignant disorders. Among the different variables, “years of professional experience” appears to explain the different diagnostic skills for oral lesions found among dentists.¹⁷ Therefore, there is a clear need for greater emphasis on oral cancer education for healthcare professionals in general.^18,19

The World Health Organization (WHO) has recommended that, in addition to dentists, other healthcare professionals should be involved in oral cancer opportunistic screening.^10,18 Therefore, the aim of this study was to assess the diagnostic skills of different primary healthcare professionals regarding oral mucosal lesions, comparing the performance of dentists with that of nondentists. The secondary aim of this study was to describe a continuing education course on oral medicine to be offered to primary healthcare workers as a continuing education strategy.

Materials and Methods

This analytical cross-sectional study with an educational intervention was previously approved by the local research ethics committee (protocol No. 07/402) and was conducted from August 2014 to December 2015. A Web-based self-instruction course on the diagnosis of oral mucosal lesions was offered to healthcare professionals in public health services enrolled in TSRS. In total, 47 participants (32 dentists, 12 nurses, and 3 dietitians) enrolled in the course. The course syllabus included different aspects of the diagnostic process, mainly focusing on the clinical characteristics of oral lesions and covering ∼24 h of educational material (Table 1). For each topic, a case discussion-based activity was provided, in which participants were required to apply the theoretical knowledge they had acquired, such as lesion description or differential diagnosis.

Table 1.

Course Syllabus and Duration of Time Allotted to Each Topic

	DURATION (MIN)
SUBJECT	LECTURE	CASE DISCUSSION
Clinical examination	57	60
Anamnesis	10
Extraoral examination	11
Intraoral examination	13
Elementary lesions	14
Anatomic variations of the oral cavity	9
Complementary examinations	151	60
Oral white lesions	40	60
Oral ulcers	53	60
Proliferative lesions	58	60
Bullous lesions	29	60
Tongue disorders	43	60
Potentially malignant disorders	15	60
Pigmented lesions	42	60
Intraosseous lesions of the jaws	154	60
Oral cancer	58	60
Adverse effects of the use of bisphosphonates	60	60
Total	760	720

The lectures were available in Flash files prepared from Microsoft PowerPoint software, with an accompanying audio recording. The lectures were accessible by Moodle, an e-learning platform hosted on the Web portal of UFRGS (www.ufrgs.br/ead). The main aim of the course was to promote continuing education activities and improve awareness of different professional categories about oral cancer and potentially malignant disorders.

To evaluate their diagnostic skills, the participants were invited to complete a test. Twenty-seven participants took the voluntarily test. Characteristics of the sample population are presented in Table 2. The test package used in this study was based on the protocol proposed by Seoane et al.¹⁶ and was composed of 33 clinical images obtained from the digital manual for the early diagnosis of oral neoplasia by the International Agency of Research in Cancer (IARC).²⁰ Benign lesions included candidiasis (n = 3, 9.1%), recurrent aphthous ulcer (n = 2, 6.1%), pyogenic granuloma (n = 2, 6.1%), papilloma (n = 1, 3.0%), hemangioma (n = 2, 6.1%), neurofibroma (n = 1, 3.0%), mucocele (n = 1, 3.0%), fissured tongue (n = 1, 3.0%), geographic tongue (n = 1, 3.0%), median rhomboid glossitis (n = 1, 3.0%), traumatic ulceration (n = 1, 3.0%), gingival hyperplasia (n = 1, 3.0%), and pemphigus vulgaris (n = 2, 6.1%). Potentially malignant disorders were represented by leukoplakia (n = 4, 12.1%), lichen planus (n = 3, 9.1%), and erythroplakia (n = 1, 3.0%), and malignant lesions, including early squamous cell carcinoma (SCC) (n = 3, 9.1%) and advanced SCC (n = 3, 9.1%). All the photographs were coded and randomized according to a random numbering technique. No information about clinical history, patient age and sex, lesion location, signs and symptoms, the course of the lesions, or exploratory findings was available. Questions in black were considered incorrect to avoid favoring responses that do not lead to decision-making. Two questions were asked for each image:

Table 2.

Demographics of Participants Completing the Test Before Taking the Course

CHARACTERISTICS	NONDENTISTS (n = 8)	DENTISTS (n = 19)	SPECIALISTS (n = 3)	p ^*
Age
Mean (SD)	32.71 (6.70)	33.78 (8.39)	42.00 (0.00)	0.20^*
Gender
Male	1	5	1
Female	7	14	2
Professional experience (years)
Mean (SD)	4.36 (2.88)^a	10.18 (8.48)^b	20.00 (0.00)^c	0.01^*
Attendance in the course (%)
Mean (SD)	26.53 (26.53)	44.57 (37.38)	—	0.26^**

Values followed by different lowercase letters are statistically different from each other. Missing information: two cases for age (dentists, n = 1 and nondentists, n = 1) and one case for years of professional experience (nondentists, n = 1).

ANOVA/Tukey; ^**Student's t test.

ANOVA, analysis of variance; SD, standard deviation.

Question 1. In your opinion, what is the diagnostic category for this lesion?

(a) A benign lesion, including inflammatory lesions, infectious diseases, or benign tumors.

(b) A potentially malignant disorder, also known as oral precancer or oral premalignant lesion, that is, risk of oral cancer development.

Question 2. What is your clinical impression (i.e., presumptive diagnosis)?

The responses were entered into a database. The percentages of correct classification, sensitivity, and specificity were calculated for each participant. Sensitivity and specificity calculations were based on this analysis, considering potentially malignant disorder and malignant as positive cases, because both diagnostic categories contain high-priority cases.¹⁶

The calculation of nonresponse rate was based on the percentage of unanswered questions in Question 1. Three oral medicine/oral pathology specialists also took the test and were considered the gold standard. The course was not accessible to the participants at any time.

Statistical Analysis

Descriptive statistics (mean ± standard deviation) were used to summarize data from the test in the different groups. Data distribution was assessed by the Shapiro–Wilk's test. Analysis of variance (ANOVA)/Tukey or Kruskal–Wallis/Dunn tests was used to compare the performance of different professional categories based on this analysis.

Results

Response Rate

The tests were returned by 27 of 50 enrolled participants (19 dentists and 8 nondentists), representing a response rate of 54%.

Characteristics of the Sample

Demographics of the participants were assessed and are summarized in Table 2. There were remarkable differences in the years of experience among different professional categories. A large number of enrolled participants failed to attend classes regularly. The mean attendance of the 27 participants who complete the test was 44.19%, ranging from 2% to 100%.

Comparison Between the Diagnosis Skills of Dentists and Nondentists

Table 3 shows a comparison between the different professional categories on the basis of different criteria. With regard to classification, specialists displayed a higher percentage of correct responses than dentists and nondentists. Nondentists exhibited 50% and 72% of nonresponse for classification and clinical impression, respectively. Dentists and nondentists showed relatively good and comparable sensitivity (p = 0.16, ANOVA). With regard to specificity, dentists and specialists performed better than nondentists (p < 0.01, ANOVA/Tukey).

Table 3.

Percentage of Correct Responses Given by Participants for Classification and Clinical Impression

VARIABLES	NONDENTISTS (n = 8)	DENTISTS (n = 19)	SPECIALISTS (n = 3)	p
Classification
Correct response	40.9 (12.2)^a	64.8 (10.2)^b	87.9 (3.0)^c	<0.01^*
Nonresponse rate	50.0 (45.1)^a	5.7 (11.9)^b	0.0 (0.0)^b	<0.01^**
Clinical impression
Correct response	4.5 (11.7)^a	45.2 (9.4)^b	70.7 (6.3)^b	<0.01^**
Nonresponse rate	72.0 (25.0)^a	19.8 (20.0)^b	1.0 (1.7)^b	<0.01^**
Sensitivity	68.8 (11.1)	63.7 (15.8)	81.0 (4.1)	0.16
Specificity	39.3 (20.6)^a	70.0 (16.6)^b	95.5 (3.1)^b	<0.01^*

Data are expressed as mean% ± standard deviation. Values followed by different lowercase letters are statistically different from each other.

ANOVA/Tukey; ^**Kruskal–Wallis/Dunn.

Cases with the lowest percentage of correct classification by dentists included malignant lesion/early SCC (5.26%), premalignant lesion/leukoplakia (15.79%), benign lesion/traumatic ulcer (15.79%), malignant lesion/early SCC (26.32%), benign lesion/candidiasis (26.32%), premalignant lesion/lichen planus (31.58%), malignant lesion/early SCC (31.58%), and premalignant lesion/lichen planus (31.58%). The most challenging cases for nondentists (nurses and nutritionists) were premalignant lesion/leukoplakia (no correct response), benign lesion/recurrent aphthous stomatitis (no correct response), benign lesion/papilloma (no correct response), malignant lesion/early SCC (no correct response), malignant lesion/advanced SCC (no correct response), benign lesion/hemangioma (no correct response), premalignant lesion/lichen planus (12.50%), and malignant lesion/advanced SCC (12.50%).

Some cases presented a low level of correct responses even for specialists. These cases could indicate different differential diagnoses, which become challenging, as shown in Figure 1.

Fig. 1.

Pictures of cases that specialists considered challenging because it was not possible to perform full examination: (A) pseudomembranous candidiasis (correct response) and lichen planus, (B) lichen planus (correct response) and geographic tongue, (C) leukoplakia (correct response) and geographic tongue, (D) early SCC (correct response) and reactional lesion (inflammatory hyperplasia, or pyogenic granuloma), (E) advanced SCC (correct response) and primary syphilis, and (F) leukoplakia (correct response) and frictional keratosis. SCC, squamous cell carcinoma. Source: Digital manual for the early diagnosis of oral neoplasia. International Agency of Research in Cancer—IARC²⁰

Discussion

The majority of previous studies have focused on the knowledge of and attitudes held by dental professionals in relation to oral cancer.^21,22 While providing important information, these studies did not evaluate how accurate professionals were in screening and detecting oral cancer or potentially malignant disorders. Obviously, dental professionals need these skills to support their decisions to refer patients as well as to determine the level of priority of each case. The purpose of our preliminary study was to assess the diagnostic skills of dentists and nondentists working in primary healthcare services. This initiative is in accordance with the WHO strategy against oral cancer, which recommends the training of primary healthcare professionals (dentists and nondentists) for the detection of early lesions.⁶

There are very few studies available on the diagnostic skills of dentists regarding oral cancer. The method of evaluation proposed in the present study is based on a method used by Seoane et al.,¹⁶ in which the ability of dentists to discriminate the nature of oral lesions is defined on the basis of pictures. Although each study uses a similar approach, direct comparison between the results of the two studies is difficult because the images used were not the same. Consequently, it is likely that the level of testing between the two studies differs significantly. Nevertheless, our results indicate that dentists and nondentists show a good and comparable sensitivity to that found in the study of Seoane et al.¹⁶ This observation supports the potential of nondentists to contribute to oral cancer detection, despite showing a need for improvement in their diagnostic abilities. In the assessment of specificity, dentists performed much better than nondentists, and this may be attributable to their specific background and training during undergraduate education as well as their professional experience.

According to Lopez-Jornet et al.,¹⁷ dentists who have more years of professional experience display better performance in terms of the detection of oral cancer. This is in agreement with our data, showing that specialists having more years of experience and training performed better than other professionals.

In a previous study, Jullien et al.²³ used a set of 80 pictures of lesions and normal oral mucosa to evaluate the diagnostic skills of healthcare professionals. Comparison of our results with the results of that study is difficult because of methodological differences. It should be noted that the approach of Jullien et al. was based on a set of pictures that included normal oral mucosa. Therefore, a higher proportion of positive cases could have helped participants to perform better.

Interestingly, all early SCC cases were associated with the highest percentage of incorrect classification by dentists. This finding is expected because these cases sometimes mimic other lesions and are difficult to detect for any professional.¹ Furthermore, cases presenting the greatest difficulty to participants mainly included white and ulcerative lesions. This observation indicates that the knowledge of differential diagnosis regarding those particular groups of lesions should be improved. Our data and those of previous studies discussed above indicate that the clinical presentation and diagnostic criteria for SCC at that stage should be strengthened in the future by continuing education courses.

In contrast, the lowest percentage of correct classifications by nondentists included nodular, white, and ulcerative lesions and a high proportion of benign lesions, demonstrating that lack of knowledge has no relationship with the characteristics of the lesion. Another important finding is the high frequency of unanswered questions, which indicates low confidence among that professional category with respect to discriminating oral lesions. Considering the fact that our proposal to involve more healthcare professionals in oral cancer screening, long-term courses on oral cancer screening should be systematically offered. Alternatively, oral medicine and/or oral pathology could also be included in the nursing curriculum to address this issue.

Interestingly, community nurses see that their position is favorable for improving opportunistic oral cancer screening; however, they acknowledge that this is not done regularly.¹⁰ Our results point to lack of training as the main reason underlying inconsistent screening among nurses.¹⁰ Offering a course to those professionals would be a valuable strategy. Unfortunately, this group of healthcare professionals showed poor course compliance and few participants returned a completed test, creating difficulty in the assessment of their diagnostic skills. This could be explained, at least in part, by constraints imposed by the test because these professionals had little background to inform a specific clinical impression.²⁴ In future studies, it would be better to focus only on the capacity to identify the characteristics of suspected lesions (malignant and premalignant) because they are high-priority cases for referral to specialists. In these cases, professionals would be able to refer patients, contributing to the early diagnosis of suspected lesions.

Dentists working in primary healthcare services also recognize that other healthcare activities such as vaccination campaigns could be used to increase opportunistic screening for oral lesions. However, this type of collaborative care is not common. Moreover, the perception of primary healthcare professionals is that multidisciplinary approaches should be adopted when trying to reach a greater part of the population.¹⁷

The present study has some limitations. The first consideration is the sample size, which did not have a large number of participants representing each professional category. In addition, more conspicuous differences may have been detectable if more specialists had participated. Another important issue is the low course attendance. One possible reason for low attendance among nondentists was the unpleasant experience of being tested because asking about their clinical impression (Question 2) can expose gaps in their knowledge.²⁴ In addition, because the set of pictures did not include images of patients without lesions,²³ the accuracy of lesion detection was not properly assessed. A test based on pictures without any clinical supporting information such as the main complaint, duration of the lesion, medical history, use of medications, exposure to risk factors, and history of recent diseases does not allow thorough evaluation.²⁵ If proper conditions for oral examination were available, specialists would probably have performed better.

The final issue is the inherent limitation of a Web-based self-instruction course, for example, lack of face-to-face contact with an instructor.²⁶ This issue could explain, at least in part, the low frequency in the present course. In this sense, creating forums of discussion by chat and/or monitoring the access of participants may be helpful in increasing participation in the course.

It is known that there are knowledge gaps in the discrimination of oral lesions. These gaps could be improved by continuing education courses about oral cancer prevention and detection.^16,27 One-time activities of continuing education present limitations for improving knowledge regarding oral lesion detection. Moreover, these activities cannot cause substantial changes in daily practices in primary healthcare services.¹⁷ We attempted to offer a full course on oral medicine, exploring clinical aspects of the main oral lesions as well as each step of the diagnosis process. This approach allowed the inclusion of a wide range of topics, including oral examination and theoretical information about the different types of oral lesions. To achieve a higher attendance rate, an alternative approach may be to place more focus on case discussion than conventional lectures because this strategy seems to be more highly accepted by participants.²⁸ Another approach for increasing attendance would be to monitor participant attendance, thereby reinforcing the importance of consistency in the learning process.

Conclusion

In conclusion, both dentists and nondentists have a fairly good capacity for discriminating the nature of oral lesions. Early SCC is a very challenging situation and remains an important issue to be addressed. Distance learning courses may improve the knowledge required for better oral cancer detection, keeping in mind that low motivation and compliance are important obstacles that remain to be overcome.

Footnotes

Disclosure Statement

No competing financial interests exist.

References

van der Waal

. Are we able to reduce the mortality and morbidity of oral cancer; Some considerations. Med Oral Patol Oral Cir Bucal, 2013; 18:e33–e37.

National Institute of Health.. Estimates 2016 - Incidence of cancer in Brazil. Rio de Janeiro, 2016. Available at www.inca.gov.br/wcm/dncc/2015/por-tipos.asp (last accessed February 2, 2016 ).

Biazevic

, Castellanos

, Antunes

, Michel-Crosato

. Trends in oral cancer mortality in the city of Sao Paulo, Brazil, 1980–2002 [in Portuguese]. Cad Saude Publica, 2006; 22:2105–2114.

Gomez

, Seoane

, Varela-Centelles

, Diz

, Takkouche

. Is diagnostic delay related to advanced-stage oral cancer? A meta-analysis. Eur J Oral Sci, 2009; 117:541–546.

Vilela

, Allison

. An investigation of the correlates of sense of coherence in a sample of Brazilians with head and neck cancer. Oral Oncol, 2010; 46:360–365.

Petersen

. Oral cancer prevention and control—The approach of the World Health Organization. Oral Oncol, 2009; 45:454–460.

Borges

, Sena

, Ferreira

, Roncalli

. Mortality for oral cancer and socioeconomic status in Brazil [in Portuguese]. Cad Saude Publica, 2009; 25:321–327.

Ministry of Health.. Lei 8080. Unified Health System Brazil, 1990. Available at www.planalto.gov.br/ccivil_03/Leis/L8080.htm (last accessed February 2, 2016 ).

Mendes

. Health care networks [Portuguese]. Cien Saude Colet, 2010; 15:2297–2305.

10.

Macpherson

, McCann

, Gibson

, Binnie

, Stephen

. The role of primary healthcare professionals in oral cancer prevention and detection. Br Dent J, 2003; 195:277–281;discussion 63.

11.

Ergun

, Ozel

, Koray

, Kurklu

, Ak

, Tanyeri

. Dentists' knowledge and opinions about oral mucosal lesions. Int J Oral Maxillofac Surg, 2009; 38:1283–1288.

12.

Diamanti

, Duxbury

, Ariyaratnam

, Macfarlane

. Attitudes to biopsy procedures in general dental practice. Br Dent J, 2002; 192:588–592.

13.

Wan

, Savage

. Biopsy and diagnostic histopathology in dental practice in Brisbane: Usage patterns and perceptions of usefulness. Aust Dent J, 2010; 55:162–169.

14.

Schwamm

. Telehealth: Seven strategies to successfully implement disruptive technology and transform health care. Health Aff (Millwood), 2014; 33:200–206.

15.

Chen

, Hobdell

, Dunn

, Johnson

, Zhang

. Teledentistry and its use in dental education. J Am Dent Assoc, 2003; 134:342–346.

16.

Seoane

, Warnakulasuriya

, Varela-Centelles

, Esparza

, Dios

. Oral cancer: Experiences and diagnosis skills elicited by dentists in North-western Spain. Oral Dis, 2006; 12:487–492.

17.

Lopez-Jornet

, Camacho-Alonso

, Martinez-Beneyto

, Seoane-Leston

. Influence of years of professional experience in relation to the diagnostic skill of general dental practitioners (GDPs) in identifying oral cancer and precancerous lesions. Int Dent J, 2008; 58:127–133.

18.

Lombardo

, da Cunha

, Carrard

, Bavaresco

. Delayed referrals of oral cancer patients: the perception of dental surgeons [in Portuguese]. Cien Saude Colet, 2014; 19:1223–1232.

19.

Seoane-Leston

, Velo-Noya

, Warnakulasuriya

, Varela-Centelles

, Gonzalez-Mosquera

, Villa-Vigil

, Rodriguez-Lozano

, Diz-Dios

. Knowledge of oral cancer and preventive attitudes of Spanish dentists. Primary effects of a pilot educational intervention. Med Oral Patol Oral Cir Bucal, 2010; 15:e422–e426.

20.

International Agency of Research in Cancer—IARC.. Digital manual for the early diagnosis of oral neoplasia. Available at http://screening.iarc.fr/atlasoral.php (last accessed February 2, 2016 ).

21.

Applebaum

, Ruhlen

, Kronenberg

, Hayes

, Peters

. Oral cancer knowledge, attitudes and practices: a survey of dentists and primary care physicians in Massachusetts. J Am Dent Assoc, 2009; 140:461–467.

22.

Vijay Kumar

, Suresan

. Knowledge, attitude and screening practices of general dentists concerning oral cancer in Bangalore city. Indian J Cancer, 2012; 49:33–38.

23.

Jullien

, Downer

, Speight

, Zakrzewska

. Evaluation of health care workers' accuracy in recognising oral cancer and pre-cancer. Int Dent J, 1996; 46:334–339.

24.

Siriphant

, Drury

, Horowitz

, Harris

. Oral cancer knowledge and opinions among Maryland nurse practitioners. J Public Health Dent, 2001; 61:138–144.

25.

Eusterman

. History and physical examination, screening and diagnostic testing. Otolaryngol Clin North Am, 2011; 44:1–29.

26.

Kemp

, Grieve

. Face-to-face or face-to-screen? Undergraduates' opinions and test performance in classroom vs. online learning. Front Psychol, 2014; 12:1278.

27.

Lopez-Jornet

, Camacho-Alonso

, Molina-Minano

. Knowledge and attitudes about oral cancer among dentists in Spain. J Eval Clin Pract, 2010; 16:129–133.

28.

Freudenberg

, Bockisch

, Beyer

. An approach towards problem-based learning in virtual space. GMS Z Med Ausbild, 2010; 27:Doc 73.