Reliability of mobile phone teledentistry in dental diagnosis and treatment planning in mixed dentition

Abstract

Introduction

The aim of the study was to test the reliability of mobile phone teledentistry in the diagnosis and treatment planning of dental caries of children in mixed dentition.

Methods

A total of 57 cases, aged 6–12 years, among six examiners were used yielding 342 comparisons. The patients were examined by a dentist who was calibrated in the recording of oral findings in children using the World Health Organization (WHO) oral health assessment form for children (version 2013), which served as the gold standard for diagnosis. Six paediatric dentists calibrated using the same WHO form served as the teledentistry group and made their diagnoses using only the images sent to them without the use of radiographs. The pictures obtained were stored on an online cloud platform (Google Drive). The sharing links for these pictures were forwarded to participating dentists using a social media application (WhatsApp Messenger, Facebook Corp., Mountain View, CA).

Results

This study showed greater sensitivity than specificity, and though both sensitivity and specificity were constantly above 80%, it can be stated that the current model has a higher chance of false positive results than false negative results. The reliability of teledentistry in this study is greater in primary teeth than in permanent teeth.

Discussion

Although the use of teledentistry without radiographs is not as accurate as clinical examination, mobile phone teledentistry offers acceptable reliability for the initial diagnosis of caries in children.

Keywords

Telemedicine teledentistry dental caries

Introduction

Dental caries is a chronic disease of childhood that has reached epidemic proportions.¹ Despite significant progress in reducing oral diseases in developed countries over the past three decades, dental caries remains a significant public health problem, particularly among disadvantaged people and people living in remote areas.² Despite the growth of dentistry in Saudi Arabia, there are still very few areas with specialised paediatric dental care. Specialists tend to be concentrated in the major towns and cities.³ This increases the challenge of providing high-quality paediatric dental care to every part of the Kingdom. Paediatric dentistry is a complex field with the interaction of many different specialties. It is very difficult to locate such multi-disciplinary care in smaller towns, and almost impossible to obtain it in the remote villages of the Kingdom.³ As a result of this maldistribution in the dental workforce, many rural and remote communities are left underserved, leading to untreated oral disease.⁴ Workforce shortages, sparsely populated regions, funding challenges and the decreasing cost of, and advances in, technology have resulted in an increased interest in the adoption of telemedicine services.⁵ One of the potentially viable solutions to address geographical hurdles and the unavailability of dentists is mobile teledentistry.⁶ Teledentistry is a domain of telemedicine that emerges from the combination of information communication technology (ICT) and dentistry. Teledentistry is defined as the use of electronic information and telecommunications technologies to support long-distance clinical oral health care, patient and professional health related education, public health, and health administration.⁷ There are two telemedicine modalities: real-time consultation and store-and-forward. For most dental applications, the store-and-forward method provides excellent results without excessive costs for equipment or connectivity.⁸ Mobile teledentistry is a subset of telemedicine that incorporates cellular phone technology and store-and-forward telemedicine into oral care services. Despite dental photography becoming an integral part of daily dental practice, it has rarely been used as the means of diagnosis, consultation, or referral in routine practice. However, evidence on the use of smartphone cameras in epidemiological dental research is rare.⁴ Given the need for specialist consultation in the remote parts of Saudi Arabia, there is a need for a study to examine the reliability of mobile phone teledentistry. Since mixed dentition is the period of maximum dental change for a child, this study focusses on the use of mobile phone teledentistry in mixed dentition. The aim of the study is to test the reliability of mobile phone teledentistry in the diagnosis of dental caries of children in mixed dentition.

Methods

Ethical considerations

Data were collected after ethical approval from the Riyadh Elm University Institutional Review Board (IRB). Signed informed consent for the use of photographs was obtained from the parents of children whose pictures were used in the study. Confidentiality of the identity of the patient was maintained by issuing each patient file with a separate study number. The examiners were blinded to the true identity of the patient and were only made aware of the study number.

Sample size and power

A total of 57 cases among six examiners was used, yielding 342 comparisons. The post hoc sample power test showed that for an effect size of 0.02 (low effect size) a power of 0.97 was obtained. The effect size for both patients to be selected and the number of examiners to be calibrated was based on the existing literature on the reliability of teledentistry.^9,4,12

Inclusion and exclusion criteria

Patients aged between 6 and 12 years who presented with a chief complaint of dental caries with or without pain and whose parents consented to participate in this study were selected from the clinics of the Riyadh Elm University. Patients whose parents did not consent to participate in the study were excluded. Since one of the objectives of the study was to test the ability of dentists to diagnose dental disease in the presence of multiple lesions, children who were asymptomatic and presented only for dental check-up were also included in the study.

Selection and calibration of dentists

The patients were examined by a dentist who was calibrated in the recording of oral findings in children using the World Health Organization (WHO) oral health assessment form for children (version 2013). This served as the “gold standard” for diagnosis and treatment planning. The examining dentist was first calibrated using the self-calibration methods whereby he performed repeat examinations on 20 children aged between 6 years and 12 years. Six paediatric dentists (clinicians with at least 5 years’ experience) were calibrated using the same WHO form with the primary examiner as a benchmark. Initial calibration of the dentists to familiarise them with the WHO form was done in a dental clinic with adequate lighting. Each of these six dentists served as the teledentistry group and made their diagnoses using only the images sent to them without the use of radiographs.

Examination of the oral cavity

The examination of the oral cavity was conducted using WHO category I examination criteria (examination with professional light and radiographs). An average of seven oral photographs were taken, and the appropriate radiographic records were obtained using the American Academy of Paediatric Dentistry (AAPD) guidelines for radiographs in children. Seven pictures were taken on a mobile phone camera (iPhone 7, Apple Corp. Cupertino, CA) using autofocus and automated settings. The pictures included two extraoral photographs (anterior and lateral view) and five intra-oral photographs. The intraoral photographs comprised of an anterior closed mouth picture, upper occlusal, lower occlusal, right lateral and left lateral image (Figure 1). These were in keeping with the guidelines of AAPD recommendations for photographic records.

Figure 1.

Intraoral photographs included an anterior closed mouth picture, upper occlusal, lower occlusal, right lateral and left lateral image.

Store and forward

The pictures obtained (Figure 1) were stored in an online cloud platform (Google Drive). The sharing link of these pictures was forwarded to the participating dentists using a social media application (WhatsApp Messenger, Facebook Corp., Mountain View, CA) (Figure 2).

Figure 2.

Sharing link of pictures forwarded to participating dentists using WhatsApp.

Charting of oral findings

Both the examining dentist and the teledentistry group charted their findings on the paediatric version of the WHO oral health assessment form (2013; http://www.who.int/oral_health/publications/pep_annex2formchildrentooth.pdf).

Validation of the diagnosis

The accuracy of the diagnosis was measured by applying Cohen’s kappa. Inter-examiner reliability was measured using the intraclass correlation coefficient (ICC) and Cronbach’s alpha.

Results

Demographic data

The sample of photos comprised a total of 57 children aged between 6 and 9 years. The mean age of the sample studied was 7.79 years (SD ± 1.52). While the females (7.88 years, SD ± 1.72) were slightly older than the males (7.71 years, SD ± 1.39), there was no significant difference between them (Table 1).

Table 1.

Mean age of the sample of photographed children.

GSex	N	Mean	SD	T	Sig
Male	32	7.7188	1.39664	−0.388	0.699
Female	25	7.8800	1.73973	−0.388	0.699

Patients were classified according to their chief complaint as those reporting with pain, those who had dental caries and those with no chief complaint. Although care was taken to try to ensure an even distribution of pain, those with dental caries (n = 21) comprised the largest part of the sample (Table 2).

Table 2.

Distribution of the sample according to the chief complaint of the child.

	Chief complaint			Total
	Pain	Dental caries	No complaint/ check-up	Total
GSex
Male	13	13	6	32
Female	6	10	9	25
Total	19	23	15	57

The kappa score for the calibration of examiners in the clinic showed reliability between 0.842 and 0.921. The mean kappa score was 0.901, which indicated that, when clinical examination and radiographs were available, the examiners showed low intra-examiner variability and excellent reliability (>0.9).

A total of 1368 teeth were examined and recorded by the control examiner, with diagnosis confirmed by radiographic examination. The pictures of the patients were sent to the six examiners, and 14 of the 1368 were either not recorded or not coded by all of them. The final reliability analysis was performed on 1354 teeth (99.0%) (Table 3). When dental caries was examined in the primary teeth it was observed that the mean decayed missing primary tooth index (dft) was 3.21 (SD ± 1.2). The decayed component (d = 2.7, SD ± 1.9) was greater than the filled component (f = 1.51, SD ± 0.3). In the permanent dentition the overall decayed missing filled permanent tooth index (DMFT) was 1.8(SD ± 0.8) with the decayed component (D = 1.2, SD ± 0.9) greater than the filled component (F = 0.6, SD ± 0.3). There were no permanent teeth missing due to caries (M = 0) in the photographed children.

Table 3.

Total number of teeth examined and readings for each score.

	Frequency	Percentage	Valid percentage	Cumulative percentage
Valid
Sound permanent tooth	373	27.3	27.5	27.5
Sound primary tooth	393	28.7	29.0	56.6
Caries in primary tooth	417	30.5	30.8	87.4
Filled primary tooth with caries	24	1.8	1.8	89.1
Filled primary tooth without caries	6	0.4	0.4	89.6
Missing primary tooth due to caries	10	0.7	0.7	90.3
Fissure sealant in primary tooth	4	0.3	0.3	90.6
Space maintainer primary teeth	17	1.2	1.3	91.9
Caries in the permanent tooth	80	5.8	5.9	97.8
Filled permanent tooth with caries	3	0.2	0.2	98.0
Filled permanent tooth with no caries	5	0.4	0.4	98.4
Permanent tooth missing due to caries	3	0.2	0.2	98.6
Fissure sealant	3	0.2	0.2	98.8
Fixed prosthesis, crown or veneer	1	0.1	0.1	98.9
Unerupted	11	0.8	0.8	99.7
Not recorded	4	0.3	0.3	100.0
Total	1354	99.0	100.0
Missing
System	14	1.0
Total	1368	100.0

When the overall agreement between the examiners (using teledentistry) and the control was measured using the kappa score, it was observed that the kappa scores ranged between 0.795 and 0.882. Although the mean kappa score (0.812) was still higher than the WHO-recommended value of 0.8, the scores were lower than those seen when the examiners were calibrated chairside (Table 1). There was high specificity and acceptable sensitivity in the values observed (Table 4).

Table 4.

Overall agreement between the examiners and the control.

Parameter	Examiner 1	Examiner 2	Examiner 3	Examiner 4	Examiner 5	Examiner 6
Sensitivity	85.2%	74.1%	80.1%	78.1%	80.1%	77.2%
Specificity	89.1%	87.2%	88.4%	83.3%	83.8%	84.1%
Kappa	0.882	0.795	0.823	0.823	0.828	0.795

The intraclass correlation coefficient showed that, although overall agreement through average scores was very high (0.960), the specific agreement was relatively low (0.799), suggesting that the cause for variation could be specific measurements, rather than an overall lack of reliability (Table 5).

Table 5.

Overall inter-examiner variability.

	Intraclass correlation^b	95% Confidence interval		F test with true value 0
	Intraclass correlation^b	Lower bound	Upper bound	Value	df1	df2	Sig
Single measures	0.799^a	0.785	0.813	24.909	1345	6725	0.000
Average measures	0.960^c	0.956	0.963	24.909	1345	6725	0.000

Two-way mixed effects model where people effects are random and measures effects are fixed.

The estimator is the same, whether the interaction effect is present or not.

Type C intraclass correlation coefficients using a consistency definition. The between-measure variance is excluded from the denominator variance.

This estimate is computed assuming the interaction effect is absent because it is not otherwise estimable.

Of the 1354 teeth examined, there were 868 primary teeth and 486 permanent teeth. The kappa scores in the primary dentition were higher than those in the permanent dentition. Overall, the kappa scores in permanent dentition ranged from 0.605 to 0.732, and for primary dentition, they ranged from 0.739 to 0.747 (Table 6), which was lower than the overall reliability. To determine further the reason for the reduced reliability, the scores of the inter-examiner reliability for each procedure was evaluated using the ICC and Cronbach’s alpha (Table 7 and Table 8).

Table 6.

Overall agreement between the examiners and the control according to dentition.

Parameter	Examiner 1	Examiner 2	Examiner 3	Examiner 4	Examiner 5	Examiner 6
Primary dentition (n = 868)	0.747	0.739	0.753	0.753	0.747	0.739
Permanent dentition (n = 486)	0.732	0.605	0.701	0.701	0.732	0.605

Table 7.

Inter-examiner variability for primary teeth.

Control	Intraclass correlation^b	95% Confidence interval
Control	Intraclass correlation^b	Lower bound	Upper bound	Sig
Overall	0.901^a	0.892	0.910	0.000
	0.985^c	0.983	0.986	0.000
Sound primary tooth
Single measures	0.278^a	0.235	0.324	0.000
Average measures	0.697^c	0.649	0.742	0.000
Caries in primary tooth
Single measures	0.699^a	0.665	0.732	0.000
Average measures	0.933^c	0.922	0.942	0.000
Filled primary tooth with caries
Single measures	0.533^a	0.362	0.715	0.000
Average measures	0.872^c	0.773	0.938	0.000
Filled primary tooth without caries
Single measures	0.770^a	0.490	0.956	0.000
Average measures	0.953^c	0.852	0.992	0.000
Indicated for extraction
Single measures	0.569^a	0.261	0.881	0.000
Average measures	0.888^c	0.679	0.978	0.000

Table 8.

Reliability among examiners for values in the primary teeth.

Control	Cronbach's alpha	No. of Items
Sound primary tooth	0.697	6
Caries in primary tooth	0.933	6
Filled primary tooth with caries	0.872	6
Filled primary tooth without caries	0.953	6
Missing primary tooth due to caries	0.888	6

There was a low intraclass correlation coefficient score for sound permanent teeth and for teeth with secondary caries. Although there was significant correlation (Table 8), the Cronbach alpha values showed relatively low values for the sound teeth (0.607). For teeth with secondary caries, there was a relatively high Cronbach’s alpha (0.897) suggesting agreement among examiners, despite the relatively low ICC (0.533)

When the inter-examiner variability for the permanent teeth was examined, it was observed that the lowest intra-class correlation was observed for the permanent teeth with secondary caries (0.033), followed by fissure sealants (0.166) and sound permanent teeth (0.360) (Table 9). When this was contrasted with the inter-examiner reliability (Table 10), it was observed that although the scores for fissure sealants (0.600) and filled permanent tooth with caries (0.200) were unacceptable, the scores for sound permanent tooth (0.820) was acceptable (Table 10).

Table 9.

Inter-examiner variability for permanent teeth.

Control	Intraclass correlation^b	95% Confidence interval
Control	Intraclass correlation^b	Lower bound	Upper bound	Sig
Sound permanent tooth
Single measures	0.360^a	0.317	0.407	0.000
Average measures	0.798^c	0.765	0.828	0.000
Caries in the permanent tooth
Single measures	0.398^a	0.306	0.503	0.000
Average measures	0.823^c	0.756	0.876	0.000
Filled permanent tooth with caries
Single measures	0.033^a	−0.121	0.873	0.324
Average measures	0.194^c	−3.105	0.980	0.324
Filled permanent tooth with no caries
Single measures	0.621^a	0.278	0.938	0.000
Average measures	0.920^c	0.729	0.991	0.000
Fissure sealant
Single measures	0.167^a	−0.082	0.930	0.133
Average measures	0.583^c	−1.123	0.989	0.133
Unerupted
Single measures	0.318^a	0.083	0.713	0.001
Average measures	0.766^c	0.389	0.946	0.001
Not recorded
Single measures	0.500^a	0.075	0.978	0.006
Average measures	0.875^c	0.363	0.997	0.006

Table 10.

Reliability among examiners for values in the permanent teeth.

Reliability statistics
Control	Cronbach's alpha	No. of items
Sound permanent tooth	0.820	6
Caries in the permanent tooth	0.846	6
Filled permanent tooth with caries	0.200	6
Filled permanent tooth with no caries	0.946	6
Fissure sealant	0.600	6
Unerupted	0.788	6
Not recorded	0.900	6

Discussion

The need for studies on teledentistry

Teledentistry is a relatively new topic, and this has meant that there is a shortage of established protocols to evaluate the efficacy of teledentistry.^9,10 The adoption of any new technology requires field testing and reliability analysis.¹¹ While there have been few studies in the West testing the efficacy of established teledentistry programmes,^12–14 there are virtually no studies in the Middle East that have evaluated the reliability of teledentistry as a method to diagnose dental caries in children. The scope of teledentistry is vast, and there are studies that have looked at its uses in orthodontics, traumatic dental injuries and even oral diagnosis.^11–15 However, dental caries is by far the single largest oral health issue facing children in Saudi Arabia.¹⁶ It was for this reason that the current study focused on dental caries. Most studies on dental caries have used the DMFT or dft indices to evaluate the reliability of teledentistry.^11,16–18 While this has generally yielded high reliability and has been previously used in studies from the USA, the method has one limitation. The overall DMFT/dft, while excellent for epidemiological purposes, is of little use from the point of view of diagnosis or treatment planning. The WHO dentition status and treatment need index is a far more sensitive tool that records not only the presence or absence of decay but also factors such as fissure sealants, secondary caries and abutments, which are missed by the DMFT/dft index.^13,19–21 The rationale of using the dentition status treatment need index was to test the true sensitivity and specificity of teledentistry when used as diagnostic and treatment planning aid.

Teledentistry using mobile applications

This study included smartphones, which have made the capture and storage of digital images easier. This has been a boon to dentistry, as photography and dentistry go hand in hand for diagnosis, treatment planning and documentation.²² In the 2010s, there was a giant leap in mobile technologies, and Summerfelt stated that by utilizing current technologies, dental professionals can digitally acquire and transmit diagnostic data to a distant dentist for triage, diagnosis, and patient referral.²³ Based on these advances, Estai et al. conducted a study in which the photographs of participants' teeth were taken by a teledental assistant using a smartphone camera.⁶ These intraoral photographs were directly uploaded from an Android application to a cloud-based server.⁶ The use of the mobile phone, however, has not been restricted to images alone. Based on that, this study used the WhatsApp™ application to send the link of uploaded photos making it easier for them to be passed to the examiners. In addition, the WhatsApp™ Messenger application is the most widely used in Saudi Arabia, thus making it quicker to get a response from the examiners.

Store and forward teledentistry

The concept of telemedicine, and, by extension, teledentistry, is not new. However, the concept of the storage of information on the cloud has changed the way information can be transmitted.^13,20,21 Until the first decade of the 21st century, information transmitted via teledentistry was limited to the amount that could be uploaded and transmitted. This had often meant that information transmitted was restricted, and the download and storage was confined to computers.²¹ The widespread use of the cloud as a platform to store data has resulted in the ability to access large amounts of information from portable devices such as mobile phones and tablets. The current study used the store and forward technology to transmit data in sets of an average of seven photographs, which would otherwise be difficult to forward.^21,24 Although the current model did not include radiographs, the store and forward platform would allow the transmission of radiographic images if necessary.

Reliability of store and forward teledentistry

The overall reliability of the model used in this study was in keeping with studies from established teledentistry centres in England and the USA.^6,14,15,24 This seems to initially suggest that teledentistry is reliable for the diagnosis of dental caries. This study, however, aimed to go further than previous studies by using the dentition status and treatment need index, rather than the overall DMFT/dft; it was hoped that we could identify specific causes for lack of reliability. Overall, it was observed that the current model showed greater sensitivity than specificity and, though both sensitivity and specificity were constantly above 80%, it can be stated that the current model has a higher chance of false positive results than false negative results. This is in keeping with other studies that have assessed the sensitivity and specificity of teledentistry.¹³ The reliability of teledentistry in our study was greater in primary teeth than in permanent teeth. It must, however, be kept in mind that the current sample was comprised of far more primary teeth than permanent teeth. We also found that the greatest challenge to reliability was the detection of sound teeth, secondary caries and fissure sealants. While the absence of radiographs in the current teledentistry model could explain the difficulty in detecting secondary caries, the reason for the inability to detect sound teeth needs to be explored further. The fact that these problems were more serious in permanent teeth also merits further investigation. The images used in this study were from a hand-held mobile phone devices (iPhone 7, Apple Corp.) rather than the specialized intra-oral cameras used in previous studies.^6,9 The main reason for this was to explore the reliability of mobile phone based technology which has been shown to be effective in the diagnosis of dental trauma.¹² The results of our study show that while the lack of specialized camera equipment or lighting provided adequate diagnosis for dental caries lesions, the photographic quality was insufficient to adequately detect fissure sealants, which were typically placed on the posterior most teeth (the first permanent molar) of the child’s mouth.

Limitations of the study

The results of the study must be viewed keeping in mind certain limitations. The absence of radiographs in the study reduces the reliability of the diagnosis. However, this was done intentionally, as the purpose of the study was to test the validity of diagnoses made in the absence of modern dental facilities. Furthermore, the study only looks at a period of the mixed dentition. The reason for choosing the age group 6–9 years was to measure the validity of teledentistry in a period of rapid change. This, however, limits the accuracy of diagnoses in permanent teeth, especially the erupting first permanent molars. It should be remembered that the principal aim of this study was to assess the reliability of teledentistry in the detection and diagnosis of dental caries. To this end, the more specific and less sensitive WHO index was used, which looked at teeth rather than surfaces. The use of the more sensitive surface indices (DMFS/dfs) were beyond the scope of the current study and could form the basis for a future study.

Conclusion

Mobile phone teledentistry offers acceptable reliability for the initial diagnosis of caries in children. The use of teledentistry without radiographs is not as accurate as clinical examination. Further research into the field with larger sample sizes and field trials may shed more light on the topic.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

References

Garcia-Cortes

Mejia-Cruz

Medina-Cerda

, et al. Experience, prevalence, severity, treatment needs for dental caries and care index in Mexican adolescents and young adults. Rev Invest Clin 2014; 66: 505–511.

Estai

Kanagasingam

Tennant

, et al. A systematic review of the research evidence for the benefits of teledentistry. J Telemed Telecare 2017; 24(3): 147–156. 1357633x16689433. doi:10.1177/1357633x16689433.

AlShammery

. Dentistry in Saudi Arabia and the challenges of Vision 2030. Saudi J Oral Sci 2016; 3: 59–60. doi:10.4103/1658-6816.188080.

Estai

Kanagasingam

Huang

, et al. Comparison of a smartphone-based photographic method with face-to-face caries assessment: A mobile teledentistry model. Telemed J E Health 2017; 23: 435–440. doi:10.1089/tmj.2016.0122.

Bradley

Black

Noble

, et al. Application of teledentistry in oral medicine in a community dental service, N. Ireland. Br Dent J 2010; 209: 399–404. doi:10.1038/sj.bdj.2010.928.

Estai

Kanagasingam

Huang

, et al. The efficacy of remote screening for dental caries by mid-level dental providers using a mobile teledentistry model. Community Dent Oral Epidemiol 2016; 44: 435–441. doi:10.1111/cdoe.12232.

Marino

Ghanim

. Teledentistry: a systematic review of the literature. J Telemed Telecare 2013; 19: 179–183. doi:10.1177/1357633x13479704.

Daniel

Kumar

. Teledentistry: a key component in access to care. J Evid Based Dent Pract 2014; 14(Suppl): 201–208. doi:10.1016/j.jebdp.2014.02.008.

Kopycka-Kedzierawski

Bell

Billings

. Prevalence of dental caries in Early Head Start children as diagnosed using teledentistry. Pediatr Dent 2008; 30: 329–333.

10.

Martin

Nelson

Bhavsar

, et al. Feasibility assessment for using telehealth technology to improve access to dental care for rural and underserved populations. J Evid Based Dent Pract 2016; 16: 228–235. doi:10.1016/j.jebdp.2016.08.002.

11.

Purohit

Singh

Dwivedi

. Utilization of teledentistry as a tool to screen for dental caries among 12-year-old school children in a rural region of India. J Public Health Dent 2017; 77: 174–180. doi:10.1111/jphd.12195.

12.

de Almeida Geraldino

Rezende

da-Silva

, et al. Remote diagnosis of traumatic dental injuries using digital photographs captured via a mobile phone. Dent Traumatol 2017; 33: 350–357. doi:10.1016/j.sdentj.2017.03.002.

13.

Estai M, Bunt S, Kanagasingam Y, et al. Cost savings from a teledentistry model for school dental screening: an Australian health system perspective. Aust Health Rev. 2018; 42(5): 482–490. DOI: 10.1071/ah16119.

14.

Kravitz

Burris

Butler

, et al. Teledentistry, do-it-yourself orthodontics, and remote treatment monitoring. J Clin Orthod 2016; 50: 718–726. doi:10.1111/eje.12265.

15.

Haddad

Skelton-Macedo

Abdala

, et al. Formative second opinion: qualifying health professionals for the unified health system through the Brazilian Telehealth Program. Telemed J E Health 2015; 21: 138–142. doi:10.1089/tmj.2014.0001.

16.

AlKlayb

Assery

AlQahtani

, et al. Comparison of the effectiveness of a mobile phone-based education program in educating mothers as oral health providers in two regions of Saudi Arabia. J Int Soc Prev Community Dent 2017; 7: 110–115. doi:10.4103/jispcd.JISPCD_95_17.

17.

Chang

Plotkin

Mulligan

, et al. Teledentistry in rural California: a USC initiative. J Calif Dent Assoc 2003; 31: 601–608.

18.

Kopycka-Kedzierawski

Billings

McConnochie

. Dental screening of preschool children using teledentistry: a feasibility study. Pediatr Dent 2007; 29: 209–213.

19.

Mandall

O'Brien

Brady

, et al. Teledentistry for screening new patient orthodontic referrals. Part 1: A randomised controlled trial. Br Dent J 2005; 199: 659–662. discussion 653. doi:10.1038/sj.bdj.4812930.

20.

Estai

Bunt

Kanagasingam

, et al. Diagnostic accuracy of teledentistry in the detection of dental caries: a systematic review. J Evid Based Dent Pract 2016; 16: 161–172. doi:10.1016/j.jebdp.2016.08.003.

21.

Estai

Kanagasingam

Xiao

, et al. A proof-of-concept evaluation of a cloud-based store-and-forward telemedicine app for screening for oral diseases. J Telemed Telecare 2016; 22: 319–325. doi:10.1177/1357633x15604554.

22.

Desai

Bumb

. Digital dental photography: a contemporary revolution. Int J Clin Pediatr Dent 2013; 6: 193–196. doi:10.5005/jp-journals-10005-1217.

23.

Summerfelt

. Teledentistry-assisted, affiliated practice for dental hygienists: an innovative oral health workforce model. J Dent Educ 2011; 75: 733–742.

24.

Irving

Stewart

Spallek

Blinkhorn

. Using teledentistry in clinical practice, an enabler to improve access to oral health care: a qualitative systematic review. J Telemed Telecare 2017; 24(3): 129–146. 1357633x16686776. doi:10.1177/1357633x16686776.