Evaluation of digital slit-lamp videos for paediatric anterior segment telemedicine consultations

Abstract

Introduction

This study aimed to assess the validity of telemedicine consultations using digital slit-lamp videos to detect anterior segment pathology in a paediatric population.

Methods

A paediatric anterior segment specialist simultaneously performed and recorded anterior segment examinations using the Topcon digital-ready slit lamp. Components of the examination included the eyelids/eyelashes, conjunctiva/sclera, cornea, anterior chamber, iris and lens. Masked to clinical findings, a paediatric ophthalmologist reviewed and graded the live video feed transmitted at 4 Mbps. At least three months later, both ophthalmologists graded the stored videos. We compared the sensitivity, specificity, percent agreement and weighted kappa (κ) of diagnosing anterior segment pathologies via live-streamed and store-and-forward video clips compared to the in-person standard examination.

Results

Examinations of 89 eyes from 45 children (5–17 years old) with known anterior segment pathology were included. Agreement between live-streamed and in-person standard examinations for conjunctiva/sclera, anterior chamber, iris and lens findings was almost perfect (sensitivity 89–96%, specificity 95–100%, κ = 0.87–0.97). Substantial agreement was found for cornea pathology (sensitivity 88%, specificity 90%, κ = 0.72), and moderate agreement was found for eyelids/eyelashes pathology (sensitivity 54%, specificity 92%, κ = 0.46). Store-and-forward results were similar, though slightly better for eyelids/eyelashes and slightly worse for conjunctiva/sclera.

Discussion

Digital slit-lamp videos hold promise for synchronous and asynchronous telemedicine in diagnosing paediatric anterior segment pathologies.

Keywords

Digital slit lamp anterior segment paediatric telemedicine teleophthalmology,diagnostic accuracy

Introduction

Anterior segment pathologies such as cataracts, ulcers and keratitis can cause irreversible visual impairment that may affect a child’s quality of life. Unfortunately, geographic and socio-economic disparities can limit access to paediatric vision care services and affect the timely diagnosis.¹,² With the growing availability of commercial devices and broadband use, telemedicine provides an effective means for physicians to provide care beyond the constraints of the in-person clinic examination and to expand their reach to underserved areas.

Telemedicine for diagnosing anterior segment diseases has been primarily limited to store-and-forward models, in which videos or images are captured and sent to a remote specialist for review. This has been done through the use of smartphone cameras (e.g. iTouch 5G), smartphone camera attachments (e.g. Corneal Cell-Scope), portable diffuse light ophthalmic cameras (e.g. Nidek Versacam and PictorPlus) and portable slit-lamp cameras.^3–6 There have been limited studies on real-time anterior segment telemedicine examinations in adult patients using a standard slit-lamp bio-microscope and a video camera attachment that show diagnostic promise.^7–10 However, to our knowledge, there have been no such studies to date in a paediatric population. The ability to use the digital slit lamp accurately through synchronous telemedicine for paediatric patients has the potential to extend the reach of an already limited pool of paediatric ophthalmologists to geographically distant or underserved communities. Validating the equipment is an essential primary step in establishing a viable real-time telemedicine platform.

The purpose of this study was to assess the accuracy of detecting anterior segment pathology in a paediatric population via live-streamed and store-and-forward telemedicine using videos from a digital slit lamp.

Methods

This prospective study was conducted from September 2013 to August 2018 at the Vision Center at Children’s Hospital Los Angeles (CHLA). The project was approved by the Institutional Review Board at CHLA and adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from parents or legal guardians of all participants, and written assent was obtained for subjects aged seven and older.

Telemedicine system

The telemedicine system employed the Polycom® RealPresence® Group 500 videoconferencing system (Polycom, San Jose, CA). The first six subjects were examined on the Topcon SL-D2 with DC-3 camera attachment (Topcon, Tokyo, Japan), with videos formatted at 900p and 30 frames per second. For the subsequent 39 patients, the newer Topcon SL-D4 digital slit lamp with DC-4 camera attachment (Topcon) was used, with videos formatted at 964p and 20 frames per second. All videos were transmitted at 4 Mbps. The digital slit lamp connects to a local computer by Ethernet cord, via a USB connector, to stream and record videos in Topcon’s EZ Capture software simultaneously. To transmit videos securely in real time to a remote examiner, the desktop is shared through an encrypted Polycom-to-Polycom video call over the hospital’s internal network.

Study design

Children were eligible for enrolment if they were between 4 and 17 years of age, previously diagnosed with ocular anterior segment disease and able to participate in a slit-lamp examination. Children were excluded if they were developmentally delayed or wheelchair bound (and thus could not physically sit at the slit lamp) or if they could not cooperate for a slit-lamp examination. The authors felt that children younger than four years old were much less likely to sit still consistently long enough for a slit-lamp examination. Prior surgical intervention did not exclude patients from this study. Both eyes of the patients were included in the study when eligible. A designated research coordinator called to enrol patients who were scheduled to see the paediatric anterior segment specialist without the input of the ophthalmologists in the study and without prior knowledge of diagnoses.

The study examiners consisted of two ophthalmologists: physician 1 and physician 2. A paediatric anterior segment specialist (physician 1, B.J.R.) simultaneously performed and recorded anterior segment examinations using the Topcon digital slit lamp. These conventional in-person examinations were considered the pragmatic standard. Given that physician 1 is one of few exclusively paediatric anterior segment specialists in the USA, her examination was felt to represent a strong in-person standard examination. All videos were taken under ambient clinic room lighting. Subjects were not dilated, and the components of the examination included the eyelids/eyelashes, conjunctiva/sclera, cornea, anterior chamber, iris and lens. A remote paediatric ophthalmologist (physician 2, S.N.), who was masked to the subject’s identity and demographic information, viewed and graded the examinations by synchronous video feed (live streamed). Communication between the remote examiner and the in-person examiner was limited (no history or symptoms were relayed), but the remote examiner occasionally prompted the in-person physician to record features again as needed. Both examiners independently re-evaluated saved videos of all examinations a minimum of three months after the initial examination. The purpose of this was to measure intra- and inter-observer variability, as well as to compare telemedicine reliability by live-streamed (synchronous) versus store-and-forward (asynchronous) review. These stored videos were not edited in any way. Subjects were not excluded from the study if the videos were of poor image quality.

The agreement in detecting anterior segment disease was evaluated between in-person standard (physician 1) and live-streamed (physician 2) examinations, as well as between in-person standard (physician 1) and store-and-forward (physician 2) examinations. The intra-grader agreement between her in-person standard (physician 1) and her own store-and-forward examination (physician 1) was also calculated to assess whether store-and-forward telemedicine and in-person standard examinations were equivalent in diagnosing anterior segment pathologies. Similarly, intra-grader agreement for physician 2 (live streamed and store and forward) was calculated to assess the degree of agreement between examination modalities conducted over video. Inter-physician agreement was assessed by comparing physician 1’s and physician 2’s store-and-forward results.

Statistical analysis

Data were collected using Microsoft Excel (Microsoft Corporation, Redmond, WA), and analysis was carried out using Stata/SE v14.2 (StataCorp LLC, College Station, TX). Sensitivities and specificities to detect anterior segment pathologies from either live-streamed or store-and-forward videos compared with the in-person standard examination were calculated with 95% confidence intervals. In order to determine the magnitude of intra- and inter-observer agreement, the weighted kappa (κ) statistic was used as a quantitative measure, and is discussed here based on the following scale: <0, less than chance agreement; 0.01–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; 0.81–0.99, almost perfect agreement; 1.00, perfect agreement.¹¹ For kappa agreement assessment, diagnoses were subdivided into abnormal (e.g. blepharitis) and partially abnormal (e.g. minimal or mild blepharitis); abnormal diagnoses were scored as 1 to indicate a condition, while partially abnormal conditions were scored as 0.5 for the purpose of weighted kappa calculation. In cases where a patient’s in-person examination results included two diagnoses, if physicians rating patients over video (synchronous or asynchronous) detected both diagnoses, a score of 1 was assigned to the abnormal examination, whereas if only one of two diagnoses was detected, a score of 0.5 was assigned to this partial result (e.g. keratic precipitates and band keratopathy noted in person, but only keratic precipitates noted on video). All partial diagnoses were treated as ‘normal’ (i.e. mild blepharitis = 0) for the purpose of calculating sensitivity and specificity; incomplete diagnoses (e.g. detecting keratic precipitates, but missing band keratopathy, as above) were treated as ‘abnormal’ (counted as 1) when calculating sensitivity and specificity. Sensitivity values were compared between video sources using McNemar’s test. All p-values <0.05 were considered statistically significant.

Results

Forty-five children (89 eyes) were enrolled in this study. Both eyes were analysed in all patients except for one patient who refused examination of the second eye. The median age of participants was 12 years (range 5–17 years). Twenty-five (55%) were male, and 20 (45%) were female. Anterior segment abnormalities detected during the in-person standard examination for each patient are shown in Table 1.

Table 1.

List of anterior segment abnormalities diagnosed in person.

Category	Condition	Number diagnosed
Eyelids/eyelashes
	Blepharitis	37
	Haemangioma	2
	Epiblepharon	2
	Pigmentation	2
Conjunctiva/sclera
	Papillae	19
	Injection	7
	Haemorrhage	2
	Melanosis	1
Cornea
	Cornea scar	19
	Band keratopathy	11
	Graft	7
	Neovascularization	7
	Haab’s striae	6
	Cornea clouding	2
	Keratic precipitates	2
	Pannus	2
	Contact lens	1
	Posterior embryotoxon	1
	Ulcer	1
	Vogt’s striae	1
Anterior chamber
	Tube shunt	17
	Shallow	6
	Iris cyst	1
Iris
	Irregular pupil	13
	Peripheral iridotomy	5
	Atrophy	3
	Aniridia	2
	Iridocorneal adhesion	2
	Posterior synechiae	2
	Peripheral anterior synechiae	1
Lens
	Aphakia	12
	PCIOL	7
	Posterior capsular cataract	7
	Nuclear sclerosis	2
	Anterior pigment	1

Diagnostic performance of video-based examinations

Live streamed

Reviews of live-streamed examinations detected nearly all anterior chamber and lens disorders during the in-person examinations (sensitivity = 96%, specificity = 95–100%; see Table 2, part a). There was at least 96% agreement for these diagnoses, with weighted kappas indicating almost perfect agreement. Detection of disorders of the iris and conjunctiva/sclera was also similar (89–90% sensitivity; 95–97% specificity; 93–94% agreement), although weighted kappa calculations were affected by higher rates of possible agreement by chance (iris: κ = 0.84; conjunctiva/sclera κ = 0.87; both still considered almost perfect agreement). Detection of corneal abnormalities was slightly reduced compared to other diagnoses, but these were still detected most of the time (88% sensitivity; 90% specificity; 87% agreement; κ = 0.72, indicating substantial agreement). Detection of disorders of the eyelids/eyelashes was most difficult over live-streamed video, as only 54% of cases were detected. However, 92% of non-cases were correctly excluded. Although there was 74% agreement, high agreement expected by chance resulted in a much lower weighted kappa value than for other diagnoses (κ = 0.46, moderate agreement).

Table 2.

Diagnostic performance and agreement.

Category	Sensitivity % (95% CI)	Specificity % (95% CI)	Agreement %	Weighted κ (95% CI)
(a) Live streamed (physician 2) vs. in-person standard (physician 1)
Eyelids/eyelashes	54 (37–70)	92 (81–98)	74	0.46 (0.32–0.61)
Conjunctiva/sclera	90 (73–98)	97 (89–100)	94	0.87 (0.76–0.98)
Cornea	88 (76–96)	90 (76–97)	87	0.72 (0.44–0.99)
Anterior chamber	96 (79–100)	100 (95–100)	99	0.97 (0.92–1.00)
Iris	89 (70–98)	95 (87–99)	93	0.84 (0.71–0.96)
Lens	96 (81–100)	95 (87–99)	96	0.90 (0.80–1.00)
(b) Store and forward (physician 2) vs. in-person standard (physician 1)
Eyelids/eyelashes	72 (55–85)	94 (84–99)	80	0.59 (0.41–0.74)
Conjunctiva/sclera	66 (46–82)	98 (91–100)	88	0.69 (0.53–0.86)
Cornea	92 (81–98)	87 (73–96)	90	0.79 (0.67–0.92)
Anterior chamber	92 (73–99)	100 (95–100)	98	0.94 (0.86–1.00)
Iris	96 (80–100)	83 (71–91)	87	0.71 (0.56–0.86)
Lens	89 (71–98)	95 (87–99)	93	0.84 (0.72–0.96)
(c). Store and forward (physician 1) vs. in-person standard (physician 1)
Eyelids/eyelashes	80 (64–91)	92 (81–98)	85	0.70 (0.53–0.83)
Conjunctiva/sclera	72 (53–87)	97 (89–100)	89	0.73 (0.58–0.89)
Cornea	86 (73–94)	92 (79–98)	88	0.74 (0.47–1.00)
Anterior chamber	83 (63–95)	100 (95–100)	96	0.88 (0.77–0.99)
Iris	96 (80–100)	87 (77–94)	90	0.77 (0.64–0.91)
Lens	89 (71–98)	90 (80–96)	91	0.78 (0.37–1.00)
(d) Store and forward (physician 1) vs. store and forward (physician 2)
Eyelids/eyelashes	–	–	76	0.51 (0.31–0.64)
Conjunctiva/sclera	–	–	88	0.66 (0.48–0.85)
Cornea	–	–	84	0.67 (0.42–0.92)
Anterior chamber	–	–	96	0.88 (0.76–0.99)
Iris	–	–	90	0.79 (0.66–0.92)
Lens	–	–	96	0.88 (0.41–1.00)
(e) Live streamed (physician 2) vs. store and forward (physician 2)
Eyelids/eyelashes	–	–	79	0.54 (0.39–0.66)
Conjunctiva/sclera	–	–	91	0.77 (0.63–0.92)
Cornea	–	–	88	0.76 (0.66–0.87)
Anterior chamber	–	–	99	0.97 (0.91–1.00)
Iris	–	–	89	0.76 (0.62–0.89)
Lens	–	–	96	0.90 (0.80–1.00)

Sensitivity and specificity were not calculated for comparisons between raters of video feeds.

CI: confidence interval.

Store and forward

The video recording of each examination was stored and reviewed by both the in-person and remote examiners at least three months following the study visit. When the remote examiner (physician 2) reviewed the stored videos, sensitivity was high in most categories (cornea 92%, anterior chamber 92%, iris 96% and lens 89%; see Table 2, part b). Agreement with the in-person standard examination occurred >87% of the time (κ = 0.69–0.94) for all categories except the eyelids/eyelashes (80%, κ = 0.59). Disorders of the eyelids/eyelashes (72% sensitivity) and those of the conjunctiva and sclera (66% sensitivity) were detected less frequently. Although physician 2’s store-and-forward results agreed with the in-person standard examination 80% of the time for eyelids/eyelashes and 88% of the time for conjunctiva/sclera, weighted kappa values were lower (κ = 0.59 and κ = 0.69, respectively).

Compared to her performance with live-streamed video, physician 2’s detection rates for most abnormalities over store and forward were similar in many categories (cornea, anterior chamber, iris and lens). However, disorders of the eyelids/eyelashes were detected with significantly greater frequency (p = 0.02) when reviewing store-and-forward video (72% sensitivity) than live-streamed video (54%), and disorders of the conjunctiva/sclera were detected with significantly less frequency (p = 0.008) over store and forward (66% sensitivity) compared to live-streamed video (90%).

When physician 1 evaluated the store-and-forward videos, her performance varied, depending on diagnostic category (see Table 2, part c). Disorders of the iris were detected most frequently, but this category also included the most false-positives compared to the in-person examination (96% sensitivity, 87% specificity, 90% agreement; κ = 0.77). Disorders of the conjunctiva/sclera were detected less frequently than other categories, with almost no false-positives (72% sensitivity, 97% specificity, 89% agreement; κ = 0.73). For all other categories, sensitivity was between 80% and 89%, while specificity was between 90% and 100%. Agreement was ≥85% across all categories, including the eyelids/eyelashes, and kappa values indicated substantial agreement for all categories (κ = 0.70–0.78) except for the anterior chamber, which had almost perfect agreement (κ = 0.88).

When compared to physician 2’s store-and-forward performance, physician’s 1 store-and-forward detection rates of iris and lens diagnoses were nearly identical (96% and 89%, respectively; see Table 2, parts b and c). Physician 1 was marginally less successful at detecting anterior chamber abnormalities (83% sensitive) than physician 2 reviewing either stored or live-streamed video (92% and 96% sensitivity, respectively), but significantly more successful at detecting disorders of the eyelids or eyelashes (80% sensitivity) than physician 2 reviewing live-streamed videos (p = 0.008). Detection of cornea and conjunctiva/sclera abnormalities by physician 1 was roughly comparable to the performance of physician 2 reviewing the same video source asynchronously.

Agreement between asynchronous video examination results

After accounting for partial and incomplete diagnoses, agreement between all evaluations of video sources (live streamed by physician 2 and store and forward for both physicians) was substantial or almost perfect (κ = 0.67–0.97) and ≥87% across most diagnostic categories (Table 2, parts a–e). The cornea had slightly less (84%) but still substantial (κ = 0.67) agreement between physicians (Tables 2, part d). The eyelids/eyelashes had the lowest agreement between physicians 1 and 2 (76%; κ = 0.51; Table 2d) and between physician 2’s live-streamed and store-and-forward results (79%; κ = 0.54; Table 2, part e), reflecting lower agreement in this category throughout the study. For the eyelids/eyelashes, physician 1 was more likely to agree with her own ratings (85% agreement; κ = 0.70; Table 2 part c), while physician 2 was less likely to agree with her own ratings. Although physician 2 agreed with her own assessments slightly more frequently in the cases of conjunctival/scleral, corneal or anterior chamber abnormalities, agreement with physician 1 in these categories was similar. For disorders of the iris or lens, agreement was nearly identical.

Discussion

Confidence with telemedicine among eye-care providers is increasing, but at least one third continue to feel ‘not at all confident’ in using remote screening for eye care.¹² This underscores the importance of continuing to validate telemedicine equipment to build confidence in the quality and accuracy of its use.

Previous studies using real-time telemedicine for slit-lamp anterior segment examinations have been done on adult patients but not children.^7–10 Only one of these studies, by Threlkeld et al., reported sensitivities and specificities.⁸ They reported low sensitivities for anterior chamber (0%; 0/3), corneal (50%; 5/10: 9 corneal scars and 1 keratitis included) and lens findings (52%; 9/17), with slightly better sensitivities for conjunctival (72%; 18/25), eyelid (81%; 42/52, including 45 blepharitis) and iris (85%; 11/13) findings.⁸ In contrast, our study demonstrated better sensitivities (Table 2, part a), ranging from 88% to 96% across all categories, with the exception of the eyelids/eyelashes (54%). The higher sensitivities are likely due to significantly increased bandwidth capacity (4 MBps vs. 1.544 Mbps). Our lower sensitivity for detecting eyelid/eyelash abnormalities will be discussed further in the next paragraph, and it is important to note that this was mainly due to differences in calling blepharitis. Specificities from the Threlkeld et al. study were 90–100% across all categories, aside from eyelids, which had 79% specificity.⁸ In contrast, specificities from our study (Table 2, part a) ranged from 90% to 100% across all categories, including the eyelids/eyelashes.

Our study demonstrates that diagnoses across the anterior segment of the eye can be detected with substantial or almost perfect agreement through live-streamed (κ = 0.72–0.97) or store-and-forward telemedicine (κ = 0.69–0.94), with the exception of the eyelids/eyelashes (κ = 0.46 via live streamed and κ = 0.59 via store and forward; both moderate agreement). There are multiple reasons for why this category had reduced sensitivity and agreement. The discrepancy was due to differences in noting blepharitis, which is the least worrisome of diagnoses encountered, and is often not addressed unless the patient is symptomatic. There was no patient history provided over live-streamed or store-and-forward telemedicine, which is typically what would prompt a more thorough look at the eyelids and lashes. Limited communication would not be an issue in a true consultation situation. However, for the purposes of this study, we did not want to influence the viewing physician’s diagnoses. Lighting also may have been too bright to capture the eyelids/eyelashes well on video, whereas it likely did not hinder diagnoses in person. There were also clearly some physician differences in calling blepharitis, as physician 1 was much more likely to agree with herself (store and forward vs. in-person standard examination), whereas physician 2 was more variable in her detection of blepharitis. Also, it is possible that physician 1 was calling milder forms of blepharitis that physician 2 did not note or noted to be mild (which was counted differently in the statistical analyses). Physician 2 did perform better in detection of blepharitis over store and forward, which may be due to the ability to pause, rewind and review as needed, in contrast to the live-streamed video where this was not possible. It is important to note that specificities for the eyelids/eyelashes were high across the board (Table 2, parts a–c). Interestingly, there was lower sensitivity for detection of conjunctival/scleral abnormalities on store-and-forward compared to live-streamed telemedicine. This was due to discrepancies in calling papillae. This may have been overlooked due to focus on other parts of the examination, since allergic conjunctivitis is also somewhat less concerning a finding, especially compared to corneal and intraocular pathology, which were detected more consistently throughout our study. The Topcon digital slit lamps used in this study also had a 30-second time limit for video recording. During the in-person standard examination, physician 1 was notified each time the video stopped recording in order to minimise any loss of examination footage. However, it is very possible that some aspects of the examination were missed during the live-streamed and store-and-forward examinations, simply because parts of the examination were not recorded.

It is important to note that no videos were excluded due to poor image quality. However, two patients who were initially consented did not participate in the study. One was a very photophobic five-year-old, and the other was a very uncooperative five-year-old.

It is likely that some diagnoses were neglected to be noted on the in-person data sheet, as they were noted on store-and-forward examinations by both physicians but not the in-person standard examination. There were eight such instances: blepharitis for one eye, irregular pupil for one eye, iris atrophy for two eyes, pupillary membrane for one eye, pigment on anterior capsule for two eyes and cortical cataract for one eye. These are all diagnoses that likely did not have clinical relevance, but this does demonstrate that due to human error, a single physician’s in-person examination is not a perfect standard. Given the scarcity of paediatric anterior segment specialists, it would be difficult to obtain their consensus opinion for a more robust pragmatic standard.

Our study has several limitations. There is no standardised protocol for grading anterior segment pathology, which makes it difficult to interpret what different physicians note. We did not examine any children with anterior uveitis, and thus cannot comment on the ability to detect anterior chamber cell and flare by digital slit lamp. Both graders were on the hospital’s internal network during video streaming. So, transmission speeds were consistently at 4 Mbps. Streaming over the Internet between remote sites, especially those with limited bandwidth, can result in variable image quality. Also, this study utilised two ophthalmologists to minimize variability resulting from varying examiner skill. Including providers of different levels on the patient side and more ‘consulting’ physicians on the viewing side would confirm and solidify the generalisability of this telemedicine system. Another limitation of our study is that the median age of participants was 12 years old. Although a few five-year-olds were enrolled, future studies should look at more children in the younger (four- to five-year) age groups. Other methods of examining and imaging the anterior segment of children with developmental delays and children younger than four years of age need to be explored in order for telemedicine to be extended to those populations. In conclusion, video clips from the digital slit lamp hold promise for use in synchronous and asynchronous telemedicine consultations. It is likely with the addition of patient history in a focused consultation, the sensitivities and specificities for detecting anterior segment pathologies would further improve.

Footnotes

Acknowledgements

This study was given as a poster presentation at the American Association for Pediatric Ophthalmology and Strabismus Annual Meeting, Washington DC, 2018.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This research was supported in part by a grant from the Margie and Robert E. Petersen Foundation. The funding organisation had no role in the conduct of this research.

ORCID iD

Divya Mulchandani

References

Ehrlich

Anthopolos

Tootoo

, et al. Assessing geographic variation in strabismus diagnosis among children enrolled in Medicaid. Ophthalmology 2016; 123: 2013–2022.

Repka

MX.

Pediatric ophthalmology and strabismus of the future. J AAPOS 2007; 11: 323–324.

Woodward

Musch

Hood

, et al. Teleophthalmic approach for detection of corneal diseases: accuracy and reliability. Cornea 2017; 36: 1159–1165.

Maamari

Ausayakhun

Margolis

, et al. Novel telemedicine device for diagnosis of corneal abrasions and ulcers in resource-poor settings. JAMA Ophthalmol 2014; 132: 894–895.

Woodward

Bavinger

Amin

, et al. Telemedicine for ophthalmic consultation services: use of a portable device and layering information for graders. J Telemed Telecare 2017; 23: 365–370.

Kumar

Yogesan

Constable

IJ.

Telemedical diagnosis of anterior segment eye diseases: validation of digital slit-lamp still images. Eye 2009; 23: 652–660.

Shimmura

Shinozaki

Fuagawa

, et al. Real-time telemedicine in the clinical assessment of the ocular surface. Am J Ophthalmol 1998; 125: 388–390.

Threlkeld

Fahd

Camp

, et al. Telemedical evaluation of ocular adnexa and anterior segment. Am J Ophthalmol 1999; 127: 464–466.

Murdoch

Bainbridge

Taylor

, et al. Postoperative evaluation of patients following ophthalmic surgery. J Telemed Telecare 2000; 6: 84–86

10.

Smith

Bainbridge

Burns

, et al. Evaluation of telemedicine for slit lamp examination of the eye following cataract surgery. Br J Ophthalmology 2003; 87: 502–503.

11.

Viera

Garrett

JM.

Understanding interobserver agreement: the kappa statistic. Fam Med 2005; 37: 360–363.

12.

De Lott

Newman-Casey

Lee

, et al. Change in ophthalmic clinicians’ attitudes toward telemedicine during the coronavirus 2019 pandemic. Telemed J E Health. Epub ahead of print 17 July 2020. DOI: 10.1089/tmj.2020.0222.