Comparison of a Novel Handheld Telehealth Device with Stand-Alone Examination Tools in a Clinic Setting

Abstract

Background and Objective:

Research demonstrates that telemedicine is effective in pediatric settings but little is published to validate the quality of the data acquired by remote peripheral examination devices to accurately inform clinical decision-making.

Introduction:

The primary aim was to compare a novel Food and Drug Administration (FDA)-cleared multifunctional remote examination device (Tyto) with other stand-alone digital examination devices. The secondary aim was to ascertain whether either device produced images or sounds better able to provide clinical information to clinicians caring for children.

Materials and Methods:

Otoscopic images and heart and lung sounds from 50 patients of ages 2–18 years were acquired using the novel device and a stand-alone digital otoscope and stethoscope. Data were stored on a secure server for review by physicians (two pulmonary faculty, two general faculty, two cardiology faculty, and two cardiology fellows). Reviewers were blinded and they reviewed images and audio files in a randomized manner. Images and sounds were scored in terms of quality using a Likert scale. Means and standard deviations (and t-tests to compare those means) were calculated. Individual (heart sounds, lung sounds, and otoscopic images) and aggregate scores were compared.

Results:

The novel device provided higher sound and image quality with less chance of an inability to make a diagnosis than the stand-alone devices. The novel device had a superior mean comparative diagnostic score with a high intra- and inter-reliability of cardiac, pulmonary, and otoscopic diagnosis.

Discussion and Conclusions:

The novel device outperformed the stand-alone digital stethoscope and otoscope and was better able to provide usable data to support a clinical encounter.

Introduction

Both the American Academy of Pediatrics and the Academy of Family Medicine have recommended guidelines for scheduled pediatric clinic visits.^1,2 These visits are typically performed in an office or in a clinic setting, incorporating a history and physical examination, developmental assessment, aspects of preventative medicine, and immunizations. Assessment of the ill child may occur in the office, urgent care, or emergency department settings.³ There is increasing interest in the provision of these visits in remote care settings that may also include the home, school, day care, or other health care facilities. To ensure quality encounters, clinicians and patients must utilize reliable validated diagnostic equipment and streamlined methods of data collection and transfer. In recent years, digital remote examination tools, such as the digital stethoscope and otoscope, have been incorporated into clinical practice and particularly in telemedicine solutions.^4
–6

Telemedicine is the provision of health care utilizing telecommunication technologies augmented by the use of peripheral examination tools.^7,8 Pediatric telehealth services have been incorporated into health care systems, hospitals, emergency rooms, out-patient clinics, schools, and day-care settings with evidence showing clinical effectiveness for the diagnosis and treatment of acute illness. These models often strive to replicate in-person services and as such, models have been published in the peer-reviewed literature.^9

–14 There is evidence that remote consultation can modify health practices and treatment compliance in peripheral environments for specific diseases.¹⁵

The introduction of a telemedicine model in suburban childcare centers using validated diagnostic tools has resulted in significantly reduced pediatric office and emergency department visits¹⁶ along with the additional benefit of a reduction in parental absences from work. The utilization of digital diagnostic technologies in clinical practice has particular potential in underserved and remote areas, potentially reducing interhospital transfers and waiting times to access specialty services and improved clinical outcomes for children.

The Tyto device (TytoCare Ltd., Israel) is a novel examination system that includes a built-in examination camera, an infrared thermometer, a wireless communication unit, a lithium ion battery, and a touch screen. The system also incorporates a digital stethoscope, a digital otoscope, and a tongue depressor. The Tyto platform enables the capacity for live video or store and forward applications, and users can be directed by voice- or on-screen instructions to obtain images and sounds to comport with the standard of care by enabling a remote physical examination. Neither the camera nor thermometer was used in this study.

The purpose of the study was to assess the clinical validity and reliability of a next-generation novel Food and Drug Administration (FDA)-cleared, multifunction comprehensive telehealth examination tool and compare it with the stand-alone digital peripheral examination devices currently deployed in our telehealth program. The primary aim of this study was to determine whether the novel device showed diagnostic equivalency with the commercially available stand-alone FDA-cleared telehealth examination devices routinely used in our telehealth program. There are many other devices on the market that were not studied. A secondary aim was to compare the images and sounds obtained with each of the devices to define which was best able to provide the physician with accurate clinical information. No attempt was made to make a clinical diagnosis such as aortic stenosis or otitis media from the data. The study was done to assess validity of a novel device.

Materials and Methods

Both the novel and stand-alone devices adhere to the International Electrotechnical Commission standard for medical products and have received 510K clearance by the U.S. FDA.

The study was approved by the local Institutional Review Board with parental/guardian consent obtained in each case for participation. Data were prospectively collected from children of ages 2–18 years presenting for their scheduled visit to a Pediatric Cardiology Clinic at the University of Virginia Children's Hospital (Charlottesville, VA) affiliated with the UVA Health System and the UVA School of Medicine. A standard physical examination was performed in the clinic by faculty physicians. Once consent was obtained, the study nurse obtained heart sounds, lung sounds, and images of both tympanic membranes with the TytoCare device and the two stand-alone digital examination devices (One Digital Stethoscope (Thinklabs Medical LLC) and the Horus HD Digital Scope System (JEDMED Instrument Co). The images and sounds were randomized by device before data acquisition and subsequent review.

In each examination, the following information was recorded from each participant:

Four heart sounds from the novel device and four from the stand-alone stethoscope.

Six lung sounds from the novel device (front/back of body) and six from the stand-alone stethoscope.

Two ear images from the novel device (left/right) and two from the stand-alone otoscope.

All data were loaded onto a secure server. The sounds and images were reviewed by eight physicians (two fellows in cardiology, two pediatric pulmonologists, two general pediatricians, and two pediatric cardiologists). The images and sounds were reviewed on a secured website and the reviewers were blind to the device and the subject.

Exclusion criteria for the study were patients with skin complaints, which might limit device use, patients with cognitive impairment, and cases wherein parental or guardian consent could not be obtained. Skin conditions such as severe inflammation might limit cooperation from discomfort. No patients who consented were excluded for skin conditions or cognitive impairment. The heart examination with each of the devices comprised recordings of the four standard auscultation points (aortic, pulmonic, tricuspid, and mitral). Lung auscultation was conducted at six standardized points (two anterior and four posterior), recording an 8 s audio at each site.

All reviewers recorded their opinions of the quality of the images and sounds on a Likert scale between 1 and 5 (where 1 = very good and 5 = very poor) such that higher mean values signified worse overall diagnostic quality. Categorical ratings were made on a scale for diagnosis at each anatomic site such that 1 = no clinical finding, 2 = significant clinical finding, and 3 = presence/absence of significant clinical finding could not be determined based upon the information provided.

Statistical Methods

Statistical analyses were performed using IBM SPSS^® Version 13.0 Software (SPSS, Inc., Chicago, IL). As there were 24 recordings and images per patient (10 auscultation recordings plus 2 images with both the novel and the stand-alone devices), a minimum of 30 patients would be needed for the provision of 360 data points assessable by each of the measurement systems (720 total). This calculation exceeded the sample size needed to demonstrate a probability difference of 0.3 between the groups.¹⁷ Quality assessments of the devices were recorded as means + standard deviation with comparisons of continuous variables using the two-sampled t-test or the Wilcoxon rank sum test where appropriate. Chi-squared, Fisher's exact testing, and proportional Z tests were used where appropriate for comparisons of quality on the categorical ratings and for diagnostic group assignment. The internal consistency of the data was measured with Cronbach's alpha and the reproducibility with the intraclass correlation coefficient.¹⁸ It was assumed that those who were rating the devices and measuring each data point were representative of the rating population as a whole, permitting estimates of both inter- and intrareliability, where a preset threshold >0.80 was considered good. Confidence limits of 95% were determined with p values <0.05 considered significant.

Results

Between July and October 2016, 50 children were enrolled in the study that included 23 males and 27 females (mean overall age 10.5 years [range 2–17]). Of the cohort, 12 had known structural cardiac abnormalities, with 1 child having a history of arrhythmia. The remainder of the children were referred because of a range of conditions including murmur, chest pain, palpitations, and syncope.

Images and sounds obtained with the novel device were judged as of higher quality when compared with those obtained with the stand-alone peripheral device (mean quality score overall = 2.8 + 1.05 vs. 3.39 + 0.94, respectively, where smaller means equaled better quality; p < 0.001). Table 1 gives comparisons between the novel and the stand-alone device with regard to image and sound quality. The novel device was more likely to be rated higher overall, with less chance that the clinician was unable to document a clinical finding when using the device (p = 0.001) as compared with the stand-alone devices. Both the Cronbach alpha and intraclass correlation coefficients for both inter- and intrareliability exceeded the preset threshold of 0.80 (0.84–99, 0.90–0.99, respectively). Table 2 gives similar comparisons for clinical findings, in which the novel device was more likely than the stand-alone devices in enabling the clinicians to detect a clinical finding (p < 0.0001) as well as for ear, heart, and lung findings, respectively (p < 0.0001, p < 0.0001, p = 0.004, respectively). Similarly, the Cronbach alpha and intraclass correlation coefficients for both inter- and intrareliability exceeded the preset threshold of 0.80 for diagnosis (0.85–0.99, 0.89–0.99, respectively).

Table 1.

Statistical Summary: Image and Sound Quality Comparisons

ASSESSMENT	ANALYSIS	RECORDED DATAPOINTS	p
Overall image/sound quality	χ² (Chi-squared)	Tyto 1,381	0.0001
Overall image/sound quality	χ² (Chi-squared)	UVA^a 1,511	0.0001
Proportions without a diagnosis	Proportional Z-test	Tyto 0.53	0.001
Proportions without a diagnosis	Proportional Z-test	UVA 0.48	0.001
Image/sound quality mean score (SD)	Two sample t-test	Tyto 2.80 (1.05)	0.0001
Image/sound quality mean score (SD)	Two sample t-test	UVA 3.39 (0.94)	0.0001
Image quality—ear	χ² (Chi-squared)	Tyto 696	0.0001
Image quality—ear	χ² (Chi-squared)	UVA 797	0.0001
Sound quality—heart	χ² (Chi-squared)	Tyto 350	0.0001
Sound quality—heart	χ² (Chi-squared)	UVA 381	0.0001
Sound quality—lung	χ² (Chi-squared)	Tyto 335	0.0001
Sound quality—lung	χ² (Chi-squared)	UVA 353	0.0001
Image/sound quality—inter-rater reliability	Cronbach alpha/intraclass correlation coefficient	0.84–0.99
Image/sound quality—intra-rater reliability	Cronbach alpha/intraclass correlation coefficient	0.90–0.99

Auditory comparisons were made with the One Digital Stethoscope (Thinklabs Medical LLC) and imaging comparisons were made with the Horus HD Digital Scope System (JEDME Instrument Co). Both are listed as standard of care UVA devices used in clinical practice.

SD, standard deviation.

Table 2.

Statistical Summary: Diagnostic Comparisons

ASSESSMENT	ANALYSIS	RECORDED DATAPOINTS	p
Overall diagnosis	χ² (Chi-squared)	Tyto 1,378	0.0001
Overall diagnosis	χ² (Chi-squared)	UVA 1,498	0.0001
Proportions without a diagnosis	Proportional Z-test	Tyto 0.56	0.001
Proportions without a diagnosis	Proportional Z-test	UVA 0.48	0.001
Diagnostic mean score	Two sample t-test	Tyto 1.66	0.0001
Diagnostic mean score	Two sample t-test	UVA 1.95	0.0001
Diagnosis—ear	χ² (Chi-squared)	Tyto 694	0.0001
Diagnosis—ear	χ² (Chi-squared)	UVA 774	0.0001
Diagnosis—heart	χ² (Chi-squared)	Tyto 350	0.0001
Diagnosis—heart	χ² (Chi-squared)	UVA 372	0.0001
Diagnosis—lung	χ² (Chi-squared)	Tyto 334	0.004
Diagnosis—lung	χ² (Chi-squared)	UVA 352	0.004
Diagnosis—inter-rater reliability	Cronbach alpha/intraclass correlation coefficient	0.85–0.99
Diagnosis—intra-rater reliability	Cronbach alpha/intraclass correlation coefficient	0.89–0.99

Discussion

This prospective cohort study from a single outpatient pediatric cardiology clinic demonstrated that the quality of images and sounds obtained using the novel device was of higher quality than those obtained using stand-alone remote examination devices routinely used in the University of Virginia telemedicine program. The novel device, which incorporates a digital otoscope, stethoscope, examination camera, and thermometer, was shown to more adequately enable remote diagnosis (no camera images or temperature data were collected in this study, so no comment can be made). There was a high level of intra- and inter-reliability for the recorded measurements of the heart, lungs, and ears. In evaluating children referred to a specialized pediatric cardiology clinic, clinicians more correctly assessed abnormal clinical findings using the novel device.

There are several issues that may be seen as limitations of this study. The participants were asked to participate on the day of a scheduled visit to the cardiology clinic. Most children were well with no cardiac pathology. It would be expected that the majority would have normal heart sounds, normal lung sounds, and normal ear examinations without significant findings. The ear images were not reviewed by an otolaryngologist and for the most part did not reveal pathology other than occasional scarring of the tympanic membrane (noted by the reviewers). Many of the ear examinations were limited by earwax that was not removed. The aim of the study was to compare the novel stand-along device with standard digital device tools. The aim of the study was not to make a diagnosis or confirm findings from the clinic visit with the novel or standard device.

The value and reliability of assessment of heart sounds with a digital stethoscope have been confirmed previously in pediatric patients.¹⁹ Although most murmur referrals in healthy children >1 year of age do not reveal any cardiac pathology, the ability to efficiently evaluate or follow patients using validated remote examination tools may improve access to care, triage, and reduce the burden of travel for patients and their families.^20,21

Introduction in the 1980s of electronic stethoscopes attempted to improve sound amplification and filtration²² with recent implementation of noise removal algorithms capable of cancelling internally and externally derived extraneous noises likely to interfere with lower amplitude murmurs.⁴ The level of agreement with the novel stethoscope and the standard electronic device was high in our study. Given that there still remains some debate around breath sound terminology on auscultation,²³ future definitions will influence the level of interobserver agreement for any digital device used. There are some basic practical differences between the novel and other digital stethoscopes. As an example, the One Digital Stethoscope (Thinklabs Medical LLC) has no application interface and requires the attachment of separate connectors to the audio channels, with hand manipulation of the audio filter. The hardware connections of this system have the potential to degrade the audio quality, whereas the Tyto system (which has all of its hardware in-built along with embedded filtration software) appears to have a superior sound quality and is much easier to use due to the touch screen interface.

There was high rating of the images and a high reliability with the novel device tympanic membrane diagnosis when compared with those of stand-alone digital otoscope. Our findings are in keeping with similarly reported rates using other digital equipment of enhanced visualization over conventional microscopy,²⁴ with similar levels of reported diagnostic inadequacy using other digital equipment.²⁵ Some of the conditions limiting the use of the digital otoscope may be device related but many issues are nondevice related (such as insufficient visualization of the tympanic membrane and/or occlusion of the ear canal with cerumen). The literature is somewhat confusing since studies are heterogeneous in their reporting of either incomplete drum visualization or lack of diagnosis when there is excess cerumen. The stand-alone digital otoscope studied provides high-resolution imagery; however, the field of view of this instrument is limited as is the distance of functioning of the automatic zoom. This invariably necessitates some manual focusing of the image. By comparison, the novel device permits a full field view of the ear canal and the tympanic membrane with preliminary white balancing of the image at the commencement of the examination and automated in-built imaging focus during the procedure.

Overall, the data concerning the efficacy of the use of digital technologies in pediatric clinical management are somewhat difficult to interpret, principally because of system variations. This may in part, however, be obviated in the future after the establishment of network groups such as the Health Experts online at Portsmouth (HELP) system in 2014, the Supporting Pediatric Research on Outcomes and Utilization of Telehealth (SPROUT) established in 2015, and consensus decision reporting for the classification of recorded lung sounds.^26,27 The validation of diagnostic digital technology in the clinic may also provide a repository of quality sounds and images in a virtual library, which may be used for training purposes.²⁸

There are many issues that impact the use of telemedicine in pediatric populations. Those issues have included reimbursement, licensure, bandwidth, electronic medical record integration, credentialing, technology choices, consumer demand, and practice guidelines.^29

–33 Equally important are concerns that care outside the context of the primary or specialty medical home, particularly when patients and their families are seeking “direct to consumer” services that may fragment care and may not favorably compare with the standard-of-care in-person visit.

In support of the use of telemedicine for pediatric populations, this study demonstrates that remote examination tools can provide high-quality data that can inform telehealth examinations. Such data may permit more widespread screening for medical conditions of childhood warranting medical attention.

Conclusion

In summary, the novel device (Tyto) met both of the articulated study aims. In the first instance, the novel device performed better than stand-alone digital examination devices utilized in our telehealth program. In the second instance, use of the novel device resulted in lower rates of diagnostic failure with high intra- and inter-reliability for examination of the heart, lungs, and ears. In our study, the device was managed by a registered nurse with basic training in its use; there is great potential for use of the novel device by parents at home or personnel in a school or day-care facility to collect the relevant data for transmission to a remotely located clinician to inform clinical decision-making and in particular wherever possible within the context of the medical home.³⁴ This approach may augment care of children with special needs or medical fragility.³⁵

Footnotes

Acknowledgments

The authors thank the children who participated, Rose Allen, RN, who did the data collection, and the Office of Telemedicine for technical assistance. This study was funded by a grant from TytoCare, Ltd.

Authors' Contributions

Dr. McDaniel conceptualized and designed the study, drafted the initial article, and reviewed and revised the article. Dr. Novicoff carried out data analysis, conceptualized and designed the study, drafted the initial article, and reviewed and revised the article. Mr. Gunnell and Mr. Cattell Gordon designed the data collection, storage, and randomization of the data. They provided access to the secure website for data review. All authors approved the final article as submitted and agree to be accountable for all aspects of the work.

Disclosure Statement

No competing financial interests exist.

References

American Academy of Pediatrics, Council on School Health. School-based health centers and pediatric practice. Pediatrics, 2012; 129:387–393.

Conners

, Kressly

, Perrin

, Richerson

, Sankrithi

; Committee on Practice and Ambulatory Medicine; Committee on Pediatric Emergency Medicine; Subsection on Telehealth Care; Section on Emergency Medicine; Subcommittee on Urgent Care; Task Force on Pediatric Practice Change; Collaborators. Non-emergency acute care: When it's not the medical home. Pediatrics, 2017; 139:pii:e20170629.

American Academy of Pediatrics, Committee on Pediatric Emergency Medicine. Pediatric care recommendations for freestanding urgent care facilities. Pediatrics, 2014; 133:950–953.

Leng

, Tan

, Chai

KTC

, Wang

, Ghista

, Zhong

. The electronic stethoscope. BioMed Eng OnLine, 2015; 14:66.

Kevat

, Kalirajah

, Roseby

. Digital stethoscopes compared to standard auscultation for detecting abnormal pediatric breath sounds. Eur J Pediatr, 2017; 176:989–992.

Moberly

, Zhang

, Yu

, et al. Digital otoscopy versus microscopy: How correct and confident are ear experts in their diagnoses?. J Telemed Telecare, 2018; 24:453–459.

Weinstein

, Lopez

, Joseph

, et al. Telemedicine, telehealth and mobile health applications that work: Opportunities and barriers. Am J Med, 2014; 127:183–187.

Dorsey

, Topol

. State of Telehealth. N Engl J Med, 2016; 375:154–161.

McIntosh

, Cirillo

, Wood

, Dozier

, Alarie

, McConnochie

. Patient evaluation of an acute care pediatric telemedicine service in urban neighborhoods. Telemed J E Health, 2014; 20:1121–1126.

10.

Heath

, Salerno

, Hopkins

, Hertzig

, Caputo

. Pediatric critical care telemedicine in rural underserved emergency departments. Pediatr Crit Care Med, 2009; 10:588–591.

11.

Otero

, Lopez-Magallon

, Jaimes

, et al. International telemedicine in pediatric cardiac critical care: A multicenter experience. Telemed J E Health, 2014; 20:619–625.

12.

Ronis

, McConnochie

, Wang

, Wood

. Urban telemedicine enables equity in access to acute illness care. Telemed J E Health, 2017; 23:105–112.

13.

Fang

, Collura

, Johsnon

, et al. Emergency video telemedicine consultation for newborn resuscitations: The Mayo Clinic experience. Mayo Clin Proc, 2016; 91:1735–1743.

14.

McConnochie

, Ronis

, Wood

, Ng

. Effectiveness and safety of acute care telemedicine for children with regular and special healthcare needs. Telemed J E Health, 2015; 21:611–621.

15.

Wood

, Clements

, McFann

, Slover

, Thomas

, Wadwa

. Use of telemedicine to improve adherence to American Diabetes Association standards in pediatric type 1 diabetes. Diabetes Technol Ther, 2016; 18:7–14.

16.

McConnochie

, Tan

, Wood

, et al. Acute illness utilization patterns before and after telemedicine in childcare for inner city children: A cohort study. Telemed J E Health, 2007; 13:381–390.

17.

Cantor

. Sample size calculations for Cohen's kappa. Psychol Methods, 1996; 1:150–153.

18.

Bravo

, Potvin

. Estimating the reliability of continuous measures with Cronbach's alpha or the intraclass correlation coefficient: Toward the integration of two traditions. J Clin Epidemiol, 1991; 44:381–390.

19.

Gupta

, May

. Managing a “new” murmur in healthy children and teens. Clin Pediatr (Phila), 2017; 56:357–362.

20.

Geggel

, Horowitz

, Brown

, Parsons

, Wang

, Fulton

. Parental anxiety associated with referral of a child to a pediatric cardiologist for evaluation of a Still's murmur. J Pediatr, 2002; 140:747–752.

21.

Belmont

, Mattioli

. Accuracy of analog telephonic stethoscopy for pediatric telecardiology. Pediatrics, 2003; 112:780–786.

22.

Hahn

. On stethoscope design: A challenge for the biomedical circuit designers. Biomed Sci Instrum, 2001; 37:499–503.

23.

Pasterkamp

, Brand

, Everard

, et al. Towards the standardization of lung sound nomenclature. Eur Respir J, 2016; 47:724–732.

24.

Eikelboom

, Mbao

, Coates

, Atlas

, Gallop

. Validation of tele-otology to diagnose ear disease in children. Int J Pediatr Otorhinolaryngol, 2005; 69:739–744.

25.

Swanepoel de

, Clark

, Koekemoer

, et al. Telehealth in audiology: The need and potential to reach underserved communities. Int J Audiol, 2010; 49:195–202.

26.

Lin

, Cole

, Chin

, Mahnke

. The Health Experts online at Portsmouth (HELP) system: One-year review of adult and pediatric asynchronous telehealth consultation. SAGE Open Med, 2016; 4:1–6.

27.

Aviles-Solis

, Vanbelle

, Halvorsen

, et al. International perception of lung sounds: A comparison of classification across some European borders. BMJ Open Respir Res, 2017; 4:e000250.

28.

Cook

, Erwin

, Triola

. Computerized virtual patients in health professions education: A systematic review and meta-analysis. Acad Med, 2010; 85:1589–1602.

29.

Olson

, McSwain

, Curfman

, Chuo

. The current pediatric telehealth landscape. Pediatrics, 2018; 141:e20172334.

30.

Dharmar

, Sadorra

, Leigh

, Young

, Nesbitt

, Marcin

. The financial impact of a pediatric telemedicine program: A children's hospital's perspective. Telemed J E Health, 2013; 19:502–508.

31.

Dowie

, Mistry

, Young

, et al. Telemedicine in pediatric and perinatal cardiology: Economic evaluation of a service in English hospitals. Int J Technol Assess Health Care, 2007; 23:116–125.

32.

Satou

, Rheuban

, Alverson

, et al. Telemedicine in pediatric cardiology: A scientific statement from the American Heart Association. Circulation, 2017; 135:e648–e678.

33.

McSwain

, Bernard

, Burke

, et al. American Telemedicine Association operating procedures for pediatric telehealth. Telemed J E Health, 2017; 9:699–706.

34.

Committee on Pediatric Workforce, Marcin

et al., The use of telemedicine to address access and physician workforce shortages. Pediatrics, 2015; 136:202–209.

35.

American Academy of Pediatrics, Medical Home Initiatives for Children with Special Needs Project Advisory Committee. The medical home. Pediatrics, 2002; 110(1 pt 1):184–186.