Abstract
BACKGROUND:
To compare oxygen saturation (SpO2) and heart rate (HR) recorded by a reference wired pulse oximeter to a wireless pulse oximeter in inpatient neonates.
METHODS:
Term infants born≥37 + 0 weeks and preterm infants born≤35 + 0 weeks gestation were enrolled and time-matched data pairs were obtained. The primary outcome was intraclass correlation coefficient and r-values between the two oximeters for heart rate and oxygen saturation.
RESULTS:
Thirty term and 20 preterm neonates were enrolled. There was a high degree of correlation between the two oximeters for HR (r = 0.926) among all 50 infants, and excellent interclass correlation (ICC = 0.961), but there were no bradycardia episodes in either term or preterm infants. There was a lesser degree of correlation for SpO2 values in the term and preterm groups (r = 0.242; 0.521, respectively) along with moderate interclass correlation (ICC = 0.719) but few episodes of hypoxemia≤90% occurred in enrolled subjects.
CONCLUSIONS:
There were no significant differences between the wireless and reference wired oximeters for assessing HR. There was less correlation between the two oximeters for monitoring SpO2 in both the term and preterm group. Wireless pulse oximetry may have practical advantages for use in inpatient neonates, but additional studies are needed that include bradycardia and desaturation events to delineate this question.
Introduction
Pulse oximetry is an indispensable tool for inpatient monitoring of heart rate (HR) and oxygen saturation (SpO2) across the age spectrum, including neonates [1, 2]. The clinical use of pulse oximeters has reduced the need for frequent invasive arterial blood sampling, and has improved patient surveillance by providing continuous information about cardiorespiratory status. Wired pulse oximeters, however, have some potential limitations due to connecting cables to the infant, which can be challenging when conflicting with care goals such as couplet bonding in the delivery room or during postpartum infant surveillance in the newborn nursery. In a recent study in the immediate postnatal period, the use of wired pulse oximetry as part of a quality bundle with other observation tools was associated with a reduced incidence of sudden unexpected postnatal collapse, but nursing staff noted that the wired oximeter interfered with maternal–child contact and other patient care needs during skin-to-skin care [3]. Another study evaluating wireless oximeters in the outpatient setting noted that the average time to detection of a reliable SpO2 reading on a wireless pulse oximeter was approximately 30 seconds, potentially aiding clinicians in clinical decision making [4].
Wireless pulse oximetry has the potential to be less intrusive and hence more adaptable in selected inpatient clinical settings but it has not previously been evaluated in inpatient settings. A recent study acknowledged the potential usefulness of wireless pulse oximetry technology during skin-to-skin care in the delivery room [5]. Our primary objective in this proof-of-concept pilot study was to evaluate the potential role of wireless pulse oximetry in hospitalized neonates. We designed this pilot study to compare a reference wired pulse oximeter to a wireless oximeter in stable full term infants in the newborn nursery and in preterm neonates in the Neonatal Intensive Care Unit (NICU).
Methods
Study population
We enrolled two separate cohorts: full term neonates in the newborn nursery born at ≥37 + 0 weeks gestation, and preterm neonates in the NICU born ≤35 + 0 weeks gestation. To be eligible, full term neonates had to be≥18 hours old and to have passed their routine congenital heart disease screen. Eligible preterm neonates needed to (1) be breathing spontaneously, (2) have a current weight ≥1800 grams to facilitate an adequate fit of the wireless oximeter sock, and (3) still be at risk for apnea of prematurity-related bradycardia or desaturation events.
Study design
This is a single-center proof-of-concept pilot study in hospitalized neonates to assess the potential utility of a wireless pulse oximeter compared to a reference wired pulse oximeter for detection of SpO2 and HR. For this study, all monitor alarms were disabled. As the reference wired oximeter, we utilized the Nonin 8500M™, an FDA approved, representative pulse oximeter commonly used in clinical practice [6, 7]. As the wireless oximeter, we utilized the Owlet Smart Sock™ Version 2.0, a previously available commercial wireless pulse oximeter marketed as a wellness device that is not FDA-approved for use as a medical device. The wireless oximeter sensor was placed inside the pre-sized, manufactured fabric sock with a Velcro strap and wrapped around the infant’s foot. The smallest size (size 1) Owlet Sock™, intended for infants zero to one month of age, was utilized for this study. The manufacturer’s recommended minimum infant weight for proper fit of this sock is six pounds (2700 grams) [8]. However, since the sock fit infants weighing as low as 1800 grams, we selected 1800 grams as the lower weight limit for this study in order to assess wireless oximeter use in preterm NICU patients. To achieve the best fit possible, we ensured that the Velcro strap could be fastened around the infant’s foot and the sensor would lay flat across the infant’s foot. Enrolling patients below the manufacture’s recommended minimum weight allowed our team to be able to recruit our goal sample size for the preterm population. Our study was approved by the Walter Reed National Military Medical Center IRB, and informed parental consent was obtained prior to study enrollment.
Data acquisition
The wired oximeter was placed on one foot according to standard protocol and covered with a self-adherent wrap to prevent light interference. The wireless oximeter was placed inside the pre-sized fabric sock and placed on the other foot. Simultaneous recordings were started once the infant was quiet or sleeping. Full term neonates in the newborn nursery were monitored for 20 minutes. Preterm neonates in the NICU were monitored for 60 minutes in order to maximize the opportunity to capture hypoxic and bradycardic events. Whenever movement artifact was detected, defined as an oximeter registering a low signal reading, the recording was paused until the movement artifact resolved. Motion artifact in the reference wired oximeter was detected when the monitor light turned from red to orange, at which point HR and SpO2 data would not display. Motion artifact in the wireless oximeter was detected when the graphic user interface (GUI) notification symbol registered a numerical value for motion on the study laptop computer. Paired data points were used for final analysis if they were obtained during a period without movement artifact.
The reference wired oximeter data were stored in the oximeter internal memory, which was then extracted to an Excel™ spreadsheet. The wireless oximeter data were transmitted to a laptop computer and stored in an Excel™ spreadsheet. The base station provided by the manufacturer for wireless data acquisition in the home was not used for this study because of concern for connectivity issues related to security settings of the hospital’s wireless network.
The wireless oximeter had an averaging time of 12 seconds and the reference wired oximeter had an averaging time of four seconds. To enable data analysis in spite of this difference in averaging time, time-matched data pairs from recorded data were collected (e.g., wireless oximeter reading at 4:13:44 PM was matched with the wired oximeter reading at 4:13:44 PM). In the time-matched data pair, the wireless oximeter’s SpO2 and HR reading at the date/time stamp was the previous 12 second average value, and the wired oximeter’s SpO2 and HR reading at the date/time stamp was the previous four second average value. A total of 7,832 time-matched data pairs (3874 term and 3958 preterm) were collected. There was a large number of normal SpO2 values (> 90%) and a small number of abnormal SpO2 values (≤90%), and only fourteen infants had low SpO2 values. In order to prevent ascertainment bias due to the small number of low SpO2 values (≤90%) from either oximeter, all low SpO2 values from these 14 infants (322 data pairs) were included in the analysis. For these 14 infants, we randomly selected one normal SpO2 data pair (> 90%) for each low SpO2 data pair from the same infant, resulting in 297 data pairs; this is due to one infant having fewer normal SpO2 data pairs (n = 52) than low SpO2 data pairs (n = 77). This resulted in 322 low SpO2 and 297 normal SpO2 data pairs. In addition, to include some data from all study infants, we enriched the total sample by randomly selecting 5 additional normal SpO2 data pairs from the 36 infants not having any low SpO2 pairs≤90% (resulting in 180 normal data pairs). A grand total of 799 data pairs was used in the final analysis; 322 low SpO2 data pairs and 477 (or 297 + 180) normal SpO2 data pairs.
Sample size estimation
In this study, the observed difference in heart rate (bpm) between wired and wireless was 0.51 with an intraclass correlation coefficient of 0.961, wired standard deviation of 18.8 and wireless standard deviation of 18.4. A sample of 799 data pairs achieved a power of 86.9% with a one-sided test and 78.9% with a two-sided test.
The observed difference in oxygen saturation (%) between wired and wireless was 1.8 with an intraclass correlation coefficient of 0.719, wired standard deviation of 4.2 and wireless standard deviation of 3.3. A sample of 799 data pairs achieved a power of 100% with a one-sided test and 100% with a two-sided test.
Data analysis
Data analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC, USA). Gender, birth weight, and gestational age were analyzed using descriptive statistics and summarized for the full term and preterm cohorts. The relationship between the two oximeters for SpO2 and HR was analyzed within the term and preterm cohorts by Pearson’s correlation coefficient (r) and scatterplots. We overlaid the original regression line and the jittered data to prevent superimposing identical data points. The jittered data was generated by adding a random noise from –0.5 to 0.5. The primary outcome was the correlation and inter-oximeter agreement between the two oximeters for SpO2 and HR recorded in the term and the preterm cohorts. Inter-oximeter agreement was evaluated using Bland-Altman plots and intraclass correlation coefficient (ICC).
Results
Between June 2019 and May 2020, 50 neonates were enrolled in the study; 30 born full term and 20 born preterm (Table 1). All approached families in the term and preterm cohort consented and were enrolled. Infant characteristics for each cohort are summarized in Table 1. Seventeen of the preterm infants (85%) were < 2,700 grams, the minimum weight recommended by the manufacturer of the Owlet Sock™.
Baseline demographics. Mean (SD) unless otherwise indicated
Baseline demographics. Mean (SD) unless otherwise indicated
There was a high degree of correlation between the two oximeters for HR among all 50 infants (r = 0.926; Fig. 1). However, no HR events < 90 bpm occurred in either group. Using Bland-Altman plots to assess agreement among all 50 infants, the overall intraclass correlation coefficient (ICC) was excellent (ICC = 0.961), with roughly equal intraclass correlations for term (ICC = 0.889) and preterm (ICC = 0.917) neonates, respectively (Fig. 2). Given that were there no HR events < 90 bpm, sensitivity and specificity were unable to be calculated.
Scatterplot of 799 paired heart rate data points: 174 from 30 term infants (r = 0.806) and 625 from 20 preterm infants (r = 0.848) comparing the reference wired oximeter (y-axis) and wireless oximeter (x-axis). Overall (n = 50) correlation (r = 0.926). Data points were jittered to avoid superimposing identical data pairs.
Bland-Altman plot of heart rate (bpm). The bias is zero, with confidence limits of ±1.96×SD. Data points were jittered to avoid superimposing identical data pairs. 30 full term neonates, 174 data pairs.
Bland-Altman plot of heart rate (bpm). The bias is zero, with confidence limits of ±1.96×SD. Data points were jittered to avoid superimposing identical data pairs. 20 preterm neonates, 625 data pairs.
In the term cohort (Fig. 3a) and the preterm cohort (Fig. 3b), there was a lesser degree of correlation between the two oximeters (r = 0.242 and r = 0.521, respectively). Using Bland-Altman plots to assess oximeter agreement among all 50 infants, the overall ICC was moderate at 0.719, but with poor agreement (ICC = 0.365) in the term group and moderate agreement (ICC = 0.673) in the preterm group (Fig. 4). Calculated sensitivity and specificity for SpO2 for full term neonates was 0.00% (95% CI: 0.00% to 16.11%) and 96.08% (95% CI: 91.66% to 98.55%), respectively. Calculated sensitivity and specificity for SpO2 for preterm neonates was 21.46% (95% CI: 16.63% to 26.94%) and 90.66% (95% CI: 87.19% to 93.44%), respectively.
Scatterplot of paired oxygen saturation data pair values comparing the reference wired oximeter (y-axis) and wireless oximeter. Data points were jittered to avoid superimposing identical data pairs. 174 data pairs from 30 term neonates.
Scatterplot of paired oxygen saturation data pair values comparing the reference wired oximeter (y-axis) and wireless oximeter. Data points were jittered to avoid superimposing identical data pairs. 625 data pairs from 20 preterm neonates.
Bland-Altman plot of oxygen saturation (%). The bias is zero, with confidence limits of ±1.96×SD. Data points were jittered to avoid superimposing identical data pairs. 174 data pairs from 30 full term neonates.
Bland-Altman plot of oxygen saturation (%). The bias is zero, with confidence limits of ±1.96×SD. Data points were jittered to avoid superimposing identical data pairs.625 data pairs from 20 preterm neonates.
We have compared the accuracy of a reference wired oximeter to a wireless pulse oximeter for HR and SpO2 in this proof-of-concept study in term infants in the newborn nursery and preterm neonates in the NICU. When compared with the reference oximeter, the wireless oximeter showed a high degree of agreement for HR in both full term and preterm neonates, although no HR values < 90 bpm occurred in either cohort. For SpO2, the wireless oximeter showed a lesser degree of correlation in the full term cohort compared to the preterm cohort. In the preterm cohort, there was a moderate degree of correlation between the reference and wireless oximeters for SpO2 values.
Although wired pulse oximetry is the current standard of care for monitoring oxygen saturation in the NICU and well-baby nursery, validation of a wireless pulse oximeter for neonatal use has the potential to reduce wire burden during routine cares and improve maternal and infant bonding during skin to skin. Chung et al reported the use of a battery powered, wireless chest unit in a pilot study of 50 patients in both the NICU and PICU setting. They found that the average differences between the wireless oximeter compared with a reference wired oximeter (Phillips™) for HR and SpO2 was ≤±5 bpm for HR and < 3.5% for SpO2 in a sample of 20 patients. Both of these were within the guideline set by the US Food and Drug Administration (FDA) when comparing pulse oximeters [5]. Paul et al conducted a quality improvement project to develop an intervention to reduce the number of sudden unexpected postnatal collapse (SUPC) events in the delivery room during skin to skin care. A wired pulse oximeter was placed within 10 minutes of life with the primary outcome being subsequent cases of SUPC. The incidence of SUPC went from 5 cases to 0 cases over a 17 month period. However, 15% of mothers reported using a wired oximeter interfered with skin-to-skin care, and 58% of nurses reported that the wired oximeter took away from other patient care responsibilities [3].
This is the largest study in neonates comparing a wireless oximeter to a reference wired oximeter. One other trial compared the Masimo™ Radical-7 wired pulse oximeter to two wireless oximeters, the Owlet Smart Sock™ and the Baby Vida™ [9]. In that report, the Owlet Smart Sock™ had fair sensitivity and specificity for hypoxemia, but the sample size was small (30 infants) and only ten randomly selected, non-blinded data pairs per subject were used for analysis. Also, the extent of clinically significant discrepancies was unclear.
We chose to evaluate the Owlet Smart Sock™ instead of the Baby Vida™ because the Owlet Smart Sock™ appeared to have the largest social media following and was featured on several mainstream media platforms compared to the Baby Vida™.
We chose the Nonin 8500M™ as the reference pulse oximeter based on published data showing clinical equivalency to other pulse oximeters in routine clinical use [6, 7]. Singh et al compared the Nonin™ to the Phillips™ oximeter in a prospective cohort study in adult patients assessing oximeter bias between SpO2 and arterial oxygen saturation (SaO2). They found that the Nonin™ had less bias (degree of difference between SpO2 and SaO2) compared to the Phillips™ and displayed better sensitivity and positive predictive value than the Phillips™ oximeter [6]. Dixon et al compared the Nonin™ to the Masimo™ oximeter in shivering, post-operative adult patients to assess the degree of signal dropout and SpO2 and HR detection. Although the Masimo™ displayed more dropouts and a longer average dropout duration than the Nonin™, they concluded that there was no statistically significant difference between the two oximeters [7]. Although the Nonin™ showed comparability to Masimo™ and Phillips™ in adult patients, the applicability of these data to infants is unclear.
As of October 2021, Owlet™ Inc. has since discontinued marketing and selling the Owlet Smart Sock™ after the FDA issued a warning letter notifying the company that the product did not have premarket approval as a medical device [10].
Limitations
Our study has several limitations. First, we only monitored term and preterm neonates for 20 and 60 minutes, respectively, which limited the amount of data collected and hence the opportunity to capture clinically significant events. Since no bradycardic (HR < 90 bpm) events occurred in either the term or preterm infant cohort, we were unable to assess accuracy and reliability of the wireless oximeter in detecting clinically relevant bradycardia. Also, there were few O2 saturation events ≤90% in the term cohort, which limited the ability to assess accuracy of the wireless compared to reference wired oximeter for detecting hypoxemia. Second, the averaging times for both oximeters were different (4 seconds for the reference, 12 seconds for the wireless) which may have led to discrepancies in statistical calculations. We mitigated this by selecting, time-matched data pairs from recorded data as mentioned above for the final statistical analysis. Third, although the sock fit preterm infants weighing less than the manufacturer’s recommended minimum weight of 2700 grams, a less-than-optimal fitting sock may have adversely affected accuracy of wireless oximeter SpO2 detection and hence correlation with the wired reference oximeter. Of note, however, this sock size limitation did not affect HR correlations with the reference oximeter, and all HR values were in the normal range in both cohorts. Fourth, we modified the method for data acquisition with the wireless oximeter; we used a laptop rather than the base station recommended for home use by the manufacturer in order to facilitate direct comparison and analysis of the reference and wireless data pairs for HR and SpO2. Our results may thus not be generalizable to home use of this wireless pulse oximeter, but the purpose of this proof-of-concept pilot study was to evaluate the potential role of wireless pulse oximetry in hospitalized neonates, not to validate the off-the-shelf wireless oximeter including the base station for data acquisition and transmittal from home. Fifth, we only assessed one reference oximeter and one wireless oximeter, and our results may not be generalizable to other pulse oximeters. We also used a wired oximeter as the reference standard for oxygenation, as opposed to arterial PaO2 which is the gold standard. Finally, this is a single-site study, so our results may not be generalizable to other clinical sites and different infant or neonatal cohorts.
Conclusion
Compared with the reference wired pulse oximeter, the wireless oximeter showed no significant clinical differences for HR, but no bradycardic events occurred with either oximeter. Overall accuracy for SpO2 was moderate, but was limited by the paucity of SpO2 values in the hypoxic range and potentially further limited in the preterm cohort by lack of an appropriately sized sock for smaller preterm infants. Although these results are consistent with an inpatient role for wireless pulse oximetry in neonates in the newborn nursery and NICU, larger studies are needed with study designs sufficient to capture clinically significant bradycardia and desaturation events.
Footnotes
Acknowledgments
We thank Brian Robertson, MD and Teotimo Andrada, MS (Department of Sleep Medicine, Walter Reed National Military Medical Center, Bethesda, MD, USA) and Jacob Collen, MD (Department of Medicine, Walter Reed National Military Medical Center, Bethesda, MD, USA) for providing the reference oximeter and supporting its use. We thank Owlet Care Inc.™ for loaning the wireless oximeter and laptop used in this study.
Disclosures
The authors have no conflicts of interest to disclose. The views expressed in the article are those of the authors and do not reflect the official policy of Uniformed Services University, the Department of Defense or the United States Government.
Owlet Care Inc.™ was not involved in study design or conduct, data analysis, manuscript preparation or review, or in final approval of the manuscript for submission, and no author has any financial or advisory relationship with Owlet.
Human research statement
We affirm that this research submitted to the journal was conducted in accordance with the ethical standards of all applicable national and institutional committees and the World Medical Association’s Helsinki Declaration. Our study was approved by the Walter Reed National Military Medical Center IRB, and informed parental consent was obtained prior to study enrollment.