Validation of a Standalone Smartphone Application for Measuring Heart Rate Using Imaging Photoplethysmography

Abstract

Background:

The aim of this study was to determine the accuracy of a freely available smartphone application, Cardiio app (Cardiio, Inc., Cambridge, MA), to measure heart rate from the finger or face using imaging photoplethysmography, by comparing against an FDA-cleared pulse oximeter at rest, and after moderate to vigorous exercise.

Methods:

A total of 40 healthy adults participated in this study. Participants engaged in a period of rest, followed by 3 min of moderate to vigorous intensity cycling on a stationary bicycle. Heart rate measurements were obtained from both the finger and face of participants using the Cardiio app at rest, immediately after exercise, 1-2 min after exercise, and 2–3 min after exercise. Concurrent heart rate readings using an FDA-cleared finger pulse oximeter served as the reference measurement.

Results:

There was a very strong agreement between heart rate measurements obtained using the Cardiio app and the pulse oximeter, both at rest (r = 0.99 for finger, r = 0.97 for face) and after exercise (r = 0.99 for finger, r = 0.97 for face). At rest, the accuracy of the Cardiio app was ±1.58 beats per minute (bpm) (or ±2.27%) using the finger mode and ±2.28 bpm (or ±3.17%) for the face mode, compared to the pulse oximeter. After moderate to vigorous exercise, the accuracy of the Cardiio app was ±2.97 bpm (or ±2.79%) using the finger mode and ±5.31 bpm (or ±4.50%) for the face mode, compared to the pulse oximeter.

Conclusion:

The Cardiio app provided accurate heart rate measurements from the finger and face, both at rest and after exercise.

Introduction

Wearable sensors, smartphones, and mobile applications (apps) are poised to play an important role in the emergence of a new health economy that is decentralized and focused on the consumer, convenience, and prevention.¹ For example, smartphone apps can provide on-the-spot measurements of heart rate and breathing rate using the built-in camera^2,3 or accelerometer.⁴ A user's lung function can also be measured using a smartphone-based spirometer that leverages the built-in microphone.⁵ Recent work has demonstrated that smartphone apps can also be used to detect the presence of atrial fibrillation, the most commonly sustained arrhythmia and a leading cause of stroke.^6,7

The underlying principal of acquiring the cardiovascular signal for many of the standalone smartphone apps that measure heart rate is known as imaging photoplethysmography (PPG). PPG is a noninvasive optical technique for measuring the volumetric changes in the blood vessels beneath the skin introduced in the 1930s.⁸ Conventional PPG utilizes a light-emitting diode to illuminate a person's skin at the measurement site and a photodiode to measure the changes in light absorption to produce a pulsatile waveform that corresponds to the timing of each heartbeat.⁹ The use of a mobile phone camera and flash to perform PPG using contact-based video imaging was first reported in 2010.^2,10 In this arrangement, a user places a finger over the rear camera, the built-in flash is used to illuminate the finger tissue, and the amount of light reflected from the finger skin is captured by the camera. Spatial averaging of the incoming video frames produces a PPG pulse waveform. Alternatively, it is also possible to obtain contact-free PPG measurements from a distance. The feasibility of using a webcam to extract the PPG pulse waveform remotely from a person's face was demonstrated by Poh et al. at MIT.^11,12 Instead of using a dedicated light source, this approach relies on ambient lighting to provide illumination. Collectively, these techniques based on principals of PPG but using a camera to measure the variations in transmitted or reflected light through video imaging are now called imaging PPG.¹³

As the number of health apps made available to the public increases, there is a need for more validation studies to shed light on the performance of these apps. The vast majority of developers of smartphone apps that measure heart rate have not published information regarding accuracy. Previously, we developed a freely available smartphone app, Cardiio (Cardiio, Inc., Cambridge, MA), that provides both contact-based (from a user's finger) and contact-free (from a user's face) heart rate measurements based on imaging PPG.¹⁴ The aim of this study was to determine the accuracy of the Cardiio smartphone app by comparing heart rate readings against an FDA-cleared pulse oximeter at rest and after moderate to vigorous exercise.

Methods

Participants

A total of 40 adult volunteers within the community of Cambridge, MA voluntarily participated in this prospective study. The study was conducted at the facilities of Cardiio, Inc. The age, gender, and skin color of each participant was recorded. Skin color was quantified based on the Fitzpatrick classification scale, which ranges from type 1 (lightest) to type 6 (darkest).¹⁵ Individuals with known heart conditions (e.g., tachycardia, atrial fibrillation, and arrhythmia) or pacemakers were excluded from the study. The 40 participants were diverse in age (age range: 22–55 years, mean ± standard deviation [SD]: 29.52 ± 6.14 years), gender (21 male, 19 female), and Fitzpatrick skin color (range of Fitzpatrick levels: type 1–6, mean ± SD: 3.18 ± 1.28). Six participants (15%) were of dark complexion (Fitzpatrick type 5 and 6).

Heart Rate Measurements

The “Cardiio-Heart Rate Monitor +7 Minute Workout Exercise Routine for Cardio Health and Fitness” (Cardiio) iOS app¹⁴ (version 3.3.1) was installed on an iPhone 6S (Apple, Inc.). Both the finger and face modes available on the Cardiio app for taking heart rate measurements were evaluated in this study. When using the finger mode, participants were asked to hold the iPhone in their right hand and to gently cover the back camera with a finger. In the face mode, participants were asked to look straight ahead and hold the iPhone in front of their face. Participants were instructed to remain still and hold the iPhone steady throughout the measurement process. Each Cardiio app measurement took ∼20 s to complete and save.

The reference device used to compare accuracy was an FDA-cleared finger pulse oximeter (K081285), the Nonin Onyx II Model 9560 (Nonin Medical, Inc.). The pulse oximeter was placed on the left index finger of the participants. The pulse oximeter wirelessly transmitted (using Bluetooth) time-stamped heart rate readings at a frequency of 1 Hz, which were recorded on a laptop.

Study Protocol

During the baseline period, participants were asked to sit and relax for 1 min on the seat of a stationary exercise bicycle (Exerpeutic; Paradigm Health and Wellness, Inc.). Resting heart rate measurements were then obtained from the Cardiio app using both the finger and face mode. Concurrent heart rate measurements from the pulse oximeter were recorded on the laptop. Next, participants engaged in moderate to vigorous intensity cycling on the exercise bike for 3 min. Participants were instructed to increase their cycling speed and resistance level after the first minute and again after the second minute. For the final 30 s of exercise, participants were asked to go all-out and cycle as fast as they could at the highest resistance setting they could tolerate. At the end of the exercise period, heart rate was immediately measured using both the finger and face mode of the Cardiio app. Heart rate measurements (finger and face) were obtained again around 1–2 min after exercise and 2–3 min after exercise. Concurrent heart rate measurements from the pulse oximeter were recorded on the laptop. Measurements at these different time points were designed to capture a wide range of heart rates that reflect different exercise intensity levels.

Statistical Analysis

Heart rate measurements obtained from the Cardiio app were exported as an Excel (CSV) file and compared against the closest time-stamped pulse oximeter readings. Pearson's correlation coefficients and the corresponding p-values were calculated to provide an indication of the relationship between the recorded heart rates from the Cardiio app and pulse oximeter. Bland–Altman plots¹⁶ were used for combined graphical and statistical interpretation of the two measurement techniques. The differences between heart rate measurements from the Cardiio app and the pulse oximeter were expressed as beats per minute (bpm) and plotted against the averages. The mean and SD of the differences, mean of the absolute differences, and 95% limits of agreement were calculated. Accuracy was determined by calculating the root-mean-squared-error (RMSE) of the heart rate measurements by the Cardiio app, HR_CardiioHRM , against reference measurements by the pulse oximeter, HR_PulseOx , which accounts for both bias and precision. RMSE was calculated both in terms of bpm and as a percentage difference from HR_PulseOx . \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\usepackage{upgreek}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}& {\rm {RMSE}} \ ( {\rm {bpm}} ) = \sqrt \frac {{{{{{ \sum \nolimits_{k = 1}^n { \left( {H{R_{Cardiio}} \left[ k \right] - H{R_{PulseOx}} \left[ k \right] } \right) } }^2}}}}}{{n}} , \ \\ & { \rm{RMSE}} \ ( { \rm{ \% }} ) = \sqrt \frac {{{{{{ \sum \nolimits_{k= 1}^n { \left( {{{ \left( {H{R_{Cardiio}} \left[ k \right] - H{R_{PulseOx}} \left[ k \right] } \right) } \over {H{R_{PulseOx}} \left[ k \right] }} \times 100 \% } \right) } }^2}}}}}{{ n}}\end{align*} \end{document}

Results

Heart Rate Measurements at Rest

The distribution of heart rates measured at rest, based on the minimum, 25th percentile, median, 75th percentile, and maximum values, is shown as a box plot in Figure 1. The mean heart rate captured at rest was 71.31 ± 10.00 bpm (range: 45–99 bpm).

Fig. 1.

Heart rate levels for the four measurement time points in the study protocol.

Figure 2 shows the correlation plots for the finger (Fig. 2A) and face modes (Fig. 2B) of the Cardiio app for heart rate measurements taken at rest (n = 40 pairs each). There was a significant (p < 0.001) and very strong correlation between heart rate measurements using the pulse oximeter and corresponding heart rate measurements using the finger mode (r = 0.99) and also between heart rate measurements using the pulse oximeter and corresponding heart rate measurements using the face mode (r = 0.97).

Fig. 2.

Scatter plots comparing heart rate measurements at rest between the pulse oximeter and Cardiio app using the (A) finger mode and (B) face mode.

The differences between heart rate measurements at rest recorded by the Cardiio app and pulse oximeter from all participants under the finger and face modes are displayed in Bland–Altman plots (Fig. 3A, B, respectively). The limits of agreement (±1.96SD) reflect where 95% of all differences between measurements are expected to lie. For the finger mode, the mean bias was 0.30 ± 1.57 bpm with 95% limits of agreement from −2.78 to 3.38 bpm; the mean absolute bias was 1.30 ± 0.91 bpm. For the face mode, the mean bias was 0.15 ± 2.30 bpm with 95% limits of agreement of −4.37 to 4.67 bpm; the mean absolute bias was 1.55 ± 1.69 bpm. At rest, the RMSE of the Cardiio app was 1.58 bpm (2.27%) for the finger mode and 2.28 bpm (3.17%) for the face mode.

Fig. 3.

Bland–Altman plots demonstrating the agreement between heart rate measurements at rest from the pulse oximeter and Cardiio app using the (A) finger mode and (B) face mode. The dash-dot lines represent the mean and 95% limits of agreement (±1.96 SD). SD, standard deviation.

Heart Rate Measurements After Exercise

Box plots of heart rates recorded immediately after, 1–2 min after, and 2–3 min after exercise are displayed in Figure 1. The mean heart rate recorded immediately after exercise was 136.35 ± 22.44 bpm (range: 61–178 bpm); mean heart rate 1–2 min after exercise was 109.04 ± 16.41 bpm (range: 64–135 bpm); and mean heart rate 2–3 min after exercise was 102.03 ± 14.12 bpm (range: 63–130 bpm). Overall, the mean postexercise heart rate across all measurements was 115.89 ± 23.31 bpm (range: 61–178 bpm). Applying the conventional formula for measuring maximum heart rate, HR _max = 220-age, the heart rates recorded reflected a wide range of intensity levels, including both vigorous-intensity exercise levels (70–85% HR _max) and moderate-intensity exercise levels (50–70% HR _max).

Figure 4 shows the correlation plots for the finger (Fig. 4A) and face modes (Fig. 4B) of the Cardiio app for heart rate measurements taken after exercise (n = 118 pairs for finger mode, n = 117 pairs for face mode). Five postexercise measurement pairs (two in finger mode, three in face mode) were incomplete and excluded due to an inability to compute heart rate by the pulse oximeter (two instances), an inability to compute heart rate by the Cardiio app (one instance), and loss of pulse oximeter data through the Bluetooth connection (two instances). There was a significant (p < 0.001) and very strong correlation between heart rate measurements using the pulse oximeter and corresponding heart rate measurements using the finger mode (r = 0.99) and also between the heart rate measurements using the pulse oximeter and corresponding heart rate measurements using the face mode (r = 0.97).

Fig. 4.

Scatter plots comparing heart rate measurements after exercise between the pulse oximeter and Cardiio app using the (A) finger mode and (B) face mode.

Figure 5A and B shows the differences between heart rate measurements after exercise recorded by the Cardiio app and pulse oximeter from all participants under the finger and face modes, respectively, in Bland–Altman plots. For the finger mode, the mean bias was −0.03 ± 2.98 bpm with 95% limits of agreement of −5.88 to 5.81 bpm; the mean absolute bias was 1.61 ± 2.50 bpm. For the face mode, the mean bias was −0.49 ± 5.31 bpm with 95% limits of agreement of −10.89 to 9.91 bpm; the mean absolute bias was 1.94 ± 4.96 bpm. After exercise, the RMSE of the Cardiio app was 2.97 bpm (2.79%) for the finger mode and 5.31 bpm (4.50%) for the face mode.

Fig. 5.

Bland–Altman plots demonstrating the agreement between heart rate measurements after exercise from the pulse oximeter and Cardiio app using the (A) finger mode and (B) face mode. The dotted lines represent the mean and 95% limits of agreement (±1.96 SD).

Discussion

Previously, the accuracy of heart rate measurements of the Cardiio app was reported to be 97.6–99.2% with respect to a clinical pulse oximeter.¹⁷ A summary of the results from the present study is presented in Table 1. The results indicate that there is a very strong agreement between heart rate measurements obtained using the Cardiio app and a validated pulse oximeter, both at rest (r = 0.99 for finger, r = 0.97 for face) and after exercise (r = 0.99 for finger, r = 0.97 for face). At rest, the accuracy (RMSE) of the Cardiio app was ±1.58 bpm (or ±2.27%) using the finger mode and ±2.28 bpm (or ±3.17%) for the face mode, compared to a pulse oximeter. These results are consistent with the findings of the original studies describing heart rate measurements at rest obtained from the face using a webcam.^11,12 After moderate and strenuous exercise, the accuracy of the Cardiio app was ±2.97 bpm (or ±2.79%) using the finger mode and ±5.31 bpm (or ±4.50%) for the face mode, compared to a pulse oximeter. The slight increase in error was likely due to motion artifacts that affected measurement accuracy. For example, we observed that two measurement outliers using the face mode were obtained from participants who were breathing heavily after exercise and were not holding still.

Table 1.

Comparison of Heart Rate Measurements Using the Cardiio App and a Pulse Oximeter

	AT REST		AFTER EXERCISE
STATISTIC	FINGER	FACE	FINGER	FACE
Mean bias (bpm ± SD)	0.30 ± 1.57	0.15 ± 2.30	−0.03 ± 2.98	−0.49 ± 5.31
Mean absolute bias (bpm ± SD)	1.30 ± 0.91	1.55 ± 1.69	1.61 ± 2.50	1.94 ± 4.96
Correlation coefficient, r	0.99^a	0.97^a	0.99^a	0.97^a
RMSE (bpm)	1.58	2.28	2.97	5.31
RMSE (%)	2.27	3.17	2.79	4.50

Indicates significance at p < 0.001.

bpm, beats per minute; RMSE, root-mean-squared-error; SD, standard deviation.

A limitation of this study is that we did not evaluate the accuracy of heart rate measurements on people with arrhythmias or pacemakers. Arrhythmias that are accompanied by pulse deficits may affect the accuracy of PPG-based heart rate measurements¹⁸ and remain as a topic for future investigations. As the Cardiio app is not cleared by the FDA, it is not intended to be used as a medical device.

The American National Standards Institute (ANSI)/Association for the Advancement of Medical Instrumentation (AAMI) standard establishes the minimum performance requirements for electrocardiograph (ECG) heart rate monitors. Based on ANSI/AAMI EC13:2002, the accuracy requirements for ECG heart rate measurements are up to ±5 bpm or ±10%, whichever is greater.¹⁹ Although the Cardiio app is not a medical device or subject to the ANSI/AAMI standards,²⁰ the accuracy of the app falls well within these specifications. These results do not imply, however, that the app should be used in place of a pulse oximeter or an ECG heart rate monitor.

The ability to acquire cardiovascular signals and accurately measure heart rate using a smartphone has the potential to augment current telemedicine apps that provide video-based consultations by aiding in diagnosis of certain conditions. Furthermore, convenient and on-demand monitoring of vital signs such as heart rate may contribute to better management of patients with chronic diseases, especially those who have limited access to medical care. Nonetheless, to fully realize the promise improving healthcare, health apps that are intended for use in the diagnosis of disease or other conditions will need to obtain clearances from the appropriate regulatory bodies, such as the FDA. As the influence of health apps on telemedicine and e-health is likely to grow significantly over the next few years, we anticipate an increase in the number of peer-reviewed validation studies of such health apps in the near future as a step toward gaining regulatory approval and clinical acceptance.

Conclusion

The Cardiio app provides accurate heart rate measurements from the finger and face, both at rest and after exercise. Users should remain still during measurements as motion may affect the performance of the app.

Footnotes

Disclosure Statement

Ming-Zher Poh and Yukkee C. Poh are cofounders and employees of Cardiio, Inc., and have an ownership stake in the company. There are no other potential conflicts of interest relevant to this study.

References

PWC HRI. New Health Economy Healthcare's new entrants: Who will be the industry's Amazon.com? . 2014. Available at http://pwchealth.com/cgi-local/hregister.cgi/reg/pwc-hri-new-entrants.pdf (last accessed October 17, 2016 ).

Jonathan

, Leahy

. Investigating a smartphone imaging unit for photoplethysmography. Physiol Meas, 2010; 31:N79.

Scully

, Lee

, Meyer

, et al. Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Trans Biomed Eng, 2012; 59:303–306.

Hernandez

, McDuff

, Picard

. Biophone: Physiology monitoring from peripheral smartphone motions. Paper presented at 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Millan, Italy, 2015.

Larson

, Goel

, Boriello

, Heltshe

, Rosenfeld

, Patel

. SpiroSmart: Using a microphone to measure lung function on a mobile phone. Paper Presented at Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, PA, 2012.

McManus

, Lee

, Maitas

, et al. A novel application for the detection of an irregular pulse using an iPhone 4S in patients with atrial fibrillation. Heart Rhythm, 2013; 10:315–319.

Chan

, Wong

, Poh

, et al. Diagnostic performance of a smartphone-based photoplethysmographic application for atrial fibrillation screening in a primary care setting. J Am Heart Assoc, 2016; 5:e003428.

Hertzman

. Photoelectric plethysmography of the fingers and toes in man. Exp Biol Med, 1937; 37:529–534.

Allen

Photoplethysmography and its application in clinical physiological measurement. Physiol Meas, 2007; 28:R1.

10.

Pelegris

, Banitsas

, Orbach

, Marias

A novel method to detect heart beat rate using a mobile phone. Paper Presented at Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE, Buenos Aires, Argentina, 2010.

11.

Poh

M-Z

, McDuff

, Picard

. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express, 2010; 18:10762–10774.

12.

Poh

M-Z

, McDuff

, Picard

. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans Biomed Eng, 2011; 58:7–11.

13.

Sun

, Thakor

. Photoplethysmography revisited: From contact to noncontact, from point to imaging. IEEE Trans Biomed Eng, 2016; 63:463–477.

14.

Cardiio—Heart Rate Monitor +7 Minute Workout Exercise Routine for Cardio Health and Fitness. Available at https://itunes.apple.com/us/app/cardiio-touchless-camera-pulse/id542891434 (last accessed July 12, 2016 ).

15.

Fitzpatrick

. The validity and practicality of sun-reactive skin types I through VI. Arch Dermatol, 1988; 124:869–871.

16.

Bland

, Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, 1986; 1:307–310.

17.

El-Amrawy

, Nounou

. Are currently available wearable devices for activity tracking and heart rate monitoring accurate, precise, and medically beneficial?. Healthc Inform Res, 2015; 21:315–320.

18.

Sneed

, Hollerbach

. Accuracy of heart rate assessment in atrial fibrillation. Heart Lung, 1991; 21:427–433.

19.

Cardiac Monitors, Heart Rate Meters, and Alarms. American National Standard (ANSI/AAMI EC13: 2002), Association for the Advancement of Medical Instrumentation, Arlington, VA, 2002:1–87.

20.

General Wellness. Policy for Low Risk Devices—Guidance for Industry and Food and Drug Administration Staff. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Devices and Radiological Health. 2016.