Validation of a New Heart Rate Measurement Algorithm for Fingertip Recording of Video Signals with Smartphones

Abstract

Introduction: This study investigates the accuracy of a heart rate (HR) measurement algorithm applied to a pulse wave. This was based on video signals recorded with a smartphone. The results of electrocardiographic HR and standard linear heart rate variability (HRV) analysis were used for reference. Materials and Methods: On a total of 68 subjects, an electrocardiogram (ECG) and the pulse curve were simultaneously recorded on an Apple iPhone 4S. The HR was measured using an algorithm developed by the authors that works according to a method combining the detection of the steepest slope of every pulse wave with the correlation to an optimized pulse wave pattern. Results: The results of the HR measured by pulse curves were extremely consistent (R > 0.99) with the HR measured on ECGs. For most standard linear HRV parameters as well, high correlations of R ≥ 0.90 in the analysis were achieved in the time and frequency domain. Conclusion: In conclusion, the overall accuracy of HR and HRV indices of pulse wave analysis, based on video signals of a smartphone, with the developed algorithm was sufficient for preclinical screening applications.

Introduction

With today's processing power and sensor technology, smartphones have the potential to function as mobile diagnostic and monitoring systems for both medical and lifestyle and fitness applications. The use of smartphones for recording vital parameters through existing sensors enables mobile diagnostics at home without the need for additional hardware or accessories. With modern signal analysis methods, a pulse curve can be derived from a video signal recorded on the user's fingertip by the smartphone's camera. This pulse curve can supply data on cardiovascular status. The heart rate variability (HRV) is an essential parameter for detecting regulatory disorders of the autonomic nervous system.^1
–3 These can manifest as stress-related diseases such as burnout syndrome,^4,5 diabetes,⁶ cardiovascular diseases,⁷ or depression.⁸ Nowadays, such lifestyle-related conditions are becoming even more prevalent, often with fatal consequences. Early detection by HRV measurement followed by appropriate therapeutic interventions can help to diminish the effects of these diseases. Smartphone-based solutions for recording vital parameters make early preclinical risk screening for lifestyle-related illnesses fast and easy. Therefore, this study investigated a proprietary algorithm developed by the authors for measuring heart rate (HR) based on iPhone video signals.

According to the guidelines issued by the Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology in 1996, the beat-to-beat variability is determined from an electrocardiogram (ECG) over less than 30 min for short-term HRV analysis.¹ In the present study, the HRV was calculated from the video signal obtained from a fingertip pressed against a smartphone camera for 5 min. The pulse wave extracted from the video signal reflects the blood pulsation caused by cardiac activity. Various studies have previously shown that the HRV from ECG recordings and the pulse rate variability based on photoplethysmographic data correlate by up to 0.99.^9
–11 The aim of these investigations was to verify the accuracy of smartphone-derived HRV parameter analysis with the developed algorithm compared to HRV parameter analysis from ECG recordings. The tachogram forms the basis for HRV analysis. Therefore, it is imperative to also compare the HR determined from the smartphone video signal and the ECG alongside the conventional HRV parameters.

Data and Methods

Selection and Description of Study Participants

In total, 68 women (n = 28) and men (n = 40) were enrolled in this study. The study groups consist of 45 patients from a cardiologic outpatient ambulance and 23 healthy controls. The group of controls was recruited from the Ernst-Abbe University of Applied Sciences in Jena. ECGs and pulse curves were recorded simultaneously with an Apple iPhone 4S on all of these subjects. Subject characteristics are summarized in Table 1. All subjects were informed about their participation in this study and provided their informed written consent to participate. The study was conducted in compliance with the Declaration of Helsinki and approved by a local ethics committee.

Table 1.

Number of Subjects, Sex, Age, Height, Weight, and BMI (Mean ± Standard Deviation)

Number	68
No. of men/women	40/28
Age, years	51.7 ± 18.83
Height, cm	172.8 ± 9.87
Weight, kg	80.1 ± 15.72
BMI, kg/m²	26.8 ± 4.84

BMI, body mass index.

Data Recording and Processing

Data acquisition

ECG recordings were taken at rest for a period of 5 min (patients: Cardiologic Explorer; enverdis GmbH, Jena, Germany, sampling frequency 1116 Hz; controls: Portapres TNO Biomedical Instrumentation, Amsterdam, Netherlands, sample frequency 1600 Hz). The pulse curve was recorded simultaneously being measured on the Apple iPhone 4S (Apple, Inc., Cupertino CA). To obtain the pulse curve, all subjects were asked to press their fingertip against the camera lens of an iPhone 4S (sample frequency ∼30 Hz) and switch the flash to continual operation.

Further on, a 2-min measurement after 3 min of physical exercise was recorded from controls.

Data processing

To improve the quality of the ECG signals, a median filter was used. After R peak detection, the tachogram was calculated from the measured RR intervals. For this purpose, the consecutive RR distances were plotted against the number of beats.

The pulse curve was derived from the green channel of the recorded video signal. The signals were filtered with a band-pass filter with a lower cutoff frequency of 0.5 Hz and an upper cutoff frequency of 7 Hz. Unlike conventional methods for determining pulse rate (simple peaks, foot point, or inflection point determination),¹⁰ a combination of detection of the steepest slope of each pulse wave and a correlation method with a pulse wave pattern was used. Initially, the inflection points of the increasing flank are detected for each heart period. These represent the first possible candidate (A) for measuring the HR. To increase the robustness of the method, a second way to find periods was applied. The original signal was correlated with a pattern of a pulse wave. The positive peaks of the cross-correlation correspond to the points with the best agreement between pattern and original, representing additionally possible candidates (B) for the positions of the individual cardiac cycles. The resulting cardiac cycles were determined from candidates A and B by applying the nearest neighbor method with a defined time frame for the tachogram (Fig. 1).

Fig. 1.

Pulse curve derived from the video signal of an iPhone 4S. The circles mark the detected points in the steepest slope in each pulse wave. Original pulse wave solid line, filtered pulse wave dashed line.

Parameters of Short-Term HRV

For HRV parameter analysis, only NN intervals were considered, that is, only the intervals between QRS complexes, formed by depolarization of the sinus node.¹ Extrasystoles and other disruptions were eliminated and the corresponding points on the tachogram replaced using an algorithm for adaptive variance estimation.¹² In 13 recordings, the proportion of extrasystoles and other disturbances was larger than 5%. These were not considered in the HRV analysis. The HRV analysis was conducted with two additional detection methods (peak and foot point detection) to investigate the robustness of the algorithm for pulse wave detection used here.

The standard parameters of the HRV analysis in the time and frequency range were calculated similarly to the Task Force at rest over 5 min (Tables 2 and 3).¹ The Fast Fourier Transformation with a Blackman–Harris window was used for the spectral analysis.

Table 2.

Selected HRV Parameters in the Time Domain

PARAMETER	UNIT	DESCRIPTION
sdNN	ms	Standard deviation of the NN intervals
rmssd	ms	Mean squared differences of consecutive NN intervals
norm rmssd	n.u.	Rmssd normalized to the length of the tachogram
Shannon entropy	bit	Describes the variability of the observed values in bits

HRV, heart rate variability.

Table 3.

Selected HRV Parameters in the Frequency Domain

PARAMETER	UNIT	DESCRIPTION
LF	ms²	Power in the low-frequency range; 0.04–0.15 Hz
HF	ms²	Power in the high-frequency range; 0.15–0.4 Hz
Total power	ms²	Variance of all NN intervals; ≤0.4 Hz
LFn	n.u.	Normalized low-frequency power
HFn	n.u.	Normalized high-frequency power

Statistical Analysis

The standard method for assessing agreement between the two measurement techniques is a Bland–Altman plot.¹³ The difference in the values measured by the two techniques, or their ratio, is plotted against the mean of the measured values of both measurement techniques. Another measure for the agreement of methods is their correlation and the related correlation coefficient. Determination of the correlation coefficient and the Bland–Altman plot are both conventional methods for comparing two medical diagnostic methods.^9,10,14 Statistical analysis was conducted with MATLAB (The MathWorks, Inc, Natick, MA).

Results

To validate the accuracy of the HR from the iPhone data compared to the ECG, randomly chosen 10-s intervals were included in further analysis. To this extent, an effort was made to only concentrate on even coverage of HR range and to only use intervals with a relatively constant HR. This resulted in 80 intervals covering a range of 47–135 bpm. The correlation between iPhone 4S and ECG was a maximum at R > 0.99 (Fig. 2). The results of the correlation of the three different detection methods (A: proprietary algorithm, B: foot point, and C: peak detection) for the HRV parameters within the time and frequency range are summarized in Table 4. Figure 2 shows the plots of the comparison of HR and HRV parameters sdNN (time range) and HFn (frequency range). The HRV parameters sdNN, rmssd, Shannon entropy, LF, and total power show very high correlation (R ≥ 0.90). For the parameters norm rmssd, LFn, HF, and HFn, the correlation between the iPhone data and ECG recordings was slightly lower at 0.89 ≤ R ≤ 0.73.

Fig. 2.

Scatterplots with regression function and Bland–Altman plots of the determined HR (bpm; top) and HRV parameters sdNN (ms; center) and HFn (n.u.; bottom). HR, heart rate; HRV, heart rate variability.

Table 4.

Results on the Accuracy for Measuring Heart Rate and Selected HRV Parameters Through iPhone Compared to the ECG, Calculated Correlation Coefficients R According to Detection Method

METHOD	A	B	C	VALUE RANGE
HR	>0.99 (R + + +)			47–135 bpm
HRV parameters
Time domain
sdNN	0.98 (R + + +)	0.93 (R + + +)	0.93 (R + + +)	9.3–109.7 ms
rmssd	0.90 (R + +)	0.67 (R−)	0.77 (R +)	7.2–73.7 ms
norm rmssd	0.89 (R + +)	0.69 (R−)	0.80 (R + +)	0.008–0.094 n.u.
Shannon entropy	0.99 (R + + +)	0.98 (R + + +)	0.99 (R + + +)	2.27–5.59 bit
Frequency domain
LF	0.99 (R + + +)	0.96 (R + + +)	0.95 (R + + +)	8–9951 ms²
HF	0.88 (R + + +)	0.83 (R + +)	0.86 (R + + +)	11–3414 ms²
Total	0.98 (R + + +)	0.94 (R + + +)	0.93 (R + + +)	54–14609 ms²
LFn	0.73 (R +)	0.74 (R +)	0.73 (R +)	30.6–90.36 n.u.
HFn	0.74 (R +)	0.66 (R −)	0.67 (R −)	8.29–64.04 n.u.

A, determining a combination of pattern correlation and inflection points of the pulse curve; B, measuring the pulse wave peak; and C, determining foot point of the pulse curve, assessment: R + + + (≥0.9), R + + (0.8–0.9), R + (0.7–0.8), R − (<0.7).

ECG, electrocardiogram; HR, heart rate.

Discussion

The aim of this study was to investigate the accuracy of a developed algorithm for determining HR and HRV using a pulse wave based on video signals and recorded with a smartphone (here: iPhone 4S). Today, there are several applications on the market using smartphone cameras to extract HR and HRV.^15

–19 Investigations of Bolkhovsky et al.¹⁸ already showed that mobile phones are able to be an accurate monitor for vital parameters such as HR or HRV. Lenskiy and Aitzhan¹⁶ as well as Scully et al.¹⁷ use smartphone video recordings of fingertip for HRV calculation and compared the results with those extracted from ECG recordings. Their investigations showed a good agreement, but correlation results were not provided. Kwon et al.¹⁹ used facial videos recorded with the front camera of a smartphone for measuring HR and developed an iPhone application called FaceBEAT. In the current study, the results of the HR and HRV analysis from ECG recordings were also used as a reference. Over a range of 47–135 bpm, the HR from the iPhone data correlates very well with the HR from the ECG data with R > 0.99. Schäfer and Vagedes¹⁰ reviewed the summarized results of studies investigating the accuracy of pulse rate variability (PRV) as an estimate of HRV analysis by the photoplethysmographic and ECG technology. Likewise, they found a high agreement (R ≥ 0.99) between HR measured by photoplethysmography and ECG recordings.^20
–22 Furthermore, the investigation of Schäfer and Vagedes reveals¹⁰ that artifacts in the HR determination by plethysmograms have a greater influence than ECG recordings. This was also found in the present studies. In the Bland–Altman plot of HR, a slight increase can be identified based on the deviation of HR between ECG and pulse wave in the HR range of over 90 bpm (Fig. 2). The higher HRs were reached by measuring the controls after 3 min of physical exercise. Physical exercise induced several artifacts during the measurements.

The majority of the HRV parameters (sdNN, rmssd, Shannon entropy, LF, Total) produced a sufficiently high agreement between iPhone 4S and ECG at R ≥ 0.90. A lower correlation was achieved in the parameters HF, HFn, LFn, and norm rmssd. For the parameters HF, HFn, rmssd, and norm rmssd, mainly short-term beat-to-beat fluctuations were accounted for in the calculations, which is why even minor deviations between the tachograms have a great impact on the correlation. Due to error propagation, the normalization of the parameters LFn and HFn leads to an increase in the deviation of the iPhone and ECG data. This explains the differences in the correlation coefficients of LF (R = 0.99) and HF (R = 0.88) as opposed to LFn (R = 0.73) and HFn (R = 0.74). In the studies reviewed by Schäfer and Vagedes,¹⁰ correlation values of R ≥ 0.9 were achieved for all HRV parameters when comparing HRV analysis by plethysmography with ECG recordings. The fact that some HRV parameters reached correlation values of R < 0.90 in the results presented here is likely attributable to the low and variable sample frequency of the video signals of ∼30 Hz. As a result, a slightly lower signal pulse wave quality (compared to photoplethysmography) recorded with constant sample frequency can lead to a certain imprecision in signal analysis. Table 4 clearly shows that the algorithm (A) with inflection point detection in combination with the correlation method works more reliably than the simple methods of peak (B) and foot point detection (C). The correlation coefficient of the detection method (A) developed by the authors is located above those of the peak and foot point detection in all HRV parameters, except Shannon entropy and LFn, where equal values were achieved. Peng et al.²³ also investigated different detection methods to determine HR and HRV with smartphone camera signals and used ECG recordings as reference. They also recognized that the results of the inflection point detection are more accurate than the results of maxima or the minima detection.²³ In comparison, in this study, a combination of the inflection point and a correlation method with a pulse wave pattern was used to measure HR. The parameters HR, sdNN, rmssd, and LF achieved better or equal results in this study, whereas the parameters of the frequency domain total power, HF, LFn, and HFn achieved less correlation.²³ The reason for different results could be the difference of study participants. Their study population included only young and healthy subjects. In the present investigation, the study group consisted of healthy subjects as well as patients of a cardiologic praxis with a wide range of age. Also, Peng et al. used an autoregressive model,²³ whereas in the current study, a Fast Fourier Transformation with a Blackman–Harris window was used for the frequency parameter extraction.

The HRV analysis using smartphone data is sufficiently precise and enables its application in everyday life, to monitor ones health condition and recognize risks for several diseases early. Lee et al.²⁴ investigated the analysis of HRV recorded with an iPhone 4S to detect persistent atrial fibrillation. Furthermore, it is possible to determine the stress level by means of acquisition and analysis of HRV. Investigations of Melillo et al.²⁵ showed that nearly all HRV parameters measured by students at their examinations were reduced. A further application field of HRV analysis is in sport medicine. In this area, HRV parameters can be used for diagnosis of overreaching and overtraining, for example.²⁶

Conclusion

This study was conducted to validate a new algorithm for HR measurement using smartphone signals. The HR determination based on pulse curves, derived from a video signal using an iPhone 4S, produced a very high agreement with the HR from ECG recordings. Overall, the accuracy in HRV analysis achieved with the proprietary algorithm developed by the authors was sufficient for preclinical screening applications and as an indicator for the need for further diagnostic exploration.

Footnotes

Disclosure Statement

No competing financial interests exist. Andrea Seeck, PhD, Andreas Mainka and Thomas Huebner, PhD are employers of Preventicus GmbH and developed the algorithm for pulse wave analysis of smartphone recordings.

References

Taskforce. Heart rate variability. Standards of measurement, physiological interpretation, and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Eur Heart J, 1996; 17:354–381.

Lombardi

. Chaos theory, heart rate variability, and arrhythmic mortality. Circulation, 2000; 101:8–10.

Kleiger

, Stein

, Bigger

. Heart rate variability: Measurement and clinical utility. Ann Noninvasive Electrocardiol, 2005; 10:88–101.

Jarczok

, Jarczok

, Mauss

, Koenig

, Li

, Herr

, Thayer

. Autonomic nervous system activity and workplace stressors—A systematic review. Neurosci Biobehav Rev, 2013; 37:1810–1823.

Uusitalo

, Mets

, Martinmäki

, Mauno

, Kinnunen

, Rusko

. Heart rate variability related to effort at work. Appl Ergon, 2011; 42:830–838.

Risk

, Bril

, Broadbridge

, Cohen

. Heart rate variability measurement in diabetic neuropathy: Review of methods. Diabetes Technol Ther, 2001; 3:63–76.

Bigger

Jr. , Fleiss

, Steinman

, Rolnitzky

, Schneider

, Stein

. RR variability in healthy, middle-aged persons compared with patients with chronic coronary heart disease or recent acute myocardial infarction. Circulation, 1995; 91:1936–1943.

Glassman

, Bigger

, Gaffney

, Van Zyl

. Heart rate variability in acute coronary syndrome patients with major depression: Influence of sertraline and mood improvement. Arch Gen Psychiatry, 2007; 64:1025–1031.

Khandoker

, Karmakar

, Palaniswami

. Comparison of pulse rate variability with heart rate variability during obstructive sleep apnea. Med Eng Phys, 2011; 33:204–209.

10.

Schäfer

, Vagedes

. How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram. Int J Cardiol, 2013; 166:15–29.

11.

, Zhao

, Ju

, Shin

, Lee

, Shelley

, Chon

. Can photoplethysmography variability serve as an alternative approach to obtain heart rate variability information?. J Clin Monit Comput, 2008; 22:23–29.

12.

Wessel

, Voss

, Malberg

, Ziehmann

, Voss

, Schirdewan

, Meyerfeldt

, Kurths

. Nonlinear analysis of complex phenomena in cardiological data. Herzschrittmacherther Elektrophysiol, 2000; 159–173.

13.

Bland

, Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, 1986; 1(8476):307–310.

14.

Flatt

, Esco

. Validity of the Ithlete^™ smart phone application for determining ultra-short-term heart rate variability. J Hum Kinet, 2013; 39:85–92.

15.

Pelegris

, Banitsas

, Orbach

, Marias

. A novel method to detect heart beat rate using a mobile phone. Conf Proc IEEE Eng Med Biol Soc, 2010; 2010:5488–5491.

16.

Lenskiy

, Aitzhan

. Extracting heart rate variability from a smartphone camera. J Inf Commun Converg Eng, 2013; 11:216–222.

17.

Scully

, Lee

, Meyer

, Gorbach

AM.

, Granquist-Fraser

, Mendelson

, Chon

. Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Trans Biomed Eng, 2012; 59:303–306.

18.

Bolkhovsky

, Scully

, Chon

. Statistical analysis of heart rate and heart rate variability monitoring through the use of smart phone cameras. Conf Proc IEEE Eng Med Biol Soc, 2012; 2012:1610–1613.

19.

Kwon

, Kim

, Park

. Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. Proc Annu Int Conf IEEE Eng Med Biol Soc, 2012; 2012:2174–2177.

20.

McKinley

, Shapiro

, Bagiella

, Myers

, De Meersman

, Grant

, Sloan

. Deriving heart period variability from blood pressure waveforms. J Appl Physiol, 2003; 95:1431–1438.

21.

Rauh

, Limley

, Bauer

R-D

, Radespiel-Troger

, Mueck-Weymann

. Comparison of heart rate variability and pulse rate variability detected with photoplethysmography. Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, 2004; 5474:115–126.

22.

, Yang

, Taylor

, Stein

. A comparison of photoplethysmography and ECG recording to analyse heart rate variability in healthy subjects. J Med Eng Technol, 2009; 33:634–641.

23.

Peng

R-C

, Zhou

X-L

, Lin

W-H

, Zhang

Y-T

. Extraction of heart rate variability from smartphone photoplethysmograms. Comput Math Methods Med, 2015; 2015:1–11.

24.

Lee

, Reyes

, McManus

, Mathias

, Chon

. Atrial fibrillation detection using a smart phone. Annu Int Conf IEEE Eng Med Biol Soc, 2012; 2012:1177–1180.

25.

Melillo

, Bracale

, Pecchia

. Nonlinear heart rate variability features for real-life stress detection. Case study: Students under stress due to university examination. Biomed Eng Online, 2011; 10:96.

26.

Hottenrott

, Hoos

, Esperer

. [Heart rate variability and physical exercise. Current status]. Herz, 2006; 31:544–552.