Vibration pattern recognition using a compressed histogram of oriented gradients for snoring source analysis

Abstract

BACKGROUND:

Snoring source analysis is essential for an appropriate surgical decision for both simple snorers and obstructive sleep apnea/hypopnea syndrome (OSAHS) patients.

OBJECTIVE:

As snoring sounds carry significant information about tissue vibrations within the upper airway, a new feature entitled compressed histogram of oriented gradients (CHOG) is proposed to recognize vibration patterns of the snoring source acoustically by compressing histogram of oriented gradients (HOG) descriptors via the multilinear principal component analysis (MPCA) algorithm.

METHODS:

Each vibration pattern corresponds to a sole or combinatorial vibration among the four upper airway soft tissues of soft palate, lateral pharyngeal wall, tongue base, and epiglottis. 1037 snoring events from noncontact sound recordings of 76 simple snorers or OSAHS patients during drug-induced sleep endoscopy (DISE) were evaluated.

RESULTS:

With a support vector machine (SVM) as the classifier, the proposed CHOG achieved a recognition accuracy of 89.8% for the seven observable vibration patterns of the snoring source categorized in our most recent work.

CONCLUSION:

The CHOG outperforms other single features widely used for acoustic analysis of sole vibration site.

Keywords

Snoring source analysis vibration pattern recognition compressed histogram of oriented gradients multilinear principal component analysis

1. Introduction

Snoring is a prevalent sleep-related disorder affecting approximately 25% of females and 45% of males in the general population [1]. Defined as loud upper airway breathing caused by vibrations of the pharyngeal tissues during sleep [2], snoring may disturb the sleep of a bedpartner or others and may further become a cause of social disturbance. Snoring is also an important indicator of obstructive sleep apnea/hypopnea syndrome (OSAHS) [3], which is characterized by repetitive events of hypopnea and apnea resulting from partial or complete collapse within the upper airway [4]. This disease will decrease sleep quality, leading to daytime sleepiness and poor job performance. Furthermore, it increases the risk of hypertension, stroke and cardiovascular diseases [5,6].

There are several treatments for snoring such as continuous positive airway pressure (CPAP), oral appliances, and surgical treatment [7–9]. Since their effectiveness depends on the patient selection as well as the treatment procedure choice [9], the knowledge of anatomic structures of vibration and obstruction is necessary for a targeted and successful treatment.

Drug-induced sleep endoscopy (DISE) is a widely used method to determine the sites of vibration and obstruction through direct visual observation of patients’ pharyngeal cavity [10]. However, since patients are required to be induced to sleep by intravenous sedation with endoscopic equipment inserted into their pharyngeal cavity, DISE is invasive and time-consuming. Considering that snoring sounds carry significant information about the dynamic of the upper airway [11], acoustic analysis becomes a promising means for identifying the snoring source, which possesses many advantages such as lower cost, convenience, and non-invasion.

To date, an increasing amount of studies have attempted to find the relationship between different snoring sources and various acoustic features such as crest factor [12], peak and centre frequencies [13], fundamental and formant frequencies [14], statistical moment coefficients of skewness and kurtosis [15], wavelet features [16], and a combination of multiple features [17]. However, most of these studies only analyzed snoring events originated from vibration of sole soft tissue. But in practice, there are abundant snoring events related to combinatorial vibrations of multiple soft tissues, which are more common especially for OSAHS patients [18].

In this work, a total of seven vibration patterns of snoring sources in line with the work of Beijing Hospital, China [14] are recognized, including vibrations of one, two, or more soft tissues. Since the acoustic characteristics of the first several post-apnea snoring events are always irregular to some extent due to the fact that the upper airway shape always changes gradually after apnea [19], only those recordings with at least 10 consecutive steady snoring events were chosen for analysis, during which there is no noticeable occurrence of apnea or hypopnea.

Recently, texture-based descriptors have been successfully used in the field of image recognition [20]. When snoring events’ spectrograms are given as the input images, these descriptors are able to depict the spectral information [21], which should be used as the features related to the vibration patterns of the snoring source. Motivated by these studies, we propose a new feature denoted as compressed histogram of oriented gradients (CHOG), which is obtained by compressing the histogram of oriented gradients (HOG), a set of texture-based descriptors [20], via the multilinear principal component analysis (MPCA) algorithm [22]. Then, a support vector machine (SVM) is used to classify the vibration patterns of the snoring source.

The rest of this paper is organized as follows: Section 2 describes the method of data acquisition and annotation. Section 3 illustrates the proposed feature and recognition procedure. Detailed experimental results are provided in Section 4. Section 5 presents further discussions. Lastly, Section 6 concludes the work.

2. Data acquisition and annotation

This work was approved by Ethics Committee at Beijing Hospital, China, and all the snoring recordings were obtained from the Otolaryngolgy Department of Beijing Hospital. We used the recordings from 76 patients diagnosed as either simple snorers or OSAHS patients with an apnea-hypopnea-index (AHI) ≤45 assessed by polysomnography (PSG). Informed consent of these patients was obtained prior to the study. The patients’ demographic information (age, body mass index (BMI), and AHI) is listed in Table 1.

Table 1
Demographic information of the 76 patients

Mean Standard deviation

Age (years) 37.9 8.7

BMI (kg/m²) 26.0 3.2

AHI (events/h) 16.7 16.0

	Mean	Standard deviation
Age (years)	37.9	8.7
BMI (kg/m²)	26.0	3.2
AHI (events/h)	16.7	16.0

DISE was performed in the operating room by the same otolaryngologist and anesthesiologist for the purpose of snoring source determination. All patients were in the supine position and were given topical nasal administration of aerosolized 3% ephedrine and 2% lidocaine, and were then induced to sleep by injecting dexmedetomidine and propofol intravenously. When patients snored steadily, a video fibrolaryngoscopy (Machida ENT-30PIII), which was connected to a high-speed video system (Karl Storz Image 1 HUB HD), was inserted through a naris into the patients’ pharyngeal cavity. The vibrations of soft tissues in the upper airway were observed and recorded by the video system [19]. In parallel with the video recording, the snoring sounds were collected with a non-contact microphone (20–20 kHz, CHZ-213, BAST) positioned at about 30cm above the patient’s mouth, which was connected to a data acquisition system (developed by Nanjing University of Science and Technology) configured with a sampling frequency of 16 kHz and a resolution of 16-bit. Only snoring recordings with at least 10 consecutive steady snoring events were used for analysis in order to eliminate the influence of shape changes in the upper airway and to keep the characteristics of snoring events predominantly related to the vibration patterns of the snoring source [14].

According to the DISE results, the snoring sounds mainly generated from the soft palate, lateral pharyngeal wall, tongue base, and/or epiglottis. Among these tissues, soft palate is the most frequent vibration site, followed by lateral pharyngeal wall. In our most recent work, seven vibration patterns of the snoring source were observed as follows [14]: (1) sole vibration of soft palate; (2) obvious vibration of soft palate with mild vibration of epiglottis or tongue base; (3) sole vibration of epiglottis; (4) no obvious vibration of any tissues (snoring might be generated by the airflow passing through the extremely narrow upper airway since the soft palate almost touches the post pharyngeal wall); (5) combinatorial vibration of soft palate and lateral wall with mild vibration of epiglottis or tongue base; (6) combinatorial vibration of soft palate and lateral wall; (7) sole vibration of lateral pharyngeal wall. For simplicity, these seven categories are denoted in the sequel as P, P+E/T, E, NONE, ALL, P+L and L, respectively. We removed the recordings with strong background noise and manually extracted snoring events from the recordings. As a result, 1037 events in total with length ranging from 0.22 s to 2.18 s were available, whose detailed information is provided in Table 2.

Table 2

Detailed information of the recordings and snoring events used in this work

Category	Number of recordings	Number of snoring events
Sole soft palate (P)	17	180
Soft palate + epiglottis/tongue base (P+E/T)	12	171
Sole epiglottis (E)	3	49
No obvious vibration (NONE)	8	131
Soft palate + lateral pharyngeal wall + epiglottis/tongue base (ALL)	11	169
Soft palate + lateral pharyngeal wall (P+L)	12	166
Sole lateral pharyngeal wall (L)	13	171

3. Method

In this section, we illustrate the recognition procedure (Fig. 1). The individual processing steps are described in the following subsections.

Fig. 1.

Block diagram of the recognition procedure in this work.

3.1. Feature extraction

Extracting significant information from snoring events is the major challenge in snoring source analysis. In this work, we extracted CHOG from spectrograms to characterize the spectral texture of snoring events. The extraction process consists of three steps: spectrogram resizing, HOG descriptors extraction, and MPCA-based feature compression.

3.1.1. Spectrogram resizing

The snoring events used in this work were divided into frames of 64 ms with a shift of 32 ms between the adjacent frames. Then the short-time Fourier transform (STFT) with hamming window was applied to each frame to obtain the spectrograms, which can be considered as the input images of the following HOG descriptors extraction. Because HOG calculation requires the input size to be the same, these spectrograms were all resized to 64 × 256 by means of the nearest neighbor interpolation [23].

3.1.2. HOG descriptors extraction

HOG descriptors were used to describe the local object appearances based on the distribution of local gradients. The computation steps are as follows [20,21]:

Given a resized spectrogram of a snoring event, compute the gradient at each point. For example, consider a spectrogram S (x, y) with amplitude H (x, y), where x is the index of temporal frames and yis the index of Fourier coefficients, its horizontal and vertical gradients, denoted as G _x(x, y) and G _y(x, y), respectively, are calculated with the simple [−1 0 1] mask as $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle G_{x}(x,y)=H(x+1,y)-H(x-1,y)\end{eqnarray}$ (1) $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle G_{y}(x,y)=H(x,y+1)-H(x,y-1).\end{eqnarray}$ (2) Then the gradient magnitude G (x, y) and orientation 𝛼(x, y) of the point (x, y) are obtained as $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle G(x,y)=\sqrt{G_{x}(x,y)^{2}+G_{y}(x,y)^{2}}\end{eqnarray}$ (3) $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle {\alpha}(x,y)=\tan ^{-1}\left(\frac{G_{y}(x,y)}{G_{x}(x,y)}\right).\end{eqnarray}$ (4)

Divide the spectrogram into small non-overlapped regions, i.e. cells. At each point, compute a function of the gradient magnitude as the weighted vote of the corresponding gradient orientation bin. Then, a local gradient histogram for each cell denoted as a cell descriptor is constructed by accumulating the votes of all points within the cell in each orientation bin.

Group cells into large regions, i.e. blocks, with 50% overlap between neighboring blocks. Each block descriptors is the concatenation of all cell descriptors within the block. It is worth noting that local normalization of each block independently turns out to be essential for good performance due to a great variable range of gradient strengths. In addition, overlapped blocks reuse cells in final descriptors with different normalization, which enhances reliability [20].

Specifically, given a spectrogram with size 64 × 256, a cell and a block are defined as a region with size 8 × 8 and 16 × 16, respectively. The cell descriptor is calculated for each cell with nine gradient orientation bins, so that each block descriptor is 36-dimensional which concatenates four cell descriptors and is then normalized independently. The final HOG descriptors are obtained as a combination of all block descriptors with a dimension of 7 × 31 × 36, denoted as a third-order feature tensor sample $\{\mathbf{X}_{m}\in \mathbb{R}^{7\times 31\times 36},m=1,2,\ldots ,M\}$ , where M is the number of snoring events under analysis.

Conventionally, HOG descriptors are required to be vectored before being fed into a classifier. However, considering that the overlapped blocks increase the reliability of classification but generate plenty of redundancy, transforming HOG descriptors (7 × 31 × 36) into vectors (7812 × 1) will result in high computational cost. Furthermore, vectorization may break the original structure of feature set and lose the spatial correlation information. In order to overcome these drawbacks, we applied the MPCA algorithm for the purpose of feature compression.

3.1.3. MPCA-based feature compression

Principal component analysis (PCA) is a popular unsupervised linear technique for vector dimension reduction. As a generalization of PCA, MPCA was proposed in [22] for tensor input instead of a conventional vector. To be specific, MPCA is a tensor decomposition algorithm with orthogonality constraints, which decomposes a tensor into a core tensor multiplied by matrices each along one mode (dimension).

Given a set of third-order feature tensor samples $\{\mathbf{X}_{m}\in \mathbb{R}^{I_{1}\times I_{2}\times I_{3}},m=1,2,\ldots ,M\}$ with I _n, n = 1, 2, 3 the size of the nth–mode, a fourth-order tensor $\mathbf{X}\in \mathbb{R}^{I_{1}\times I_{2}\times I_{3}\times M}$ is formed as the concatenation of all samples along the fourth dimension. It is worth noting that all samples are normalized by subtracting the mean tensor before concatenation. Then the decomposition is implemented excluding the fourth-mode, which yields $\begin{eqnarray}\displaystyle \mathbf{X}=\mathbf{Y }\times _{1}\mathbf{U}^{(1)}\times _{2}\mathbf{U}^{(2)}\times _{3}\mathbf{U}^{(3)} & & \displaystyle\end{eqnarray}$ (5) where $\mathbf{U}^{(n)}\in \mathbb{R}^{I_{n}\times P_{n}},P_{n}\lt I_{n},n=1,2,3$ is the factor matrix along the nth-mode with orthogonality constraints. $\mathbf{Y }\in \mathbb{R}^{P_{1}\times P_{2}\times P_{3}\times M}$ is the core tensor projected from original tensor space $\mathbb{R}^{I_{1}}\otimes \mathbb{R}^{I_{2}}\otimes \mathbb{R}^{I_{3}}$ to subspace $\mathbb{R}^{P_{1}}\otimes \mathbb{R}^{P_{2}}\otimes \mathbb{R}^{P_{3}}$ spanned by factor matrices It can be considered as a concatenation of the third-order compressed feature tensors for all samples $\{\mathbf{Y }_{m}\in \mathbb{R}^{P_{1}\times P_{2}\times P_{3}},m=1,2,\ldots ,M\}$ . Y × _n U ⁽ⁿ⁾represents the product of the tensor Y with the factor matrix U ⁽ⁿ⁾ along the nth-mode [24]. The illustration of tensor decomposition is shown in Fig. 2.

Fig. 2.

Illustration of tensor decomposition.

The main challenge of MPCA is to find appropriate factor matrices to project the original tensor as optimally as possible while retaining the maximum information. The rank P _n is determined depending on the number of dominant eigenvalues of the product X _(n) X _(n) ^T, where X _(n) is the nth-mode matricized version of X. In other words, P _n is set as the minimum value satisfying the following inequality [25] $\begin{eqnarray} \arg \min _{P_{n}}{\displaystyle \frac{\mathop{\sum }_{i=1}^{P_{n}}{\lambda}_{i}}{\mathop{\sum }_{i=1}^{I_{n}}{\lambda}_{i}}}>{\theta} \end{eqnarray}$ (6) where 𝜆_i, i =1,2, …, I _n are the eigenvalues of X _(n) X _(n) ^T with 𝜆₁ ≥𝜆₂ ≥⋯ ≥𝜆_{I
_n}, and 𝜃 is the threshold defined as the ratio of the remained portion after truncation to the total scatter. Then the factor matrix U ⁽ⁿ⁾ is initialized as the P _n eigenvectors corresponding to the first P _n eigenvalues. The optimal solution can be obtained by the iterative Alternating Least Squares (ALS) algorithm [26], which estimates factor matrices sequentially by minimizing the loss function.

The decomposition process was implemented by the N-way Toolbox [27]. The threshold 𝜃 was set to 0.9 and the corresponding P ₁, P ₂, and P ₃, were 6, 27, and 23, respectively. Considering that each element of the compressed tensor, denoted as Y _m(p ₁, p ₂, p ₃) with p _n the index in the nth mode (1 ≤ p _n ≤ P _n), is almost uncorrelated with others, all elements are sorted in descending order according to the Fisher score that is defined as the ratio of the between-class scatter over the within-class one [28] $\begin{eqnarray}\displaystyle {\phi}(p_{1},p_{2},p_{3})={\displaystyle \frac{\displaystyle \mathop{\sum }_{c=1}^{C}N_{c}\cdot [\overline{\mathbf{Y }}_{c}(p_{1},p_{2},p_{3})-\overline{\mathbf{Y }}(p_{1},p_{2},p_{3})]^{2}}{\displaystyle \mathop{\sum }_{m=1}^{M}[\mathbf{Y }_{m}(p_{1},p_{2},p_{3})-\bar{\mathbf{Y }}_{cm}(p_{1},p_{2},p_{3})]^{2}}} & & \displaystyle\end{eqnarray}$ (7) where C is the number of vibration patterns, N _c is the number of samples in pattern c, and cm is the pattern label for the sample m. Y _c is the mean tensor of pattern c and Y is the total mean tensor. Then the first and also the most significant K elements are chosen to constitute the final CHOG feature, where K is set to 120 in this work.

3.2. SVM-based classification

We employed SVM for the classification task, which is a robust supervised learning method widely applied to pattern recognition. Since SVM was originally designed for binary classification by constructing a hyperplane based on the theory of structural risk minimization [29], we utilized the “one-against-one” strategy [30] for multi-class classification in this work. LIBSVM [31] was used to implement SVM with radial basis function (RBF) kernel while the two parameters, gamma and cost, were set to 0.125 and 4, respectively.

4. Experimental results

4.1. Experimental setup

Two hundred independent trials were conducted for performance evaluation. In each trial, the 1037 snoring events listed in Table 2 were randomly separated into two sets with 60% events for training and the other 40% for testing. Meanwhile, the percentage of training and testing snoring events in each category of vibration patterns was kept the as same as 60:40. Note that the factor matrices calculated from the feature tensor of the training set were preserved for using in the testing stage.

4.2. Performance metrics

We employed recognition accuracy to evaluate the overall recognition performance $\begin{eqnarray}accuarcy={\displaystyle \frac{N_{C}}{N_{T}}}\times 100\%\end{eqnarray}$ (8) where N _C is the number of correctly classified events and N _T is the total number of test events.

In addition, the performance of each category of vibration pattern was measured by the three common metrics: precision, recall, and F-score, which were then averaged over 200 independent random trials.

4.3. Results

In this section, we compared the recognition performance of the proposed CHOG and other widely-used features including crest factor (CF), fundamental frequency (F0), formants, spectral frequency features (SFF), power ratio between below and above 800 Hz (PR800), sub-band energy ratio (SER), Mel-scale frequency cepstral coefficients (MFCC), Empirical Mode Decomposition-Based Features (EMDF) and wavelet energy features (WEF) [16,17,32]. The box plot, as shown in Fig. 3, is utilized to depict the distribution of recognition accuracy values over 200 trails. It demonstrates that CHOG is superior to other features with an averaged accuracy of 89.8%, followed by MFCC and SFF, achieved 83.0% and 80.7%, respectively. By contrast, the average accuracy metrics of PR800, F0 and CF are below 40%, indicating that they may not be suitable for the snoring source recognition task.

Fig. 3.

The box plot of recognition accuracy for all features. The central line represents the median. The bottom and top edges of the box indicate the first and third quantiles, respectively. The whiskers extend to the extreme non-outliers and the outliers are symbolized as ‘+’.

In order to further investigate the performance of these features, the comparison of averaged precision, recall and F-score over 200 trials are provided in Tables 3, 4, and 5. It can be observed that the performance metrics of CHOG are higher than those of other features for almost all categories of the vibration pattern, except for pattern E for which the metrics of CHOG are slightly lower than those of MFCC.

Table 3

Comparison of all features in terms of the averaged precision for each category

Category	Precision (%)
	CHOG	CF	F0	Formants	SFF	PR800	SER	MFCC	EMDF	WEF
P	89.4	24.3	29.2	49.8	80.6	31.9	43.7	81.3	62.2	78.4
P+E/T	95.5	25.9	23.6	54.0	85.7	24.9	42.7	88.6	59.1	82.5
E	96.5	0.2	1.4	71.9	93.9	0.0	76.8	99.2	82.9	93.3
NONE	90.3	34.8	38.1	54.6	79.8	44.8	58.4	82.4	64.2	81.1
ALL	89.9	33.4	34.3	46.5	80.4	30.6	58.5	84.6	64.7	77.5
P+L	85.5	23.0	34.3	48.4	78.0	32.1	57.6	77.9	56.1	68.6
L	87.6	18.5	17.6	47.7	77.6	29.1	46.0	80.4	49.1	68.7

Table 4

Comparison of all features in terms of the averaged recall for each category

Category	Recall (%)
	CHOG	CF	F0	Formants	SFF	PR800	SER	MFCC	EMDF	WEF
P	90.4	50.7	46.5	52.2	83.8	56.4	63.5	84.0	62.7	80.1
P+E/T	95.1	10.4	11.6	56.2	83.9	26.7	38.1	85.9	64.6	79.5
E	90.9	0.0	0.2	38.5	89.6	0.0	28.0	91.0	69.9	89.7
NONE	92.1	40.9	64.9	55.2	82.2	33.7	54.8	87.0	61.7	83.0
ALL	89.0	50.8	39.1	47.0	77.9	41.2	58.0	83.0	62.5	79.8
P+L	83.5	9.0	33.7	48.5	75.7	34.9	52.6	76.3	55.1	67.1
L	88.9	10.8	8.0	47.2	78.1	3.1	42.0	80.1	48.8	67.1

Table 5

Comparison of all features in terms of the averaged F-score for each category

Category	F-score
	CHOG	CF	F0	Formants	SFF	PR800	SER	MFCC	EMDF	WEF
P	0.898	0.327	0.357	0.508	0.821	0.407	0.517	0.825	0.622	0.791
P+E/T	0.952	0.138	0.145	0.549	0.846	0.257	0.401	0.871	0.616	0.808
E	0.934	0.000	0.003	0.494	0.914	0.000	0.395	0.948	0.752	0.912
NONE	0.911	0.374	0.478	0.546	0.807	0.380	0.563	0.845	0.626	0.819
ALL	0.893	0.401	0.363	0.466	0.790	0.349	0.580	0.837	0.634	0.785
P+L	0.843	0.120	0.338	0.483	0.767	0.332	0.547	0.769	0.554	0.676
L	0.881	0.129	0.105	0.472	0.777	0.054	0.437	0.800	0.488	0.677

5. Discussion

Note that the threshold 𝜃 in ((6)), which determines the factor matrices rank P _n, n = 1, 2, 3, and the CHOG dimension K may bring a non-negligible impact on the performance of CHOG. In this section, we investigated the influence of these two parameters. The threshold 𝜃 was set to 0.9, 0.8, 0.7, and 0.6, while the corresponding ranks $\{P_{1},P_{2},P_{3}\}$ of the three factor matrices were calculated to be $\{6,27,23\}$ , $\{5,23,15\}$ , $\{4,19,10\}$ , $\{3,15,6\}$ , respectively. For each 𝜃, the value of K ranged from 20 to 200 with an increment of 20. According to the results shown in Fig. 4, the best performance was achieved when 𝜃 and K are set to 0.9 and 120, respectively. Furthermore, it shows that when 𝜃 is not less than 0.7 with K ranging from 80 to 140, there is only a slight difference between an accuracy metric, which implies that a stable performance can be achieved for the CHOG in a wide range of dimension values.

Fig. 4.

Influence of threshold 𝜃 and dimension K on the performance of CHOG.

In addition, according to the experimental results, the performance metrics for vibration patterns P+E/T and E are higher than those for pattern P+L. To be specific, the confusion matrix after one trial is provided in Table 6. There is only one snoring event of pattern P+E/T and pattern E that is misclassified, while a small part of events are misclassified into each other for three pairs of patterns: P and P+L, NONE and L, ALL and P+L.

Table 6

Confusion matrix after one trial using CHOG

Actual category	Predicted category							Comments
	P	P+E/T	E	NONE	ALL	P+L	L
P	64	0	1	0	4	3	0	Sole soft palate
P+E/T	1	67	0	0	0	0	0	Soft palate + epiglottis/tongue base
E	1	0	19	0	0	0	0	Sole epiglottis
NONE	3	0	0	45	0	0	4	No obvious vibration
ALL	0	0	0	0	61	5	2	Soft palate + lateral pharyngeal wall + epiglottis + tongue base
P+L	4	0	0	0	5	54	3	Soft palate + lateral pharyngeal wall
L	0	0	0	8	0	1	59	Sole lateral pharyngeal wall

Snoring solely origniating from vibration of epiglottis (vibration pattern E) is relatively rare in practice and the snorers who have this type of snoring cannot be treated by regular ENT surgical procedures. In this work, we distinguished snoring events of pattern E from others with a high accuracy, which helps to avoid ineffective surgery for these snorers. In addition, the difference between snoring events in patterns ALL and P+L is the slight flutter of epiglottic/tongue, which is always indiscernible even under the observation of DISE. Therefore, the result shows that the confusion between these two patterns in our experiment is consistent with the outcome of DISE. However, the differences between snoring events in vibration patterns P and P+L, NONE and L are relatively significant in DISE. According to the analysis of detailed experimental results, we found that misclassified events are concentrated in several recordings, and the reason is under further investigation.

Considering that CHOG depicts the texture information of spectrogram from an image processing perspective while MFCC and SFF, which achieve comparable performance to CHOG as depicted in Fig. 3, characterize the spectral and Mel spectral information respectively, we fused these features in order to investigate the impact of multi-feature analysis strategy on recognition performance. ReliefF algorithm [33] was applied to the aggregation of CHOG, MFCC, and SFF to calculate a contribution weight for each feature element. Since a higher positive weight represents a more important contribution, all feature elements are ranked in descending order of weight and the first 120 feature elements are selected to construct the fusion feature set, among which CHOG, MFCC and SFF contribute 50%, 26.7%, 23.3%, respectively. Expermential results after 200 trials show that the fusion feature set achieves an average recognition accuracy of 92.6%. It demonstrates that fusing multiple diversified features together may bring an improved recognition performance, and multi-feature analysis strategy can be adopted in future research to construct a more comprehensive and discriminative feature set.

6. Conclusion

1. It was demonstrated that the noncontact acoustic analysis is a promising alternative to determine the snoring source of patients, which is neither complicated nor invasive compared to conventional methods such as DISE.

2. The snoring source was categorized into seven vibration patterns in our recent work according to the different combinations of upper airway soft tissues vibrating simultaneously.

3. The proposed feature CHOG can characterize the textural information of the snoring sound spectrograms. As a result, it achieved an accuracy of 89.8% for recognizing the seven vibration patterns of the snoring source based on the sound recordings of 76 simple snorers or OSAHS patients during DISE.

4. Experimental results showed that the snoring source recognition potency of CHOG was better than other widely-used single features, while a higher recognition accuracy could be obtained if CHOG was fused with other features such as MFCC and SFF.

Footnotes

Acknowledgements

The work was supported by the National Natural Science Foundation of China under Grant Nos 61271410 and 61401203.

Conflict of interest

None to report.

References

Young

Palta

Dempsey

Skatrud

Weber

and Badr

, The occurrence of sleep-disordered breathing among middle-aged adults, N. Engl. J. Med. 328(17) (1993), 1230–1235.

Thorpy

M.J.

, Classification of sleep disorders, J. Clin. Neurophysiol. 7 (1990), 67–81.

Dalmasso

and Prota

, Snoring: analysis, measurement, clinical implications and applications, Eur. Respir. J. 9(1) (1996), 146–159.

Strollo Jr

P.J.

and Rogers

R.M.

, Obstructive sleep apnea, N. Engl. J. Med. 334(2) (1993), 99–104.

Arzt

Young

Finn

Skatrud

J.B.

and Bradley

T.D

, Association of sleep-disordered breathing and the occurrence of stroke, Am. J. Respir. Crit. Care. Med. 172(11) (2005), 1447–1451.

Marin

J.M.

Carrizo

S.J.

Vicente

and Agusti

A.G

, Long-term cardiovascular outcomes in men with obstructive sleep apnoea-hypopnoea with or without treatment with continuous positive airway pressure: an observational study, Lancet 365(9464) (2005), 1046–1053.

Kribbs

N.B.

Pack

A.I.

Kline

L.R.

Smith

P.L.

Schwartz

A.R.

Schubert

N.M

, Objective measurement of patterns of nasal CPAP use by patients with obstructive sleep apnea, Am. Rev. Respir. Dis. 147(4) (1993), 887–895.

Dioguardi

and Al-Halawani

, Oral appliances in obstructive sleep apnea, Otolaryngol. Clin. North. Am. 49(6) (2016), 1343–1357.

K.K.

, Surgical therapy for adult obstructive sleep apnea, Sleep Med. Rev. 9(3) (2005), 201–209.

10.

Croft

C.B.

and Pringle

, Sleep nasendoscopy: a technique of assessment in snoring and obstructive sleep apnoea, Clin. Otolaryngol. Allied. Sci. 16(5) (1991), 504–509.

11.

Pevernagie

Aarts

R.M.

and Meyer

M.D.

, The acoustics of snoring, Sleep Med. Rev. 14(2) (2010), 131–144.

12.

Hill

P.D.

Lee

B.W.

Osborne

J.E.

and Osman

E.Z.

, Palatal snoring identified by acoustic crest factor analysis, Physiol. Meas. 20(2) (1999), 167–174.

13.

Agrawal

Stone

McGuinness

Morris

and Camilleri

A.E.

, Sound frequency analysis and the site of snoring in natural and induced sleep, Clin. Otolaryngol. Allied. Sci. 27(3) (2002), 162–166.

14.

Peng

Huang

Jia

, Acoustic analysis of snoring sounds originating from different sources determined by drug-induced sleep endoscopy, Acta. Otolaryngol. 137(8) (2017), 872–876.

15.

Beeton

R.J.

Wells

Ebden

Whittet

H.B.

and Clarke

, Snore site discrimination using statistical moments of free field snoring sounds recorded during sleep nasendoscopy, Physiol. Meas. 28(10) (2007), 1225–1236.

16.

Qian

Janott

Zhang

Heiser

and Schuller

, Wavelet features for classification of vote snore sounds. in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Shanghai, China, 2016, pp. 221–225.

17.

Qian

Janott

Pandit

Zhang

Heiser

and Hohenhorst

, Classification of the excitation location of snore sounds in the upper airway by acoustic multifeature analysis, IEEE Trans. Biomed. Eng. 64(8) (2017), 1731–1741.

18.

Jia

Gao

Huang

Peng

, Investigation of the source of snoring sound by drug-induced sleep nasendoscopy, ORL J. Otorhinolaryngol. Relat. Spec. 77(6) (2015), 359–365.

19.

Yang

Qin

Huang

Peng

and Xu

, Acoustic characteristics of snoring sound in patients with obstructive sleep apnea hypopnea syndrome, J. Clin. Otorhinolaryngol. 26(8) (2012), 360–363.

20.

Dalal

and Triggs

, Histograms of oriented gradients for human detection. in: Proceedings of the IEEE Computer Society Conference on Computer Vision & Pattern Recognition, IEEE, San Diego, CA, USA, 2005, pp. 886–893.

21.

Rakotomamonjy

and Gasso

, Histogram of gradients of time–frequency representations for audio scene classification, IEEE/ACM Trans. Audio Speech Lang. Process 23(1) (2015), 142–153.

22.

Plataniotis

K.N.

and Venetsanopoulos

A.N.

, MPCA: multilinear principal component analysis of tensor objects, IEEE Trans. Neural Netw. 19(1) (2008), 18–39.

23.

Parker

Kenyon

R.V.

and Troxel

D.E.

, Comparison of interpolating methods for image resampling, IEEE Trans. Med. Imaging. 2(1) (1983), 31–39.

24.

Bader

B.W.

and Kolda

T.G

, Algorithm 862: MATLAB tensor classes for fast algorithm prototyping, ACM Trans. Math. Software. 32(4) (2006), 635–653.

25.

Phan

A.-H.

and Cichocki

, Tensor decompositions for feature extraction and classification of high dimensional datasets, Nonlinear Theory and its Applications, IEICE 1(1) (2010), 37–68.

26.

Kolda

T.G.

and Bader

B.W

, Tensor decompositions and applications, Siam Rev. 51(3) (2009), 455–500.

27.

Andersson

C.A.

and Bro

, The N-way Toolbox for MATLAB, Chemometr. Intell. Lab. 52(1) (2000), 1–4.

28.

Yan

Yang

Zhang

Tang

and Zhang

H.-J.

, Discriminant analysis with tensor representation. in: Proceedings of the IEEE Computer Society Conference on Computer Vision & Pattern Recognition, IEEE, San Diego, CA, USA, 2005, pp. 526–532.

29.

Cortes

and Vapnik

, Support vector network, Mach. Learn. 20(3) (1995), 273–297.

30.

Hsu

C.W.

and Lin

C.J.

, A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Netw. 13(2) (2002), 415–425.

31.

Chang

C.-C.

and Lin

C.-J.

, LIBSVM: A library for support vector machines, ACM Trans. Intell. Syst. Technol. 2(3) (2011), 1–27.

32.

Qian

and Zhao

, Automatic detection, segmentation and classification of snore related signals from overnight audio recording, IET Signal Process 9(1) (2015), 21–29.

33.

Kononenko

Simec

and Robnik-Sikonja

, Overcoming the myopia of inductive learning algorithms with RELIEFF, Appl. Intell. 7(1) (1997), 39–55.

Vibration pattern recognition using a compressed histogram of oriented gradients for snoring source analysis

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSION:

Keywords

1. Introduction

2. Data acquisition and annotation

Table 1 Demographic information of the 76 patients Mean Standard deviation Age (years) 37.9 8.7 BMI (kg/m2) 26.0 3.2 AHI (events/h) 16.7 16.0

3.1.1. Spectrogram resizing

3.1.2. HOG descriptors extraction

4. Experimental results

4.1. Experimental setup

4.2. Performance metrics

Footnotes

Acknowledgements

Conflict of interest

References

Table 1
Demographic information of the 76 patients

Mean Standard deviation

Age (years) 37.9 8.7

BMI (kg/m²) 26.0 3.2

AHI (events/h) 16.7 16.0