Influence of tree-based multi-layer node information on scoring accuracy and speed of speaker verification

Abstract

In the speaker verification task based on Gaussian Mixture Model-Universal Background Model (GMM-UBM), by constructing the UBM as a tree structure, the kernel Gaussians suitable for different speakers can be quickly selected, which speeds up the modeling of speaker acoustic space by GMM. The tree-based kernel selection algorithm (TBKS) introduces a beam-width, which increases the candidate range of kernels and improves the kernel selection accuracy. In this paper, we improve the TBKS algorithm by introducing a recall rate to adjust the number of nodes recalled in each layer of the tree structure. This adjustment refines the quantity and resolution of Gaussian distributions in various subspaces within the acoustic space, compensating for the loss caused by discarding some significant Gaussians erroneously. Speaker verification experiments are carried out based on the Aishell2 dataset. The results reveal that the modified TBKS algorithm reduces EER by 7.5% relatively and increses computational reduction factor to 42.93, enhancing both recognition accuracy and speed. In addition, the test speech is spliced into different lengths and common environmental noise is added to verify the universality of the improved algorithm.

Keywords

Speaker verification fast scoring gaussian mixture model tree-based kernel selection

1 Introduction

The Gaussian Mixture Model (GMM) is commonly used to model data distributions and finds wide applications in fields such as cluster analysis, pattern recognition, and anomaly detection [1, 2]. In the task of speaker verification, the Gaussian Mixture Model-Universal Background Model (GMM-UBM) [3, 4] is used to model the acoustic space for general background and specific speakers. GMM-UBM first trains the UBM, and then the speaker-specific GMM is obtained based on the UBM and speaker-specific speech data. Acoustic space modeling via GMM eliminates the need for massive training data or long-term training processes. In addition, GMM-UBM requires less computational resources, making it possible to realize real-time applications on resource-constrained embedded devices [5, 6].

However, in the verification phase, it takes up most of the operation time to calculate the likelihood values of all Gaussian components in UBM and the speaker model [7]. In order to reduce the computation time in the validation phase, the following three perspectives are usually considered: the number of feature vectors to be scored in a test utterance (downsampling) [8], the dimension of the feature vector, and the number of Gaussian components in the UBM. Typically, only Gaussians near the feature vectors are selected as the kernels for likelihood calculations. This approach reduces the number of Gaussians that need to be considered, as these chosen kernels contribute significantly to the likelihood value.

In order to achieve fast search of kernels, researchers have proposed various methods. Yu et al. introduced the Sorted GMM [9], a method that arranges Gaussian components in a sorted manner and subsequently identifies the optimal Gaussian index through a codebook search applied to the feature vectors. Roland et al. analyzed three hash GMMs [10]. For each Hash GMM component, a candidate list of indexes was generated containing the indexes of the Gaussian components used for validation. Rahim et al. combined Gaussian selection with frame selection [11]. Xiang et al. proposed Structural GMM [12], in which each Gaussian component in the UBM are used as leaf nodes in a tree structure to narrow down the selection of kernels through a top-down hierarchical search. This process of selecting kernels has also been effectively utilized in GMM-SVM models [13] as well as Ivector-based models [14 –16]. The introduction of the aforementioned fast selection methods has led to a substantial enhancement in scoring speed. However, since it’s not possible to precisely identify the top Gaussians with the highest global likelihood values, a compromise must be made within a tolerable range, sacrificing some recognition accuracy.

Enhancing recognition accuracy while maintaining fast scoring has become a key focus in the practical application of speaker verification systems in real-world scenarios. Building upon their work in reference [12], Xiang et al. employed a Multi-Layer Perceptron (MLP) as a speaker identity discriminator after computing the likelihood values. However, the additional MLP requires a significant amount of training data, which in turn increases computational load. Xiong et al. proposed the TBKS (tree-based kernel selection) algorithm [19], which directly optimizes the strategy for selecting kernel distributions. By introducing a beam-width, the TBKS algorithm expands the local search range, thus reducing the likelihood of selecting core distributions incorrectly. The TBKS algorithm balances recognition accuracy and speed. With just a 1% increase in EER, the number of Gaussians that need to be calculated can be reduced by a factor of 15.2, and the algorithm remains robust to additive background noise [20].

However, the TBKS algorithm sacrifices the acoustic information contained in the discarded Gaussian components to ensure recognition speed. To address this issue, this paper proposes an improved approach aimed at recalling some of the discarded but relatively important Gaussian components to optimize the kernel selection process based on TBKS. The structure of this paper is organized as follows. Firstly, hierarchical clustering is performed on each single Gaussian component of the UBM to create a tree-like structure. Secondly, the matching degree between each candidate node at each layer of the tree structure and the speech frames is computed, serving as the basis for selecting kernels. Finally, while determining the search range for each layer, some nodes outside the beam-width are recalled, and their aggregate nodes are added to the kernel group. The effectiveness of the improved TBKS algorithm is validated through speaker verification experiments on the Aishell2 dataset.

Fig. 1

Adaptation process of the speaker model.

2 Speaker verification based on GMM-UBM

The Gaussian mixture model (GMM) is a linear combination of several Gaussian probability density functions. It can be used to represent the characteristic distribution of any speech feature vector by the model parameters Λ = {w_i, μ_i, Σ_i}, which are the weight, mean vector, and diagonal covariance matrix respectively for the i-th component, where i = 1, 2, ⋯ , M. The speaker-independent universal background model (UBM) [21] and speaker models are commonly represented by GMM, in which speaker models are adapted from UBM through maximum a posteriori (MAP). A set of D-dimensional feature vectors X = {x₁, x₂, ⋯ , x_T} are extracted from overlapping windows (the so-called frames) of speech, where T is the number of frames. The likelihood value of the feature vector on GMM indicates the fitness of the feature vector on GMM. Speaker verification amounts to computing the log-likelihood ratio of sequence between the UBM and speaker model, which determines the correction of the claimed speaker identity. To avoid arithmetic underflow, the likelihood ratio is usually converted to the difference of the log-likelihoods and is called the likelihood score: $\begin{matrix} Λ (X) = \frac{1}{T} \sum_{t = 1}^{T} (log p (x_{t} ∣ λ_{tar}) \\ - log p (x_{t} ∣ λ_{UBM})) \end{matrix}$ (1) where p (x_t ∣ λ) is the likelihood function of the t-th vector on GMM represented by λ: $p (x_{t} ∣ λ) = \sum_{i = 1}^{M} w_{i} p_{i} (x_{t})$ (2) where p_i (·) is the probability density function (pdf) of the i-th Gaussian component: $\begin{matrix} \begin{matrix} p_{i} (x_{t}) & = \frac{1}{(2 π)^{D / 2} | Σ_{i} |^{1 / 2}} \\ \times exp {- \frac{1}{2} (x_{t} - μ_{i})^{'} Σ_{i}^{- 1} (x_{t} - μ_{i})} \end{matrix} \end{matrix}$ (3)

According to Eqs. (2) and (3), the computation load of likelihood values on all M Gaussian components occupies most of the scoring time. However, a side benefit of MAP adaptation is that it keeps the correspondence between the Gaussians of the target-speaker model and the UBM. This allows the kernels selected from UBM to be directly applied to the speaker model, thereby reducing the time to calculate the likelihood score.

Figure 1 illustrates the adaptation process of a speaker model. For a feature vector x_t of the speaker, only a few nearby Gaussians update the parameters and get closer to x_t. The more feature vectors are input, the more Gaussians are updated, and the more accurate the model expresses speaker acoustic information. As shown in the figure:

(1) When calculating the likelihood value of x_t on GMM, only a few Gaussians near it contribute the most to the likelihood, while the Gaussians far away from it almost have no contribution.

(2) If x_t fits well with a Gaussian in UBM, it will also fit well with the corresponding Gaussian in the target speaker model, and vice versa.

In this paper, only the mean vectors are updated during MAP adaptation. Firstly, calculate the posterior probability of x_t on the i-th Gaussian component: $\Pr (i ∣ x_{t}) = \frac{w_{i} p_{i} (x_{t})}{\sum_{j = 1}^{M} w_{j} p_{j} (x_{t})}$ (4) Secondly, calculate the sufficient statistics of x_t on GMM: $n_{i} = \sum_{t = 1}^{T} \Pr (i ∣ x_{t})$ (5) $E_{i} (X) = \frac{1}{n_{i}} \sum_{t = 1}^{T} \Pr (i ∣ x_{t}) x_{t}$ (6) Finally, the adaptation of mean vectors is expressed by ${\hat{μ}}_{i} = α_{i} E_{i} (X) + (1 - α_{i}) μ_{i}$ (7) where $α_{i} = \frac{n_{i}}{n_{i} + r}$ controls the degree to which the new statistics are used to update the old statistics. The relevance factor r is typically set to 16.

For each input feature vector, top C highest-scoring components are selected as kernels after the likelihood values are calculated on all M Gaussians in UBM and applied to the speaker model. C is typically set to 5 [4]. At this time, the number of Gaussians to be calculated is reduced from 2M to M + C. According to Eq. (2), since C is usually far less than M, the computational cost of calculating the likelihood value is reduced by almost a half. However, the traversal search requires a calculation of all Gaussian components, which is still not efficient enough. It becomes essential to select kernels quickly and accurately for system performance improvement.

3 Kernel selection in tree structure

To use as few Gaussians as possible to more accurately represent the local acoustic space where the feature vector is located, this paper builds an L-layer tree to perform hierarchical clustering of all Gaussians in UBM based on the TBKS algorithm. When selecting the kernels, search from top to bottom. The candidate nodes of the current layer are arranged in descending order according to the likelihood values. Some nodes enter the next layer to search, some nodes are reserved as part of the kernel set, and the remaining nodes are pruned. Thus, an adaptive resolution GMM is constructed for the local acoustic space.

3.1 Construction of the tree structure

In the tree structure, the leaf nodes (M in total) correspond exactly to the components in UBM. The nodes generated in the first L - 1 layers are the merging of corresponding child nodes, and so on, the root node is the merging of M leaf nodes. Before constructing the tree structure, it is necessary to define the distance between two single Gaussians and the combined Gaussian model parameters of multiple Gaussians.

As one of the common gaussian distance measures, symmetrized Kullback–Leibler (KL) divergence [17] represents the sum of the relative entropy of two Gaussians. When diagonal covariance matrices are assumed, the distance d_KL (m, n) between two Gaussian components G_m ∼ N (μ_m, Σ_m) and G_n ∼ N (μ_n, Σ_n) can be simplified to $\begin{matrix} d_{KL} (m, n) = & \int G_{m} (x) ln \frac{G_{m} (x)}{G_{n} x} dx \\ + \int G_{n} (x) ln \frac{G_{n} (x)}{G_{m} (x)} dx \\ = & \frac{1}{2} \sum_{k = 1}^{K} [\frac{σ_{m}^{2} (k) + {(μ_{m} (k) - μ_{n} (k))}^{2}}{σ_{n}^{2} (k)} \\ + \frac{σ_{n}^{2} (k) + {(μ_{m} (k) - μ_{n} (k))}^{2}}{σ_{m}^{2} (k)} - 2] \end{matrix}$ (8) where μ_m (k) is the k-th element of the mean vector μ_m, $σ_{m}^{2} (k)$ is the k-th element of the covariance matrix, and K is the feature dimension. The centroid of a set of Gaussians Φ can be estimated by maximum likelihood (ML) estimation [18]. The pdf parameters of the centroid are calculated as follows $\begin{matrix} μ_{c} (k) = \frac{\sum_{i \in Φ} w_{i} μ_{i} (k)}{\sum_{i \in Φ} w_{i}} \\ σ_{c}^{2} (k) = \frac{\sum_{i \in Φ} w_{i} (σ_{i}^{2} (k) + μ_{i}^{2} (k))}{\sum_{i \in Φ} w_{i}} - μ_{c}^{2} (k) \\ w_{c} = \sum_{i \in Φ} w_{i} \end{matrix}$ (9) where w_i is the weight of the i-th Gaussian. We refer to the nodes obtained from clustering as parent nodes, and correspondingly, the nodes used for clustering are referred to as child nodes.

After determining the number of layers L and the number of branches of all nodes in the upper L - 1 layers, the tree is constructed as follows [17].

Step 1 Set the root node to be R, and its Gaussian index set to be G = {1, 2, ⋯ , M}. Calculate the mean and covariance of R according to Eq. (9). In this case, the weight is 1.

Step 2 Set the current node to be r, and its Gaussian index set to be g. If r has no child nodes, then stop clustering; Otherwise, the child nodes r_child are initiated by selecting components from g according to the Minimax method [17].

Step 3 Fine-tune the initial value of child nodes r_child by linear interpolating with corresponding father node

$\begin{matrix} μ_{r_{child}} (k) = & (1 - α) μ_{r} (k) + α μ_{r_{child}} (k) \\ {\hat{σ}}_{r_{child}}^{2} (k) = & (1 - α) (σ_{r}^{2} (k) + μ_{r}^{2} (k)) \\ + α (σ_{r_{child}}^{2} (k) + μ_{r_{child}}^{2} (k)) - {\hat{μ}}_{r_{child}}^{2} (k) \end{matrix}$ (10)

Step 4 According to Eq. (8), classify all components in g to the nearest child node and update the model parameters of the child nodes according to Eq. (9).

Step 5 Return to Step 4 and execute, until the distances between each Gaussian component in g and corresponding child node remain unchanged.

Fig. 2

Construction of the tree structure.

According to the construction process above, the number of branches in the last non-leaf layer is unknown, and the number of nodes in the last layer is equal to the total Gaussian number of UBM. Figure 2 shows the distribution of tree-based Gaussian nodes in the acoustic space. Each layer of the tree represents a more precise or higher-resolution GMM for the acoustic space than the layers near the root node.

3.2 Tree-based kernel selection algorithm

After the tree structure is established, the local optimal search for kernels can be performed from top to bottom. Normally, Gaussians near the boundary of two clusters will be hard classified into one of them. They may be discarded by mistake if these two clusters have similar likelihood values. The TBKS algorithm introduces beam-width K to search for C representative kernels. As shown in Fig. 3, the search process is as follows [22].

Step 1 Set the index set of the candidate nodes in the current layer to be G and put all the child nodes under the root node into G.

Step 2 Calculate the likelihood values of the feature vector on all nodes in G and then sort nodes in descending order.

Step 3 If the candidate nodes are leaf nodes, select out the first C highest-scoring ones from G as the kernels. Otherwise, empty G and put the first K highest-scoring nodes into it, and return to Step 2.

Fig. 3

Construction of the TBKS algorithm.

The introduction of beam-width K balances accuracy and scoring computation. The number of candidate nodes in each layer increases with the increase of K, which leads to an increase in the computing load of the likelihood value. However, the accuracy of the local optimal search is improved due to the consideration of the Gaussian components near the decision boundary.

3.3 Improvements to the TBKS algorithm

Although the TBKS algorithm expands the search range of the kernels, the accuracy of the speaker verification system still drops a bit. First, the number of branches in each layer is fixed, which results in some nodes being hard classified to a farther cluster. Second, one of the two nodes with similar likelihood values may still have to be discarded near the beam-width. Thus, some representative Gaussians will be missed, and the acoustic space where each feature vector is located cannot be fully characterized.

For the nodes near the beam-width, although their representation ability is lower than the nodes in the beam width, they still contain some useful information. If K keeps increasing, the child nodes of such ambiguous nodes will be included in the set of candidate nodes at the next layer. However, these child nodes are likely to be removed at the next level due to their fairly low representational capability. Therefore, this paper directly retains the nodes near the beam-width as part of the kernels and uses the lower-resolution merged Gaussians in the upper L - 2 layers (excluding the root node layer) to roughly represent the acoustic space that is far away from the feature vector and have lower significance.

Fig. 4

Construction of the improved TBKS algorithm.

The construction process is shown in Fig. 4(a). The search starts from the root node. In the beam-width, all the child nodes of selected nodes are scored and sorted in descending order layer by layer. The first K child nodes are treated as candidate nodes in the next layer. Some lower-scoring child nodes are directly retained as components of the kernel set, while the rest of the lowest-scoring child nodes are pruned.

In this paper, the recall rate ξ is introduced to adjust the proportion of retained child nodes. ξ = 1 means that all child nodes outside the beam-width are recalled. When ξ equals to 1, it is the TBKS algorithm, in which only K child nodes are retained. The kernel set of the feature vector is composed of the retained merged gaussian nodes (such as node 3,4,5,6) in upper L - 2 layers and the original Gaussian nodes (such as node 1,2) in the last layer. Figure 4(b) shows the distribution of the kernel set in the acoustic space. Compared with TBKS, the computational cost of selecting the kernels is constant. Instead of discarding nodes outside the beam-width directly, this paper retains some useful nodes to compensate for the loss caused by the tree structure and the local searching. The implementation steps are as follows:

Step 1 Set the index set of the candidate nodes in the current layer to be G and put all the child nodes under the root node into G.

Step 2 Calculate the likelihood values of the feature vector on all nodes in G and then sort nodes in descending order.

Step 3 If the candidate nodes are leaf nodes, select out the first C highest-scoring ones from G as the kernels, including them along with the aggregated nodes recalled from the previous L-2 layers as kernels. Otherwise, empty G, put the first K highest-scoring nodes into it, select suboptimal nodes based on the recall ratexi, compute aggregated Gaussian nodes according to Eq. (9), and return to Step 2.

In the scoring stage, the Gaussians used in the UBM and the speaker model need to be corresponding for a feature vector. Thus, the tree structure of the speaker model and the UBM should be kept consistent. Since the Gaussian distribution of non-leaf nodes is required, the index sets of nodes at all layers of the speaker model’s tree structures should also be matched with the UBM’s. When enrolling the speaker model, M leaf nodes are obtained by MAP adaption from UBM, and then the model parameters of nodes in each layer are calculated according to Eq. (9).

3.4 Computational reduction factor

In section 3.3, the resolution and number of Gaussians in each subspace of the speaker are adjusted according to the local acoustic space where the feature vector is located. The original GMM is replaced by a few kernels. In this paper, the computational reduction factor [18] is chosen to represent the speed up of a speaker verification system. Taking the traversal search as the baseline, the ratio of the Gaussian number required is calculated before and after using the fast selection algorithm. Without considering the Gaussian sorting or searching time, the expression is as follows: $F^{'} = \frac{M + C}{M_{ubm} + C_{sm}}$ (11) where M, C, and C_sm, M_ubm are the number of Gaussians to be calculated in UBM and the number of kernels to be selected under the traversal search and improved TBKS algorithm respectively. M_ubm is approximated by $M_{ubm} = \frac{1}{T} \sum_{t = 1}^{T} \sum_{l = 2}^{L} n_{l, t}$ (12) where n_l,t is the number of nodes evaluated in layer l for the feature vector x_t.

Let the branch number set of the first L - 1 layer be R = {R₁, R₂, ⋯ , R_L-1}, and C original Gaussians are selected in the last layer. The number of kernels C_sm is given by $C_{sm} = \sum_{l = 2}^{L - 1} C_{l} + C$ (13) where C_l = ξ (R_l-1K - K), ξ is the recall rate, R_l-1 is the branch number of the upper layer, K is the beam-width. In this paper, each layer of the tree structure uses the same beam-width and recall rate.

4 Experiments and results

In this section, trees with several structures are built according to the process in section 3.1. All the components in UBM are hierarchically clustered. As described in section 3.3, the search of kernels is carried out from top to bottom. The candidate nodes will be retained, discarded, or further subdivided according to their importance on the likelihood values. The kernel set is composed of the nodes retained in each layer. Based on the traversal search, the effects of different tree structures, beam-widths, and recall rates on accuracy and recognition speed are discussed. Considering the application of speaker verification in real scenes, the robustness of the system under different audio durations and two common kind of noises is also explored.

4.1 Processing of the speech data

In this paper, the experiments are based on Aishell2 [23], a Mandarin Chinese dataset recorded in a quiet indoor environment. The duration of each record is about 3∼5 seconds. 300 people are selected to train UBM with 1,024 mixtures. Use records in the dataset to splice 30s audio for each person. Another 50 people are selected for enrollment and test, with a balanced ratio of male to female. After the voice activity detection, 13-dimensional Mel-Frequency cepstral coefficients (MFCC) are extracted from speech. Typically, the frame length is 200ms and the neighboring frame is overlapped by 80ms. Then, 13 delta-cepstral coefficients and 13 delta-delta-cepstral coefficients are concatenated to the cepstral vector to generate a 39-dimensional vector. Finally, these vectors are processed with CMVN [24] to eliminate convolution noise.

4.2 Configurations of the tree structure

Before the speaker verification experiments, we investigate the effect of the kernel number C in traversal search and the most suitable C is selected for the baseline. As shown in Fig. 5, as C increases, the EER obtained through traversal search gradually rises and tends to be stable. This indicates that most Gaussians contribute a little to the accuracy. Hence, this paper takes the traversal search with C = 1 as the baseline, EER = 2.1633.

Fig. 5

Influence of kernel number on the performance of the traversal search.

Note that, although the best accuracy can be obtained using only one kernel, the fast scoring algorithms cannot accurately select the best Gaussian component. Only by increasing the number of kernels can the loss caused by the wrong selection be minimized.

Set the layer number of the tree to be L, and the set of branch number in upper L - 1 layers to be R = {R₁, R₂, ⋯ , R_L-1}. The TBKS algorithm indicates that the computational reduction factor can reach the maximum when R₁ = K × R₂ = ⋯ = K × R_L-1, and maintain the value when L > 5. Hence, 6 types of tree structures are selected to discuss the influence of beam-width K and recall rate ξ on fast verification system. The number of leaf nodes with the highest likelihood values of being selected as part of the kernels in the last layer is 4.

Fig. 6

Performance under different tree structures.

Figure 6 shows the relationship between EER and actual computational reduction factor F′ when the beam-width K is set to {1, 2, 3} and the recall rate ξ is set to {0, 0.05, 0.1, 0.2, 0.3}. When ξ equals to 0, it is the TBKS algorithm. As shown in Fig. 6(a), L5 : {1, 16, 4, 4, *} indicates that the tree has 5 layers, and the branch number in the first L - 2 layers is {16, 4, 4}. ’*’ indicates that the branch number in the penultimate layer is uncertain. It can be seen from the figures above:

(1) Among the six tree structures, there are parameter configurations that achieve higher recognition rates than the baseline. Except for structure L3 : {1, 16, *}, the other tree structures all can achieve the best performance, where EER is 2, and relatively improves 7.5% compared to the baseline system.

(2) As shown in Fig. 6(e), among all tree configurations that achieve the optimal recognition performance with EER = 2, structure L5 : {1, 8, 4, 4, *} experiences the greatest reduction in the actual computational reduction factor, with F′ = 42.93. At this point, the beam-width K = 1 and the recall rate ξ = 0.1.

(3) Within the same tree configuration, when ξ = 0.05, the number of nodes recalled per layer is 1 ∼ 2. In comparison to xi = 0, there is a significant improvement in recognition performance. As illustrated in Fig. 6(d), when ξ = 0.05, the EER can decrease by up to around 50%, while the reduction in Gaussian number is only around 6.2%.

(4) After recalling a portion of the nodes, when ξ > =0.1, most tree structures can achieve the best accuracy, and the EER remains unchanged, indicating that the recognition performance tends to be stable.

4.3 Robustness under different audio durations

In this section, the robustness of the speaker verification system under different speech length t = {30, 15, 9, 3}s is discussed. To ensure the consistency and integrity of the speech, all the test audios are spliced from the records in the dataset. The enroll audio duration of each speaker remains 30s.

According to the experimental results in the previous section, the TBKS algorithm achieved the best performance under L4 : {1, 32, 8, *}, K = 2. For the improved TBKS algorithm, three configurations with better performance are selected according to different heights of the tree: L5 : {1, 8, 4, 4, *}, L4 : {1, 32, 8, *}, L3 : {1, 32, *}, where K = 1, ξ = 0.1.

Fig. 7

Performance under different audio durations.

As shown in Fig. 7, the improved TBKS algorithm leads to a decrease in EER for various tree structures. For any speech length, there is at least one optimal tree configuration with EER lower than that of the traversal search or the original TBKS algorithm. Notably, the structure L4 : {1, 32, 8, *} maintains the best recognition performance across different speech lengths. When t = 3s, the accuracy of all algorithms decreases greatly, indicating that short test audio is a big bottleneck in the application of speaker verification applications.

4.4 Robustness under simple noises

In this section, we investigate the noise robustness of TBKS after the introduction of recall rate in two common noisy environments. Two sets of testing conditions with babble noise and factory noise from the NoiseX-92 database is evaluated at 15dB, 30dB, 50dB SNR. Also, the duration of speech has been considered (i.e. 30s, 15s, 9s, and 3s). According to the experimental results in Section 4.3, the TBKS algorithm is still set as L4 : {1, 32, 8, *}, K = 2, and the improved TBKS algorithm is set as L4 : {1, 32, 8, *}, K = 1, ξ = 0.1.

Fig. 8

Performance under different noises and audio durations.

The performance is shown in Fig. 8, and it can be seen that:

(1) When the speech length is t = 30s, under different SNRs and noise types, the improved TBKS algorithm demonstrates a notable decrease in EER compared to both traversal search and the original TBKS algorithm.

(2) In a total of 24 testing conditions, the TBKS algorithm with recall rate has the highest accuracy for 19 times, and the lowest accuracy for 2 times, which indicates that the improved algorithm can also remain robust in complex environments.

(3) Compared with Factory noise, the improved TBKS algorithm is more robust to Babble noise and maintains the best accuracy.

5 Conclusions

Based on the tree-structured GMM-UBM speaker verification system, we optimize the TBKS kernel selection algorithm to enhance recognition performance. During the kernel selection process at each layer of the tree structure, a portion of important nodes outside the beam width are recalled, resulting in aggregated nodes used as kernels, thereby improving the modeling of speaker acoustic space. The improved algorithm is validated on the Aishell2 dataset, investigating the influence of node information within the tree structure on recognition speed and accuracy. Furthermore, experiments are conducted under various speech lengths, noise types, and signal-to-noise ratios to examine the robustness of the enhanced TBKS algorithm.

With the introduction of recall rate, a smaller number of lower-resolution nodes are utilized to represent the less crucial acoustic space, ensuring that kernels cover the local acoustic space where speech frames are located as much as possible. When the recall rate is set to 0, it corresponds to the TBKS algorithm. While the introduction of recall rate slightly increases the number of kernels, it significantly improves recognition accuracy. For instance, in the tree structure L4 : {1, 32, 8, *}, when the recall rate is set to 0.05, compared to the TBKS algorithm, the number of kernels increases by 1-2, while the EER decreases by around 50%. Additionally, in comparison to traditional traversal search, the improved TBKS algorithm notably accelerates the kernel selection process, resulting in a maximum reduction factor of gaussian components up to 42.93.

The improved TBKS algorithm also enhances the robustness of the speaker verification system. Across various combinations of speech lengths (t = 30, 15, 9, 3s), noise types (Babble and Factory), and Signal-to-Noise Ratios (SNR = 50, 30, 15), we compared the recognition accuracy of traversale search, TBKS, and the improved TBKS algorithm. In most scenarios, the improved TBKS algorithm exhibited the highest recognition accuracy. When t = 30s, the improved TBKS algorithm showcased significant performance improvement. However, for t = 3s, the performance of all three algorithms declined, indicating that short speech remains a challenge in speaker verification tasks.

References

Warren Liao

, Clustering of time series data–a survey, Pattern Recognition 38(11) (2005), 1857–1874.

Ravika Naika An overview of automatic speaker verification system, Intelligent computing and information and communication: proceedings of 2nd international conference, 2017.

Reynolds

D.A.

An overview of automatic speaker recognition technology, IEEE international conference on acoustics, speech, and signal processing, 2002.

Reynolds

D.A.

, Quatieri

T.F.

and Dunn

R.B.

, Speaker verification using adapted Gaussian mixture models, Digital Signal Processing 10 (2000), 19–41.

Moon

Y.S.

, Leung

C.C.

, Pun

K.H.

Fixed-point GMM based speaker verification over mobile embedded system, Proceedings of the 2003 ACM SIGMM workshop on Biometrics methods and applications, 2003.

Tianxiang Chen , Elie Khoury Speaker embedding conversion for backward and cross-channel compatibility, IEEE international conference on acoustics, speech and signal processing (ICASSP), 2022.

Zhang

, Zheng

, Xu

Data-driven Gaussian component selection for fast GMM-based speaker verification, Twelfth Annual Conference of the International Speech Communication Association, 2011.

McLaughlin

, Reynolds

D.A.

, Gleason

A study of computation speed-ups of the GMM-UBM speaker recognition system, Sixth European conference on speech communication and technology, 1999.

and Li

, Speaker Validation Based on Clustering and Sorted Gaussian Mixture Model, Jisuanji Gongcheng/Computer Engineering 37(23) (2011), 162–164.

10.

Auckenthaler

, Mason

J.S.

Gaussian selection applied to text-independent speaker verification, A Speaker Odyssey-The Speaker Recognition Workshop, 2001.

11.

Saeidi

, Kinnunen

, Mohammadi

H.R.S.

, Rodman

, Fränti

Joint frame and Gaussian selection for text independent speaker verification, In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 2010.

12.

Xiang

, Berger

Structural Gaussian mixture models for efficient text-independent speaker verification, INTERSPEECH, 2002.

13.

McClanahan

, De Leon

Towards a More Efficient SVM Supervector Speaker Verification System using Gaussian Reduction and a Tree-Structured Hash, Technical Report, Sandia National Lab, Albuquerque, NM (United States), 2013.

14.

McClanahan

and De Leon

P.L.

, Reducing computation in an i-vector speaker recognition system using a tree-structured universal background model, Speech Communication 66 (2015), 36–46.

15.

Zheng

, Xu

Data-driven tree structure based UBM reconstruction for speaker verification, The 9th International Symposium on Chinese Spoken Language Processing, 2014.

16.

Gaka

and Jaciów

, Speaker identification using fuzzy i-vector tree, Journal of Intelligent & Fuzzy Systems 37(4) (2019), 4937–4949.

17.

Shinoda

and Lee

C.H.

, A structural Bayes approach to speaker adaptation, IEEE Transactions on Speech and Audio Processing 9(3) (2001), 276–287.

18.

Xiang

, Berger

Efficient text-independent speaker verification with structural Gaussian mixture models and neural network, IEEE Transactions on Speech and Audio Processing, 2003.

19.

Xiong

, Zheng

, Tomas

, Song

, Soong

and Wu

, A tree-based kernel selection approach to efficient Gaussian mixture model–universal background model based speaker identification, Speech Communication 48(10) (2006), 1273–1282.

20.

Xiong

, Zheng

and Song

, Tree-structure-universal background model based efficient speaker identification, Journal-Tsinghua University 46(7) (2006), 1305.

21.

Reynolds

D.A.

Comparison of background normalization methods for text-independent speaker verification, Fifth European Conference on Speech Communication and Technology, 1997.

22.

Xiong

, Zheng

T.F.

, Song

, Wu

Combining selection tree with observation reordering pruning for efficient speaker identification using GMM-UBM, IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005.

23.

, Na

, Liu

, Bu

Aishell-2: Transforming mandarin asr research into industrial scale, arXiv preprint arXiv:1808.10583, 2018.

24.

Furui

, Cepstral analysis technique for automatic speaker verification 29(2) (1981), 254–272.