Optimization learning of hidden Markov model using the bacterial foraging optimization algorithm for speech recognition

Abstract

Nowadays, the speech recognition applications can be found in several activities, and their existence as a field of study and research lasts for a long time. Although, many studies deal with different problems, in security-related areas, biometric identification, access to the Smartphone… Etc. In automatic speech recognition (ASR) systems, hidden Markov models (HMMs) have widely used for modeling the temporal speech signal. In order to optimize HMM parameters (i.e., observation and transition probabilities), iterative algorithms commonly used such as Forward-Backward or Baum-Welch. In this article, we propose to use the bacterial foraging optimization algorithm (BFOA) to enhance HMM with Gaussian mixture densities. As a global optimization algorithm of current interest, BFOA has proven itself for distributed optimization and control. Our experimental results show that the proposed approach yields a significant improvement of the transcription accuracy at signal/noise ratios greater than 15 dB.

Keywords

Automatic speech recognition acoustic information bacterial foraging optimization algorithm BFOA/HMM Gaussian mixture densities Baum-Welch

1. Introduction

Speech recognition is the capacity to listen to talked words and distinguish different sounds exhibit in it, recognize them as words of some known language. In the computer area, the Speech recognition is might be characterized as the capacity of the computer framework to acknowledge talked words in the sound configuration like wav and afterward create its substance in the content arrangement. Automatic speech recognition (ASR) [1] simulates human listening, it transforms speech into text.

This sort of conversion should be independent of vocabulary size, accent, speaker characteristics such as male or female etc. A more technical definition is given by Jurafsky, where he defines ASR as the building of system for mapping acoustic signals to a string of words. He continues by defining automatic speech understanding (ASU) as extending the goal to producing some sort of understanding of the sentence. Speech recognition is basically a pattern recognition problem. This involves extracting features from the input signal waves and classifying them to classes using pattern matching model. Performance of ASR system is measured on the basis of recognition accuracy, complexity, and robustness. Advantages of automatic speech recognition are accessibility for the deaf, cost reduction through automation, searchable text capability.

The problem of ASR is to program a computer to take digitized speech samples and print the words that a human would recognize when listening to the same sound. ASR has grown roughly in proportion to other areas of pattern recognition because of the desire to invent the machine capable of making complex decisions, and, practically, one that could function as swiftly as humans.

Although we have learned a great deal about how to build practical and useful speech recognition systems, there remain a number of fundamental questions about the technology to which we have no definitive answers. Unmistakably the speech signal is a standout amongst the most complex signals that we have to manage. It is created by a human’s vocal system and in this way difficult to be described by a straightforward 2-dimensional model of sound spread. While there exist various advanced numerical models which endeavor to simulate the speech production system, their demonstrating capacity is as yet restricted.

In this work, we investigate the effectiveness of using the Bacteria Foraging Optimization Algorithm (BFOA) for optimizing the structure and parameter learning of HMM. The BFOA, proposed by Passino [2], is a newcomer to the family of nature-inspired optimization algorithms. For over the last five decades, optimization algorithms like Genetic Algorithms (GAs) [3, 4], Evolutionary Programming (EP) [5], Evolutionary Strategies (ES) [6], which are inspired by evolution and natural genetics, dominated optimization algorithms field. Recently natural swarm inspired algorithms like Particle Swarm Optimization (PSO) [7], Ant Colony Optimization (ACO) [8] have found their way in this research area and proved their effectiveness. Following the same trend of swarm-based algorithms, Passino proposed the BFOA [2]. Furthermore, it is possible to design operators that favor biologically plausible changes to the structure of an HMM. That is, to ensure that modules of the states are kept intact. The objective of this work is to propose algorithms that improve this quality. The criterion used to quantify the quality of HMM is the probability that a given model generates a given observation. To solve this problem, we use as we have already mentioned a BFOA hybridization with HMM.

This paper is organized as follows, the first section is an introduction, the second, in which work motivations are exposed, is an overview of previous related works. The third section presents the proposed approach, the ASR system based BFOA/HMM and details about the used method. The experimental results of the proposed method are discussed in section four. Finally, section five concludes the paper with some perspectives.

2. Related work

To our best learning, the point of the machine and human interaction is to utilize the most regular method of communicating, through our speech. The execution of the programmed ASR systems depends on traditional HMMs which depend on maximum likelihood estimation (MLE) systems. Different models have been investigated by specialists like neural and Bayesian systems, discriminative training strategies, state duration modeling and the use of support vector machines with HMM.

The most punctual endeavors to devise systems for ASR by machine were made in the 1950’s. In 1952’s, at Bell Laboratories, Biddulph, and Balashek [9] fabricated a system depended heavily on measuring spectral resonances during the vowel region of each digit. In 1970’s speech recognition research achieved various huge turning points. To start with, the isolated word or discrete utterance recognition became a viable and usable technology based on the fundamental studies by Velichko and Zagoruyko in Russia [10, 11, 12].

In recognizing syllables or isolated words, the human auditory systems perform above chance level already at $-$ 18dB signal-to-noise ratio (SNR) and significantly above it at $-$ 9dB SNR. No ASR system is able to achieve performance close to that of human auditory systems in recognizing isolated words or phonemes under severe noisy conditions, as has been confirmed recently in an extensive study by Sroka [13].

Norris in 2008’s [14] presented a Bayesian model of continuous speech recognition, which is based on shortlist and shared many of its key assumptions: parallel competitive evaluation of multiple lexical hypotheses, phonologically abstract pre-lexical and lexical representations, feed-forward architecture with no online feedback, and a lexical segmentation algorithm based on the viability of chunks of the input as possible words.

Neural network models are powerful speech recognition engines [15]. Their ability to classify data and ability in parallel processing pave the way for speech recognition. A typical neural network consists of the input layer, hidden layer, output layer. Input layer receives the input signal and transfers the data to the hidden layer. Hidden layer computes the action function and all the necessary calculations are done in this layer. After computations output is transferred to the output layer. Graves in 2013’s [16] investigated deep recurrent neural networks (RNNs), which combined the multiple levels of representation, that have proved so effective in deep networks with the flexible use of long-range context that empowers RNNs. When they trained end-to-end with suitable regularization, they found that deep Long Short-term Memory of RNNs achieve a test set error of 17.7% on the TIMIT phoneme recognition benchmark.

Support Vector Machines (SVM) supervised learning models with associated learning algorithms. They are used for classification and regression analysis. They analyze the data and recognize patterns. Text-independent speaker recognition uses as their features a compact representation of a spoken utterance, known as i-vector [17]. Rather than estimating an SVM model per speaker, according to the one versus all discriminative paradigms, the Pairwise Support Vector Machine (PSVM) approach classifies a trial, consisting of a pair of i-vectors, as belonging or not to the same speaker class. Training a PSVM with a large amount of data, however, is a memory and computationally expensive task.

Generalized variable Hidden Markov Model (GVP-HMM) is used for speech recognition in a noisy environment [10]. A crucial task of automatic speech recognition systems is to robustly handle the mismatch against a target environment introduced by external factors such as environmental noise. When these factors are of time-varying nature, this problem becomes seven more challenging.

The used language in this study is Arabic. It is well known that Arabic is the fifth most widely spoken language in the world with approximately 300 million native speakers, cutting across a wide geographical area from North Africa to the Middle East. It is also one of the six official languages adopted in the United Nations and represents the official language in some twenty-two countries, whereas there are substantial Arabic-speaking communities in many countries. Arabic is also the liturgical and worship language for more than one billion and a half Muslims worldwide [18].

Moreover, many challenges are faced by Arabic speech recognition. For example, Arabic has short vowels which are usually ignored in the text, which add more confusion to the ASR decoder. Additionally, Arabic has many dialects where words are pronounced differently. Elmahdy and Gruhn in [19]summarized the main problems in Arabic speech recognition, which include Arabic phonetics, discretization problem, grapheme-to-phoneme relation, and morphological complexity. Bourouba in 2006’s [20] presented a new HMM/support vectors machine (SVM) (k-nearest neighbor) for the recognition of isolated spoken words. Sagheer in 2005’s [21] proposed a novel visual speech features representation system. They used it to comprise a complete lip-reading system. While Muhammad in 2011’s [22] evaluated conventional ASR system for six different types of voice disorder patients speaking Arabic digits. Mel-frequency cepstral coefficients (MFCCs) and Gaussian mixture model GMM/HMM are used as features and classifier, respectively. The recognition result is analyzed for types of diseases.

3. Asr system based BFOA/HMM

The structure of an ASR system is divided into three modules according to their role. The first module is feature extraction to capture the most relevant and discriminate characteristics of the signal to recognize. The second subsystem is the training module, whose function is to create the knowledge about the speech and language to be used in the system. The final module is the recognition one, whose function is tried to figure out the meaning of the input speech given in the testing phase.

Figure 1 shows the different stages in the processes of learning and recognition of the proposed system. Each element in this figure will be described further.

Figure 1.

An overall architecture of an ASR system.

Figure 2.

Steps of RASTA-PLP analysis.

3.1 Pre-processing and features extraction

The audio features extraction consists of using pre-processing techniques such as signal enhancement and environment sniffing to help preparation of the incoming audio stream for the features extraction step [23]. Generally, the audio is broken down into sliding window “frames” and features are extracted from each frame. In many cases, the frame size is on the order of 10 ms. In the literature, many results are reported about the audio features extraction for clean and noisy speech conditions. The MFCCs and linear prediction coefficients (LPCs) are the most commonly used acoustic features and additional research is ongoing in the field of noise robust acoustic features.

After the acoustic feature extraction step, the first and second derivatives of the data are usually concatenated with the original data to form the final features vector. The original data is also known as the static coefficients while the first and second derivatives are known as delta and delta-delta or acceleration coefficients.

The frequency response of the communication channel influences easily most speech parameter estimation techniques. For this reason, Hermansky in 1991’s [24] developed a technique that is more robust to such steady-state spectral factors in speech. This approach is conceptually simple and computationally efficient.

The proposed method for speech analysis is based on Relative-Spectral Analysis-Perceptual Linear Predictive method (RASTA-PLP) for feature extraction. This technique is an improvement of the traditional PLP method, which consists in a special filtering of the different frequency channels of a PLP analyzer. The previous filtering is done to make speech analysis less sensitive to the slowly changing or steady-state factors in speech. The RASTA method replaces the conventional critical-band short-term spectrum in PLP and introduces a less sensitive spectral estimation. For the RASTA-PLP features, an additional filtering is applied after decomposition of the spectrum into critical bands. This RASTA filter suppresses the low modulation frequencies which are supposed to stem from channel effects rather than from speech characteristics.

The process of computing RASTA-PLP coefficients is showed in Fig. 2.

Concerning the filter, the transfer function is denoted by Eq. (1):

$\displaystyle H\left(z\right)=0.1z^{4}\times\frac{2+z^{-1}-z^{-3}-2z^{-4}}{1-0% .98z^{-1}}$ (1)

The low cut-off frequency of the filter determines the fastest spectral change of the log spectrum. The high cut of frequency indicates the fastest spectral change which is preserved in the output parameters. The higher values of the band-pass filter attenuate the convolution noise. The RASTA filter has a long time constant of the integration that is of 500 ms for the particular filter of Eq. (1). It means that the current analysis result depends on previous outputs stored in the memory of the recursive RASTA filter. Thus, the analysis results depend on the time in where the analysis starts. The PLP and RASTA-PLP streams were augmented by their delta features [24].

In this work, the sampled speech data are segmented into 0.025 seconds frames with a 0.010 seconds overlapping of two consecutive windows. After application of a Hamming window, each of these frames is analyzed using the RASTA-PLP speech analysis technique, in order to make speech analysis more robust to spectral distortions.

3.2 Proposed method

The well-known Baum-Welch algorithm can be used to simultaneously estimate the state transition probabilities and the observation probabilities in a maximum likelihood framework from the sequence of observations. The states in HMM are assumed as discrete values. However, the observations can have either discrete (limited) values, modeled with probabilities in matrix B, or continuous (or continuous discretized), modeled by conditional probability density functions.

We applied BFOA to optimize the Baum-Welch algorithm in left-to-right HMM. The results between the HMM system and hybrid BFOA/HMM were compared to analyze how BFOA can improve the recognition rate in hybrid GA/HMM.

3.2.1 Hidden Markov Model (HMM)

Speech recognition systems generally assume that the speech signal is a realization of some message encoded as a sequence of one or more symbols. To perform the reverse operation of recognizing the underlying symbol sequence given a spoken utterance.

In this study, we used the Hidden Markov Models (HMM) [25], which have emerged as the predominant technology in speech recognition filed in recent years. These model shave proved to be the most appropriate to the speech recognition problems.

The HMMs are a class of models in which the distribution that generates an observation depends on the state of an underlying but unobserved Markov process [26]. Thereby, HMM is a combination of two processes, namely a Markov chain which determines the state at time $t$ , $S_{t}=s_{t}$ , and a state-dependent process which generates the observation $X_{t}=x_{t}$ depending on the current state $s_{t}$ . In most cases, a different distribution is imposed for each possible state of the state space. A hidden Markov Model is characterized by:

•
$S$ : the set of states.
•
$O$ : the sequence of observations, each of which is drawn from a vocabulary, $<o_{1},o_{2},\ldots,o_{T}>$ .
•
$N$ is the number of states in the model.
•
$M$ is the number of mixtures in the random function.
•
$\pi={\{}\pi_{i}{\}}$ : initial state distribution

$\displaystyle\pi_{i}=P\left({q_{1}=i}\right),1\leqslant i\leqslant N$ (2)
•
$A={\{}a_{ij}{\}}$ : the transitional probability matrix

$\displaystyle a_{ij}=P\left({q_{t+1}=j|q_{t}=i}\right),$ (3) $\displaystyle 1\leqslant i,j\leqslant N,1\leqslant t\leqslant T$
•
$B={\{}b_{j}\left(o\right){\}}$ : the probability distribution at each state, where $b_{i}(o_{t})$ is the probability of observation $o_{t}$ generated from state $i$ .

The HMM outputs are modeled as continuous density output probabilities by representing each output probability as a Gaussian mixture. Thus, the output probability for an HMM in state $j$ is given by:

$\displaystyle b_{j}(o)=\sum_{k=1}^{M}w_{ji}N(o,\mu_{jk},\sigma_{jk}^{2})$ (4)

Here, the sum is over the mixtures ( $k$ ), where $N(o,\mu_{ji},\sigma_{ji}^{2})$ denotes a Gaussian distribution for observation vector $o$ with mean vector $\mu_{jk}$ and variance matrix $\sigma_{ji}^{2}$ and $w_{ji}$ are the mixture weights for each state.

As the probability distribution for the output probabilities must integrate to one,

$\displaystyle\sum_{k=1}^{M}w_{ji}=1$ (5)

There are three (03) fundamental problems related to the HMM; the evaluation problem, the problem of determination of the path of states and the problem of learning (supervised or unsupervised). There are other problems related to HMM such as the overflow of representation of numbers in the machine, insufficient data for learning, the update of models when processes vary in time, the choice of HMM architecture the best adapted to data, the choice of a good initial estimate of the probabilities of the HMM. The Forward algorithm allows the calculation of the likelihood of the observations, the Viterbi algorithm finds the optimal path which is the most likely to follow and the Baum-Welch algorithm performs the supervised learning (re-estimate the parameters of HMM) [24].
3.2.2 Bacterial Foraging Optimization Algorithm (BFOA)

Bacteria Foraging Optimization Algorithm (BFOA) developed by Liu and Passino [27] is a generally new populace based calculation; it is a nature-inspired meta-heuristic calculation, which emulates the foraging behavior of E. coli found in a human digestive tract.

As indicated by scrounging hypothesis, the bacteria need to search for the nutrients and achieve the nutrients in such a manner that energy consumption per unit time is minimized.

Table 1
Description of parameters

Parameter name	Description
$p$	The dimension of the search space
$S$	Total number of bacteria in the population
Nc	The number of chemo-tactic steps
Ns	The swimming length
Nre	The number of reproduction steps
Ned	The number of elimination-dispersal events
Ped	Elimination-dispersal probability
$C(i)$	The size of the step taken in the random direction specified by the tumble

A gathering of bacteria forages for the nutrient region and flag others about the area of the supplements with the assistance of chemical attractants. Likewise, they help other creature to keep away from the poisonous locales by generating chemical repellents. The way of finding the regions of high nutrients by the bacteria can be modeled as an optimization process. BFOA has been applied in various optimization problems associated with the real-world and therefore already gained the attention of the researchers in the domain.

The swarm of microorganisms “Bacteria” S acts as takes after [2]:

•

Bacteria are discretionarily spread in the map of nutrients.

•

Bacteria travel towards high-nutrient regions on the map. Those get their nourishment inadequate, expanded long and will split into two equal parts at a reasonable temperature. Besides those situated in a low nutrient district will scatter and the individuals who located in a toxic area will die.

•

Bacteria located in the promising districts of their environment will attempt to pull in others microorganisms by producing chemical attractants.

•

Bacteria are situated in the most elevated nutrient districts.

•

Bacteria scatter as to search for new nutrient districts in their environment.

Bacteria foraging behavior searching conduct incorporates four primary advances: Chemo-taxis (tumble and swimming), swarming, Reproduction and Elimination-dispersal.

3.2.3 The proposed BFOA/HMM training

In this study, we explore the application of BFOA in order to optimize the learning of the HMMs. As other swarm intelligence algorithms, BFOA is inspired from the social and cooperative actions found in nature. In fact, the optimization process here consists in the way bacteria rummage around for the high nutrient regions. The application of BFOA is various in real-world optimization problems and therefore gained the attention of the researchers in the field.

Suppose that we want to find the minimum of $J(\theta)$ where $\theta\in\Re^{p}$ (i.e. $\theta$ is a $p$ -dimensional vector of real numbers), and no measurements or an analytical description of the gradient $\nabla J(\theta)$ is available. The four principal mechanisms observed in a real bacterial system are imitated by BFOA, which are: chemo-taxis, swarming, reproduction, and elimination-dispersal to solve this non-gradient optimization problem.

The definition of a chemotaxis step to be a tumble followed by a tumble or a tumble followed by a run is as follows:

Let $j$ be the index for the chemotaxis step, $k$ the index for the reproduction step, and $l$ the index of the elimination-dispersal event. The parameters are described in the Table below:

Let $P(j,k,l)={\{}\theta^{i}(j,k,l)|i=1,2,\ldots,S{\}}$ represent the position of each member in the population of the $S$ bacteria at the $j^{\text{th}}$ chemo-tactic step, $k^{\text{th}}$ reproduction step, and $l^{\text{th}}$ elimination-dispersal event.

Here, $J(i,j,k,l)$ denote the cost at the location of the $i^{\text{th}}$ bacterium $\theta^{i}(j,k,l)\in\Re^{p}$ the indices are dropped and refer to the $i^{\text{th}}$ bacterium position as $\theta^{i}$ ). Note that we will interchange ably refer to $J$ as being a “cost” (using terminology from optimization theory) and as being a nutrient surface (in reference to the biological connections). A brief description of the four prime steps in BFOA is presented below:

Chemo-taxis

Biologically the movement of E. Coli bacterium is restricted into two different ways, either it can swim for a period of time in the same direction or it may tumble for the entire lifetime. In the BFOA, a unit step in random direction represents a “tumble” and a unit step with the same direction in the last step indicates a “run”, which simulates the chemo-taxis procedure is used in Eq. (6). $\theta_{i}^{j}$ presents the position of the $i^{\text{th}}$ bacterium in the $j^{\text{th}}$ chemo-taxis step, $C(i)$ is the step length during the $j^{\text{th}}$ chemo-taxis and $\emptyset(i)$ is a unit vector which stands for the swimming direction after a tumble. It can be generated by Eq. (7), where $\Delta i$ is a randomly produced vector with the same dimension of the problem:

$\displaystyle\theta_{i}^{j+1}=\theta_{i}^{j}+C(i).\emptyset(i)$ (6) $\displaystyle\emptyset(i)=\frac{\Delta(i)}{\sqrt{\Delta^{T}(i)\Delta(i)}}$ (7)

In each chemotaxis step, the bacterium generated a tumble direction firstly. Then the bacterium moves in the direction using Eq. (6). If the nutrient concentration in the new position is higher than the last position, it will run one more step in the same direction. This procedure continues until the nutrient get worse or the maximum run step is reached. The maximum run step is controlled by a parameter called Ns.

Swarming

Interesting group behavior has been observed for several motile species of bacteria including E. coli and S. Typhimurium, where intricate and stable spatiotemporal patterns (swarms) are formed in the semisolid nutrient medium. A group of E. Coli cells arranges themselves in a travelling ring by moving up the nutrient gradient when placed amidst a semisolid matrix with a single nutrient chemo-effecter. The cells when stimulated by a high level of succinate, release an attractant aspartate, which helps them to aggregate into groups and thus move as concentric patterns of swarms with high bacterial density.

BFOA simulates this social behavior by representing the combined cell-to-cell attraction and repelling effect. Now combined cell-to-cell attraction and repelling effects can be modeled as:

$\displaystyle J_{cc}(\theta,P(j,k,l))=\sum_{i=1}^{s}J_{cc}^{i}(\theta,\theta^{% i}(j,k,l))J_{cc}(\theta,P(j,k,l))=\sum_{i=1}^{s}\left[-d_{\textit{att}}\exp\!% \left(\!-w_{\textit{att}}\!\sum_{m=1}^{p}(\theta_{m}-\theta_{m}^{2})^{2}\!% \right)\right]+\sum_{i=1}^{s}\left[-d_{\textit{rep}}\exp\!\left(\!-w_{\textit{% rep}}\exp\!\sum_{m=1}^{p}(\theta_{m}-\theta_{m}^{i})^{2}\!\right)\right]$ (9)

where $J_{cc}(\theta,P(j,k,l))$ is the objective function value to be added to the actual objective function (to be minimized) to present a time-varying objective function, $S$ is the total number of bacteria, $p$ is the number of variables to be optimized, which are present in each bacterium and $\theta=(\theta_{1},\theta_{2},\ldots,\theta_{p})^{T}$ is a point in the $p$ -dimensional search domain. $d_{\textit{att}}$ , $w_{\textit{att}}$ , $h_{\textit{rep}}$ , $w_{\textit{rep}}$ are different coefficients that should be chosen properly [2, 27].

Reproduction

For every $N_{c}$ time of chemo-tactic steps, a reproduction step is taken in the bacteria population. The bacteria are sorted in descending order by their nutrient obtained in the previous chemo-tactic processes. Bacteria in the first half of the population are regarded as having obtained sufficient nutrients so that they will reproduce. Each of them splits into two (duplicate one copy in the same location). Bacteria in the residual half of the population die, and they are removed out from the population. The population size remains the same after this procedure. For bacterial, a reproduction step takes place after all chemo-tactic steps. Now health of $i^{\text{th}}$ bacterium:

$\displaystyle J_{he}^{i}=\sum_{j=1}^{N_{c}+1}J(i,j,k,l)$ (10)

Elimination and dispersal

In nature, the changes of environment where population lives may affect the behaviors of the population. For example, the sudden change of temperature, nutrient concentration and the flow of water. All these may cause bacteria in the population to die or move to another place. To simulate this phenomenon, eliminate-dispersal is added in the BFOA. After every $N_{re}$ time of reproduction steps, an eliminate-dispersal event happens. For each bacterium, a random number is generated between 0 and 1. If the random number is less than a predetermined parameter, known as $P_{e}$ , the bacterium will be eliminated, and a new bacterium is generated in the environment. In BFOA, the dispersion event happens after a certain number of reproduction processes and some bacteria are chosen depending upon the probability to be killed and moved to another position within the environment [27, 28].

Generally, in the problem of HMM learning, the individuals are HMM. It is necessary to encode the HMM tore present the bacterium at $j^{\text{th}}$ chemo-tactic on which the BOFA operators will be applied. In the second case, the BFOA is used to calculate the $w$ , $\mu$ and $\sum$ parameters.

The population of size $S$ was created randomly. The most natural encoding is to build the bacteria by reorganizing all the coefficients of the HMM. The simplest way is to juxtapose all rows of all matrices. Thus, a coding in real numbers is obtained while respecting the constraints related to the HMM.

3.3 Recognition

A discriminated model is used to do Recognition, where learning will be associated with each word learned in HMM. Recognition will be done by calculating for each known HMM its probability of generating the word to recognize. The word will be recognized that the associated HMM obtained a maximum score.

The HMM who has the highest probability of generating the input data is selected in decision stage. In our experiment, the decision is performed using the Viterbi algorithm.

The Viterbi algorithm is used to find the best state sequence $q_{1}^{*},q_{2}^{*},\ldots,q_{T}^{*}$ given the observation sequence $Q_{1},Q_{2},\ldots,Q_{T}$ . The highest probability along a single path, which accounts for the first $k$ observations and ends in $S_{i}$ at time $k$ , is defined as:

$\displaystyle\delta_{k}(i)=\max_{q_{1},q_{2},\ldots,q_{k-1}}$

(11) $\displaystyle\quad P[q_{1}q_{2}\ldots q_{k}=S_{i},O_{1},O_{2},\ldots,O_{k}|\lambda]$

It can be induced that:

$\displaystyle\delta_{k+1}(j)=\max_{i}[a_{ij}\delta_{k}(i)_{1}].b_{j}(O_{k+1})$ (12)

The best state sequence can be retrieved by keeping track of the argument that maximizes the previous equation for each $k$ and $j$ . The observation probability distribution $b_{j}(O_{k})$ is a Gaussian mixture likelihood function as mentioned above.
4. Experimental results and discussion

The performance analysis of various speech recognition systems is evaluated. Speech recognition has a big potential in becoming an important factor of interaction between human and computer. A successful speech recognition system has to determine features not only present in the input pattern at one point in time but also features of input pattern changing over time. The most common performance measure is word error rate. This measure is computed by comparing a reference transcription output by the speech recognizer. From this comparison it is possible to compute the number of errors, which typically belongs to three categories:

Insertions I (when in the output of ASR it is present a word not present in reference), deletions D (a word is missed in ASR output) and substitutions S (a word is confused with another word).

4.1 Database

In order to test the proposed system, a multi-speakers database was used for a speech recognition task. Therefore, we used the AVARB database [26]. It contains pronunciations of Arabic words isolated, taken at a sampling frequency of 16 KHz.

Video data were captured using a digital camera with a 690 $\times$ 340 pixel resolution at 30 frames/s frame rate with variations of pose (profile view and front view) and saved in AVI format. The database is collected from 18 speakers (2 female and 16 male), these speakers are from different regional dialects, and each speaker pronounces each word 9 times (Arabic numerals from zero to nine) with different modes of pronunciation (normal, slow and fast). The distance between the camera and the speaker as well the luminance are adjustable to add more diversification in the audio and the visual streams during the learning, the average distance is to 16.5 cm. In our basic corpus which contains only isolated words, the size of each record is 2 seconds which is enough time to utter a word slowly in Arabic.

4.2 Experimental results

As a standard procedure in evaluating machine learning techniques, the dataset is split into training and test sets. Therefore, we have used 2/3 of the data for the learning stage and there maining 2/3 to test the effectiveness of our ASR system.

The proposed ASR system that uses RASTA-PLP method as audio features, and a BFOA/HMM for the speech modeling, was implemented as described in the previous sections. The whole programming was implemented in MATLAB. As a result, we obtain a matrix of 27 parameters by integrating the first and second derivative of the parameters. The BFOA/HMMs recognizers were built using the Hidden Markov Model Toolkit (HTK) [29].

Various kinds of the instance with different BFOA control parameters have been solved with our algorithm, in order to evaluate the performance of the proposed system, we have compare it with the classical training by the baum-Welch method. Each instance is run 15 times with a different number of the mixture; also the used maximum number of iteration for EM algorithm is 40. In the tables below, the obtained maximum average $P(o|\lambda)$ values after 150 generations are listed as follows:

Table 2
BFOA parameters training HMM for AVARB database

$S$	$N_{c}$	Number of stats	Number of mixtures	Average $P(o\|\lambda)$ HM	Average $P(o\|\lambda)$ BFOA/HM
20	15	3	7	$-$ 91.1111	$-$ 110.6788
20	21	5	4	$-$ 89.0189	$-$ 101.1718
20	23	7	8	$-$ 89.2434	$-$ 101.1955
20	25	7	7	$-$ 96.0007	$-$ 104.2815
20	30	5	6	$-$ 96.4333	$-$ 107.0122
20	35	7	6	$-$ 98.1012	$-$ 107.8234

We can clearly notice from Table 2 that the results are varied according to the parameters training of the BFOA, also to the number of stats and the number of mixture. However, with $S=$ 20, $N_{c}=$ 21 and $M=$ 4, the average maximum likelihood is the highest one on the both case. Therefore, these parameters value were chosen to be used in the proposed BFOA/HMM.

To estimate the reliability of this system we must be calculated recognition rate that will allow us to evaluate the performance of each algorithm in order to compare them. For that, we choose an additional model which is the multilayered perceptron (MLP) neural networks. We use an MLP with a single hidden layer, and the input layer consists of X neurons, such that X represents the resulting classes of clustering.

Figure 3.

Influence of the number of chemo-tactic steps on the recognition rates of the ASR system.

The net is trained by at first choosing little random weights and interior limits, and displaying all training data repeatedly. Weights are adjusted after every trial until converge and the cost function is reduced to an acceptable value.

The activation functions used were a hyperbolic tangent for the hidden layer and sigmoid for the output layer. The hyperbolic tangent function is given by:

$\displaystyle f(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$ (13)

The sigmoid function is given by:

$\displaystyle f(x)=\frac{1}{1+3^{-x}}$ (14)

After several tests over the training and test data, we find that the fitness function in BFOA is better with 21 iterations, which yields better results for optimizing HMM parameters. In Fig. 3, we can notice also that the BFOA fitness function increase faster, substantially at the beginning iterations.

In the following, we present our experimental results of the proposed ASR system over a range of noise levels using these three models. In order to simulate various noise conditions, the artificial white Gaussian noise was added.

The experiment was conducted under a mismatched condition, where the recognizers were trained at 20 dB SNR, and acoustic white Gaussian noise, ranging from $-$ 5 dB to 20 dB in steps of 5 dB SNR, was added to the test data.

Figure 4.

Recognition rates of ASR system for varying SNR levels.

We performed evaluation and comparison of the performance of the system by calculating the average recognition rate for each SNR levels, overall utterances for each word. Moreover, the HMM model is trained by using the traditional method (Baum-Welch algorithm). The Fig. 4 shows that in most of the cases, the recognition rates obtained with the proposed BFOA/HMM system provides better results in compare is onto those obtained by the HMM-based system, and the percentage increase amount from 1.4% to 15.3%, along with the increase in the size of the population.

5. Conclusion

Automatic Speech Recognition is viewed as an essential piece of human-computer interfaces that are conceived to utilize discourse, among different means, to accomplish regular, inescapable figuring. The state of ASR lacks robustness, to channel and environment noise continues to be a major impediment.

ASR system takes a human discourse as an information and requires a series of words as yield. The issue of consequently perceiving discourse with the assistance of a computer is a troublesome issue, and the purpose behind this is the unpredictability of the human dialect. Lack of linguistic corpora for dialect language models seems to be a difficulty in implementing an ASR. This stems from trouble in gathering sentences with vernaculars. The extent of the issue ought to be widened into larger vocabularies, continuous speech.

We presented in this paper an approach for an ASR system using BFOA/HMM with a Gaussian mixture density for modeling the Arabic speech. In order to further optimize the solution found by the Baum-Welch algorithm.

Based on the obtained results, we can conclude that the system modeled by HMM and trained by our BFOA/HMM have higher rates of recognition than the HMM trained by the Baum-Welch algorithm. Moreover, the classification results of Baum-Welch are greatly dependent on the initialization of the parameters in contrast to the BFOA, which gives stable results independently of the initial values.

As perspective, we are planning to cover more issues about improving the performance of the system, by enhancing the size of our database and increasing the number of speakers. In addition, we intend to test our system with other alternatives methods of fusion.

References

Ratnadeep

and Abdulmalik

, Automatic Speech Recognition Techniques: A Review, 2nd International Conference on Cognitive Knowledge Engineering, Aurangabad, Maharashtra, India, 2016.

Passino

K.M.

, Biomimicry of bacterial foraging for distributed optimization and control, IEEE Control System Magazine 22(2) (2002), 52–67.

Kaur

Srivastava

and Kumar

, Genetic Algorithm for Combined Speaker and Speech Recognition using Deep Neural Networks, journal of telecommunications and information technology, 2018.

Benmachiche

Bouhadada

Laskri

M.T.

and Zendi

, A dynamic navigation for autonomous mobiles robots, Intelligent Decision Technologies, ISSN 1875-8843 (E) 10(1) (2016).

Loyd

and Amo

, Analysis of independent roulette selection in parallel ant colony optimization, Evolutionary Computation 25(2) (2017).

Saborido

Ruiz

and Luque

, Global WASF-GA: An evolutionary algorithm in multi-objective optimization to approximate the whole pareto optimal front, Evolutionary Computation 25(2) (2017).

Serraji

Didi Omar

and Boumhidi

, Multi swarm optimization based adaptive fuzzy multi agent system for microgrid multi-objective energy management, International Journal of Knowledge-based and Intelligent Engineering Systems 20(4) (2016), 229–243.

Dorigo

and Gambardella

L.M.

, Ant colony system: A cooperative learning approach to the traveling salesman problem, IEEE Transactions on Evolutionary Computation 1(1) (1997), 53–66.

Biddulph

and Balashek

, Automatic recognition of spoken digits, Journal of the Acoustical Society of America 24(6) (1952), 637–642.

10.

Velichko

V.M.

and Zagoruyko

N.G.

, Automatic recognition of 200 words, International Journal of M An-Machine Studies 223(2) (1970).

11.

Iter

Huang

and Jermann

, Generating adversarial examples for speech recognition, Technical Report, 2017.

12.

Panayotov

F.V.

Chen

Povey

and Khudanpur

, Librispeech: An ASR corpus based on public domain audio books, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp, 5206–5210.

13.

Sroka

and Braida

, Human and machine consonant recognition, Speech Comm 45(4) (2005), 401–423.

14.

Norris

and Mcqueen

J.M.

, Shortlist B: A bayesian model of continuous speech recognition, Psychological Review 115(2) (2008), 357–395.

15.

Carlini

and Wagner

, Towards evaluating the robustness of neural networks, In IEEE Symposium on Security and Privacy, 2017.

16.

Graves

Mohamed

A.R.

and Hinton

, Speech recognition with deep recurrent neural networks, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, British Columbia, Canada, 2013, pp. 6645–6649.

17.

Cumani

and Laface

, Large-Scale Training of Pair wise Support Vector Machine for Speaker Recognition, IEEE transactions on audio, speech and language processing, 2014.

18.

Kettani

, 2010 world Muslim population, in: The 8𝑡ℎ Hawaii International Conference on Arts and Humanities, 2008.

19.

Elmahdy

and Gruhn

, Modern standard Arabic based multilingual approach for dialectal Arabic speech recognition, in: Proceedings of the 8𝑡ℎ International Symposium on Natural Language, 2009.

20.

Bourouba

Djemili

Bedda

and Snani

, New hybrid system (supervised classifier/HMM) for isolated Arabic speech recognition, 2nd Information and Communication Technology (ICTTA’06), 2006, 1264–1269.

21.

Sagheer

Tsuruta

Taniguchi

R.I.

and Maeda

, Hyper column model vs. fast DCT for feature extraction in visual Arabic speech recognition, in: Proceedings of the Fifth IEEE International Symposium on Signal Processing and Information Technology, 2005, pp. 761–766.

22.

Muhammad

Almalki

Mesallam

and Farahat

, Automatic Arabic digit speech recognition and formant analysis for voicing disordered people, in: IEEE Symposium on Computers and Informatics (ISCI), 2011, pp. 699–702.

23.

Gupta

Bansal

and Choudhary

, Performance comparison of robust speech recognition using different feature extraction techniques, International Journal of Computer Science and Mobile Computing 8(Issue.1) (2019), 125–130.

24.

Hermansky

Morgan

Bayya

and Kohn

, RASTA-PLP Speech Analisys, ICSI Technical Report TR-91-069, Berkeley, California, 1991.

25.

Rabiner

, A tutorial on hidden markov models and selected applications in speech recognition, in Proceedings of the IEEE 77(2) (1989), 257–286.

26.

Makhlouf

Lazli

and Bensaker

, Structure evolution of hidden markov models for audiovisual arabic speech recognition, International Journal of Signal and Imaging Systems Engineering, IJSISE 9(1) (2016), 55–66.

27.

Liu

and Passino

K.M.

, Biomimicry of social foraging bacteria for distributed optimization, models, principles, and emergent behaviors, Journal of Optimization Theory and Applications 115(3) (2002), 603–628.

28.

Maheshwari

D.K.

, Bacteria in Agrobiology: Plant probiotics. Berlin: Springer. ISBN 978-3-642-27514-2. 2012.

29.

Young

Evermann

Gales

Hain

Kershaw

Liu

X.A.

Moore

Odell

Ollason