Towards voice based prediction and analysis of emotions in ASD children

Abstract

Voice processing has proven to be an eminent way of recognizing the various emotions of the people. The objective of this research is to identify the presence of Autism Spectrum Disorder (ASD) and to analyze the emotions of autistic children through their voices. The presented automated voice-based system can detect and classify seven basic emotions (anger, disgust, neutral, happiness, calmness, fear and sadness) expressed by children through source parameters associated with their voices. Various prime voice features such as Mel-frequency Cepstral Coefficients (MFCC) and Spectrogram are extracted and utilized to train a Multi-layer Perceptron (MLP) Classifier to identify possible emotions exhibited by the children thereby assessing their behavioral state. This proposed work therefore helps in the examination of emotions in autistic children that can be used to assess the kind of training and care required to enhance their lifestyle.

Keywords

Emotion recognition Autism Spectrum Disorder voice processing MFCC

1 Introduction

The recognition of emotions from voice is one of the thought-provoking tasks in the domain of voice processing. There are two classes of features namely time domain and frequency domain features which depict the variation of emotions associated with the voice. The features of voice such as pitch, energy rate, jitter, shimmer, short-time energy, short-time zero crossing, Mel spectrum features contribute to the distinction of emotions which varies from person to person.

The neurodevelopmental disorders of the humans can be best reflected through their voices. Thus analyzing the voice, their disparities in modulation, and recognizing the variations of emotions collectively contribute to the identification of such disorders.

Autism Spectrum Disorder (ASD) is one such complex neurodevelopment disorder that involves considerable challenges in communication and social interaction. The children with ASD find difficulty in relating themselves with the people around them. The symptoms of autism for some children may emerge during initial childhood period and some children behave normally until toddlerhood but gradually develop signs of autism. This aspect of ASD leads to confusion and misunderstanding among the people in rural areas in differentiating ASD children from toddlers which may prevent those children from getting proper training and guidance to face their future. Thus training of the children with autism relies on the identification of autism in children, analysis of level of autism persisting in the children, and finally the coaching of those children that aids in improving their lifestyle.

The discrepancy in the voice of autistic children from that of the normal children can be realized in the delay in the babbling and non-synchronization of vocal patterns found in the infants during their first year of age [12]. The variation of emotions in children with ASD differs from that of the Typically Developing (TD) children in the stipulated period. The pattern of variation from one emotion to another emotion in typically developing children and autistic children in a time-variant event-driven environment is studied and analyzed. This assists in identifying and discriminating the autistic children from the non-autistic children. These emotional disparities in autistic children vary according to the level of autism prevailing in the children [22]. With the dawn of various machine learning algorithms [10], emotion recognition through voice [8] acts as a screening tool for autism. Thus this paper focuses on examining the features of voice signals and analyzing the emotional variations of autistic children and normal toddlers using Artificial Neural Network (ANN) algorithm.

The paper is structured in such a way that Section 2 explains the literature review related to the work mentioned in the paper. Section 3 depicts the methodology of the implemented work. Section 4 portrays the results and the insights attained from the experimental analysis and Section 5 concludes the paper.

2 State of Art

Besides identifying the presence of autism in children, it is also essential to recognize the emotions in those children to analyze the behavior. The emotional variations could be better diagnosed with the various features concerned with the voice. The features differs from male to female, children to adults, more precisely these features vary from one person to person. Quite a few researchers have contributed to this emotion recognition process which is seldom protracted to the identification of autism through voice.

Abhijit et al. [1]., proposed the phenomenon of identifying the emotions in ASD through the acoustic features. This system incorporates features such as frequency variants, Linear Prediction Cepstrum Coefficients (LPCC) to understand the emotions in ASD and normal children which has been used as a mechanism to detect autism. The KNN (k-nearest neighbor) algorithm is employed to perform the classification process of ASD and normal children. Boaz et al. [2]., proposed a perception system for recognizing emotions in ASD children based on the prosodic content of their voice and suggested that the stimuli of High Functioning ASD shall be improved with the integration of identified emotions along with lexical and prosody content. Daniel et al. [18] employed the prosodic content of the speech such as intensity, duration and pitch to assess the role of pitch in better identification of ASD in people. Yasufumi et al. [19] suggest that the ability to understand the emotions through prosodic content of the speech develops a little early in typically developed children than ASD children.

The phenomenon of identifying the emotions through features such as pitch variations [5] also plays a major role in the analyzing the social cognition of autistic children. This paper suggests that distinguishing basic vocal pitch directly links to emotion recognition in ASD children and suggests that analyzing the low-level sensory features of voice contributes to the identification of autism. Cheol et al. [4] add to the research of emotion recognition in autism through vocal stereotyping through which the behavioral patterns of the ASD children can be analyzed. The findings from [16] acclaim that the processing mechanism of emotions through voice by ASD differs from that of the TD children. This likewise recommends that emotion recognition through voice helps to better understand the social communication capability of autistic children.

M.C.Kirana et al. [3] proposed a different way of analyzing the emotions in ASD children by examining the intensity of the voice of the concerned child. The authors revolved around four emotions such as happy, sad, afraid and angry which are recognized with the levels of the intensity computed using the Fast Fourier Transform. This research suggests that the lower intensity of sound results in positive emotions such as happy and higher intensity of sound corresponds to negative emotions such as sad.

There exists noteworthy research in emotion recognition using MFCC features [6] associated with the voice. Using these features in the process of recognizing the emotions such as sadness, happiness, anger, disgust, fear, surprise [10] is considered to be effective in clearly classifying the emotions in normal children and autistic children. These emotions are explored in a deeper sense to realize the variation of emotions in autistic and non-autistic children.

The phenomenon of emotion recognition in ASD children through voice has made significant progress in the field of research with the blend of voice processing [7] and machine learning algorithms [11]. Utterance classifier using a neural network [9] was proposed by analyzing the emotions to better understand the response factor of autistic children and normal children. Chin kim et al. [24] suggested that these MFCC features contribute to the better realization of emotions using machine learning algorithms. Deshmukh et al., [14] commend that machine learning based voice analysis provides a better clarity in identifying the emotions in ASD children. However, to improve the accuracy of classification, better feature extraction and feature analysis need to be encompassed. Thus analyzing these features and their variations in a time variant environment lead to behavioral analysis and identification of autism in children.

3 Methodology

As the voice of humans effectively depicts the basic emotions through pitch, modulation and tone, recognizing the emotions through voice [20] has proven to be an eminent process. But the process of identifying the emotions is quite difficult as the emotions are independent and their interpretation is also challenging. Though it is a difficult process, recognition of emotions from voice has found its inevitable role in many applications. Hence this process can be used for recognizing the existence of ASD and analyzing their emotions.

Moreover, the emotions of children vary dynamically in a particular time frame, analyzing their emotions through voice helps us to understand the stability of their behavior [16]. Though autistic children do not express their emotion as typically developing children [4], this process would act as a screening method to differentiate ASD children from normal children.

The recognition of emotion through voice in autistic children commences with the pre-processing of voice, followed by the extraction of features from voice, classifying and analyzing the emotions through the various features associated with the voice. The model depicting the flow of the entire work is given in Fig. 1. The implementation model of the proposed work involves four steps which are described as follows:

Pre-processing of Dataset: The dataset is converted into a suitable form for extraction of required features.

Extraction of Features: This process involves the extraction of audio features such as energy, pitch and other features from voice.

Training of Model: This step consists of training the model to predict the emotions from the extracted features.

Testing of Model: This step validates the developed model and calculates the accuracy of the model.

Fig. 1

Flow diagram of emotion analysis in ASD.

3.1 Pre-processing and conversion of voice signal

The process of recognizing the emotions begins with the loading of the dataset. To perform this function, a special utility function called glob() function from the module called glob is used to retrieve the pathnames associated with the sound files in the dataset. To make the voice suitable for extracting its features, the voice signal obtained from the input dataset is processed in frames of 20 ms to 40 ms and these frames are overlapped to maintain a smooth transition. Windowing is done to minimize discontinuities and avoid spectral distortions. The FFT (Fast Frequency Transform) is applied to the windowing signal to convert the signal from the domain of time to the domain of frequency.

3.2 Feature extraction

This is a module that changes the waveform associated with the voice into a parametric form of representation at a comparatively reduced data rate. To perform this function, a special python package called librosa [21] is employed. The audio signal used by the deployed package for the feature extraction is characterized as a one-dimensional array, with the sampling rate denoting the frequency in which values of the audio signal shall be sampled. This part of the implementation is used to retrieve the following features from the input audio file.

3.2.1 Mel Frequency Cepstral Coefficient (MFCC)

This feature characterizes the short-term power spectrum of the corresponding sound. This represents the real value corresponding to the cepstral of the short-time signal resulting from the Fast Fourier Transform (FFT) of that particular signal. The extraction of MFCC consists of a series of steps such as pre-processing, framing (or) windowing, conversion into FFT spectrum, extraction of mel-spectrum. Table 1 shows the algorithm for MFCC extraction [24].

Table 1
Algorithm for MFCC extraction

Step 1: Divide the input sound into frames by applying a win dowing function

Step 2: Take the Fourier transform of each frame of the corre sponding signal

Step 3: Plot the powers of the spectrum resulting from step 3 against mel scale

Step 4: Take the logarithmic value of the powers obtained at each mel frequencies

Step 5: Compute the Discrete Cosine Transform of mel powers

Step 6: Retrieve the amplitude values of the resulting spectrum as mfcc features

Figure 2. shows a sample visual representation of MFCC of anger emotion in the dataset in which the x-axis corresponds to the time and y-axis corresponds to the MFCC values concerned with anger emotion. The region matching the red color in the figure specifies larger values of the MFCC coefficients whereas the blue color specifies smaller values. This representation varies for every other emotion as the MFCC coefficients associated with each emotion are different.

Fig. 2

MFCC representation of anger emotion.

3.2.2 Mel-scaled Spectrogram Frequency (mel)

The auditory system of humans does not have the capability of interpreting the pitch in a linear fashion that increases with frequency. To solve this, the mel-scale was designed in 1937 by Stevens, Volkmann and Newman [25] and framed the term ‘mel’ from the word ‘Melody’.

Mel is a frequency scale that is generally used to segregate and characterize audio signals and this perceptual scale of pitches is judged by listeners who are placed at an equal distance from one another. The solitary resolution of the experimentation was to define the auditory system of humans concerning a linear scale. The result of the experiment revealed that the frequency range of 0-1000 Hz perceives the pitch in a linear order. However this linear scale becomes logarithmic above 1000 Hz, which is 40 decibels above threshold of the listener.

A spectrogram is a visual depiction of various bands of frequencies of the audio signal as the signal is analog and non-periodic. A spectrogram can be produced by applying Fast Fourier Transform (FFT); This FFT is computed on overlapping several windowed segments of signals from which we get the resulting Spectrogram. Mapping y-axis frequency to ‘mel’ scale we get Mel Spectrogram.

Figure 3 depicts the visual representation of Mel Spectrogram of sample anger emotion. This is obtained by mapping the frequency of typical anger emotion present in the dataset in the scale of ‘mel’ (y-axis) to time (x-axis) & corresponding color dimension (amplitude) in decibels.

Fig. 3

Visual representation of Mel Spectrogram of anger emotion.

The mel frequency scale is defined as: $mel = 2595 * log 10 (1 + \frac{hertz}{700})$ (1) where hertz corresponds to the frequency values of the voice which should be mapped to the mel scale. Table 2 shows the algorithm for the computation of mel spectrogram [25].

Table 2

Steps for computation of Mel Spectrogram

Step 1: Sample the input sound into smaller frames

Step 2: Compute the Fast Fourier Transform(FFT) of each frame to convert the respective frame from time to frequency domain.

Step 3: Separate the frequency spectrum of the entire frames equally spaced frequencies.

Step 4: Transform the magnitude of the signal into its elements, which matches the frequencies in the corresponding mel scale.

3.2.3 Chroma

This feature of the voice corresponds to the 12 different classes of the pitch associated with the voice. Although representations of Mel-scale are generally used, they deliver a quiet low resolution of pitches and pitch classes. The representations of chroma or pitch class are frequently applied to convert harmony against suppressing variations in loudness, timbre and other aspects. Figure 4. portrays the visual representation of chromagram of sample anger emotion in the dataset using librosa package function [21]. The figure shows the different pitch classes represented by various colors corresponding to the sample emotion considered.

Fig. 4

Chromagram representation of anger.

3.2.4 Tonnetz

The tonnetz provides a different representation of harmony and pitch. A group of pitches namely a chord can be labeled as a single centroid point in the corresponding space. This feature encompasses such tonal centroids in the form of coordinates which are represented in a six-dimensional interval space. This is computed by the multiplication of chroma vector and the transformation matrix [23] where chroma vector is a vector representation obtained for each frame from 12 different pitch classes associated with the voice and the transformation matrix corresponds to the base of six-dimensional space. Thus tonnetz is used to represent the harmonic network associated with the voice. Table 3 shows the algorithm for feature extraction from voice.

Table 3
Algorithm for Feature Extraction

Step 1: Read the audio file in the dataset using sound file.read()

Step 2: Retrieve the sample rate S of each audio file using sound file.samplerate

Step 3: For each audio file X, extract the raw features such as mfcc, mel and chroma

If chroma exists in the audio file, then = STFT(X), where

STFT is Short-time Fourier Transform

Retrieve feature values using librosa.feature_name()

by passing Y as an input

Features = librosa.feature.Chroma(Y)

If mfcc exists in the audio file

Retrieve mfcc values using librosa.feature_name() by

passing Y as an input

Features = librosa.feature.mfcc(X, n_mfcc = 40)

If mel exists in the audio file

Compute the mel-scaled spectrogram of the audio

file.

Features = librosa.feature.melspectrogram(X)

Step 4: Store the retrieved values in an array stacked horizontally

resultFeatures = np.array()

resultFeatures = np.hstack (resultFeatures, Features)

3.3 Classification and analysis of emotions

Once the features are extracted, the emotions are predicted for the identified features using the Multi-layer Perceptron (MLP) Classifier. This classifier is a model of feedforward Artificial Neural network. Unlike other classification algorithms, this MLP classifier has an built-in neural network to perform classification.

A MLP classifier involves minimum three layers of nodes: an input layer, a hidden layer and an output layer. Excluding the first layer of input nodes, each node is designated as neuron that deploys a non-linear activation function. MLP employs a supervised learning technique known as backpropagation for training. Hence this work implements an MLP classifier to predict and classify the emotions.

The MLP classifier implemented in this work has one input layer, one hidden layer of 300 nodes and an output layer with the adaptive learning rate. The activation function used is Rectified Linear Unit (ReLU) function and the loss function is optimized using the stochastic gradient-based optimizer. Table 4 depicts the algorithm for emotion recognition from voice.

Table 4
Algorithm for emotion recognition from voice

Step 1: Define the base class of emotions i.e (anger, happiness,

disgust, fear, sadness, neutral, calmness)

Step 2: Import the dataset and pre-process the dataset.

Step 3: For each audio file in the dataset,

•Perform extraction of features such as mfcc, chroma, mel from the audio file using extractFeature()

•Store the retrieved features and emotions as

xFeatures and yEmotions respectively

xFeatures.append(features_extracted)

yEmotions.apped(emotions_identified)

Step 4: Split the data as training and testing from xFeatures and

yEmotions with the test data size 0.2

Step 5: Initialize the MLP Classifier and train the model

Step 6: Test the model for suitable identification of emotions

and predict the accuracy

Step 7: Analyze the emotions to understand and compare the

stability of behavior of ASD children and TD

4 Experimental results and discussion

4.1 Dataset explanation

The data set used in this implementation contains the voice of the two classes of children who were identified as ASD and TD. Both the classes of children are observed in a similar environment and the voices have been recorded. This process takes 185 audio files categorized as ASD and 215 audio files categorized as TD. In the entire dataset, 80% of the files were used as training dataset and 20% of the files were used as test dataset in both the categories. Though the number of voice files considered for ASD seems to be minimum due to the challenges involved in data acquisition, the exploratory analysis of emotions using the dataset seems to be near to the processing capability. The experimental analysis performed in this paper focuses on exploring and analysing the emotions of the ASD children and TD children.

4.2 Classification of emotions

The classification of emotions is performed with the main aspect of articulating a fundamental screening mechanism that can be deployed as a suggestive factor of autistic behavior shortly. The proposed system revolves around seven major emotions such as anger, disgust, calmness, fear, happiness, neutral and sadness which exhibit the behaviour of humans. This classification and identification of emotions are performed through MLP classifier. MLP classifier proved to perform better to find the concerning emotion and classify them since it consists of an in-built neural network.

Table 5 shows the probability of each emotions corresponding to both ASD and TD. From Table 5, it can be inferred that the construal of emotions in ASD seems to be neutral, anger, disgust, fear, calmness, sadness, and happiness in maximum to a minimum order. It can be seen from Fig. 5. that the ASD children show neutral as their major emotion which is followed by anger or disgust as the next consecutive emotion with varied behaviour of the concerned children.

Table 5
Probability of Emotions in ASD and TD

Emotions ASD TD

Anger 0.4402 0.2756

Calmness 0.1294 0.2941

Disgust 0.2875 0.1831

Fear 0.1143 0.2444

Happiness 0.1588 0.4333

Neutral 0.5328 0.1163

Sadness 0.1176 0.0833

Emotions	ASD	TD
Anger	0.4402	0.2756
Calmness	0.1294	0.2941
Disgust	0.2875	0.1831
Fear	0.1143	0.2444
Happiness	0.1588	0.4333
Neutral	0.5328	0.1163
Sadness	0.1176	0.0833

Fig. 5

Comparison of emotions in ASD & TD.

The experimental studies indicate that TD children show happiness as the key emotion. Without any interference of the objects, the toddlers too exhibit neutral as their major emotion. This also interprets that the normal children show a maximum of happiness emotion which is followed by other emotions depending upon the atmosphere around them. However, in the case of autistic children, several objects can be shown, of which certain objects may contribute to change in the emotion exhibited by them.

Table 6 shows the probability distribution of these seven emotions for ASD children derived as a result of experimental analysis. The above table shows the probability distribution which corresponds to the variation of emotions. From this table, the probability of any particular emotion to be predicted as that of the same emotion seems to be highest which commends the accuracy of the analysis. Figure 6. insights that maximum emotion prevailing in autistic children seems to be neutral taking the highest probability value in the experimental analysis which justifies the results shown in Table 5. Through this probability analysis, drifting from one emotion to other can be realized which may be used as a basic technique for discriminating between autistic children and toddlers.

Table 6

Probability Distribution of Emotions in ASD

Emotions	Anger	Calmness	Disgust	Fear	Happiness	Neutral	Sadness
Anger	0.440231	0.058962	0.241379	0.117931	0.056604	0.051887	0.035377
Calmness	0.321001	0.129412	0.251001	0.058824	0.058824	0.073529	0.037001
Disgust	0.350011	0.094787	0.297501	0.125101	0.075829	0.052133	0.033175
Fear	0.314286	0.030612	0.085714	0.152861	0.040816	0.112245	0.020408
Happiness	0.326471	0.044118	0.205882	0.117847	0.158824	0.117647	0.029412
Neutral	0.333301	0.087719	0.26246	0.070175	0.052632	0.532807	0.035088
Sadness	0.352941	0.100001	0.211765	0.117647	0.075001	0.0749	0.1176

Fig. 6

Flow of emotions in ASD.

The flow of emotions in TD children seems to vary smoothly may be from happiness to sadness, and then to anger or disgust and so on. But in autistic children, this smooth transition may not be happening which was inferred from the conventional clinical analysis method [17]. Their major response would remain as neutral emotion which may unexpectedly turn into anger [15] or disgust emotion that may drive them to show certain behavior which cannot be predicted. However, this variation of emotions may differ from children to children based upon the environment around them [26]. Henceforth this analysis can be used as a preliminary screening technique that can predict the autistic nature in children in the near future.

The trained model computes the probability values of each emotion and calculates the differences between them in successive iterations until there is no greater variation in the probability values across the emotions. The results are tabulated in Table 7 in which the probabilistic differences between the emotions could be observed that helps to understand the proximity emotions associated with each emotion of ASD and TD children. These deviations between the emotions helps us to realize the next consecutive emotion and their flow towards revealing their behavior.

Table 7

Comparison of difference in probability values of ASD and TD

Classification Type	Emotions	Anger	Calmness	Disgust	Fear	Happiness	Neutral	Sadness
ASD	anger	0	0.38127	0.02885	0.30235	0.38363	0.08834	0.43485
	calmness	0.21516	0	0.07059	0.46816	0.48651	0.23047	0.06241
	disgust	0.06251	0.28271	0	0.26241	0.27167	0.06537	0.41357
	fear	0.21306	0.40981	0.05857	0	0.04098	0.23516	0.02169
	happiness	0.22765	0.44711	0.06471	0.04098	0	0.22835	0.48234
	neutral	0.19951	0.40509	0.24658	0.46231	0.39428	0	0.46308
	sadness	0.23534	0.01762	0.22417	0.00475	0.49635	0.38342	0
TD	anger	0	0.22516	0.32114	0.36178	0.43917	0.22972	0.11263
	calmness	0.12456	0	0.22541	0.41527	0.33161	0.08179	0.00134
	disgust	0.31972	0.22676	0	0.05634	0.41716	0.217859	0.24081
	fear	0.34762	0.39571	0.06341	0	0.40889	0.10562	0.46825
	happiness	0.44917	0.34259	0.42441	0.41836	0	0.45783	0.02548
	neutral	0.23472	0.07195	0.22367	0.103171	0.45734	0	0.33859
	sadness	0.10873	0.00235	0.24138	0.45348	0.01359	0.33531	0

Figure 7 shows that the proximity emotions of anger are found to be disgust and neutral as inferred from the probability variation of the emotions for ASD children. This shows the next emotion of anger can be disgust or neutral as the probability differences between those emotions are minimum. This can also be inferred form the overall probability value given in Table 5 which justifies the analysis. Likewise, for TD children the observation indicates that the emotion next to anger might be pictured as calmness and sadness. Similarly, Fig. 8. provides the observation that anger and disgust are the proximity emotions for neutral in autistic children which is vindicated from the experimental insights.

Fig. 7

Proximity emotion of anger.

Fig. 8

Proximity emotion of neutral.

In the same manner, Fig. 9. depicts that the proximity emotions for disgust in ASD children are anger and neutral which justifies the experimental findings. The differences in probabilities provides insights regarding the change in emotions and thereby helps in the categorization between ASD and TD. Thus the analysis illustrates that the emotions of the majority of autistic children revolve around neutral, anger and disgust on most occasions.

Fig. 9

Proximity emotion of disgust.

4.3 Performance evaluation of emotion analysis

The testing is performed with the 20% of the datasets in both categories to ensure the correctness of the emotion predicted as a result. The measures such as precision and recall are used to evaluate the performance of the proposed model. Table 8 shows the performance measures for the emotion analysis in ASD children. The accuracy of the emotion analysis process is determined by the performance of the training and the testing datasets. The accuracy is calculated from the values of actual emotions and the predicted emotions. On first execution of the process, the accuracy was found to be 79.25% and during the second run it was found to be 82.56% and the next consecutive runs gave 82.75% of accuracy. On an average, the performance accuracy of the process was found to be 81.52%. Figure 10. depicts the accuracy factor involved in the process of emotion analysis. This can be further enhanced by improving the training process with more combinations of features and by increasing the size of the data considered for the process.

Table 8
Performance Measures

Emotion / Performance Measures Precision Recall

Anger 0.92 0.88

Calmness 0.91 0.82

Disgust 0.80 0.85

Fear 0.78 0.84

Happiness 0.77 0.87

Neutral 0.95 1.00

Sadness 0.75 0.88

Emotion / Performance Measures	Precision	Recall
Anger	0.92	0.88
Calmness	0.91	0.82
Disgust	0.80	0.85
Fear	0.78	0.84
Happiness	0.77	0.87
Neutral	0.95	1.00
Sadness	0.75	0.88

Fig. 10

Accuracy of emotion analysis.

5 Conclusion

The identification of emotions in ASD children is extremely important to provide proper training and coaching to the concerned autistic children. There are many techniques and systems available to identify the presence of autism disorder in children. The paper focuses on the flow of seven basic emotions such as anger, disgust, neutral, happiness, calmness, fear and sadness which was evaluated by the probabilistic distribution values associated with the emotions. From the analysis, it can be perceived that the major emotion shown by the toddlers moves around happiness and calmness while the major emotion exhibited by the autistic children revolves around neutral, anger and disgust [13]. The emotion analysis performed in this paper also recommends the major emotion and the next prime emotion shown by the children through which the behavior of the children can be analyzed. The accuracy of this analysis revolves around 80% which can be further enhanced with the inclusion of more features. Thus the proposed analysis helps to assist the process of identification of autism through their voice and to analyze the emotions exhibited by them observing their natural behavior.

References

Mohanta

, Mukherjee

and Kumar Mittal

, Acoustic Features Characterization of Autism Speech for Automated Detection and Classification, 2020 National Conference on Communications (NCC), (2020), pp. 1-6, doi: 10.1109/NCC48643.2020.9056025

Ben-David

B.M.

, Ben-Itzchak

, Zukerman

, Yahav

and Icht

, The Perception of Emotions in Spoken Language in Undergraduates with High Functioning Autism Spectrum Disorder: A Preserved Social Skill, Journal of Autism and Developmental Disorders, 2019.

Kirana

M.C.

, Lubis

M.Z.

and Amalia

E.D.

, The Effect of Sound Manipulation to Know Response Rate in Autism Children Using FFT, International Conference on Applied Engineering (ICAE), (2018), pp. 1-5, doi: 10.1109/INCAE.2018.8579418

Min

C-H.

and Fetzner

, Vocal Stereotypy Detection: An Initial Step to Understanding Emotions of Children with Autism Spectrum Disorder, 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (2018), pp. 3306-3309, doi: 10.1109/EMBC.2018.8513050

Schelinski

and von Kriegstein

, The Relation Between Vocal Pitch and Vocal Emotion Recognition Abilities in People with Autism Spectrum Disorder and Typical Development, Journal of Autism and Developmental Disorders, 2019.

Attawibulkul

, Kaewkamnerdpong

and Miyanaga

, Noisy Speech Training in MFCC-based Speech Recognition with Noise Suppression Toward Robot Assisted Autism therapy, Biomedical Engineering International Conference, 2017

Deng

, Cummins

, Schmi

, Qian

, Ringeval

and Schuller

, Speech-based Diagnosis of Autism Spectrum Condition by Generative Adversarial Network Representations, In Proceedings of DH’17, 2017.

Ashok Kumar

and Mazher Iqbal

J.L.

, Machine Learning Based Emotion Recognition using Speech Signal, International Journal of Engineering and Advanced Technology, ISSN: 2249 – 8958, Volume-9 Issue-1S5, December, 2019.

Wijesinghe

, Samarasinghe

, Seneviratne

, Yogarajah

and Pulasinghe

, Machine Learning Based Automated Speech Dialog Analysis Of Autistic Children, 11th International Conference on Knowledge and Systems Engineering (KSE), 2019.

10.

Nithya Roopa

, Prabhakaran

and Betty

, Speech Emotion Recognition using Deep Learning, International Journal of Recent Technology and Engineering, ISSN: 2277-3878, Volume-7 Issue-4S, November 2018.

11.

van den Bekerom

, Using Machine Learning for Detection of Autism Spectrum Disorder, Twente Student Conference, 2017.

12.

Garrido

, Watson

L.R.

, Carballo

, Garcia-Retamero

and Crais

E.R.

, Infants At-Risk for Autism Spectrum Disorder: Patterns of Vocalizations at 14 Months, International Society for Autism Research, Wiley Periodicals, 2017.

13.

Cristina , et al., Emotion recognition in typical and atypical development – a technology-based paradigm, 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), 2019.

14.

Deshmukh

, Gaonkar

, Golwalkar

and Kulkarni

, Speech based Emotion Recognition using Machine Learning, 3rd International Conference on Computing Methodologies and Communication (ICCMC), 2019.

15.

Valerie , et al., Enhanced Sensitivity to Angry Voices in People with Features of the Broader Autism Phenotype, Journal of Autism and Developmental Disorders, 2018.

16.

Matsumoto

, et al., Behavioral Study on Emotional Voice Perception in Children with Autism Spectrum Disorder, American Academy of Pediatric Neuropsychology 2017.

17.

Stronach

Sheri T.

and Wetherby

Amy M.

, Observed and Parent-Report Measures of Social Communication in Toddlers With and Without Autism Spectrum Disorder Across Race/Ethnicity, American Journal of Speech-Language Pathology, 2017.

18.

Hubbard

Daniel J.

, et al., Production and perception of emotional prosody by adults with autism spectrum disorder, International Society for Autism Research, Wiley Periodicals, 2017.

19.

Yasufumi , et al., Characteristics of the Understanding and Expression of Emotional Prosody among Children with Autism Spectrum Disorder, Autism Open Access, DOI: 10.4172/2165-7890.1000185, 2016

20.

Ringeval

, Marchi

, Grossard

, Xavier

, Chetouani

, et al., Automatic Analysis of Typical and Atypical Encoding of Spontaneous Emotion in the Voice of Children, Proceedings INTERSPEECH 2016, 17th Annual Conference of the International Speech Communication Association (ISCA), 2016.

21.

McFee

, Raffel

, Liang

, Ellis

Daniel P.W.

, McVicar

, Battenber

and Nietok

, librosa: Audio and Music Signal Analysis in Python, Proceedings of 14th Python in Science Conference, 2015.

22.

Santos

Joao F.

, et al., Very early detection of Autism Spectrum Disorders based on acoustic analysis of pre-verbal vocalizations of 18-month old toddlers, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013.

23.

Harte

, Sandler

and Gasser

, Detecting Harmonic Change In Musical Audio, Proceedings of the 1st ACM workshop on Audio and music computing multimedia, 2006.

24.

Kim

, et al., Mel-Frequency Cepstral Coefficient Analysis in Speech Recognition, International Conference on Computing & Informatics, 2006.

25.

Stevens

S.S.

and Volkmann

, The Relation of Pitch to Frequency: A Revised Scale,pp, The American Journal of Psychology53(3) (1940), 329–353.

26.

Dongxin , et al., Objective Child Behavior Measurement with Naturalistic Daylong Audio Recording and its Application to Autism Identification, International Conference of the IEEE EMBS, 2012.