Deep quaternion Fourier transform for salient object detection

Abstract

Salient object detection plays a vital role in image processing applications like image retrieval, security and surveillance in authentic-time. In recent times, advances in deep neural network gained more attention in the automatic learning system for various computer vision applications. In order to decrement the detection error for efficacious object detection, we proposed a detection classifier to detect the features of the object utilizing a deep neural network called convolutional neural network (CNN) and discrete quaternion Fourier transform (DQFT). Prior to CNN, the image is pre-processed by DQFT in order to handle all the three colors holistically to evade loss of image information, which in-turn increase the effective use of object detection. The features of the image are learned by training model of CNN, where the CNN process is done in the Fourier domain to quicken the method in productive computational time, and the image is converted to spatial domain before processing the fully connected layer. The proposed model is implemented in the HDA and INRIA benchmark datasets. The outcome shows that convolution in the quaternion Fourier domain expedite the process of evaluation with amended detection rate. The comparative study is done with CNN, discrete Fourier transforms CNN, RNN and masked RNN.

Keywords

Convolutional neural networks quaternion complex variable object detection image enhancement discrete quaternion Fourier transform

1 Introduction

Object detection is the localization of the targeted object in the image or video. The advantage is to understand and analyze scenes in images or videos such as tracking or counting the objects [1, 2]. The detected object is denoted by the boundary box or pixel masking of the object [3, 4]. Some of the applications include vehicle detection, human behavior identification, machine inspection, image retrieval and security-related problems. The hardware device for the application is the intellective video camera. Many challenges occurs while capturing the images in indoor and outdoor areas like illumination variation, appearance changes of the moving objects, occlusion, complex background and shadow etc. Prior to the deep learning technique era, the object is detected in the following three steps 1) Proposal generation 2) Feature extraction and 3) Region classification. Proposal generation searches the location of the objects in the image namely region of interest (ROI). In ROI, the feature is extracted by sliding window process in the image with some of the feature descriptors like Gradient Location and Orientation Histogram (GLOH) [5], Speeded up Robust Features (SURF) [6], scale-invariant feature transform (SIFT) and principal component analysis (PCA) etc. The region classification labels the covered regions and the techniques used to classify regions are Adaboost, SVM and Cascade learning. The occurrence of more false positives in proposal generation leads to complex features of the image are unable to handle and the optimization is discretely carried out for each step so there is no global optimization is done for the whole process. To overcome these inhibitions, deep learning [7, 8] approach achieved remarkable progress in computer vision applications. The handcrafted feature descriptor [9] used in classical models turned into an automatic feature descriptor from raw pixels while training the data in deep learning model namely CNN which is the biological inspired model to extract the high-level and low-level features of the image.

In this paper, to accelerate the CNN process in terms of computational efficiency, the image is processed in Fourier domain. Nowadays, different kinds of Fourier transform is used in many computer vision applications for pre-processing the image to achieve efficient outcome [31]. Fourier transform cannot holistically handle the color image. Hence, in this paper, we have handled the quaternion complex number, which is a four dimensional complex number which resembles the three colour channels with zero real part in Fourier domain and performs discrete quaternion Fourier transform namely DQFT handles the three color channels of the pixel which simultaneously congruous for authentic-time as well as offline immensely colossal to increment the computational efficiency. Then, the features of such preprocessed images are trained using CNN and finally the image is re-transferred into the spatial domain to yield the detection of object. The Fig. 1 shows the process flow diagram of the proposed work.

Fig. 1

Process flow diagram.

The rest of the paper is structured as follows: Section 2 presents the related works about the current methods of object detection, Section 3 gives the background definitions of the methods which was used in the proposed system, Section 4 discusses the proposed work, Section 5 gives the experimental results with the comparative analysis and the conclusion is given in Section 6.

2 Related works

In literature, object detection mainly focuses on the detection of features and also behaves like a classifier [10]. Considering the video frames, background subtraction is a preprocessing step for efficient object detection [11]. Readers can refer, haar features with a sliding window for human face detection, HOG feature with SVM for pedestrian detection [12]. Deep learning for object detection comes under two categories [13]: (a) Two-stage detector and (b) One-stage detector. Category (a) detects the object based on proposal generation and region classification, Recurrent Convolutional Neural Network (RCNN) is one of the example of (a). Category (b) takes all the image pixels as a whole input to the model, so it doesn’t require a two-step process. Sermanet et al. developed object detection using one-stage detection method [14]. Alexnet is one of the architecture of CNN which consists of five convolutional layers and three fully connected layers [15], GoogLeNet is developed by google which is deep convolutional network implements convolution and pooling layers simultaneously [16, 17]. GoogLeNet contains Inception Modules, so that it can perform variant sizes of convolutions and concatenates the filters to the next layer. AlexNet does not supports filter concatenation. Chu implemented CNN model for unary object proposal and for feature extraction of the object [18]. Chen cumulates region proposal network with object detection which has a feature extrapolating layer to detect the object categories [19]. Li induced CNN in the course and fine detection phase. The course phase, detects the coarse candidate regions which contain objects. The fine phase, object candidate regions are cropped and classified as objects and backgrounds [20]. Kang developed a temporal convolution framework to handle video-predicated object detection for temporal information [21]. Quaternion Convolution neural network is utilized for color image relegation and denoising theme identification of telephone conversation between agents and customer [22, 23], image reconstruction [24], image forensics [25] and for detecting lake, grass, forest, and town areas [26]. Ren et al. detects the objects based on two modules. The first one contains deep fully convolutional network which proposes regions, and the second module is the object detector [38]. Mask RCNN is a two-stage pipeline, with an RPN as the first stage and the second stage prognosticating the class and box offset [39]. Our proposed method overcomes the computational complexity of above discussed methods with better efficiency of object detection.

3 Background

3.1 Quaternion complex number

It is a four dimensional complex number [27] Q = a + bi + cj + dk or (a, b, c, d) with one real unit and three imaginary units i, j, k where a, b, c, d are real numbers with the following multiplications laws ij = - ji = k ; jk = - kj = i ; ki = - ik = j ; i² = j² = k² = ijk = -1, i.e., $\begin{matrix} i & j & k \\ i & - 1 & k & - j \\ j & - k & - 1 & i \\ k & j & - i & - 1 \end{matrix}$ (1)

Let Q₁ = a₁ + b₁i + c₁j + d₁k and Q₂ = a₂ + b₂i + c₂j + d₂k. Addition of two quaternion complex numbers is given by $\begin{matrix} Q_{1} + Q_{2} = (a_{1} + a_{2}) + (b_{1} + b_{2}) i + (c_{1} + c_{2}) \\ j + (d_{1} + d_{2}) k \end{matrix}$ (2) Multiplication of two quaternion complex numbers is obtained by $\begin{matrix} Q_{1} Q_{2} & = & (a_{1} + b_{1} i + c_{1} j + d_{1} k) (a_{2} + b_{2} i + c_{2} j + d_{2} k) \\ = & \begin{matrix} a_{1} a_{2} + a_{1} b_{2} i + a_{1} c_{2} j + a_{1} d_{2} k + \\ b_{1} a_{2} i + b_{1} b_{2} i^{2} + b_{1} c_{2} ij + b_{1} d_{2} ik + \\ c_{1} a_{2} j + c_{1} b_{2} ji + c_{1} c_{2} j^{2} + c_{1} d_{2} jk + \\ d_{1} a_{2} k + d_{1} b_{2} ki + d_{1} c_{2} kj + d_{1} d_{2} k^{2} \end{matrix} \end{matrix}$ (3) Using (1) in (3), $\begin{matrix} = (a_{1} a_{2} - b_{1} b_{2} - c_{1} c_{2} - d_{1} d_{2}) + \\ (a_{1} b_{2} + b_{1} a_{2} + c_{1} d_{2} - d_{1} c_{2}) i + \\ (a_{1} c_{2} - b_{1} d_{2} + c_{1} a_{2} + d_{1} b_{2}) j + \\ (a_{1} d_{2} + b_{1} c_{2} - c_{1} b_{2} + d_{1} a_{2}) k \end{matrix}$ (4) the Equation (4) is represented interms of matrix form as follows, $\begin{matrix} Q_{1} Q_{2} = (\begin{matrix} a_{1} & - b_{1} & - c_{1} & - d_{1} \\ b_{1} & a_{1} & - d_{1} & c_{1} \\ c_{1} & d_{1} & a_{1} & - b_{1} \\ d_{1} & - c_{1} & b_{1} & a_{1} \end{matrix}) (\begin{matrix} a_{2} \\ b_{2} \\ c_{2} \\ d_{2} \end{matrix}) \\ Q_{1} Q_{2} = (\begin{matrix} a_{1} a_{2} - b_{1} b_{2} - c_{1} c_{2} - d_{1} d_{2} \\ a_{1} b_{2} + b_{1} a_{2} + c_{1} d_{2} - d_{1} c_{2} \\ a_{1} c_{2} - b_{1} d_{2} + c_{1} a_{2} + d_{1} b_{2} \\ a_{1} d_{2} + b_{1} c_{2} - c_{1} b_{2} + d_{1} a_{2} \end{matrix}) \end{matrix}$ (5) Example, let Q₁ = 1 +2i + 3j + 6k and Q₂ = 1 + i + j + k, $\begin{matrix} Q_{1} Q_{2} = (\begin{matrix} 1 & - 2 & - 3 & - 6 \\ 2 & 1 & - 6 & 3 \\ 3 & 6 & 1 & - 2 \\ 6 & - 3 & 2 & 1 \end{matrix}) \\ (\begin{matrix} 1 \\ 1 \\ 1 \\ 1 \end{matrix}) = (\begin{matrix} - 10 \\ 0 \\ 8 \\ 6 \end{matrix}) \end{matrix}$ Thus, Q₁Q₂ = -10 + 0i + 8j + 6k.

3.2 Pure quaternion representation of the image

The discrete color image f(m, n) image matrix can be transformed into the quaternion complex matrix by encoding the red, green, and blue as a pure quaternion with zero real part, i.e., $f (m, n) = 0 + [r (m, n) i + g (m, n) j + b (m, n) k]$ where, m - number of rows in the image, n - number of columns in the image. The advantage of quaternion color image processing [29] is to treat the triple color of each pixel as a whole unit which leads to the simultaneous process of each color channel.

3.2.1 Conversion of 2D image into quaternion image

For any pure quaternion μ₁ = μ(1, 1) i + μ(1, 2) j + μ(1, 3) k, p = p(1, 1) i + p(1, 2) j + p(1, 3) k, the pixel intensity of quaternion image is obtained by $\begin{matrix} Q (m, n) = [a (m, n) + b^{'} (m, n) μ_{1}] \\ + μ_{2} [c^{'} (m, n) + d^{'} (m, n) μ_{1}] \end{matrix}$ (6) where $μ_{2} = \frac{μ_{1} p - p μ_{1}}{| μ_{1} p - p μ_{1} |}$ μ₃ = μ₂μ₁ b′(m, n) , c′(m, n) and d′(m, n) are obtained by the below matrix multiplication where μ₁, μ₂, μ₃, p are given by $[\begin{matrix} b^{'} (m, n) \\ c^{'} (m, n) \\ d^{'} (m, n) \end{matrix}] = [\begin{matrix} μ (1, 1) & μ (1, 2) & μ (1, 3) \\ μ (2, 1) & μ (2, 2) & μ (2, 3) \\ μ (3, 1) & μ (3, 2) & μ (3, 3) \end{matrix}] [\begin{matrix} b (m, n) \\ c (m, n) \\ d (m, n) \end{matrix}]$ The following illustration explains to find the pixel intensity of the image in the quaternion domain.

Let $μ_{1} = \frac{1}{\sqrt{3}} (i + j + k)$ and p = 0i + 0j + k. Consider the pixel values of rgb i.e., b, c, d values of the image are given by (192, 230, 114), then μ₂ is obtained as follows $μ_{2} = \frac{- 1.154 j + 1.154 i}{\sqrt{1.332 + 1.332}} = - 0.707 j + 0.707 i$ and μ₃ = μ₂μ₁ =(0.707i - 0.707j)(0.577i + 0.577j + 0.577k) using Equation (5) $μ_{3} = (\begin{matrix} 0 & - 0.707 & 0.707 & 0 \\ 0.707 & 0 & 0 & - 0.707 \\ - 0.707 & 0 & 0 & - 0.707 \\ 0 & 0.707 & 0.707 & 0 \end{matrix}) (\begin{matrix} 0 \\ 0.577 \\ 0.577 \\ 0.577 \end{matrix})$ $μ_{3} = (\begin{matrix} 0 \\ - 0.408 \\ - 0.408 \\ 0.816 \end{matrix})$ which implies that μ₃ = -0.408i - 0.408j + 0.816k and b′(m, n) , c′(m, n) and d′(m, n) is obtained using Equation (8). $(\begin{matrix} b^{'} (n, m) \\ c^{'} (n, m) \\ d^{'} (n, m) \end{matrix}) = (\begin{matrix} 0.577 & 0.577 & 0.577 \\ 0.707 & 0.707 & 0 \\ 0.408 & 0.408 & - 0.816 \end{matrix}) (\begin{matrix} 192 \\ 230 \\ 114 \end{matrix})$ $\begin{matrix} b^{'} (n, m) \\ c^{'} (n, m) \\ d^{'} (n, m) \end{matrix} = (\begin{matrix} 309.27 \\ - 26.87 \\ - 79.15 \end{matrix})$ The pixel intensity of the quaternion image is for (192, 230, 114) is obtained as 309.27i - 26.87j - 79.15k.

3.3 Discrete quaternion Fourier transform

Fourier transform is a fundamental transform used in both signal and image processing. It converts a signal (sample format) or image (pixel format) from a spatial domain into amplitudes and phases of sinusoid’s in frequency domain [30, 31]. Discrete Fourier transform (DFT) is called the sampled Fourier transform which takes the necessary samples rather than the all pixels of the image to represent it in the frequency domain. The size of the Fourier image is equal to the size of the image in the spatial domain. Generalization of a Fourier transform is the Discrete Quaternion Fourier Transform (DQFT) [32 –34] in which quaternion complex number executes in the place of complex number and the whole process is defined as the symplectic decomposition which is explained below

3.3.1 Symplectic decomposition

Once the image is converted into the pure quaternion image, it is decomposed into two components, the first part contains real part and red components and the second part consists of green and blue components. The 2D-DFT is applied to both parts discretely and the cumulative results of DFT’s produces the DQFT which is briefly explained in the following steps and the respective images are shown in Fig. 2:

Fig. 2

From left to right: Original image, (i, j, k) component for processing the quaternion image, Discrete quaternion Fourier transform of image.

Step1: The decomposition of input image:

$Q (m, n) = q_{a} (m, n) + q_{b} (m, n) μ_{2}$ (7) where, $\begin{matrix} q_{a} (m, n) & = & a (m, n) + b^{'} (m, n) μ_{1} \\ q_{b} (m, n) & = & c^{'} (m, n) + d^{'} (m, n) μ_{1} \end{matrix}$ Step2: Applying discrete Fourier transform to the quaternion decomposition. $\begin{matrix} Q (u, v) = \frac{1}{\sqrt{MN}} \sum_{m = 0}^{M - 1} \sum_{n = 0}^{N - 1} e^{- μ 2 π (\frac{mu}{M} + \frac{nv}{N})} q_{a} (m, n) \\ + \\ [\frac{1}{\sqrt{MN}} \sum_{m = 0}^{M - 1} \sum_{n = 0}^{N - 1} e^{- μ 2 π (\frac{mu}{M} + \frac{nv}{N})} q_{b} (m, n)] μ_{2} \end{matrix}$ (8)Step3: Process of DQFT image $QF (m, n) = Q (u, v) \frac{1 - μ_{3}}{2} + Q (u, - v) \frac{1 + μ_{3}}{2}$ (9)

3.4 Convolutional neural network

The object feature extraction plays an important task in object detection. There are many feature extractors namely PCA, SIFT, HOG, etc [35]. A Convolutional neural network is one of the deep learning methods which extract the features of the image. To detect the targeted object, object features have to be trained. Once the features are trained the targeted object can be easily detected. There are three different layers in CNN, which is explained with quaternion image and the process flow of the CNN architecture is shown in Fig. 3. 1) convolutional layer 2) pooling layer 3) fully connected layer.

Fig. 3

CNN architecture.

4 Quaternion Fourier convolution

This section briefly explains about our proposed model. Once the pre-processing is done using DQFT, the whole color image becomes a 2D pure quaternion matrix. The convolution product (⊙) which is the rudimentary step in CNN, the kernel matrix K(m, n) of order X x X is applied in the convolutional layer. The quaternion Fourier convolution QFC(m, n) is defined by $\begin{matrix} QFC (m, n) = \sum_{x = 1}^{X} \sum_{y = 1}^{X} \\ QF (x + m - 1, y + n - 1) ⊙ K (x, y) \end{matrix}$ (10) In QFC(m, n) convolution occurs only one time due to 2D matrix Q(m, n) whereas in the classical CNN, the convolution is three times to every pixel of three-color system. Also, classical CNN transforms the 3-channel color image into a single-channel feature map, so there is a loss of information but it is recovered in the QFC(m, n). The computational time for spatial and Fourier convolution for small scale images may be more or less identically same. However, for larger images, spatial convolution takes more time, rather Fourier convolution scales at low. The next layer of CNN is the pooling layer which is additionally one of the building blocks of CNN. Its functionality is to reduce the size of the pixels of the image for fewer parameters and computation. It is also called the down sampling layer which controls over-fitting on the training. The max pooling method is used in this proposed work. $\begin{matrix} MP (m, n) = {max}_{x = 1}^{M - 1} \\ {{max}_{y = 1}^{N - 1} QFC (m + x - 1, n + y - 1)} \end{matrix}$ (11) Each layers contains linear and non linear transformation.There are many non-linear operations present for the CNN architecture they are ReLU, sigmoid and tan etc. After the process convolution and pooling ReLU is used as the non-linear activation function. The convolution pooling along with ReLU is applied three times in the CNN architecture to extract the prominent features. Next to these layer, the inverse QDFT is applied to yield the color image for the fully connected layer. $\begin{matrix} IQ (m, n) = \frac{1}{\sqrt{MN}} \sum_{m = 0}^{M - 1} \sum_{n = 0}^{N - 1} \\ e^{- μ 2 π (\frac{mu}{M} + \frac{nv}{N})} QF (m, n) \end{matrix}$ (12)

In the fully connected layer, to connect the neurons from one layer to the next layer the below two equations are used. $Z = w^{T} x + b$ (13) where Z is the output of the linear function, w is the weight, x is the 2D data and b is the bias. The activation function which is the non-linear operation is applied to the output of the linear function. Here, the sigmoid activation f(Z) is used which brings the values between 0 to 1. The mathematical expression is given as $f (Z) = \frac{1}{(1 + e^{- Z})}$ (14) Where Z, is the input given to the sigmoid function. The way the system adjusting the filter values in the training of CNN is called backpropagation. The four steps involved in the backpropagation i) forward pass ii) loss function iii) backward pass and iv) weight update. In the forward pass, the first set of training image does not give any preference to any class. Then, the loss function is calculated by the equation $loss function = \sum \frac{1}{2} (T - PO)^{2}$ (15) where T is the expected output, PO is the predicted output for the epoch. In backward pass the loss function calculated over the output layer is passed back through the network to the hidden units. The last step of backpropagation is to update the weights of input to hidden layers.

4.1 Transfer learning with CNN

CNN architecture is a subsidiary for training and testing data. 75% of image dataset used for training. Only if the training images are more than the testing, precision will be more efficient. Transfer learning is the re-usability of a previously trained model for a new problem. The advantages of transfer learning includes preserve training time, efficient performance of neural networks and not requires a lot of data. In deep learning, this method does the initial training of a CNN for a particular task like object classification and object detection. The pre-trained ImageNet model contains the ImageNet architecture and learned weights. This pre-trained model used in our proposed for defining the initial weights of CNN network to performs the detection task. This process is used to lessen the computational costs of training a deep neural network from the beginning, or to hold the necessary feature extractors trained during the initial stage.

4.1.1 Algorithm for training

Input: f(RGB Image) Output: Classified Object with Label

Step 1 The input image f(m, n) is converted into quaternion image Q(m, n) by using the Equation (9).

Step 2 Conversion of quaternion image into Fourier domain by e (11).

Step 3 Initialization of CNN network weights using the pretrained model of Imagenet.

Step 4 Do the steps (5) to (9) for every epoch untill error rate reduced to zero:

Step 5 According to the CNN architecture, the convolution and pooling are done thrice by Equations (12) and (13)

Step 6 Next, the ReLU which is a non-linear function is applied to the previous output.

Step 7 To convert the image from Fourier domain to spatial domain by Equation (14), the inverse QFT is applied to the output obtained from Step 6.

Step 8 Then the fully connected layer is applied to the spatial domain image using the Equations (15) and (16)

Step 9 Calculate the error rate by Equation (17) for backward propagation.

The Remaining 25% of the images from the dataset executed for testing. Steps 1, 2, 5, 6, 7 and 8 without backward propagation in training algorithm is applicable for the testing process and from the response map the object is detected.

5 Experiment analysis

The experiment is virtually conducted in the ubuntu environment using python programming language. Tensorflow and Keras libraries in the python programming used to implement the deep learning concept of CNN.

5.1 Data analyzation

In order to do experiment analysis, we have used INRIA [40], HDA [41] which has both static and moving person dataset. In these datasets, training and testing data were separately given. The training image has pixels of 96 x 160, the testing image has a resolution of 70 x 134. To avoid the boundary conditions for processing, the testing image size is scaled as same as the training images [36]. For positive and negative training 1208 and 1218 images taken respectively. The images are taken from indoor, outdoor, city, mountain, and beach. HDA person dataset is for the dataset of high definition surveillance. 18 cameras taken videos duration of 30 mins. The videos are taken from different cameras of resolution 640 x 480, 1280 x 800, 2560 x 1600. The dataset is analyzed in both the spatial and Fourier convolution domain. For computation complexity, whenever the image resolution is less, the result yielded will be more or less the same otherwise there will be a major impact on the quaternion Fourier convolution process which results in reduced computational time rather then the spatial convolution.

5.1.1 Comparative analysis

The performance of the proposed method is efficiently compared with the classical CNN [37] (which examines the raw spatial image as the input from which the object feature is extract) and with the discrete Fourier transform (DFT) based CNN, the resultant images are displayed in Fig. 4. On moving to large and larger dataset, the spatial image will require more computation time and memory. In DFTCNN, the spatial image is transformed into Fourier domain. Processing of image in Fourier domain reduce the computational time. Still there exists a computation complexity because of handling r, g and b values as individual components. So still there exits a computation and recollection requirement problem. To overcome this situation proposed method (DQFT) uses quaternion complex number to be processed with Fourier to handle the rgb components simultaneously. Then DQFT image is given to the CNN to handle the limitations of the anterior models (CNN, DFTCNN). RCNN [38] is a method which integrates CNN with the Regional Proposal method. Training is expensive for a larger dataset and testing process is gradual. Masked RCNN [39] is a simple effective framework for object instance segmentation which overcomes the shortfall of RCNN but it has a major drawback of avail in authentic-time applications because of the computational complexity. Our proposed method DFTCNN is cost-efficient and most suitable for real time applications.

Fig. 4

Object Detection Results of DQFTCNN with other Current Methods.

Jin et al. [42] used quaternion Fourier transform in spatial domain for orientation detection which helps dynamically to update the vector median filter with respect to size and shape. But, our proposed work enhances the image using quaternion Fourier transform as a pre-processing step. To improve and incorporate all the color channels in the singe component form, we have used quaternion with Fourier transform. Here, the CNN is used train the human object features. Guo et al. [43] handled Quaternion Fourier transform (QFT) to obtain the location of salient areas as well as to represent the image as spatial-temporal saliency. In QFT, they have been extracted only phase QFT, by leaving the magnitude. They have handled the three imaginary components as intensity, color and motion feature. But in our proposed work, we have taken r, g and b color components as the three imaginary components and also used the whole QFT, instead of phase QFT. Because, our concept of work is to enhance the image, not to detect the location of the object.

5.1.2 Evaluation metrics

The performance of the proposed work is evaluated using the mean average precision (mAP) method. In this method, after calculating the precision and recall, mAP is computed by using the average precision. Precision (P) gives the true detected objects by the total number of true positive and false positive objects. Recall (R) gives the measurement of the true detected objects by the total number of true positive and false negative objects in the image. In mAP the precision and recall computed using the below equations: $P = \frac{T_{p}}{T_{p} + F_{p}}, R = \frac{T_{p}}{T_{p} + F_{n}}$ (16) where T_p - true positive, F_p - false positive and F_n - false negative. The mean average precision (mAP) is computed by $mAP = \frac{1}{classes} \sum_{c \in classes} \frac{T_{p}}{T_{p} + F_{p}}$ (17) Compared to classical CNN, the proposed method works better for a larger dataset with high detection rate. DFTCNN inclines more expeditious implementation than CNN but proposed DQFTCNN works faster than these two methods. Our proposed method produce efficient object detection, the average precision values compared to CNN, DFTCNN, RCNN, and MaskRCNN are given below:

In Table 1, the 2^nd, 3^rd and 4^th column represents the average precision values of the images. Figure 5 shows the mAP results for different methods.

Table 1

mAP evaluation metric for Different methods with proposed method for INRIA and HDA datasets

Methods	INRIA	HDA1	HDA2	mAP
Average precision values
CNN	40.3	62.3	67.84	56.81
DFTCNN	53.56	74.82	79.56	69.31
RCNN	51.38	78.16	73.25	67.59
MaskRCNN	76.93	82.45	79.64	79.67
DQFTCNN	73.62	81.74	89.53	81.63

Fig. 5

Graphical representation of mAP.

6 Conclusion

Our proposed method of object detection using CNN with discrete quaternion Fourier transform (DQFT) influences the holistic color image process leads to less computation and more efficient detection. The novelty of our work is DQFT processed image is passed to the CNN process. Further, background modeling is used for background subtraction to eliminate non-moving objects in the background. The experimental analysis shows that the results obtained from the proposed method are more efficient than the prior works. The proposed work can be directly incorporated for the real-time applications for security management and for the surveillance system.

In future, the work has been extended with other algebraic approaches like octonion number and other statistical modeling which helps to enhance the image by considering more features of the object. So, there occurs ease of CNN training for detecting the objects. Added to that, GPU implementation makes more powerful while handling larger dataset.

Footnotes

Acknowledgments

The work of R.Sundara Rajan was partially supported by Project No. 2 /48(4)/

2016/NBHM-R D-II/11580, National Board of Higher Mathematics (NBHM), Department of Atomic Energy (DAE), Government of India.

References

Zheng

, Li

and Yao

, LHRNet: Lateral hierarchically refining network for salient object detection, Journal of Intelligent & Fuzzy Systems 37(2) (2019), 2503–2514.

Ramík

D.M.

, Sabourin

, Moreno

and Madani

, A machine learning based intelligent vision system for autonomous object detection and recognition, Applied Intelligence 40(2) (2014), 358–375.

Arya

, Singh

and Agrawal

R.K.

, A novel combination of second-order statistical features and segmentation using multi-layer superpixels for salient object detection, Applied Intelligence 46(2) (2017), 254–271.

Wang

and Wu

, Salient object detection using biogeography-based optimization to combine features, Applied Intelligence 45(1) (2016), 1–17.

Liang

, Liu

, Xu

, Xiang

and Zou

, Multi-task gloh feature selection for human age estimation, In 2011 18th IEEE International Conference on Image Processing, pp. 565–568, 2011.

, Deng

, Li

, Lei

, Liang

, Xu

and Li

, Advanced SURF features based flexible object detection, In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1910–1915, 2019.

Revathi

and Rajalaxmi

T.M.

, Deep learning for people counting model, Soft Computing for Problem Solving-Advances in Intelligent Systems and Computing (Springer), 1048, 2020.

Erhan

, Szegedy

, Toshev

and Anguelov

, Scalable object detection using deep neural networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147–2154, 2014.

, Hu

, Wang

, Wei

and Dai

, Learning region features for object detection, In Proceedings of the European Conference on Computer Vision ECCV, pp. 381–395, 2018.

10.

Zhao

Z.Q.

, Zheng

, Xu

S.T.

and Wu

, Object detection with deep learning: A review, IEEE Transactions on Neural Networks and Learning Systems 30(11) (2019), 3212–3232.

11.

Kim

, Lee

, Han

and Kim

Y.M.

, A hybrid framework combining background subtraction and deep neural networks for rapid person detection, Journal of Big Data 5(1) (2018), 1–22.

12.

Zhu

, Yeh

M.C.

, Cheng

K.T.

and Avidan

, Fast human detection using a cascade of histograms of oriented gradients, IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2 (2006), 1491–14.

13.

, Sahoo

and Hoi

S.C.

, Recent advances in deep learning for object detection, Neurocomputing, 2020.

14.

Sermanet

, Eigen

, Zhang

, Mathieu

, Fergus

and LeCun

, Overfeat: integrated recognition, localization and detection using convolutional networks, in: arXiv preprint arXiv:1312.6229, 2013.

15.

Liu

, Ouyang

, Wang

, Fieguth

, Chen

and Liu Pietikäinen

, Deep learning for generic object detection: A survey, International Journal of Computer Vision 128(2) (2020), 261–318.

16.

Szegedy

, Toshev

and Erhan

, Deep neural networks for object detection, In Advances in Neural Information Processing Systems, pp. 2553–2561, 2013.

17.

Szegedy

, Liu

, Jia

, Sermanet

, Reed

, Anguelov

, Erhan

, Vanhoucke

and Rabinovich

, Going deeper with convolutions, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9, 2015.

18.

Chu

and Cai

, Deep feature based contextual model for object detection, Neurocomputing 275 (2018), 1035–1042.

19.

Chen

, Lu

and Fan

, S-CNN: Subcategory-aware convolutional networks for object detection, IEEE Transactions on Pattern Analysis and Machine Intelligence 40(10) (2017), 2522–2528.

20.

and Wang

, Object detection using convolutional neural networks in a coarse-to-fine manner, IEEE Geoscience and Remote Sensing Letters 14(11) (2017), 2037–2041.

21.

Kang

, Ouyang

, Li

and Wang

, Object detection from video tubelets with convolutional neural networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 817–825, 2016.

22.

Zhu

, Xu

and Chen

, Quaternion convolutional neural networks, In Proceedings of the European Conference on Computer Vision (ECCV), pp. 631–647, 2018.

23.

Parcollet

, Morchid

, Linares

and De Mori

, Quaternion convolutional neural networks for theme identification of telephone conversations, In 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 685–691, IEEE, 2018.

24.

Parcollet

, Zhang

, Morchid

, Trabelsi

, Linarès

, De Mori

and Bengio

, Quaternion convolutional neural networks for end-to-end automatic speech recognition, arXiv preprint arXiv:1806.07789, 2018.

25.

Yin

, Wang

, Luo

, Zhai

, Jha

S.K.

and Shi

Y.Q.

, Quaternion convolutional neural network for color image classification and forensics, IEEE Access 7 (2019), 20293–20301.

26.

Shang

and Hirose

, Quaternion neural-network-based PolSAR land classification in poincare-sphere-parameter space, IEEE Transactions on Geoscience and Remote Sensing 52(9) (2013), 5693–5703.

27.

Grigoryan

A.M.

and Agaian

S.S.

, Tensor transform-based quaternion Fourier transform algorithm, Information Sciences 320 (2015), 62–74.

28.

CChen

, Yang

, Jeon

and Zhang

, Kernel quaternion principal component analysis and its application in RGB-D object recognition, Neurocomputing 266 (2017), 293–303.

29.

Rizo-Rodríguez

, Méndez-Vázquez

and García-Reyes

, Illumination invariant face recognition using quaternion-based correlation filters, Journal of Mathematical Imaging and Vision 45(2) (2013), 164–175.

30.

da Silva

L.C.

and Lima

J.B.

, The quaternion Fourier number transform, Circuits, Systems, and Signal Processing 37(12) (2018), 5486–5506.

31.

, Zhendong

and Cheolkon

, Fast Fourier transform networks for object tracking based on correlation filter, IEEE Access, pp. 6594–6601, 2018.

32.

Bahri

, Discrete quaternion Fourier transform and properties, International Journal of Mathematical Analysis 7(25) (2013), 1207–1215.

33.

Pei

S.C.

, Ding

J.J.

and Chang

J.H.

, Efficient implementation of quaternion Fourier transform, convolution, and correlation by 2-D complex FFT, IEEE Transactions on Signal Processing 49(11) (2001), 2783–2797.

34.

Hitzer

, Quaternion domain Fourier transform, In Proceedings of the 10-th International Conference on Clifford Algebras and their Applications in Mathematical Physics, 2014.

35.

Parekh

H.S.

, Thakore

D.G.

and Jaliya

U.K.

, A survey on object detection and tracking methods, International Journal of Innovative Research in Computer and Communication Engineering 2(2) (2014), 2970–2978.

36.

Liu

, Fang

, Zhao

, Wang

and Zhang

, Implementation of training convolutional neural networks, arXiv preprint arXiv:1506.01195, 2015.

37.

Pratt

, Williams

, Coenen

and Zheng

, Fcnn: Fourier convolutional neural networks, In Joint European Conference on Machine Learning and Knowledge Discovery in Databases-Springer, pp. 786–798, 2017.

38.

Ren

, He

, Girshick

and Sun

, Faster r-cnn: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6) (2016), 1137–1149.

39.

, Gkioxari

, Dollár

and Girshick

, Mask r-cnn, In Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969, 2017.

40.

INRIA Person Dataset: http://pascal.inrialpes.fr/data/human. In ICCV.

41.

HDA Person Dataset: http://vislab.isr.ist.utl.pt/hda-dataset.

42.

Jin

, Song

and Zhang

, Denoising color images based on local orientation estimation and CNN classifier, Journal of Mathematical Imaging and Vision, pp. 1–27, 2020.

43.

Guo

, Ma

, Zhang

, Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform, In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, 2008.