Cattle face recognition under partial occlusion

Abstract

The occlusion in the real feedlot environment is ubiquitous, and the current research based on the cattle face recognition under occlusion conditions is almost non-existent. Thus, an attention mechanism module with high accuracy and low model complexity is designed to incorporate into MobileNet so that the cattle face under occlusion can be identify accurately, which is the RGB images captured in the ranch environment. In this paper, we also construct a Simmental cattle face image dataset for data modeling and method evaluation, which contains 10,239 images of 103 cattle. The experimental results show that when the occluder is in the upper left and lower right corner, if the occlusion rate is less than 30%, the value of Top_1 reaches more than 90%; if it is less than 50%, the value of Top_1 is more than 80%. Even if the middle part occludes lots of important information, the occlusion rate of 40% has an accuracy of more than 80%. Furthermore, comparing the proposal model with MobileNet, the parameter and model size are equal, and the amount of calculation as a cost increase a little. Therefore, the proposal model is suitable to transplant to the embedded system in the future.

Keywords

Partial occlusion attention mechanism cattle face recognition convolutional neural network image classification

1 Introduction

In recent years, animal biometrics has been a very popular and promising research field [1 –3]. Among them, the identification of cattle plays an important role in registration and traceability [4 –6]. In the past few years, with the widespread and successful use of biometric applications, cattle face recognition has received widespread attention [7]. For example, a combination of traditional feature extraction operators and image processing techniques is used to recognize cattle faces in the references [8 –10]. The above method has not been widely used because it relies excessively on manually designed components and has to go through a complicated parameter adjustment process, and poor generalizability and robustness. With the development of deep learning [11, 12], convolutional neural networks (CNNs) have made huge breakthroughs in many fields. Reference [13] shows that CNNs are feasible in solving the problem of animal biometrics. Therefore, the cattle face recognition based on CNNs is studied in the references [14 –16], artificially designed components are not dependent and have high recognition accuracy. The above studies are all aimed at cattle face recognition under constrained conditions, while the cattle face images collected in the real feedlot environment are all unconstrained, that is to say, the occlusion phenomenon is inevitable. Therefore, we verify the recognition of the occluded image by the traditional CNNs on the dataset we made. The experimental results are shown in Fig. 1. The blue shows that the recognition accuracy of traditional CNNs for positive unobstructed cattle face images is above 90%; The green shows the recognition accuracy results of the traditional CNNs after 40% of the cattle face is occluded. It can be found that the recognition accuracy of each network is less than 84%. The results show that the traditional CNNs have better feature extraction ability for unoccluded data, and the recognition effect of occluded data needs to be improved urgently. And it can be seen from the figure that Mobilenet [17] has a good recognition accuracy compared with other models under occlusion conditions, and also has a very low parameter, which is very useful for the transplantation of embedded systems.

Fig. 1

The accuracy of traditional convolutional neural network for cattle face recognition with or without occlusion.

At present, the research on image recognition under occlusion conditions is focused on human face recognition [18 –23]. There are no reports on the research of cattle face recognition under occlusion conditions. Therefore, we decided to study the cattle face recognition under occlusion conditions by combining the unique characteristics of cattle and based on the research results and methods of face recognition under occlusion conditions. One of the methods of face recognition under occlusion conditions is to use the attention mechanism to distinguish the occluded and unoccluded part, so that the feature information of the unoccluded part is focused and extracted by the network, and various methods have been proposed to integrate the attention mechanism into CNNs [21 –23]. Therefore, on the basis of the above research, a channel attention mechanism module with high accuracy and low model complexity is designed in this paper. The specific manifestation is the introduction of Lp Pooling which is a complex spatial aggregation strategy and the use of lightweight Efficient Channel Attention (ECA) module to reduce the amount of overall network parameters. From Fig. 1, it can be found that MobileNet has the advantages of lightweight and high recognition accuracy in occluded images, so it is selected as the baseline network of the model of the cattle face recognition under occlusion. And the accuracy of cattle face recognition under occlusion conditions is verified on the dataset we make. The experimental results show that the attention mechanism module we designed can improve the accuracy of cattle face recognition under occlusion conditions.

Our main contributions are summarized as follows:

Dataset. Since there is no public standard face dataset of Simmental beef cattle, the data is collected and screened in a real feedlot environment. And a system based on Yolov3 to automatically filter background noise is designed to automatically generate the dataset for classification task, and on this basis, Python language is used to automatically add occlusion factors.

Architecture of attention mechanism. By analyzing the traditional channel-based attention architecture, Lp Pooling is introduced to improve the representation ability of the global receptive field, which is a more complex spatial aggregation strategy; The traditional channel excitation method is realized by a fully connected bottleneck structure, which carries a large number of trainable parameters, so the fully connected structure is replaced by ECA-Net, which is a lighter module.

Improve the accuracy of cattle face recognition under occlusion conditions. The attention mechanism module we designed is plugged into MobileNet. Experiments on the cattle face dataset under occlusion conditions showed that it can distinguish between occluded and unoccluded parts. And the identity of the cattle is distinguished based on the characteristics of the unoccluded part.

The rest of this paper is organized as follows. The Section 2 describes the process and details of the dataset production. The Section 3 explains our attention mechanism architecture. The Section 4 explains the experimental configuration and analyzes the experimental results. The Section 5 summarizes this paper.

2 Preparation and description cattle face image database

This section mainly introduces the acquisition of Simmental beef cattle dataset, the extraction of key frames, the filtering of background noise and the addition of occlusion factors. Figure 2 shows the flow chart of dataset production.

Fig. 2

The production process of cattle face dataset.

Since Simmental beef cattle is not only the second largest breed in the world after Holstein cows, but also the largest breed of beef cattle in the world, the research object of this paper is Simmental beef cattle. The video data in this paper was collected by members of the same group who went to the cattle farm twice, where the video images of 48 cattle were collected at the Shengyuan beef cattle farm in Hebei, China from July to August 2018; Another 55 cattle were collected in Kulun Banner, Tongliao, Inner Mongolia Autonomous Region, China from August to September 2020. All the cattle in the cattle farm were kept in a large open-air environment surrounded by fences. A handheld mobile camera bracket was used to enter the farm to collect cattle videos at close range, the equipment used was a Sony camera. The resolution of the image was 1920×1080, and the fps was set to 50 to reduce the impact of blurring caused by the relative increase in movement of the cattle. During the shooting, the sunny weather without fog and haze was selected, and the shooting was carried out at 8 : 00-16 : 00. Finally, 295 videos of 103 cattle were shot. Each video was about 2-4 min long and saved in mp4 format. Our raw and annotated datasets have been shared at https://github.com/Oliver6999/cattle_dataset/tree/main.

2.1 Extraction of cattle face image

After obtaining the cattle face videos with background noise, the framing software of Free Video to JPG Converter was used to decompose each video to 1000 frames. The image after framing contain a large number of invalid data, such as it only contains the background, the deflection angle of the cattle face is too large to recognize any valid facial information, occluded entirely by other cattle, and the adjacent images are very similar. Aiming at the problems caused by the above-mentioned framing, a combination of manual selection and algorithmic filtering is used to deal with in this paper. Manual selection is mainly aimed at the first three problems, and the images after manual selection are sent to the algorithm composed of Structural Similarity (SSIM) [24] to filter the similarities between adjacent images. The filtering algorithm is as follows:

Algorithm 1.

Key frame extraction

Inputs:

F_i{The i-th image};

N{Image number of the cattle sequence};

Operation process:

For i = 1 to N

S_i,i +1: Similarity computation between two adjacent images;

if S_i,i +1 ≥ Th; Delete the repeated image, Th is the set

threshold

If S_i,i +1 < Th; No processing and return to the for loop

End for

Outputs:

The similarity of the image sequence is lower than Th

Key frame extraction
Inputs:
F_i{The i-th image};
N{Image number of the cattle sequence};
Operation process:
For i = 1 to N
S_i,i +1: Similarity computation between two adjacent images;
if S_i,i +1 ≥ Th; Delete the repeated image, Th is the set
threshold
If S_i,i +1 < Th; No processing and return to the for loop
End for
Outputs:
The similarity of the image sequence is lower than Th

Where the value of Th is set to 0.32, the number of raw images of 103 cattle is 68260, and the number of images after manual selection and algorithm filtering is 10239 (The number of images of each cattle is 100 except for the four cattle whose numbers are 77, 92, 91, and 79), and about 85% of the images are discarded.

2.2 Filter out background noise

The images after manual selection and algorithm filtering contain background noise, which is contrary to the form of the dataset required by the classification task [25, 26]. The traditional method of filtering background noise is manual cropping. This method not only is time-consuming, but also has low accuracy. Thus, an automatic image capture algorithm based on YOLOv3 [27] is designed to automatically generate a dataset for classification tasks. This method has the characteristics of short time and high accuracy. The algorithm flow chart is shown in Fig. 3:

Fig. 3

Background noise filtering flowchart.

In the training process, 1/5 of all the data were randomly selected for the labeling of YOLOv3 model data. The K-Means algorithm is replaced by K-Means++ [28] to improve the accuracy of anchor box clustering; The DarkNet-53 feature extraction network is replaced by MobileNet to improve the speed of model detection. Then the labeled data is inputted into YOLOv3 for training and the weights are saved. In the testing process, all the data are loaded into the YOLOv3 model and the weights learned in the training process are transplanted to the testing process. The task of the traditional testing process is to detect the target image and draw it with a rectangular box on the raw image. Therefore, the image cropping module is embedded on this basis, that is, the coordinates of the upper left corner and the lower right corner of the rectangular box are passed into the cutting function, the function of automatic cropping is realized by loop.

2.3 Add occlusion factor

Because the occluded cattle face image collected in the real feedlot environment can neither control the occluded area nor the occlusion rate, the actual collected cattle face image with occlusion does not meet the most basic conditions of the experiment (control variables). Therefore, the most direct Image Compositing method [29] is used to apply the occluder to the positive unobstructed cattle face dataset in this section. The raw cattle picture is normalized to M*M, and filled with bilinear interpolation [30]. The size of the occluder can be calculated by M*M*ration, which is percentage of the raw cattle picture is occluded. And the position of the occluder can be obtain by artificially setting the coordinates. The specific operations are shown in Fig. 4. The artificially determine the coordinates of the upper left corner of the occluder (Lw and Lh), where Lw and Lh are in the range of 0 M. According to the size M * M * ration of the occluder, the coordinates of the lower right corner can be obtained. This operation is implemented by the Python language, and a loop is added to avoid the time spent on manual cutting and pasting.

Fig. 4

The representation of the occluded cattle face image.

For specific parameter settings, the raw cattle face image is normalized to 64*64*3. Since a ratio greater than 0.6 is meaningless, the ratio is set to 0-0.6 to calculate the size of the occluder, and different proportions of the occluders are added to the upper left, middle and lower right corners of the raw cattle face picture.

3 Method

The convolution kernel, as the core of the CNNs, is usually regarded as the fusion of spatial information and channel information in the local receptive field. When channel information is fused, the contribution of each channel is the same. Inspired by this, Hu et al. [21] proposed Squeeze-and-Excitation network (SENet) to explicitly model the interdependence between each channel. Specifically, the importance of each channel is obtained by passing the SENet on the feature map, which is get by the convolution operation. And then according to the importance, the essential features are extracted and the useless features for the current task are suppressed. This is similar to the brain signal processing mechanism of the human visual system, which selects and focuses on some salient areas after a series of partial scans, while ignoring the content of other areas [31].

The purpose of cattle face recognition for partial occlusion is to make the computer learn to ignore the feature information of the occluded part of pictures, extract the feature information of the other part. And the feature information is sent to the top layer of the CNNs for classification and recognition. Therefore, a novel attention mechanism module is designed and plugged into MobileNet to further improve the ability of model to distinguish between occluded and unoccluded parts in this paper. The novel attention mechanism module mainly includes two parts: squeeze and excitation.

Squeeze: The feature map is a combination of spatial information and channel information. Since SENet wants to explicitly model the interdependence between each channel, the influence of spatial information should be eliminated firstly, feature compression along the spatial dimension is performed. And global Average pooling (GAP) [32] is adopted to turn each two-dimensional feature channel into a real number, which has a global receptive field to some extent. Woo et al. [23] proposed the Convolutional Block Attention Module (CBAM) architecture can emphasize or suppress and refines intermediate features effectively. Experiments show that Global Max Pooling (GMP) [33] can also compress spatial information and the combined use of GAP and GMP has the highest accuracy, which means that more complex aggregation strategies represent higher recognition accuracy. Inspired by this, the max pooling, average pooling and Lp Pooling [34] are used to compress spatial information at the same time in this paper. Lp Pooling is illustrated by formula 1 [34]. When p = 1, the result of Lp Pooling is sum pooling, which is proportional to average pooling; when p⟶ ∞, it can be found that the result is max pooling through the assisted calculation of scientific computing software. Therefore, when p is between 1 -∞, it can be regarded as a compromise between max pooling and average pooling, which may obtain other unique effects. $f (X) = \sqrt[p]{\sum_{x \in X} x^{p}}$ (1)

Excitation: The squeeze operation obtains the global description feature of each two-dimensional feature channel, and then two fully connected bottleneck structures are used to obtain the dependency between each channel in the traditional excitation method. It can be seen from Table 2 in Section 4.3 that the parameter amount of MobileNet is 3.31M. After adding the SENet module, the parameter amount becomes 3.44M. Since there are no additional trainable parameters in the pooling method of the squeeze part, the added 0.13M is all of the two-layer fully connected structure. The increase of parameters means that the training time becomes longer, the model size becomes larger, and the time to load the model in the actual testing process will also become longer. Therefore, ECA-Net which is a more lightweight excitation method is used to replace the traditional two-layer fully connected structure in this section.

Table 2

Experimental results of squeeze modules composed of different pooling methods on the positive unobstructed cattle face dataset

Model	Top_1(%)	Top_5(%)	MFLOPs	Model size(MB)	Parameter(M)	Average test time(ms)
MobileNet	94.97	99.44	47.27	12.77	3.31	2.19
AvgPool(SE)	95.38	99.38	47.81	13.28	3.44	2.27
MaxPool	95.21	99.66	47.80	13.28	3.44	2.27
LpPool	95.60	99.66	47.80	13.28	3.44	2.24
Max+Avg	95.70	99.51	48.34	13.28	3.44	2.27
Max+Lp	95.41	99.61	48.33	13.28	3.44	2.27
Avg+Lp	95.75	99.66	48.34	13.28	3.44	2.25
Max+Avg+Lp	96.09	99.66	48.86	13.28	3.44	2.26

Qilong et al. [35] proposed an ECA module for deep CNNs, which avoids dimensionality reduction and effectively captures cross-channel interaction information, aiming to ensure computational performance and model complexity. The weight of each channel is calculated only by considering the interaction between the i-th channel and its k neighbors. The calculation formula is shown as follows. $ω_{i} = σ (\sum_{j = 1}^{k} ω_{i}^{j} y_{i}^{j}), y_{i}^{j} \in Ω_{i}^{k}$ (2)

Where y_i represents the i-th input channel, $Ω_{i}^{k}$ represents the set of k adjacent channels of y_i, σ represents the sigmoid function, and the parameter amount involved in this process is K×C. In order to further improve the performance of the model, and then share the trainable parameters of all channels, the final formula is shown in formula 3, and the parameter amount becomes K. And the one-dimensional convolution with the convolution kernel size K is used to realize the information interaction between the channels, so the value of the parameter K needs to be determined next. $ω_{i} = σ (\sum_{j = 1}^{k} ω^{j} y_{i}^{j}), y_{i}^{j} \in Ω_{i}^{k}$ (3)

Although the optimal value range of the convolution kernel K can be adjusted manually, the manual cross-validation will cost lots of computing resources. And further analysis shows that there is a certain mapping between K and the number of channels C, and since the number of channels is usually an exponential multiple of 2, an exponential function with base 2 is used to express the mapping relationship between K and C, so the author proposes a method for adaptively adjusting the size of the convolution kernel, its calculation formula is shown as follows. Where the constantsγand b are set to 2 and 1 according to the experiment. $k = ψ (C) = {| \frac{{log}_{2}^{(C)}}{γ} + \frac{b}{γ} |}_{odd}$ (4)

Therefore, the feature map is calculated by the novel attention mechanism module, illustrated as formula 5, where ⊗ is the element-wise multiplication, and ECA can be calculated by formulas 3 and 4. $\begin{matrix} \tilde{X} = σ (ECA (Global_Max_Pool (X)) \\ + ECA (Global_Average_Pool (X)) \\ + ECA (Global_Lp_Pool (X))) \otimes X \end{matrix}$ (5)

Based on the above analysis, the architecture of attention mechanism module in this paper is shown in Fig. 5. The squeeze part uses the parallel connection of Max Pool, Avg Pool and Lp Pool to aggregate to obtain more effective features. Among them, Lp Pool is used in the attention mechanism architecture for the first time and achieved surprising effects. Then connect with the lightweight ECA-Net to get the activation value of each channel reduce the parameter amount of SENet.

Fig. 5

Architecture diagram of the novel attention mechanism module.

4 Experiment

In this section, ablation analysis is first conducted to verify the feasibility and effectiveness of this algorithm, and MobileNet is adopted as the basic network architecture. The attention mechanism module we designed is plugged in MobileNet and the architecture of the proposal network is shown in Fig. 6. Since the channel attention is composed of squeeze and excitation, an effective method to compose the squeeze module is first searched, and an effective method to reduce the excess of traditional excitation parameters is considered.

Fig. 6

The architecture diagram of the proposal cattle face recognition under partial occlusion network.

4.1 Performance evaluation

To evaluate the performance of the classification network, accuracy (Top_1 and Top_5), parameters, the model size, MFLOPs and the average test time are common performance standards. Among them, parameters and MFLOPs are our focus because of the demands for transplanting model to embedded systems in the future. Top_1 is given as the number of correctly identified images as a percentage of the testing dataset. Top_5 is given as the number of images with the correct label included in the first five classification probabilities as a percentage of the testing dataset. The model size is considered as the storage space occupied by the model and parameters. The average detection time is the average time it took each photo of the cattle face to be identified. The parameters and MFLOPs are calculated by formulas 6 and 7 to represent the complexity of the model. $Params = C_{0} \times (k^{2} \times C_{i} + 1)$ (6) $FLOPs = C_{0} \times (k^{2} \times C_{i} + 1) \times H \times W$ (7)

Where C₀ represents the number of output channels, C_i represents the number of input channels, K represents the size of the convolution kernel, and H,W represents the height and width of the output feature map.

4.2 Experiment setup

All experiments in this paper are based on the hardware platform shown in Table 1 for modeling and training. And all the models and ablation analysis processes in this paper are trained on the basis of the following parameters and hyperparameters. The 10239 cattle face images in the dataset are randomly shuffled and divided into training, validation set and testing set according to the ratio of 6 : 2:2. And the input image size is reduced to 64×64×3, batch_size is 32, the learning rate is 0.001, the loss function is CrossEntropyLoss, the optimizer is Adam [36], and the number of epochs is set to 600.

Table 1
Hardware experiment platform configuration

Name Specific configuration

Platform Ubuntu18.04

CPU Intel i5 9400F @2.9GHz

Memory 16GB

Hard disk(SSD/HDD) 512GB/1TB

GPU NVIDIA GeForce RTX2060, 6GB

GPU acceleration CUDA10.0, cuDNN7.6.5

Deep learning framework Pytorch1.6.0 [37]

Name	Specific configuration
Platform	Ubuntu18.04
CPU	Intel i5 9400F @2.9GHz
Memory	16GB
Hard disk(SSD/HDD)	512GB/1TB
GPU	NVIDIA GeForce RTX2060, 6GB
GPU acceleration	CUDA10.0, cuDNN7.6.5
Deep learning framework	Pytorch1.6.0 [37]

4.3 Ablation analysis

Squeeze module: The first part of the SENet is to generate channel-based statistics by using global average pooling. In practice, both max pooling and Lp pooling have the ability of average pooling to fuse spatial information and reduce feature dimensions. Therefore, this section compares the effects of max pooling, average pooling, Lp pooling and their combined use on the accuracy of cattle face recognition through experiments. The excitation part in the above ablation experiment is still completed by the FC layer in the SE module, and it is plugged after the last convolutional layer of MobileNet. The experimental results of using different pooling methods to form the squeeze module are shown in Table 2. We observe that max pooling and Lp pooling are as meaningful as average pooling, and both have improved accuracy relative to MobileNet. And the best results are obtained by using max pooling, average pooling and Lp pooling at the same time. Therefore, although the combined use of the three pooling methods has brought about a small increase in MFLOPs, the accuracy is improved without reducing the detection speed, so the initial SE network that only uses average pooling to fuse spatial information can be replaced.

Excitation module: The second part of the SENet is to realize the importance of channel information evaluation by using a two-layer fully connected structure. The characteristic of the fully connected structure is that there are many trainable parameters. As shown in Table 2, the parameter amount of the SE module is 3.44M, which is an increase of 0.13M compared to MobileNet. Since the pooling process of the squeeze module has no trainable parameters, all the 0.13M added are the parameters of the fully connected layer, which aggravates the burden of the training process and increases the model size. Therefore, this section verifies the accuracy of replacing the excitation part with ECA module in cattle face recognition. The experimental results are shown in Table 3. Firstly, the calculation amount, parameter amount and model size of the combination of average pooling+ECA module are significantly reduced compared with the combination of average pooling+FC, and is consistent with the baseline network MobileNet. That is to say, although the attention mechanism module is plugged into the baseline network, no parameters are added, and only a small amount of calculation is added as a cost. Finally, the combination of the three pooling methods and the ECA-Net is used as the attention mechanism of this paper.

Table 3
Experimental results of different excitation methods on the positive unobstructed cattle face dataset

Model Top_1(%) Top_5(%) MFLOPs Model size(MB) Param.(M) Average test time(ms)

MobileNet 94.97 99.44 47.27 12.77 3.31 2.19

AvgPool+FC(SE) 95.38 99.38 47.81 13.28 3.44 2.27

AvgPool+ECA 95.99 99.69 47.28 12.77 3.31 2.41

ours 96.35 99.79 47.29 12.77 3.31 2.41

Model	Top_1(%)	Top_5(%)	MFLOPs	Model size(MB)	Param.(M)	Average test time(ms)
MobileNet	94.97	99.44	47.27	12.77	3.31	2.19
AvgPool+FC(SE)	95.38	99.38	47.81	13.28	3.44	2.27
AvgPool+ECA	95.99	99.69	47.28	12.77	3.31	2.41
ours	96.35	99.79	47.29	12.77	3.31	2.41

4.4 Image classification in occluded cattle face

The robustness of our proposed method is tested for cattle face recognition under partial occlusion conditions in this section. Since the mutual occlusion between the faces of two cattle is more frequent in the actual environment, the face of other cattle is chosen as the occluder in the paper. In order to verify the anti-interference of the algorithm in this paper, we choose to set different occlusions in the upper left corner, middle and lower right corner of the cattle face picture. Figure 7 shows a 0 to 60% occlusion of the cattle face on the different part.

Fig. 7

Images of a cattle face under different occlusion rates and parts.

MobileNet is used as the baseline network, and the settings of parameters and hyperparameters are consistent with section 4.2. The results of specific test samples are shown in Table 4. The results show that under different occlusion rates, the recognition accuracy of the upper left corner and the lower right corner of the cattle face image is almost the same after being occluded. When the occlusion rate is less than 30%, the value of TOP_1 can reach more than 90%; when the occlusion rate is less than 50%, the value of TOP_1 can reach more than 80%; when the occlusion rate is more than 60%, the accuracy drops sharply. It can be seen from Fig. 7 that when the occlusion rate is 60%, most of the facial features of the cattle are missing, and it is difficult to achieve high recognition accuracy only by relying on the remaining facial information. Since the occluder occludes more facial information in the middle part, the recognition accuracy is much lower than when the occluder is in the upper left corner and the lower right corner. When the occlusion rate is less than 20%, the value rate of TOP_1 can reach more than 90%; when the occlusion rate is less than 40%, the value of TOP_1 can reach more than 80%; when the occlusion rate reaches more than 50%, the accuracy drops sharply. And this can be verified from Fig. 8. When the occlusion rate is lower than 40%, the two curves fit well and basically overlap; then as the occlusion rate increases, the fit of the two curves continues to deteriorate. The value of TOP_1 is also declining. The above experimental results show that our proposed method has good experimental results on the occluded cattle face dataset, and can meet the daily cattle face recognition task under partial occlusion.

Table 4

Cattle face recognition results under different occlusion rates and parts. Among them, Left, Right and Center represent the upper left corner, lower right corner and middle of the cattle face image respectively, which are occluded

Accuracy	0	10%	20%	30%	40%	50%	60%
Left_Top_1(%)	96.35	93.99	93.84	91.63	88.55	82.07	67.54
Left_Top_5(%)	99.79	99.43	98.41	97.38	95.69	91.63	80.79
Right_Top_1(%)	96.35	94.72	93.06	90.47	87.25	81.09	69.96
Right_Top_5(%)	99.79	99.51	98.97	97.17	94.85	90.52	81.44
Center_Top_1(%)	96.35	93.89	90.18	86.47	81.05	71.81	57.40
Center_Top_5(%)	99.79	99.46	98.44	95.90	91.84	88.18	78.94

Fig. 8

Convergence curves of the accuracy of the training and the validation set under different occlusion rates, when the occluder is in the middle of the cattle face picture.

5 Conclusion

The occlusion phenomenon in the real feedlot environment is ubiquitous, and the application of the existing model is very bad in this environment. Thus, we research the cattle face recognition under the occlusion condition by combining the unique features of the cattle and the attention mechanism. In this paper, an attention mechanism module with high accuracy and low model complexity is designed and plugged into MobileNet to improve the effect of cattle face recognition under occlusion. Experiments show that when the occlusion rate is less than 20%, the accuracy of our proposed method is more than 90%; when the occlusion rate is less than 40%, the accuracy is more than 80%. Moreover, compared with the MobileNet, no parameter or model size is increased, and only a small increase in the amount of calculation is taken as a cost. The experimental results of the traditional CNNs and our proposed method on the dataset where 40% of the cattle face image is occluded are shown in Table 5. The experimental results show that the model we designed can distinguish the occluded and unoccluded part in the image, and extract the feature information of the unoccluded part to recognize and classify. And our novel model is suitable for transplanting into the embedded system applying in the real feedlot environment. Furthermore, our proposal method can be applied to recongnize the others animals under occlusion not only by face but also by back, side and so on.

Table 5

The results of our proposed method and traditional CNNs on the cattle face dataset with an occlusion rate of 40%

Model	Parameter(M)	MFLOPs	Top_1(%)	Top_5(%)
VGG16 [38]	40.32	1281.69	81.78	91.21
ResNet50 [39]	23.72	335.75	83.73	91.55
InceptionV3 [40]	22.00	5729.31	83.83	92.67
DenseNet [41]	7.19	2871.28	82.17	90.67
SqueezeNet [42]	0.79	745.50	78.75	86.37
MobileNet [17]	3.31	47.27	83.00	91.74
ours	3.31	47.29	88.55	95.69

Footnotes

Acknowledgments

Thanks for the support of national natural science fund project (61640011) and the Natural Science Foundation of Inner Mongolia (2016MS0617).

References

Weiwei Zhang , Jian Sun and Xiaoou Tang , From tiger to panda: animal head detection, IEEE Transactions on Image Processing 20(6) (2011), 1696–1708.

Kühl

H.S.

and Burghardt

, Animal biometrics: quantifying and detecting phenotypic appearance, Trends in Ecology & Evolution 28(7) (2013), 432–441.

Helena Russello , Rik van der Tol and Gert Kootstra , T-LEAP: Occlusion-robust pose estimation of walking cows using temporal information, Computers and Electronics in Agriculture 192 (2022), 106559.

Barry

, Gonzales-Barron

, McDonnell

, Butler

and Ward

, Using muzzle pattern recognition as a biometric approach for cattle identification, American Society of Agricultural and Biological Engineers 50(3) (2007), 1073–1080.

Juan Larregui

, Dario Cazzato and Silvia Castro

, An image processing pipeline to segment iris for unconstrained cow identification system, Open Computer Science 9(1) (2019), 145–159.

Carlos Silva

, Daniel Welfer

and Cláudia Dornelles

, The pattern recognition in cattle brand using bag of visual words and support vector machines multi-class, Inteligencia Artificial 21(61) (2018), 1–13.

Kumar

, Tiwari

and Singh

S.K.

, Face recognition for cattle. Paper presented at the 2015 Third International Conference on Image Information Processing (ICIIP); 2015 Dec 21–24; Waknaghat, India, IEEE, 65–72.

Kim

H.T.

, Choi

H.L.

, Lee

D.W.

and Yoon

Y.C.

, Recognition of individual Holstein cattle by imaging body patterns, Asian-Australasian Journal of Animal Sciences 18(8) (2005), 1194–1198.

Cai

and Li

, Cattle face recognition using local binary pattern descriptor. Paper presented at the 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference; 2013 Oct 29 –Nov 1; Kaohsiung, Taiwan, IEEE, 1–4.

10.

Kumar

, Tiwari

and Singh

S.K.

, Face recognition of cattle: can it be done?. Proceedings of the National Academy of Sciences, India Section A: Physical Sciences 86(2) (2016), 137–148.

11.

Voulodimos

, Doulamis

and Protopapadakis

, Deep learning for computer vision: A brief review, Computational Intelligence and Neuroscience 2018 (2018), 7068349.

12.

O’Mahony

, Campbell

, Carvalho

, Harapanahalli

, Hernandez

G.V.

, Krpalkova

, Riordan

and Walsh

, Deep learning vs. traditional computer vision. Paper presented at the Science and Information Conference. 2019 April 24; Springer, Cham. 128–144.

13.

Hansen

M.F.

, Smith

M.L.

, Smith

L.N.

, Salter

M.G.

, Baxter

E.M.

, Farish

and Grieve

, Towards on-farm pig face recognition using convolutional neural networks, Computers in Industry 98 (2018), 145–152.

14.

de Lima Weber

, de Moraes Weber

V.A.

, Menezes

G.V.

, Junior

A.d.S.O.

, Alves

D.A.

, de Oliveira

M.V.M.

, Matsubara

E.T.

, Pistori

and de Abreu

U.G.P.

, Recognition of Pantaneira cattle breed using computer vision and convolutional neural networks, Computers and Electronics in Agriculture 175 (2020), 105548.

15.

Yao

, Hu

, Liu

, Kuang

and Gao

, Cow face detection and recognition based on automatic feature extraction algorithm. Paper presented at the Proceedings of the ACM Turing Celebration Conference-China. 2019 May; China, 95:1–5.

16.

Andrew

, Gao

, Campbell

, Dowsey

A.W.

and Burghardt

, Visual identification of individual holstein friesian cattle via deep metric learning. 2020;arXiv preprint, arXiv:2006.09205.

17.

Howard

A.G.

, Zhu

, Chen

, Kalenichenko

, Wang

, Weyand

, Andreetto

and Adam

, Mobilenets: Efficient convolutional neural networks for mobile vision applications. 2017;arXiv preprint, arXiv:1704.04861.

18.

Jialian Wu , Chunluan Zhou , Ming Yang , Qian Zhang , Yuan Li and Junsong Yuan , Temporal-Context Enhanced Detection of Heavily Occluded Pedestrians. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 05 August 2020, 1–13.

19.

Guan’an Wang , Shuo Yang , Huanyu Liu , Zhicheng Wang , Yang Yang , Shuliang Wang , Gang Yu , Erjin Zhou and Jian Sun , High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identifification. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 1063–6919.

20.

Lei Ke , Yu-Wing Tai and Chi-Keung Tang , Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers. 2021 Computer Vision and Pattern Recognition arXiv:2103.12340.

21.

, Shen

and Sun

, Squeeze-and-excitation net-works. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2018; Salt Lake City, UT, USA, IEEE, 7132–7141.

22.

Park

, Woo

, Lee

J.-Y.

and Kweon

I.S.

, Bam: Bottleneck attention module. 2018;arXiv preprint, arXiv:1807.06514.

23.

Woo

, Park

, Lee

J.-Y.

and Kweon

I.S.

, Cbam: Convolutional block attention module. Paper presented at the Proceedings of the European conference on computer vision (ECCV). 2018; Munich, Germany, Springer, 3–19.

24.

Wang

, Bovik

A.C.

, Sheikh

H.R.

and Simoncelli

E.P.

, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing 13(4) (2004), 600–612.

25.

Tharwat

, Gaber

and Hassanien

A.E.

, Two biometric approaches for cattle identification based on features and classifiers fusion, International Journal of Image Mining 1(4) (2015), 342–365.

26.

Kumar

and Singh

S.K.

, Automatic identification of cattle using muzzle point pattern: a hybrid feature extraction and classification paradigm, Multimedia Tools and Applications 76(24) (2017), 26551–26580.

27.

Redmon

and Farhadi

, Yolov3: An incremental improvement. 2018;arXiv preprint, arXiv:1804.02767.

28.

Arthur

and Vassilvitskii

, k-means++: The advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms. 2006 Jun 06; Stanford. p. 1027–1035.

29.

Szeliski

, Computer vision: algorithms and applications. Springer Science & Business Media. 2010.

30.

Kirkland

E.J.

, Bilinear interpolation. In Advanced Computing in Electron Microscopy (2010), 261–263.

31.

Corbetta

and Shulman

G.L.

, Control of goal-directed and stimulus-driven attention in the brain, Nature Reviews Neuroscience 3(3) (2002), 201–215.

32.

Lin

, Chen

and Yan

, Network in network. 2013;arXiv preprint, arXiv:1312.4400.

33.

Azizpour

, Sharif Razavian

, Sullivan

, Maki

and Carlsson

, From generic to specific deep representations for visual recognition. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) workshops. 2015; Boston, MA, USA, IEEE, 36–45.

34.

Rawat

and Wang

, Deep convolutional neural networks for image classification: A comprehensive review, Neural Computation 29(9) (2017), 2352–2449.

35.

Qilong

, Banggu

, Pengfei

, Peihua

, Wangmeng

and Qinghua

, ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. 2020;arXiv preprint, arXiv:1910.03151.

36.

Kingma

D.P.

and Ba

, Adam: A method for stochastic optimization. 2014;arXiv preprint, arXiv:1412.6980.

37.

Pytorch. http://pytorch.org/ Accessed: 2017-11-08.

38.

Simonyan

and Zisserman

, Very deep convolutional networks for large-scale image recognition. 2014;arXiv preprint, arXiv:1409.1556.

39.

, Zhang

, Ren

and Sun

, Deep residual learning for image recognition. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2016; Las Vegas, NV, USA, IEEE, 770–778.

40.

Szegedy

, Vanhoucke

, Ioffe

, Shlens

and Wojna

, Rethinking the inception architecture for computer vision. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2016; Las Vegas, NV, USA, IEEE, 2818–2826.

41.

Huang

, Liu

, Van Der Maaten

and Weinberger

K.Q.

, Densely connected convolutional networks. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2017; Honolulu, HI, USA, IEEE, 4700–4708.

42.

Iandola

F.N.

, Han

, Moskewicz

M.W.

, Ashraf

, Dally

W.J.

and Keutzer

, SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and<0.5 MB model size. 2016;arXiv preprint, arXiv:1602.07360.

Cattle face recognition under partial occlusion

Abstract

Keywords

1 Introduction

Table 1 Hardware experiment platform configuration Name Specific configuration Platform Ubuntu18.04 CPU Intel i5 9400F @2.9GHz Memory 16GB Hard disk(SSD/HDD) 512GB/1TB GPU NVIDIA GeForce RTX2060, 6GB GPU acceleration CUDA10.0, cuDNN7.6.5 Deep learning framework Pytorch1.6.0 [37]

Footnotes

Acknowledgments

References

Table 1
Hardware experiment platform configuration

Name Specific configuration

Platform Ubuntu18.04

CPU Intel i5 9400F @2.9GHz

Memory 16GB

Hard disk(SSD/HDD) 512GB/1TB

GPU NVIDIA GeForce RTX2060, 6GB

GPU acceleration CUDA10.0, cuDNN7.6.5

Deep learning framework Pytorch1.6.0 [37]