SCS-YOLOv5s: A cattle detection and counting method for complex breeding environment

Abstract

Cattle detection and counting is one of the most important topics in the development of modern agriculture and animal husbandry. The traditional manual monitoring methods are inefficient and constrained by factors such as site. To solve the above problems, a SCS-YOLOv5 cattle detection and counting model for complex breeding scenarios is proposed. The original SPPF module is replaced in the YOLOv5 backbone network with a CSP structured SPPFCSPC. A CA (Coordinate Attention) mechanism is added to the neck network, as well as the SC (Standard Convolution) of the Neck network is replaced with a light convolution GSConv and Slim Neck is introduced, and training strategies such as multi-scale training are also employed. The experimental results show that the proposed method enhances the feature extraction ability and feature fusion ability, balances the localization accuracy and detection speed, and improves the use effect in real farming scenarios. The Precision of the improved network model is improved from 93.2% to 95.5%, mAP@0.5 is improved from 94.5% to 95.2%, the RMSE is reduced by about 0.03, and the FPS reaches 88. Compared with other mainstream algorithms, the comprehensive performance of SCS-YOLOv5 s is in a leading position, with fewer missed and false detections, and the strong robustness and generalization ability of this model are proved on multi-category public datasets. Applying the improvement ideas in this paper to YOLOv8 s also yields an increase in accuracy. The improved method in this study can greatly improve the accuracy of cattle detection and counting in complex environments, and has good real-time performance, so as to provide technical support for large-scale cattle breeding.

Keywords

Cattle detection counting attention mechanism occlusion complex environments

1 Introduction

The health of cattle determines the development and economic efficiency of dairy farming. Precision husbandry plays an important role in the process of intensive cattle breeding, and has achieved more and more positive results in terms of pasture protection, animal welfare, workforce optimization, livestock traceability, understanding of disease trajectories and quarantine of major diseases, vaccination, production management, tracking and health monitoring, and large-scale cattle ownership management [1–4]. In the process of intelligent farming, along with the birth, death and sale of cattle, the inventory and counting of cattle has become an important issue in the process of intelligent breeding. Therefore, real-time counting of dense herds is of great significance for the construction of modern animal husbandry. Currently, cattle counting on farms is mainly manual, resulting in a lot of wasted human and material resources, extremely low efficiency and large errors in counting results. The use of wearable devices for monitoring has great advantages over manual work [5], but can cause stress to cattle. Contactless methods based on computer vision have become an important tool for intelligent cattle counting in recent years [6], saving workforce input and improving management efficiency, while offering significant advantages in terms of animal welfare, and playing an important role in intelligent cattle breeding.

However, with the increasing scale of the ranch and the increasing real-time requirements of digital farming, the inventory and counting of cattle are facing new challenges. On the one hand, it is vital to improve the accuracy of cattle counting, while on the other hand, it is necessary to ensure the real-time performance of the algorithm.

Recent research in the field of livestock population counting at home and abroad includes three main methods, which are based on object detection, instance segmentation and density estimation, respectively. Instance segmentation method and density estimation method have their own advantages, but their shortcomings are also obvious, neither can satisfy the requirements of accuracy and speed at the same time.

Instance segmentation has a good performance in solving the occlusion problem, but its biggest drawback is the large model and slow inference speed, which makes it difficult to meet real-time requirements.

Methods based on density estimation are mostly applied to UAV scenarios with a large number of objects and high density. However, the biggest shortcoming of the density estimation method is the low accuracy of counting, which is difficult to meet the needs of large-scale aquaculture.

Compared to the other two methods, the object detection method is the most widely used and can achieve both high accuracy and fast inference to meet real-time requirements. Therefore, the counting method based on object detection was used in this study.

What’s more, most of the existing cattle counting studies only focus on a single breeding scene, which leads to poor generalization ability of the model, and the small number of cattle is less affected by external influences such as occlusion and background interference. To solve the above problems, this study uses rich experimental scene dataset, taking into account interference such as occlusion and complex backgrounds. In order to improve the detection accuracy and real-time performance, this study proposes a cattle detection and counting method based on improved YOLOv5. The main contributions of this paper are as follows:

1)
The fast spatial pyramid pooling in the backbone using the CSP cross-stage local connectivity idea fully integrates the features of each layer of the network, improves the feature extraction capability, and enhances the accuracy of detection.
2)
In order to make the network pay more attention to the foreground image and resist the interference of the complex background, the CA mechanism is added to Neck so that the network pays more attention to the detailed features of the cattle, which can effectively solve the occlusion problem.
3)
Replacing the SC of Neck with GSConv, the same CSP idea is used to introduce Slim Neck containing GSConv Block, which balances the accuracy and speed of the model and ensures the real-time performance of the algorithm.

2 Related work

2.1 Methods based on instance segmentation

Among the many types of inventory and counting algorithms, in order to improve the accuracy, many researchers have conducted many studies for instance segmentation methods. Xu et al. [7] applied Mask R-CNN to cow counting, and achieved good results in both wild grazing and captive breeding scenarios. Hu et al. [8] solved the occlusion problem well by improving this network and integrating the deep and shallow features of the network. Based on SOLO v2, Wang et al. [9] proposed a high-density herd pig counting model integrating multi-scale features, which has good performance on public datasets. Wei et al. [10] used Faster R-CNN based Transfer learning method to detect and count wheat grains, with a low error rate. Singh et al. [11] combined Mask-RCN and a bidirectional convolutional long-term memory network, the counting accuracy outperformed more sophisticated techniques.

2.2 Methods based on density estimation

When targeting dense scenes and small objects in the UAV’s field of view, the instance segmentation method is no longer able to meet the speed requirements, and the density estimation method becomes a new option. Zhao et al. [12] used Bayesian Loss model to count sheep and achieved good results. Liu et al. [13] proposed a crowd density estimation method based on Multiple Information Aggregation, which solved the problems of difficulty in feature extraction and insufficient feature fusion. For the problem of duplicate detection, Soares et al. [14] proposed a good countermeasure in their CNN network structure, which outperformed other methods in the UAV scene. Tian et al. [15] used the ResNeXt model to learn the mapping from image features to density, and integrated the density map to estimate the total number of pigs, and achieved good verification results on a real breeding scene dataset. Gao et al. [16] improved GSRNet, a crowd counting network, to obtain predicted density maps by sensing multi-scale features and integrating the density maps to achieve pig number estimation, which showed high accuracy and robustness. YAO et al. [17] used AdaBoost and SVM classifiers to extract features and set thresholds to achieve counts and population density estimates of white-backed flycatchers. To address the issue of scale variation and background complexity, Yu et al. [18] developed a Multi-scale Aware Module and a Hybrid Attention Module, resulting in more accurate crowd counting.

2.3 Methods based on object detection

Existing object detection methods mainly include traditional algorithms and deep learning-based methods. The teams represented by He et al., Liu et al., Wang et al., Song et al., Qin et al. [19–24] did a lot of research on cow object detection and feature extraction using traditional object detection algorithms, which promoted the development of cow object detection technology from theory to application. However, compared with deep learning methods, there is still a large room for improvement.

Deep learning-based object detection methods are widely used in livestock object detection and counting, such as One-Stage methods represented by YOLO, SSD, Retina-Net, and Two-Stage methods represented by Faster R-CNN, Grid R-CNN and others [25–30]. YOLO has become one of the most popular algorithms in the field of object detection over the past few years. It can be seen that object detection methods have been widely used in cattle detection and counting.

2.3.1 High-accuracy object detection methods

Weber et al. [31] used 24 architectures of YOLOv4 and YOLOv5 models to evaluate the feasibility of cattle identification and counting in UAV images, and achieved good results. Ahn et al. [32] applied YOLOv4-Tiny to a development board to realize the detection and counting of pigs in pigpens. Ning et al. [33] used the improved YOLOv5 model to achieve accurate counting of pigs in different breeding scenarios. Ding et al. [34] implemented the YOLOv5 model on the Jetson Nano platform by quantizing the network structure and reducing the model’s arithmetic power requirements to achieve real-time accurate detection of lactating piglets. Hu et al. [35] used YOLO as a detection network to locate the head, torso and tail of cows, and Tassinari et al. [36] also used this network successfully realize real-time detection and identification of cows. Weng et al. [37] proposed a channel-pruned YOLOv5 network-based method for cattle face detection, number of parameters and FLOPs were reduced by 86.10% with only a 1.6% reduction in mAP_0.5. In order to capture long-distance dependencies, Wang et al. [38] designed an improved attention module based on global context information and convolution, and combined with porous spatial pyramid pooling to realize multi-scale fusion of high-level semantic information, improving the model’s feature extraction capability for dense cow images. For the purpose of improving recognition accuracy and detection real-time, Xing et al. [39] proposed an improved SSD algorithm SFM-SSD based on shallow feature module and achieved good results. Song et al. [40] achieved accurate corn cob detection using a rich experimental dataset and by incorporating an attentional mechanism to suppress the interference of occlusion.

2.3.2 Multi-scale object detection methods

Due to the comprehensive consideration of accuracy and speed of detection for scale-varying objects, many people have made efforts in related fields. Khan et al. [41] proposed the MFNet and MFNet-FA networks to solve the problems of computational complexity, accuracy limitation and scalability. Three versions with different model sizes are proposed to meet the multi-scale detection requirements. MFNet-M and MFNet-L enable high accuracy bird and UAV detection, while MFNet-S has good real-time performance and is well suited for deployment on limited hardware. In addition, in order to overcome the problem of small size of UAVs and low visibility at night, and to improve the detection accuracy of infrared images at night. They proposed an infrared image object detector called GAANet, which uses an automatic anchor calculation method and replaces the traditional convolution with ghost convolution, and introduces an ultra-small object feature extractor and detector. In the experimental results on birds, UAVs and helicopters, the mAP@50, Recall and Precision are improved by 2.5%, 2.3% and 1.4% respectively [42]. Although the research content is directed at different objects, they focus on the detection problem of multi-scale objects, and we focus on solving the occlusion problem, but ultimately the purpose is to improve the detection accuracy and ensure the real-time algorithm, which is valuable for our work.

3 Materials and methods

3.1 Image acquisition and preprocessing

The experimental videos were collected from the Jiayu Breeding Professional Cooperative located in Horinger County, Hohhot, Inner Mongolia, China. The acquisition device was two 4-megapixel network cameras (DS-2CD3T47EWD-L, Hikvision, China) with a resolution of 2560 pixels×1440 pixels. The cameras were installed in the middle of the outer wall of the 40 m×20 m cowshed at a height of approximately 3.5 m. The surveillance videos were temporarily stored in a Video Cassette Recorder (VCR) and downloaded via remote access over the network, with the data acquisition schematic shown in Fig. 1. The video dataset contains a total of 272 videos during the day and night from July 24 to 26, 2022, with each video lasting approximately 5 minutes. Key frames were extracted from the video using FFmpeg, images that did not contain individual cattle were manually removed, and multi-object images were labelled using LabelImg, saving the labelled file in xml format.

Fig. 1

Schematic diagram of data acquisition.

3.2 Construction of experimental dataset

In order to increase the diversity of samples and improve the adaptability of the algorithm to different environments, the dataset used in the experiment is composed of four parts. After data cleaning in the Baidu EasyData platform and removing images with similarity > 0.85 and sharpness < 5, a total of 2282 images were obtained, and Table 1 gives the detailed composition of the experimental dataset. The first part is the 925 images obtained from the surveillance video extraction frames collected by Jiayu Breeding Professional Cooperative, including 624 images during daytime and 301 images during nighttime. In the second part, 318 images are selected from OpenCows2020, a public dataset of cow backs. The third part is a public dataset downloaded from the Roboflow website, containing 460 images from 4 scenes. The fourth part is the dataset with the class of cow in the VOC dataset, which consists of 579 images. Figure 2 shows the composition of the four parts with a total of 8 scene datasets.

Table 1
Detailed composition of experimental data

Constitution Surveillance OpenCows Dataset downloaded VOC

video from Roboflow 2007

July 24–26, 2022 2020 Scenarios Cow

Day Night 1 2 3 4

Quantity 624 301 318 44 276 67 73 579

Constitution	Surveillance	OpenCows	Dataset downloaded	VOC
Quantity	624	301	318	44	276	67	73	579

Fig. 2

Composition of all scenario datasets.

By analyzing the number of cattle in the dataset images, the number of cattle in each image ranges from 1 to 22, with the majority of the images focusing on 1–15 cattle. In addition, according to the number of cattle in each image, the density of cattle is divided into three levels: sparse, dense and crowded, which are represented by blue, orange and red respectively, as shown in Fig. 3.

Fig. 3

Distribution of the number of cattle in the images of the dataset.

Combined with the camera mounting height analysis, there are a lot of occlusion cases in the experimental dataset, which is one of the main research issues in this paper. The small amount of experimental data and the density of individual cattle lead to poor model robustness, so the experiments used Mosaic, random scaling, translation, rotation, color transformation, saturation transformation, and brightness transformation to improve the model generalization. The dataset was divided into a training set, a validation set and a test set in the ratio of 8 : 1:1, with 1824 images in the training set, 229 images in the validation set and 229 images in the test set.

3.3 SCS-YOLOv5s object detection model

Compared to the latest YOLOv6, YOLOv7 and YOLOv8 models, YOLOv5 has the characteristics of small size and fast inference, while YOLOv5 is divided into ten different structures such as n, s, m, l, x, n6, s6, m6, l6, x6, etc., of which YOLOv5s has both accuracy and speed advantages. In addition, compared with other object detection algorithms, YOLOv5 has been quite mature in the industrial field [43]. Considering the above factors, YOLOv5s is chosen as the base model to ensure real-time performance and improve detection accuracy.

YOLOv5s mainly consists of three parts, including backbone network, neck network and prediction layer. The backbone network is mainly responsible for feature extraction, and the main body is CSPDarknet53. Through the stacking of Conv and C3 modules, features are fused across stages to reduce the number of parameters and improve the feature extraction capability. The neck network combines Feature Pyramid Networks and Path Aggregation Network, making full use of the fine-grained information of the shallow structure and the semantic information of the deep structure to enhance the feature fusion effect. The prediction layer consists of three detection branches, which generate category and location information for large, medium and small objects respectively.

In this paper, the backbone network of the model is first improved to enhance model accuracy, followed by replacing the standard convolution module in the neck and adding an attention mechanism to balance the accuracy and speed requirements for real-time detection of cattle. In the backbone of the network, a CSP cross-stage local network structure is adopted based on the original SPPF network, which combines the original features with the SPPF output to improve feature extraction ability by fusing gradient changes into the entire feature map. A Coordinate Attention mechanism is added after layers 17, 21 and 25 of the model neck, respectively, which makes the network more focused on the cattle object and improves detection performance. The GSConv module is also used instead of the Conv module to balance the accuracy and speed of the model, and Slim Neck is introduced to reduce the model’s floating-point operation in the model, achieving greater computational cost effectiveness. The improved model is called SCS-YOLOv5s, SCS is the initials of SPPFCSPC, CA and Slim Neck respectively, and its structure is shown in Fig. 5.

Fig. 4

Structure diagram of CA mechanism.

Fig. 5

Structure diagram of SCS-YOLOv5s.

3.3.1 SPPFCSPC was used to improve the feature extraction ability

In the original YOLOv5 model, SPPF can avoid image distortion caused by cropping and scaling operations on images regions, while solving the problem of repeated extraction of image features by convolutional neural networks, saving computational costs and improving computational speed. In the improved model of this study, the idea of CSP cross-stage local network is introduced into SPPF to fully integrate the deep and shallow features of the network, maximize the number of branching paths, and improve the learning ability of the network. The structure of SPPFCSPC is shown in the lower left of Fig. 5, and the structure in the blue dashed box in the figure is the original SPPF network.

3.3.2 The CA mechanism was integrated to reduce complex background interference

In conventional object detection tasks, complex backgrounds tend to make the network ignore the detailed features of the target object and extract a large number of background features, which seriously affects the detection effect. To solve the above problems, this study proposes a YOLOv5 network integrating CA mechanism for the complex light and environment of pastures.

CA decomposes channel attention into two 1-D feature encoding processes, and aggregates features along two spatial directions respectively, which not only captures remote dependencies, but also retains precise location information, generating a pair of direction-sensitive and location-sensitive attention maps respectively to enhance the representation of the object of interest. The structure of CA mechanism is shown in Fig. 4.

3.3.3 Slim Neck is introduced to ensure the detection speed

Considering the needs for real-time object detection, the detection network should consider both accuracy and speed. Accordingly, all the standard convolutions in the neck of the network are replaced by GSConv with both accuracy and speed advantages, and its structure is shown in Fig. 6.

Fig. 6

GSConv module structure diagram.

GSConv is the product of combining SC and DSC (Depth-wise Separable Convolution), retaining the feature extraction power of SC, taking the advantage of DSC in reducing the number of parameters. The SC-generated information is permeated into every part of the DSC-generated information using shuffle. In addition, the introduction of Slim Neck enables the detector to be more computationally cost effective, saving computational resources.

3.4 Training parameter setting and evaluation metrics

In this paper, the default image resolution is 640×640, the Batch Size is 32, the AdamW optimizer is used, the momentum is 0.937, training 200 epochs on a single GPU. To describe the effectiveness of model detection, the Precision, mAP (mean Average Precision) and FPS (Frames Per Second) are used as evaluation metrics for the object detection algorithm. The closer the values of Precision and mAP are to 100%, the better the detection effect is. The RMSE (Root Mean Square Error) is used as an evaluation indicator for cattle counting, the closer the RMSE is to zero, the lower the counting error is, and the better the effect is.

Precision evaluates the accuracy of the classifier’s prediction, the number of correctly predicted positive samples as a proportion of the total number of predicted positive samples. mAP measures how well each category is detected on average, there is only one category in this paper, so it is also equivalent to AP, which is the area enclosed by the PR curve. mAP evaluation metrics with 2 IOU thresholds are used in the paper respectively, namely mAP@0.5 (mAP with IoU set to 0.5) and mAP@0.5 : 0.95 (mAP with IoU from 0.5 to 0.95 with step size 0.05). For the above object detection evaluation metrics with four values of TP (positive samples predicted as positive samples), TN (negative samples predicted as negative samples), FP (negative samples predicted as positive samples) and FN (positive samples predicted as negative samples). In addition, the number of object detection classes is defined as c, then the above indicators can be calculated by the following formulas:

Precision = \frac{TP}{TP + FP}

(1)

AP = \int_{0}^{1} P (R)

(2)

mAP = \frac{1}{c} \sum_{j = 1}^{c} {AP}_{j}

(3)

RMSE is commonly used as a measure of the dispersion between predicted and true values, that is, the deviation between the model’s predicted and true cattle numbers. RMSE is calculated as follows:

RMSE = \sqrt{\frac{1}{n} \sum_{i = 1}^{n} ({\hat{y}}_{i} - y_{i})^{2}}

(4)

In Equations (4), ${\hat{y}}_{i}$ represents the number of cattle predicted by the model, and y_i is the actual number of cattle. By finding the RMSE, the overall counting effect of the model can be assessed, providing a reference for the application of the model in practical counting.

4 Results

The workstation configurations used in the experiments are shown in Table 2.

Table 2
Experimental software and hardware configuration parameters

Hardware and software Parameters

configuration

CPU 12th Gen Inter(R) Core(TM)

i9-12900k @3.19 GHz

GPU NVIDIA GeForce RTX 3090Ti

Operating system Windows10

Memory 64G

Programming languages Python3.8

CUDA 10.2

Deep Learning Framework PyTorch1.12.1

Hardware and software	Parameters
CPU	12th Gen Inter(R) Core(TM)
	i9-12900k @3.19 GHz
GPU	NVIDIA GeForce RTX 3090Ti
Operating system	Windows10
Memory	64G
Programming languages	Python3.8
CUDA	10.2
Deep Learning Framework	PyTorch1.12.1

4.1 Optimizer selection

The most widely used optimizers for object detection tasks are SGD and Adam. However, SGD may fall into a local optimum with large fluctuations. Adam parameters are smoother and different adaptive learning rates are calculated for different parameters. AdamW, on the other hand, incorporates weight decay, which adds the regular term directly to the backpropagation process and is more computationally efficient. The validation loss curves when using different optimizers are shown in Fig. 7. Obviously, the model converges faster and the curves are smoother when using AdamW, and the losses are relatively lower after smoothing.

Fig. 7

Comparison of validation loss curves using different optimizers.

4.2 Ablation experiment

In order to verify the effectiveness of each improvement method, ablation experiments were designed in this paper to verify the effects on the experimental results for one improvement point, different combinations of two improvement points and three improvement points, respectively, as shown in Table 3.

Table 3
Comparison of ablation experiments results

Method Precision E-mail: mAP@0.5. RMSE GFLOPS FPS

SPPFCSPC CA Slim Neck (%)↑ (%)↑ ↓ (B)↓ (Hz)↑

—— 93.2 94.5 1.1578 15.8 118

✓ 94.3 95.0 1.1840 20.9 123

✓ 94.7 94.8 1.0878 15.8 109

✓ 94.8 95.3 1.1077 15.5 97

✓ ✓ 94.5 95.2 1.1523 20.9 96

✓ ✓ 93.8 95.0 1.1386 20.3 98

✓ ✓ 95.3 95.0 1.0911 15.2 98

✓ ✓ ✓ 95.5 95.2 1.1292 20.4 88

Method	Precision	E-mail: mAP@0.5.	RMSE	GFLOPS	FPS
	——		93.2	94.5	1.1578	15.8	118
✓			94.3	95.0	1.1840	20.9	123
	✓		94.7	94.8	1.0878	15.8	109
		✓	94.8	95.3	1.1077	15.5	97
✓	✓		94.5	95.2	1.1523	20.9	96
✓		✓	93.8	95.0	1.1386	20.3	98
	✓	✓	95.3	95.0	1.0911	15.2	98
✓	✓	✓	95.5	95.2	1.1292	20.4	88

As can be seen from the table, all three improvement points were able to obtain an improvement in accuracy as well as a reduction in counting error. After replacing SPPF with SPPFCSPC, the Precision and mAP@0.5 of the model are increased by 1.1% and 0.5% respectively, and it also has a small improvement in FPS, which shows that the cross-stage idea can improve the detection speed of model. After introducing CA mechanisms, the Precision is increased by 1.5%, mAP@0.5 is increased by 0.3%, and the RMSE is reduced by 0.07. After using Slim Neck, the Precision and mAP@0.5 of the model are greatly improved, while the model floating point operation is lower than the original network by 0.3B, which is a good balance between accuracy and computational cost.

When the three improvement points are combined in pairs, compared with only one improvement point, Precision and mAP@0.5 gain different degrees of increase, the RMSE also increases slightly, but the FPS decreases slightly with the increase of the number of model parameters. The Precision value of the improved model is increased from 93.2% to 95.5%, the mAP@0.5 value is increased from 94.5% to 95.2%, the RMSE is also reduced by about 0.03, and the FPS can still maintain a high level to meet the requirements of real-time.

The counting effects of SCS-YOLOv5s and YOLOv5s were examined on 229 test set images, and the improved model has fewer missed and false detections compared to the original model, as shown in Fig. 8, where the blue dots represent true values, the red dots represent detected values, and the orange line represents cases where the true value is greater than the detected value, and the opposite is true for the green line. Figure 9 shows the counting error comparison statistics between YOLOv5s and SCS-YOLOv5s, where the horizontal coordinates indicate the distribution of counting errors in the 229 test images, and the vertical coordinates are the corresponding number of images under each error. The pink color represents that there are more images with the same counting error for YOLOv5s than for SCS-YOLOv5s, while the blue color represents the opposite. As can be seen from the figure, SCS-YOLOv5s has the largest number of cases with an error of 0, and the number of cases with an error of 5-6 is zero, and their error is reduced to 4. Although the improved model has a slightly larger number of cases with an error of 2 cattle than YOLOv5s, it zeroes out a large number of cases with an error of 1, and exhibits much better robustness, which improves the model’s detection performance and counting accuracy in the same environment.

Fig. 8

Comparative statistics of YOLOv5s and SCS-YOLOv5s counting errors.

Fig. 9

Comparison of counting errors between YOLOv5s and SCS-YOLOv5s.

4.3 Comparative experiment

4.3.1 Use different attention mechanisms

In the field of image processing, the attention mechanism can essentially achieve focusing on important information with high weights and ignoring irrelevant information with low weights, and important information can be selected under different usage situations by adjusting the weights. Different attention mechanisms are suitable for different application scenarios. This paper finds the most suitable attention mechanism for this model by introducing an attention mechanism to efficiently extract features and adapt to the complex cattle breeding environment through comparative experiments.

Table 4 presents the experimental results of the seven commonly used attention mechanisms, focusing on their accuracy and Root Mean Square Error, the attention mechanisms are added to the CA location as shown in Fig. 5.

Table 4
Add different attention mechanisms

Attention Precision E-mail: mAP@0.5. RMSE GFLOPS FPS

mechanism (%)↑ (%)↑ ↓ (B)↓ (HZ)↑

—— 93.2 94.5 1.1578 15.8 118

SE 93.7 94.6 1.1156 15.8 111

CBAM 94.1 94.6 1.0838 15.8 112

SK 93.7 94.7 1.0919 68.8 94

SimAM 94.4 94.4 1.2345 15.8 105

NAM 93.3 94.8 1.1465 15.8 114

Shuffle 93.9 94.7 1.0259 15.8 110

CA 94.7 94.8 1.0878 15.8 109

Attention	Precision	E-mail: mAP@0.5.	RMSE	GFLOPS	FPS
——	93.2	94.5	1.1578	15.8	118
SE	93.7	94.6	1.1156	15.8	111
CBAM	94.1	94.6	1.0838	15.8	112
SK	93.7	94.7	1.0919	68.8	94
SimAM	94.4	94.4	1.2345	15.8	105
NAM	93.3	94.8	1.1465	15.8	114
Shuffle	93.9	94.7	1.0259	15.8	110
CA	94.7	94.8	1.0878	15.8	109

The experimental results show that CA has the most significant effect on improving model accuracy and reducing counting errors. Precision is improved by 1.5% compared to no attention mechanism used, mAP@0.5 is improved by 0.3%, while RMSE is also reduced by 0.07.

In order to compare the effect of CA more intuitively, the visualization results of the heatmap of the middle layer features of the model are output, as shown in Fig. 10. Four scenarios are randomly selected to verify their effects, each column represents one scenario, the 1st row is the original input image, the 2nd row is the original YOLOv5 network heat map, and the 3rd–9th rows are the heat maps after adding seven attention mechanisms respectively. Compared with other attention mechanisms and no attention mechanism added, all target instances are detected by adding CA mechanism, and the features extracted by CA are more focused on the target itself and contain less background information, which can distinguish the individual cattle well from dense scenes. While other attention mechanisms extract a lot of background information and even fail to extract target instances, and the performance of other attention mechanisms is poor in dense breeding scenes. Therefore, this study chooses an improved approach based on the CA mechanism.

Fig. 10

Comparison Experiment of Attention Mechanisms.

4.3.2 Comparison experiments of attention mechanism positions

In order to verify the effect of the location of the attention mechanism on the performance of the model, the following experiments were designed: the attention mechanisms were added at the four locations shown in Fig. 11, and the P/2 layer of the Backbone, and the P/4, P/8, P/16 and P/32 layers of the Head were output to compare the differences between the feature maps. In Fig. 11, ding172 is added before Head, ding173 is added after Backbone, while ding174 and ding175 are scattered in Neck. The four additions cover almost the entire network structure, which gives a more comprehensive picture of the effect of different additions on the feature extraction process of the network.

Fig. 11

Schematic diagram of the experimental comparison of attention mechanism positions.

Figure 12 shows the visualization results of the feature maps added by the CA mechanism at different locations. The top two panels show the input image and its annotation map respectively, while the bottom four columns from left to right indicate the visualization results of the feature maps at each of the four added locations. Figure 12(a)–(e) show the feature maps of the middle layers of Backbone’s P/2, Head’s P/4, P/8, P/16 and P/32 respectively.

Fig. 12

Results of feature map visualization for CA at different locations.

As can be seen in Fig. 12(a), the complete cattle body features were extracted in all four approaches as more shallow features were extracted at layer P/2 of Backbone, before which no attention mechanism was added. Figure 12(b) and (c), on the other hand, extracted more intermediate layer features and were able to roughly see the outline of the cattle, with better results in mode ding172 ding173 ding175 than mode ding174. Figure 12(d) and (e) extract more semantic features through the stacking of convolutional layers. Compared with mode ding174 ding175, mode ding172 ding173 contain rich semantic features and extract more spatial features, although they are deeper networks, they can roughly see the outline of the cattle. Comparing the overlapping parts of the cattle in Fig. 12(d) and (e), mode ding172 can better distinguish the boundary between the two cattle, while mode ding173 joins the outlines of the two cows as a whole. It is clear that the attention mechanism added to mode ding172 can better solve the occlusion problem, and this improved mode is finally chosen in this paper.

4.3.3 Comparison with other network models

In this paper, the one-stage SSD model, the two-stage Faster RCNN model, YOLOv5s, YOLOv6s, YOLOv7 and YOLOv8s models were selected for comparison experiments with the SCS-YOLOv5s model proposed in this paper, the Precision, mAP@0.5, mAP@0.5 : 0.95, Model size, and detection speed of each model were respectively compared, and the experimental results are shown in Fig. 13. Compared with Faster RCNN and SSD, SCS-YOLOv5s has significant advantages in accuracy and speed. Except for SCS-YOLOv8s, Precision and mAP@0.5 of SCS-YOLOv5s are the highest among all algorithms, and its mAP@0.5 : 0.95 is also second only to the most advanced YOLOv8s and SCS-YOLOv8. YOLOv6s has lower values than SCS-YOLOv5s for both mAP evaluation metrics, and its larger model and lower frame rate limit its usage scenarios. Although mAP@0.5 : 0.95 of YOLOv7 is higher than the improved model, its model size is about 2.5 times that of SCS-YOLOv5s, and the frame rate is also inferior, making it difficult to meet the real-time requirements.

Fig. 13

Comparison of experimental results of different network models.

In addition, compared with YOLOv8s, the two most important accuracy evaluation indexes Precision and mAP@0.5 of SCS-YOLOv5s are higher than those of YOLOv8s, but they are inferior in terms of model size and detection speed. More importantly, applying the method proposed in this paper to YOLOv8s, the improved model is called SCS-YOLOv8s. Compared to YOLOv8s, the mAP@0.5 and mAP@0.5 : 0.95 of SCS-YOLOv8s has improved by 0.9% and 0.8%, respectively, sacrificing about a quarter of the detection speed, but still maintaining good real-time performance. Therefore, the improved model proposed in this paper is competitive and the optimization method used is effective.

As an important tool for comprehensively assessing the performance of object detection algorithms, the P-R curve usually uses AUC (Area Under Curve) to determine whether the model is good or bad, and the larger the area represents the better performance of the detection algorithm. This paper utilizes the experimental results of the above mainstream YOLO series of object detection algorithms to draw P-R curves, which are compared with the SCS-YOLOv5s proposed in this paper, as shown in Fig. 14. As can be seen from the figure, SCS-YOLOv5s has the best performance, followed by YOLOv8s, YOLOv6s and YOLOv7 have similar performance, while YOLOv5s has the worst performance, which is consistent with our expectation and proves once again the effectiveness of the proposed method.

Fig. 14

P-R curves of different YOLO models.

4.3.4 Comparison and verification on public dataset

In order to verify the robustness and generalization ability of the method in this paper, a comparison experiment on the PASCAL VOC 2007 public dataset was designed, as shown in Table 5. The experimental results show that the index data of SCS-YOLOv5s model are greatly improved, among which Precision is increased by 4.7%, mAP@0.5 is increased by 2.3%, and mAP@0.5 : 0.95 is increased by 1.9% . It can be seen that as the experimental data in this paper contains numerous farming environments, the proposed method can adapt to the changes of various scenarios and has certain anti-interference ability, which not only obtains improvement in single-category detection tasks, but also performs better in multi-category detection such as VOC dataset, showing strong robustness.

Table 5
Comparison experiment on PASCAL VOC 2007

Model Precision (%)↑ mAP@0.5 (%)↑ mAP@0.5 : 0.95 (%)↑

YOLOv5s 71.9 72.3 44.2

SCS-YOLOv5s 76.6 (+4.7) 74.6 (+2.3) 46.1 (+1.9)

Model	Precision (%)↑	mAP@0.5 (%)↑	mAP@0.5 : 0.95 (%)↑
YOLOv5s	71.9	72.3	44.2
SCS-YOLOv5s	76.6 (+4.7)	74.6 (+2.3)	46.1 (+1.9)

4.4 Detection results and visual analysis

In order to demonstrate the detection and counting effect of SCS-YOLOv5 algorithm on cattle, images were randomly selected from the test set for testing, and some error detection and counting results were analyzed, as shown in Fig. 15. Among them, Fig. 15(1)–(6) are the detection effect diagram of SSD, Faster RCNN, YOLOv5s, YOLOv6s, YOLOv7 and YOLOv8s respectively, and Fig. 15(7) is the detection effect diagram of the proposed SCS-YOLOv5s.

Fig. 15

Visualization of the test results of different model.

The yellow rectangular boxes in Fig. 15 represent missed cattle detection and the cyan dashed boxes represent false detection targets. The SSD shows more missed detections and fails to detect all cattle even without occlusion, indicating that the SSD performs poorly in object detection tasks with complex backgrounds. While the Faster RCNN, YOLOv5s and YOLOv6s all show varying degrees of false detections, mostly in areas where cattle overlap or features change significantly, and the above algorithms fail to distinguish cattle contours correctly. In addition, the YOLOv5s model has missed detections in (b)–(d). Through analysis, the location of the missed cattle in (b) is obscured by the camera text and is far away, and the ground color and fence color also interfere with the detection process. The missed detection in (c) is mainly caused by the occlusion between cattle, even if 80% of the area is occluded, SCS-YOLOv5s can still detect it. In (d), two cattle are close together and can easily be mistaken for one cattle, which the improved model can also easily detect and achieve accurate counts. Although the mAP@0.5 : 0.9 of YOLOv7 and YOLOv8s is higher than SCS-YOLOv5s, there are also missed detections, which shows that it does not show a strong competitive edge in improving the occlusion. In summary, the improved model SCS-YOLOv5s proposed in this paper achieves better results in improving the effect of occlusion and complex environment on detection results.

5 Conclusion

The method proposed in this paper achieves a large improvement in accuracy, ensures better real-time performance, and at the same time well improves the occlusion problem between cattle in complex breeding scenes with high anti-interference ability. Compared with the current mainstream object detection and counting methods, it shows strong competitiveness, provides ideas and directions for people in related research fields, and further enhances the feasibility of automatic cattle counting.

However, the method proposed in this paper also has shortcomings, for example, the model size of SCS-YOLOv5s nearly doubled, while mAP only gained a small amount of enhancement, which may be attributed to the relatively small number of farming scenarios and the small amount of data. What’s more, in the application scenario of cattle counting in the UAV field of view, which contains a large number of small object cattle individuals. Due to the limitations of the sliding window, the situation of large changes in the scale of the cattle has not been considered for the time being.

Relevant researchers can collect data from more scenes in future studies and obtain better quality datasets through image super-resolution reconstruction and more image enhancement methods to further improve the model accuracy. In addition, model pruning and knowledge distillation can greatly reduce the number of model parameters, which is a good choice for future model deployment and lightweight applications. In the future research, we will combine the multi-object-tracking algorithm and the Re-Identification framework to further optimize the model performance and improve the accuracy and application scope of cattle counting. Of course, cattle detection and counting based on UAV scenarios is another focus of the work, and automatic counting of sheep and pigs will also be involved.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Footnotes

Acknowledgments

This research was funded by the National Natural Science Foundation of China under Grant 61966026. The authors appreciate the funding organization for their financial supports.

References

Bianchi

M.C.

Bava

Sandrucci

, et al. Diffusion of precision livestock farming technologies in dairy cattle farms[J], Animal16(11) (2022), 100650.

Aquilani

Confessore

Bozzi

, et al. Precision Livestock Farming technologies in pasture-based livestock systems[J], Animal16(1) (2022), 100429.

Awad

A.I.

Hassaballah

, Bag-of-visual-words for cattle identification from muzzle print images[J], Applied Sciences9(22) (2019), 4914.

Hossain

M.E.

Kabir

M.A.

Zheng

, et al. A systematic review of machine learning techniques for cattle identification: Datasets, methods and future directions[J], Artificial Intelligence in Agriculture6 (2022), 138–155.

Simanungkalit

Barwick

Cowley

, et al. Use of an ear-tag accelerometer and a radio-frequency identification (RFID) system for monitoring the licking behaviour in grazing cattle[J], Applied Animal Behaviour Science244 (2021), 105491.

Mahmud

M.S.

Zahid

Das

A.K.

, et al. A systematic literature review on deep learning applications for precision cattle farming[J], Computers and Electronics in Agriculture187 (2021), 106313.

Wang

Falzon

, et al. Automated cattle counting using Mask R-CNN in quadcopter vision system[J], Computers and Electronics in Agriculture171 (2020), 105300.

Hu Yunge, Cang Yan Qiao Yulong, , Design of intelligent pig counting system based on improved instance segmentation algorithm[J], Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE)36(19) (2020), 177–183.

Wang Rong, Gao Ronghua, Li Qifeng, , et al. High-density Pig Herd Counting Method Combined with Feature Pyramid and Deformable Convolution[J], Transactions of the Chinese Society for Agricultural Machinery53(10) (2022), 252–260.

10.

Wei

W.U.

Yang

Rui

L.I.

, et al. Detection and enumeration of wheat grains based on a deep learning method under various scenarios and scales[J], Journal of Integrative Agriculture19(8) (2020), 1998–2008.

11.

Singh

Gupta

Shukla

, Activity detection and counting people using Mask-RCNN with bidirectional ConvLSTM[J], Journal of Intelligent & Fuzzy Systems43(5) (2022), 6505–6520.

12.

Jianmin Zhao, Xuedong Li Baoshan Li, , Algorithm of sheep dense counting based on unmanned aerial vehicle images[J], Laser & Optoelectronics Progress58(22) (2021), 2210013.

13.

Liu Guanghui, Wang Qinmeng, Chen Xuanrun, , et al. A multivariate information aggregation method for crowd density estimation and counting[J], Optics and Precision Engineering30(10) (2022), 1228–1239.

14.

Soares

V.H.A.

Ponti

M.A.

Gonçalves,

R.A.

, et al. Cattlecounting in the wild with geolocated aerial images in large pastureareas[J], Computers and Electronics in Agriculture189(2021), 106354.

15.

Tian

Guo

Chen

, et al. Automated pig counting using deep learning[J], Computers and Electronics in Agriculture163 (2019), 104840.

16.

Gao Yun, Li Jing, Yu Mei, , et al. High-density pig counting net based on multi-scale aware[J], Transactions of the Chinese Society for Agricultural Machinery52(9) (2021), 172–178.

17.

Qing Yao, Xian,

Liu,

, et al. Automated counting of rice planthoppers in paddy fields based on image processing[J],–, Journal of Integrative Agriculture13(8) (1736), 1745.

18.

Qian

, et al. MHANet: Multi-scale hybrid attention network for crowd counting[J], Journal of Intelligent & Fuzzy Systems(Preprint):1–11.

19.

He Dongjian, Meng Fanchang, Zhao Kaixuan, , et al. Recognition of calf basic behaviors based on video analysis[J], Transactions of the Chinese Society for Agricultural Machinery47(9) (2016), 294–300.

20.

Liu Dong, Zhao Kaixuan He Dongjian, , Real-time target detection for moving cows based on Gaussian mixture model[J], Transactions of the Chinese Society for Agricultural Machinery47(5) (2016), 288–294.

21.

Wang Shaohua, He Dongjian Liu Dong, , Automatic recognition method of dairy cow Estrus behavior based on machine vision[J], Transactions of the Chinese Society for Agricultural Machinery51(4) (2020), 241–249.

22.

Song Huaibo, Li Tong, Jiang Bo, , et al. Automatic detection of multi-target ruminate cow mouths based on Horn-Schunck optical flow algorithm[J], Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE)34(10) (2018), 163–171.

23.

Song Huaibo, Yin Xuqiang, Wu Dihua, , et al. Detection of moving cows based on adaptive kernel density estimation algorithm[J], Transactions of the Chinese Society for Agricultural Machinery50(5) (2019), 196–204.

24.

Qin Lifeng, Zhang Xiaoqian, Dong Mingxing, , et al. Target extraction of moving cows based on multi-feature fusion correlation filtering[J], Transactions of the Chinese Society for Agricultural Machinery52(11) (2021), 244–252.

25.

Redmon

Divvala

Girshick

, et al. You only look once: Unified, real-time object detection[C],:, Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), 779–788.

26.

Liu

Anguelov

Erhan

, et al. Ssd: Single shot multibox detector[C],Springer, Cham,:, European conference on computer vision, (2016), 21–37.

27.

Lin

T.Y.

Goyal

Girshick

, et al. Focal loss for dense object detection[C],:, Proceedings of the IEEE international conference on computer vision, (2017), 2980–2988.

28.

Ren

Girshick

, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[J], Advances in Neural Information Processing Systems (2015), 28.

29.

Yue

, et al. Grid r-cnn[C],:, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2019), 7363–7372.

30.

Zaidi

S.S.A.

Ansari

M.S.

Aslam

, et al. A survey of modern deep learning based object detection models[J],:, Digital Signal Processing (2022), 103514.

31.

de Lima Weber,

, de Moraes Weber,

V.A.

de Moraes,

P.H.

, et al. Counting cattle in UAV images using convolutional neural network[J], Remote Sensing Applications: Society and Environment29 (2023), 100900.

32.

Ahn

Son

Kim

, et al. EnsemblePigDet: Ensemble Deep Learning for accurate pig detection[J], Applied Sciences11(12) (2021), 5577.

33.

Ning Yuanlin, Yang Ying, Li Zhenbo, , et al. Detecting and counting pig number using improved YOLOv5 in complex scenes[J], Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE)38(17) (2022), 168–175.

34.

Ding Qi’an, Liu Longshen, Chen Jia, , et al. Object detection of suckling piglets based on Jetson Nano and YOLO v5[J], Transactions of the Chinese Society for Agricultural Machinery53(3) (2022), 277–284.

35.

Dai

Shen

, et al. Cow identification based on fusion of deep parts features[J], Biosystems Engineering192 (2020), 245–256.

36.

Tassinari

Bovo

Benni

, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn[J], Computers and Electronics in Agriculture182 (2021), 106030.

37.

Weng

Liu

Zheng

, Cattle face detection method based on channel pruning YOLOv5 network and mobile deployment[J],&(Preprint):, Fuzzy Systems1–18.

38.

Wang

Bai

Gao

, et al. Oestrus detection in dairy cows by using atrous spatial pyramid and attention mechanism[J], Biosystems Engineering223 (2022), 259–276.

39.

Xing Yongxin, Sun Youdong Wang Tianyi, Individual recognition of dairy cow based on improved SSD algorithm[J], Computer Engineering and Applications58(02) (2022), 208–214.

40.

Song

Zhang

, et al. Detection of maize tassels for UAV remote sensing image with an improved YOLOX model[J], Journal of Integrative Agriculture22(6) (2023), 1671–1683.

41.

Khan

M.U.

Dil

Alam

M.Z.

, et al. Safespace mfnet: Precise and efficient multifeature drone detection network[J], IEEE Transactions on Vehicular Technology2023.

42.

Khan

M.U.

Misbah

Kaleem

, et al. GAANet: Ghost auto anchor network for detecting varying size drones in dark[J], arxiv preprint arxiv:2305.03425, 2023.

43.

Guo

Wang

Yang

, et al. Msft-yolo: Improved yolov5 based on transformer for detecting defects of steel surface[J], Sensors22(9) (2022), 3467.

Constitution	Surveillance		OpenCows	Dataset downloaded				VOC
	video			from Roboflow				2007
	July 24–26, 2022		2020	Scenarios				Cow
	Day	Night		1	2	3	4
Quantity	624	301	318	44	276	67	73	579

Hardware and software	Parameters
configuration
CPU	12th Gen Inter(R) Core(TM)
	i9-12900k @3.19 GHz
GPU	NVIDIA GeForce RTX 3090Ti
Operating system	Windows10
Memory	64G
Programming languages	Python3.8
CUDA	10.2
Deep Learning Framework	PyTorch1.12.1

Method			Precision	E-mail: mAP@0.5.	RMSE	GFLOPS	FPS
SPPFCSPC	CA	Slim Neck	(%)↑	(%)↑	↓	(B)↓	(Hz)↑
	——		93.2	94.5	1.1578	15.8	118
✓			94.3	95.0	1.1840	20.9	123
	✓		94.7	94.8	1.0878	15.8	109
		✓	94.8	95.3	1.1077	15.5	97
✓	✓		94.5	95.2	1.1523	20.9	96
✓		✓	93.8	95.0	1.1386	20.3	98
	✓	✓	95.3	95.0	1.0911	15.2	98
✓	✓	✓	95.5	95.2	1.1292	20.4	88

Attention	Precision	E-mail: mAP@0.5.	RMSE	GFLOPS	FPS
mechanism	(%)↑	(%)↑	↓	(B)↓	(HZ)↑
——	93.2	94.5	1.1578	15.8	118
SE	93.7	94.6	1.1156	15.8	111
CBAM	94.1	94.6	1.0838	15.8	112
SK	93.7	94.7	1.0919	68.8	94
SimAM	94.4	94.4	1.2345	15.8	105
NAM	93.3	94.8	1.1465	15.8	114
Shuffle	93.9	94.7	1.0259	15.8	110
CA	94.7	94.8	1.0878	15.8	109

SCS-YOLOv5s: A cattle detection and counting method for complex breeding environment

Abstract

Keywords

1 Introduction

2.1 Methods based on instance segmentation

2.2 Methods based on density estimation

2.3 Methods based on object detection

2.3.1 High-accuracy object detection methods

2.3.2 Multi-scale object detection methods

3 Materials and methods

3.1 Image acquisition and preprocessing

Table 1 Detailed composition of experimental data Constitution Surveillance OpenCows Dataset downloaded VOC video from Roboflow 2007 July 24–26, 2022 2020 Scenarios Cow Day Night 1 2 3 4 Quantity 624 301 318 44 276 67 73 579

3.3.2 The CA mechanism was integrated to reduce complex background interference

3.3.3 Slim Neck is introduced to ensure the detection speed

4.3.1 Use different attention mechanisms

Table 5 Comparison experiment on PASCAL VOC 2007 Model Precision (%)↑ mAP@0.5 (%)↑ mAP@0.5 : 0.95 (%)↑ YOLOv5s 71.9 72.3 44.2 SCS-YOLOv5s 76.6 (+4.7) 74.6 (+2.3) 46.1 (+1.9)

Declaration of competing interest

Footnotes

Acknowledgments

References

Table 1
Detailed composition of experimental data

Constitution Surveillance OpenCows Dataset downloaded VOC

video from Roboflow 2007

July 24–26, 2022 2020 Scenarios Cow

Day Night 1 2 3 4

Quantity 624 301 318 44 276 67 73 579

Table 5
Comparison experiment on PASCAL VOC 2007

Model Precision (%)↑ mAP@0.5 (%)↑ mAP@0.5 : 0.95 (%)↑

YOLOv5s 71.9 72.3 44.2

SCS-YOLOv5s 76.6 (+4.7) 74.6 (+2.3) 46.1 (+1.9)