Abstract
The edge features of peanut grain image are not considered in peanut grain integrity detection, and there are many noises, resulting in low accuracy of peanut grain integrity detection and poor effect of image edge noise removal. Therefore, this paper designs a peanut seed integrity detection method based on deep learning convolution neural network. The edge contour of peanut grain image is closed, the edge contour curve characteristics of peanut grain image are extracted, the noise area of peanut grain edge image is determined according to the filter window of Gaussian kernel function, and the wavelet descriptor in contour description operator is used to reduce the edge noise of peanut grain image. Input the image into the convolution neural network, update the weight by gradient descent method, construct the peanut seed integrity detection model, output the peanut seed integrity detection results, and realize the peanut seed integrity detection. The experimental results show that the proposed detection method can improve the accuracy of peanut grain detection and has certain feasibility.
Keywords
Introduction
Peanut is known as longevity fruit, and millennium seed. It has high nutritional value. It is an important edible oil resource and a high-quality plant protein resource. It is widely cultivated all over the world [1]. Many large peanut processing and consumption countries such as the EU have strengthened the quality control of imported peanuts [2]. China stipulates in the quality standards and specifications of exported high-quality peanut varieties that the purity of exported peanut kernel should reach more than 95%, and that of different varieties should not exceed 5% [3].
With the continuous expansion of endogenous production scale, whether peanut export is smooth is an important factor affecting domestic peanut production. After China’s accession to the WTO, with the intensification of competition in the international peanut export market, the requirements for peanut quality are becoming more and more strict. It has the function of single variety or single item detection [4], which is far from meeting the actual production needs. How to strengthen the standardized production of peanut, improve the quality of peanut and promote the transformation of peanut production from traditional quantity type to modern quality benefit type is an important issue. Therefore, the detection of peanut grain integrity is the top priority of peanut seeds detection. In recent years, the vigorous development of deep learning has promoted the progress of target detection technology. Convolutional neural network is an algorithm that has attracted extensive attention [5]. It has good robustness in target detection. When the light intensity changes or the background information is complex, it still has the advantages of high detection accuracy and fast detection speed. At the same time, it also has good detection effect in detecting small targets. However, there are many deficiencies in peanut grain image detection [6]. Therefore, researchers continue to study and obtain better methods.
Zhang et al. [7] proposed a method for identifying imperfect grains to solve the problems that the images collected during traditional image acquisition cannot be transmitted to the computer in real time and marked and processed in time, and to realize the application of image processing technology in wheat imperfect grain recognition. The method is based on OpenCV computer vision library and Python language, and combines the vgg16 neural network model of Keras framework to recognize and test the wheat grain. This method improves the accuracy of grain integrity detection, but the detection language is complex and has some shortcomings. Zhu et al. [8] designed a grain integrity image detection system based on CNN. The image database and morphological feature database of single grain wheat are established. Four typical convolutional neural networks were used to establish wheat grain integrity recognition model. However, the development of the system is relatively simple, and the detection effect can not be used. Zhu et al. [9] proposed an excellent seed screening method based on image recognition and convolution neural network. The data set of grain quality is established, and the convolution neural network is designed to extract the characteristics of grain image. To improve the classification accuracy and real-time performance, the convolution neural network is optimized from the aspects of designing and selecting the convolution neural network structure, reducing over fitting, accelerating the training convergence speed and enhancing the robustness of the network. This method improves the speed of grain detection comparing with traditional machine learning classification methods, but there is a certain deviation in the output results, which needs to be corrected.
In view of the shortcomings of the above methods, a peanut seed integrity detection method based on deep learning convolution neural network is designed in this paper. By collecting the peanut seed image, the image edge is segmented to remove the noise in the edge image, and the deep learning convolution neural network is introduced to construct the integrity detection model. The main technical route of this paper is as follows:
Design a CCD camera, camera link interface unit and FPGA to read image data such as CCD camera data, and collect peanut grain image. With the help of MRF algorithm, the probability of joint value in random field is calculated, the pixel points are matched with the edge data points of peanut grain image after segmentation, and the edge data points of peanut grain image are divided into three forms to complete the edge segmentation of peanut grain image. Close the edge contour of peanut grain image, extract the edge contour curve characteristics of peanut grain image, determine the noise area of peanut grain edge image according to the filtering window of Gaussian kernel function, and complete the edge noise reduction of peanut grain image by using the wavelet descriptor in the contour description operator; Input the peanut grain image into the main feature extraction network, determine the convolution layer of the deep learning neural network, extract the original feature map, reweighted peanut grain feature map, input the image into the convolution neural network, update the weight by gradient descent method, construct the peanut grain integrity detection model, output the peanut grain integrity detection results, and realize the peanut grain integrity detection. Experimental analysis. In this section, the performance of the integrity detection method proposed by the study will be tested, and the similar algorithms on the market will be used to compare with the algorithm proposed in the study during the testing process. Conclusion. This part is to sort out the research as a whole and summarize the research content and results.
Peanut seed image acquisition
In the integrity detection of peanut grain image, we need to collect the peanut grain image first, and take the collected peanut grain image as the research object of this paper to realize the design of detection method. Therefore, this paper first collects the image of peanut seeds. In the method proposed in the research, the main process of peanut seed image acquisition is as follows. First, the CCD camera is used to acquire and output the stored data, and the data is stored in the external DDR for the algorithm to use. It mainly includes the CCD camera, camera link interface unit, FPGA reading CCD camera data and other image data [10].
CCD camera
The width of conveyor belt commonly used in industrial and agricultural screening site is 300 mm and the running speed is 1.5 m/s. The minimum area that can be identified is 0.15
The camera Link interface unit design
The camera outputs 28 bit image parallel data, of which 24 bits are RGB true color images and 4 bits are control signals. The basic configuration mode of camera link is selected in this design, and its working principle is shown in Fig. 1.
Schematic diagram of working principle of camera link interface.
In this mode, the camera link interface contains a transmitter and a receiver. The transmitter in the camera converts 28 bit parallel data into 4 channels of low-voltage differential signals and 1-bit clock signal into 1 channel of low-voltage differential signals. Ds90cr286 chip is selected as the receiver to convert the above five low-voltage differential signals into 28 bit data signal and 1-bit clock signal [12]. The circuit schematic diagram of the chip is shown in Fig. 2.
Schematic diagram of ds90cr286 chip interface circuit.
In addition to the 28 bit image parallel output signal, there are four camera input control signals (CC1-CC4) and serial communication signals in the I/O signal of CCD camera. Dslv047a chip is used to realize the conversion of 4-channel control signals, and ds90lv019 chip is used to realize the conversion of serial communication signals.
A dual port RAM is established inside the FPGA to cache one line of data and notify the DSP to receive it. Use Xilinx’s IP core to build dual port RAM. When calling the generated dual port RAM in the program file, it is necessary to operate the IP core by example. The camera data output is 24 bits, but the dual port RAM input defined here is 32 bits. Therefore, set the lower 24 bits as RGB values and the upper 8 bits as invalid bits. Similarly, in the process of dual port RAM reading, all addresses with 4n-l (n is a positive integer) are required to be invalid bits, and these addresses need to be skipped through programming. The input and output timing of dual port RAM in FPGA is shown in Fig. 3.
Schematic diagram of input and output timing of peanut grain image acquisition dual port RAM.
It can be seen from Fig. 3 that when the row synchronization is high, the dual port RAM starts reading and writing. In the process of writing dual port RAM, the rising edge of the clock writes data. At the same time, the number of addresses is added by 1, and the data width is 24 bits. In the process of dual port RAM reading, when the RD pin is the falling edge, the data corresponding to the address is output, and the output data width is 8 bits. The data of the low address is output first, and the invalid bit address will be skipped at the same time. The data comparison shows that the dual port RAM can work reliably.
In the peanut seed image acquisition, CCD camera, camera link interface unit and FPGA are designed to read the image data of CCD camera to collect the peanut seed image.
In the preprocessing process of peanut kernel image, the first part is to segment the peanut edge image. This step uses the MRF algorithm, which can better reflect the spatial correlation of image edge data. In order to optimize the effect of the final detection, it is also necessary to perform noise reduction processing on the noise at the edge of the image. When the image edge noise is too large, it will have a negligible negative impact on the integrity detection results.
Edge segmentation of peanut grain image
In the integrity detection of peanut grain image, its edge feature is the typical feature of the target image. Therefore, the edge image of peanut needs to be segmented first. MRF algorithm is used to segment the edge of peanut grain image to reflect the spatial correlation of peanut grain image edge data [13].
Assuming that
where
Based on the random value probability value, the peanut grain image edge data to be divided, assuming the pixel set is
In the edge segmentation of peanut grain image, the probability of joint value in random field is calculated with the help of MRF algorithm, so that the pixel points are matched with the edge data points of peanut grain image after segmentation, and the edge data points of peanut grain image are divided into three forms to complete the edge segmentation of peanut grain image.
According to the edge of the segmented peanut grain image, the noise in the image edge is removed to improve the integrity detection of peanut grain. In this paper, the edge noise of peanut grain image is removed by using contour feature recognition method. This method mainly obtains its characteristics according to the edge curve of the research object, and then carries on the corresponding target processing. In the edge denoising of peanut grain image, the contour of peanut grain image is closed, and any point is selected, and the length of a point in this area is:
In the formula,
In the Fourier descriptor, when the coefficient term meets the one-dimensional signal of the research object, the research purpose of peanut grain noise reduction can be completed. The edge contour curve of peanut grain image expressed by this method reflects the directional characteristics of grain edge image as follows:
In the formula,
Based on the edge features of peanut grain image determined above, the gradient value and gradient direction of each data point in peanut grain edge image are determined according to the filtering window of Gaussian kernel function [14], and the area with noise in the edge of peanut grain image is determined, that is:
In the formula,
Finally, the wavelet descriptor in the profile description operator is used to complete the edge noise reduction of the peanut kernel image, namely:
In the formula,
In the edge denoising of peanut grain image, the edge contour of peanut grain image is closed, the edge contour curve characteristics of peanut grain image are extracted, the noise area of peanut grain edge image is determined according to the filter window of Gaussian kernel function, and the wavelet descriptor in contour description operator is used to complete the edge denoising of peanut grain image.
According to the above determined peanut grain edge image, it is used as the basic data of this paper to complete the detection of peanut grain integrity. In the final detection of this paper, deep learning and convolutional neural network are combined to improve the effect of peanut grain integrity detection.
Deep learning convolutional neural network
Due to the ability to select the region of interest, the attention mechanism applying in machine learning image processing can improve its processing ability. Researchers focus their attention on the recognizable area of the image dynamically instead of directly processing all the information of the image. The deep neural network imitates the human visual signal processing mechanism when recognizing fine-grained images: quickly scan the global image, find the target area of interest, and then suppress other useless information. Earlier attempts to use visual attention mechanism for fine-grained image recognition have achieved good results.
Convolutional neural network (CNN) is a feedforward neural network, which is good at dealing with machine learning problems related to images. At present, convolutional neural network has even surpassed human beings in the task of image recognition [15]. CNN is an improved version of the traditional neural network. Its advantage is to share the convolution kernel, do not need to manually select the features, train the weight to get the features, and the classification effect is good. Its disadvantage is the large amount of data processing and the high requirements for configuration. Its special structure of local weight sharing has unique advantages in image processing. Its layout is closer to the actual biological neural network, and weight sharing reduces the complexity of the network and effectively avoids the complexity of data reconstruction during feature extraction and classification. Therefore, the combination of deep learning convolutional neural network improves the effectiveness of image detection. Its basic principle is shown in Fig. 4.
Basic principle diagram of deep learning convolutional neural network.
To summarize Fig. 4, the network structure and data characteristics of the convolutional neural network lead to its excellent training and classification effect, and the network complexity is not high. Adding an attention mechanism to it can theoretically further optimize the detection effect.
Based on the deep learning convolutional neural network, after the peanut grain image is input to the backbone feature extraction network, the convolutional layer of the deep learning neural network is the output feature graph of conv1, conv2 and conv3, respectively, and each feature graph can be represented as
At this point, the peanut grain signature map changes into a one-dimensional feature descriptor.
The attention vector m in deep learning was calculated based on the one-dimensional feature descriptors of the peanut grain image as determined above, thus obtaining:
Variables
Based on the determined deep learning peanut grain image feature map, the customs clearance convolution neural network further determines the integrity of peanut grain image. The training process of convolutional neural network is similar to that of traditional neural network. Its training is a parameter adjustment process, which needs to go through forward propagation stage and back propagation stage.
Convolution layer forward propagation process: the last layer of the output of the input is the current layer, through the activation function layer by layer, set the
Back propagation process of convolution layer: it is essentially a method of calculating parameter derivative, which can be considered as the application of chain rule in matrix derivation. It is usually used to train multi-layer perceptron, that is:
where
Based on the above output peanut grain characteristic map, in order to avoid large detection error, the neuron error is processed, that is:
At this time, the error equation of peanut grain image completion output layer is:
Then calculate the parameters and partial derivatives in the convolution neural network to obtain:
where
Update the weight by gradient descent method, and set all peanut grain image features as all zero matrix, that is:
The updated parameter weights are:
where
For each type of target, a linear ridge regressor is used for refinement, and the input is the output of Alex net pool. The regression considers that the candidate region and cross truth are linear, and the following results are obtained:
Among them,
Finally, the constructed peanut grain integrity detection model is:
Among them,
Input the peanut seed image into the trunk feature extraction network, determine the convolution layer of the deep learning neural network, extract the original feature map, reweighted the peanut seed feature map, input the image into the convolution neural network, update the weight by gradient descent method, construct the peanut seed integrity detection model, output the peanut seed integrity detection results, and realize the peanut seed integrity detection.
Experimental scheme design
In order to verify the feasibility of the proposed detection method, an experimental analysis is designed. In the experiment, peanut seeds from an agricultural food factory were selected as the research object, and their high-quality peanut seeds were selected to effectively detect their integrity. Before the experiment, peanut was threshed to obtain relevant parameters such as variety, axle weight, axle volume, grain length, grain width, grain number, grain weight, grain volume, water content and so on. The operation of the experimental environment is shown in Fig. 5.
Specific experimental operation environment.
The image of peanut seeds can reflect the internal physiological characteristics to a great extent, and help researchers to judge the integrity of seeds without damage. In the experiment, 100 grains were manually selected and placed on the image acquisition device. Through the grain phenotype measurement software, the multi grain image was cut and the single grain image was extracted. The integrated image data was used for variety classification training. Peanut grain phenotype measurement software meets the needs of quickly obtaining a large number of data information such as grain length, grain width and grain color. The collected picture data are segmented according to the characteristics of grain shape, color and texture. Due to the randomness of grain placement, there will be adhesion, crisscrossing, and shielding of grains. Some experimental peanut seeds were processed by software, and the data obtained are shown in Fig. 6.
Image of peanut grain of experimental sample.
According to the segmented peanut grain image in Fig. 6, multiple peanut grain images are randomly selected as the sample images for the study, and the selected images are effectively trained. The experimental sample training peanut grain image is shown in Fig. 7.
Peanut seed experimental training data set.
Test accuracy of peanut grain integrity by different methods.
Error analysis of peanut grain edge detection.
Based on the above determined experimental environment and the experimental peanut seed training set, the experimental analysis was carried out. In order to highlight the feasibility of this method, the experiment is carried out mainly by comparing this method, Literature [7] method and Literature [8] method. Literature [7] method is an imperfect grains identifying method using vgg16 neural network and Literature [8] method is a rain integrity image detection system based on CNN. In the experiment, the accuracy of peanut grain integrity detection, the error of peanut grain edge detection and the accuracy of peanut grain image noise reduction are taken as experimental indicators for experimental analysis.
In the experiment, the methods of this paper, Literature [7] and Literature [8] are analyzed to analyze the accuracy of peanut grain integrity detection of the trained experimental sample image set. The results are shown in Fig. 8.
By analyzing the experimental results in Fig. 8, it can be seen that there are some differences in the accuracy of peanut grain integrity detection of the trained experimental sample image set by the methods in this paper, Literature [7] and Literature [8]. Among them, from the trend of detection accuracy curve, it can be seen that the detection accuracy of this method is always higher than 90%, and always higher than the other two methods. Therefore, it can be seen that the detection effect of this method is better, the integrity of peanut seeds can be detected, and the results are more reliable.
In the experiment, the methods of this paper, Literature [7] and Literature [8] are analyzed to analyze the error of peanut grain edge detection of the trained experimental sample image set. The results are shown in Fig. 9.
By analyzing the experimental results in Fig. 9, it can be seen that there are some differences in the detection errors of the methods in this paper, Literature [7] and Literature [8] for the edge detection of the peanut grain image trained in the experiment. Among them, this method can completely detect the edge of peanut grain image, while there are certain errors in the edge image of peanut grain detected by the methods of Literature [7] and Literature [8]. Therefore, it can be seen that the integrity of peanut grain detected by this method is more reliable.
In the experiment, the methods of this paper, Literature [7] and Literature [8] are further analyzed to compare the accuracy of noise reduction of peanut grain image in the trained experimental sample image set. The results are shown in Table 1.
Accuracy analysis of peanut grain image noise reduction (%)
Accuracy analysis of peanut grain image noise reduction (%)
By analyzing the experimental data in Table 1, it can be seen that the methods in this paper, Literature [7] and Literature [8] have high accuracy in noise reduction of peanut grain images of the trained experimental sample image set. But in contrast, the noise reduction accuracy of this method is higher than that of the other two methods. This is because this method closes the edge contour of peanut grain image, extracts the edge contour curve characteristics of peanut grain image, determines the noise area of peanut grain edge image according to the filter window of Gaussian kernel function, and uses the wavelet descriptor in the contour description operator to complete the edge noise reduction of peanut grain image, which further improves the noise reduction performance of the proposed method and provides a basis for peanut grain integrity detection.
This paper designs a peanut kernel integrity detection method based on deep learning convolution neural network. Camera link interface unit and FPGA are designed to read the image data of CCD camera and collect the image of peanut seeds. MRF algorithm is used to process peanuts images. The wavelet descriptor in contour description operator is used to reduce the edge noise of peanut grain image. According to the experimental results, the accuracy of the peanut kernel integrity detection algorithm proposed in the study is always higher than 90% in the actual test, while the detection accuracy of the vgg16 neural network algorithm and the CNN image integrity detection system for comparison are distributed at 70% to 85%. The accuracy of the proposed algorithm is more than 10% higher than that of the comparison model in most situations. In addition, the noise reduction ability of the proposed algorithm can reach 97% at the highest and 95% at the lowest, and the noise reduction ability in each case is higher than that of the comparison algorithm. The proposed detection method can improve the accuracy of peanut grain detection and has certain feasibility.
