Abstract
In order to solve the problem of real-time and accurate recognition of coal gangue in the intelligent separation system of coal gangue, an online visual recognition algorithm of coal gangue based on BLOB analysis and machine learning is proposed. It filters the easily recognized gangue or coal by triple filter model with small calculation, which only discriminating the suspected gangue image extremely difficult to recognize. The remaining small amount of suspected coal gangue image is distinguished by calculating the local characteristic parameters and inputting them into the SVM classification model. The algorithm has been applied to the intelligent sorting system of coal gangue and verified by experiments. The test results show that it improves the recognition rate of coal gangue and ensures the real-time detection.
Introduction
Coal gangue separation is an important means for coal production enterprises to ensure coal quality. At present, coal mines in China basically remove coal gangue manually, which is not only labor-intensive, but also inefficient and harmful to health. Therefore, all coal mining enterprises are anxious to develop an automatic sorting robot to replace manual sorting. On the premise of compatibility with the existing production equipment of coal mine, the intelligent separation system of coal gangue will be limited by environmental and field factors, so the industrial application equipment has not been applied to the separation of medium and large coal gangue (particle size more than 200 mm). The main technical difficulty is the identification of coal gangue. At present, the mainstream technology to identify large and medium-sized coal gangue is image recognition technology, which includes two categories: one is the selection of characteristic parameters, that is, analyzing the physical characteristics of coal gangue, selecting and designing different parameters for image evaluation. The specific parameters can be one or more, and the segmentation threshold is used for classification judgment after obtaining the parameter evaluation value [1, 2, 3, 4, 5]; another is to select the existing evaluation parameters, use machine learning method, get the correct classification model through learning and training, and then use the classification model to correctly identify coal gangue [6, 7, 8, 9, 10, 11, 12].
According to the description of papers 1, the identification method of single parameters, such as gray mean and gray variance, is adopted in coal gangue identification, and its identification rate is less than 70%; however, using machine learning methods such as neural network and support vector machine, the recognition rate can reach more than 90%. However, no matter which machine learning method is adopted, the higher the accuracy, the more kinds of image feature parameters need to be extracted, so it takes much more time than a single parameter. Taking a coal gangue gray image with 1600
After the acquired image which includes coal or coal gangue is evaluated by one or more parameters, the coal gangue recognition results could be gotten by comparing the evaluation value and threshold value. This method is always used on condition that there has a clear distinction between the background and the target. By extracting the edge of the contour and determining the object field, this method could achieve good results. However, in the process of coal transportation, the actual evaluation effect is often not good due to the pollution of belt and the interference of external factors such as dust and vibration. The combination of multi-parameter evaluation and machine learning can be identified by the classification model. However, this recognition will bring a large amount of calculation for multi-feature parameter evaluation, so that real-time analysis is difficult to meet the requirements of online recognition. In addition, the image samples used in machine learning are images with a conveyor belt as the background. Due to belt pollution, the parameter evaluation cannot truly reflect the real characteristics of coal and gangue, making the classification model invalid. Therefore, it is of great significance to study the online recognition method of coal gangue image and realize the online intelligent sorting of coal gangue.
Based on the research of various algorithms, an on-line image recognition algorithm of coal gangue is proposed in this paper. For the suspected coal gangue which is difficult to identify, firstly the distribution boundary region is accurately defined, and then the characteristic parameters are estimated by the pixels of the image in this area. After that, the classification model is used to identify coal and gangue, which can not only locate the target, but also reduce the computing time. For the obvious coal gangue which is easy to be identified, it can be identified by single parameter evaluation.
Basic principle of BLOB analysis method
Usually, the image can be divided into background and object. Most of the target objects have their own basic characteristics. The BLOB analysis method belongs to the method of extracting image features. Its main purpose is to combine the pixels with similar gray values in the image to form BLOB blocks. The main processing flow of the BLOB analysis method is shown in Fig. 1.
Firstly, the grayscale image of the target is acquired by CCD. Then, the segmentation threshold of the image is extracted through the gray difference between the target and the background. The commonly used methods are: OTSU method, iterative method, and maximum entropy method. After the threshold is found, the image is divided into two categories: target and background. The image above the threshold is defined as the target, and the following is defined as the background. After threshold segmentation, the connected parts of the object in each line of the image can be extracted through run-length scanning. The run-length connect marking is mainly used to analyze whether the run-length between each line are connected. The connected run-lengths are marked with the same block number [13, 14, 15, 16]. Through the run-length connect marking, all connected run-lengths are merged into the run set of the same block, which is called BLOB. After calculating and analyzing the data in the patch run array, the characteristic parameters of the block could be got, such as area, center of gravity, distribution range, and so on. Because the target feature always occupies a large area, the feature with the largest area could be regarded as the target. The distribution parameters (W, H, limX, limY) of the feature and the range of the target could be obtained by extracting the feature parameters. W and H represent the width and height of the target, and limX and limY represent the upper-left coordinates of the target distribution.
BLOB feature data with the largest area
BLOB feature data with the largest area
BLOB analysis method processing flow.
BLOB analysis results of coal gangue images.
Figure 2 shows the distribution area of coal gangue extracted from the coal gangue image by BLOB analysis algorithm, and the BLOB characteristic data of the largest partition is shown in Table 1.
Through the extraction of feature distribution data, the analysis range of image data is reduced from 1600
BLOB analysis method could locate the target effectively and consume little time when there is a certain difference between the background and the target. In the real work environment, the coal gangue color is very close to the conveyor. It means it’s hard to get the target boundary especially for the image which only has coal or just belt background. Before using the blob analysis method, Gray scale methods is used to remove most cases that are difficult to distinguish boundaries.
Basic principle of support vector machine
SVM (Support vector machine) is a machine learning method based on statistical learning theory. The classification model based on minimum structural risk has strong generalization ability for actual data. It has more advantages in small samples, nonlinear and high-dimensional identification problems. It is also a binary classification tool to solve machine learning problems with optimization methods. The essence of coal gangue recognition is to divide the coal flow on the conveyor belt into non-gangue and gangue. Non gangue includes coal, debris, and background, which is a typical binary classification problem and is suitable for learning and training with support vector machines.
For a given collection of training samples TS
When dealing with the nonlinear classification problem, the sample data is mapped from the original space to the high-dimensional feature space by nonlinear mapping, and the classification hyperplane is established in the high-dimensional feature space. The optimal classification function is:
In which
In many problems, the distribution of data is nonlinear and separable, making it impossible to use linear models for accurate classification or regression prediction. By using kernel functions, we can map data from the original space to a higher dimensional feature space, thereby making the data linearly separable in the new space.
The procedure of Coal gangue classification model establishment.
A good SVM classification model is very important for gangue recognition. The procedure is shown in Fig. 3. Firstly coal and gangue images are acquired by the image collection devices. Secondly image blob analysis is executed and the target area is identified. Then parameters evaluation is executed on images target area. All the data are collected and divided into test set and training set. After that the training set data are put into the SVM function to get the model parameter. Finally the SVM model is gotten.
The basic principle of support vector machine can get the classification model of coal and gangue, which mainly needs: rich coal and gangue image samples (including training and testing samples), the selection of characteristic parameters for classification, and obtaining a sufficient data set. At the same time, according to the judgment effect of each single parameter of coal gangue in the literature, the reflection of the characteristics of coal gangue in the image and whether the characteristic parameters are independent, comparative analysis and screening. Many scholars have conducted experimental comparison on image parameters of coal gangue recognition [1, 4, 5]. Zhang L does some experiments to test six gray-scale features and four texture features for coal gangue identification. Six groups of coal and gangue images with the same number were selected for feature extraction experiments, and the obtained data were plotted into a box-plot. It’s found that characteristics of gray skewness, gray variance and texture contrast have highest discrimination on coal and gangue images [5]. We do the similar experiments to select effective evaluation parameters. The parameters with good results are mainly as follows: GLCM texture feature parameters, image grayscale feature parameters, and focus function. Nine parameters such as texture (correlation coefficient), texture (contrast), gray mean, third moment, histogram peak, gray distribution range, Brenner gradient function and entropy are selected to form the feature vector
Nine evaluation characteristic parameters
Schematic diagram of the classification model.
After determining the characteristic parameters to be extracted, the image sample collection of coal and gangue is obtained first. The image of each sample is analyzed by BLOB analysis method. At the same time, after obtaining the distribution range of the corresponding target, we use the gray value of the pixels in the distribution range to calculate the nine characteristic parameters of the sample. After completion, you can get the support vector (
After establishing the classification function model, we used the verification set to test the trained model. Then the classification accuracy is used as the performance index to evaluate the classification.
Coal gangue samples were selected from Tongxin Mine, Datong, Shanxi Province. There were 56 samples, including 28 pieces of coal and 28 pieces of gangue, all of which were similar in color and difficult to identify. All sample images were first placed on the conveyor belt and grey images were collected. Next, BLOB analysis was carried out by software to get the local images of the concerned objects. After the completion, 9 characteristic parameters of the local images were calculated and recorded, and coal and gangue were marked at the same time. Coal is 1, and gangue is 0, and the obtained data constitute a sample data collection. The data are divided into two parts: training group and testing group, of which 1–18 and 29–46 are the training groups, that is, the data of 18 small probability gangue samples and 18 coal samples are used for model training, and the remaining 19–28 and 47–56 are used for testing. After the test and training groups are selected, the data are normalized. After the completion, the parameters are evaluated by using the function SVMcgForClass of LibSVM, and the optimal parameters are obtained by searching the values of c and g in the exponential grid of 2.
Figure 5 is the result of fine parameter selection. It can be seen from the figure that under the K-CV method (
Figure 6 is the prediction result obtained by testing the test samples with the obtained classification model. It can be seen from the figure that the prediction accuracy rate is 100%, which shows that the classification effect of the SVM classification model based on image BLOB analysis is good.
Comparison results of four evaluation methods
Comparison results of four evaluation methods
SVC parameter selection results.
Prediction results of classification model.
Design of online coal gangue identification algorithm
From the analysis of the previous problems, it can be seen that the main difficulty of online identification of coal gangue lies in the definition of the target range of coal gangue and the calculation time. The method of establishing the mathematical model by machine learning for classification judgment has high accuracy, but it takes more time to extract the characteristic parameters for the whole map analysis, which leads to real-time failure; BLOB analysis method can effectively define the target range, thus narrowing the image analysis area. Therefore, it is helpful to solve the problems of real-time and accuracy by combining them in online image recognition of coal gangue.
Although the model established by BLOB analysis and machine learning can distinguish coal and gangue well and greatly reduce the calculation time, it still takes a lot of time to analyze a coal or gangue image and calculate 9 evaluation characteristic parameters. Therefore it cannot be directly applied to the gangue sorting system. However, many previous studies have shown that the evaluation speed of a single parameter is the best. Therefore, the method of multiple filtering can be adopted to identify the easily distinguishable scenes first, and the classification model can be used to identify the gangue for the difficult-to-identify image scenes.
Table 3 list the comparison results of four evaluation methods. As we can see, third moment evaluation method and grey variance method could be used as the whole image evaluation because of low time cost. If the threshold is set enough high and low, the obvious gangue could be identified immediately. Blob analysis and Grey variance could be used when there is not obvious gangue in the image. The gangue most difficult to identify is achieved using the support vector machine method.
Figure 7 is the flow chart of the online identification algorithm of coal gangue based on triple filter model. Gray images collected by CCD can be divided into four categories: background images, no gangue images, obvious gangue images, and suspected gangue images. Through the first filtration, the gray value of the image is distinguished by the third-order moment evaluation, and then according to the obvious characteristics of the coal gangue image with large third moment evaluation value, the image with small evaluation value can be regarded as no coal gangue and does not need follow-up processing. The background image and no gangue image can be filtered by this processing. The image with larger third-order moment evaluation value shows that there is a certain probability of coal gangue. The second filtration is carried out on the image that meets this condition, that is, it is judged according to a single characteristic parameter. For example, through the grey variance value to judge: if the variance value exceeds the threshold, we can judge that there must have gangue in the image, and this step will filter out the obvious coal gangue image. If the variance value does not exceed the threshold, BLOB analysis and local grey variance method are used to filter the suspicious gangue image. After BLOB analysis, the distribution area of the target object is located, and then the local variance within the region is calculated. If the local variance value is greater than the set threshold, it means that there is coal gangue in the image. If the local variance value is lower than threshold, third filter is executed. Nine local characteristic parameters within the distribution range of the target object are calculated, and the values of the target characteristic parameters are substituted into the set classification model for discrimination. After calculation, the classification result is obtained, where 1 is gangue and 0 is coal. After the third filtering, the image is processed by the support vector machine model for only some suspected gangue images which are difficult to distinguish. This can save most of the computer processing time.
Flow chart of online identification algorithm of coal gangue based on triple filter model.
According to the characteristics of intelligent sorting system for coal gangue, a set of user application programs, integrating image acquisition, analysis and processing, and output control is developed .It uses Boland C++ as the development platform and the online image recognition technology of coal gangue described in this paper.
The on-line identification and sorting system of coal gangue in Tongxin Coal Mining Plant is installed and debugged in the field. The equipment mainly includes: 2 sets of visual inspection system, 2 sets of gangue picking robot, 1 set of belt transmission system. The device is shown in Fig. 8.
Equipment of intelligent sorting system for coal gangue.
To test the gangue identification effect of the algorithm in Fig. 7, the gangue sorting system was used to carry out the sample identification experiment. All samples are provided by Tongxin Coal Mine, and the proportion of coal and gangue is 50% and 50%, respectively, including 60 pieces of coal and 60 pieces of gangue. The length range of all samples is 300 mm-600 mm, and the conveyor belt speed is 0.3m/s. The coal gangue recognition algorithm selects the grey variance method, BLOB analysis
The experimental process is as follows: (1) 120 coal gangue samples with similar color were selected from the coal mine as training samples, and the support vector machine classification model was obtained by the method shown in 2.3, and 10 coal gangue samples were used as test samples. (2) using the grey variance method, BLOB analysis
Experimental results of coal gangue identification
From the experimental results, it can be seen that the online identification algorithm of coal gangue based on triple filter model greatly reduces the probability of discrimination by using support vector machine model, while the classification model based on BLOB analysis and machine learning is not only effective, but also saves the calculation time. Therefore, the algorithm effectively improves the recognition rate of coal gangue and shortens the average recognition time, which can meet the requirements of real-time detection. The gangue identification algorithm is actually used in the gangue picking line of the factory, the automatic gangue picking can be realized under the condition that the coal flow rate is not particularly large, which greatly reduces the labor intensity of workers and improves the work efficiency.
An online image recognition algorithm of coal gangue based on BLOB analysis and machine learning is proposed.
Firstly, the images of coal and coal gangue samples, which are difficult to identify, are selected, and the local image of the research target is obtained after BLOB analysis. Then the image sample set is constructed based on the local image. After completion, three categories and nine evaluation parameters are selected as feature vectors to evaluate the image of the sample set and construct the data set. The kcv cross validation method is used to optimize the model parameters of the support vector machine, and the correct classification model is trained and established by using the classification method of support vector machine. On this basis, an online image recognition algorithm is put forward, which filters the easily recognized gangue or coal by triple filter model with small calculation, and only discriminates the suspected gangue image which is extremely difficult to distinguish by calculating the local characteristic parameters and inputting them into the classification model. This method solves the problem of image recognition in the process of coal gangue separation, improves the accuracy of coal gangue recognition, and ensures the real-time detection. The optimization of intelligent coal gangue separation system will mainly focus on the establishment of more image databases and networking in the future.
Footnotes
Acknowledgments
This work has the “Intelligent Measurement and Control Technology” Innovation Team Project of Guangdong Education Department (2020KCXD065), and “Intelligent Manufacturing Equipment Technology Engineering Center of Guangdong Education Department” (2022GCZX025) project support. Thanks the reviewers and editors for their valuable comments on this paper. I will revise the paper accordingly.
