Abstract
It is the key step of the classification and recognition to segment 3D point cloud. Aiming at the shortcomings of super-voxels concavo-convex segmentation algorithm for 3D point cloud, an efficient segmentation method based on multi-feature fusion is proposed. Firstly, the noises of 3D point cloud are removed by the statistical outlier removal filter, and the denoised 3D point cloud is simplified by the voxel grid filter. Secondly, it is divided into many voxels with the same size by the octree, and the voxels with the smallest mean curvature in local neighbourhood are used as seed voxels for regional growth to form super-voxels. Next, the super-voxels adjacency graph is structured and the concave-convex feature, continuous feature and colour feature between adjacent super-voxels are calculated as the weight on their edges. Finally, an arbitrary super-voxel is selected as the seed super-voxel to regional growth based on multi-feature weights of the edges in order to achieve the segmentation of the point cloud. The experimental results show that the proposed method whose segmentation speed, stability and accuracy are higher than existing methods greatly improves the over-segmentation or under-segmentation of the super-voxels segmentation algorithm.
Keywords
Introduction
With the development of artificial intelligence, the machine vision gradually expands from the two-dimensional image to the 3D image. 3D point cloud, as one typical representative of the 3D image, has been widely used. 3D point cloud segmentation is the process of dividing the point cloud into multiple regions which has the same features in the same area. It is a vital part in the registration, classification and recognition of 3D point cloud. The segmentation effect plays a decisive role in the follow-up work. Because of high redundancy, uneven density and uncertain structure of point cloud, the segmentation still faces great challenges.
In general, there are five kinds of 3D point cloud segmentation methods: point cloud segmentation based on edge detection [1, 2], point cloud segmentation based on region growth [3, 4], point cloud segmentation based on feature clustering [5, 6], point cloud segmentation based on model [7, 8] and point cloud segmentation based on graph [9, 10]. In recent years, 3D point cloud segmentation by super-voxels has gradually attracted the attention of some scholars. At first, some scholars extended the pixels of the two-dimensional image to the voxels in 3D image, and 3D point cloud is studied by voxels. Point cloud or point cloud video are analysed, clustered and segmented by super-voxels in the literatures [11–15]. The advantages of these methods are to directly segment the point clouds without training models, and whose effect is ideal. However, there are still some shortcomings: the unreasonable selection of seed voxels will lead to the instability of segmentation effect, especially when seed voxels selected are in concave edges, which will result in over-segmentation or under-segmentation; in the 3D point cloud segmentation procedure, extremely close objects that are not connected each other cannot be segmented; overlapping objects with different colours cannot be divided. Therefore, a point cloud data segmentation method based on multi-feature fusion is proposed.
The shortcomings of these algorithms are improved by this paper. Firstly, the noises of 3D point cloud are removed by the statistical outlier removal filter, and the denoised 3D point cloud is simplified by the voxel grid filter, respectively. Secondly, the voxel with the smallest mean curvature is regarded as the seed voxel to form super-voxel, in order to improve the stability of the algorithm. In 3D point cloud segmentation procedure, concave-convex feature, colour feature and continuous feature are fused together for segmentation in order to improve the accuracy of the algorithm.
Related work
The super-voxel concave-convex segmentation algorithm was first proposed by SC Stein [15] in 2014. On the basis of super-voxel clustering segmentation, it is performed cluster again by the concave-convex feature between super-voxels.
Super-voxel formation
Firstly, the Octree [16] is used for dividing the 3D point cloud into many voxels with the specified size. Secondly, some voxels which keep certain distance are evenly selected as seed voxels. The distance, colour difference and normal-weighted feature distance between the seed voxel and the adjacent voxels is calculated, and voxel with the minimum feature distance is merged into the seed voxel. Regional growth is carried out according to this growth criterion until there is no renewable growth of voxel, and finally the super-voxels are formed. The formation process of super-voxel is shown as Fig. 1.

Formation process of super-voxels.
In mathematics, the convex body is defined as the geometry which the line between any two points is in the surface or inside. Most of the objects are convex bodies in nature, but the connections among different objects are generally concave. Thus, SC Stein [17] used the concave-convex feature for 3D point cloud segmentation on the basis of super-voxels segmentation. The process of concave-convex segmentation is as follows: A seed super-voxel is selected arbitrarily and is marked. Starting from the seed super-voxel, regional growth and marking are carried out according to whether there is a convex surface between the seed super-voxel and the adjacent super-voxel, and the growth is stopped when there is no new convex surface. An unmarked super-voxel is selected as the next new seed super-voxel. Repeating the above process until all the super-voxels are marked.
Methods
Pre-processing of 3D point cloud
Removing noises of 3D point cloud
Because of the influence of external factors and scanning equipment measurement errors, there are always some noises in 3D point cloud acquired by 3D laser scanner. Therefore, the noises of 3D point cloud are removed before segmentation.
The 3D point cloud usually has some outlier noises obtained by the 3D laser scanner so the statistical outlier removal filter [18] is selected to filter it so as to remove the noises. The process is as follows: Calculate the average distance d from each point to its nearest K neighbourhood points. Calculate the expectation dm and standard deviation s of the mean distance d, and calculate the distance threshold dt by formula (1);
If d>dt, the outliers are filtered out, otherwise no.
The result of the experiment is shown in Fig. 2. The noises of the raw 3D point cloud reduce significantly and the edge of 3D point cloud is smoother by the statistical outlier removal filter. The experimental result shows that the filtering algorithm can filter out the outlier noise points on the premise of keeping point cloud feature, which lays a foundation for the subsequent 3D point cloud segmentation.

Contrast figures before and after the statistical outlier removal filtering.
Because of great amount or high density of the acquired 3D point cloud, computational speed declines, this seriously affects the rate and time of the subsequent 3D point cloud segmentation and recognition. Therefore, the paper uses the voxel grid filter [19] to reduce the number of 3D point cloud on the premise of keeping the geometric features unchanged, thus the segmentation speed is improved.
This filtering process is as follows: The 3D point cloud is divided into many small voxels by the Octree. Calculate the gravity centre of each voxel; Using gravity centre point to replace all point cloud in voxel.
This method is slower than using voxels geometric centre. The gravity centre refers to the equivalent point of the object gravity. For the object with regular shape and uniform density, the geometric centre is equal to the gravity centre. But when point cloud is curved or the density is not uniform, the gravity centre is more accurate than the geometric centre. The simplified effect is shown in Fig. 3, the scenes 1 and scenes 2 were 211,794 and 962,546 points before simplifying which are simplified as 111,355 and 412,108 points respectively. On the premise of keeping the raw 3D geometric feature unchanged, the 3D point cloud density decreased obviously. If there are more curved objects in the scene, the effect is better.

Contrast results before and after simplifying.
The Octree is used for dividing the 3D colour point cloud into many individual voxels. In the process of forming super-voxels, seed voxels are selected first. At present, voxels which keep a certain distance are selected as seed voxels for regional growth to form super-voxels. Although this method is simple, it has great instability. When the selected seeds fall on the concave edge of the object, the two sides 3D point cloud of the concave edge will be regarded as the same class, which leads to the super-voxels error growth and segmentation fault.
In this paper, the voxel with the smallest curvature in the local area is selected as the seed voxel to make a regional growth and to form super-voxel, thus avoiding the wrong segmentation due to the seeds falling on the concave edge. The mean curvature [20-24] of all points in a voxel is calculated as follows:
The discrete curvature K
L
of a point z in the voxel is calculated by formula (2).
Where α ij + β ij are diagonals of connecting z i and z j edges respectively. N(i) is the set of z i neighbourhood points. Amin represents the minimum neighbourhood area of point z i .
The average curvature of all points is the expected value of the curvature of all points in the voxel, it is calculated by formula (3).
Where N is the number of all points in the voxel.
Concave-convex feature
In mathematics, the convex body is defined as the geometry which the line between any two points is in the surface or inside. Most of the objects are convex bodies in nature, but the connections among different objects are generally concave. Based on this, 3D point cloud was segmented by using concave-convex feature, which is calculated by the extended convexity criterion and the sanity criterion.
(1) The extended convexity criterion
Whether the connection between these is convex or concave can be inferred from the relation of the surface normal to the vector joining their centroids. As shown in Fig. 4, consider two adjacent super-voxels with centroids at the positions

Concave or convex feature.
Concave-convex feature can be expressed by the following formula (4).
The basic convexity criterion can be expressed by the following formula (5).
(2) The sanity criterion
When two super-voxel adjacent surfaces are discontinuous but only have one join point, the extended convexity criterion mentioned above may sometimes be misjudged as a convex relation. Based on this, a sanity criterion is proposed to detect whether two adjacent super-voxels adjacency surfaces are truly connected, so as to judge whether convex relationship is formed between the two surfaces.
The complete sanity criterion is defined by the following formula (6).
Where
When the super-voxel adjacency graph is used for concave-convex segmentation, it is impossible to effectively segment the objects which contain super-voxels with close distance but unrelated. In this paper, the continuous feature detection is added to the construction of super-voxels adjacency graph, which can effectively divide the two objects which are very close but not connected.
The principle of continuous feature is to calculate the distance d
t
between each super-voxel and its adjacent super-voxel in the constructed super-voxel adjacency graph. If the distance exceeds the set threshold d, there is no adjacency edge between the two super-voxels. Otherwise, the adjacent edge exists. d
t
is calculated by formula (7), (8) and (9).
Where R
s
is the distance between the super-voxels,

Contrast diagrams before and after joining continuous feature.
The experimental result shows that two close but disconnected sofa parts can be effectively segmented after the continuity detection.
At present, the colour feature is not used in point cloud segmentation. When divided a number of objects neatly stacked but having different colours, they will be segmented into one class, resulting in segmentation errors. In this paper, the colour feature is added in 3D point cloud segmentation. The process of which is as follows. Calculate the average colour value of each super-voxel. The colour difference between adjacent super-voxels is calculated one by one. If the difference is bigger than the set colour threshold, the adjacent super-voxel is not merged and stops growing.
The effect of colour detection is shown in Fig. 6.

Compared results before and after adding color feature.
The experimental result shows that the super-voxel concave-convex segmentation algorithm divides the two books, brown and black side by side, into the same class, which leads to the error segmentation. The colour feature detection can effectively segment the two books into different two classes.
The noises of 3D point cloud are removed by the statistical outlier removal filter, and the denoised 3D point cloud is simplified by the voxel grid filter, respectively. The Octree is used for dividing the point cloud data into many voxels with the same size, the voxel with the smallest mean curvature is regarded as the seed in local neighbourhood, and the super-voxels are formed by region growth. The super-voxels adjacency graph is structured. The concavity-convexity, continuity, and colour information between adjacent super-voxels are calculated as the weights on their edges. An arbitrary super-voxel is selected as the seed, and regional growth is performed according to the weight value of the edge in order to segment the 3D point cloud. If there is an unclassified super-voxel, it will be viewed as a new seed and repeat (4) until all super-voxels are classified.
Results and discussion
Experimental dataset and platform
The 3D point cloud dataset of this paper was scanned by KinectV1, its sensor configuration parameters are shown in Table 1. To greatly verify the segmentation ability of the algorithm the dataset is composed of 100 3D point cloud indoor scenes. Each scene includes sofas, tables, chairs, caps, water cups, boxes and other objects with the corner angles. The experimental platform is Intel Core i5-8250, with 8G memory, Windows 10 64-bit operating system, VS2015VC++win64 console application, and open source point cloud library PCL1.8.0.
Performance comparison in each aspect
Performance comparison in each aspect
In order to verify the effect of the proposed algorithm, the proposed algorithm is executed on 3D point cloud provided by the above dataset on the premise of setting the same parameters, and compared with the effect of super-voxels concave-convex algorithm. The segmentation result is marked with randomly generated colours. The results of various performances comparison are listed. Experimental results show that in 50 scenes, the proposed algorithm not only improves the speed, stability and accuracy of 3D point cloud data segmentation, but also is applicable to point cloud data of some complex scenes. Some representative experimental results are selected, as shown in Fig. 7 and Table 1.

Contrast of algorithms.
Figure 7(a1)&(a2) are the raw data of two scenes. Figure 7(b1)&(b2) are the results performed by super-voxels concave-convex segmentation algorithm. Figure 7(c1)&(c2) are the results performed by region growth algorithm. Figure 7(d1)&(d2) are the results performed by the proposed method in this paper. By contrast, most targets in the scenes can be separated by super-voxels concave-convex segmentation algorithm but with over-segmentation or under-segmentation result. In scene 1, the two sofas are not separated from the wall behind them while the wall behind them is divided into two parts. In addition, the chair on the far right is divided into two parts (the armrest and the rest). In scene 2, the white cup, the cap edge and the table are divided into one class, and the two chair feet with obvious concave feature are not separated. Some targets in the scenes can be separated by region growth algorithm but with over-segmentation result. In scene 1, the objects on the table and armrests of single sofa are over-segmented. In scene 2, the left side of the box and the right side of the computer case with obvious concave feature were separated one class with the ground, the cap and the yellow soda can on the table were not separated completely. The over-segmentation or under-segmentation is caused by the unreasonable selection of seed voxel, the absence of continuous feature and colour feature in the process of regional growth. Figure 7(d1)&(d2) are results of the proposed method by this paper, which can well solve the over-segmentation or under-segmentation problems existing in the super-voxels concave-convex segmentation and the region growth algorithm, and the segmentation effect is better visually.
The proposed method is better than the super-voxels concave-convex segmentation algorithm and the region growth algorithm from four aspects in Table 1. Running time is reduced almost by 50% especially. The number of clustering is quite close to the number observed by the human eyes. After a lot of experiments in the above dataset, the average segmentation accuracy of super-voxels concave-convex segmentation algorithm is 88.7%, the average segmentation accuracy of region growth algorithm is 86.9%, and the method proposed by this paper improves 3.4% and 5.2%.
In addition, experiments are also carried out in the slightly messy 3D point cloud scene. The experimental results are shown in Fig. 8. In Fig. 8(a), the yellow flashlight in the lower left corner of the table and the red flashlight in the middle have not been separated. besides the cup and the cap in the upper right corner of the table are divided into one category in Fig. 8(b). The rear two chairs were divided into one class, some of the objects on the table are over-segmented in Fig. 8(c). However, the above subjects are separated by the proposed method in Fig. 8(d). Therefore, the proposed method has stronger robustness and more accuracy for slightly messy scenes.

Effect comparison of messy scene.
Aiming at the shortcomings of the current super-voxels concave-convex segmentation algorithm, a new segmentation method is proposed, which with multi-feature fusion characteristics for 3D point cloud data.
The advantages of the proposed method are as follows: Firstly, the 3D point cloud data obtained by the 3D scanner are noisy or mass, which affects the segmentation effect and running time. the noises of 3D point cloud are removed by the statistical outlier removal filter, and the 3D point cloud is simplified by the voxel grid filter, respectively. Secondly, in the process of super-voxels forming, the voxels with the smallest mean curvature are selected as the seeds, which avoid the error segmentation that voxel seeds are at the concave edge. Finally, super-voxels concave-convex segmentation algorithm cannot segment many close but unconnected objects as while as neatly arranged objects with different colours. In this paper, the continuous feature and colour feature are added in the process of concave-convex segmentation, which can effectively solve the above problems. The findings show that the proposed algorithm can greatly achieve segmentation results, increase the accuracy of segmentation, and shorten the running time.
In addition, the parameters of this algorithm need to be manually debugged. How to adjust parameters adaptively according to local geometric characteristics on the basis of maintaining good segmentation effect is the focus and direction of future research.
Competing interests
The authors declare that they have no competing interests.
Funding
Not applicable.
Authors’ contributions
RXL carried out 3D point cloud data segmentation method studies, participated in the new algorithms alignment and drafted the manuscript. WW carried out the implementation and experimental analysis of the algorithm. XSJ conceived of the study, and participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.
