Moving object segmentation in point cloud data using hidden Markov models

Handling uncertainty in the problem inputs

The MOS problem requires two inputs: a sequence of point cloud data, and the corresponding 6-DOF pose of the sensor.

Point cloud measurements are typically recorded using LiDAR sensors. LiDARs return a set of range measurements at specified heading and elevation angles at frequencies typically between 5 and 20 Hz. The sensors exhibit range measurement uncertainty that increases with distance. Additionally, the intrinsic parameters (beam heading, elevation angles) are only known to a level of uncertainty and require careful calibration to allow correct interpretation of the range measurement (Nouiraa et al., 2016).

Transforming the point cloud measurements into a frame relative to previous measurements (e.g. a map frame) requires accurate sensor pose estimation. The sensor’s pose is generally estimated by Global Navigation Satellite Systems (GNSS) or point cloud odometry, such as MOLA (Blanco-Claraco, 2024). These pose estimates inherently contain uncertainty, which, combined with the sensor’s measurement uncertainty, propagates in transforming the point cloud to the common reference frame. The effects are more pronounced at greater ranges, where pose errors result in substantial errors in the transformed measurements. Furthermore, if the platform’s pose solution and LiDAR are offset, as in the case of using GNSS for platform pose, accurate extrinsic calibration is required to locate the LiDAR relative to the navigation solution (D’Adamo et al., 2018).

LiDAR provides accurate range measurements at high frequencies but has known sensing limitations. Sensor measurements are sparse at long ranges, making it challenging to separate noisy detections from objects with only a few measurements. The behaviour of range returns from reflective surfaces is unpredictable and leads to incorrect beliefs about space occupancy, resulting in subsequent false detections. However, these surfaces are commonly encountered, and algorithms need to account for these sources of uncertainty. Sensor returns from vegetation and atmospheric obscurants such as dust also exhibit unpredictable behaviour (Phillips et al., 2017).

Providing real-time performance

Systems aim to provide detection results at rates commensurate with the cycle times of decision-making processes that consume the information. High-density scanners capture more information about the environment, but processing the increased measurements leads to greater computational demand. Modern LiDARs provide millions of measurements per second at high frequencies. Processing significant amounts of data in real-time applications requires hardware that is typically uncommon on mobile robots. Alternatively, the data volume can be reduced through pre-processing, but this generally introduces quantization and diminishes the level of detail in information about the scene.

Providing generalized performance

MOS has numerous applications ranging from robots interacting with humans (Falque et al., 2023) to vehicles navigating unstructured environments (Wojke and Häselich, 2012). An approach is desired that provides generalized performance across varying platform dynamics, sensor characteristics, and application environments. Learning-based approaches commonly fail to generalize performance between different sensors, platforms, and operational environments due to the large variation in the possible inputs. A common property of all sensors is the information provided about occupied and free space, and we use this to form confident beliefs about dynamic measurements.

Benchmarking metrics

Numerous labelled datasets are available for benchmarking the performance of MOS algorithms in comparison to existing methods. Each benchmark dataset includes a sequence of scans, in some cases the sensor’s estimated pose, and a set of labelled ground truth scans. A labelled ground truth scan provides a binary classification of static and dynamic points in the scan. These scans have either been annotated manually (Schmid et al., 2023), by a static map generation approach (Lim et al., 2023), a deep-learning network (Chen et al., 2022), or a combination of any. While numerous labelled datasets exist, the definition of a moving point varies significantly, leading to an inaccurate comparison between algorithms. This raises the question of what defines a moving object?

An object is moving if its pose is changing relative to a fixed reference frame, regardless of the object’s velocity and previous dynamic state. Given the k-th point cloud located in a fixed reference frame, for example, the map frame, sensor measurements corresponding to objects that are moving are classified as dynamic. The remaining measurements are static. Measurements corresponding to objects that (i) were moving and have stopped moving, (ii) move at a future instance (t = k + 1), or (iii) have the potential to move, are not dynamic objects given the earlier description.

This work focuses on detecting objects that are currently moving. We acknowledge the difficulty in generating ground-truth labels using experimental data, with labelling often capturing other instances.

Learning-free approaches

Learning-free approaches are generally categorized into methods identifying discrepancies between successive scans registered in a common frame and methods constructing a continuous representation of the environment updated sequentially with new observations. In common, these approaches query changes in free space to provide cues for detecting dynamic objects.

Learning-based approaches

Most recent MOS approaches are learning-based. These methods use labelled data to train deep-learning networks that learn patterns to identify dynamic objects in point cloud data. The idea is to train these networks with sufficient high-quality labelled data to allow for accurate performance in unseen instances with variations in the input data quality, that is, sensor characteristics, sensor noise, and pose estimation uncertainty. The performance usually depends on the design of the network architecture and the quality of the labelled data used for training.

Convolutional Neural Networks (CNNs) frequently form the foundation of MOS deep-learning models. Chen et al. (2021) use sequential range images with a CNN to identify residuals in point clouds registered over a receding window to separate static and dynamic points. The algorithm performs accurately in scenarios similar to its training set but shows significantly reduced performance when evaluated with datasets collected with differing motion profiles and diverse dynamic objects. To improve the robustness of the network across applications, Mersch et al. (2022) employ spatiotemporal 4D convolutions combined with a binary Bayes filter for a recursive fusion of predictions over a receding horizon. This unique approach to training a CNN allows for significant improvements in the generalized performance across testing scenarios as the network trains to detect changes in a sequence of scans compared to appearance-based techniques. Mersch et al. (2023) extend this to include volumetric beliefs and increase dynamic detection while recursively updating a local static map of the environment. Wang et al. (2023) also uses 4D convolutions with instance detection and feature fusion to better identify static and dynamic objects in the scan.

Several other architecture designs also achieve accurate performance, including the use of dynamic graph CNNs (Wang and Solomon, 2021), leveraging semantic and motion labels in a range image-based CNN (Kim et al., 2022), and using a bird’s-eye view approach for motion detection (Zhou et al., 2023).

The critical hurdle involves generalizing performance across various operating environments and sensors with differing point cloud densities and scan patterns. The recently introduced HeLiMOS dataset by Lim et al. (2024) emphasizes the importance of generalization and provides labelled data from four different LiDARs along the same sequence. The performance of existing methods on the different LiDAR datasets without retraining highlights the poor generalized performance. Furthermore, the dependence on the training inputs and consequent variation in results reveals the gap for a robust approach to the MOS problem.

Map representation and update

A global map frame, M , is defined to indicate the origin of the environment, commonly located at the origin of the first scan. At time k, the map, M M , k , is discretized using voxels, v, of a user-configured size, Δ. The voxels are augmented with properties alongside their position to model spatiotemporal behaviour. The properties are summarized in Figure 3 and detailed throughout this section.OPEN IN VIEWER

Without uncertainty, detecting dynamic objects is as simple as updating voxel occupancy with new observations. If a voxel’s state changes from free to occupied, it suggests a dynamic measurement. However, uncertainties in point cloud measurements, sensor pose, or poor sensing conditions lead to incorrect or missed detections. Existing beliefs should be fused probabilistically to handle the various sources of uncertainty. As the state of each voxel is not directly interpretable due to the associated uncertainty, an HMM is used to represent the state of each voxel in the map, M M , k .

Representing each voxel using an HMM (Rabiner, 1989) requires defining: the n states of the voxel, S = {S₁, …, S _n }, the transition probabilities between the states captured in the state transition matrix A ∈ R n × n , and a process to generate the likelihood of the i-th voxel being in each state at time k given the sensor observations, encoded in the measurement conditional densities, B i , k ∈ R n × n . Let the i-th voxel’s state vector, x ^ i , k ∈ R n × 1 , denote the probability of being in each state at time k. The voxel’s initial state is given in x ^ i , 0 . Once defined, a voxel’s state is efficiently updated using the recursive HMM filter (Elliott et al., 2008),

x ^ i , k = η i , k B i , k A x ^ i , k − 1 ,

(1)

Each voxel, v, is represented using three states (n = 3), S = {unobserved, occupied, free}, with each voxel initialized in the unobserved state, x ^ i , 0 = 1 0 0 T . The state transitions are set as

A = p ( S 11 ) p ( S 21 ) p ( S 31 ) p ( S 12 ) p ( S 22 ) p ( S 32 ) p ( S 13 ) p ( S 23 ) p ( S 33 ) = 1 − 2 ϵ 0 0 ϵ 1 − ϵ ϵ ϵ ϵ 1 − ϵ ,

(2)

Interpretation of a new observation

The new point cloud at time k in the sensor frame, P S , k , is transformed by the current sensor pose estimate, T M → S , k , to locate the scan points in the map frame,

P M , k = T M → S , k P S , k .

(3)

The interpretation of the current scan provides information about occupied and free voxels. However, believing the information and overwriting previous beliefs directly generates suboptimal results, as uncertainty leads to a rapid transition of voxels between occupied and free states. This is problematic as the main cue used for dynamic detection is the transition of previously free voxels turning occupied. Instead, the likelihood of each observed voxel being free and occupied is used to construct a belief of the current state of the voxel. The measurement conditional densities of the i-th voxel being in a particular state given an observation are

B i , k = diag ( 0 , L v i o , 1 − L v i o ) ,

(4)

An observed voxel is likely to be occupied if it is close to a voxel in the voxelized scan, P M , k ′ . Similarly, an observed voxel is likely to be free if it is distant from a voxel in the voxelized scan. This information can be captured using a Gaussian Distance Field (GDF), which is derived from the commonly used Euclidean Distance Field (EDF) (Oleynikova et al., 2016) (Figure 2(c)). First, an EDF is constructed for the current scan. Figure 4 shows an example. For each observed voxel, v i ∈ P M , k ′ o b s , the closest distance, d _i , to the nearest voxel in the voxelized scan, v j ∈ P M , k ′ , is computed and augmented to voxel v _i ’s properties. This Euclidean distance is used to calculate the likelihood of being in each state by evaluating an unnormalized Gaussian function at d _i ,

L v i o = exp − d i 2 / 2 σ o 2 ,

(5)

OPEN IN VIEWER

Figure 4.

A voxelized scan is shown on the left, captured by the LiDAR at the origin. The computed Euclidean Distance Field (EDF) of the point cloud is shown on the right. The EDF values are transformed by a Gaussian function to compute an occupancy likelihood. Voxels close to occupied measurements are likely to be occupied, and voxels located away are more likely to be free.

A voxel’s current state, S_i,k, is updated when the probability of being in a particular state, x_i,k, is greater than a predefined threshold, p_min, else, the voxel remains in the previous state,

S i , k = S a r g m a x i ( x ^ i , k ) max ( x ^ i , k ) > p min .

(6)

Dynamic point identification

This process allows for an efficient probabilistic update of each voxel’s state in the global map. A voxel’s transition between free and occupied space is used to seed the detection of dynamic objects. The HMM per voxel allows for an accumulation of confidence before transitioning state. Figure 5 illustrates the process of estimating dynamic voxels with the following detailing each step.OPEN IN VIEWER

Detecting changes in voxel states

The first step is to identify voxels from the current voxelized scan, P M , k ′ , that changed state in the updated voxel map, M M , k ,

P M , k ′ c h g = v i ∈ P M , k ′ S i , k ≠ S i , k − 1 and S i , [ k − 1 , k ] ≠ S 1 .

(7)

Transitions to and from the unobserved state (S₁) are neglected as these are new observations. The resulting subset of voxels, P M , k ′ c h g ⊂ P M , k ′ , contain true and false dynamic detections.

Capturing neighbourhood behaviour using a spatiotemporal (4D) convolution

The change detection allows for likely dynamic voxels to be identified. However, each voxel is modelled independently, and changes in the voxel’s neighbourhood are not examined. Dynamic objects are likely to occupy space composed of several voxels, given a sufficiently small discretization resolution. Two heuristics can be stated without assuming the object’s shape or dynamics to keep the algorithm application agnostic; (i) a static voxel with many dynamic neighbours is likely a missed detection (false negative), and (ii) a dynamic voxel with many static neighbours is likely a false detection (false positive). A spatial (3D) convolution is performed to identify missed detections to reduce the false negatives and suppress noisy detections to decrease false positives.

While both heuristics significantly improve the results, they do introduce unwanted effects. As a consequence of heuristic (i), detections at sparse, and often long ranges, are missed as they exhibit similar behaviour to noisy measurements. Heuristic (ii) captures parts of static objects when a dynamic object moves close to static parts of the environment. This depends on the voxel size, as it is common for a voxel to capture both static and dynamic parts of the environment (e.g. feet and wheels contacting the ground). The benefits of performing the convolution significantly outweigh the unwanted side effects.

For each occupied voxel, v i ∈ P M , k ′ , the likelihood of being dynamic, L v i dyn , is calculated by summing state changes in the voxel’s local neighbourhood. A kernel, K m ∈ R m × m × m , is convolved with each voxel in P M , k ′ to compute the likelihood of the voxel being dynamic,

L v i dyn = v i ∗ K m , v i ∈ P M , k ′ = max ∑ α ( v j ) ∀ v j ∈ K m ( v i ) , 0 ,

(8)

where α(v _j ) is given by

α ( v j ) = 1 , if v j ∈ P M , k ′ c h g ∩ S ( v j ) ≠ S 1 , 0 , else if v j ∉ P M , k ′ c h g ∩ S ( v j ) ≠ S 1 , − 1 , otherwise .

(9)

The condition, S(v _j ) ≠ S₁, is added to avoid including detections that are close to the boundary between unobserved (S₁) and observed (S₂, S₃) space, as voxelization is known to be inaccurate at this boundary. A dynamic detection close to this boundary is penalized as in the third constraint of equation (8) to allow for the detection of high-confidence dynamic voxels only. Voxels missed due to this constraint are recovered in the final step of the pipeline. The likelihood of being dynamic increases with the number of voxels in the convolution kernel, K _m , that have changed state. A post-processing local neighbourhood median filter is also applied to smooth the likelihood values.

The spatial 3D convolution (equation (8)) is extended over a receding local window of size w _l , to compute a spatiotemporal 4D convolution,

L v i dyn = ∑ j = k − w l k v i ∗ K m , v i ∈ P M , j ′ .

(10)

The addition of the temporal convolution over w _l allows for further suppression of incorrect detections. For example, a dynamic detection in one scan and static in the remaining scans reduces the likelihood of that voxel being dynamic. At the end of this stage, all voxels in P M , j ′ have been assigned a likelihood of being dynamic (Figure 2(e)).

Automatic thresholding the 4D convolution

The spatiotemporal convolution assigns a high likelihood for dynamic voxels and a low likelihood for static voxels. The likelihood of a voxel being dynamic depends on many factors: the voxel size, Δ; the size of the convolution kernel, K _m ; the sparsity of the original point cloud, P S , k ; and the shape, size, and speed of the moving object. It is difficult to manually set a threshold to separate voxels into static and dynamic classes based on their likelihood. The automatic thresholding algorithm by Otsu (1979), commonly used in image processing, is applied to the convolution scores to compute a dynamic threshold, γ, that provides a binary distinction between static and dynamic voxels (Figure 2(f)). A lower bound on the threshold, γ_min, is required as the algorithm continues to provide separation for static scans due to the presence of noisy detections,

P M , k ′ d y n = { v i ∈ P M , k ′ | L v i dyn > max ( γ , γ min ) } .

(11)

It is important to mention that this only estimates the high-confidence dynamic voxels. The lower threshold, γ_min, corresponds to the minimum number of dynamic neighbours of voxel v _i in its neighbourhood defined by the kernel size K _m in the local window, w _l .

Preserving high-confidence dynamic voxels and extending to low-confidence areas

High-confidence dynamic voxels from the previous scan are preserved in the current scan (Figure 2(g)). The recursive preservation is independent of the receding window, w _l , used in previous operations. This aims to help capture the complete object. All dynamic predictions from the previous scan are not preserved, as these may contain false positive detections. Instead, only voxels with a likelihood of being dynamic, L v i dyn , greater than the minimum threshold, γ_min, are included,

P M , k ′ d y n = P M , k ′ d y n ∪ { v i ∈ P M , k − 1 ′ d y n | L v i dyn > γ min } .

(12)

This helps to preserve only the high-confidence predictions.

A nearest-neighbour dilation is applied to the high-confidence voxels to grow the dynamic detection results into neighbouring regions (Figure 2(g)), which were previously disregarded due to strict constraints to avoid identifying false positives,

P M , k ′ d y n = P M , k ′ d y n ∪ { v i ∈ P M , k ′ | ∃ v j ∈ K 3 ( v i ) : v j ∈ P M , k ′ d y n } .

(13)

The final step involves extracting the original point cloud measurements from P S , k that are captured in P M , k ′ d y n , and labelling them as being P S , k static or P S , k dyn .

Summary

Using HMMs provides a probabilistic approach to modelling the environment while extending image processing techniques to point cloud data filters the dynamic detections. The proposed approach uses a simple pipeline composed of three stages: (i) voxelizing a new scan, (ii) probabilistically integrating the new scan into the existing map, and (iii) detecting changes in the environment’s state using spatiotemporal (4D) convolutions. Table 1 provides a complete listing of the algorithm’s configuration parameters.

OPEN IN VIEWER

Table 1.

Summary of all algorithm configuration parameters and purpose.

Parameter	Units	Default	Purpose
ϵ	prob	0.005	The configuration parameter to construct the state transition matrix for updating each voxel’s state given an observation using the HMM filter. The matrix encodes the transition probabilities within states and is fixed for the MOS task
Δ	m	0.2	Voxel resolution used for discretizing the new scans and the local map
σ o	m	0.2	The lumped uncertainty in estimating a voxel’s occupancy given the sensor measurement and corresponding pose. The value is used for generating the measurement conditional densities. We typically set the uncertainty to reflect the uncertainty in the pose estimates
p min	prob	0.99	Probability threshold for a voxel transitioning to a state
m	voxels	5	Convolution kernel size (m × m × m)
γ min	voxels	3	A lower bound on the automatic Otsu threshold. The value corresponds to the minimum number of dynamic voxels in the kernel over the 4D convolution
w l	scans	5	Length of the local receding window for the 4D convolution computation
w g	scans	300	Length of the global receding window used to reset voxels. This is useful for handling erroneous observations and pose drift
r max	m	—	The maximum sensor range used for truncating new observations and maintaining the local map size. This is also the maximum MOS range

Performance benchmarking

The following evaluates the algorithm’s performance on nine datasets to support the claims of providing generalized performance across platform dynamics, sensor characteristics, and operating environments while using the same algorithm configuration. Results for two different voxel sizes are included, Δ = 0.2 m and Δ = 0.25 m, with corresponding lumped uncertainties of σ _o = 0.2 m and σ _o = 0.25 m. While the accuracy results are similar, a significant computational benefit is achieved using the slightly larger voxel size. A trend is observed in some results where the larger voxel size returns a better result as it captures the complete object, however, this is usually at the expense of increasing false positives. This is a consequence of the quantization. The average runtime of each dataset is detailed in the corresponding section. A comparison with existing methods is provided when runtime results are published by the corresponding method or the open-source package results are reproducible. The reader’s attention is drawn to the variability in the ground truth of the benchmark datasets.

Moving event dataset (MOE)

The MOE dataset by Chen et al. (2024) provides a series of simulated and real-life sequences consisting of various diverse dynamic objects aimed at improving moving event detection using LiDAR sensors. Simulation datasets are beneficial as the sensor’s pose and the MOS labels are known. The authors benchmark existing algorithms on three simulated sequences: Sequence 00 is a mobile robot exploring an indoor apartment with humans as dynamic objects in the Webots simulator, sequence 01 is an outdoor city scene traversed by a vehicle in the Carla simulator, and sequence 02 is recorded in a crowded pedestrian setting in the Gazebo simulator. Table 4 shows the published benchmark updated with the method proposed in this paper. The proposed algorithm performs consistently for all sequences, providing the best results for sequences 00 and 02. The benchmark uses the mean of the per-scan IoU metric for benchmarking performance, which is sensitive to the number of dynamic detections per scan. Most other benchmarks in this section use the total IoU, which provides a total sum of the dynamic detections for the entire sequence, reducing the sensitivity compared to the mean IoU.

OPEN IN VIEWER

Table 4.

Evaluation on the MOE dataset with best results in bold. 4DMOS and MapMOS were evaluated using their open-source packages, with the delayed results separated at the bottom of the table. Results for all other methods are as documented by Chen et al. (2024). The ground truth uses type (I) labelling.

Method	Sequence IoU (%)
	00	01	02
Removert	29.7	2.8	42.1
ERASOR	37.8	2.8	62.7
Octomap	32.8	3.1	65.2
Dynablox	32.0	19.5	49.2
DOD	78.6	14.2	59.5
M-detector	30.5	17.4	4.4
MotionBEV	0.2	5.5	6.9
InsMOS	49.5	28.2	37.9
4DMOS, online	71.2	26.1	66.0
This paper Δ = 0.20 m	80.0	38.2	81.9
This paper Δ = 0.25 m	70.6	40.5	77.6
4DMOS, delayed	78.0	25.7	76.2
MapMOS, volume	27.8	50.3	78.7

Sequences 00 and 02 are processed at 40 Hz for a maximum 50 m MOS range, whereas sequence 01 processes at 10 Hz for a maximum 20 m range and 3 Hz for a 50 m range. Sequences 00 and 02 are indoor environments, providing dense scans with a low maximum range, whereas sequence 01 is recorded in an outdoor environment with dense long ranges. The per-frame computation is proportional to the number of measurements (scanning density) and the scanning range. This relationship is observed for all datasets. The algorithm does not discard measurements that are greater than the configured maximum range. Instead, the ray is truncated at the maximum range to accurately model free space.

HeLiMOS

The HeLiMOS dataset provides the ground-truth labels for a single sequence from the HeLiPR dataset (Jung et al., 2024) captured simultaneously by four LiDARs: a Livox Avia, an Aeva Aeries II, a Velodyne VLP-16, and an Ouster OS2-128. The dataset is unique in highlighting the importance of sensor-agnostic MOS, with the purpose of providing consistent performance independent of the sensor used to generate the point cloud data. We achieve consistent performance across all sensors as demonstrated in Figure 6 and Table 5. The sensor pose is estimated using SiMpLE (Bhandari et al., 2024). 4DMOS and the proposed approach provide instantaneous detection of dynamic objects and use a small temporal history only, whereas MapMOS predicts all moving objects, even when they transition to being static. We outperform 4DMOS in all scenarios, except for the Aeva dataset with the delayed variant of 4DMOS.OPEN IN VIEWER

Figure 6.

We achieve consistent performance in the HeLiMOS (Lim et al., 2024) irrespective of the sensor used to record point cloud data. The figure shows the same instance captured by four different LiDARs. The proposed algorithm correctly labels dynamic objects (green), with minimal missed detections (orange), and false positives (red). The figure is best viewed in colour.

OPEN IN VIEWER

Table 5.

Evaluation on the HeLiMOS dataset with best results in bold (L: Livox Avia, A: Aeva Aeries II, O: Ouster OS2-128, V: Velodyne VLP-16). Results for other methods are as documented by Lim et al. (2024). The delayed results are separated at the bottom of the table. The ground truth uses type (II) labelling.

Method	Sequence IoU (%)
	L	A	O	V
4DMOS, online	52.1	54.0	64.2	4.7
MapMOS, scan	58.9	63.2	81.4	4.3
This paper, Δ = 0.20 m	50.7	68.0	73.0	35.8
This paper, Δ = 0.25 m	51.9	69.0	74.2	36.1
4DMOS, delayed	59.0	58.3	70.4	5.4
MapMOS, volume	62.7	66.6	82.9	5.8

We present only the generalized performance of alternative approaches without retraining them on the new dataset, as retraining would undermine our goal of evaluating algorithm robustness in unseen environments and new applications. An interesting study was conducted by the authors of the dataset, who trained 4DMOS and MapMOS with various combinations of the sensor data to examine the impact on performance. They acknowledge the significant improvements required for existing MOS methods to operate in a sensor-agnostic approach, providing further motivation for this work. Dynablox constructs a global TSDF map using the point cloud data and hence requires significant computational memory to be evaluated on the HeLiMOS dataset. We evaluate a fraction of the dataset and qualitatively analyze its performance compared to the proposed approach in Figure 7. Dynablox performs well with dense point cloud data in the Ouster sequence, but has missed and incomplete detections for the other sensors, which record lower-density point clouds. The sensor-specific configuration parameters were updated where appropriate, or else configured to provide the best result possible. The proposed approach provides better detection for all sensors.OPEN IN VIEWER

Figure 7.

A comparison between the performance of Dynablox and the proposed approach for all sequences using their default configurations. Dynablox performs well with high-density point clouds in the Ouster sequence. However, low-density scan patterns in the other sequences result in incomplete detections. The figure is best viewed in colour.

Examining the average precision and recall in Table 6 indicates the performance is precise, with the main limitation in capturing the entire object.

OPEN IN VIEWER

Table 6.

Precision and recall results of the proposed approach on the HeLiMOS dataset (L: Livox Avia, A: Aeva Aeries II, O: Ouster OS2-128, V: Velodyne VLP-16).

Metric (%)	Sequence
	L	A	O	V
Precision	95	95	90	94
Recall	53	72	81	37

It is important to mention that the ground-truth labels any dynamic objects that have moved throughout the sequence. This includes objects that transition from being dynamic to static and objects that transition from being static to dynamic. These situations occur in several scenes, drastically reducing the recall. The effect is detrimental to the Livox (L) results for all methods. The ground-truth labels are useful for map cleaning approaches (e.g. Removert, ERASOR), but they hinder the performance of all MOS approaches as the vehicles transition from stationary to moving. The proposed approach cannot predict if an object will move and capture these instances. Results on all LiDARs confidently detect moving objects only. Figure 8 depicts this scenario, and Table 7 quantifies the significant increase in the re-evaluated performance when scans 570–881 are correctly labelled. The Livox (L) and Aeva (A) sequences are significantly affected by the incorrectly labelled static vehicles due to their mounting locations and field of view. The Velodyne (V) sensor’s field of view is blocked at its mounting location, resulting in a minor performance change. While the Ouster (O) sequence is affected, the relative number of false positives in this sequence is outweighed by the true positive detections captured by its 360-degree FOV. For comparison, the Ouster sequence is labelled with approximately 8.4e6 dynamic measurements, whereas the Livox sequence has 1.8e6 dynamic measurements only.OPEN IN VIEWER

Figure 8.

A snapshot of the environment in scans 570-881 from the HeLiMOS dataset. Vehicles are labelled as dynamic but move in a future instance. This scenario incorrectly depicts the recall metrics of the Aeva, Livox, and Ouster sequences for all MOS methods. The Velodyne has a limited view of these vehicles due to its mounting position. The figure is best viewed in colour.

OPEN IN VIEWER

Table 7.

The re-evaluated performance by removing an instance of the incorrectly labelled dynamic detections between scans 570 to 881 for the HeLiMOS dataset (L: Livox Avia, A: Aeva Aeries II, O: Ouster OS2-128, V: Velodyne VLP-16).

Metric (%)	Sequence
	L	A	O	V
IoU (baseline)	51.9	69.0	74.2	36.1
IoU (re-evaluated)	72.1	80.3	77.7	39.0

The results for the Velodyne sensor (V) show the most significant performance discrepancy in comparison to the other LiDARs for all methods. The 16-beam scans provide challenging input, as it is difficult to discriminate between noisy detections and sparse measurements at long ranges. Figure 6 illustrates the drastic difference in scan density for all LiDARs. The proposed approach demonstrates consistent performance across all scenarios, independent of the sensor characteristics, as it uses information about free and occupied space from the point cloud data only. This is seen as an advantage of our proposed approach.

Table 8 displays a comparison of the average processing frame rate. The algorithm provides real-time results for all sensors except the Ouster due to its high-density scans. 4DMOS and MapMOS provide 10 Hz results for full-range scans but are GPU-based. The proposed algorithm uses CPU threading only.

OPEN IN VIEWER

Table 8.

The average processing rate for each sensor at varying maximum MOS ranges (L: Livox Avia, A: Aeva Aeries II, O: Ouster OS2-128, V: Velodyne VLP-16).

MOS range (m)	Frame rate (Hz)
	L	A	O	V
25	27	19	5	24
50	11	8	2	13

Apollo-Southbay

The Apollo-Southbay dataset by Lu et al. (2019) is commonly used to evaluate the generalization of learning-based approaches outside their training on the Semantic Kitti dataset. Though both datasets use Velodyne HDL-64 LiDARs, they are mounted on different vehicles at different heights relative to the ground. Table 9 displays the results of the proposed approach in comparison to existing methods benchmarked by Mersch et al. (2023). The proposed approach outperforms the online and delayed variants of 4DMOS and is second to MapMOS.

OPEN IN VIEWER

Table 9.

Evaluation on the Apollo-Southbay dataset with best results in bold. Results for other methods are as reported by Mersch et al. (2023). The delayed results are separated at the bottom of the table. The ground truth uses type (II) labelling.

Method	IoU validation (%)
LMNet	13.7
MotionSeg3D, v1	6.5
MotionSeg3D, v2	8.8
4DMOS, online	68.7
MapMOS, scan	79.2
This paper, Δ = 0.20 m	74.7
This paper, Δ = 0.25 m	75.8
4DMOS, delayed	70.9
MapMOS, volumetric	81.7

The proposed approach accurately identifies moving vehicles, cyclists, and pedestrians without any parameter tuning and identifies minimal false positives as displayed in Figure 9 (left). Like the HeLiMOS dataset, AutoMOS’s ground-truth labels (Chen et al., 2022) classify objects as dynamic if they moved at any point during the sequence. This classification approach significantly alters recall scores when vehicles stop moving.OPEN IN VIEWER

Figure 9.

The proposed approach accurately identifies objects of various sizes travelling at different speeds indicated by the green measurements. False positives shown in red arise at the boundary of the static and dynamic points, such as where the wheels touch the ground. The scan on the right shows a vehicle labelled as dynamic by the ground truth that transitioned from being dynamic to static. These instances incorrectly portray algorithm performance. The figure is best viewed in colour.

There is a sequence of 150 scans in Apollo sequence 00 where a vehicle transitions from being dynamic to static behind the instrumented vehicle. The ground truth continues to label these as measurements corresponding to a dynamic object. Figure 9 (right) shows an example of this situation. Correcting the ground truth by ignoring these measurements results in a 17% increase in recall. Furthermore, removing all such instances between scans 180–530 results in a 26% increase in recall. This represents a significant discrepancy when benchmarking results are compared to other methods that continue to label these measurements as dynamic.

We firmly maintain that a moving object should be defined as one that is actively in motion during the current scan, not one that moved previously or might move later. Our approach adheres to this definition by correctly identifying temporarily stationary vehicles as static, while the ground truth continues to label them as dynamic based on their movement history.

The algorithm processes the Velodyne HDL-64 scans at 10 Hz for a 25 m MOS range, and at 5.5 Hz for a 50 m range.

Waymo, KITTI, nuScenes, Avia

M-detector by Wu et al. (2024) benchmarks performance on three open datasets; KITTI (Geiger et al., 2012), Waymo (Sun et al., 2020), and nuScenes (Fong et al., 2022). Wu et al. (2024) provide the ground truth pointwise labels for these datasets. The KITTI dataset is labelled using the bounding boxes provided in the object-tracking sequences. The velocity of the bounding boxes is estimated using consecutive frames, with all measurements in bounding boxes having a velocity greater than a threshold, labelled as dynamic, 0.5 m/s for pedestrians and 1.0 m/s for vehicles. The authors of this paper label any dynamic detections as moving. In comparison to the other experiments in this paper, this introduces a different evaluation metric. A similar approach is used for the Waymo and nuScenes datasets. The method proposed in our work captures any moving objects, irrespective of their speed. This introduces an inaccuracy when benchmarking results. It must be noted that the KITTI ground-truth labels only capture moving objects within a 120-degree field of view due to the limited object-tracking bounding boxes. Any motion behind the sensor or on the sides is not captured. Wu et al. (2024) additionally record and label a unique indoor dataset capturing small flying objects with a Livox Avia. This dataset challenges learning-based approaches as the scan pattern, object shape, and dynamics are very different from the training data.

Figure 10 illustrates sample detections from the various datasets. The proposed approach successfully captures dynamic objects in all tests. The nuScenes dataset captures a large number of points on the vehicle itself. The sequence of Avia scans shows a small, fast-moving object correctly captured by the proposed algorithm. The example for the Waymo dataset illustrates an instance of the minimum velocity threshold, labelling the vehicle as slowing down as a static object ( < 1  m/s). Relabelling scans 28 to 97 in sequence 16 results in a 7% increase in precision and a 4% increase in the sequence’s IoU. This is a significant increase, given the minor differences in the benchmarking comparison. Furthermore, evaluating MapMOS on the same sequence results in a 13% increase in precision when using its definition of a moving point.OPEN IN VIEWER

Figure 10.

Example results on the datasets benchmarked by M-detector. The Waymo dataset uses a 64-beam custom LiDAR. The proposed approach correctly identifies moving objects. False positives occur (i) at the boundary of static and dynamic points, and (ii) due to the minimum velocity threshold on the dynamic objects. The nuScenes dataset uses a 32-beam LiDAR. A large density of points is captured on the vehicle itself, biasing the results. The Avia dataset tests the capability to capture small and fast-moving objects, providing a unique experimental study. The figure is best viewed in colour.

Table 10 displays the performance of the proposed approach on the datasets and in comparison to M-detector. The proposed algorithm performs consistently in all datasets, performing on par with M-detector and outperforming the learning-based approaches. We acknowledge the difference in the ground-truth labelling process with the minimum velocity thresholds. The proposed algorithm uses the same configuration for all sensors in all experiments used in the other datasets. M-detector configures a set of 14 parameters for each sensor, with different parameters for the depth map generation, occlusion check, event detection, and map consistency. The finer tuning helps achieve better results.

OPEN IN VIEWER

Table 10.

Evaluation on the M-detector datasets with best results in bold. Results for other methods are as documented by Wu et al. (2024). The ground truth uses type (I) labelling for the Avia dataset and type (III) labelling for the other datasets.

Method	Sequence IoU (%)
	Avia	KITTI	nuScenes	Waymo
LMNet-1	0.7	55.4	11.1	4.2
LMNet-8	0.2	63.5	3.3	2.5
SMOS	15.1	20.5	8.2	4.3
M-detector (point-out)	62.9	57.0	85.5	64.0
M-detector (frame-out)	91.6	74.6	90.2	74.6
This paper, Δ = 0.20 m	94.0	64.4	88.0	68.3
This paper, Δ = 0.25 m	90.2	64.2	89.1	70.5

Table 11 shows the average processing rates for the proposed algorithm at varying MOS ranges, compared to the results published by M-detector. A similar trend is observed with previous datasets where low-density scanners are processed significantly faster. The Avia dataset is recorded indoors.

OPEN IN VIEWER

Table 11.

The algorithm’s processing rate for varying MOS ranges for the proposed algorithm (top) and the results reported by M-detector (bottom).

MOS range (m)	Frame rate (Hz)
	Avia	KITTI	nuScenes	Waymo
25	35	10	14	9
50	28	5	8	4
Full-range	13	4	6	3
Full-range	62	11	30	12

DOALS

The DOALS dataset by Pfreundschuh et al. (2021) consists of eight sequences at four locations recorded using a handheld OS1-64 LiDAR at 10 Hz. The sequences are recorded in indoor and outdoor human-centric environments. We use SiMpLE (Bhandari et al., 2024) to estimate the sensor pose accurately and use the deskewed point clouds. Deskewing plays a significant role in correcting the jittery LiDAR motion from the handheld sensor. Schmid et al. (2023) label 10 scans per sequence that do not specifically discriminate between moving and non-moving objects, with all pedestrian-looking points labelled as dynamic. This eliminates the ambiguity in the different definitions of moving points, but also means a perfect IoU cannot be achieved. In line with the evaluation process by Schmid et al. (2023), the IoU is computed for all 10 labelled scans per sequence, with the mean value displayed in Table 12. Table 13 shows the mean processing frame rate for all sequences. The Station sequence (ST) has the slowest processing time due to its large open recording environment, which requires updating a significant number of free voxels.

OPEN IN VIEWER

Table 12.

Evaluation on the DOALS dataset with best results in bold (ST: Station, SV: Shopville, HG: Hauptgebaeude, ND: Niederdorf). Results for other methods are as documented by Schmid et al. (2023). The lower half of the table shows results for a 20 m MOS detection range. The ground truth uses type (IV) labelling.

Method	Sequence IoU (%)
	ST	SV	HG	ND
DOALS-3DMiniNet	84.0	82.0	82.0	80.0
4DMOS	38.8	50.6	71.1	40.2
LMNet (original)	6.0	7.5	4.6	3.0
LMNet (refit)	19.9	18.9	27.4	40.1
Dynablox	86.2	83.2	84.1	81.6
This paper, Δ = 0.20 m	81.9	81.6	85.6	80.9
This paper, Δ = 0.25 m	81.7	81.1	82.2	81.1
LC free space (20 m)	48.7	31.9	24.7	17.7
ST normals (20 m)	80.0	81.0	85.0	76.0
Dynablox (20 m)	87.3	87.8	86.0	83.1
This paper, Δ = 0.20 m (20 m)	88.7	84.0	87.3	82.9
This paper, Δ = 0.25 m (20 m)	86.9	84.1	83.6	82.4

OPEN IN VIEWER

Table 13.

The algorithm’s processing rate for a 20 m MOS range in comparison to Dynablox.

Method	Frame rate (Hz)
	ST	SV	HG	ND
Dynablox	17	17	17	17
This paper, Δ = 0.25 m	6	12	12	10

Results are provided for full-range (172 m) and short-range (20 m) test cases. We outperform existing learning-based approaches such as LMNet and 4DMOS and are on par with Dynablox for the full-range tests. The learning-based approaches fail to generalize performance as indicated by the low IoU. We outperform all approaches on the short-range test with similar results to Dynablox again. We achieve a mean precision of 96% and recall of 90% for all evaluated scans. The high precision indicates the low false positives, whereas the slightly lower recall highlights the challenge of capturing the entire object, but is also affected by the labelling process.

Sipailou campus

The Sipailou campus dataset by Zhou et al. (2023) consists of eight sequences recorded by a Livox Avia LiDAR mounted to an unmanned ground vehicle traversing the Southeast University Sipailou Campus. The LiDAR has a non-repetitive scanning pattern and a different field of view, providing a unique dataset to test the generalization capabilities of MOS methods. The provided sensor poses are used. Using the same configuration as other tests, we outperform all approaches as displayed in Table 14.

OPEN IN VIEWER

Table 14.

Evaluation on the Sipailou Campus dataset with best results in bold. Results for other methods are as reported by Zhou et al. (2023). The ground truth uses type (I) labelling.

Method	IoU (%)
	Validation	Test
LMNet	5.4	6.9
MotionSeg3D	6.8	6.7
4DMOS	78.5	82.3
Motion-BEV	50.4	52.0
Motion-BEV-h	70.9	71.5
This paper, Δ = 0.25 m	84.9	86.0
This paper, Δ = 0.20 m	84.8	86.1

The results for the other methods are used directly from Zhou et al. (2023) and only consider the evaluation without retraining the networks on the new data to provide a fair test of generalized performance. The results are evaluated using the Semantic KITTI API (Behley et al., 2019) to be consistent with the original evaluation process and benchmark results. Learning-based approaches such as LMNet, MotionSeg3D, and MotionBEV fail to generalize due to differences in the sensor’s characteristics. 4DMOS demonstrates strong generalization with its unique design of learning changes in sequential data, being model-independent. The proposed approach cannot capture sparse measurements at longer ranges corresponding to dynamic objects, as they are difficult to differentiate from noisy estimates. The results are processed with an average frame rate of 10 Hz for a maximum 50 m MOS range.

Case study: Detecting moving objects in an excavator’s workspace

This Section investigates the use of MOS in detecting dynamic objects within an excavator’s workspace using onboard LiDAR. Excavator operators have limited visibility from the cab, and their proximity to other machines can result in collisions, leading to operator fatalities, equipment damage, and significant operational downtime. Sensors such as LiDAR enable imaging of the workspace’s immediate environment, and extending the discussed MOS algorithms to this scenario provides the ability to identify the presence of dynamic objects in the agent’s workspace.

This case study demonstrates the performance of the proposed algorithm in detecting the presence of dynamic objects within the excavator’s workspace. The sensor data is from a field study where two Velodyne HDL-64 LiDARs were mounted on the back of an excavator. Two scenarios from the dataset are presented: (i) identifying the presence of a light vehicle that enters the work area during a shift change, and (ii) the changes in the workspace during regular operation as haul trucks come and go.

Figure 11(a) shows the field environment, displaying the instrumented excavator and haul trucks moving in its workspace. Detection results for the left sensor are shown as the excavator is loading trucks to its left only.OPEN IN VIEWER

The performance of the proposed algorithm is compared to 4DMOS (Mersch et al., 2022) and MapMOS (Mersch et al., 2023), both of which were trained on the Semantic KITTI dataset (Behley et al., 2019), and Dynablox (Schmid et al., 2023). The Semantic KITTI dataset and this case study both use the Velodyne HDL-64 sensor. However, the sensor’s motion profile, scanning frequency, mounting height, and mounting orientation are drastically different in both datasets. Figure 11(b) displays examples of scans from the Semantic KITTI dataset (left) and the excavator (right). The scan from the Semantic KITTI dataset provides high coverage of the workspace at all times. In contrast, the scan from the excavator is unstructured and has a limited view of different areas in the workspace as the machine slews on its swing axis. The proposed algorithm continues to perform well; 4DMOMS and MapMOS are less effective. Dynablox also performs well, with the high-density scans matching the sensor characteristics from its published evaluation.

All MOS approaches use the same inputs for fair testing, with Dynablox, 4DMOS, and MapMOS configured using their default parameters. The sensor’s pose is estimated using the point cloud data. All detections are performed in the sensor’s frame, allowing for testing with the open-source repositories.

The first scenario assesses the ability of the MOS approaches to identify a light vehicle and its operator within the excavator’s workspace during a shift change. Figure 12 displays the dynamic detection results for the MOS approaches on a sequence of scans as a light vehicle enters the excavator’s workspace. MapMOS and the proposed algorithm yield nearly identical results for all scans, completely capturing the moving vehicle. Dynablox performs well, only missing a small fraction of the vehicle’s bonnet in the last frame. 4DMOS provides a late partial detection, possibly due to the receding horizon used to fuse beliefs about dynamic measurements. This scenario provided a simple scenario with a well-observed vehicle, being similar to the dynamic detections in the Semantic KITTI dataset.OPEN IN VIEWER

The second scenario involves a truck maneuvering into the loading position. Figure 13 displays the performance of the MOS approaches for a sequence of scans during the scenario. 4DMOS detects the moving truck in most frames, but generates numerous false positives on the terrain, failing to identify the truck accurately. MapMOS captures the truck in the first frame, provides a partial detection for the second frame, but fails to capture any detections thereafter. The proposed algorithm and Dynablox provide consistent detection across all frames, regardless of the truck’s observability.OPEN IN VIEWER

Both scenarios provide unique scenarios for examining the performance of the MOS approaches. MapMOS performs well without being retrained, but is unable to capture dynamic measurements in partially observable conditions. Its strength lies in using the probabilistic backend to keep a history of the occupied and free space. 4DMOS fails as the scene’s observability changes throughout its receding horizon, from which the dynamic detections are estimated. Dynablox performs well with its default configuration with only minor missed detections. The dense point cloud data closely matches the DOALS dataset used to originally evaluate Dynablox’s performance. The proposed algorithm provides consistent performance in both scenarios.

Computation

Figure 14(a) displays the mean execution time per scan at varying measurement ranges for different sequences. The proposed algorithm provides real-time results for dynamic object detection within a 20 m sensor range, with computation time increasing at larger ranges. The computation time is bi-proportional to the point cloud’s size and discretization resolution. Using a smaller voxel size increases accuracy, but consequently leads to a larger computation time due to the larger number of voxels that need to be updated. The Sipailou campus dataset and HeLiMOS Velodyne (V) are the fastest due to their small point cloud size in comparison to the high-density Ouster (O) scanners. It is important to note that the 20 m range constraint is applied to the measurement’s ray to allow for accurate modelling of free space. Consequently, this means that modelling large open spaces as in DOALS Station (ST) sequences leads to a slightly longer execution time of 148 ms/scan, compared to the Shopville (SV) sequences with an execution time of 60 ms/scan.OPEN IN VIEWER

A breakdown of the computational expense of each stage is provided in Figure 14(b). A pattern is observed with the map update and dynamic detection modules being most computationally expensive. Both of these modules involve iterating over all observed voxels. The DOALS ST sequence has a greater computation time for integrating a new observation due to the raycasting operation in a large open space. Implementation of the proposed algorithm relies strongly on CPU threading to provide timely results.

The purpose of the MOS pipeline’s stages

The proposed algorithm consists of three main stages summarized in Figure 2: integrating a new observation, updating the local map, and detecting dynamic points. In its most straightforward implementation, identifying the changes in the voxel state is sufficient in labelling dynamic points. However, this is inadequate in capturing the entire dynamic object, as articulated objects, such as humans, are not completely dynamic at every instant. Table 15 displays the effect of the algorithm stages on the MOS IoU, precision, and recall for a test sequence from the DOALS dataset. Using a spatial convolution only, the algorithm correctly identifies dynamic objects as highlighted by the high precision but fails to capture the entire object, reflected in the low recall. Adding a temporal aspect significantly increases the recall, leading to a higher IoU. The dilation stage recovers voxels that were disregarded due to the strong constraints on identifying dynamic voxels, leading to a higher recall and consequent increase in IoU. The final preservation stage increases the IoU while slightly decreasing the precision, as false positive detections may be preserved.

OPEN IN VIEWER

Table 15.

The effect of different algorithm stages on the MOS IoU, precision, and recall for the DOALS Hauptgebaeude dataset. The dynamic detection modules increase recall by 56.1% at a 1.8% loss in precision.

Module	Metric (%)
	IoU	Precision	Recall
Change detection (3D)	34.0	99.8	34.0
Change detection (4D)	70.9	99.7	71.0
With dilation	86.9	99.0	87.6
With preservation	88.5	98.0	90.1

Visualizing different algorithm configurations

Figure 15 visualizes the affect of different configuration parameters on the algorithm’s performance. The first sequence of the DOALS Niederdorf sequence is used for the comparison, with the first column showing the performance of the algorithm using default parameters. The results are analyzed below.OPEN IN VIEWER

Configuration sensitivity

We take the view that an algorithm needs to be insensitive to its configuration parameters if it is to be robust. This section illustrates the effect of varying the algorithm’s configuration parameters on the performance IoU, precision, and recall.

Figure 16 shows the dependency between the lumped uncertainty, σ _o , the state transition probability threshold, p_min, and the local window size, w _l . As discussed previously, the lumped uncertainty, σ _o , corresponds to the uncertainty in observations being integrated into the local map. A very low uncertainty (e.g. σ _o = 0.1 m) corresponds to trusting the observations directly. Consequently, the results indicate a very high recall at the cost of identifying numerous false positives, as indicated by the low precision. On the other hand, a very high uncertainty (e.g. σ _o = 0.5 m) leads to dynamic objects being missed, indicated by the low recall. In this case, there is high precision as sufficient observations are required to support state transitions. An anomaly is the w _l = 10, which displays a very low precision for high uncertainties. This is attributed to accumulating state changes occurring over a longer time compared to the other window lengths and using a minimum Otsu threshold of γ_min = 3 for all test cases. The local window plays a significant role in smoothing false positives, with a higher mean precision in the results using a window size of five compared to one.OPEN IN VIEWER

Figure 17 shows the dependency between the size of the convolution kernel, m, the minimum Otsu threshold, γ_min, and the voxel size, Δ. A smaller voxel size allows for finer classification and segmentation of measurements corresponding to dynamic objects. The heatmaps illustrate degraded performance with a large voxel size. Only a single configuration parameter is varied at a time, which contributes to poor performance, as these parameters are interdependent. That is, all three parameters represent physical space. A large voxel size, combined with a large convolution kernel, results in the analysis of a larger area. This explains the poor performance with a voxel size of v = 0.5 and convolution kernel sizes of m = 7 and m = 9. The minimum Otsu threshold filters noisy dynamic detections in static scans. A large value such as γ_min = 4 is shown to reduce performance for a range of voxel sizes. The smallest voxel size continues to perform well, as the dynamic object will occupy more voxels and hence be less affected by the lower threshold.OPEN IN VIEWER

Selecting configuration parameters

The discussion of the proposed MOS pipeline’s stages, visualization of the different algorithm configurations, and analysis of their sensitivity reveal several insights into the effect on the algorithm’s performance. This Section uses these insights to outline an adaptive strategy for selecting the algorithm’s configuration parameters.

Table 1 outlines all algorithm configuration parameters and their purpose. Table 16 provides a guide for their selection, clearly highlighting interdependencies and their recommended values. The default values for the State Update and Local Map are suitable for a range of applications and do not significantly impact MOS performance. The default values provide insight into the relationship between the parameters, with the entries in Table 16 offering a clear guide for their selection. The default parameters provide the desired performance. However, changing the voxel size requires adaptation for all values in the dynamic detection module. Future work involves automatically determining these values online.

OPEN IN VIEWER

Table 16.

A guide for selecting the algorithm’s configuration parameters.

	Parameter	Parameter configuration and selection	Recommendation
State	Transition probability, ϵ	Shown to be insensitive. Both parameters are core to the HMM filter update	It is recommended to use the default parameters
	Transition threshold, pmin
Local Map	Local map radius, rmax	These parameters control the size of the local map used for dynamic detection. The local map radius defines the MOS detection range	It is recommended to use the default global window size, with the map radius configured depending on the desired MOS detection range
	Global window, w g
	Voxel size, Δ	The voxel resolution to discretize space for occupancy representation. A smaller voxel size allows for a detailed representation environment, but at the cost of an increased runtime per frame. The voxel size directly influences the configuration of subsequent dynamic detection parameters	A voxel size of 0.2 m or 0.25 m provide an adequate trade-off between accuracy and speed
Dynamic Detection	Lumped uncertainty, σ o	This value reflects the combined uncertainty in the point cloud measurements and pose estimation accuracy. This value is discretized at the voxel’s resolution	It is recommended to analyze the quality of the sensor’s measurements and the pose estimations to estimate the value of the lumped uncertainty. A dynamic per-scan uncertainty from the sensor’s pose estimate can be used if available
	Convolution kernel size, m	The kernel size used to calculate the convolution score. A small kernel is unable to capture spatial state changes, whereas a very large kernel is unable to capture small detections. The kernel represents a local 3D spatial neighbourhood of voxels over which state changes are accumulated	An intermediate choice of m = 5 is recommended for a voxel size of Δ = 0.2 m (5 × 0.2 = 1 m kernel edge). It is recommended to decrease the kernel size for larger voxel sizes, and vice versa. For example, a voxel size of Δ = 0.1 performs best with m = 7 (7 × 0.1 = 0.7 m kernel edge), whereas a voxel size of Δ = 0.5 m performs best with m = 3 (3 × 0.5 = 1.5 m kernel edge). The kernel size must be an odd integer
	Local window, w l	The convolution kernel is extended over a local window size to compute the 4D convolution. A small window may miss slow or intermittent motion, whereas a large window provides temporal smoothing but may suppress small detections	An intermediate value of three to five scans (0.3–0.5 s at 10 Hz scanning frequency) provides complete detection for most cases. Large windows such as 10 scans, leads to the smoothing of small changes
	Minimum Otsu threshold, γmin	The minimum Otsu threshold is applied to the 4D convolution score: the convolution kernel applied over the local window. This is only introduced for handling noisy detections arising in static scans. The minimum Otsu threshold is physically representing the minimum number of dynamic voxels detected within the kernel across the local window	It is recommended to set the minimum threshold lower than the kernel size to ensure small objects are captured, even if they are only dynamic for portion of the local window

Each of the algorithm’s configuration parameters has a physical meaning. Ill-defined parameters lead to poor performance. The algorithm configuration used for the benchmark tests was tuned using the first Hauptgebaeude sequence from the DOALS dataset during early testing. This allowed the authors to gain an understanding of the parameters and verify beliefs about their dependencies. Upon benchmarking, the configuration was tested with other datasets, resulting in the benchmarked results. We do not see value in optimizing over all nine datasets, as the aim is to design an algorithm that is insensitive to small changes in the configuration and has a meaningful parameter set adaptable for a variety of applications. We invest in describing these relationships to allow for easy and correct adaptation of the algorithm to different applications.

Failure modes

Figure 18 illustrates the failure modes of the proposed approach. Namely, (i) labelling objects moving into unobserved space, (ii) labelling sparse measurements corresponding to dynamic objects, (iii) capturing the complete object, for example, long vehicles such as buses, and (iv) handling large errors in the sensor’s estimated pose.OPEN IN VIEWER

The top left figure shows the failure in identifying dynamic objects moving into unobserved space (see equation (8)). This occurs due to the strict constraints enforced during the 4D convolution, where voxels cannot have unobserved neighbours. The constraint is important as raycasting is known to be inaccurate around the boundary of observed and unobserved space. Other approaches commonly introduce a delay in the detection, for example, 10 scans, to provide better detection around these boundaries.

The top right figure shows a scenario where dynamic labels are missed for sparse measurements. It is challenging to differentiate these measurements from noisy dynamic detections. Consequently, they are rejected in the 4D convolution when using the automatic Otsu thresholding. This failure occurs due to the sensor’s sparse scanning density at longer ranges.

The bottom left corner shows the partial detection of a bus. From a voxel’s perspective, it remains occupied for the entire length of the bus, during which its state does not change. Hence, it identifies the voxel to be static. This failure can be mitigated by introducing the concept of an object rather than examining it at a voxel level only.

The bottom right image shows several false detections occurring due to a significant error in the sensor’s pose. The algorithm depends on an accurate sensor pose locally. That is, large errors or jumps in pose result in incorrect detections, whereas pose drift outside the global window does not impact MOS performance. The probabilistic state update is capable of handling noisy measurements and sensor pose estimates, but large errors cannot be detected in the current implementation.