Modeling Human Steering Behavior in Teleoperation of Unmanned Ground Vehicles With Varying Speed

Abstract

Objective

This paper extends a prior human operator model to capture human steering performance in the teleoperation of unmanned ground vehicles (UGVs) in path-following scenarios with varying speed.

Background

A prior study presented a human operator model to predict human steering performance in the teleoperation of a passenger-sized UGV at constant speeds. To enable applications to varying speed scenarios, the model needs to be extended to incorporate speed control and be able to predict human performance under the effect of accelerations/decelerations and various time delays induced by the teleoperation setting. A strategy is also needed to parameterize the model without human subject data for a truly predictive capability.

Method

This paper adopts the ACT-R cognitive architecture and two-point steering model used in the previous work, and extends the model by incorporating a far-point speed control model to allow for varying speed. A parameterization strategy is proposed to find a robust set of parameters for each time delay to maximize steering performance. Human subject experiments are conducted to validate the model.

Results

Results show that the parameterized model can predict both the trend of average lane keeping error and its lowest value for human subjects under different time delays.

Conclusions

The proposed model successfully extends the prior computational model to predict human steering behavior in a teleoperated UGV with varying speed.

Application

This computational model can be used to substitute for human operators in the process of development and testing of teleoperated UGV technologies and allows fully simulation-based development and studies.

Keywords

human performance modeling teleoperation driver behavior ACT-R cognitive architecture computational modeling

Introduction

Unmanned ground vehicles (UGVs) have drawn research interest in various applications, especially in hazardous environments where it is dangerous or even impossible for human operators to be on board (Lichiardopol, 2007; Liu et al., 2017; Nguyen-Huu et al., 2009).

Most UGVs deployed in the field are teleoperated, that is, operated remotely by a human operator to control all the UGV actions and complete desired tasks. One major challenge with teleoperation is the communication time delay introduced in both the control and sensing paths. The time delay in sending the desired commands to the vehicle can lead to undesired unresponsiveness. Moreover, the time delay in the visual and sensory feedback from the UGV to the human operator can cause reduced situational awareness. Research has shown that these challenges reduce the capability of the human operator to complete tasks and missions in the teleoperative setting, and thus reduce the effectiveness of the UGV platform (Zheng et al., 2016, 2018). Understanding how time delay affects the human operator can help the development of UGV technologies to overcome this challenge.

Currently, studies on teleoperated UGV technologies are typically conducted with human-in-the-loop experiments (Vozar et al., 2018). Such experiments, however, are expensive and time consuming. If a computational human operator model were available, a fully simulation-based development approach would become feasible and allow for fast and easy evaluation of a large set of teleoperation frameworks, designs, and UGV prototypes. Such evaluations could even be moved to earlier stages of the development process, where human-in-the-loop experiments are not yet feasible. In addition, a human operator model could also be used to develop predictive interfaces to reduce the negative impact of time delays in teleoperated UGVs (Zheng et al., 2016, 2018).

For vehicles with a driver on board, the benefits of computational driver models have already been recognized and pursued through extensive research. The resulting driver models are used in place of human drivers to simulate human driver behavior and performance in various driving tasks, with methods of modeling varying significantly in terms of focus and application. Examples of these models are the two-level driver model (Donges, 1978; Salvucci & Gray, 2004), model predictive control (Guo et al., 2013), near collision control (Markkula et al., 2012), race car models (Braghin et al., 2008), multiloop methods (van der El et al., 2019), and cognitive driver models (Dehban et al., 2015; Deng et al., 2019; Liu et al., 2006; Salvucci, 2006). The availability of such computational driver models helps researchers develop a better understanding of driver behavior and accelerates the development of various driving technologies.

Compared to the onboard driving scenarios, driver modeling in teleoperation settings has been understudied. Among the few reported efforts, almost all of them focus on small ground robots (Chen et al., 2007; Ritter et al., 2006, 2007; Vozar, 2013; Vozar et al., 2018). However, the dynamics of a small ground robot and its user interface can be dramatically different compared to passenger-sized UGVs. To the best of the authors’ knowledge, the only teleoperator model for a passenger-sized UGV is the work of Mirinejad et al. (2018), where the authors applied a cognitive framework, namely the Adaptive Control of Thought-Rational (ACT-R), to model the human steering behavior for a passenger-sized UGV that operates under a constant speed. Their results demonstrate the feasibility of modeling operator behavior in the cognitive framework while capturing the effect of time delay and nonnegligible vehicle dynamics. The assumption that the UGV travels at a constant speed, however, is limiting. Specifically, in a constant speed scenario, the curvature of the turns has to be limited within a specific range to allow safe steering. In addition, the tuning method in a constant speed scenario only finds an appropriate set of parameters for a set speed, while in varying speed scenarios, the tuning needs to account for all possible turning speeds at various curvatures at the same time. Moreover, the effect of accelerating and decelerating on steering is not present when the UGV travels at a constant speed. Thus, the results in a constant speed setting cannot be directly applied in a varying speed setting. Therefore, there is a need to extend the current models to capture teleoperator behavior on a UGV with varying speed.

In this work, we aim to address this need and extend the model developed by Mirinejad et al. (2018) to varying speed scenarios. In addition to the two-point steering model in the ACT-R architecture used in Mirinejad et al. (2018), this paper introduces a second far point for speed control to allow the model to operate the UGV at varying speeds. Temporal constraints are introduced for proper coupling of the two far points for the speed and steering control models. For the purpose of this study, the computational model is developed for the path following scenario, where average lane keeping error (ALKE) is the performance metric used to evaluate the steering behavior. Because teleoperation is significantly different than onboard driving, human teleoperators are typically well trained and experienced in remotely operating a vehicle. Hence, in this study we are particularly interested in modeling the lowest human ALKE. A tuning method is therefore developed to find a parameterization that minimizes the ALKE and is robust to small perturbations in the parameters.

To evaluate and validate the computational driver model, the simulated teleoperation platform developed by Zheng et al. (2016) was used as a real-time simulation platform to collect human subject data. Human-in-the-loop experiments were conducted with 31 subjects and the collected data were compared with the performance data from the computational driver model. Statistical tests were performed to validate the equivalence of the computational model and human performance in terms of the lowest ALKE.

Driver model with steering and speed control

The driver model in this work consists of a two-point visual model for steering and a single far-point model for speed control tasks. The two-point steering model was first introduced by Salvucci and Gray (2004) and Salvucci (2006), and was later adopted into the teleoperated driving domain by Mirinejad et al. (2018) in the teleoperated steering model for constant-speed path following tasks. In the present work, a second far point is added for speed control tasks. The two models are connected and regulated through temporal and logical constraints, and a tuning method is developed to represent the lowest human ALKE in the teleoperated driving setting with varying speeds.

Steering and Speed Control Models

It has been demonstrated that the two-point visual model, a two-level control model defined based on the near and far visual regions of the roadway, can successfully describe human steering behaviors (Donges, 1978; Land & Horwood, 1995; Salvucci & Gray, 2004). In this work, the visual point at the far region, far point, is placed at the center of the upcoming roadway at a distance that is determined by the vehicle speed and a control parameter called steering time headway ( ${thw}_{st}$ ). The visual point at the near region, near point, is placed at the center of the roadway in front of the vehicle. The far point is used to predict steering control based on the curvature of the upcoming roadway to maintain lateral stability of the vehicle. The near point is used to correct lane keeping error. The control law for the two-point steering model is described as

Δ ϕ = k_{f} Δ θ_{f} + k_{n} Δ θ_{n} + k_{I} θ_{n} Δ t

(1)

where $ϕ$ is the steering angle, and $θ_{f}$ and $θ_{n}$ are the visual directions to the far point and near point. $k_{f}$ , $k_{n}$ , and $k_{I}$ are control gains that can be used to tune the control model.

In addition to modeling human steering behavior, modeling human speed control behavior is also important to capture for a UGV with varying speed. A second far point is therefore introduced in the control model, where this far point is defined similarly to the far point in the two-point steering model. The time headway for the speed control model, however, is different from that of the two-point steering model. Thus, the time headway for the speed control model is denoted as ${thw}_{sp}$ . The control law for the speed control is defined as the following:

\begin{matrix} u_{v} = 1 \\ i f u p c o m i n g r o a d w a y i s s t r a i g h t a n d Δ θ < ϵ \end{matrix}

(2)

\begin{matrix} u_{v} = 0 \\ i f u p c o m i n g r o a d w a y i s s t r a i g h t a n d Δ θ > ϵ \end{matrix}

(3)

\begin{matrix} u_{v} = k_{v} (v_{r e f} - v) \\ i f u p c o m i n g r o a d w a y i s c u r v e d \end{matrix}

(4)

where $u_{v}$ is the accelerator or brake input with a range of [−1,1]; $k_{v}$ , the proportional control gain of the speed controller; $v_{r e f}$ , the target speed; and $v$ , the current speed. We observed in our pilot studies that as time delay increases, human operators become less confident in the steering performance and the stability of the vehicle, and as a result accelerate only when the steering control is nearly stable. Therefore, in the coupled model of both steering and speed control, a logical constraint is introduced to ensure that acceleration only occurs when steering angle $Δ θ$ is within a threshold $ϵ$ as shown in Eqn. (2) and Eqn. (3). The target speed in Eqn. (4) is calculated using the following equation:

v_{r e f} = \sqrt{\frac{4 F_{t}}{m_{v} c_{r}}}

(5)

where $c_{r}$ is the curvature of the upcoming curve calculated using a three-point curvature approximation; $F_{t}$ , the maximum tire force for each tire; and $m_{v}$ , the mass of the vehicle. This formula gives an estimate of the maximum vehicle speed for the given road curvature. The maximum tolerable speed is used as the reference, because in this work the scenario of interest is such that the driving task is to be completed as quickly as possible. For far points that are placed at straight segments of the roadway, the speed controller will simply accelerate at maximum acceleration with full throttle, again with the motivation to minimize task completion time. Hence, $u_{v}$ is set to 1 in Eqn. (2).

Unlike steering performance, which is bounded by the skill level of a human operator and easily evaluated using the ALKE metric, speed performance is subjective and difficult to regulate. In our experiments, a minimum average speed is introduced as a hard constraint to regulate speed performance for both the computational driver model and human operators based on the time delay. $ϵ$ in Eqn (2) and (3) is then manually tuned based on such speed constraints such that the average speed of the vehicle for the human model is marginally above the speed constraint.

Steering and Speed Control Models in the ACT-R Architecture

The ACT-R architecture is a cognitive framework developed by Anderson et al. (2004). A cognitive framework is a computational framework to model human cognitive behaviors (Aasman, 1995; Anderson & Lebiere, 1998; Anderson et al., 2004; Laird, 2012). In this work, the ACT-R rules used by Mirinejad et al. (2018) are extended to incorporate speed control ability for the operator model.

The production rules are shown in Table 1. First, the model checks the far point for speed control and updates the speed control command Eqns. (2)–(4). Then, the model checks the near point for steering control. Finally, the model checks the far point for steering control and updates the steering control command Eqn. (1). Because steering control requires a previous steering command to issue a new steering command, the rule “process-without-old” is needed for the first several production cycles when there is no previous steering command. Rules “loop-stable,” “loop-unstable,” and “reset” check the stability of the loop and restart the process if needed. The ACT-R model uses four declarative memory chunks. One complete control cycle requires the firing of five production rules.

Table 1

ACT-R Production Rules for Steering and Speed Control Update for a UGV in the Path-Following Task

attend-far_speed

process-far_speed-update-speed

attend-near_steering

process-near_steering-attend-far_steering

process-far_steering-update-steering

process-without-old

loop-stable

loop-unstable

reset

Note. ACT-R= adaptive control of thought-rational; UGV = unmanned ground vehicle.

The ACT-R model is connected to the vehicle simulator using a MATLAB interface. The vehicle simulator in Simulink acts as an external simulator and passes the vehicle state updates to the ACT-R model. The calculated control inputs from ACT-R are fed back to MATLAB to close the control loop.

Model Tuning

Previous studies have shown that human operation behavior varies under different driving conditions for teleoperation, such as different communication delays and vehicle speeds (Mirinejad et al., 2018). Similarly, a set of control gains that is able to describe the human operator behavior under a certain teleoperation condition may not capture the human operator behavior under a different condition. Therefore, a scenario-dependent tuning method is needed to correctly portray the human operation behavior for each operation condition.

As shown by Mirinejad et al. (2018), the optimal model parameters vary as vehicle speed changes. Parameters that work well at low turning speed could become too aggressive at higher turning speeds and are susceptible to small variations in the control parameters. Therefore it is important to ensure that the tuned set of parameters are robust to perturbations.

In Eqn. (1), three control parameters are given. However, our pilot experiments have shown that variations of $k_{n}$ and $k_{I}$ have negligible impacts on the overall steering performance when the average speed of the vehicle is reasonably high. Hence, in this study, a nominal value of $k_{n}$ and $k_{I}$ is selected without a tuning process. Therefore, only $k_{f}$ needs to be tuned for steering control. As a result, a total of four parameters need tuning for the model: steering and speed control gains $k_{f}$ and $k_{v}$ , along with time headway for both steering and speed control models, ${thw}_{st}$ and ${thw}_{sp}$ .

In light of the aforementioned fact that human teleoperators are typically well trained to achieve the best possible performance in terms of the evaluation metrics of a given task, we propose a tuning strategy to find a set of parameters to achieve the best robust performance for the path following scenario with ALKE as the performance metric. The proposed tuning strategy is shown in the algorithm in Table 2. A discrete set of values for the parameters are selected manually to define the region of the search. The algorithm first sets parameter $k_{f}$ to be the largest value available. For the given $k_{f}$ , exhaustive search method is then applied to every combination of ${thw}_{st}$ , ${thw}_{sp}$ , and $k_{v}$ to find a set of optimal parameter candidates, where the maximum number of candidates in this set is set to 10.

Table 2

Tuning Strategy for Operator Model for a Teleoperated UGV in a Path-Following Task With Varying Speed

A note about the relationship between ${thw}_{st}$ , ${thw}_{sp}$ is in order here. During steering, the speed of the vehicle is reduced for the upcoming curvature of the turn, and then the steering wheel is turned to complete the steering action. Therefore, speed control needs to happen before steering control. To ensure this, the following constraint is added to the tuning strategy such that speed control has a far point further away from the near point than the steering control.

t h w_{s t} §lt; t h w_{s p}

(6)

After a set of candidate parameters is thus obtained, for every candidate in this set a perturbation of −10%, −5%, 5%, and 10% of the incremented value of every parameter is applied, and only the parameter values that are robust to every perturbation become available for selection. If no parameters are available after applying perturbation due to failures in the simulation (e.g., tire lift-off), the next largest $k_{f}$ value is selected and the procedure repeats until a parameter set is found.

Human Subject Test Study

The human subject test was a within-subject study that consisted of three sets of experiments, each of which had a training period and a data collection period. The test was conducted at the Automotive Laboratory of the University of Michigan using the teleoperation simulation testbed developed by Zheng et al. (2016). This research complied with the American Psychological Association Code of Ethics and was approved by the Institutional Review Board at the University of Michigan. Informed consent was obtained from each participant.

Participants

Thirty-one human subjects were recruited to participate in the test. The subjects were all students from the University of Michigan, Ann Arbor campus and were paid for their participation. We did not screen for motion sickness and did not observe kinetosis in the subjects during the experiment. Two subjects did not have a driver’s license and the rest of the subjects (29) had a driver’s license either in the United States or other countries. Among the participants, 24 subjects were male and 7 subjects were female. 2 subjects were between 26 and 30 years old, and 19 subjects were between 22 and 25 years old. The rest (10) were between 18 and 21 years of age. Majority of the subjects (22 out of 31) had more than 2 years of driving experience. Five subjects had between 6 months and 2 years of driving experience, and the remaining (4) subjects had less than 6 months experience. Comparatively, the subjects had less experience with simulated vehicle driving, with only 8 subjects having more than 2 years of simulated vehicle driving experience, and only 2 had more than 2 years’ experience in a pedal and steering wheel setup. Eight subjects had never driven a simulated vehicle, while 17 subjects had never driven a simulated vehicle with a pedal and steering wheel setup. Fifteen subjects had some but less than 2 years’ experience in operating a simulated vehicle. Twelve subjects had less than 2 years’ experience operating a simulated vehicle with pedal and wheel setup.

Equipment

The simulation platform used in this study is similar to the one used in Mirinejad et al. (2018), which is a fixed-based teleoperated driving simulator developed by Zheng et al. (2016). It was loaded with a 14 degrees-of-freedom vehicle dynamics model that is representative of a notional medium-duty military truck (Liu et al., 2016). The powertrain model was based on a 6L V8 diesel engine and automatic transmission modeled by Ersal et al. (2011). The lateral forces and aligning moments of the four tires are modeled using the Pacejka tire model (Liu et al., 2016).

The test station consists of a monitor for visual feedback, a Logitech G25 steering wheel, and a set of pedals. The simulation platform is realized in MATLAB 3D Animation Toolbox. The visual feedback available to the drivers is an overboard view of the vehicle and roadway environment, with a red marker at the front center of the vehicle as a visual aid for lane keeping. This setup mimics the camera feedback from a real UGV setup in teleoperation. Time delays are introduced when the control inputs are transmitted to the simulation platform and when the vehicle information is transmitted to the monitor for visual feedback. This framework simulates the time delays observed in a real teleoperation setting, where time delays are present in both directions. A 25-meter straight road is placed at the start of the track, allowing the human drivers the time to react and accelerate into the test track. A similar “warm-up” period is therefore also introduced into the simulations for the computational driver model for consistency.

Driving Scenarios

There were three scenarios in this test corresponding to no delay (0 s total delay), moderate delay (0.6 s constant total delay), and long delay (1 s constant total delay) conditions. The test was performed on two tracks to minimize the learning effect, where track 2 is the reverse of track 1 (Figure 1) to avoid inconsistency of the difficulty of the tracks. The track featured curvatures of various optimal turning speeds, which were designed to distinguish from tracks designed for constant speed and test the unique steering behavior with varying speed. For each scenario, a hard constraint was placed on the minimum average speed to complete the track. The specific values are determined in a way such that the driving task was challenging enough for the human subjects, but not impossibly difficult. The minimum average speed for the no delay scenario is set at 35 mph. In the moderate and long delay scenarios, the minimum required average speed was 32 and 30 mph, respectively, because human drivers compensate for poorer steering performance by driving slower (Van Winsum & Godthelp, 1996). A run with average speed slower than the requirement was regarded as a failed attempt and was not used in any calculation or analysis. The decrease in the allowed minimum average speed was due to the increasing difficulty of a successful run from the start to finish as the time delay increased. To aid the subjects, a reference speed sign was located at every curve on the track and this information was also available for the driver model.

Figure 1

Test track 1.

In each scenario, subjects needed to control both speed and steering to drive from the beginning of the track to the end as fast as possible. ALKE was used to evaluate the performance of each subject, and therefore was the dependent variable. The time delay was the independent variable.

Procedure

The human subject tests were conducted in January and February of 2019 in a span of 3 weeks. The test site was located at the Automotive Laboratory of the University of Michigan. For each subject, the test consisted of three 15-min sessions, each of which consisted of a 10-min training period and a 5-min data collection time. The total test time was about an hour and comprised an explanation of the procedure before the experiment began, paperwork, equipment instructions, the test sessions, and a short rest between each session. The paperwork included the consent form and an anonymous survey about their age and driving experiences.

The 10-min training period was designed to give the subjects a chance to become familiar with the setup and different delays. The test setup was similar to an actual teleoperated driving setup with the only major difference being that the vehicle was simulated with high fidelity instead of using a real vehicle. Correspondingly, the visual feedback was also simulated instead of using a real camera. This mode of driving is different from onboard driving even with no delay, because the only feedback available to the driver is the visual feedback from the monitor. Moreover, the driver has to make adjustments and change the driving style with different delay values, especially when the accelerator, brake, and the steering wheel all have control delay. This is in contrast to the constant speed scenarios, where only the steering wheel delay is relevant. The delay on speed control, specifically, makes it very challenging for the subjects to complete the track with an average speed above the minimum requirement. Therefore, this training period was necessary for the subjects to have a chance of successfully finishing the track during the 5-min data collection period.

During data collection, subjects were allowed to repeat the track as many times as they could within the 5-min period. Only the result with the lowest ALKE in the 5-min period was used for analysis for each subject, because we are interested in the lowest human ALKE performance in this study due to aforementioned reasons.

Results and discussion

The collected data points from the human subject test along with the tuned driver model results are shown in Figure 2. To obtain the driver model results, the algorithm in Table 2 is applied with $ϵ$ tuned to be 0.045 and 0.028 for 0.6 s and 1 s delay, respectively. For scenarios with no delay, this threshold is set to be a very large number and has no practical effect on the speed control. The tuned parameters for the driver model are shown in Table 3. Performance of the driver model after tuning is shown in Table 4.

Figure 2

Human subject test data and tuned driver model result.

Table 3

Tuned Driver Model Parameters

Total Delay	k _f	k _v	thw _st	thw _sp
0 s	30	0.3	1.1	2.0
0.6 s	15	2.1	1.8	2.0
1 s	9	1.2	2.0	2.7

Table 4

Tuned Driver Model Performance

Total Delay	ALKE	Average Speed
0 s	0.12 m	37 mph
0.6 s	0.29 m	32 mph
1 s	1.02 m	31 mph

Note. ALKE = average lane keeping error.

The statistics of the human subject test are shown in Table 5 and Figure 3. There are three outlier samples for no delay scenarios, where the outlier samples are defined as 3 median absolute deviations (MAD) away from the median. The data presented in Figure 3 are calculated after excluding the outlier samples.

Table 5

Human Subject Test Statistics

Total Delay	ALKE (Min)	ALKE (Mean)
0 s	0.13 m	0.51 m
0.6 s	0.30 m	1.62 m
1 s	1.00 m	2.40 m

Note. ALKE = average lane keeping error.

Figure 3

Box plot of human subject test results. The three asterisks represent the outliers.

In the algorithm in Table 2, the tuned driver model aims to minimize ALKE without using any human subject data. The research question of interest is then whether this tuning strategy is representative of the lowest human ALKE performance. In Figure 4, the comparison between the tuned driver model performance and human performance is shown. The lowest human ALKE performance recorded is very similar to the tuned driver model performance in all three delay scenarios.

Figure 4

Comparison of tuned driver model and human subject test data.

To establish the statistical equivalence between the tuned driver model performance and the top human performance, an equivalence test with risk level of $α$ = .05 is performed. The statistical test aims to test the following null hypothesis: there is no difference between the ALKE of tuned driver model and the mean of ALKE of human data minus 1.5 standard deviation ( $σ$ ). This null hypothesis claims that the tuned driver model is equivalent to the 6.8 percentile of all human subjects. However, because there is only one tuned set of parameters for the driver model, the original hypothesis cannot be directly tested using t test. Therefore, the t test is performed for a mathematically equivalent null hypothesis, namely, there is no difference between the mean of ALKE of human data and the ALKE of tuned driver model plus 1.5 $σ$ of human data.

Moreover, two more alternative hypotheses are tested to show that there is statistical evidence that the performance of the tuned driver model is bounded around 1.5 $σ$ . The first alternative hypothesis is that the ALKE of human data is less than that of the tuned driver model plus 2 $σ$ of human data (lower tailed alternative hypothesis); the second alternative hypothesis is that the ALKE of human data is more than that of the tuned driver model plus 1 $σ$ of human data (upper tailed alternative hypothesis).

The results of the one sample t test are shown in Tables 6 -8 for the three delay conditions. p values are .711, .286, and .975 for no delay, 0.6 s total delay, and 1 s total delay, respectively. Because the p values for all three delay scenarios are larger than the risk level $α$ , there is no significant statistical evidence to reject the null hypothesis. The p values of the alternative hypotheses for all scenarios are less than the risk level $α$ , with p values of .016, .003, .050, .0003, .005, and .004, respectively. The results show that there is strong statistical evidence that the mean ALKE of human data is bounded between 1 σ and 2 σ plus the ALKE of the tuned driver for all three delay scenarios.

Table 6

One Sample t Test Result for No Delay Scenario

Hypothesis	p Value
Null hypothesis at 1.5 σ	.711
Lower tailed hypothesis at 2 σ	.016
Upper tailed hypothesis at 1 σ	.003

Table 7

One Sample t Test Result for 0.6 s Total Delay Scenario

Hypothesis	p Value
Null hypothesis at 1.5 σ	.286
Lower tailed hypothesis at 2 σ	.050
Upper tailed hypothesis at 1 σ	.0003

Table 8

One Sample t Test Result for 1 s Total Delay Scenario

Hypothesis	p Value
Null hypothesis at 1.5 σ	.975
Lower tailed hypothesis at 2 σ	.005
Upper tailed hypothesis at 1 σ	.004

The results from the t tests, combined with results illustrated in Figure 4, have shown that the proposed model can successfully predict the top human performance after proper tuning. It is worth re-emphasizing that the tuned model results are obtained without human data; that is, the model is not tuned to match any human data as a reference. This shows that the proposed model can successfully predict the upper limit of the ability of a human operator in a teleoperated path following task. To our best knowledge, this is the first model that offers such predictive capability in scenarios in which the operator is allowed to vary the vehicle speed.

In practice, the results reported here are useful, because operators of teleoperated UGVs are typically well trained, which is expected to place them in the top percentiles of human performance. The developed model could therefore be applied in various simulations in place of such human operators, allowing fully simulation based research, development, and engineering studies.

Conclusion

In this work, an operator model for a teleoperated path following setting is extended from the constant speed case to the varying speed case. The purpose of the model is to predict the best steering performance in terms of the lowest ALKE that can be expected from human operators when they are in charge of controlling both the steering and speed. To enable this extension, a far-point speed control is added and the driver model is built upon the ACT-R cognitive framework. A tuning strategy is developed to seek for the model parameters that yield the lowest ALKE in a robust way. Validation data are obtained through human subject tests. Results show that the proposed model can successfully predict top human behavior in various time delay situations and could be used in place of a human operator in the loop for simulations of a UGV in teleoperation.

The findings encourage further development of cognitive models to predict UGV operators’ performance. Such predictive capability would facilitate fully simulation-based evaluation of new UGV technologies, thereby accelerating development cycles and reducing the cost and risks associated with human subject tests. These benefits do not need to be limited to the teleoperation mode, on which this work focused. If the cognitive models can be extended to shared control modes, in which teleoperators share the control authority with onboard autonomy, then the models can benefit the development and evaluation of semi-autonomous solutions, as well.

Mirinejad et al. (2018) have shown that speed can have a strong impact on the ALKE performance in constant speed settings. In this work, we minimize the impact of speed on the steering performance by introducing a hard constraint on the minimum average speed. Best ALKE performance can therefore be obtained at an average speed slightly above the constraint. This formulation, however, cannot predict the statistical distribution of human operators. Therefore, one of the recommended directions for future work is to develop more complex speed control rules and algorithms to predict the statistical distribution of the average speed of human operators.

Another potential future work is to develop a tuning method that does not optimize performance, but predicts the variation of human operators. A tuning method designed around the optimization concept can successfully predict top human performance as shown in this work. However, the inherent disadvantage of this method is that it cannot predict average or below average human performance without existing human data. It is potentially desirable to have a tuning method to capture all levels of human performance.

The performance evaluation metric used in this paper focuses on the lane keeping performance. For applications in which performance is not evaluated in terms of lane keeping, the applicability of the developed tuning method needs further study and validation.

Key points

A computational model for human operators in a teleoperation setting would allow fully simulation-based development of related unmanned ground vehicle (UGV) technologies. It would also allow researchers to predict the impact of time delays in such systems.

A human operator model that can control both speed and steering in the path following task would extend the range of scenarios where the computational model can be used in place of a human operator in the simulations.

This work extends the computational operator model for a UGV at constant speed to introduce speed control into the loop. The extended model builds upon the cognitive framework adopted by the constant speed model and adds an additional far-point visual model for speed control. Speed and steering control are coupled and regulated through a logical constraint.

The developed computational model is validated through human subject tests. According to the statistical analysis, the proposed model can successfully predict top human steering performance in terms of lowest average lane keeping error (ALKE) for a teleoperated UGV with varying speed.

The model could be used in place of a human operator in the simulations to predict human behavior in the path following task.

Footnotes

Acknowledgments

This work was supported by the Automotive Research Center in accordance with Cooperative Agreement W56HZV-19-2-0001 U.S. Army Ground Vehicle Systems Center, Warren, Michigan. DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited. OPSEC# 3207.

Author Biographies

Chen Li is a PhD student in mechanical engineering at the University of Michigan. He received his MSc degree in electrical engineering from the University of Michigan in 2017.

Yue Tang is an algorithm engineer at Aptiv. She received her MSc degree in mechanical engineering from the University of Michigan in 2018.

Yingshi Zheng is an autonomous vehicle algorithm engineer at Aptiv. He received his PhD in mechanical engineering from the University of Michigan in 2018.

Paramsothy Jayakumar is a senior technical expert at the US Army GVSC in Warren, Michigan. He received his PhD in structural dynamics from the California Institute of Technology in 1987.

Tulga Ersal is an associate research scientist at the University of Michigan. He received his PhD in mechanical engineering from the University of Michigan in 2007.

References

Aasman

. (1995). Modelling driver behaviour in Soar [Doctoral dissertation]. University of Groningen, Netherlands.

Anderson

J. R.

Bothell

Byrne

M. D.

Douglass

Lebiere

Qin

. (2004). An integrated theory of the mind. Psychological Review, 111, 1036–1060.doi:10.1037/0033-295X.111.4.1036

15482072

Anderson

J. R.

Lebiere

. (1998). The atomic components of thought. Lawrence Erlbaum.

Braghin

Cheli

Melzi

Sabbioni

. (2008). Race driver model. Computers & Structures, 86, 1503–1516.doi:10.1016/j.compstruc.2007.04.028

Chen

J. Y. C.

Haas

E. C.

Barnes

M. J

. (2007). Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man and Cybernetics, Part C, 37, 1231–1245.doi:10.1109/TSMCC.2007.905819

Dehban

Menhaj

Sajedin

. (2015). Neuro-ACT cognitive architecture applications in modeling driver’s steering behavior in turns. AUT Journal of Modeling and Simulation, 47, 21–29.

Deng

Cao

Lyu

. (2019). Modeling the effect of limited sight distance through fog on car-following performance using QN-ACTR cognitive architecture. Transportation Research Part F: Traffic Psychology and Behaviour, 65, 643–654.doi:10.1016/j.trf.2017.12.017

Donges

. (1978). A two-level model of driver steering behavior. Human Factors, 20, 691–707.doi:10.1177/001872087802000607

Ersal

Brudnak

Salvi

Stein

J. L.

Filipi

Fathy

H. K

. (2011). Development and model-based transparency analysis of an internet-distributed hardware-in-the-loop simulation platform. Mechatronics, 21, 22–29.doi:10.1016/j.mechatronics.2010.08.002

10.

Guo

H. Y.

Chen

. (2013). Understanding and modeling the human driver behavior based on MPC. IFAC Proceedings Volumes, 46, 133–138.doi:10.3182/20130904-4-JP-2042.00085

11.

Laird

J. E

. (2012). The Soar cognitive architecture. MIT Press.

12.

Land

Horwood

. (1995). Which parts of the road guide steering? Nature, 377, 339–340.doi:10.1038/377339a0

7566087

13.

Lichiardopol

. (2007). A survey on teleoperation. Technische Universiteit Eindhoven, DCT rapporten, 155.

14.

Liu

Jayakumar

Stein

J. L.

Ersal

. (2016). A study on model fidelity for model predictive control-based obstacle avoidance in high-speed autonomous ground vehicles. Vehicle System Dynamics, 54, 1629–1650.doi:10.1080/00423114.2016.1223863

15.

Liu

Jayakumar

Stein

J. L.

Ersal

. (2017). Combined speed and steering control in high speed autonomous ground vehicles for obstacle avoidance using model predictive control. IEEE Transactions on Vehicular Technology, 66, 8746–8763.doi:10.1109/TVT.2017.2707076

16.

Liu

Feyen

Tsimhoni

. (2006). Queueing network-model human processor (QN-MHP): A computational architecture for multitask performance in human-machine systems. ACM Transactions on Computer-Human Interaction, 13, 37–70.doi:10.1080/00423114.2016.1223863

17.

Markkula

Benderius

Wolff

Wahde

. (2012). A review of near-collision driver behavior models. Human Factors, 54, 1117–1143.doi:10.1177/0018720812448474

23397819

18.

Mirinejad

Jayakumar

Ersal

. (2018). Modeling human steering behavior during path following in teleoperation of unmanned ground vehicles. Human Factors, 60, 669–684.doi:10.1177/0018720818769260

29664713

19.

Nguyen-Huu

Titus

Tilbury

Ulsoy

. (2009). Reliability and failure in unmanned ground vehicle (UGV). GRRC Technical Report 2009-01. University of Michigan.

20.

Ritter

F. E.

Kukreja

Amant

R. S

. (2007). Including a model of visual processing with a cognitive architecture to model a simple teleoperation task. Journal of Cognitive Engineering and Decision Making, 1, 121–147.doi:10.1518/155534307X232811

21.

Ritter

F. E.

Van Rooy

Amant

R. S.

Simpson

. (2006). Providing user models direct access to interfaces: An exploratory study of a simple interface with implications for HRI and HCI. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 36, 592–601.doi:10.1109/TSMCA.2005.853482

22.

Salvucci

D. D

. (2006). Modeling driver behavior in a cognitive architecture. Human Factors, 48, 362–380.doi:10.1518/001872006777724417

16884055

23.

Salvucci

D. D.

Gray

. (2004). A two-point visual control model of steering. Perception, 33, 1233–1248.doi:10.1068/p5343

15693668

24.

van der El

Pool

D. M.

Mulder

. (2019). Measuring and modeling driver steering behavior: From compensatory tracking to curve driving. Transportation Research Part F: Traffic Psychology and Behaviour, 61, 337–346.doi:10.1016/j.trf.2017.09.011

25.

Van Winsum

Godthelp

. (1996). Speed choice and steering behavior in curve driving. Human Factors, 38, 434–441.doi:10.1518/001872096778701926

26.

Vozar

. (2013). A framework for improving the speed and performance of teleoperated mobile manipulators [Doctoral dissertation]. University of Michigan.

27.

Vozar

Storms

Tilbury

D. M

. (2018). Development and analysis of an operator steering model for teleoperated mobile robots under constant and variable latencies. Robotica, 36, 167–186.doi:10.1017/S0263574716000679

28.

Zheng

Brudnak

M. J.

Jayakumar

Stein

J. L.

Ersal

. (2016). An experimental evaluation of a model-free predictor framework in teleoperated vehicles. IFAC-PapersOnLine, 49, 157–164.

29.

Zheng

Brudnak

M. J.

Jayakumar

Stein

J. L.

Ersal

. (2018). A predictor based framework for delay compensation in networked closed-loop systems. IEEE/ASME Transactions on Mechatronics, 23, 2482–2493.doi:10.1109/TMECH.2018.2864722