Abstract
The use of mobile phones while driving has been a hot topic in the field of driving safety for decades. Although there are few studies on the influence of gesture control on in-vehicle secondary tasks, this study aims to investigate the impact of gesture-based mobile phone use without touching while driving from the perspective of multiple-resource workload owing to visual, auditory, cognitive, and psychomotor resource occupation. A novel gesture control technique was adopted for secondary task interactions, to recognize the gestures of drivers. An experiment was conducted to study the influences of two interaction modes, traditional touch-based mobile phone interaction and gesture-based mobile phone interaction, on driving behavior in three different cognitive level task groups. The results indicate that gesture-based mobile phone interaction can improve driving performance with regard to lateral position-keeping ability and steering wheel control; nevertheless, it has no significant impact on longitudinal metrics such as driving speed, driving speed variation, and throttle control variation. Gesture-based mobile phone interactions have a larger effect on secondary tasks with medium cognitive load but not on actual operation tasks. It was also verified that the performance of gesture-based mobile phone interaction was better in secondary mobile phone tasks such as switching (e.g., switching songs) and adjusting (e.g., adjusting volume) than the traditional interaction mode. This study provides the theoretical and experimental support for human–computer interaction using gesture-based mobile phone interactive control in future automobiles.
The past few years have witnessed an acceleration in the development of mobile phones. As convenient communication tools, they are now essential to millions of people. With the rapid growth of the number of vehicles in China ( 1 ), a considerable number of people are using their mobile phones while driving, and this has become one of the crucial factors leading to crashes. A driver’s lack of attention is one of the major causes for disengagement in automated vehicles ( 2 ). A multitude of studies have shown that mobile phone use while driving is one of the main factors for distraction ( 3 – 7 ). The driver’s attention can be diverted by mobile phone use, resulting in a higher likelihood of crashes ( 5 ). A study from California confirmed that crashes caused by cell phone usage result in severe injuries ( 8 ). A cross-sectional study was conducted in May 2014 targeting 986 male students at King Saud University, Riyadh, Saudi Arabia. Many students conceded that texting (77.0%) and speaking on handheld mobile phones while driving (83.9%) pose more risks than hands-free driving (35.9%) ( 9 ). Mobile phone use while driving can lead to poor vehicle performance, for example, a longer brake reaction time, an undulating lane course, a fluctuating driving speed, and an inconsistent following distance and time headway ( 10 – 15 ). Distraction while driving impairs driving performance, and different distractors have been associated with different levels of risk ( 16 ). Using a hand-held mobile phone and interacting with devices with touchscreens increases the risk of a crash approximately fourfold to fivefold compared with situations with no distractions ( 17 ). Research has shown that pressing the buttons, scrolling the sliders, and reading a small display while driving is quite dangerous ( 18 – 20 ).
Multitasking, such as using mobile phones while driving, can cause interference between primary and secondary tasks ( 21 ). Ergonomically speaking, when one has a multi-task assignment, the primary task is the one that takes priority. That is, it is the task that should receive the largest amount of allocated mental and physical resource. The secondary tasks are subordinate or incidental to a primary task for which multiple tasks are required for completion. Multiple resource theory ( 22 ), which holds that information resources can be divided into four categories of visual, auditory, cognitive, and psychomotor (VACP), views performance decrement as a shortage of these different resources and describes humans as having limited capability for processing information. Cognitive resources are limited, and a supply-and-demand problem occurs when an individual performs two or more tasks that require a single resource. Excess workload caused by a task using the same resource can cause problems and result in errors or slower task performance. If primary and secondary tasks have similar demand levels for the same resource or for multiple resources, common resources are shared, degrading task performance ( 22 ). Data suggest that the effects of cognitive load on lane maintenance may depend on the type of secondary task in addition to the predictability of the primary task ( 23 ). Cullen ( 24 ) summarized the four resources as 28 types of behavioral elements as shown in Table 1, and they were rated from 0 to 7. The VACP rating scale measured the demand on resources from basic abstract behaviors. Therefore, workload could be predicted after breaking down the task to a behavioral level. Driving is a visually dominant, complex cognitive behavior that generates demands primarily on a driver’s visual, cognitive, and motor resource dimensions ( 25 ). The driver generally performs a variety of tasks, including those directly involved in driving and those that are not. Usually, the primary task is to steer the car and be aware of potential road hazards. Secondary tasks refer to in-vehicle tasks that require the driver to divide attention ( 26 ). Driving behavior requires more visual attention and visual channel occupation than normal activities. Visual distraction has a highly negative impact on the driver ( 27 ). Therefore, in-vehicle secondary tasks that compete for visual resources will reduce the performance of both primary and secondary tasks. For example, selecting music or answering the phone while driving increases the amount of time spent by the drivers taking their eyes off the roadway, as the tasks require the use of the same visual resource. As driving includes tasks that require high demands, drivers must be warned against using all devices, except for the radio and CD players, or the use of such devices should be banned ( 26 ). The in-vehicle secondary tasks can divert driver attention, thereby degrading their driving performance and increasing the risk of crashing. Moreover, for rural roadways, this correlation is even stronger as such roadways have a large amount of phone usage while driving ( 7 ).
Twenty-Eight Behavioral Elements and Their Rating Scales ( 24 )
Driving is primarily a visual task that requires a high visual workload, and it is reported that 95% of information is obtained and processed by vision ( 28 ). There is a “15-second rule” that indicates that the time spent on continuously operating visual secondary tasks should not exceed 15 s, otherwise it will increase the risk of a crash ( 29 ); however, other researchers argued that 15 s away from the roadway is a significantly high number. Research has also found that a glance more than 2 s away from the roadway increased the likelihood of crashing or nearing a crash by almost twofold ( 30 ). Auditory information is the second most important channel for information perception. Human responses to auditory information are much faster than to visual information ( 31 ). It can be observed from the multiple resource theory that receiving auditory information from secondary tasks does not occupy the visual channel of the primary task while driving, and drivers can receive auditory information from all directions. Driving with hands-free devices was perceived to be safer; however, the attitude of drivers is that such devices are overall unsafe ( 32 ). However, auditory information is also a source of distraction for the attention required for driving ( 33 ). If the visual modality is largely occupied owing to the secondary task, the visual load increases, leading to more crashes ( 34 , 35 ).
There are numerous studies on mobile phones being used while driving ( 36 – 38 ). The association between mobile phone use while driving and background information has also been studied ( 39 , 40 ). Recently, Louveton et al. ( 41 ) evaluated certain mobile phone touch gestures while driving. However, experiments on the use of gesture control-based mobile phones and their impact on in-vehicle secondary tasks have not been reported widely yet. Gesture or touchless technology does not occupy visual and auditory resources, the two most important modalities in driving. Gesture control has become a promising novel interaction mode for in-vehicle information control. Humans use a wide variety of gestures to perform simple actions such as pointing and more complex actions such as expressing feelings and communicating with others ( 42 ). Gesture recognition is believed to be promising for in-vehicle computers or mobile phone applications to understand human body language, and it is more natural than the legacy textual user interface and graphical user interface without the aid of additional interaction devices (mouse, keyboard, touchscreen, etc.). Gesture recognition has been used in the fields of amusement, sign language translation, remote control, and virtual assembly ( 43 ). Quite a few studies have shown that there are significant benefits to using gestures for secondary task interactions, including reducing the task recovery time between completion of a secondary task and returning to the primary task ( 44 ), decreasing the interruptions of primary driving tasks ( 45 ), reducing the task load, and decreasing the task completion time significantly ( 46 ).
This study explores the influence of gesture control on driving by analyzing driving performance and the driver’s workload comprehensively. An experiment was conducted using the open racing car simulator (TORCS) to compare the driving performance and driver’s workload in two interaction modes, traditional touch-based mobile phone interaction and gesture-based mobile phone interaction. The interactions were studied under situations involving three cognitive levels: non-cognitive tasks, medium cognitive level tasks, and actual operation tasks. The cognitive tasks require participants to mentally process new information and use that information later in the same or similar situation ( 47 ). This study hypothesized that (1) gesture-based mobile phone interaction can influence the control ability of the driver in both lateral and longitudinal directions; (2) the effect of gesture-based mobile phone interaction while driving will be larger in secondary tasks with high cognitive load; and (3) gesture-based mobile phone interaction will facilitate the secondary tasks more than in the traditional interaction mode.
Methodology
Thirty-six people participated in this experiment. There were three groups in the experiment: the non-cognitive task group, medium cognitive level task group, and actual operation task group. Each group had two interaction modes: traditional mobile phone interaction and gesture-based mobile phone interaction. Twelve participants were assigned to each group. The participants were asked to complete the primary driving task and the in-vehicle secondary task. Each participant had an opportunity to perform one test to get familiarized with the driving simulator and gesture control device. Driving performance and subjective mental workload were measured, and a two-way analysis of variance (ANOVA) was used to determine whether there are any statistically significant differences between different cognitive level groups and different interaction modes.
Participants
Thirty-six people were recruited through social media networks and the WeChat platform to participate in this experiment (28 males, 8 females; the average age was 29 years). There were no restrictions based on handedness or gender. All the participants were required to have more than two years of driving experience, knowledge about the driving simulator, driving games and gesture control technology, and normal vision and hearing. First, their informed consent was obtained; then, they were assured them that their information will be secure and informed that the data they provided would be used only for scientific research. All procedures followed an approved institutional protocol.
Experimental Facility
This study used TORCS) ( 48 ), a gesture control device by Vidoo (Sharpnow Inc., China) ( 49 ), a smartphone with a 5.5-inch display, and two notebooks (one for data analyzing and the other for display). Figure 1 shows the layout of the overall experimental facility. Car controls were provided by a G27 steering wheel and pedal set (Logitech Inc., U.S.) ( 50 ). The simulated car was a Peugeot 406. Car telemetry data and real-time data were generated by Visual Studio 2015 C++, which allowed for synchronization and off-line data analysis. The smartphone with a 5.5-inch display was connected with Vidoo and was placed beside the driver’s dominant hand. The participants could adjust the position and the angle of the phone according to his or her needs using an adjustable support. Vidoo, which is directly equipped with an independent computing core, can be connected with various types of terminal devices and processes data without a PC.
Experimental facilities and driving scene.
Performance Measures
The dependent variables include driving performance and subjective mental workload. After reviewing numerous driving safety literatures, driving speed ( 51 ), speed variability ( 52 ), variation of lateral position, variation of steering wheel position ( 53 ), and variation of throttle control ( 54 ) were adopted as measures of driving performance. Variation of lateral position denotes the standard deviation of the lateral distance between the vehicle and the central axis of the lane; variation of steering wheel angle denotes the standard deviation of the steering angle of the steering wheel; and variation of throttle control denotes the standard deviation of the accelerator pedal displacement. The recording frequency was 50 Hz, based on ISO standards. After a driving task was finished, the drivers were asked to evaluate the task load using the six-scale NASA task load index (NASA-TLX) ( 55 ), which ranged from 0 to 9 (for performance, 0 indicates failure and 9 perfect; for the remaining five scales, 0 indicates very low and 10 very high). NASA-TLX is a widely used, subjective, multidimensional assessment tool that rates perceived workload to assess a task or other aspects of performance. NASA-TLX includes six scales: mental demand, physical demand, temporal demand, performance, effort, and frustration ( 56 ). The load index value was defined as the overall workload.
Experimental Scenario and the Design of In-Vehicle Secondary Tasks
The driving situation in this study was as simple as to avoid interference from unrelated external factors. The participants were asked to drive the car in an expressway outside the city in a panoramic simulated environment. The landscape was a flat textured surface with a two-way road, two lanes that were 7.5-m wide, and green vegetation on each side, which is typical for highways in China. The speed limit was 120 km/h, the weather was sunny, and there was no direct strong sunlight.
The primary task was driving, which required the participants to maintain a stable speed of 80 km/h and stay as close possible to the centerline of the lane. The driver was first asked to ensure smooth driving; in addition, the driver should as far as possible complete the secondary tasks, that is, tasks related to mobile prompts. To minimize the influence of the background light from the environment, the display brightness was adjusted according to the requirements of the participants before the experiment, and no further changes were made during the experiment.
The independent variables include interaction mode and cognitive level. Cognitive level was the between-subject variable, which consisted of three categories: non-cognitive task, medium cognitive level task, and actual operation task. Interaction mode was the within-subject variable, which includes two levels: traditional mobile phone interaction mode and gesture mobile phone interaction mode.
Table 2 shows details of the task scenario design. There were three groups in this experiment: the non-cognitive task group, medium cognitive level task group, and actual operation task group. When designing the cognitive resource load task, the resource occupancy of the visual and motor dimensions should be separated as much as possible. Therefore, the auditory channel with low impact on the primary driving task was used in this study to obtain cognitive task information. The study used 0-back for the first difficulty level (sign/signal recognition, rating about 3.7) and 1-back for the second difficulty level (sign/signal recognition+recall, rating 5.3~7). In the non-cognitive task group, a 0-back task was used to simulate a non-cognitive secondary task because 0-back tasks require few cognitive demands. For a better comparison of the differences between the non-cognitive and cognitive task groups, only the auditory channel is used for information input.
Task Scenario Design
The participants were required to do the N-back (0-back or 1-back, as described previously) tasks by clicking the corresponding numbers on the screen using their fingers or using gestures of 0 to 9 to represent the corresponding numbers in the traditional interaction mode. In the gesture-based mobile phone interaction mode group, the participants were asked to use gestures of 0 to 9 to represent the corresponding number. The participants listened to a sequence of ten digits presented as audio, with the starting utterance of each digit spaced 3 s apart. In the 0-back task, the participants had to repeat the most recent digit that was presented, whereas in the 1-back task, the participants had to repeat the digit in a backward sequence from the current digit. Each digit (0–9) was presented once in the sequence in a randomized order ( 57 ). All the phone-based secondary tasks were generated randomly.
Table 3 shows the corresponding behaviors of traditional mobile phone interaction and gesture-based mobile phone interaction in the actual operation task. Three types of gesture-based mobile phone interaction tasks were built based on gesture control: playing/pausing/closing, switching, and adjusting. These three tasks were considered for simulating the operations in the in-vehicle secondary task of daily mobile phone use. In this group, the traditional operation of a music player in the driving mode of Netease cloud music application (V3.8.1) was compared with the gesture-based mobile phone interaction. There are two reasons for choosing music for comparison. First, operating a music player is quite common and representative of daily driving; second, this operation comprehensively covers the VACP behavior elements from the perspective of a multi-resource workload.
Traditional Mobile Phone Interaction and Gesture-Based Mobile Phone Interaction in the Actual Operation Task
The gestures were set for right-handed people, and Vidoo supports the handedness setting.
All participants were assigned randomly, and there was no significant difference in the average age and driving experience of each group. In non-cognitive and cognitive task groups, the participants were asked to rest for 3 s after completing each secondary task (ten secondary tasks in each interaction mode). In the actual operation task group, three types of music operation, playing/pausing/closing, switching, and adjusting, should be completed. The 90-s experimental time after steady driving was divided into three parts, and every participant was asked to perform the three types of operation randomly (using Latin Square arrangement) in three 30-s intervals. In each interval, only one type of operation was performed, and there were no more than ten secondary tasks.
Procedure
First, the participants were asked to complete a profile questionnaire (basic information and driving experience) and an informed consent form (approximately 1.5 min). Second, the experimental content was explained to the participants and the operation of the driving simulator was introduced (approximately 3 min). Each participant was given an opportunity to perform a simple baseline task using TORCS, that is, to drive for 5 min without secondary tasks. Third, the experimenter demonstrated the primary and secondary tasks (approximately 3 min). The participants were asked to be seated in the driving simulator and adjust the position of the seat, pedal, brake, cell phone, steering wheel angle, display light, and system volume under the guidance of the experimenter. Then, the experiment was started. TORCS, Vidoo, a smartphone, and two notebooks were used during the experiment. Finally, the experimenter confirmed the quality of all the collected data and paid the participants (approximately 1 min).
Once the participants reached the required speed of 80 km/h and drove steadily, they were required to complete the corresponding secondary tasks in three groups. These groups would end once all the secondary tasks were done. The participants performed a post hoc evaluation of the task workload using the six-scale NASA-TLX after each group was completed. Scores from low to high meant that the workload was gradually increasing. Each participant was also asked to evaluate the gesture-based mobile phone interaction for technical improvements.
Data Analysis
The driving data and vehicle control data of the driver were recorded in real time by TORCS, and SPSS (Statistical Product and Service Solutions) was used for data analysis. ANOVA was used to determine whether there are any statistically significant differences between different cognitive level groups and different interaction modes. The Kolmogorov–Smirnov test was used to verify that both data and residuals were normally distributed and the Bartlett test to verify the homogeneity of variance. Both assumptions were consistently confirmed. The criteria for overall statistical significance were set at α = 0.05.
Results
Driving Performance
When comparing the performance of traditional interaction with gesture-based mobile phone interaction in the non-cognitive task group (as shown Table 4), variation of lateral position and variation of steering wheel position showed significant differences, namely, p-value was 0.012 and <0.001, respectively. There was no significant difference in average speed, speed variation, and the variations of throttle control.
Analysis of Variance of the Influence of the Interaction Mode on Driving Performance in Different Task Groups
The throttle control in the driving simulation platform fluctuated in the range of 0 to 1.41.
The steering wheel angle in the driving simulation platform fluctuated in the range of −1 to +1.
In the medium cognitive level task group, the cognitive load of 1-back tasks was medium. As shown in Table 4, the interaction mode mainly affected variation of lateral position (M = 0.646, 0.493) and variation of steering wheel position (M = 0.015, 0.010), where the p-value was 0.020 and 0.013 by comparing the driving performance for traditional interaction mode and gesture-based interaction mode, respectively.
As shown in Table 4, for play/pause/close, switch, or adjust operations, the variations of lateral position and steering wheel position showed significant differences. The p-value was 0.005 (variation of lateral position) and 0.001 (variation of steering wheel position) in play/pause/close and switch operations, and 0.028 (variation of lateral position) and 0.009 (variation of steering wheel position) in the adjust operation when comparing the average performance for traditional interaction mode and gesture-based interaction mode, respectively. Although the interaction mode affected the lateral position and steering wheel position, the p-value in the adjustment operation (0.028, 0.009) was higher than that of the other two types (0.005, 0.001).
The improvement of the lateral position and steering wheel controllability in gesture-based mobile phone interaction was also studied. The level of improvement can be computed using the following formula:
where ImpRatio is the level of improvement, VarTrad is the variation of traditional interaction mode, and VarGest is the corresponding variation of gesture-based mobile phone interaction mode. The ImpRatio values of the variation of lateral position (m) and variation of steering wheel position (−1 to +1) are 38.9% and 47.2% in the non-cognitive task group and 23.7% and 33.3% in the medium cognitive level task group, respectively. Compared with the secondary tasks without cognitive workload, gesture-based mobile phone interaction under medium cognitive level had less impact on driving performance.
Subjective Workload
There was no significant difference in subjective workload except for performance in the non-cognitive task group (see Figure 2). The ratings of mental demand (2.140, 2.150), physical demand (2.410, 2.370), temporal demand (2.710, 2.410), effort (2.165, 2.040), and frustration (1.250, 1.240) for two different modes were similar, which indicated that the experience of the participants had no significant difference in the five scales. However, the rating of performance in the traditional interaction mode was significantly higher than that of gesture-based mobile phone interaction mode (4.540, 3.120; p = 0.016), which indicated that the gesture-based mobile phone interaction had a lower workload and was even more satisfying for the drivers.
Subjective workload of the two interaction modes in different task groups using the NASA-TLX questionnaire: (a) non-cognitive task, (b) medium cognitive level task, and (c) actual operation task.
There was no significant difference in subjective workload except for performance in the medium cognitive level task group (see Figure 2). The rating of performance in the traditional interaction mode was higher than in the gesture-based mobile phone interaction mode (6.540, 5.010; p = 0.024). Therefore, drivers who did the secondary tasks with medium cognitive load while driving were more satisfied with gesture-based mobile phone interaction. Although the results in the non-cognitive and cognitive task groups were similar, the p-value for performance in the medium cognitive level task group was higher than that of the non-cognitive task group, which may suggest that the advantage of gesture-based mobile phone was more noticeable in the scenario requiring less cognitive load.
During the actual operation task, the results were much different from those in the other two groups. There was no significant difference in physical demand (3.245, 3.145), performance (4.156, 4.045), effort (2.165, 2.040), and frustration (1.250, 1.240). However, the subjective workload of gesture-based mobile phone interaction was higher than that of traditional mobile phone interaction in mental demand (3.571, 4.457; p = 0.087) and temporal demand (1.570, 2.145; p = 0.092). This suggested that cognitive resources were occupied when the drivers used the gesture-based mobile phone, and driving was not as smooth during the actual operation tasks.
Discussion
The purpose of this study was to investigate the influence of traditional mobile phone interaction and gesture-based mobile phone interaction on driving performance. Three groups—non-cognitive, medium cognitive level, and actual operation task groups—were designed to study this influence.
The results suggest that the gesture-based mobile phone interaction, compared with the traditional mobile phone interaction, improves the driver’s lateral position stability and steering wheel stability significantly in both non-cognitive and cognitive tasks, and the driver’s degree of self-satisfaction in gesture-based mobile phone interaction is higher. The main reason for the improvement is that gesture-based mobile phone interaction does not produce temporary visual distractions compared with traditional mobile phone interaction. The driver used gestures to complete secondary tasks to be able to allocate their visual resources to the road instead of the phone and maintain better lane-keeping ability. The results support the theory of automatic information processing ( 58 ) and illustrate the negative impact of hand operations and their occupation of, and dependence on, visual resources in the traditional mobile phone interaction mode on driving performance. The gesture-based mobile phone interaction has a larger effect in non-cognitive tasks. One possible explanation is that in phone-use secondary tasks with medium cognitive workload, the restriction of cognitive resources is more significant than that of visual resources, which results in a larger effect in the gesture-based mobile phone interaction mode without cognitive workload. However, it may also relate to the sample feature (28 males, 8 females, the average age was 29 years) in the experiment.
The results of the actual operation task group suggest that gesture control has less impact on driving behavior than touch gesture operations, thus leading to better driving performance, especially for lane-keeping and handling. For the abnormalities in adjusting operation tasks (the advantage of gesture-based mobile phones was diminished compared with the other two operation tasks) and the subjective workload (subjective workload of gesture-based mobile phone interaction was higher than that of traditional mobile phone interaction in mental demand and temporal demand), the questionnaires revealed that some of the participants still felt slightly uncomfortable with this new mode of operation even after completing the gesture control training. Most of the participants pointed out that the gesture control device did not provide timely response and partially delayed the adjusting operation, which further caused the participants to glance at the phone screen for actual volume feedback, thereby worsening driving performance and increasing time pressure. This also provides valuable feedback for further improvement of the gesture control technique.
The research suggests that the gesture-based mobile phone interaction mode, as an ideal alternative for in-vehicle phone-use secondary tasks, has positive effects on driving behaviors and subjective workload and is more critical to lateral control during driving. Gesture control technology is expected to replace some interactions involving visual demand in operating traditional mobile phones, which can significantly improve the driver’s driving performance. This study also provides guidance for the application of gesture control technology in phone-use secondary tasks. For instance, in-vehicle information interaction gestures should be natural, simple, and intuitive, and drivers should choose the interaction mode with the least visual, cognitive, and motor workload when the phone must inevitably be used. However, related ergonomics research, especially in-vehicle applications, is still in its infancy. There is no persuasive experimental data and research to support or oppose the application of gesture control technology in vehicles, especially considering its use with mobile phones.
Potential limitations of this study include the number and age of participants, the disadvantages of the gesture control device, and the driving simulator. The sample (36 participants) is not big for a mixed within-subject (two phone interaction modes) and between-subject (three cognitive levels) design. Young people are more familiar with and receptive to new technologies. The participants, who were recruited with experience of driving and gesture control, are relatively young and may not represent drivers of all ages. Age seems to significantly affect driving behavior ( 59 ). The effect of interaction types, visual and auditory, on the performance of older drivers was significant, and the effect of secondary task engagement may vary by interaction types and driver age ( 60 ).
Therefore, the conclusions of this study cannot simply be extended to driving groups of all ages. In addition, the study is limited by the gesture control device and the driving simulator. The gesture recognition and response speed of the device still has some delays, and the designated gesture database cannot be replaced temporarily. Besides, situations in which some participants are not familiar with some gestures cannot be ruled out. The experience and proficiency of gesture-based interaction use is also an important factor. Long-term use can increase their exposure to gesture-based interaction and enhance their ability to handle the events when driving, which may lead to different results. Furthermore, the experiment was conducted using a low-fidelity desktop driving simulator. The duration of the experimental condition was short, and the aspect of vibrations in the simulated driving situation was not considered. As a result, there may be some differences in applying the conclusions of this study to real driving situations.
In the current study, the impact of participant age and gender were not considered. The differences in age and gender may influence driving performance and behavior, which may lead to different results. Future research should consider including male and female groups with differences in age and experience, to ensure that the findings are consistent with those of this study. A larger sample should be selected for testing and the influence of individual differences on driving performance in the two interaction modes should be investigated. A second line of future research should investigate the influence of combining gestures and operations on drivers. This study only used one music application when comparing traditional mobile phone interaction with gesture-based mobile phone interaction during the actual operation task group. Experiments may require a more detailed design, and the comparison of different applications and their operation will also enrich the application of gesture control technology in vehicles. Further, future research should also investigate how differences in driving experience influence driving performance in the two interaction modes.
Conclusions
This study compared the influence of traditional mobile phone interaction and gesture-based mobile phone interaction on driving performance from the perspective of multi-resource workload. Based on the multiple resource theory and its subsequent studies of the VACP rating technique, which measured the demand on VACP resource, three task groups were designed with different cognitive levels. The study results show that gesture-based mobile phone interaction can improve driving performance in lateral position-keeping ability and steering wheel control. However, the hypothesis that gesture-based mobile phone interaction can influence the control ability of the driver in longitudinal direction is not supported. To the authors’ surprise, the gesture-based mobile phone interaction has a larger effect in the medium cognitive level than non-cognitive and actual operation tasks, which need thorough investigation. This may be related to the impact of age, gender, or number of participants. Moreover, the gesture chosen as representative should be natural, simple, and intuitive and make drivers feel comfortable; otherwise, driving performance may become worse. Introducing gesture control devices into in-vehicle information systems could reduce the occupation of visual channel owing to the use of electronic equipment such as mobile phones. This study provides a theoretical basis for the design and improvement of mobile phones or in-vehicle information systems.
Footnotes
Author Contributions
The authors confirm contribution to the paper as follows: study conception and design: Jianwei Niu, Xingguo Liu; data collection: Xingguo Liu; analysis and interpretation of results: Jianwei Niu, Xingguo Liu, Dan Wang, Yulin Zhou; draft manuscript preparation and revision: Jianwei Niu, Dan Wang, Yulin Zhou. All authors reviewed the results and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
