Abstract
News concerning autonomous cars are becoming more and more common today. There are recordings of vehicles in self-driving mode having an accident as well as footages in which they operate properly, in an errorless way. What can cause this fundamental difference? Either a software problem or the inaccuracy of the data emitted by the sensors or an incorrect decision issued by the central unit. This article is going to show the various ways in which the decisions of the central unit can be influenced and so the passengers and the environment of the vehicle can be endangered. The aim is not to affect the trade of the autonomous cars in a negative way but, on the contrary, to attract the attention of the manufacturers to make them get prepared for and protect their cars against these dangers. At the end of the article there are going to be some suggestions made on how to install a module that can recognize external manipulations in self-driving cars to make their operation more secure.
Keywords
Introduction
In order to place the autonomous cars slowly into our everyday lives from the world of science fiction, there was a need for new, or rather more accurate sensors [1, 2] and also a major development of artificial intelligence [3]. Some research focused on the autonomous cruise construction for remotely controlled ships [4] and the decision-making for the self-driving navigation of naval [5] or autonomous underwater vehicle [6] connected to our research, but we are interested for the autonomous vehicles.
Other connecting papers focused on safety mistake, security aggression, and achievable countermoves for self-driving vehicles [7], shows Blockchain-based security attack flexible patterns for self-driving cars [8], using stpa method and the six-step model to integrate autonomous vehicle safety and security [8]. Some other research gives an overview of autonomous vehicles safety [10] or a study on cyber-security of self-driving and remotely controlled transportations [11].
Our paper gives detailed descriptions of the sensors, and of the possible Situations suitable for misleading self-driving vehicles [12]. The article is going to show the various ways in which the decisions of the central unit can be influenced and so the passengers and the environment of the vehicle can be endangered.
At the end of the article there are going to be some suggestions made on how to install a module that can recognize external manipulations in self-driving cars to make their operation more secure.
Sensors in self-driving vehicles
In 2014 the ’Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems’, the J3016 regulation of The Society of Automotive Engineers (SAE) International determined the categories of autonomous vehicles, which was updated as a result of the circumstances in 2016. At present the motor industry uses the J3016_201609 regulation [13]. According to the regulation there are 6 levels of self-driving vehicles. On level 0 there are the vehicles in which no self-driving function or driving supporting system can be found. On the highest level, on level 5 there are the vehicles which are absolutely suitable for operating without a driver. The people in the vehicles on this level are simple passengers and every single task in connection with driving is completed by the system.
The currently available sensors used by car manufacturers are LiDAR, radar, camera and UH sensors. The vehicle gathers information on its environment with their help.
LiDAR
LiDAR (Light Detection and Ranging) makes a 3D point cloud within the distance of 100 m and this way it can distinguish between the various objects and the background. By applying this system we can avoid situations in which a suddenly appearing light-coloured vehicle in a light background is not detected by the camera, which is exactly what happened at the maladventure of a Tesla Model S in May 7, 2016. [14].
The quality of its resolution is not as good as that of an HD camera. Nevertheless, it provides a considerable amount of data due to its 3D feature. Its advantage is that it can be used at night, too and there is no need for road lighting as it can be observed in the test of the Ford [15].
It has some disadvantages; it is costly and cannot be used properly in all meteorological conditions. It does not provide reliable information in case of snow or fog.
Radar
Radar is un-costlier than the ray or the laser solution and gives less information than the HD camera because of its bad resolution. However, its data are not influenced by weather conditions or visibility. The result radar provides is an object list containing the size and the distance of objects. However, in vehicular traffic it is not only the reflections from certain objects that have to be taken into account but the reflections from the road, the guard rail and any other objects around. It can be used at keeping distance as well as at pre-indicating braking. It is due to the multiple reflection of radar that, for instance, in December 2016 Tesla managed to avoid an accident in the Netherlands. Although the camera could not see that the vehicle in front of the one preceding the self-driving car started using the emergency brake, the radar detected the reflections of that car on the asphalt. Therefore, the car could start braking in time to avoid the crash [16].
Camera
The HD resolution cameras are comparatively low-priced and are capable of distinguishing colours (in appropriate circumstances). It is possible to install them all around the car-body; approximately 10 of them are enough to monitor the whole environment of the vehicle. Due to their resolution and the frequency of recordings (30 photos a second), they provide a lot of information that has to be processed by the central unit.
UH sensors
Ultrasound sensors fitted in cars are used for parking, or for giving warnings about the traffic in the rear area when changing lanes. The reason for their suitability for these tasks is their short range (approximately 10 m). They are inexpensive and reliable devices.
Central Unit
Fulfilling the self-driving function requires a great amount of calculations since the data to be processed are provided by several sensors and cameras. All of them are tasks to be performed in a strictly limited period of time for bringing the car to a halt at red traffic lights cannot possibly take long minutes, neither can turning the steering wheel swiftly to the side in order to avoid colliding with an oncoming vehicle ... etc.
These high-efficiency computers have to fit in the vehicles in a way that there is enough room left for the passengers and the luggage. Although the Michigan Micro Mote (M3) company has produced the smallest working computer in the world, its capacity for calculations would not be sufficient to guide an autonomous car properly [17].
GPE architecture is more suitable to perform this task. It makes it possible to process data simultaneously since the information to be processed is gathered from several sensors and cameras at the same time.
There is a sudden increase in the demand for GPUs since they have been used in bitcoin mining recently. This has led to the duplication of the prices of the cards, which hinders their use in researches and so impedes the development of researches such as ETI [18].
However, due to the increased demand there is higher income that can be used to develop GPUs. In consequence, NVIDIA introduced a new target hardware with 2 petaflops, developed for artificial intelligence, at San Jose GPU Technology Conference in March 2018 [19].
However, this equipment would take up too much room in a car. Thus, car manufacturers and autonomous car programming companies are waiting either for the reduction of its size or for the Drive PX Pegasus system announced by NVIDIA at GTC Europe 2017 Conference in the autumn of 2017. The basic size of this system is the same as the size of a number-plate; its capacity is 320 billion operations a second, which is still less than the calculating capacity of the one mentioned before, though it is promised to be sufficient to provide for the 5-level self-driving function [20].
There have been experiments to control a car simply based on the data provided by the cameras. In February 2018 a Porsche Panamera was controlled by a Huawei Mate 10 Pro mobile phone. This type of mobile contains the Kirin 970 system chip with a neural processing unit (NPU), with the help of which the calculations required by MI can be accelerated [21]. NPUs are extremely good at image processing, just like in the Apple A11 chip, and they proved themselves to be sufficient at the 30 m/h test, too. They enabled the mobile phone to recognize the various situations and make the vehicle react to them by directing the screen of the camera fitted on the top of the car. It would be an exaggeration to claim that even this phone would be able to control a car under all circumstances since during the test there was only one camera used with the help of which only a very small range of situations occurring during a drive can be examined. Still, it highlights the fact that a further developed NPU together with the GPUs can assist in controlling a car. Indeed, it can even replace them and so it would be possible to reduce the consumption of energy by the units installed in autonomous cars.
Development of artificial intelligence
Artificial intelligence, which has appeared in every field of industry, is based on the recent results of the researches in artificial neural networks [22], computer vision [23] and the multi-layer boolean neural network [24].
It is obvious that it was also the idea of the use of artificial intelligence that presented itself at the development of autonomous cars where decisions have to be made based on the neverending flow of the immense amount of information emittted by the various sensors. In the field of industry the artificial intelligence makes decisions about selecting the faulty products on the basis of image recognition, hence a bad decision can result in the elimination or disposal of good products or the misjudging of faulty items. In the area of human resources management the applied artificial intelligence can prefer the representatives of one of the sexes in the course of selecting the suitable applicants for a position [25]. In case of self-driving cars a bad decision can cost human lives as we have already witnessed in the Uber accident [26]. One of the research groups at MIT has managed to create the World’s first psychopath, A1, which highlighted the liability of teaching artificial intelligence [27].
Due to the results of the artificial intelligence researches of recent years it is out of question that artificial intelligence should be used in autonomous cars in order to make an appropriate decision at an appropriate speed. On the basis of the information provided by the data gained from various sensors and cameras, data processing and decision making have to be performed in 1/30 of a second. During the teaching process it would be the easiest to take only the rules of the road into consideration at decision making. This would be easier to justify legally, too in case of accident.
The constructors at Tesla have generated a so-called shade mode for the Autopilot. It also analyses the traffic situations while a human driver is driving and transfers these data to the centre so that they can be used at the development of the next version. This is a great idea since accomplished drivers can avoid several perilous situations. However, it can generate problems if an inexperienced driver’s bungling appears as a proposed solution.
There are several types of target hardware found in vehicles nowadays. By applying one of the latest developments of NVIDIA an interactive world can be created, which can not only be used in the field of the toy industry to make the settings more spectacular but also in the process of teaching self-driving cars to simulate real-life surroundings easily.
Hyundai CRADLE has invested in Perceptive Automata startup business whose technique is to guess the pedestrians’ thoughts, for instance whether they change their mind while crossing the road and decide to let the autonomous car continue on its way [28].
From the traffic point of view, it can be equally important to recognize uncertain drivers during the transition period when apart from autonomous cars there will be traditional vehicles still in use. Accomplished drivers can judge how much the driver of the preceding car is in control of their own car on the basis of the movements of their vehicle. If the driver of the car in front of you shows signs of uncertainty, you try to overtake them as soon as possible or decide to fall behind and keep a longer distance between the two cars so that you can avoid the potentially dangerous situations deriving from this attitude. In case of self-driving cars, artificial intelligence could be prepared to recognize the uncertain drivers in its environment so that it could safely get far away from them as soon as possible.
Situations suitable for misleading self-driving vehicles
Some research concentrate on obstacle detection and avoidance based on reinforcement leatning [29] or There are no 100% safe systems existing and this will be true in case of autonomous cars as well. According to John S. Chen, the director of BlackBerry, his company will be able to develop a 90% safe system, but it will require continuous supervision so that that safety level can be maintained [30].
This article is not examining the malfunction of the system of autonomous cars but situations which can be suitable for confusing the artificial intelligence in the self-driving vehicles or for forcing it to make a decision which is beneficial only for the one causing the damage. The aim of this article is to make the manufacturers test these situations as well as similar situations during the developing phase so that the transportation in the next decades, which will be overrule by autonomous vehicles, will be more safe and secure.
Let’s have a look at some examples that already reveal the deficiencies of the present systems.
The problem the self-driving system of Cadillac Supercruise has is similar to that of the one Tesla had in the autumn of 2016. If the sunbeams reach it from a wrong angle, it loses its sense of locality [31].
Tesla models now are capable of keeping the lane by themselves and changing lanes if it is necessary according to their judgement. Their cameras monitor the road paint indicating the lanes to facilitate the proper use of the steering wheel. The experts of the Chinese Keen Security Labs, which is an IT security company, could make the vehicle change lanes and advance in the lane of the oncoming traffic with the help of some well-placed white spots [32].
The experts at McAfee have deceived a Tesla Model S. They stuck a 5 cm-long black tape on a road sign indicating the maximum speed limit as 35 m/h in a way that they extended the meeting point of the two arcs in the middle of the number 3 so that it could be read as an 8 under way. During the test the system detected the speed limit indicated on the road sign as 85 m/h instead of 35 m/h and immediately started accelerating [33].
Simon Weckert switched on the google maps application on 99 mobiles phones, placed them in a hand-cart that he started pulling along the street while walking. The application indicated traffic jam in the given road-section for the other drivers though the streets were empty. What should a vehicle decide to do in such a case? Should it react on the basis of the information gathered from its own sensors and cameras that show the streets are empty or on the basis of the map application that signs a traffic jam [34]?
Volvo is remanding 736340 cars because of a software problem. In cold weather these cars do not slow down if the cameras detect another vehicle, a cyclist or a pedestrian despite the fact that the cars indicate that a perilous situation has arisen. The software seems to misinterpret the signals of the installed temperature sensors and therefore it fails to activate the automatic emergency brake. Here you can see in what a danger a casual or an intended error in the software can rush the passengers of the cars into [35].
The situations I have devised can also be apt to confuse the artificial intelligence operating in autonomous cars. It can be noticed that there are situations that are even less likely to be fended off than the ones mentioned above.
The potential dangers of the changes in human behaviour
Autonomous cars will be able to recognize situations and react to them more quickly than people, making road traffic safer. Recognizing this swiftness during the transition period when traditional vehicles still take part in traffic, and trusting the fast reactions of their cars, the drivers of self-driving cars might create situations that they should never do in the presence of traditional cars. They might unexpectedly overtake them in their lanes, could outrun them at crossroads or suddenly precede them when taking the motorway. These actions can slow traffic down or even cause accidents if there is a traditional vehicle behind the autonomous car whose driver will be able to avoid accident due to the fast reaction of their car but will force the other driver to deal with a situation that they are unable to due to their slower reaction time. This way an accident involving a self-driving car can also be induced. If a car suddenly overtakes it when it is followed by a traditional car, the traditional car will run into the autonomous car. Due to its swift reactions the self-driving car will avoid crashing into the car that has preceded it and generated the perilous situation. As a result, the car that has caused the accident will remain undamaged.
Deceiving the lane tracking system
Modern vehicles are equipped with a lane tracking system that keeps the car in the lane if the road paint is of appropriate quality. A Tesla can be deceived in self-driving function. The digital light technology of Mercedes is capable of projecting various symbols or even a lane in front of the car with the help of the mirrors installed in the headlights [36]. It is possible to project a lane next to a bicycle in a similar way, thus making their presence in traffic safer. This technology can also be used to project a fake lane for an autonomous car and so it can be confused and diverted from its original route. In Europe yellow tapes on the road mark a detour. If the denoted lanes are altered with such yellow tapes, traffic lanes can be diverted into each other thus causing immense traffic jams in cities since, according to their original function, the yellow lines on the road have to be prioritised over the white lines (Fig. 1).
Yellow tapes as modified traffic lanes in Hungary.
The road sign recognition system is able to make the adaptable course control set the speed of the vehicle to the velocity restriction indicated on a road sign. A research has pointed out that these systems can be misinform, too. Now, in this research the emphasis is on the possibility of changing transform situations and that of causing accidents. If a no-entry sign that, for instance, closes off a pedestrian precinct is covered by a one-way road sign at both ends, what decision will autonomous cars make? If the white part of the no-entry sign is covered by the number 70, it will become a road sign determining speed limit at once. On the motorway if the numbers are covered on a road sign denoting the maximum speed limit as 130 km/h, it will turn into a no-entry sign. Can the traffic be stopped and traffic jams induced with this movement? The question arises whether the vehicle should trust its installed road map information or nullify it because of the signs its cameras have recognized and alter its preferment accordingly? What should have a higher antecedence? A map that is occasionally modified and that might even include fake information if it has been hacked? Or the road sign recognized at the given moment and that can be part of a deliberate hoax?
Blinding
At decision making one of the most important parts of autonomous cars is the set of data provided by its cameras. What happens if the car advancing in front of a self-driving car bespatters the field of view of the cameras in splashy weather? The windscreen can be cleaned even on the way, but the surfaces of the side cameras cannot. The windscreen can be besmirched if oil is poured on it from a vehicle passing it. Some of the sensors, like LiDAR, can be blinded in foggy or snowy weather or by a haze machine. The cameras can be made blind by laser; it is similar to the effect of the setting sun that can be a problem for vehicles today. In such case the autonomous car stops and does not continue on its way so the passengers can be made to be late for an important meeting for example or miss the departure of the train or the plane they were going to take.
Captivating, forcing a vehicle to modify way, persecution
Self-driving vehicles will be developed to side-step accidents. If a self-driving vehicle is encircled by pedestrians or cyclists, it will be uncapable to start and on SAE level 5 even the passengers of the car will have no possibility to take control of the car to be able to escape from the situation. An autonomous car can be captivated while on the way on the base of a parallel logic. If motorcyclists or conventional cars encircle it and permanently reduce their speed, they can force the autonomous vehicle to decelerate or even to stop.
By encircling an self-driving vehicle programmed to side-step accidents it can be forced, for instance, to leave the motorway and take the nearest exit, thus modifying its original route.
If traditional vehicles or motorcycles are aligned behind a self-driving vehicle and keep accelerating while advancing close enough to it, it can be made to advance faster and or it can even be pursued [37]. The question is whether it can be forced to advance at a speed that is too high for the car to deal with, so that its swift decision making and reactions are not fast enough to be able to side-step an accident. Is it allowed to install an escape function into the autonomous cars, with the help of which passengers can get out of such situations and so their lives will be saved even at the cost of damaging the car? But what happens if the passengers are in a rush and while waiting at the red traffic lights they decide to make the vehicle run into the crowd of pedestrians crossing the street? It could even make the vehicle apt to perform a terrorist attack.
Forming levels of priority to evaluate data deriving from various resources
In the future in the traffic dominated by completely autonomous vehicles it will be inevitable for the participants to communicate. This communication can be a solution for several problems, for instance the car in front of us can indicate a pothole before our vehicle detects it. In many cases it would be more advantageous if the vehicles could share their experiences of any irregularities and thus preparing the others to avoid them and so decreasing the number of possible accidents (graph 2). The question is whether they should share this information about the differences from the available map information with the central office or in case of a pothole with the company responsible for road repairs or simply with the cars nearby. The advantage of the last one is that the pressure on the central office mitigates since there is no need to inform all the vehicles in the city about that pothole and so the network traffic will diminish. In this case it is advisable to install a unit in the autonomous vehicles, which can share and receive data within a short distance via a standardized protocol. It has to be unanimously regulated what anomalies should be reported to the central office and which ones should be shared with only the surrounding vehicles. Although it is obvious that the communication between vehicles can be of great assistance in avoiding dangers, it can also pose some threat. If this communication system can be hacked, the vehicle sending false warnings can be the cause of accidents or can divert the other vehicles from their routes. To avoid this, it is worth considering what priorities should the data deriving from various resources have in the process of decision-making.
I suggest placing the map information of the car or the ones given by the central office on the lowest level since the information owned does not necessarily contain the latest data of an accident or contingent road repairs, for example when there is a sudden pipe burst, except if the information is continuously updated by the central office. But still there is a danger that the central data might be manipulated and do not reveal the actual happenings.
I would place the information received from nearby vehicles on the middle level. However, they could be altered or out of date, too, for instance the car reported to have stopped and stayed in the middle of the road has already left the scene by the time we get there.
I would place the information detected by the own sensors of the autonomous car on the top level since they are up-to-date, reflecting the actual conditions (Table 1).
Priority levels of different data sources in autonomous vehicles
Priority levels of different data sources in autonomous vehicles
It might also happen that the information received from various resources significantly differ from each other, for example according to the map the maximum speed limit is 130 km/h, according to the nearby vehicles the cars advancing in the front, which are traditional vehicles that are out of the scope of the sensors, move at the speed of 70 km/h and the camera detects a no-entry sign.
In this case the vehicle could be subjected to the manipulation discussed above, which will pose new threats to the passengers and the environment in case of an accident. I suggest that before the data deriving from various resources are transmitted to decision-making, there should be another module based on artificial intelligence such as External Manipulation Recognition System (EMRS) installed in the vehicles, which could be taught by the manufacturers themselves considering their own willingness to take risks (Fig. 2). The tasks of this module should be to recognise the dangers imposed by external manipulation and to make suggestions on the evaluation of the different data from various sources by recognising the situation. In such cases it could also happen that the priority order of the data resources, which has already been suggested, can be altered in the interest of safe advancement.
The place of the External manipulation system in the system of autonomous vehicles.
The fuzzy logic is a wide use technology to solve different mathematical problems [38–40], but we can use it in healthcare [41] and in special case for investment for winemaking too [42]. The operation of the module recognising external manipulation would be based on fuzzy logic since a certain situation is not always clear enough to enable us to make a decision by the aid of simple computer logic. The knowledge of this module could be updated at the occasional servicing visits while adopting the appropriate security measures.
It means not only the steering control could be based on Fuzzy logic [43] and the safety evaluation model for smart driverless car [44], but also we can use the fuzzy technology for recognizing the external manipulations too. The knowledge of this unit can be updated during a service visit with appropriate security measures.
The aim of writing this study was to draw the attention of the manufacturers producing autonomous vehicles to situations that could be perilous by influencing the decision-making process of the central unit. In view of this, dealing with such situations and recognizing situations that enable external manipulation can be prepared for thus making the advancement of self-driving cars safer in traffic.
Conflicting information provided by various resources are also suitable for confusing the central unit. In order to avoid this a priority system grading the various data resources has been set up.
The module that recognizes external manipulation can be taught by the factories and updated while servicing. The suggested area for the installation of this module is the area between the sensors and the central unit in the system of the autonomous vehicles.