Multi-agent simulation of autonomous industrial vehicle fleets: Towards dynamic task allocation in V2X cooperation mode

Abstract

The smart factory leads to a strong digitalization of industrial processes and continuous communication between the systems integrated into the production, storage, and supply chains. One of the research areas in Industry 4.0 is the possibility of using autonomous and/or intelligent industrial vehicles. The optimization of the management of the tasks allocated to these vehicles with adaptive behaviours, as well as the increase in vehicle-to-everything communications (V2X) make it possible to develop collective and adaptive intelligence for these vehicles, often grouped in fleets. Task allocation and scheduling are often managed centrally. The requirements for flexibility, robustness, and scalability lead to the consideration of decentralized mechanisms to react to unexpected situations. However, before being definitively adopted, decentralization must first be modelled and then simulated. Thus, we use a multi-agent simulation to test the proposed dynamic task (re)allocation process. A set of problematic situations for the circulation of autonomous industrial vehicles in areas such as smart warehouses (obstacles, breakdowns, etc.) has been identified. These problematic situations could disrupt or harm the successful completion of the process of dynamic (re)allocation of tasks. We have therefore defined scenarios involving them in order to demonstrate through simulation that the process remains reliable. The simulation of new problematic situations also allows us to extend the potential of this process, which we discuss at the end of the article.

Keywords

Multi-agent planning cooperative mobile robot collective problem solving multi-agent simulation communication V2X

1. Introduction

The smart factory leads to a strong digitalization of industrial processes [1, 2], but also to continuous communication between the different tools, systems and workstations, integrated into production, storage and supply chains. Among the challenges of Industry 4.0 are the development and optimization of data, product and material flows [3]. Certain technological bricks have been defined [4], in particular for the use of autonomous mobile systems [5]: automated guided vehicle (AGV), autonomous industrial vehicle (AIV), and other autonomous/intelligent mobile robot [6].

The deployment of fleets of AIVs is problematic on several levels: the location of the vehicles, the fluidity of traffic, the perception of disturbances in the environment (dynamics), the vehicle heterogeneity, the acceptability by employees and even their cooperation. Currently, to achieve their goals, vehicles have navigation autonomy linked to rails, physical or virtual beacons. They follow predetermined trajectories while detecting obstacles to avoid collisions. Their decision-making capacities are often limited to following these predetermined trajectories and stopping in the event of obstacles. During incidents, the presence of obstacles or broken down vehicles, for example, the modification of routes is ensured by the central system or a supervisor, which sends a change of mission order to the vehicles.

The level of vehicle autonomy is well characterized in the context of the road vehicle: six levels of autonomous driving have been proposed [7, 8]. However, this is not the case for autonomous industrial vehicles. The autonomy of AIVs is often limited to local visibility of their environment, as well as visibility and knowledge of other AIVs operating in the same environment. The ability of AIVs in the same fleet to exchange information between themselves, with the active elements of the infrastructure that they encounter on their route, or with the human beings who operate in their environment, should improve the decision-making autonomy of these vehicles, make the solutions collectively defined more robust, and enable greater adaptability to changing traffic conditions [9].

For an AIV to be autonomous, it must be able to manage and control a set of tasks well discussed in the literature: perception [10, 11], mapping [11, 12], task allocation [13], localization [14, 15], path planning [16, 17] and path finding [18], motion planning [14], and vehicle management [19, 20]. The autonomy of an AIV fleet can be further increased if the AIVs can collectively manage and optimize the task allocation problem: that means they collectively assign the set of tasks to the set of AIVs with a high level of efficiency (for instance, minimization of energy costs or mission completion time) [13, 21].

Task allocation and planning are often managed centrally, even semi-centrally when global and local planning are differentiated [22]. For the proper functioning of autonomous and dynamic systems, the requirements of flexibility, robustness and scalability, lead to consider decentralized mechanisms to react to unexpected situations. Autonomy and decentralization are two excessively linked notions to the extent that an autonomous system operates and make decisions autonomously, and a system is decentralized if the decision, which are made, are not centrally controlled [23, 24]. The problem of task allocation [25, 26], tasks which grouped together can constitute missions, must therefore be thought of in a decentralized way [13]. However, before being definitively adopted, decentralization must first be modelled and simulated [23]. The concept of multi-agent systems is well suited to perform this type of modelling and simulation [27, 28, 29]. Furthermore, many tools have been developed to facilitate agent-based modelling and simulation (ABMS) [30]. It should also be noted that multi-agent systems are also used to physically control AGVs [31, 32, 33].

Some studies have focused on leveraging data from other autonomous vehicles within a fleet to control the physical behaviour of a specific autonomous vehicle from the same fleet. This is the case with reference [34], which proposes a suspension control strategy based on a deep learning algorithm. Also, based on deep learning algorithms, techniques for controlling connected autonomous vehicles are well-documented in the literature. In [35], the authors addressed this topic for adaptive convoy control, while [36], within the same subject, specifically considers communication failure situations.

In this article, we start by presenting a state of the art on the allocation of tasks in centralized or decentralized modes as well as on the different types and standards of V2X communication. In Section 3, we propose a dynamic task (re-)allocation process for an AIV fleet. In Section 4, we successively present the agent model designed to simulate this task (re-)allocation process, the simulation interface used and then the different test scenarios proposed in an industrial context such as a smart warehouse. We discuss the different results obtained in simulation in Section 5. Finally, we conclude on the proposed dynamic task allocation-reallocation process, and then we present different work perspectives.

2. State of the art

2.1 Task allocation for mobile multi-robots

Task allocation consists of optimally assigning a set of tasks to be performed by agents, actors, robots or processes, grouped and organized in a global system [37]. This is the case for mobile multi-robot systems [25, 26] or the AIV fleets addressed in this article [38, 39].

In global way, the multi-robot task planning includes two processes: 1) a multi-robot task decomposition that refers to how a team mission can be decomposed into several subtasks which can be completed by the robots, and 2) the multi-robot task allocation that consist to determinate how each subtask can be assigned to one robot [40]. Thus, two major objective functions are defined for analysing a solution to the multi-robot task allocation problem: the makespan (number of time steps required for all robots to reach their tasks), and sum of costs (sum of time steps requires by each robot to reach its task) [41].

In the field of mobile robotics, the taxonomy presented in [13] makes it possible to better characterize the task allocation process. A task can be assigned to one or more robots, and several tasks can be assigned to heterogeneous robots or multitasking robots. These tasks can be allocated instantly or extended over time. As many combinations as exhaustively detailed by numerous surveys on the issue of Multi-robot task allocation [26].

Task allocation in multi-robot systems is complex and the tasks themselves may have many time, precedence or resource constraints [37]. It is then necessary to determine the objectives to be optimized, in particular among: the travel cost (time, distance, fuel or battery consumption), the fitness (quality of task performing), the reward (gain of task completing), the priority (urgency of task completing), and the utility (balance between cost and fitness-reward) [13].

For reasons of flexibility, robustness and scalability, we are interested in decentralized task allocation solutions. These solutions must be able to assign tasks to a fleet of homogeneous or heterogeneous robots. Different models of solutions are proposed in the literature, mainly the following three: 1) solutions based on optimization (exact algorithms, dynamic programming, heuristics and metaheuristics) [26]; 2) solutions based on the Contract Net Protocol, particularly in the field of multi-agent systems (an initiating agent sends a call for proposals to the entire community of agents, chooses the best proposal received, then informs all the agents choice) [42]; or 3) solutions based on the market concept (announcement by an auctioneer, submission by bidders, selection by the auctioneer and award by the auctioneer) [43]. Solutions based on the market concept can easily be applied in a distributed context, where each mobile robot is able to become an auctioneer [44]. For each situation, a single mobile robot is appointed auctioneer [45]. He retains this role until the situation is definitively managed.

2.2 V2X communication and cooperation

In order to be able to successfully perform all the tasks assigned to them, AIVs must coordinate and therefore cooperate and share information about their activity and their perception of the environment. Four types of coordination can then be implemented according to increasing degrees of autonomy: centralized (with the assistance of a coordinator), negotiated (with respect for a specific negotiation protocol), agreement (with dynamic protocol) and emergent (with self-organizing algorithms) [46]. The last three coordination approaches are characterized as decentralized, they therefore require that the AIVs be connected to each other [47, 48, 49] and/or communicate frequently [32]. The link between V2X communication and cooperation, often presented from the perspective of cooperative control, is widely emphasized and extensively documented in the literature, as exemplified in [50] and [51].

The communication that takes place between vehicles is commonly referred to as vehicle-to-vehicle communication (V2V) [52]. While the literature widely covers V2V communications, it seems relevant to highlight specific variations, such as decentralized intersection traffic light synchronization [53], simulation of mixed traffic with cooperative lane changes [54], and traffic light optimization [55]. Vehicles can also request or receive information from the infrastructure, in particular to warn them of the presence of obstacles. This communication is then called vehicle-to-infrastructure (V2I) communication [52], and is highly developed in environments such as smart warehouses [56].

Many systems use these two last modes of communication to intelligently manage and coordinate connected vehicles in problematic situations such as crossing an intersection [57], accessing a highway [58], allocating a parking space in a smart car park [59], platooning and traffic flow optimization [46], or multiple vehicle cooperation and collision avoidance [60].

Communication between intelligent vehicles and pedestrians/humans is also used to increase vehicle autonomy and ensure better safety for humans in shared human-robot environments [61]. This communication is called vehicle-to-pedestrian communication (V2P) [62]. It also makes it possible to set up collaboration between humans and autonomous vehicles, for example when a vehicle has detected an obstacle that requires the intervention of a human operator to clear it. In the rest of the article, we will call vehicle-to-everything communications (V2X) all of the three communication modes presented above (V2V, V2I, and V2P) [63].

The ETSI (European Telecommunication Standard Institute) has defined standard communication messages for intelligent transport systems (ITS), which we have transposed and used in previous works [64]. The Decentralized Environmental Notification Message (DENM) (ETSI standard EN 302 637-3 [65]) are alert messages, issued at the time of an unexpected event in order to notify it, and therefore to cooperate by broadcasting information in the geographical area concerned. The Cooperative Perception Messages (CPM) (ETSI standard TR 103 562 [66]) provide warning and help vehicles make decisions on their route. Thus, if an AIV detects an obstacle obstructing an aisle of a warehouse, it can signal it to the other AIVs which can then recalculate and plan a new route, if necessary to be able to accomplish their tasks. An infrastructure camera can also detect obstacles and send this type of message to AIVs. Note that beyond AIVs, recent literature suggests various types of V2V communications. For instance [67], proposes, in the field of urban transportation, a dynamic control strategy for a fleet of connected buses using a multi-agent system that leverages bus history along with traffic information. This approach enables each bus to adjust its movement based on weighted information from downstream buses.

Another type of V2V communication could be useful to improve cooperation between AIVs in carrying out their tasks. Indeed, if an AIV finds itself blocked by obstacles in a warehouse aisle, broken down or generally unable to perform the task in progress, it sends a DENM message by default. It could then be useful for him to send a cooperative message to delegate the realization of his task with the necessary information.

We therefore propose a new Cooperative Task Message (CTM), which would allow in particular delegating a task. Häfner et al. [68] propose a protocol with four new types of messages, including the Cooperative Response Message (CRM) for transmitting the response to a request for cooperation. The AIV agents, modelled in the simulation part of this article (cf. §4), will use this type of message in feedback from the CTM messages to signify their agreement to take charge of a task for example.

3. Tasking process for AIV fleets

The common objective of the AIVs belonging to the same fleet is to perform all the tasks assigned to them while respecting a certain number of time and priority constraints. In this context, given $T$ a set of tasks to be performed, a task $\tau\in T$ is defined by the following tuple Eq. (1):

$\displaystyle\tau=\,<\tau_{\textit{id}},\kappa,p_{\textit{start}},p_{\textit{% end}},t_{\textit{start}},t_{\textit{end}},\textit{pr},\sigma>$ (1)

Where $\tau_{\textit{id}}$ is the task identifier; $\kappa$ is the task category (for example in a warehouse, move goods, $i . e$ .: “load the goods at a starting point to bring it to an ending point”); $p_{\textit{start}}$ is the task starting point; $p_{\textit{end}}$ is the task ending point; $t_{\textit{start}}$ is the task starting time; $t_{\textit{end}}$ is the task ending time; pr is the task priority; and $\sigma$ is the task status.

A set of $n$ tasks can be grouped to form a mission $m$ defined as follows Eq. (2):

$\displaystyle m=\,<\tau_{1},\ldots,\tau_{n}>$ (2)

Given two sets $V$ and $T$ , such that $V=\{v_{1},\ldots,v_{n}\}$ is a set of $n$ AIVs and $T=\{\tau_{1},\ldots,\tau_{m}\}$ is a set of $m$ tasks, we define the two following functions: the function $C$ which allows to calculate the cost $c_{{}_{i},v_{j}}\in C$ of performing a task $\tau_{i}$ by an AIV $v_{j}$ Eq. (3), and the function ${A}$ which allows to allocate each task $\tau_{i}\in T$ to an AIV $v_{j}\in V$ , depending on the cost $c_{{}_{i},v_{j}}$ Eq. (4).

$\displaystyle{C}:T,V\to C$ (3)

$\displaystyle{A}:T,V,C\to A$ (4)

Figure 1.

Task allocation process.

The task allocation process that we have defined is based on a market model type solution [45]. The flexibility of this solution allows a good adaptation for a decentralized system. In the rest of the article, we will apply and test this task allocation process to a fleet of homogeneous mobile robots, loading and unloading goods in a warehouse. Its process is depicted in Fig. 1.

When a task has been defined by an organizational actor (call supervisor in the rest of the article), it is sent by the supervisor using a CTM message to an available AIV ( $i . e .$ , having no tasks to be accomplished at this time), who will play the role of auctioneer. Then, this AIV auctioneer will send an acknowledgment using a CRM message, upon receipt of the work package.

For greater efficiency, before starting the auctions, the AIV auctioneer can cluster certain tasks received. This involves, in particular, associating tasks having ending points and starting points in common. For example, consider the task $\tau_{1}$ “Bring material from a parking source node n^∘21, to a storage point node n^∘13” ( $\tau_{1}=$ [21, 13]). An associated task $\tau_{2}$ could be to go to a parking lot after completing task $\tau_{1}$ , for instance: “Take material from storage point node n^∘13 and bring it to parking node n^∘25” ( $\tau_{2}=$ [13, 25]). Then, the two clustered tasks $\tau_{1}$ and $\tau_{2}$ are represented by the mission $m_{i}=$ < $\tau_{1}$ , $\tau_{2}$ > $=$ <[21, 13], [13, 25]> (in the tables of section §5, we simplify the writing of the mission $m_{i}$ by using the triplet <21, 13, 25>).

Following the association of tasks to form missions, these are sent to all vehicles by the AIV auctioneer. Then, AIVs calculate costs to perform the various missions, taking into account a set of performance indicators (distance, energy, time, etc.), and produce their bids. Each AIV returns all of its bids to the AIV auctioneer. This runs a simple optimization algorithm: the clustered tasks are listed in priority order. Subsequently, the AIV that bid with the lowest cost for a mission wins the auction (strategy of choice by the auctioneer for the least expensive proposal).

Figure 2.

CTM and CRM messages exchanged during task allocation.

To summarize at the communication level, the AIV auctioneer allocates a task (eventually clustered tasks) to each chosen AIV, sending to it via a CTM message. The receipt of this CTM message by an AIV ends with a CRM message sent to the auctioneer in order to inform him of his acceptance. The different interactions between the supervisor, the AIV auctioneer and the other AIVs are identified in the sequence diagram of Fig. 2. This allocation mechanism is also used for task reallocation. Indeed, a robot can re-auction a task becoming an auctioneer in turn to manage the reallocation of all or part of its tasks. The robot offering the best bid will then add the reallocated task(s) to the set of tasks it must perform.

4. Multi-agent simulation of AIV fleets

4.1 Agent model for AIVs

Many simulation-based approaches have been proposed in literature in the context of Industry 4.0, mainly: Agent-based modelling and simulation, Discrete Event Simulation, System Dynamics, Virtual Reality, Augmented Reality, Artificial Intelligence, Petri Nets simulation, Hybrid Simulation (characterized by the combination of two or more simulation methods, i.e., multi-paradigm model), Digital Twins, or Virtual Commissioning [23]. Among these approaches, the use of the agent paradigm to simulate or model complex, interactive, adaptive, distributed or cooperative systems has become common [27]. Indeed, the properties for which the agent-based approach is most suitable are: modularity [69], decentralization [24, 28], autonomy [30], flexibility [70] and agility [23]. Agent-based systems (ABS) have thus been proposed in many engineering fields, such as for industrial applications [71], for intelligent manufacturing [72, 73, 74], for supply chain management [75], for autonomous vehicles [29, 76], or for AGV [31]. In this last area, the case study “AGV control in an industrial bakery” proposed by Mes and Gerrits is well representative of the developments of ABS to control AGVs in the context of industry 4.0 [33]. The distribution of agents (decentralization) allows the systems that implement them to be more flexible and reactive. On the other hand, the agent concept is well suited for modelling and simulating cyber-physical systems [77] including AGVs or AIVs.

The basic definition of an agent is something that acts, to which it is useful to add three key properties: autonomous, interactive and adaptive. Thus, an agent is an autonomous entity that can adapt to and interact with its environment or other agents [42].

Other properties can be associated with the concept of agent: situated, social, flexible, proactive, and robust [33]; but also, mobile, intelligent, rational, temporally continuous, coordinative, cooperative, competitive, rugged (able to deal with errors and incomplete data robustly) [78]. These different properties will be discussed in Section 5, in relation to the results obtained during the simulation of the scenarios proposed to illustrate this article.

The basic behaviour of an agent can be modelled by an automaton [79]: 1) an agent perceives inputs using detectors (sensors); 2) it analyses, processes and interprets its inputs (recognizes, normalizes, etc.); 3) it determines the actions to be performed by interpreting the data, examining its goals and its current state; then 4) it performs the actions through effectors (its outputs).

The design of an agent-based system (ABS), often studied from a processual or methodological point of view [80], supposes that the designer proceeds with a local vision to respect the fact that each agent manages its own knowledge and actions (autonomy). ABS design support languages are numerous [13, 81]. The support tools, both for agent-based modelling and for agent-based simulation, are also very numerous: JADE, GAMMA, Matlab, MASON, NetLogo, AnyLogic, SN2, Sinalgo, etc. [82, 83, 84].

In [85], we proposed a four-step ABS design method that largely refers to AUML (Agent Unified Modelling Language) [86]:

•
Step 1. Definition of use case diagrams (services provided by ABS).
•
Step 2. For each use case, draw sequence diagrams representing the interactions (exchanges of messages and scheduling) between the agents involved.
•
Step 3. From the sequence diagrams, which identify the agents, the objects and their interactions, create a class diagram: the objects are associated with classes, the messages exchanged (requests for service between objects) are translated into operations on classes, parameters associated with operations are translated into class attributes – it may be possible to complete this diagram with a collaboration diagram.
•
Step 4. From the class diagram, define the behaviours of each agent using a state or activity diagram. These behaviours integrate a whole set of functionalities fulfilled by the agents. As an example, Mes and Guerrits specify a set of six generic functionalities to be fulfilled by AGV agents [33]: DemandManagement, ParkManagement, VehicleScheduling, VehicleRouting, ConflictResolution and BatteryManagement.

Figure 3.
Simulator architecture: dynamic elements in red, static in green, and not related to the environment in purple.

Figure 4.
a) Representation of the circuit, and b) directed graph corresponding to the circuit.

Figure 3 presents the agent architecture developed to simulate the AIV traffic situations that we wish to study. Each agent of this architecture has its own knowledge (in particular on the other AIV agents, on the traffic environment, as well as on the paths and tasks allocated), and has functional capacities of observation, communication, decision and of action [39].

In this architecture, an industrial infrastructure (production lines, warehouses, etc.) is deployed in an environment, and is composed of a circuit and active elements such as beacons, tags, stations and cameras. These active elements are modelled as agents. Static or dynamic obstacles (e.g. human operators) may be present in the environment. Human workers or operators, with whom AIV agents can communicate to carry out cooperative activities, are also modelled as agents.

AIV agents, grouped in the same agent community, perform missions defined by paths on the traffic map. They are equipped with a radar that allows them to evolve in a partially known environment, and can thus avoid obstacles and collisions between them. The internal architecture of these agents is defined in such a way as to allow them to manage the four essential functions of an autonomous mobile robot: perception, localization, planning and control [87].

Figure 5.
Multi-agent simulation interface.

AIV agents cooperate with each other to optimize the performance of a set of missions, which are transmitted to them by a supervising agent who acts as an organizing service (these missions can be sent to the AIV in packets or in continuous flow). The active elements of the infrastructure (cameras, tags, beacons, stations, etc.) participate in the cooperation [88], in particular by contributing to the safety of AIV travel. To ensure this cooperation based on inter-agent communications in VTX mode, different types of standardized messages are used (CAM, MCM, CPM, CTM, CRM and DENM).

The environment chosen in this article, to illustrate our scenarios of problematic situations, is a typical warehouse presented in Bechtsis et al. [89]. Certainly, this environment is small, but it allows us to detail very finely and from an educational point of view all the scenarios that we have defined, in particular the 4 scenarios studied in the following sections. However, the framework presented in Fig. 3 was developed to model larger warehouses and to simulate problematic traffic situations involving a large number of AIVs (for example, it is currently used to simulate the activity of a hundred baggage conveyor robots in an airport).

The warehouse circuit shown in Fig. 4a comprises several intersections, in which the vehicles can arrive from different sides as in a warehouse. This type of traffic plan offers the different characteristics of an industrial environment and allows us to carry out simulated experimental tests, in accordance with realistic scenarios of an industrial context. Five AIV agents are integrated into this environment corresponding to the five parking spaces available in this environment. One of the major interests of simulation is to be able to test the size of the vehicle fleet. Also, if the flow of tasks proposed to the AIVs becomes too great, leading to waiting times that are too long for the allocation of these tasks, the simulation conditions could be easily adapted in the environment with the addition of new AIVs and of AIVs parking lots.

The costs in distance between the different nodes of the circuit are represented in the directed graph of Fig. 4b. They have been chosen and applied to favour certain directions of circulation. These costs are used to find the shortest paths in the graph (minimization of distance costs in this case), in order to optimize the times for performing tasks by AIV agents. The interested reader will find presentations and discussions on the different types of optimization algorithms that allow a mobile robot to determine a shortest path (Dijkstra, A, D, Genetic algorithm or Particle Swarm Optimization) in many surveys [14, 38, 90].
4.2 Multi-agent simulation interface

The simulation interface presented in Fig. 5 has been designed generically to integrate different types of traffic plans. We use and develop this simulator for various laboratory experiments, and for teaching engineering students. In order to facilitate community utilization, we are releasing the code of our project, on the Github page: https://gitlab.inria.fr/jgrosset/AIV_Simulator. Its implementation respects the agent model presented in Fig. 3.

Figure 6.

a) Simulation of the detection of an obstacle by a camera agent, b) sequence diagram of the scenario sc1.

Figure 7.

a) Simulation of an AIV agent breakdown at access point n^∘14, b) sequence diagram of the scenario sc2.

This interface is divided into five frames:

•

Frame 1: visualization of the warehouse presented in Fig. 4a. The white squares represent tag agents, used by AIV agents to locate themselves. AIV agents are visualized by coloured circles, and obstacles are surrounded by a red circle of variable size corresponding to their level of obstruction in the aisle. The four camera agents of the infrastructure are identified by a black square evoking their viewing area of the aisle.

•

Frame 2: application management and its various features. It is thus possible: 1) to simulate the four types of scenarios illustrated in the rest of the article, as well as a random scenario; 2) to generate obstacles randomly on the circuit; 3) to emulate a robot or camera failure; or 4) to view a model of the circuit with the node numbers, as in Fig. 4b, by clicking on the circuit button.

•

Frame 3: supervision of AIV agents. This frame makes it possible to visualize the missions assigned to the various AIV agents, their paths, their statuses and other information useful to the supervisor agent. When an AIV agent plays the role of auctioneer, he is graphically identifiable by a frame.

•

Frame 4: supervision of camera agents. This frame makes it possible to identify their status, their position and their detection of obstacles. When a camera agent has detected an obstacle, it is graphically identifiable by a black frame.

•

Frame 5: task supervision. This frame makes it possible to monitor the progress of the performance of the tasks allocated to the AIVs. The states of the different task attributes are updated there: task identifiers, task starting point and ending point, and task states (attribute, in progress, blocked, completed).

The traffic plan chosen, and presented in the form of a directed graph in Fig. 4b, makes it possible to start a set of problem scenarios that can be easily configured in the interface. In the following, we will focus on three scenarios called sc₁ , sc₂ and sc₃ . The missions will be named $m_{i}$ and the nodes $n_{i}$ . These different scenarios are detailed in the following subsection.

4.3 Dynamic task (re-)allocation scenarios

In [64], we proposed AIV agent blocking scenarios that highlighted the need to increase cooperation/communication between agents if we wanted to effectively manage these problems. In [39], we showed that the use of MCM, CAM and DENM messages made it possible to respond effectively to the problem of avoiding obstacles and collisions between AIV agents.

To verify the ability of AIV agents to carry out their missions while cooperatively managing the problems of obstacles or AIV agent breakdown, we have defined four new scenarios. These scenarios include five AIV agents, simulating five real AIVs of the same type, and illustrate the different types of V2X cooperation/communication that allow agents to anticipate problems and thus improve the performance of their missions collectively: V2I for the first scenario, V2V for the second, and V2P for the third. All information concerning the communication during these scenarios is resumed in the Table 10. The scenarios are run in the environment shown in Fig. 4a (a simplified warehouse). VIAs perform simple tasks: 1) load goods at a source storage point, then 2) drop them off at a destination storage point. Each AIV agent has knowledge of the environment, i.e. the position of aisles, intersections, parking lots, storage points, battery replacement points and active elements of the infrastructure such as camera agents.

Figure 8.

a) Simulation of a blocked task at access point n^∘15, b) sequence diagram of the scenario sc3.

The first scenario sc1 is represented in Fig. 6a and 6b. It makes it possible to test the contribution of the cooperation between the AIV agents and the infrastructure agents for the performance of the tasks entrusted to the AIV agents. In this scenario, the camera agent placed to monitor the area around storage point n^∘14 detects an obstacle and send a CPM message to the five AIV agents. The AIV agents, whose mission involves passing through the aisle obstructed by the obstacle, are able to re-plan their rout in advance. Thus, this cooperation with the infrastructure makes it possible to avoid waiting for an AIV agent to detect the obstacle with its LIDAR and to warn the four other AIV agents when it passes near the obstacle (therefore saving time on detection). This scenario allows measuring the performance of a collective strategy including the infrastructure compared to a collective approach based only on V2V communications between AIV agents.

The sc2 scenario corresponds to an inability for an AIV agent to complete his mission. This one can be blocked by obstacles or have a breakdown but without this preventing it from communicating (Fig. 7a and 7b). In this case, it is the dynamic task allocation mechanism presented in Section 3, which is launched to reallocate the unfinished mission. The blocked AIV agent becomes an auctioneer. He transmits all the tasks he had to perform to the four other AIV agents using a CTM message. The AIV agents bid according to their situation and the tasks they are performing, which allows the AIV auctioneer agent to make his choice for the reallocation of the tasks he cannot complete.

Scenario sc3, depicted in Fig. 8a, illustrates the ability of an AIV agent to handle the blocking problem when a task cannot be completed due to an event occurring on the warehouse circuit or in the defined environment to perform the task. It is further assumed that this blocking could not be detected by an infrastructure agent and therefore that its resolution was carried out by an AIV agent. For example, if a stock point designated as a target in the mission of an AIVi agent is inaccessible (for instance, because of the presence of several obstacles), then the AIVi agent must be able to inform the four other AIV agents that its mission cannot be carried out, using a CPM message. Subsequently, a human or an AIVj agent having the ability to clear the obstacles in the aisle can intervene in response to the request made to it by the AIVi agent, by sending a DENM message (Fig. 8b). The task that could not be performed before the human intervention is put back to auction as soon as the human has informed the AIVi agent that the aisle is clear again, by sending a CPM message. The AIVi agent then temporarily becomes an auctioneer to manage the reallocation of the task. This prevents the AIVi agent from waiting for human intervention to be able to continue its mission, and can possibly complete another tasks.

Scenario sc4 presents a situation similar to Scenario sc3, where an AIV agent encounters what seems to be an obstacle, as detected by a camera. However, in this instance, the camera’s assessment is faulty, and there is no actual obstruction present. Upon detecting the apparent obstacle, the camera notifies both the five AIV agents and the worker. However, upon closer inspection, it is revealed that the obstacle does not exist. Subsequently, the worker promptly sends two messages to the five AIV agents: a CPM message confirming the absence of an obstacle at the specified location, and a DENM message alerting about the failure of the camera agent.

5. Results and discussion

The four scenarios presented in the previous section were tested with the same dataset. The different types of V2X communications illustrated in the scenarios are summarized in the Table 9. The choice therefore fell on an allocation of tasks by packet, rather than continuously. A supervisor agent sends 10 tasks to an available AIVi agent (when the AIV agents have no more missions to perform, they inform the supervisor agent). The AIVi agent starts by clustering the tasks in missions, and then offers them up for auction.

The four scenarios were analysed with the following performance indicators:

•
tasks to be performed,
•
tasks fully completed,
•
difference between the number of tasks to be performed and the number of tasks fully performed by each AIV agent,
•
total distance covered by each AIV agent,
•
load distribution (in particular to distribute the energy costs between the different vehicles and facilitate the management of battery charging).

Table 1
Test sets for the four scenarios

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> <21,14,25> 2/2 124

AIV2 <22,13,22> <22,13,22> 2/2 116

AIV3 <23,15,21> <23,15,21> 2/2 112

AIV4 <24,14,23> <24,14,23> 2/2 114

AIV5 <25,13,24> <25,13,24> 2/2 114

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] <21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24> 10/10 580

Table 2
Obstacle obstructing an aisle, detected by the AIV4 agent

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> <21,14,25> 2/2 164

AIV2 <22,13,22> <22,13,22> 2/2 116

AIV3 <23,15,21> <23,15,21> 2/2 112

AIV4 <24,14,23> <24,14,23> 2/2 204

AIV5 <25,13,24> <25,13,24> 2/2 114

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] <21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24> 10/10 710

Table 3
Obstacle obstructing an aisle, detected by a camera agent

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> [[21,14],[14,25]] 2/2 144

AIV2 <22,13,22> [[22,13],[13,22]] 2/2 116

AIV3 <23,15,21> [[23,15],[15,21]] 2/2 112

AIV4 <24,14,23> [[24,14],[14,23]] 2/2 154

AIV5 <25,13,24> [[25,13],[13,24]] 2/2 114

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] [[21,14],[14,25]] [[22,13],[13,22]] [[23,15],[15,21]] [[24,14],[14,23]] [[25,13],[13,24]] 10/10 640

Table 1 corresponds to the performance of all the tasks by the AIV agents, without them encountering any problem. This table includes the performance indicators listed above: the tasks requested then allocated to each AIV agent, the tasks actually performed by the AIV agents, the ratio between the tasks allocated and performed, as well as the total distance covered by each AIV agent (distance in meters in this case).

Table 4
Breakdown during part 1 of the AIV2 mission

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> <21,14,25> 2/2 124

AIV2 <22,13,22> 0/2 24

AIV3 <23,15,21> <23,15,21> <22,13,22> 4/2 236

AIV4 <24,14,23> <24,14,23> 2/2 114

AIV5 <25,13,24> <25,13,24> 2/2 114

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] <21,14,25> <23,15,21> <22,13,22> <24,14,23> <25,13,24> 10/10 612

Table 5
Breakdown during part 2 of the AIV2 mission

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> <21,14,25> [13,22] 3/2 234

AIV2 <22,13,22> [22,13] 1/2 44

AIV3 <23,15,21> <23,15,21> 2/2 112

AIV4 <24,14,23> <24,14,23> 2/2 114

AIV5 <25,13,24> <25,13,24> 2/2 114

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] <21,14,25> [[13,22],[22,13]] <23,15,21> <24,14,23> <25,13,24> 10/10 618

Table 6
Stock point n^∘15 has become inaccessible

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> <21,14,25> 2/2 124

AIV2 <22,13,22> <22,13,22> 2/2 116

AIV3 <23,15,21> 0/2 22

AIV4 <24,14,23> <24,14,23> 2/2 114

AIV5 <25,13,24> <25,13,24> 2/2 114

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] <21,14,25> <22,13,22> <24,14,23> <25,13,24> 8/10 490

The results obtained during the execution of the scenario sc1 corresponding to Fig. 6a are given in Table 3. The obstacle was detected by the camera, which saves time on obstacle detection. Indeed, we tested this same scenario by disabling the camera agent. It was necessary to wait for the AIV4 agent to arrive near the obstacle for it to be detected by its LIDAR. These results appear in Table 2. Thus, the total distance covered is 640 in the case where the camera agent detects the obstacle (Table 3), and 710 if the camera agent is deactivated (Table 2). This makes it possible to verify that cooperation with the infrastructure via camera agents can save time for the detection of obstacles, in particular by anticipating problems, and thus minimize distances for the performance of the missions of AIV agents.

The scenario sc2 was simulated in two cases:

•
the delegation of a complete mission (two clustered tasks),
•
the delegation of the second part of a mission (only one task).

These delegations of mission by an AIV agent can occur when the latter is unable to perform the mission in progress, following a breakdown or a blockage in an aisle for example. Thus, in Tables 4 and 5, it is possible to observe that the AIV2 agent could not finalize its mission because its number of tasks performed is not equal to its number of tasks to be carried out. In Table 5, the AIV2 agent was able to perform one task out of two of its mission. He then started the task reallocation process, which resulted in the second uncompleted task being auctioned off. This task was won and performed by the AIV1 agent. The latter therefore perform three tasks, whereas two tasks had initially been assigned to him. The second test for task delegation corresponds to Table 5 where it is possible to see that the entire mission of the AIV2 agent has been reallocated. In this case, it was the AIV1 agent who took over the complete mission, while minimizing the overall distance covered.

Table 7
The stock point n^∘15 has become inaccessible, then cleared by a worker (human operator)

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> <21,14,25> 2/2 124

AIV2 <22,13,22> <22,13,22> 2/2 116

AIV3 <23,15,21> <23,15,21> 2/2 112

AIV4 <24,14,23> <24,14,23> 2/2 114

AIV5 <25,13,24> <25,13,24> 2/2 114

Worker <1,15,1> <1,15,1> 2/2 116

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] <21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24> 10/10 580AIVs $+$ 116Worker

Table 8
A worker checks the presence of camera-detected obstacle

Agent Requested and allocated tasks Completed tasks Ratio Distance covered

AIV1 <21,14,25> <21,14,25> 2/2 124

AIV2 <22,13,22> <22,13,22> 2/2 116

AIV3 <23,15,21> <23,15,21> 2/2 112

AIV4 <24,14,23> <24,14,23> 2/2 114

AIV5 <25,13,24> <25,13,24> 2/2 114

Worker <1,15,1> <1,15,1> 2/2 116

Global/supervisor [[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]] <21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24> 10/10 580AIVs $+$ 116Worker

Figure 9.
Simulation of a worker checking for an obstacle at storage point 15 detected by the camera, b) sequence diagram of the scenario sc4.

The results of the simulated dataset with a problem accessing a stock point appear in Table 6. They correspond to the simulation of the sc3 scenario with the blocking of stock point n^∘15 identified in Fig. 8. We can notice that the AIV3 agent who had two tasks related to the deposit or the retrieval of stock at stock point n^∘15 could not perform his tasks. Only eight of the ten tasks provided by the supervisor agent could be performed in this scenario. It is therefore necessary in this case, that an AIV agent or a human can come and unblock the situation (Fig. 8b).

Table 9
The various V2X communications in the different scenarios proposed

Scenario Problem/Solution Descriptive Figure/Result Table Communication

Sc0Nominal case AIVs perform all tasks (mission, allocation, path planning) without encountering any problems. Figure 4: Representation of the circuit
Table 1: Test sets for the 3 scenarios
I2V: Supervisor gives tasks to auctioneer AIV V2V: CTM and CRM messages for the task allocation (auctions, reception and feedback)

Sc1Obstructed aisle If an obstacle obstructs an aisle, then an AIV that has to cross this aisle must quickly replan its path. Cooperation AIV $\leftrightarrow$ infrastructure Figure 6: Simulation of the detection of an obstacle by a camera agent
Table 2 : Obstacle detected by an AIV

Table 3: Obstacle detected by a camera
I2V: – Supervisor gives tasks to auctioneer AIV $\!\!\!$     – Camera send a CPM “Obstacle detected” V2V: CTM and CRM messages for the task allocation (auctions, reception and feedback)

Sc2AIV break-down An inability for an AIV to complete its mission (obstacles or breakdown). Cooperation between the AIVs (for sharing missions). Figure 7: Simulation of an AIV breakdown at access point n^∘14 Breakdown of an AIV mission during part 1 (
Table 4 ) or part 2 ( Table 5)
I2V: Supervisor gives tasks to auctioneer AIV V2V: – CTM and CRM messages for the task allocation (auctions, reception and feedback) $\!\!$ – CTM messages to share missions

Sc3Inaccessible stock point An inability for an AIV to complete its mission due to an event occurring in the defined environment to perform the task (stock point is inaccessible). Cooperation AIV $\leftrightarrow$ Worker (to remove an obstacle). Figure 8: Simulation of the access point n^∘15 blocked
Table 6 : Stock point n^∘15 has become inaccessible

Table 7: Stock point n^∘15 has become inaccessible, and then cleared by a worker.
I2V: – Supervisor gives tasks to auctioneer AIV $\!\!\!$     – Camera send CPM: obstacle detected I2P: CPM message for an obstacle at stock point V2V: – CTM and CRM messages for the task allocation (auctions, reception, and feedback) $\!\!$ – CTM messages to share missions V2P: DENM messages for a blocking problem P2V: CPM message indicating no more obstacle

Sc4Camera failure If a camera sends false information, this information has to be checked. Cooperation AIV $\leftrightarrow$ Worker or between the AIVs (to verify information sent by the camera) Figure 9: Simulation of a worker checking for an obstacle at stock point n^∘15
Table 8: A worker checks the presence of camera-detected obstacle
I2V: – Supervisor gives tasks to auctioneer AIV $\!\!\!$     – Camera send CPM: obstacle detected I2P: CPM message for an obstacle at stock point V2V: – CTM and CRM messages for the task allocation (auctions, reception, and feedback) $\!\!$ – CTM messages to share missions V2P: DENM messages for a blocking problem P2V: – CPM message indicating no more obstacle $\!\!\!\!$      – DENM message for the camera failure

Table 10
The symbols means agent properties are captured ❬ or partially captured ❷, and the properties are: P1 – Situated; P2 –Social; P3 – Flexible; P4 – Proactive; P5 – Robust; P6 – Mobile; P7 – Intelligent; P8 – Rational; P9 – Temporally continuous; P10 – Coordinative; P11 – Cooperative; P12 – Competitive; P13 – Rugged

Scenarios Agent properties

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13

Sc0 ❬ ❬ ❷ ❷ ❬ ❷ ❬ ❬ ❬

Sc1 ❬ ❬ ❬ ❬ ❷ ❬ ❬ ❬ ❷

Sc2 ❬ ❬ ❬ ❷ ❷ ❬ ❬ ❷ ❬ ❬ ❬ ❷

Sc3 ❬ ❬ ❬ ❬ ❷ ❬ ❬ ❷ ❬ ❬ ❬ ❷

Sc4 ❬ ❬ ❷ ❷ ❬ ❬ ❬ ❬ ❬ ❬ ❬

The modified scenario, with human intervention and all tasks completed, is named sc3’. It is defined in Fig. 8b and the simulation results are presented in Table 7. Furthermore, the supervisor agent can also be informed so that it does not request the performance of other tasks related to this storage point as long as it is not accessible.

Finally, the results of the scenario sc4 presented in Fig. 9 appear in Table 8. The results obtained are the same as those in Table 7. The time differences compared to the scenario sc3 occur if there are tasks continuous tasks. Because the AIVs will continue to pass through this area, and they will not have to wait for clearance by a worker or another robot related to an obstacle, as in scenario sc3. This scenario sc4 demonstrates qualitative gains and robustness in processing. It highlights the importance of verifying information provided by the infrastructure because it can be faulty. This scenario underscores the importance of reliable infrastructure and effective communication channels among human operators, AIVs, and surveillance systems to address inaccuracies and ensure operational efficiency.

We indicated in Section 4.1 that a certain number of properties can be associated with the concept of agent: situated, social, flexible, proactive, robust, mobile, intelligent, rational, temporally continuous, coordinative, cooperative, competitive, rugged (able to deal with errors and incomplete data robustly). The four scenarios presented in this article, as well as the one proposed in a previous article [39], which we will call Sc0, make it possible to verify the relevance of agent-based simulation. Indeed, all of the above properties are addressed during the realization of these scenarios (Table 10), where the AIVs agents will:

•
carry out their missions in an environment where they will be located;
•
communicate with each other, with the infrastructure and with workers, to establish collective intelligence;
•
pursue a common objective of carrying out all tasks by cooperating with each other, with the infrastructure or with a worker;
•
re-plan their paths and missions, or reallocate tasks, if necessary;
•
listen to other AIVs and active elements of infrastructure, and continue to act even if they are blocked;
•
coordinate themselves by using an auction mechanism for the allocation of tasks;
•
collectively check possibly incorrect information and communicate with a worker to resolve any problems;
•
act even when having incomplete data when receiving information without the AIVs being able to verify it themselves.

6. Conclusion and perspectives

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	<21,14,25>	2/2	124
AIV2	<22,13,22>	<22,13,22>	2/2	116
AIV3	<23,15,21>	<23,15,21>	2/2	112
AIV4	<24,14,23>	<24,14,23>	2/2	114
AIV5	<25,13,24>	<25,13,24>	2/2	114
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	<21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24>	10/10	580

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	<21,14,25>	2/2	164
AIV2	<22,13,22>	<22,13,22>	2/2	116
AIV3	<23,15,21>	<23,15,21>	2/2	112
AIV4	<24,14,23>	<24,14,23>	2/2	204
AIV5	<25,13,24>	<25,13,24>	2/2	114
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	<21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24>	10/10	710

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	[[21,14],[14,25]]	2/2	144
AIV2	<22,13,22>	[[22,13],[13,22]]	2/2	116
AIV3	<23,15,21>	[[23,15],[15,21]]	2/2	112
AIV4	<24,14,23>	[[24,14],[14,23]]	2/2	154
AIV5	<25,13,24>	[[25,13],[13,24]]	2/2	114
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	[[21,14],[14,25]] [[22,13],[13,22]] [[23,15],[15,21]] [[24,14],[14,23]] [[25,13],[13,24]]	10/10	640

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	<21,14,25>	2/2	124
AIV2	<22,13,22>		0/2	24
AIV3	<23,15,21>	<23,15,21> <22,13,22>	4/2	236
AIV4	<24,14,23>	<24,14,23>	2/2	114
AIV5	<25,13,24>	<25,13,24>	2/2	114
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	<21,14,25> <23,15,21> <22,13,22> <24,14,23> <25,13,24>	10/10	612

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	<21,14,25> [13,22]	3/2	234
AIV2	<22,13,22>	[22,13]	1/2	44
AIV3	<23,15,21>	<23,15,21>	2/2	112
AIV4	<24,14,23>	<24,14,23>	2/2	114
AIV5	<25,13,24>	<25,13,24>	2/2	114
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	<21,14,25> [[13,22],[22,13]] <23,15,21> <24,14,23> <25,13,24>	10/10	618

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	<21,14,25>	2/2	124
AIV2	<22,13,22>	<22,13,22>	2/2	116
AIV3	<23,15,21>		0/2	22
AIV4	<24,14,23>	<24,14,23>	2/2	114
AIV5	<25,13,24>	<25,13,24>	2/2	114
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	<21,14,25> <22,13,22> <24,14,23> <25,13,24>	8/10	490

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	<21,14,25>	2/2	124
AIV2	<22,13,22>	<22,13,22>	2/2	116
AIV3	<23,15,21>	<23,15,21>	2/2	112
AIV4	<24,14,23>	<24,14,23>	2/2	114
AIV5	<25,13,24>	<25,13,24>	2/2	114
Worker	<1,15,1>	<1,15,1>	2/2	116
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	<21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24>	10/10	580AIVs $+$ 116Worker

Agent	Requested and allocated tasks	Completed tasks	Ratio	Distance covered
AIV1	<21,14,25>	<21,14,25>	2/2	124
AIV2	<22,13,22>	<22,13,22>	2/2	116
AIV3	<23,15,21>	<23,15,21>	2/2	112
AIV4	<24,14,23>	<24,14,23>	2/2	114
AIV5	<25,13,24>	<25,13,24>	2/2	114
Worker	<1,15,1>	<1,15,1>	2/2	116
Global/supervisor	[[23,15],[25,13]] [[22,13],[24,14]] [[21,14],[15,21]] [[13,24],[13,22]] [[14,23],[14,25]]	<21,14,25> <22,13,22> <23,15,21> <24,14,23> <25,13,24>	10/10	580AIVs $+$ 116Worker

Scenario	Problem/Solution	Descriptive Figure/Result Table	Communication
Sc0Nominal case	AIVs perform all tasks (mission, allocation, path planning) without encountering any problems.	Figure 4: Representation of the circuit Table 1: Test sets for the 3 scenarios	I2V: Supervisor gives tasks to auctioneer AIV V2V: CTM and CRM messages for the task allocation (auctions, reception and feedback)
Sc1Obstructed aisle	If an obstacle obstructs an aisle, then an AIV that has to cross this aisle must quickly replan its path. Cooperation AIV $\leftrightarrow$ infrastructure	Figure 6: Simulation of the detection of an obstacle by a camera agent Table 2 : Obstacle detected by an AIV Table 3: Obstacle detected by a camera	I2V: – Supervisor gives tasks to auctioneer AIV $\!\!\!$ – Camera send a CPM “Obstacle detected” V2V: CTM and CRM messages for the task allocation (auctions, reception and feedback)
Sc2AIV break-down	An inability for an AIV to complete its mission (obstacles or breakdown). Cooperation between the AIVs (for sharing missions).	Figure 7: Simulation of an AIV breakdown at access point n^∘14 Breakdown of an AIV mission during part 1 ( Table 4 ) or part 2 ( Table 5)	I2V: Supervisor gives tasks to auctioneer AIV V2V: – CTM and CRM messages for the task allocation (auctions, reception and feedback) $\!\!$ – CTM messages to share missions
Sc3Inaccessible stock point	An inability for an AIV to complete its mission due to an event occurring in the defined environment to perform the task (stock point is inaccessible). Cooperation AIV $\leftrightarrow$ Worker (to remove an obstacle).	Figure 8: Simulation of the access point n^∘15 blocked Table 6 : Stock point n^∘15 has become inaccessible Table 7: Stock point n^∘15 has become inaccessible, and then cleared by a worker.	I2V: – Supervisor gives tasks to auctioneer AIV $\!\!\!$ – Camera send CPM: obstacle detected I2P: CPM message for an obstacle at stock point V2V: – CTM and CRM messages for the task allocation (auctions, reception, and feedback) $\!\!$ – CTM messages to share missions V2P: DENM messages for a blocking problem P2V: CPM message indicating no more obstacle
Sc4Camera failure	If a camera sends false information, this information has to be checked. Cooperation AIV $\leftrightarrow$ Worker or between the AIVs (to verify information sent by the camera)	Figure 9: Simulation of a worker checking for an obstacle at stock point n^∘15 Table 8: A worker checks the presence of camera-detected obstacle	I2V: – Supervisor gives tasks to auctioneer AIV $\!\!\!$ – Camera send CPM: obstacle detected I2P: CPM message for an obstacle at stock point V2V: – CTM and CRM messages for the task allocation (auctions, reception, and feedback) $\!\!$ – CTM messages to share missions V2P: DENM messages for a blocking problem P2V: – CPM message indicating no more obstacle $\!\!\!\!$ – DENM message for the camera failure

In the context of the current smart factory, mobile robots must become increasingly autonomous in order to perform their missions effectively, i.e. optimize their activity according to performance indicators such as distances covered, energy consumed, time for perform missions, availability, etc. Autonomy and decentralization are two excessively linked notions to the extent that an autonomous system operates and make decisions autonomously, and a system is decentralized if the decision, which are made, are not centrally controlled. Therefore, we proposed a dynamic task (re-)allocation process model for autonomous industrial vehicles, managing their activity in a decentralized context. We then developed a multi-agent application to be able to simulate this process and test it on different scenarios of problematic traffic situations. The proposed scenarios allow us to move towards strong cooperation between AIV agents, but also between AIV agents and infrastructure agents (cameras, tags, beacons, etc.). The V2X communication implemented to enable this cooperation is an essential element of our decentralized agent-based simulation approach. We have shown that it brings more flexibility and robustness in the management of problematic dynamic situations. We wish to accentuate these types of cooperation to increase the autonomy of the AIVs that we use in real experiments.

The different perspectives that emerge from our work are data fusion and shared memory of AIV agents. For example, how to merge data related to the detection of an obstacle by an AIV agent and by a camera agent at different times. We want also to work on ways to verify the presence of an obstacle, for example by asking an AIV agent to go and verify the presence of it. A shared memory would allow AIV agents to have, for example, global information on task delegation requests, but also to map the environment. For this, we plan to suppress the CRM messages, and to choose to return to all AIV agents the mission assigned to them. These prospects for enhanced cooperation would make it possible to increase the autonomy and efficiency of autonomous industrial vehicles. Furthermore, we continue to develop the simulation platform to integrate fleets of heterogeneous robots, therefore with robots that will not be able to perform all the defined tasks.

Footnotes

Acknowledgments

The authors would like to thank the Brittany region for funding the VIASIC and ALPHA projects, as part respectively of the ARED-2021-2024 call for projects entitled “The economy at the service of industry for intelligent production” and the PME 2022 call for projects entitled “Accelerate time to market of digital technological innovations from SMEs in the Greater West”.

References

. Industry 4.0: A survey on technologies, applications and open research issues. Journal of Industrial Information Integration. 2017; 6: 1-10.

Vogel-Heuser

Wang

. Industry 4.0 and Industry 5.0 – Inception, conception and perception. Journal of manufacturing systems. 2021; 61: 530-535.

Liao

Deschamps

de Freitas Rocha Loures

Ramos

LFP

. Past, present and future of Industry 4.0 – a systematic literature review and research agenda proposal. International Journal of Production Research. 2017; 55: 3609-3629.

Pereira

Romero

. A review of the meanings and the implications of the Industry 4.0 concept. Procedia Manufacturing. 2017; 13: 1206-1214.

Benotsmane

Dudás

Kovács

. The Concept of Autonomous Systems in Industry 4.0. Advanced Logistic Systems-Theory and Practice. 2018; 12(1): 77-87.

Andreasson

Bouguerra

Cirillo

Dimitrov

, et al. Autonomous transport vehicles: where we are and what is missing. IEEE Robotics and Automation Magazine. 2015; 22(1): 64-75.

Wang

Zhang

Huang

Zhao

Bella

. Safety of autonomous vehicles. Journal of advanced transportation. 2020; 2020: 1-13.

Wiseman

. Autonomous vehicles. Encyclopedia of Information Science and Technology. 5th Edition, IGI Global. 2021; 1-11.

Peres

Jia

Lee

Sun

Colombo

, et al. Industrial artificial intelligence in industry 4.0-systematic review, challenges and outlook. IEEE Access. 2020; 8: 220121-220139.

10.

Rosique

Navarro

Fernández

Padilla

. A systematic review of perception system and simulators for autonomous vehicles research. Sensors. 2019; 19(1): 648.

11.

Wang

Yang

. Artificial intelligence applications in the development of autonomous vehicles: A survey. IEEE/CAA Journal of Automatica Sinica. 2020; 7(2): 315-329.

12.

Rubio

Valero

Lopis-Albert

. A review of Mobile Robots: Concepts, Methods, Theoretical Framework, and Applications. International Journal of Advanced Robotic Systems. 2019; 16(2): 1-22.

13.

De Ryck

Versteyhe

Debrouwere

. Automated guided vehicle systems, state-of-the art control algorithms and techniques. Journal of Manufacturing Systems. 2020; 54: 152-173.

14.

Zghair

NAK

Al-Araji

. A one decade survey of autonomous mobile robot systems. International Journal of Electrical and Computer Engineering. 2021; 11(6): 4891.

15.

Djoko-Kouam

Fougères

. Towards a bijective Co-simulation model between physical and virtual environments, adapted to a platform for autonomous industrial vehicles. Automation, Control and Intelligent Systems. 2023; 11(2): 27-44.

16.

García

Villar

Tan

Sedano

Chira

. An efficient Multi-Robot Path Planning solution using A* and coevolutionary algorithms. Integrated Computer-Aided Engineering. 2023; 30(1): 41-52.

17.

Rafai

ANA

Adzhar

Jaini

. A review on path planning and obstacle avoidance algorithms for Autonomous mobile robots. Journal of Robotics. 2022; 2022: 2538220.

18.

Aizat

Azmin

Rahiman

. A survey on navigation approaches for automated guided vehicle robots in dynamic surrounding. IEEE Access. 2023; 11: 33934-33955.

19.

Raj

Kos

. A comprehensive study of mobile robot: history, developments, applications, and future research perspectives. Applied Sciences. 2022; 12: 6951.

20.

Hewawasam

Ibrahim

Appuhamillage

. Past, present and future of path-planning algorithms for mobile robot navigation in dynamic environments. Industrial Electronics Society. 2022; 3: 353-365.

21.

Deng

Shi

Shen

. Toward Energy-Efficient Routing Of Multiple AGVs with multi-agent reinforcement learning. Sensors. 2023; 23(12): 5615.

22.

Cardarelli

Digani

Sabattini

Secchi

Fantuzzi

. Cooperative cloud robotics architecture for the coordination of multi-AGV systems in industrial warehouses. Mechatronics. 2017; 45: 1-13.

23.

de Paula Ferreira

Armellini

De Santa-Eulalia

. Simulation in industry 4.0: A state-of-the-art review. Computers and Industrial Engineering. 2020; 149: 106868.

24.

Tavcar

Horvath

. A review of the principles of designing smart cyber-physical systems for run-time adaptation: Learned lessons and open issues. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2019; 49: 145-158.

25.

Jia

Meng

. A survey and analysis of task allocation algorithms in multi-robot systems. IEEE int. conf. on robotics and biomimetics, ROBIO. 2013; 2280-2305.

26.

Khamis

Hussein

Elmogy

. Multi-robot task allocation: a review of the state-of-the-art. Coop Robots Sensor Netw. 2015; 2: 31-51.

27.

Macal

. Everything you need to know about agent-based modelling and simulation. J Simul. 2016; 10: 144-156.

28.

Kamdar

Paliwal

Kumar

. A state of art review on various aspects of multi-agent system. Journal of Circuits, Systems and Computers. 2018; 27(11): 1830006.

29.

Jing

Zhan

Chen

Shi

. Agent-based simulation of autonomous vehicles: A systematic literature review. IEEE Access. 2020; 8: 79089-79103.

30.

Abar

Theodoropoulos

Lemarinier

O’Hare

. Agent based modelling and simulation tools: A review of the state-of-art software. Computer Science Review. 2017; 24: 13-33.

31.

Vis

IFA

. Survey of research in the design and control of automated guided vehicle systems. Eur J Oper Res. 2006; 170(1): 677-709.

32.

Draganjac

Miklic

Kovacic

Vasiljevic

Bogdan

. Decentralized control of multi-AGV systems in autonomous warehousing applications. IEEE Trans Autom Sci Eng. 2016; 13(2): 1433-47.

33.

Mes

Gerrits

. Multi-agent systems. In: Zijm

et al. (eds). Operations, Logistics and Supply Chain Management. Lecture Notes in Logistics, Springer Nature. 2019; 611-636.

34.

Chen

Zhao

Liao

Zhu

. A hierarchical framework for improving ride comfort of autonomous vehicles by deep reinforcement learning with external knowledge. Computer-Aided Civil and Infrastructure Engineering. 2023; 38(8): 1059-1078.

35.

Zhu

Chen

Wong

. Modeling adaptive platoon and reservation-based intersection control for connected and autonomous vehicles employing deep reinforcement learning. Computer-Aided Civil and Infrastructure Engineering. 2023; 38(10): 1346-1364.

36.

Shi

Zhou

Wang

Gong

Ran

. A Deep Reinforcement Learning based distributed connected automated vehicle control under communication failure. Computer-Aided Civil and Infrastructure Engineering. 2022.; 37(15): 2033-2051.

37.

Gerkey

Matarić

. A formal analysis and taxonomy of task allocation in multi-robot systems. The International journal of robotics research. 2004; 23(9): 939-954.

38.

Karur

Sharma

Dharmatti

Siegel

. A survey of path planning algorithms for mobile robots. Vehicles. 2021; 3: 448-468.

39.

Grosset

Ndao

Fougeres

Djoko-Kouam

Couturier

Bonnin

. A cooperative approach to avoiding obstacles and collisions between autonomous industrial vehicles in a simulation platform. Integrated Computer-Aided Engineering. 2023; 30(1): 19-40.

40.

Yan

Jouandeau

Cherif

. A survey and analysis of multi-robot coordination. International Journal of Advanced Robotic Systems. 2013; 10(12): 399.

41.

Stern

Sturtevant

Felner

Koenig

, et al. Multi-agent pathfinding: Definitions, variants, and benchmarks. In Proc. of the Int. Symposium on Combinatorial Search. 2019; 10(1): 151-158.

42.

Wooldridge

. An introduction to multiagent systems. John Wiley and Sons. 2009.

43.

Dias

Zlot

Kalra

Stentz

. Market-based multirobot coordination: a survey and analysis. Proceedings of the IEEE. 2006; 94(7): 1257-1270.

44.

Daoud

Balbo

Gianessi

Picard

. ORNInA: A decentralized, auction-based multi-agent coordination in odt systems. AI Communications. 2021; 34(1): 37-53.

45.

Hussein

Khamis

. Market-based Approach to Multi-robot Task Allocation. IEEE Int. Conf. on Individual and Collective Behaviors in Robotics. 2013; 69-74.

46.

Mariani

Cabri

Zambonelli

. Coordination of autonomous vehicles: taxonomy and survey. ACM Computing Surveys. 2021; 54(1): 1-33.

47.

Zeng

Wang

Rajan

. Decentralized coordination for truck platooning. Computer-Aided Civil and Infrastructure Engineering. 2022; 37(15): 1997-2015.

48.

Ding

Dai

Fan

Zhang

. Collaborative control of traffic signal and variable guiding lane for isolated intersection under connected and automated vehicle environment. Computer-Aided Civil and Infrastructure Engineering. 2022; 37(15): 2052-2069.

49.

Zhou

Peeta

Wang

. Cooperative control of a platoon of connected autonomous vehicles and unconnected human-driven vehicles. Computer-Aided Civil and Infrastructure Engineering. 2023; 38(18): 2513-2536.

50.

Zhou

Wang

Ding

Nie

. A 2D connected automated vehicle car-following control algorithm. Computer-Aided Civil and Infrastructure Engineering. 2023; 38(18): 2560-2575.

51.

Mohebifard

Hajbabaie

. Trajectory control in roundabouts with a mixed-fleet of automated and human-driven vehicles. Computer-Aided Civil and Infrastructure Engineering. 2022; 37(15): 1959-1977.

52.

Wen

Yao

. A survey of traffic control with vehicular communication. IEEE Trans Intell Transp Syst. 2014; 15(1): 425-432.

53.

Zhang

Yang

. Signal timing at an isolated intersection under mixed traffic environment with self-organizing connected and automated vehicles. Computer-Aided Civil and Infrastructure Engineering. 2023; 38(14): 1955-1972.

54.

Huang

Halkias

McHale

James

. Simulation of mixed traffic with cooperative lane changes, computer-aided civil and infrastructure engineering. 2022; 37(15): 1978-1996.

55.

Zhang

Guo

Zhang

. Traffic signal optimization for partially observable traffic system and low penetration rate of connected vehicles. Computer-Aided Civil and Infrastructure Engineering. 2022; 37(15): 2070-2092.

56.

Youssef

El Khoreby

Issa

Abdellatif

. Brief Survey on Industry 4.0 Warehouse Management Systems. International Review on Modelling and Simulations. 2022; 15(5): 340-350.

57.

Namazi

. Intelligent intersection management systems considering autonomous vehicles: A systematic literature review. IEEE Access. 2019; 7: 91946-91965.

58.

Rios-Torres

Malikopoulos

. A survey on the coordination of connected and automated vehicles at intersections and merging at highway on-ramps. IEEE Transactions on Intelligent Transportation Systems. 2016; 18(5): 1066-1077.

59.

Lin

Rivano

Le Mouël

. A survey of smart parking solutions. IEEE Transactions on Intelligent Transportation Systems. 2017; 18(12): 3229-3253.

60.

Muzahid

AJM

Kamarulzaman

Rahman

, et al. Multiple vehicle cooperation and collision avoidance in automated vehicles: survey and an AI-enabled conceptual framework. Scientific Reports. 2023; 13(1): 603.

61.

Da̧browska

Giel

Werbińska-Wojciechowska

. Human safety in autonomous transport systems–review and case study. Journal of KONBiN. 2021; 51(1): 57-71.

62.

Qiu

. Vehicle-to-pedestrian communication modeling and collision avoiding method in connected vehicle environment. Transportation Research Record. 2017; 2621(1): 21-30.

63.

Kokuti

Hussein

Marin-Plaza

de La Escalera

Garcia

. V2X communications architecture for off-road autonomous vehicles. 2017 IEEE Int. Conf. on Vehicular Electronics and Safety (ICVES). 2017; 69-74.

64.

Grosset

Fougères

Djoko-Kouam

Bonnin

. Collective obstacle avoidance strategy – an agent-based simulation approach. Int. Conf. on Adv. in Signal Processing and Artificial Intelligence (ASPAI’2022). Corfu, Greece. 2022; 99-104.

65.

ETSI

. [DENM] ETSI EN 302 637-3 V12.1. Intelligent Transport Systems (ITS); Vehicular Communications; Basic set of Applications. Part 3: Specification of Decentralized Environmental Notification Basic Service. ETSI, Standard. 2014.

66.

ETSI, . 103 562 V21.1. Intelligent Transport Systems (ITS). Vehicular Communications; Basic set of Applications; Analysis of the Collective Perception Service (CPS); Release 2. Standard. European Telecommunications Standards Institute: Sophia Antipolis, France. 2019.

67.

Shi

Nie

Wang

Zhou

Ran

. A distributed deep reinforcement learning based integrated dynamic bus control system in a connected environment. Computer-Aided Civil and Infrastructure Engineering. 2022; 37(15): 2033-2032.

68.

Häfner

Jiru

Roscher

Ott

Schmitt

Sevilmis

. CVIP: A protocol for complex interactions among connected vehicles. IEEE Intelligent Vehicles Symposium. 2020; 510-515.

69.

Leitao

Karnouskos

Ribeiro

Lee

Strasser

Colombo

. Smart agents in industrial cyber–physical systems. Proceedings of the IEEE. 2016; 104(5): 1086-1101.

70.

Rodrigues

Oliveira

Leitao

. Decentralized and on-the-fly agent-based service reconfiguration in manufacturing systems. Computers in Industry. 2018; 101: 81-90.

71.

Leitão

Mař’ık

Vrba

. Past, present, and future of industrial agent applications. IEEE Transactions on Industrial Informatics. 2012; 9(2): 2360-2372.

72.

Monostori

Váncza

Kumara

. Agent-based systems for manufacturing. CIRP Annals. 2006; 55(2): 697-720.

73.

Leusin

Kück

Frazzon

Maldonado

Freitag

. Potential of a multi-agent system approach for production control in smart factories. IFAC-PapersOnLine. 2018; 51(11): 1459-1464.

74.

Fougères

Ostrosi

. Holonic fuzzy agents for integrated CAD product and adaptive manufacturing cell formation. Journal of Integrated Design and Process Science. 2019; 23(1): 77-102.

75.

Vieira

Dias

Santos

Pereira

Oliveira

. Supply chain data integration: A literature review. Journal of Industrial Information Integration. 2020; 19: 100161.

76.

Dong

Woodman

Jennings

Brewerton

Birrell

Higgins

. A simulation study of platooning AV fleet service in shared urban environments with uncertainties. Transportation Engineering. 2021; 4: 100062.

77.

Vogel-Heuser

Lee

Leitão

. Agents enabling cyber-physical production systems. at-Automatisierungstechnik. 2015; 63(10): 777-789.

78.

Odell

. Agent Technology: What is it and why do we care? Enterprise Architecture. 2007; 10(1): 1-25.

79.

Fougères

. Modelling and simulation of complex systems: an approach based on multi-level agents. Int J of Computer Science Issues. 2011; 8(6): 8-17.

80.

Biswas

. Towards an agent-oriented approach to conceptualization. Applied Soft Computing. 2008; 8(1): 127-139.

81.

Bergenti

Gleizes

Zambonelli

. Methodologies and Software Engineering for Agent Systems. Kluwer. 2004.

82.

Macal

North

. Tutorial on agent-based modelling and simulation. Journal of Simulation. 2010; 4: 151-162.

83.

Dorri

Kanhere

Jurdak

. Multi-Agent Systems: A survey. IEEE Access. 2018; 6: 28573-28593.

84.

Antelmi

Cordasco

d’Ambrosio

de Vinco

. Experimenting with agent-based model simulation tools. Applied Sciences. 2022; 13: 13.

85.

Fougères

. Agents to cooperate in distributed design process. Proc. of IEEE Int. Conf. on Systems, Man and Cybernetics (SMC’04). The Hague, Netherlands. 2004; 2629-2634.

86.

Odell

Parunak

HVD

Bauer

. Extending UML for agents. In Proc. of the AOIS Workshop, 17th Nat. conf. on Artificial Intelligence, Austin, Texas. 2000.

87.

Siegwart

Nourbakhsh

Scaramuzza

. Introduction to autonomous mobile robots. Cambridge, MA, USA: MIT Press. 2011.

88.

Belbachir

El Fallah-Seghrouchni

Casals

Pasin

. Smart mobility using multi-agent system. Procedia Computer Science. 2019; 151: 447-454.

89.

Bechtsis

Tsolakis

Vlachos

Srai

. Intelligent autonomous vehicles in digital supply chains: A framework for integrating innovations towards sustainable value networks. Journal of cleaner production. 2018; 181: 60-71.

90.

Qin

Shao

Wang

Jiang

Cao

. Review of autonomous path planning algorithms for mobile robots. Drones. 2023; 7: 211.