Abstract
The allocation of resources in the cloud environment is efficient and vital, as it directly impacts versatility and operational expenses. Containers, like virtualization technology, are gaining popularity due to their low overhead when compared to traditional virtual machines and portability. The resource allocation methodologies in the containerized cloud are intended to dynamically or statically allocate the available pool of resources such as CPU, memory, disk, and so on to users. Despite the enormous popularity of containers in cloud computing, no systematic survey of container scheduling techniques exists. In this survey, an outline of the present works on resource allocation in the containerized cloud correlative is discussed. In this work, 64 research papers are reviewed for a better understanding of resource allocation, management, and scheduling. Further, to add extra worth to this research work, the performance of the collected papers is investigated in terms of various performance measures. Along with this, the weakness of the existing resource allocation algorithms is provided, which makes the researchers to investigate with novel algorithms or techniques.
Keywords
Nomenclature
Ant colony optimization algorithms Advanced Message Queueing Protocol Genetic Algorithms Application Oriented Docker Container Accelerated Particle Swarm Optimization Binary Quadratic Programming Container As A Service Cloud-Based Applications Cloud Container Platform Communication Efficient Framework Container Management Service Central processing unit Container Resource Allocation Mechanism Customized Rider Optimization Algorithm Data Center Network Data Centers’ Elastic Application Container Energy-Efficient Container-Based Scheduling Elastic Provisioning Of Virtual Machines For Container Deployment Fuzzy Inference System Genetic Algorithms Good Fit Geo-Aware Multiagent Task Allocation Approach Genetic Programming Hyper-Heuristic Graphics Processing Units Input-output Internet Of Things Information Technology Operations Lion Algorithm Large Fit Mixed Integer Linear Program Myocardial perfusion imaging Normalized Instance Latency Node Selection algorithm for container deployment Platform-As-A-Service Physical Machines Particle swarm optimization Resource Allocation Random Access Memory Resource Allocation Strategy Start Delay Software-defined networking Service-Level Agreement SDN-Based Ngroups Management Scheme Virtual Centralized Processing Unit Vienna Platform For Elastic Processes On Containers Virtual Machine Velocity-Updated Grey Wolf Optimization Workload Aware Energy Efficient Container Whale Optimization Algorithm Whale Random Update Assisted Lion Algorithm Yet Another Resource Negotiator
Introduction
In recent days, cloud computing is a trending computing paradigm. It is in great demand among the users for storage and infrastructure services, as it provides “reliable, customized, and QoS (Quality of Service) guaranteed computing” [4, 52, 58, 62, 67, 83]. The principle behind cloud computing is that the user data is stored in the data center and not locally on the internet [6, 7, 53]. Cloud computing is defined as “A model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction” [9, 20, 37]. The resources of the cloud are accessed with three models: (a) Software as a Service, in which the cloud-based applications are made available over a network to the users; (b) Platform as a Service, in which the programming tools are utilized for creating applications and (c) Infrastructure as a Service, in which the computing resources are provisioned by the operators and are arbitrarily deployed as services. In grid computing, scalable resources are especially more common, and here, the resources are utilized on a pay-per-use basis. In cloud computing, “RA is the process of assigning available resources to the needed cloud applications over the internet”. Further, if the resources allocation is not managed properly, they starve services, and it can be solved by allowing the service providers to manage the resources for each individual module [1, 29, 62, 67, 73]. Advancements in large data management, hardware, and cloud computing administration have enabled the development of deep learning algorithms that have shown to be extremely effective [2, 60, 65].
In the cloud, the applications can be managed successfully by vitalizing applications with the lightweight technology, the Containers [20, 26, 34, 54, 76]. Ciborowski says, “Due to the nature of how container images are created (highly automated build, test, and release of apps and dependencies) and the velocity of container creation and scheduling (deployment), existing IT operations processes are unequipped to support container delivery beyond simple use cases”. The container is controversy to VM in case of” container is more lightweight than a VM” [15, 62, 81], the VM encapsulates “a full set of OS, and different VMs do not share the OS”. But the only the executables are enclosed within the container and the same Operating System (OS) is shared by the same machine by different containers [10, 21, 38]. The VM instruction is translated into instruction with the hypervisor in the host. But, any privileged instruction need not be executed in the containers, and here the through the system calls, the communication between OS takes place. Recently, as an alternative to traditional virtual machines, container-based techniques like Docker, OpenVZ, and LXC(Linux Containers) are more commonly used. Compared to traditional virtual machines, containers provide more flexibility and versatility to improve resource utilization [4, 14]. Tom says, “The rapid evolution of container platform components such as orchestration, storage, networking, and systems services like load balancing are making the entire stack a moving target”. In deployment and bootup stages, the Dockers container is more convenient for administrating the system and it too exhibits higher scalability than the virtual machines. In addition, the Dockers container achieves “higher CPU and memory utilization”.
Further, the near-native performance can be provided with the container technology and based on the underlying implementations and design structures, they are classified as “system containers (e.g., LXC) and application containers (e.g., Docker)” [45–47]. Alike the application, the container also runs on top of the Operating System (OS), and with their own address space, the container is isolated from each other [66]. a container packs all the necessary application-specific software’s together and the application executables and the other dependencies [22, 50, 78, 84]. Moreover, on the same physical machine, the two particular features like the namespaces and Cgroups were executed for running different containers over it. The resources allocation is controlled by the Linux kernel, which is a Linux kernel mechanism of Cgroup (control groups). It permits the system administrators for running containers to allocate the resources like “CPU, memory, network, or any combination of them”. In the cgroups, to each container, the resources are allocated and adjusted dynamically, but here the only the prescribed quantity of resources can only be used. An abstraction of the kernel resources is provided by the namespace and these resources are encapsulated, “IDs, network interfaces, host names”, such that they have possession of isolated resources for each of the containers. Further, if the processes with the same process ID are running with namespaces in different containers, there is no conflict between them. Typically, the extant resource management strategies are designed for allocating the resources to VM. So the question arises of how these strategies can be adapted to allocate the resources in containerized cloud.
The major contributions of this survey are exhibited below:
An overview of the current state of the art regarding container resource allocation is provided within the broad sense of cloud computing, complementary to existing surveys in literature.
Different methods (algorithms technique), novel machine learning and artificial intelligence algorithms and theory utilized in this framework for better resource allocation and monitoring are summarized.
Resource management allocation models are investigated in terms of various performance measures that reduce latency, communication cost, computational time, energy efficiency.
To strengthen the containerized cloud resource allocation, this survey hopefully motivates future researchers to develop smarter and secure optimal resource allocation algorithms.
The rest of the paper is organized as: Section 2 summarizes the collected papers utilized for the survey. A comprehensive review of diverse strategies for containerized clouds is depicted in Section 3. In addition, the exhibited performance analysis of resource allocation in container clouds is addressed in Section 4. The research gaps and challenges are shown in Section 5. Finally, this survey paper is concluded in Section 6.
Literature review
Related works
In 2019, Netaji et al. [52] have developed a novel optimized container resource allocation model with the aid of a new hybridized algorithm. The authors have deployed the WR-LA to make the resource allocation of the container optimally with consideration of the objectives like the “Threshold Distance, Balanced Cluster Use, System Failure, and Total Network Distance”. Particularly, they have amalgamated the WOA and the LA to form WR-LA. In 2020, Gholipour et al. [23] have introduced a novel cloud resource management procedure by merging the container and virtual machine migration approach. This approach was based on the multi-criteria decision-making method. The proposed approach was evaluated with the ContainerCloudsim simulator in terms of energy consumption, count of migrations, and SLA violation. In 2012, He et al. [27] have developed EAC as a lightweight resource management model. The EAC-oriented platform was implemented to hold up a multi-tenant cloud in terms of resource efficiency and feasibility. In 2020, Al-Moalmi et al. [3] have developed a new approach for the container and VM allocation in a CaaS-based environment based on the two optimized objectives like power consumption and resources utilization. The WOA was implied to issues related to the optimization issues. The experimentation resultant has exhibited the proposed work’s supremacy in terms of migration time, SLA violation minimization, power consumption, and VM migration.
In 2017, Tao et al. [73] have introduced FIS for resource allocation of the container globally. This allocation of the global container resource was evaluated with FIS on the status of the nodes. The evaluation of this works was done in a containerized test environment with diverse samples and various resource allocation policies. In 2019, Adhikari and Srirama [1] had constructed a new EECS strategy based on the APSO technique. The authors have selected the suitable container with minimum delay for each of the tasks using the APSO. In the cloud servers, the resources were utilized significantly in the cloud environment with EECS-based resource scheduling. A comparative evaluation was made with the proposed method over the extant models under various performances like computational time, energy consumption, CO2 emission, Temperature emission, and resource utilization. In 2016, YuSun et al. [68] have introduced ROAR (Resource Optimization, Allocation and Recommendation System) as a modeling framework for automatic optimization of the cloud resource allocation. The domain-specific language was utilized to meet the QoS requirements in the web-based applications. Further, the QoS requirements and the cost-optimal resource configuration in multiple resource configurations were derived with the code generation and the model-based analysis. In 2020, Joseph and Chandrasekaran [30] have modeled the microservice application modules as a BQP problem in the containerized cloud infrastructure. In addition, the microservices were deployed with the Interaction Graph’s interaction information with the robust heuristic approach IntMA in an interaction-aware manner.
In 2016, Ficco et al. [19] have projected a meta-heuristic approach based on the bio-inspired coral-reefs optimization paradigm for container cloud resource allocation in the cloud-data center. Further, in the cloud-data center, the cloud elasticity was modeled, and based on the objectives of the cloud provider, the resource reallocation schema was optimized. The customer requirements were the major objective. It was formalized with the fuzzy linguistic method using the Service Level Agreements. In 2016, Ciavotta et al. [15] proposed MILP based on the queuing theory as a solution to explore the multi-cloud configuration in the container clouds. In the multi-cloud application, the running costs and the development time was reduced with improved QoS using the proposed hybrid approach. In 2018, Tang et al. [72] have built the dynamic resource allocation algorithm inclusive of the resource matching and scheduling algorithm in the cloud-edge environment. In the case of the resource scheduling problem, an optimal solution was found using the tabu search algorithm. Further, concerning the task priorities, network transmission cost and the resource location, an optimal solution was explored by the resource matching algorithm for the resource matching problems. The spurious containers were added to the cloud to overcome the issues of resource allocation and matching in terms of extended QoS and reduced network delay. In 2020, Srirama et al. [67] had developed a new container-aware application scheduling strategy in a cloud environment with the auto-scaling policy. In addition, a rule-based policy was implied in the proposed work to overcome the cold start effect in the container-aware cloud environment. They have deployed the dynamic bin-packing strategy to effectively use the computing resources with the minimal count of the PMs. In the cloud data center, the computing resources wastages were minimized with the heuristic-based auto-scaling policy. In 2018, Wan et al. [78] have applied the features of the Docker containers into the cloud data centers to meet the requirement of the microservice-based applications. The apt location for placing the containers and the task assigned to the containers was determined using the suboptimal and the CEF. The proposed framework had exhibited a reduction in the cost and was proved to be much flexible.
In 2019, Celesti et al. [12] had proposed a lightweight solution referred as container virtualization in the IoT Cloud devices to boost up the of micro-service’s provisioning of the services, setup, and management capability. They evaluated the containerized microservices during the deployment of multiple concurrent containers into the IoT device. These container virtualizations have implied the overhead, which was evaluated in terms of resources partitioning. In 2018, Jiang et al. [29] have developed an online resource allocation algorithm for data centers to achieve energy efficiency and lessened SLA violations with VM consolidation. The evaluation was made regarding the migration of path length, count of migration, and servers power usage in DCNs. In 2017, Yang et al. [81] had developed a multi-policy-aware resources allocation algorithm to allocate the required quantity of resources to perform the task in public cloud extension. In addition, under the YARN, the multipolicy-aware resource scheduling optimization model was developed for performing each of the jobs in the smart computing framework of clusters. In 2019, Xu and Buyya [79] had designed BrownoutCon for energy-efficient clouds by merging the containers and the brownout. For containers management, the docker swarm based on the BrownoutCon was transparent and with the APIs, the docker swarm’s default configurations weren’t modified. The brownout-based algorithms were implemented by customizing the container’s activities dynamically to handle the minimized energy consumption and overloads. In 2019, Saleh and Mashaly [62] had introduced three main features to the CaaS with the CloudSim Tool. The virtualization techniques like the containers and the virtual machines were supported in the datacenters. In the second feature, the application cloudlets and the dynamic simulation were dynamically scheduled with the help of the simulation time. Finally, based on the tasks, the application simulation was supported by the third feature. In 2018, Salza et al. [63] had distributed GAs in the software containers, cloud orchestration and message queues. The message queues were exploited to perform the GAs tasks scheduling parallel with the introduced AMQPGA. In addition, the distributed GAs activities were executed as CBAs for deployment and execution of the tasks. An empirically assessed was accomplished with the proposed works in terms of “execution time, speedup, overhead”.
In 2013, Espadas et al.[17] have developed a tenant-based model in cloud computing infrastructures to tackle and underutilize the deployment of the SaaS platforms. The three related policies adopted in the tenant-based model were the, tenant-based isolation, tenant-based load balancing, and tenant-based VM instance allocation. As a result, the authors have created a cost-effective, scalable environment. In 2019, Ge et al. [22] had proffered a resource provisioning framework using amalgamating the dynamic and semi-static strategies for resource provisioning in the containerized MapReduce components in the cloud platform. Further, in terms of the deadline limitation and resource allocation adjustment in the scheduler, the minimal resource requirements were estimated. In 2019, Tan et al. [70] have developed a novel problem model for a “two-level container allocation problem”. The “genetic programming hyper-heuristics combined with human-designed rules” were the hybrid approach that was developed to enhance the allocation of resources efficiency. Further, for creating as well as selecting the VM rules were developed with GPHH. In addition, the selection of PM was made using the FirstFit. In 2019, Lu et al. [43] have observed the subsequent task recovery and interrupting tasks process, as they have imposed the overheads with additional physical machines and excess CPU time consumption. In addition, the non-interruptible and interruptible tasks having the resource requirements and the particular deadline were considered. The completion deadline of the tasks was censured using alleviating the task interruption overhead. The offline optimization of the resources was done with the Integer Linear Program (ILP) in the container cloud. In 2018, Rattihalli et al. [58] have utilized the apache mesos resource management system for both the academic clouds/clusters and the commercial clusters to explore a solution to the resource estimation problem. The target behind this research was to manage the right-size resources dynamically to upgrade the overall utilization in the concerned application. In addition, within the Mesos cluster, the containerized jobs were migrated for better efficiency. In 2018, Zhou et al. [84] had proffered scheduling frameworks and explored the algorithm design space for container cloud services in offline and online schedulers. The constraints corresponding to the nonconventional scheduling were handled with the compact exponential technique. Based on the couplet of the constraints, the primal solution was determined by adopting the primal-dual framework. The proposed model was computationally efficient. In 2019, Tan et al. [71] have developed a dual-chromosome GA in container-based clouds for solving the issues related to the allocation of resources. They used real-world datasets to experiment with the proposed work over the BestFit Descending algorithm and the single-chromosome GA. The resultant have exhibited better resource allocation with less energy consumption and time.
In 2018, Lahmann et al. [35] have investigated the container resource allocations in the CCP. The application’s hosting was provided with the shared container clusters of CCP with the leverage of open source Apache mesos/marathon container orchestration with the Amazon EC2 VMs. The memory statistics were captured with synthetic workloads for testing stress applications, static deployments and memory allocations of dynamic containers. In 2016, Nakagawa and Oikawa [50] had developed a new resource management method for managing the allocated resources based on the resource consumption behavior in the container-based virtualization environment. The container with overused memory during the resource allocation is detected by the presented work and provides a limitation to it. Further, the proof-of-concept exhibited that the presented work successfully attacked the problem situation while implementing it in the Linux kernel. In 2016, Ma et al. [44] have developed a new approach for managing the allocated resources with adaptive resource management in the container-based virtualization to solve the issues regarding the MPI programs in terms of load imbalance problem in the cloud. Moreover, they adjusted the CPU resource allocation dynamically to MPI processes running in the current program execution state as container instances and system resource status. Further, within the host, the resources were adjusted dynamically with intra-host level, and in between the hosts with MPI processes, the containers were migrated together with inter-host level.
In 2016, Fan et al. [18] have developed a “VM-Container hybrid hierarchical resource scheduling mechanism” for resource allocation, which classifies tasks in terms of the virtualization type and scheduling state. In addition, the initial scheduling strategy was formulated for tasks to migrate easily in different levels of allocated tasks. The nD-PSO, which was the extended version of PSO, was utilized for scheduling problem optimization. The optimization problem regarding the resource allocation was solved with the binary operator-based 2D-BPSO and the resultant had exhibited enhancement in the resource utilization efficiency. In 2019, Kandan et al. [32] have developed a novel archetype for resource-aware placement of the selected resources and SLA based on various factors. The application performance was improved by scheduling the containers effectively. Further, the proposed model had benefitted the container orchestrator and the evaluation resultant had exhibited its efficiency in terms of CPU requirement. In 2020, Vhatkar and Bhole [74] constructed an “optimal resource allocation and management model” to ensure the placement of the containers in an optimal way. The VUGWO was established by leveraging the PSO and the GWO for optimized resource allocation. The constraints like utilization of balanced cluster, system failure, threshold distance and the overall distance of the network were also utilized as a key for the optimized resource allocation. In 2016, Kang et al. [33] have projected a WEEC brokering system for optimal allocation of the energy resources to the container applications with no compromise in the performance level. The power consumption per application was considered in the multiple cloud servers with docker containers for power heterogeneity. Further, concerning the partitioning around medoids, and k-medoids algorithm, they have applied the workload clustering.
In 2015, Monsalve et al. [49] have proposed a novel solution by extending the time-slicing concept. The presented concept offered a solution for the overutilization of the dynamic CPU resources during the resource allocation in containerized cloud environments. Then, the serializing applications were suggested as a solution for the resources oversubscription for maximal resource usage. In 2019, Runsewe and Samaan [59] have developed CRAM based on the game theory. The authors have formulated the n-player non-cooperative game as a problem among the “heterogeneous containerized streaming applications”. The “optimal Nash Equilibrium state” was obtained from the evaluation.
In 2020, Li et al. [36] have developed a two-stage adaptive placement algorithm for container cloud in order to deal with the scheduling virtual resources problems. The proposed model was based on the secondary bin packing and the resultant of the proposed algorithm had exhibited better utilization of the computing resources at the data center’s. In 2017, Nardelli [51] had discussed the formulation of EVCD. The proposed approach took the resources of the virtual machine and the heterogeneity of container requirements to achieve multiple QoS metrics and flexibility. In 2014, Xu et al. [80] had adopted a resource scheduling scheme for the container virtualized cloud environments. In the container virtualized cloud’s multi-type resource sharing environment, the resource usage was provided with higher quality using the resource scheduling approach to reduce response time and resource utilization rate. In 2017, Sangpetch et al. [64] have developed Thoth, a dynamic resource management system with docker container technology for the providers of PaaS. For each application, the number of resources was appropriately adjusted dynamically and the usage of the resources was monitored automatically. In addition, the Neural Network, Q-Learning rule-based algorithm, was elected to implement the automatic scaling algorithm. The load changes were adapted with Q-Learning, the computing resources were saved with Neural Network, and the service quality was not compromised using the rule-based algorithm. In 2016, Memishi et al. [48] analyzed the containers’ utilization in the MapReduce-based cloud systems and described AdaptCont in MapReduce systems for optimal allocation of containers. In addition, Dynamic AdaptCont and Pool AdaptCont were two selection approaches based on the foundations of feedback systems. Further, for each container, the exact quantity of the resources required was computed with Dynamic AdaptCont. In contrast, from the available pool of configurations, the predefined container was selected with Pool AdaptCont. In 2018, Dziurzanski and Indrusiak [16] developed a value optimizing container allocation approach for cluster platforms when the end-users and the container’s value vary. In addition, two dynamic allocation algorithms and a baseline static were proposed. The original Docker engine was utilized in the cloud platform and it was implemented with dynamic algorithms. The proposed approaches had significantly expanded the workload. In 2017, Patil et al. [56] had developed NSCD by using docker, which is a container manager. The authors have used the dynamic resource allocation approach in the NSCD to expand the processing speed. In addition, for effectively utilizing the resource, hadoop was combined with docker. In 2020, Kanchanadevi et al. [31] have developed a novel architecture for optimal resource allocation and load balancing in cloud computing for a container. The authors have assessed the proposed architecture based on the “GF and LF”. Further, the ACO for introduced for enhancing the quantity of the dynamic Physical Machines with diminished arrangements. In 2020, Vhatkar and Bhole [75] introduced a new C-ROA for optimally allocating the container. The effects of the system performance in addition to security were taken into consideration by the authors. Further, the “threshold distance, balanced cluster use, system failure, total network distance and security” based rescaled objective function was defined. Finally, the proposed work was evaluated in terms of cost analysis as well.
In 2019, Bhardwaj and Krishna [7] developed a new paradigm for implementing the LXD/CR container-based migration technique and for evaluating its performance over the pre-copy VM migration scheme. Initially, the container was launched into the implemented LXD with liblxc using modifying the Linux kernel. The checkpoint/restore mechanism of the CRIU technique was utilized for container migration. In addition, “web server, CPU-intensive, disk I/O and database” were used for evaluating the presented work. In 2019, Netto et al. [53] had developed Kraft by amalgamating the Kubernetes with the Raft to manage the containers during the resource allocation. The Raft consensus algorithm-based container management system was executed on the Docker containers and the Kubernetes was utilized for orchestration. The resultant had exhibited lower processing power and memory. In 2016, Baresi et al. [6] have constructed Micro-Cloud for fine-grained resource allocation. Then, in a “coordinated, multi-level, and topology-aware manner”, the resource allocation was adapted automatically with meta-workflow. The proposed micro-clouds implementation was based on ECoWare, and the resultant had exhibited average response time and improved resource management. In 2015, Niu et al. [55] developed GMTA to allocate the workflows in container-based clouds and optimize the workflow execution. The dynamic workflow restructuring strategy along with the market-like agent negotiation mechanism was included in the GMTA. The resultant had exhibited a considerable level of decrement in traffic overheads and workflow makespans. In 2016, Guan et al. [25] have designed a novel AODC to minimize the deployment cost during the allocation of the resources to the DCs. This approach also assisted in automatic scaling for every variation in the workload of cloud applications. In addition, the AODC resource allocation problem was modeled concerning the available resources, application requirements and docker’s features.
In 2019, CAI et al. [8] have introduced conTuner as an adaptive tuning framework in the Docker cluster to optimize the container online resource configuration while allocating resources for new cloud-based applications. In conTuner, the candidate resource configurations were provided by the optimized configuration pool and from the pool, the appropriate optimized configuration was acquired with a configuration optimizer. The resultant of the presented work had exhibited higher prediction accuracy. In 2014, Caton et al. [11] have presented social compute cloud for allocating as well as sharing the Infrastructure Resources of the resource owners on the personal computer(s) or smart device(s) in the virtualized containers to their social network. The evaluation was made to reach the challenging question of how resources are allocated effectively on a best effort basis within the social community. The assessment was made during the social resource allocation in terms of “welfare, allocation fairness, and algorithmic runtime”. In 2016, Huang et al. [28] have proposed SNMS for container resource and VM allocation. The clients outside the container in the cloud had acquired the resources using the address of the NGroups. During the resource allocation, the traffic routing was made with SDN as a resultant there was good flexible configuration in the container’s network during the allocation and sharing of resources. In 2019, Li et al. [39] have a new approach mathematically based on the queueing theory to evaluate the container-based cloud environment during the resource allocation and low-rate DDoS attack. The resource allocation was coordinated for different users with the proposed dynamic DDoS mitigation strategy. This approach had the potential to regulate the count of serving container instances simultaneously for different users. Therefore, the quality of service was maximized and the resultant had exhibited that the presented work are capable of effectively defeating the DDoS attack in the container-based cloud environment. In 2019, LIN et al. [40] have developed a multi-objective optimization model for scheduling the allocated resources to the container-based microservice in the cloud architectures. The multi-objective optimization model was based on the ant colony algorithm that had considered the objectives like computing and storage resources utilization and the failure rate of the physical nodes and the count of microservice requests. The pheromone updating validity was ensured in the optimal path using enhancing the selection probability. The resultant of the evaluation achieved better results in terms of “cluster service reliability, cluster load balancing, and network transmission overhead”. In 2018, Waibel et al. [77] had introduced ViePEP-C to tackle the issue related to the fluctuating demands on computational resources in the fine-grained execution environment. To allocate diverse resources and to schedule them effectively, the ViePEP-C was provided with “cloud controller, monitoring and business process execution functionalities and provided a platform”. The resultant of the presented model had exhibited a reduction in the execution cost and high service level during resource allocation. In 2020, Ranjan et al. [57] have preferred container-based virtualization for energy-efficient workflow scheduling with the allocated resources in software-defined data centers. As per the client requirements, the underlying resources were allocated flexibly and during the scheduling mechanism, the scheduling decisions was made with the runtime scheduler. Further, two-way communication (to the servers and virtual machines) during the resource allocation was made with a doubly-linked list-based access mechanism. This work was assessed in terms of “makespan, execution time, fault tolerance, energy consumption.
In 2019, Zhao et al. [82] have built a predictive auto-scaler to improve the efficiency of the container auto-scaling in the Kubernetes clusters. The major reason for developing the predictive auto-scaler based on the predictive algorithm was to enhance the resource allocation in the cloud, thereby increasing the service offerings. The evaluation resultant had exhibited a reduction in the response time even under higher workloads. In 2018, Gu et al. [24] have developed GaiaGPU for computing resources and GPU memory allocation among the containers in clouds. The resource utilization was further enhanced by adopting the dynamic resource and the Elastic resource allocation. During the resource allocating, the physical GPUs were partitioned by GaiaGPU into multiple virtual GPUs. Based on the request from the client-side, the virtual GPUs were assigned to the containers. The resultant of the GaiaGPU revealed more significant GPU resources allocation among containers with less overhead. In 2017, Suresh and Ra [69] had applied a hybrid access control model in a heterogeneous cloud environment for efficient resource allocation and sharing. With the custom-built containers, there was faster resource allocation. Based on the containerization, the authors have deployed an optimal resource allocation algorithm. They have utilized Collaborative Research (CRC) for both resource allocation and isolation. In 2017, Awada and Barker et al. [5] presented a new approach using the existing cloud-based CMS to enhance resource efficiency, scalability, and deployment density during their allocation in the containerized clouds. The containerized applications were orchestrated by joint optimization of containerized applications set together in the geographically distributed cloud with CMS. In 2018, Chen et al. [13] have developed an innovative cloud resource allocation mechanism by means of amalgamating the combinatorial double auction and simulated annealing algorithm. The combinatorial double auction had priced the cloud resources, and the effectiveness, and efficiency, was evaluated for the proposed resource allocation scheme by means of introducing the maximum turnover rate. In 2018, Liu and Yu [42] assessed Alibaba in a 24-hour trace (AT) for co-location and workload-based resource allocation in the containerized clouds. They have evaluated the specific batch job structure based on the SD and NIL.
In 2019, Liu et al. [41] have utilized the vCPU abstraction in virtualized systems of the cloud environment for discriminating the resource scope from the protection scope to allocate and manage the resources better. Then, within the resource scope, the vCPUs contained were designated by a domain that had incurred all the CPU consumption. This research had reduced the overhead by eliminating the explicit communications in between the domains and the hypervisor. In 2018, Saha et al. [61] have undergone a performance evaluation in the Chameleon cloud for “Docker and Singularity on bare metal nodes”. First, they have mapped the Docker containers with InfiniBand hardware and RDMA communication. Then, for the allocated resources, the container-ready orchestration tools were utilized to manage the resources optimally to the containers based on the parallel workloads.
Chronological review
Figure 1 demonstrates the contribution of the existing literature works to make this survey successful. The containerized cloud-based resource allocation is new to cloud computing, so very few researchers were undergone in the early periods like 2012–2015. As time goes, it has become a hot topic these days and the count of researchers tends to reach an uncountable range in the future. To build this survey as an interesting and useful one, the research works done by diverse authors during diverse time periods (i.e. 2012–2020) is collected. The contribution of the collected literature papers in the year 2012, 2013, 2014 and 2015 is 25, 2%, 3% and 3%, respectively. In the midst of 2015, the count of researchers on this subject slightly started to rise up and in the year 2016, the contribution of the collected literature papers is 17%. In addition, 20%, 30%, 12% of the contributions were given to this survey by the literature papers collected during 2017, 2018, 2019, and 2020, respectively.

Chronological review.
Resource allocation
The resource allocation is nothing but the allocation of the pool of resources like the network bandwidth, computational resources and storage for a specified consumer. The allocating resources are split into “static and dynamic allocation”. In the Static Resource Allocation strategy, the quantity of the required resources does not vary during the “lifetime of the deployed applications”. The essential resources are determined during the deployment of resources. On the other hand, the volume of the allocated resources varies during execution to satisfy the demand of the users. Further, the under-or over-provisioning occurs with the static resource allocation and fails to equalize the current demand with the volume of the allocated resources. The physical resources are utilized highly and permit server consolidation to lessen the operating costs. As per the collected literature papers concerning the subject, most researchers have been made with dynamic resource allocation. The static resource allocation was utilized in [23, 27, 32, 35, 43, 49–52, 70] and [77], respectively. Figure 2 shows the resource allocation mechanism followed in each of the literary works.

Resource allocation: dynamic or static.
During the allocation of the resources, the resources are adapted in the form of reactive control mechanism, proactive control mechanism or hybrid control mechanism. In case of the reactive control mechanism, the resources are allocated and adjusted with respect to time. In case of the proactive control mechanism, the resources are allocated and adjusted on the basis of the change in the demand. This is generally made in terms of the historical measurements. Figure 3 shows the type of resource adaptation policy implied in different research works taken into consideration. The Reactive control mechanism is followed in [1, 3, 16–19, 22, 25, 27, 30, 32, 44, 48–51, 56, 58, 62, 63, 68, 70, 73, 75, 79], and [24, 69], respectively. The Proactive control mechanism exist in [1, 5, 6, 8, 11–13, 17, 23, 28, 36, 39, 41, 42, 52, 57, 67, 72, 74, 81, 82], respectively. In addition, the hybrid approach is followed in [7, 15, 18, 29, 31, 33, 35, 43, 53, 55, 64, 71, 78, 80, 84] and [40, 77] in that order. Figure 3 depicts the resource adaptation policy.

Resource adaptation policy.
The allocation method is broadly classified based on their objectives like QoS enhancement, reduction in power consumption. The allocation methods utilized in the collected research papers taken for the survey are shown in Table 1.
Allocation methods
Allocation methods
The type of resources that is allocated for each of the uses based on their demands in various research works taken into consideration are shown in Table 2. “The definition of a resource is very important as anything, such as CPU, memory, storage, bandwidth, and application can be termed an ICT resource in cloud computing landscape”. The CPU resource is used in almost all the collected research papers.
Resource type
Resource type
The objective function can be a mathematical expression, metric, or function that needs optimization with conditions subject to system constraints. The objective function can either single or composite on the basis of the parameters. The single objective function is utilized in [5, 6, 11–13, 16, 18, 24, 29, 33, 35, 36, 42, 53, 62, 64, 67, 69, 71, 73, 78, 80, 82, 84]. The composite objective function based resource allocation is followed in [1, 1, 3, 3, 7, 8, 12, 15, 15, 17–19, 19, 22, 22, 23, 25, 27, 27, 29, 30, 30–32, 35, 43, 44, 44, 48, 50–52, 55, 56, 58, 59, 62, 63, 63, 67, 68, 68, 70, 70–72, 72, 73, 73–75, 78, 79, 79, 81, 81, 84], respectively. Figure 4 depicts the objective function based resource allocation mechanism.

Objective function based resource allocation.
The allocation operation are classified as: “(a) service migration and (b) service shutdown”. The service migration is defined as the process of transferring the application component’s local data and process states into a new CPU resource. The Service shutdown is said to be the mechanism of “automatic switching/powering off the system, hardware component, or network on the failure or abnormal termination of the previously active resource allocation”. The applicability domain of the optimization technique is said to be interoperability, and it is classified as single or multiple. The single or homogeneous resource allocation technique effect the energy efficiency on a “single datacenter resource type”. The resource dependency says about the allocation of the resources, on which basis it was accomplished. The type of container in which the resource allocation occurs is grouped on the type of container section. The Docker container is the most commonly used one. The allocation operation followed Interoperability and Resource type in the collected papers is shown in Table 3.
Allocation operation, interoperability, container type and resource
type
Allocation operation, interoperability, container type and resource type
(Continued)
Performance measures
The performance measures play a crucial role in exhibiting the supremacy of each of the works. It is taken as major criteria and the resultants acquired with each model are grouped in Table 4. The ESM improvement (%) = 99.9 in [23]. In the idle mode, the power consumed during the resource allocation is 133.2(watt) in [3]. The average latency (ms) is 11 in [68] and it is the lowest one. The lowest overhead is recorded in [70] as 0.05. The deadline hit rate under the different proportion of interruptible tasks = 0.995. The best MIPS (million instructions per second) is recorded in [36] as 37274. The lowest response time of 30 sec is recorded in [80]. The highest acceptance ratio is achieved in [25] as 100%. The proportion of low-rate DDoS malicious requests in [39] is 100%.
Performance measures
Performance measures
(Continued)
The rate at which a service processes requests is known as throughput. This can be calculated as the number of requests handled per unit of time. Several factors influence the rate at which requests move through the system, including calculation time, communication latencies, and the resource capacity of the nodes on which the requests are processed. The comparison of throughput measures utilized in various research works in cloud computing is shown in Table 5. From the analysis, it can be noticed that the ROAR [68] attains the highest throughput of 24050req/min. Similarly, the lowest throughput of 8.3843 (ops) was accomplished by IntMA in [30].
Comparative analysis of throughput measures
Comparative analysis of throughput measures
The time spent by the system executing a task, including time spent executing runtime or system services on its behalf, refers to the task’s execution time or CPU time. Table 6 shows the execution time of diverse research works in cloud computing. From the table, it can be noticed that the online resource allocation algorithm deployed in [29] takes less time of 1.47 ms. In addition, the highest run time of 1000 sec is attained by ConTuner in [8].
Comparative analysis on execution time
Comparative analysis on execution time
Various approaches developed by diverse authors are shown in Table 5. In certain cases, they have used the existing models to upgrade the resource allocation ability. Apart from this, in most of the works, new techniques or algorithms are introduced. From the Table, it is analyzed that container-based approaches have been presented in most of the research work, such as, Joint VM and container consolidation approach [23], Elastic Application Container (EAC) [27], ContainerCloudSim [62], ECMS [58], Container-Based Application approach [35], Workload aware Energy Efficient Container (WEEC) [33], Elastic provisioning of Virtual machines for Container Deployment (EVCD) [51], NSCD [56], and Application Oriented Docker Container (AODC) [25]. Further, online resource allocation algorithm, behavior-based memory resource management, resource-aware placement and SLA are established in [29, 50], and [32]. Further, MapReduce was presented in very less count by authors in [22, 81], and [48].
Analysis on different approaches and techniques deployed
Analysis on different approaches and techniques deployed
The optimization algorithm being new to the field, it has been used only in a very smaller count. The WR-LA is utilized in [52]. The WOA is deployed in [3], C-ROA in [75] and ACO in [31, 40]. The APSO technique is introduced in [1] and velocity-updated grey wolf optimization in [74]. GA is utilized for optimal container resource allocation in [63, 72], and [71]. The ROAR is utilized in [68] for cloud resource allocation in containers. The optimization algorithms deployed are shown in Fig. 5.

Optimization concept based resource allocation in a container cloud.
In the field of cloud computing, one fundamental technology is the virtual resources
that are being provided by the cloud providers in the form of VMs. In cloud
environments, most of the research related to resource allocation focuses only on
CPU-based resource allocation, which tends to overuse memory and disk spaces. The
research gap and challenges are as follows: The container technology can serve the application with
low latency by bringing the cloud closer to the end-users and thereby
reducing the network connection. But, in most cases, the resources are
not allocated dynamically, so they cannot undergo
migration. In containerized applications, the
dynamic resource allocation is costly owing to its costly migration of
the resources. The existing containers
being lightweight and portable, do not support live migration and may
cause significant delays, and as a consequence of this, the downtime is
higher. Moreover, during the migration of
the containers for efficient resource allocation, it is essential to
select a feasible destination host. When
the containers run on OS directly in the physical machine, the overhead
needs to be eliminated as it tends to expand the resource
utilization. If the resource allocation is
done effectively, it can support multiple actors and can aid in
lessening the unprovisioned hardware and power consumption. For this,
multi-objective-based optimization techniques can be a promising
solution. During the inclusion of the
container in the clouds, security is considered as an obstacle. Certain
protection mechanisms like the “host-based intrusion detection systems
and the security hardening mechanisms” can be applied to enhance the
security of containers during
allocation. However, adding these
protection mechanisms to the containerization, clouds tend to introduce
additional overhead and degrade the container environment’s scalability.
Therefore, the cloud users need to be satisfied with the SLAs agreed
provisioning of virtual infrastructure in the public
cloud. The DMA remapping needs more information
regarding cloud governance for better resource allocation. In addition,
the remote attestation model needs to focus more on the adaptation of
container images to achieve better reliability with minimized overhead
during allocation. Major trader’s containers like Amazon, Google,
Microsoft are used as an alternative to the virtual machine due to its
independent updating, simpler orchestration and better resource
utilization. These containers utilize the
“memory and computing resources” for efficient communication in the
hosting machine. The objective-based optimization algorithms can be
suggested as a solution for this issue.
Conclusion
This paper had undergone a survey on the most recent works regarding the allocation of the resources in the container. An overview of 64 research works on container-based resource allocation of diverse authors from 2012-2020 was summarized. Different methods (algorithms or techniques) and theories have been utilized for allocating the resources to containers are discussed. The review section was made in terms of diverse metrics like: resource allocation type, allocation method, type of resources allocated, adaptation policy, and objective function-based resource allocation, allocation operation and interoperability. Different optimization models used for the allocation have also been reviewed. Finally, research gaps and challenges in this field are elaborated to assure better contributions.
