Spark’s operation time predictive in cloud computing environment based on SRC-WSVR

Abstract

In Apache Spark cloud computing environment, computation capability of each node varies, together with the data size and uncertainties in application execution, these result in the differences in task execution time of a job. In order to enhance the accuracy in load execution time prediction, and reasonably guide the user to apply for Spark cluster resources, this paper researches on the execution flow of Spark job, collects the load time consumption index, and puts forward the time index fusion calculation scheme. And then, this paper researches on the multiple linear regression model and support vector machine model to explore the payload execution time and CPU Core, inputting data volume, memory size and other performance indicators. Based on the two models above, this paper proposes a Standard Regression Coefficient-based Weighted Support Vector Regression time prediction model (SRC-WSVR). Finally, through comparing the results from the prediction model proposed in this paper with conventional regression prediction model and Standard Support Vector Machine model, it proves that SRC-WSVR has a higher prediction accuracy, which can provide valid data reference for predicting Spark resource consumption.

Keywords

Weighted Support Vector Regression Machine Standard Regression Coefficient execution time prediction performance benchmark keyword five Spark

1. Introduction

Apache Spark [26,31] is an open-source cluster computing system. Spark is a fast, in-memory data processing engine with development APIs to allow data workers to execute streaming, machine learning or SQL. Spark introduces the concept of Resilient Distributed Dataset (RDD) which is particularly suitable for iterative applications such as Iterative Machine Learning and Graph Algorithms. At the same time, this type of cluster computing system with large CPU resources is also suitable for non-iterative applications. Executing Spark systems on the cloud has been a flexible choice of some enterprises and scientific research organisations. Users have to request and allocate the optimised amount of clusters resources to fulfil the application requirements, therefore, prediction models [2,27], which can help users to estimate appropriate number of clusters resources, have to be established for reducing the resources renting cost.

The features of Big Data are as follows: Volume (Amount of Data), Velocity (Speed of Data in and out), Variety (Range of Data Types and Sources) and Value (Usefulness of Data). The rise of BigData has spawned data science, which can be understood as predictive analysis and data mining, but at the same time, due to the subjective data and error analysis, it can easily lead to data dictatorship, which is terrible and should be forbidden.

This work builds the model based on Spark data analysis history information, for predicting execution time required for non-iterative application WordCount and iterative application PageRank. By considering the fact that execution time of specific operations on Apache Spark will be affected by various factors [1,17] such as data size and type of inputs, the design and implementation of algorithm [4], and computing power (e.g. number of nodes, number of execution CPU cores, available of memory size), the model is built based on Standard Regression Coefficient [13,15], so to contrast the weight of each impact factor, then obtaining prediction result with the aid of Weighted Support Vector Regression [7,8] Machine Model. This model makes use of input data size, and the number of CPU cores allocated, to predict the execution time of Spark processes.

2. Related works

Performance prediction of Spark is the basis for the optimisation of allocating cluster resources [12,18,30]. There have been works proposing performance prediction models for currently popular cloud computing platforms. Kavulya et al. [10] used behaviour analysis of Hadoop users and K-Means algorithm to measure the similarities of jobs, then to predict Hadoop process’s execution time. However, this approach required a large amount of historical data and the result accuracy is unstable. Ganapathi et al. [6,20] used machine learning algorithm to analyses the performance of the loaded historical operation information prediction system. They extracted job’s historical operation information and proposed a resources scheduling model, to ensure jobs could finish in a specific time. Article [23] worked on the issues of execution time prediction on Network-Intensive Iterative Algorithm on MapReduce. However, it mainly focussed on the iterative algorithm, which requires representable tuning data in order to achieve high prediction accuracy. Article [32] attempted to base on distributed computing jobs [24] on heterogeneous machines and to predict job completion time based on boundary-based performance modelling. Its aim is to evaluate the upper and lower limits of task completion time to predict job performance. Article [3,19,21] proposed a Self-Adjusting Framework for predicting job’s working performance under different settings. Despite having a satisfactory prediction accuracy, its differences with Spark platform made it not suitable for predicting the performance of Spark platform. This work’s findings may not be adequate to apply on Spark as Spark has a different implementation mechanism than Hadoop, but these works results are valuable references on Spark performance prediction. Wang et al. [28] proposed a Spark performance evaluation scheme based on theoretical analysis. It sampled operational results of small data sets and to predict the performance of big data sets using theoretical derivation approach, and this gave a satisfactory accuracy. Article [5,14] evaluated the performance of a cloud workstation from the perspective of the mathematical analysis model, using impact benchmarks such as CPU, internal memory and network bandwidth to conduct an integrated evaluation of the system impact caused by different parameters in an application, with the use of a Comprehensive Fuzzy Evaluation Model. The rigorous mathematical theories in this work have a strong reference, but the work lacks the descriptions on the impact relationship between parameters and system performance.

In conclusion, this work builds on results from previous works, to integrate time consumption information benchmarks [11,22] by tracing the task execution flow in Spark, then introduce the calculation methods for job execution time boundaries, to put forward a Standard Regression Coefficient-based Weighted Support Vector Regression (SRC-WSVR) time prediction model. It first performs screening calculations of standard regression coefficient for related benchmarks, and those coefficients are used as impact weight and added to training set data parameters. A non-iterative application, WordCount and an iterative application, PageRank, are used for job execution time prediction, to verify if the results lie within the boundaries set.

3. Calculate metrics

3.1. Execution flow

In Spark platform, all application executed in serial and by stages. Data sets input are divided into clusters and scattered on nodes to fulfil the parallel processing model [16,29]. The diagram in Fig. 1 illustrates the serial process execution.

Fig. 1.

Serial Process Execution Diagram.

In the diagram, each box represents a stage, and execution of the job is divided into 3 stages. The first stage is to read the input file “textFile” from the HDFS [25], it then triggers “flatMap” operand to extract each word from each row, and then trigger sthe “Map” operand to form the key-value pair of the form (word, 1). The second stage is to make use of the “reduceByKey” operand to summarise each word’s value, then triggers the “sortBy” operand to sort the statistics results of WordCount. The last stage is mainly saving the result by the “saveAsTextFile” operand.

Hence, from the analysis of Spark operation process, it is found that sub-stages are executed one after the other like a streaming line. Detailed time consumption information for each sub-stages can be obtained from historical execution information, as illustrated in Table 1.

Table 1

Consuming Time Information

Time benchmark	Benchmark pointer
Duration	The whole process execution time
Scheduler	Delay Task scheduler delay
Task Deserialization Time	Deserialization time of each task
GC Time	Garbage collection time
Result Serialization Time	Time needed for serialising results
Shuffle Read Time	Read Time for Shuffle
Shuffle Write Time	Write Time for Shuffle
Task Tim	Task completion time in a sub-stage
Getting result Time	Compute result and return the driver time

3.2. Calculations of benchmark time

Based on the time benchmark information above, this work proposes the integration of time benchmark information in each sub-stages, forming the boundary calculation method of job execution time from the perspective of job execution process.

3.2.1. Execution time

By breaking down the Spark job execution is divided into multiple stages, with each stage containing multiple tasks, this can be represented by formula (1) and (2): $\begin{array}{l} (1) & Job = {Stage}_{i} | 0 ⩽ i ⩽ M \\ (2) & {Stage}_{i} = {Task}_{i, j} | 0 ⩽ j ⩽ N \end{array}$ where M is the number of stages in the job, each job is labelled with i. N is the number of tasks in the ith stage.

Since different stages in a job are executed in sequence order, the job’s execution time can be represented as the summation of the execution time of stages, start time and clean time, represented by formula (3): $\begin{array}{rcl} (3) & JobTime = StartTime + \sum_{s = 1}^{M} {StageTime}_{s} + CleanTime \end{array}$ where $\sum_{s = 1}^{M} {StageTime}_{s}$ represents the total execution time of a job in the Mth stage.

Considering jobs are processed in a parallel manner between nodes, each stage will have a CPU core executing one task at a time. Hence, in a Spark cluster with H nodes, the number of tasks P that can be processed in a parallel manner can be calculated by formula (4): $\begin{array}{rcl} (4) & P = \sum_{i = 1}^{H} {Core}_{i} \end{array}$ ${Core}_{i}$ represents the number CPU cores in the ith node, and H is the number of nodes in a Spark cluster.

From formula (3), each stage will have specific task time. Based on that time, it can be divided into the maximum execution time of all serial tasks, startup time and clean up time, as represented by formula (5): $\begin{array}{rcl} (5) & StageTime = Startup + {max}_{c = 1}^{p} \sum_{i = 1}^{K_{c}} {TaskTime}_{c, i} + Cleanup \end{array}$ where p is the number of CPU cores, $K_{c}$ is the number of serial tasks executed by core C, ${max}_{c = 1}^{p} \sum_{i = 1}^{K_{c}} {TaskTime}_{c, i}$ is the maximum sum of execution time for $K_{c}$ serial tasks under execution stage with P CPUs. TaskTime can be further broken down as formula (6): $\begin{array}{rcl} (6) & TaskTime = DeserialisationTime + RunTime + SerialisationTime \end{array}$

DeserialisationTime is the time required for deserializing input data, SerialisationTime is the time required to serialise the results, RunTime is the actual time used for the data operation. These time information can be obtained through Spark UI job dashboard.

In short, the Spark job execution time evaluation formula can be represented by formula (7): $\begin{array}{l} JobTime = & StartTime + CleanTime \\ (7) & + {max}_{c = 1}^{p} \sum_{i = 1}^{K_{c}} (DeserialisationTime + RunTime + SerialisationTime) \end{array}$

3.2.2. Evaluation of execution time boundaries

The dynamics of virtualisation of resources inevitably have a certain impact on the result of prediction. Therefore, an evaluation of execution time boundaries is proposed in this paper, which divides the effective upper and lower bounds of the prediction time, and verifies the prediction results of the SRC-WSVR model.

Based on the job execution time evaluation formula, it is easily seen that the execution time of task in each stage influences the job execution time the most. Let a stage ${Stage}_{i}$ has n tasks, with each task has an execution time of $t_{1}, t_{2}, \dots, t_{n}$ , respectively. The number of available core is K and it also represents the number of tasks can be executed in parallel manner, Spark scheduler will then assign unexecuted tasks to idle cores using the greedy algorithm.

In an ideal situation, n tasks in a stage are allocated evenly onto the K cores, with each task executing for the same amount of time. The execution time of such stage can be represented by formula (8): $\begin{array}{rcl} (8) & T_{avg} = n \cdot t_{avg} / K \end{array}$ where $t_{avg} = \sum_{t = 1}^{n} t_{i} / n$ , it represents the average execution time of n tasks; K is the number of cores available.

However, in the actual jobs loading operation, each node is using different types of CPU, internal memory, and hard drives that may have differences performance. Task distributed on each CPU core execution time may different. Cluster scheduler will prioritise assigning tasks to idled CPU cores, and in the worst scenario it could be the longest execution time task will be the last execution task. By the greedy method, the time elapsed before executing the last task is $\frac{\sum_{i = 1}^{n - 1} t_{i}}{K}$ , and from formula (9): $\begin{array}{rcl} (9) & \sum_{i = 1}^{n - 1} t_{i} ⩽ t_{avg} \cdot (n - 1) \end{array}$ the upper bound for job execution time is of formula (10): $\begin{array}{rcl} (10) & T_{\max} = \frac{t_{avg} \cdot (n - 1)}{K} + t_{\max} \end{array}$

The upper bound and lower bound are to estimate the longest and shortest execution time of a job in a stage, respectively. Hence, the lower bound of the Spark job total execution time is: $\begin{array}{rcl} (11) & T_{down} = T_{startTime} + \sum_{j = 1}^{m} T_{j, avg} + T_{cleanTime} \end{array}$ and the upper bound is $\begin{array}{rcl} (12) & T_{up} = T_{startTime} + \sum_{j = 1}^{m} T_{j, \max} + T_{cleanTime} \end{array}$ where $T_{avg}$ and $T_{\max}$ are the average execution time and longest execution time of the ith stage in a spark job, respectively.

In summary, from the produced time-consuming information in the task operation runtime listed in Table 1, the execution time of a Spark job, upper bound and lower bound of job’s total execution time can be calculated using formulae (7), (11) and (12), respectively. From formula (4), we can observe that the execution time of a job is affected by several factors, such as the number of CPU used in the execution, input data size, and the allocated internal memory. Therefore, in part 4, a regression model is used to compute a Standard Regression Coefficient as the impact weight using job execution time-related indices, the weighted samples are then passed onto the Weighted Support Vector Regression Machine Prediction Model to analyse the predicted operation time in processing big data cluster jobs. In part 5, the accuracy of the predicted results will be compared with Multi-factor Line Regression (MLR), and Support Vector Regression machine (SVR) models.

4. Weighted support vector regression machine predictive model based on standard regression coefficient

In Standard Support Vector Regression Machine, given a training sample $D = (x_{1}, y_{1})$ , $(x_{2}, y_{2})$ , $(x_{m}, y_{m})$ , where $x_{i} \in R$ is the input, $y_{i} \in R$ is the corresponding target value, m is the number of samples. The insensitive loss function ε is defined as: $\begin{array}{rcl} (13) & y - f (x, ω) |_{ε} = \{\begin{matrix} 0, & | y - f (x, ω) | ε, \\ | z | - ε, & | y - f (x, ω) | ⩾ ε . \end{matrix} \end{array}$

$f (x, w)$ in the formula is the forecasting function constructed from training sample. y is the actual target value corresponding to x; $ε > 0$ is the design parameter, representing the error requirement of the evaluation function on sample data. Since Support Vector Machine can present data in dual, therefore, the forecasting function $f (x)$ is expressed as: $\begin{array}{rcl} (14) & f (x) = ω \cdot Φ (x) + b \end{array}$ where $ω \cdot Φ$ is the scalar product of vectors ω and Φ; ω dimension is the feature space dimension, $b \in R$ . Prediction from forecasting function $f (x)$ can be converted into the optimisation problem stated in formula (15) $\begin{array}{l} min_{ω, b, ζ, ζ^{*}} \frac{1}{2} ‖ ω ‖^{2} + C \sum_{i = 1}^{m} (ζ_{i} + ζ_{i}^{*}) \\ s.t. y_{i} - ω \cdot Φ (x_{i}) - b ⩽ ε + ζ_{i} \\ ω \cdot Φ (x_{i}) + b - y_{i} ⩽ ε + ζ_{i}^{*} \\ (15) & ζ_{i} ⩾ 0, ζ_{i}^{*} ⩾ 0, i = 1, 2, \dots, m \end{array}$ where $C > 0$ is the punitive coefficient, a larger C value indicates a harsher punishment on samples that have a training difference $| f (x_{i}) - y_{i} |$ larger than ϵ, while a smaller C value indicates a leaner punishment and cause a greater training difference. The choice of parameter C has a greater impact on the system generalisation ability, and the value ε determines whether the punishment will be applied or not. A smaller ε requires a higher Regression Estimation Accuracy and more support vector, while a larger ε requires a lower Regression Estimation Accuracy and fewer support vectors. Hence, based on sample’s property, choosing a reasonable value for C and ε helps to increase the prediction accuracy of the model.

On the other hand, as stated in formula (15), the Standard Regression Support Vector Machine method will have the same C and ε values for all samples, meaning that the punishment applied for deviation in accuracy is the same for all sample data. But in actual applications, the dependent and independent variables of the forecasting function have a weighted relationship, a different error requirement parameter $(ε)$ and punitive function punishing for deviation error $(C)$ should be picked depending on the importance of the sample. Previous works have shown that weight of the sample punitive parameter C can enhance the system performance, while the system performance does not enhance much with the weight of the sample error parameter. Hence, this work proposes the method of computing standard regression coefficient between sample parameters β, using linear regression algorithm, as the basis for calculating the importance weight of sample parameters.

4.1. Calculation of the weight of sample parameters

The linear relationship sample constructed from dependent variable Y and independent variables $x_{1}, x_{2}, \dots, x_{n}$ can be represented by formula (16): $\begin{array}{rcl} (16) & Y = β_{0} + β_{1} X_{1} + β_{2} X_{2} + \dots + β_{n} X_{n} + ε \end{array}$ where Y is the prediction value, $β_{i}$ , with $i = 1, 2, \dots, p$ representing the regression coefficient of the corresponding benchmark, ε is the error constant and regression coefficient $(β_{0}, β_{1}, \dots, β_{n})$ can be calculated from method of optimization, so that standardized regression coefficient represents the importance weight of the sample parameters.

Let $(x_{i_{1}}, x_{i_{2}}, x_{i_{3}}, x_{i_{2}} x)$ , with $i = 1, 2, \dots, n$ being the nth independent observation value of $(X_{1}, X_{2}, \dots, X_{p}, Y)$ , this is represented in the form of matrix: $\begin{array}{rcl} (17) & y = [\begin{matrix} y_{1} \\ y_{2} \\ \dots \\ y_{n} \end{matrix}], β = [\begin{matrix} β_{1} \\ β_{2} \\ \dots \\ β_{p} \end{matrix}], X = [\begin{matrix} 1 & x_{11} & \dots & x_{1 p} \\ 1 & x_{21} & \dots & x_{2 p} \\ \dots & \dots & \dots & \dots \\ 1 & x_{n 1} & \dots & x_{n p} \end{matrix}], ε = [\begin{matrix} ε_{1} \\ ε_{2} \\ \dots \\ ε_{p} \end{matrix}] \end{array}$ By vectorization, it can be expressed as: $\begin{array}{rcl} (18) & y = X β + ε \end{array}$ To make the equation set to be “solvable”, a suitable β value has to be picked to make the equations “reasonable”, hence, the residual sum of square function $S (β) = ‖ X β - y ‖ 2$ is introduced. When $β = \hat{β}$ , $S (β)$ is at the minimum, denoted by: $\begin{array}{rcl} (19) & \hat{β} = argmin (S (β)) \end{array}$ Differentiating $S (β)$ to solve for minima will have: $\begin{array}{rcl} (20) & X^{T} X \hat{β} = X^{T} y \end{array}$ Hence, when matrix $X^{T} X$ is non-singular, then β has the only solution: $\begin{array}{rcl} (21) & \hat{β} = {(X^{T} X)}^{- 1} X^{T} y \end{array}$ When fulfilling the conditions in residual significance test, benchmarks that could cause a greater impact on the sample prediction is selected. By standardising the standard weight $\hat{β}$ obtained will have the standardised weight value to be W, the expected value of each standard weight is $E (x)$ and the standard deviation is $S (x)$ . The sample weight coefficient is: $\begin{array}{rcl} (22) & W_{i} = \frac{({\hat{β}}_{i} - E (x))}{S (x)}, w_{i} \in (0, 1) \end{array}$

4.2. Weighted support vector regression machine based on standard regression coefficient

Putting formula (22) into formula (15) and add weight according to the importance weight of the sample parameters, the optimisation problem is: $\begin{array}{l} min_{ω, b, ζ, ζ^{*}} \frac{1}{2} ‖ ω ‖^{2} + C \sum_{i = 1}^{m} (ζ_{i} + ζ_{i}^{*}) \\ s.t. y_{i} - ω \cdot Φ (x_{i}) - b ⩽ ε + ζ_{i} \\ ω \cdot Φ (x_{i}) + b - y_{i} ⩽ ε + ζ_{i}^{*} \\ (23) & ζ_{i} ⩾ 0, ζ_{i}^{*} ⩾ 0, i = 1, 2, \dots, m \end{array}$ where $w_{i}$ represents the weighted coefficient of parameter C by the ith sample parameter’s weight. By solving the quadratic programming problem with inequality constraints using Method of Lagrange Multiplier, then: $\begin{array}{l} L (ω, b, α, \hat{α}, ζ, ζ^{*}, μ, \hat{μ}) = & \frac{1}{2} ‖ ω ‖^{2} + C \sum_{i = 1}^{m} W_{i} (ζ_{i} + ζ_{i}^{*}) - \sum_{i = 1}^{m} u_{i} ζ_{i} - \sum_{i = 1}^{m} \hat{u_{i}} ζ_{i}^{*} \\ (24) & + \sum_{i = 1}^{m} α_{i} (ω \cdot Φ (x_{i})) - y_{i} - ε - ζ_{i} + \sum_{i = 1}^{m} \hat{α_{i}} (y_{i} - ω \cdot Φ (x_{i}) - ε - ζ_{i}^{*}) \end{array}$ where $α, \hat{α}, μ, \hat{μ} ⩾ 0$ are Lagrange Multipliers. Let the partial derivatives of $ω, b, ζ_{i}, ζ_{i}^{*}$ be 0, then $\begin{array}{l} (25) & ω = \sum_{i = 1}^{m} (\hat{α_{i}} - α) x_{i} \\ (26) & 0 = \sum_{i = 1}^{m} (\hat{α_{i}} - α) \\ (27) & C W_{i} = α_{i} + μ_{i} \\ (28) & C W_{i} = \hat{α_{i}} + \hat{μ_{i}} \end{array}$ Putting formulae (25) to (28) into Formula (24) will obtain its duality problem $\begin{array}{l} max_{α, \hat{α}} \sum_{i = 1}^{m} y_{i} (\hat{a_{i}} - a_{i}) - ε (\hat{a_{i}} + a_{i}) \\ - \frac{1}{2} \sum_{j = 1}^{m} (\hat{a_{i}} - a_{i}) (\hat{a_{j}} - a_{j}) x_{i}^{T} x_{j} \\ = - \frac{1}{2} \sum_{i = 1}^{m} \sum_{j = 1}^{m} (\hat{a_{i}} - a_{i}) (\hat{a_{j}} - a_{j}) K (x_{i}, x_{j}) - ε \sum_{i = 1}^{m} (\hat{a_{i}} + a_{i}) + \sum_{i = 1}^{m} (\hat{a_{i}} - a_{i}) \\ s.t. \sum_{i = 1}^{m} (\hat{a_{i}} - a_{i}) = 0 \\ 0 ⩽ a_{i} ⩽ C W_{i} \\ (29) & 0 ⩽ \hat{a_{i}} ⩽ C W_{i}, i = 1, 2, \dots, m \end{array}$ The processes above fulfilled the KTT condition, $K (x_{i}, x j) = Φ (x_{i}) \circ Φ (x_{j})$ in the formula are known as the kernel function. The optimisation problem from formula (29) can solve for the optimised value of $(({\hat{α}}_{i}, α_{i}))$ , the regression estimation function is, hence, calculated as: $\begin{array}{l} f (x) & = \sum_{i = 1}^{m} (α_{i} - \hat{α_{i}}) Φ (x_{i}) \circ Φ (x_{j}) + b \\ (30) & = \sum_{x_{i} \in SupportVector} (α_{i} - \hat{α_{i}}) K (x_{i}, x_{j}) + b \end{array}$

In summary, this calculation process enables prediction to job execution time by the Standard Regression Coefficient-based Weighted Support Vector Regression Machine. The use of estimated Mean Absolute Error (MAE), estimated Mean Square Error (MSW) and goodness of fit determination coefficient $(R^{2})$ can be used as evaluation benchmark in assessing the prediction ability of the forecasting function. The formula for each of the method is listed in Table 2, with $a_{i}$ to be the actual value, $\overline{a}$ to be mean value and $p_{i}$ to be prediction value.

Table 2
Evaluation benchmark and corresponding formula

Evaluation benchmark Formula

MAE $\frac{\sum_{i = 1}^{m} | a_{i} - p_{i} |}{m}$

MSE $\frac{\sum_{i = 1}^{m} | a_{i} - p_{i} |^{2}}{m - 1}$

$R^{2}$ $\frac{\sum_{i = 1}^{m} {(p_{i} - \overline{a})}^{2}}{\sum_{i = 1}^{n} {(a_{i} - \overline{a})}^{2}}$

Evaluation benchmark	Formula
MAE	$\frac{\sum_{i = 1}^{m} \| a_{i} - p_{i} \|}{m}$
MSE	$\frac{\sum_{i = 1}^{m} \| a_{i} - p_{i} \|^{2}}{m - 1}$
$R^{2}$	$\frac{\sum_{i = 1}^{m} {(p_{i} - \overline{a})}^{2}}{\sum_{i = 1}^{n} {(a_{i} - \overline{a})}^{2}}$

Table 3

Physical configurations table for nodes

Role of the node	Master	Node 1	Node 2
Processor	IntelⓇXeonⓇCPU E5-2620 v3 @2.40 GHz	IntelⓇXeonⓇCPU E5-2620 v3 @2.40 GHz	IntelⓇXeonⓇCPU E5-2620 v3 @2.40 GHz
Number of cores	4	4	4
Internal Memory	38.4 GB	38.4 GB	38.4 GB
Hard Disk	975 GB	975 GB	975 GB
Operating System	Ubuntu 16.04 LTS	Ubuntu 16.04 LTS	Ubuntu 16.04 LTS
Cloud Environment	Spark + Hadoop (HDFS)	Spark + Hadoop (HDFS)	Spark + Hadoop (HDFS)

5. Simulations

5.1. Descriptions

To evaluate the prediction accuracy of the model proposed in this work, cluster execution experiments were used in simulations, with one of the nodes being the main node, and the two other being the task nodes. Physical configurations of each node are illustrated in Table 3.

In the evaluation process, this work used the Hadoop Distributed File System (HDSF) with default allocated memory size is 64 MB for the input source and output of data. The Apache Spark run in Standalone mode, for the performance data were collected from the UI dashboard of the Apache Spark platform and performance benchmark acquisition port provided by the Spark Metrics, the results are then exported and stored on a local hard drive in CSV format. Through analysing job performance monitoring information, StageTime could be calculated using formula (3), while TaskTime can be calculated using formula (5). At the same time, values such as the startup time, cleanup time and stage initialization time, stage cleanup time and task serialisation time could be obtained from monitoring documents, and to calculate the actual JobTime using formula (7).

One of the applications used in this work for verifying the prediction model is WordCount, a non-iterative text processing tool [9] used for word frequency statistics analysis. This is a type of loading which identifies keywords of web pages for easy categorization and processing. Another application used is PageRank, an iterative graph algorithm which gives a mark to every web pages and to calculate the importance of each of them iteratively by understanding the relationship of the links (URLs).

The detailed experiment procedures are: 1. Choice of data type and the amount of data: As tuning data, data source input to WordCount is from Wikipedia entries and output store to HDFS. PageRank used Google Web graphics, this data set has a maximum of 875713 nodes, 5105039 edges, forming a non-structured graph data set. 2. Calculation of sample parameter Weight: Standard Weight coefficient: To the size of data set is gradually growing, jobs could run on multiple CPU cores with different size of memory. The standard weight coefficient of the application’s execution time effect by the number of cores and data size is calculated using regression algorithm. 3. Prediction by Weighted Support Vector Regression Machine: Adding weight obtained in step 2 to sample-related parameters for predicting job time, and to prove if it within the job operation time range. The accuracy of the prediction result is compared with traditional prediction model to verify the model’s accuracy. Configurations of data used in experiments are listed in Table 4.

Table 4
Experiment Data Information

Application Input data size Number of Cores Task Operational Internal Memory

WordCount 2 G, 6 G, 10 G 12 96 30 G

PageRank 3 G, 8 G, 12 G 12 96 30 G

Application	Input data size	Number of Cores	Task	Operational Internal Memory
WordCount	2 G, 6 G, 10 G	12	96	30 G
PageRank	3 G, 8 G, 12 G	12	96	30 G

5.2. Result analysis

By performing regression analysis on performance data using stepwise approach, the summary table and regression variation analysis table can be obtained.

Table 5
Regression Variance Analysis Table

R $R^{2}$ Adjusted $R^{2}$ Standard evaluation error

0.873 0.861 0.846 0.6832

R	$R^{2}$	Adjusted $R^{2}$	Standard evaluation error
0.873	0.861	0.846	0.6832

From Table 5, it is observed that the response level of the linear equation to actual data (adjusted $R^{2}$ value) is 84.6%. The linear fitting equation has a better respond level when compared with using original data. Hence, constructing regression equation from tuning sample data could solve for benchmark’s corresponding parameter β. There are a number of factors affecting application’s execution time, including the number of cores used (cores), the amount of data (nums) and internal memory (memory). The standard weight coefficient of each factor’s time effect to WordCount application is listed in Table 6, while that to PageRank application is listed in Table 7.

From Table 6, it is observed that for the non-iterative application like WordCount, the benchmark “cores” account for the heaviest weight, meaning that the more CPU core is used, the shorter the job execution time and higher efficiency.

Table 6

Related benchmark weight for WordCount

Benchmark	Weight Parameter β	t test	Significance
Constant	5.395	0.437	0.001
$x_{1}$ : cores	0.780	3.615	0.001
$x_{2}$ : nums	0.213	0.507	0.001
$x_{3}$ : memory	0.417	−2.723	0.000

Table 7

Related benchmark weight for PageRank

Benchmark	$R^{2}$	Weight Parameter β	t test
Constant	8.395	0.531	0.004
$x_{1}$ : cores	0.480	3.615	0.001
$x_{2}$ : nums	0.313	0.507	0.000
$x_{3}$ : memory	0.817	−2.723	0.000

Fig. 2.

WordCount prediction result.

Fig. 3.

PageRank prediction result.

From Table 7, it is observed that for the iterative application like PageRank, benchmark “memory” accounts for the heaviest weight, meaning that the more the amount of internal memory allocated, the shorter the job execution time and higher efficiency.

Putting prediction samples confirmed by weight parameter into the model, WordCount used prediction samples with sizes 2 G, 6 G and 10 G; PageRank used prediction samples with sizes 3 G, 8 G and 12G. The number of cores used was 12 and the internal memory was 30G. Results are shown in Figs 2 and 3.

Table 8

Related benchmark weight for WordCount

Models	MSE	MAE β	$R^{2}$
SRC-WSVR	2.934	2.127	0.803
SVR	2.972	2.199	0.797
MLR	3.402	1.835	0.746

The experiment showed that the predicted execution time for both types of the job is within the prediction range. Results from repetitive experiment showed that the predicted job execution time has an error within 10% when compared with the mean value.

By comparing the results from the model proposed in this work with the evaluation benchmarks of conventional regression prediction model and standard support vector regression machine model, as from the R2 values listed in Table 8, the model proposed in this word has a higher fitting degree.

6. Conclusions

In order to use the rental service efficiently in the cloud environment, users need to apply for optimal cluster resources to reduce rental costs. Therefore, the Standard Regression Coefficient-Weighted Support Vector Regression Machine prediction model proposed in this work enables the prediction of execution time in processing large data sets of a specific type of job, through analysing the tuning data sets. Compared with the traditional prediction model, it also has a higher accuracy which could provide an estimation of resources overheads requirement to users in the production environment.

Footnotes

Acknowledgements

The subject is sponsored by the National Natural Science Foundation of P.R. China (No. 61572260, No. 61572261, No. 61672296, No. 61602261), the Natural Science Foundation of Jiangsu Province (No. BK20140886, No. BK20140888, No. BK20160089), Scientific & Technological Support Project of Jiangsu Province (No. BE2015702, No. BE2016777, BE2016185), China Postdoctoral Science Foundation (No. 2014M551636, No. 2014M561696), Jiangsu Planned Projects for Postdoctoral Research Funds (No. 1302090B, No. 1401005B), Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks Foundation (No. WSNLBZY201508), Research Innovation Program for College Graduates of Jiangsu Province (SJZZ16-0148), Postgraduate Research & Practice Innovation Program of Jiangsu Province (SJZZ16-0148).

References

Anetor,

E.E.

Osakue and

Odetunde, Parametric studies of some operating variables on spark-ignition engine performance, Arabian Journal for Science & Engineering 5(42) (2017), 1–16.

Atas and

V.C.

Gungor, Performance evaluation of cloud computing platforms using statistical methods, Computers & Electrical Engineering 40(5) (2014), 1636–1649. doi:10.1016/j.compeleceng.2014.03.017.

A.D.

Benedictis,

Rak,

Turtur and

Villano, A framework for cloud-aware development of bag-of-tasks scientific applications, International Journal of Grid and Utility Computing 7(2) (2016), 130–140. doi:10.1504/IJGUC.2016.077492.

Boyinbode

Le and

Takizawa, A survey on clustering algorithms for wireless sensor networks, International Journal of Space-Based and Situated Computing (IJSSC) 1(2/3) (2011), 137–150. doi:10.1504/IJSSC.2011.040339.

Chawla,

Chana and

Rana, Cloud-based automatic test data generation framework, Journal of Computer & System Sciences 82(5) (2016), 712–738. doi:10.1016/j.jcss.2015.12.001.

Ganapathi,

Chen,

Fox et al., Statistics-Driven Workload Modelling for the Cloud, IEEE International Conference on Data Engineering Workshops, IEEE, 2010, pp. 87–92.

Gu and

V.S.

Sheng, A robust regularization path algorithm for ν-support vector classification, IEEE Transactions on Neural Networks and Learning Systems (2016). doi:10.1109/TNNLS.2016.2527796.

Gu,

V.S.

Sheng,

K.Y.

Tay,

Romano and

Li, Incremental support vector learning for ordinal regression, IEEE Transactions on Neural Networks and Learning Systems 26(7) (2015), 1403–1416. doi:10.1109/TNNLS.2014.2342533.

Gu and

Li, Memory or time: Performance evaluation for iterative operation on hadoop and spark, in: IEEE International Conference on High Performance Computing and Communications & 2013 IEEE International Conferenceon Embedded and Ubiquitous Computing, 2013, pp. 721–727.

10.

Hamamoto,

Dong and

Babu, MapReduce programming and cost-based optimisation crossing this chasm with starfish, in: Proceedings of the Vldb Endowment, Vol. 4, 2011, pp. 1446–1449.

11.

Herodotou,

Lim,

Luo,

Borisov,

Dong,

F.B.

Cetin and

Babu, Starfish: A self-tuning system for big data analytics, in: 5th Biennial Conference on Innovative Data Systems Research, 2011, pp. 261–272.

12.

Kaushik and

D.P.

Vidyarthi, A model for resource management in computational grid for real-time jobs using game theory, International Journal of Grid and Utility Computing. 6(3/4) (2015), 232–248. doi:10.1504/IJGUC.2015.070673.

13.

Li,

Rong and

Zhang, An Internet of things QoE evaluation method based on multiple linear regression analysis, in: International Conference on Computer Science & Education, IEEE, 2015, pp. 925–928.

14.

Li,

Tan and

Wang et al., SparkBench: A comprehensive benchmarking suite for in memory data analytic platform spark, in: ACM International Conference on Computing Frontiers, ACM, 2015, pp. 1–8.

15.

Louis,

V.K.

Agrawal and

P.V.

Khadikarc, Prediction of intrinsic solubility of generic drugs using MLR, ANN and SVM analyses, European Journal of Medicinal Chemistry 45(9) (2010), 4018–4025. doi:10.1016/j.ejmech.2010.05.059.

16.

Ma and

Zhang, Introducing browser-based high-frequency cloud monitoring system using Web Socket proxy, pp. 30–46. doi:10.1504/IJGUC.2015.066389.

17.

Maher and

A.A.D.

Sarhan, Proposing a new performance index to identify the effect of spark energy and pulse frequency simultaneously to achieve high machining performance in WEDM, International Journal of Advanced Manufacturing Technology 3(1) (2016), 1–11.

18.

Mavridis and

Karatza, Performance evaluation of cloud-based log file analysis with apache hadoop and apache spark, Journal of Systems & Software 3(25) (2016), 133–151.

19.

Mershad,

Artail,

Saghir et al., A mathematical model to analyze the utilization of a cloud datacenter middleware, Journal of Network & Computer Applications 59(3) (2014), 399–415.

20.

Mozafari,

Curino,

Jindal et al., Performance and resource modeling in highly-concurrent OLTP workloads, in: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, 2013, pp. 301–312. doi:10.1145/2463676.2467800.

21.

A.A.

Munshi and

Y.A.-R.I.

Mohamed, Big data framework for analytics in smart grids, Electric Power Systems Research 6(1) (2017), 369–380.

22.

A.A.

Munshi and

Y.A.I.

Mohamed, Cloud-based visual analytics for smart grids big data, in: Innovative Smart Grid Technologies Conference, IEEE, 2016, pp. 7–12.

23.

A.D.

Popescu,

Balmin,

Ercegovac et al., PREDIcT: Towards predicting the runtime of large scale iterative analytics, in: Proceedings of the Vldb Endowment, Vol. 6, 2013, pp. 1678–1689.

24.

Saadatfar and

Deldari, A job submission manager for large-scale distributed systems based on job futurity predictor, International Journal of Grid and Utility Computing. 5(1) (2014), 50–59. doi:10.1504/IJGUC.2014.058252.

25.

Shen,

Yang,

Wu et al., Securing data services: A security architecture design for private storage cloud based on HDFS, International Journal of Grid & Utility Computing 4(4) (2013), 242–254. doi:10.1504/IJGUC.2013.057118.

26.

Thabet,

Boufaida and

Kordon, An approach for developing an interoperability mechanism between cloud providers, International Journal of Space-Based and Situated Computing. 4(2) (2014), 88–99. doi:10.1504/IJSSC.2014.062469.

27.

L.E.B.

Villalpando,

April and

Abran, Performance analysis model for big data applications in cloud computing, Journal of Cloud Computing 3(1) (2014), 1–20. doi:10.1186/2192-113X-3-1.

28.

Wang and

M.M.H.

Khan, Performance prediction for apache spark platform, in: IEEE International Conference on High PERFORMANCE Computing and Communications, IEEE, 2015, pp. 166–173.

29.

Ye,

Qian and

R.Q.

Hu, Identity-based schemes for a secured big data and cloud ICT framework in smart grid system, Security & Communication Networks 9(8) (2016), 8–15.

30.

Yuriyama and

Kushida, Integrated cloud computing environment with IT resources and sensor devices, International Journal of Space-Based and Situated Computing 1(2/3) (2011).

31.

Zhang, Spark Technology Insider: Deeply Analyzing Spark Core Architecture Design and Implementation, Mechanical Industry Press, 2015.

32.

Zhang,

Cherkasova and

B.T.

Loo, Performance modeling of MapReduce jobs in heterogeneous cloud environments, in: IEEE Sixth International Conference on Cloud Computing, IEEE, 2014, pp. 839–846.