SVR-based collaborative filtering algorithm: From dense to sparse

Abstract

Recommender systems have been very important components to prevent people from dwelling in the overwhelming information. In this paper we analyze the difference between item-based recommendation algorithms and SVR-based collaborative filtering algorithms, and it can be found that item-based method performs much better while the data is not sparse significantly, and SVR-based method performs better while the data is dense and small. On this premise we propose a method that can combine the advantages of these two methods by predicting a small part of ratings using SVR method firstly and then predicting the rest of ratings using the item-based algorithm, which can solve the problem of data sparsity to certain extend. Finally, we evaluate our results compared with the benchmark on different datasets and prove our method’s advantages.

Keywords

Support vector regression item-based recommendation collaborative filtering data sparsity

1. Introduction

For decades, the data for recommend systems to work are usually based on users and items, which are typically displayed as U-I matrices [1]. There are two major benchmarks to realize collaborative filtering (CF) so to explore the relationship between users and items [2] basically. One is the memory-based method [3, 4] including user-based CF and item-based CF. Wang et al. [5] presented a schedule-based user recommendation system with improved K-means algorithm. Their results show that the system improves the recall and precision compared to traditional one. Dixit and Jain [6] introduced another novel framework for real life recommender systems, utilizing k-prototype clustering technique in grouping contextually similar users to get a reduced set and find the contribution of different context features to control data sparsity problem. Gupta and Dixit [7] presented a scalable and optimized recommender system for e-commerce web sites, where similarity measures were used along with efficient rough set leader clustering algorithm, which helped in making accurate and fast recommendations. These methods often calculate the similarity function between users or items to predict the particular users’ unknown or unpaid items rates.

Another way of collaborative filtering is the model-based [8] method, which directly constructs a map between input rates and the predicting rates. Yu et al. [9] proposed a novel robust recommendation method based on kernel matrix factorization, therefore enhance the power of the model, and realize the robust estimation of user feature matrix and item feature matrix. Liu et al. [10] presented a method of kernelizing matrix factorization for collaborative filtering, which applied multiple kernel learning method to capture the non-linear relationships in the rating data for improving the prediction accuracy. Xian et al. [11] used Singular Value Decomposition (SVD) for recommender systems, and it was claimed that the proposed method not only protected privacy information in the raw data but also ensured the accuracy of recommendations. García-Floriano et al. [12] analyzed the prediction accuracy of SVR when applied to predict software enhancement effort. The results shown that the polynomial kernel $\varepsilon$ -SVR was statistically better than statistical regression, neural networks, association rules and decision trees, with 95% confidence. Previously researches have shown with even a little progress in the predicting process, the recommend work will get a remarkable advance [13].

However, most of memory-based algorithms suffer from the data sparsity [14, 15] since the majority of datasets lack more than 90% of the data. It means even with the best prediction algorithms, we can’t get a convincing result due to the principle of statistics.

In this paper we utilize a recommend algorithm that combines the KNN item-based CF and the support vector regression (SVR) model, and it can solve the problem of data sparsity. First, we record the majority part of rated items and users and use SVR model [16] to predict a pre-filled U-I matrix, and then, we use the item-based CF algorithm to predict the rest ratings according to the above pre-filled U-I matrix. Therefore, we can get a part of comparable accurate data rather than empty data to deal with the sparse data.

In Section 2, the related work and the proposed model are introduced and in Section 3, we present the evaluation methodology and the performance of the method. Lastly, Section 4 provides the readers with some method conclusions and plans for the future work.

2. Proposed approach

The idea of the proposed method is originated from the fact that item-based CF method always meets the problem of data sparsity. At the same time, SVR-based method performs well when the data is dense and small. For the above mentioned reasons, the proposed method can combine the advantages of these two methods by using SVR to predict a small part of ratings firstly, and then using the item-based CF algorithm to predict the rest of ratings with pre-filled and rated data, which is not significantly sparse any more.

2.1 Notation

The number of users is denoted by $N$ and the number of items is denoted by $M$ . Matrix $R$ is used to represent the user-item implicit matrix of size $N\times M$ . It shows items the users have purchased/viewed/rated. Symbols $u_{i}$ and $\textit{item}_{j}$ are used to denote the $i^{\text{th}}$ user and the $j^{\text{th}}$ item respectively, $i=1,2,\cdots,N$ , $j=1,2,\cdots,M$ . We refer to the items that the user has purchased/viewed/rated as rated items while refer to the rest as unrated items. The value of item $j$ that the user $u_{i}$ has rated will be recorded as $r_{i,j}$ . If the user $u_{i}$ doesn’t rate item $j$ , $r_{i,j}=0$ . Moreover, we use $k$ to denote the number of neighbors which we choose to predict the missing values in the item-based CF algorithm.

2.2 Motivation

Item-based collaborative filtering looks into the set of items that the active user has rated and computes how similar they are to the active item $j$ , then selects $k$ most similar items $n_{j}=(\textit{item}_{j_{1}},\linebreak\textit{item}_{j_{2}},\cdots,\textit{% item}_{j_{k}})$ as $k$ nearest neighbors (KNN) [17]. The similarity between items $j$ and $l$ , $s_{j,l}$ is often computed by Cosine similarity denoted as:

$\displaystyle s_{j,l}=\cos({\textit{item}_{j},\textit{item}_{l}})=\frac{r_{% \cdot,j}\cdot r_{\cdot,l}}{\|{r_{\cdot,j}}\|_{2}\cdot\|{r_{\cdot,l}}\|_{2}}$ (1)

where “ $\cdot$ ” denotes the dot-product of the two vectors $r_{\cdot,j}$ and $r_{\cdot,l}$ , $r_{\cdot,j}=({r_{1,j},r_{2,j},\cdots,r_{N,j}})^{T}$ , $r_{\cdot,l}=({r_{1,l},r_{2,l},\cdots,r_{N,l}})^{T}$ . The prediction $p_{i,j}$ of the item $j$ refer to $u_{i}$ is

$\displaystyle p_{i,j}=\frac{\sum\limits_{l=j_{1}}^{j_{k}}{({s_{j,l}\cdot r_{i,% l}})}}{\sum\limits_{l=j_{1}}^{j_{k}}{({s_{j,l}})}}$ (2)

For each item $j$ , the $k$ most similar items are selected to make the method faster and more precise, where $k\leqslant M$ .

The SVR model treats collaborative filtering as a regression problem using Support Vector Machine (SVM) by three ways. One way, the two-dimensional data [user, movie] associated with a rating is treated as a $({N+M})$ -element feature vector with a rating label [18]. The second way, each item is treated as a separate classification or regression problem. Given an item $i$ , a model is built to decide which side the user belongs to. The third and a more common way, each user is treated as a separate classification or regression problem [19, 20].

Both the item-based collaborative filtering and the SVR-based algorithm have their own advantages and particular restrictions, so a method is present which can combine the advantages of both algorithms while prevent the sparsity problem. An item-based method can predict rating fast and has the good scalability, but using one similarity function to evaluate the whole dataset might be not precise especially when the dataset is sparse extremely. When it comes to SVR-based method, we can use the precise power of SVM to classify data or do regression to predict a small amount of missing values, where the time cost of training process and the probability of over-fitting make this method lack of flexibility when the data is large.

Figure 1.

a. Dataset with different densities; b. Predicted values from the dense part using SVR model; c. Predict the rest part of U-I matrix using item-based CF.

After observing the data, we found the data was composed by comparably dense part and sparse part. The previous prediction method often pre-fill the missing value, as average rating of the same item rated by users or the average rating of the user, to predict the U-I matrix. In this paper, at the first step, SVR model is used to predict the missing values of the dense part, and at the second step, item-based model is used to predict the missing values of the sparse part. The simplified process is denoted in Fig. 1.

2.3 SVR-based collaborative filtering

In this part, we present our method SVR-based Collaborative Filtering (SVR-based CF), which pre-fills the comparable dense part of the U-I matrix using SVR model and then predicts the rest of missing values in the U-I matrix using KNN item-based algorithm.

Figure 2.

The prediction process of the dense part of U-I matrix. a. Select the dense part of U-I matrix corresponding to the red rectangular area, and the sparse part corresponds to the other area; b. Pre-fill the missing values of U-I matrix with the average rating of the corresponding item rated by all users; c. Train SVR model according to the dense part of U-I matrix which has been pre-filled; d. Update each pre-filled missing value with the rating predicted by SVR model; e. Output the predicted dense part of U-I matrix, the red rectangular area.

The first step, we need to choose the dense part of a dataset, i.e. the red rectangular area shown in Fig. 2a. We calculate the rated number for each user denoted as $S_{\textit{user}}$ and the rated number for each item denoted as $S_{\textit{item}}$ . Taking Movielens dataset as an example, we choose the ratings when $S_{\textit{user}}>$ 70 and $S_{\textit{item}}>$ 160. The second step is to pre-fill the missing value of item $j$ , which equals to the average rating of the item $j$ rated by $n$ users, $j=1,2,\cdots,m$ . $n$ and $m$ are the number of users and items included in the dense part of U-I matrix, respectively. As shown in Fig. 2b, we consider the $u_{i}$ as active user, $i=1,2,\cdots,n$ , and $u_{i}$ has rated the set of items $x_{i}=({r_{i,1},r_{i,2},\cdots,r_{i,j-1},r_{i,j+1},\cdots,r_{i,m}})^{T}$ , where each element of $x_{i}$ represents the rated value of corresponding item by $u_{i}$ . Meanwhile, the unrated value $y_{i}$ of item $j$ by $u_{i}$ is set to the average of $Y$ , the values of the particular item $j$ rated by users, and the values of the other items rated by users or pre-filled are recorded as $X$ . The third step, each row of $X$ is treated as training sample and each row of $Y$ is treated as the corresponding label. Then, the SVR model can be achieved accordingly. The fourth step, we consider $x_{i}$ as the input of SVR, and the output $y_{i}$ is regarded as the predicted rating of item $j$ according to $u_{i}$ , which will replace the pre-filled missing value, i.e. the average rating of the particular item $j$ rated by users. The fifth step, we need to predict and update all of the pre-filled missing values, by repeating the third and the fourth steps above, and receive the predicted dense part of U-I matrix finally, i.e. the red rectangular area shown in Fig. 2e. After the dense part of U-I matrix is predicted, we use the item-based method to predict the rest of missing values in the U-I matrix. The details of the complete algorithm are given in Table 1.

Table 1

The algorithm of SVR-based collaborative filtering

Algorithm
// Step 1: Choose the dense part of U-I matrix in which the minimum number of $S_{\textit{item}}$ and $S_{\textit{user}}$ is $T$ and $G$ respectively.
For $j={1},2,\cdots,M$ ; $i=1,2,\cdots,N$
Find $n$ users and $m$ items, satisfying $S_{\textit{user}}>G$ and $S_{\textit{item}}>T$ , and determine the dense part of U-I matrix
End for
// Step 2: Pre-fill the dense part of U-I matrix
For $j={1},2,\cdots,m$ ; $i=1,2,\cdots,n$
If $r_{i,j}=0$
$r_{i,j}=\frac{\sum_{u=n_{1}}^{n_{t}}{r_{u,j}}}{t}$ , where $r_{u,j}\neq 0$ and $t$ is the number of $r_{u,j}$ , and $\textit{flag}_{i,j}=F$
End if
End for
// Step 3: Train SVR models
For $j={1},2,\cdots,m$
Build the SVR model, where each row of $X$ is a training sample and the corresponding row of $Y$ is the label
// Step 4: Predict the missing values
For $i=1,2,\cdots,n$
If $\textit{flag}_{i,j}=F$
Predict the value of $p_{i,j}$ by SVR, and re-assign $r_{i,j}$ as $p_{i,j}$
End if
End for
End for
// Step 5: Complete the prediction of the missing values from the dense part of U-I matrix
Receive the predicted dense part of U-I matrix
// Step 6: Compute the similarity between items $j$ and $l$ , according to Eq. (1)
For $j={1},2,\cdots,M$ ; $l={1},2,\cdots,N$
$s_{j,l}=\cos({\textit{item}_{j},\textit{item}_{l}})=\frac{r_{\cdot,j}\cdot r_{% \cdot,l}}{\\|{r_{\cdot,j}}\\|_{2}\cdot\\|{r_{\cdot,l}}\\|_{2}}$
End for
// Step 7: Use item-based CF algorithm to predict the rest missing values of the U-I matrix, according to Eq. (2)
For $j={1},2,\cdots,M$ ; $i=1,2,\cdots,N$
If $r_{i,j}=0$
$r_{i,j}=p_{i,j}=\frac{\sum\limits_{l=j_{1}}^{j_{k}}{({s_{j,l}\cdot r_{i,l}})}}% {\sum\limits_{l=j_{1}}^{j_{k}}{({s_{j,l}})}}$
End if
End for

Table 2

Dataset attributes

Name	User	Item	Rating	Sparsity (%)
Movielens	943	1682	100000	93.69
Jester	57732	150	1760039	79.68
EachMovie	28303	1628	723942	98.43
ML-latest	671	9066	100000	98.36
LastFM	1852	12523	71064	99.70

Finally, given the number of missing values $n_{m}$ from the U-I matrix, the number of missing values ${n}^{\prime}_{m}$ from the dense part of U-I matrix, the time complexity of the proposed method for each step is shown as follows: (1) Choose the dense part of U-I matrix – $O({M\cdot N})$ ; (2) Pre-fill the dense part of U-I matrix – $O({m\cdot n})$ ; (3) Train SVR models – $O({m^{2}\cdot n+m^{3}})$ ; (4) Predict the missing values – $O({m\cdot n_{m}})$ ; (5) Compute the similarity – $O({M\cdot N^{2}})$ ; (6) Use item-based CF algorithm to predict the rest missing values of the U-I matrix – $O({k\cdot({n_{m}-{n}^{\prime}_{m}})\cdot M\cdot N})$ . The time cost of the traditional item-based CF method can be stated: (1) Compute the similarity – $O({M\cdot N^{2}})$ ; (2) Use item-based algorithm to predict the rest missing values of the U-I matrix – $O({k\cdot n_{m}\cdot M\cdot N})$ . By comparison, the proposed method will take more time consuming due to the prediction of the dense part of U-I matrix with SVR model. However, combining SVR and item-based CF, the proposed method can solve the problem of data sparsity and receives more accurate results. Meanwhile, it is evidently that using SVR locally is time-saving than using SVR globally.

3. Experiment evaluation

3.1 Datasets

We evaluate the results of the proposed method with different datasets, of which attributes are shown in Table 2. The sparsity value means the number of missing values divided by the U-I matrix size, i.e.

$\displaystyle\textit{Sparsity}=\frac{n_{m}}{N\times M}$ (3)

where $n_{m}$ is the number of missing values from the U-I matrix. The larger $n_{m}$ is, the sparser the dataset will be.

The MovieLens [21] and ML-latestaremovie datasets [22], which represent movie ratings, range from 1 to 5 as discrete integers. The jester dataset [23] is an online joke recommender system with ratings range from $-$ 10 to 10 as continuous real numbers. The EachMovie dataset [24] is a movie rating dataset with ratings range from 0 to 1 as discrete real numbers. Except the MovieLens and jester datasets we clean the other datasets to eliminate the unavailable data. The LastFM [25] represents different users listen to different music for vary times.

3.2 Evaluation methodology

In the experiments, we use five-fold cross-validation to evaluate the performance of the presented algorithm. Moreover, we measure the performance by computing the MAE values of different methods,

$\displaystyle\textit{MAE}=\frac{\sum\nolimits_{i=1}^{t}{|{p_{i}-r_{i}}|}}{t}$ (4)

where $p_{i}$ is the prediction value of the $i^{\text{th}}$ test sample, $r_{i}$ is the real value, and $t$ is the number of the test samples. The smaller the evaluation value MAE is, the better the algorithm will be.

3.3 Experiment results

The MovieLens dataset’s MAE values of KNN Item-based method and the proposed SVR-based CF can be seen in Fig. 3. The horizontal axis means different $k$ neighbors of the rated items, and the vertical axis means the value of MAE. Different colors and symbols of points mean different methods, and then we use five-fold cross validation so each area means the 5 different MAE values for the method respectively. The darker line within the area means the average MAE value of different methods. It can be found that the smallest MAE can be received when $k$ is near to 120, and with different $k$ nearest neighbors our presented method works slightly better than the traditional KNN item-based method.

Table 3
MAE values for different datasets

Name	Neighbor $k$	KNN item-based	SVR-based CF
Movielens	30	1.432	1.194
	120	0.795	0.701
	200	0.798	0.741
ML-latest	50	1.761	1.755
	100	1.778	1.751
	500	1.765	1.745
Jester	30	1.195	1.541
	120	0.945	1.022
	150	0.837	0.864
Eachmovie	100	1.788	1.756
	180	1.240	0.935
	240	0.905	0.897
LastFM	30	1.816	1.697
	100	1.765	1.685
	500	1.673	1.654

Figure 3.

MAE values of MovieLens dataset using KNN item-based CF and SVR-based CF.

Furthermore, as shown in Fig. 3, the MAE value of the prediction is reduced first, and then prediction error is rising. The reason is that the prediction process selects more similar neighbors, the more accurate. However, when the number of neighbors exceeds a certain degree, the items that the general public likes are included in the calculation for many times, and the error will increase inevitably. Therefore, the experiments on different datasets will change the number of neighbors according to the number of projects owned by the dataset, which better reflects the change of the MAE value generated by the algorithm when the value changes.

3.4 Performance against traditional approaches

In addition, the MAE comparisons of the proposed method SVR-based CF and the traditional KNN item-based method on different datasets are shown in Table 3. It can be found that the method proposed in this paper performs better on most of the above datasets, each of which the sparsity is very high. By predicting the SVR model iteratively and extending the trusted data reasonably, it can avoid the problem that the sparsity of the data set is higher than 90%, which makes it difficult for different algorithms and supports the shortcomings of statistical significance [26, 27]. However, when the sparsity of the Jester dataset is not very high, as shown in Table 2, it means that the missing values in the U-I matrix account for the proportion of the entire matrix size, and the missing score is less than 90%. Therefore, traditional algorithms still stand out in predicting the missing values of such data sets.

Table 4
Calculation time

Datasets	Neighbor $k$	KNN item-based (s)	SVR-based CF (s)
Movielens	120	29	407
ML-latest	50	29	251
Jester	150	83	993
Eachmovie	240	1188	2378
LastFM	100	349	1033

The other advantage of the proposed method is that with SVR model, we make it better to utilize the tag information and timing information, and the other data attributes as vectors for SVR to do regression works. In this way, we don’t need analyze the semantics of the attributes anymore and just treat them as special vectors in the training process.

Moreover, the time cost comparisons are shown in Table 4. With more accurate results, the proposed method using the iteratively predicted SVR locally increases the time consuming which is not too large. Meanwhile, it is evidently that using SVR locally is time-saving than using SVR globally.

Although the proposed method achieves better results than KNN item-based method in most cases, it also has some shortcomings. When facing different datasets, we need to choose different amounts of data to train the SVR model because different attributes result in different vector dimension. The SVR model has the possibility of over fitting, so we should choose the number of items as vector number and the user number as data quantity properly, which is also the next work in the future.

4. Conclusions

In this paper, we present a method to improve the traditional KNN item-based recommendation algorithm. It divides the prediction into two steps of predicting the dense part and the sparse part of a dataset, and combining the precise prediction of SVR and the scalability, time-saving prediction of the item-based algorithm. The proposed method enriches the data in a reasonable way to solve the data sparsity problem, even though the pre-predicted data from SVR-based method are not a hundred percent precise. The experiment results indicate that the pre-prediction of the data to enrich the sparse information are mostly better than lacking information especially when the data is extremely sparse as more than 90% lack of information, with the characteristic of small sample, high dimension and nonlinear. However, the presented method is too agility to decide which part of data should be called dense and which part should be called sparse. In order to make the algorithm better, we need to select the dense part of dataset adaptively rather than manually. In the future, we can calculate each single missing value from the most dense part gradually to the global area of U-I matrix. Another way is to utilize the context information as new vector features, and the SVR model doesn’t need to analyze the semantics of added features so the features will be easily added.

Footnotes

Acknowledgments

This work was supported by the Fundamental Research Funds for the Central Universities of China (Grant No. 3122015C021).

References

Sarwar

et al., Item-based collaborative filtering recommendation algorithms, in: 10th International Conference on World Wide Web, 2001, pp. 285–295.

Revankar

O.S.

and Haribhakta

Y.V.

, Survey on collaborative filtering technique in recommendation system, International Journal of Applicantion or Innovation in Engineering & Management 4(3) (2015), 85–91.

Shi

Larson

and Hanjalic

, Collaborative filtering beyond the user-item matrix: a survey of the state of the art and future challenges, ACM Computing Surveys 47(1) (2014), 1–45.

Kurdija

A.S.

et al., Efficient global correlation measures for a collaborative filtering dataset, Knowledge-based Systems 147 (2018), 36–42.

Wang

et al., Research on schedule-based user recommendation model based on improved K-means algorithm, Journal of Computational Methods in Sciences and Engineering 16(3) (2016), 691–700.

Jain

and Dixit

V.S.

, Recommendations with context aware framework using particle swarm optimization and unsupervised learning, Journal of Intelligent and Fuzzy Systems 36(5) (2019), 4479–4490.

Gupta

and Dixit

V.S.

, Scalable online product recommendation engine based on implicit feature extraction domain, Journal of Intelligent and Fuzzy Systems 34(3) (2018), 1503–1510.

Maleszka

, A method for profile clustering using ontology alignment in personalized document retrieval systems, Computational Collective Intelligence (2015), 410–420.

et al., A novel robust recommendation method based on kernel matrix factorization, Journal of Intelligent and Fuzzy Systems 32(3) (2017), 2101–2109.

10.

Liu

et al., Kernelized matrix factorization for collaborative filtering, in: 16th SIAM International Conference on Data Mining, 2016, pp. 378–386.

11.

Xian

et al., New SVD-based collaborative filtering algorithms with differential privacy, Journal of Intelligent and Fuzzy Systems 33(4) (2017), 2133–2144.

12.

García-Floriano

et al., Support vector regression for predicting software enhancement effort, Information and Software Technology 97(2018), 99–109.

13.

Koren

, Factorization meets the neighborhood: a multifaceted collaborative filtering model, in: 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2008, pp. 426–434.

14.

Panniello

Tuzhilin

and Gorgoglione

, Comparing context-aware recommender systems in terms of accuracy and diversity, User Modeling and User-Adapted Interaction 24(1) (2014), 35–65.

15.

Polato

and Aiolli

, Exploiting sparsity to build efficient kernel based collaborative filtering for top-N item recommendation, Neurocomputing 268 (2017), 17–26.

16.

Zhang

and Iyengar

V.S.

, Recommender systems using linear classifiers, Journal of Machine Learning Research 2(3) (2002), 313–334.

17.

Polatidis

and Georgiadis

C.K.

, A dynamic multi-level collaborative filtering method for improved recommendations, Computer Standards and Interfaces 51 (2017), 14–21.

18.

Shi

Larson

and Hanjalic

, Collaborative filtering beyond the user-item matrix: a survey of the state of the art and future challenges, ACM Computing Surveys 47(1) (2014), 1–45.

19.

Jhalani

Kant

and Dwivedi

, A linear regression approach to multi-criteria recommender system, in: 1st International Conference on Data Mining and Big Data, 2016, pp. 235–243.

20.

Nilashi

et al., A multi-criteria collaborative filtering recommender system using clustering and regression techniques, Journal of Soft Computing and Decision Support Systems 3(5) (2016), 24–30.

21.

Harper

F.M.

and Konstan

J.A.

, The movieLens datasets: history and context, ACM Transactions on Interactive Intelligent Systems 5(4) (2016), 1–19.

22.

Herlocker

J.L.

et al., An algorithmic framework for performing collaborative filtering, in: 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1999, pp. 230–237.

23.

Goldberg

et al., Eigentaste: A constant time collaborative filtering algorithm, Information Retrieval 4(2) (2001), 133–151.

24.

http://www.cs.cmu.edu/∼lebanon/IR-lab/data.html.

25.

http://www.last.fm.

26.

Grcar

et al., Data sparsity issues in the collaborative filtering framework, in: 7th International Workshop on Knowledge Discovery on the Web, 2006, pp. 58–76.

27.

Yagci

A.M.

Aytekin

and Gurgen

F.S.

, Scalable and adaptive collaborative filtering by mining frequent item co-occurrences in a user feedback stream, Engineering Applications of Artificial Intelligence 58(1) (2017), 171–184.

SVR-based collaborative filtering algorithm: From dense to sparse

Abstract

Keywords

1. Introduction

2. Proposed approach

2.1 Notation

2.2 Motivation

3.1 Datasets

Table 3 MAE values for different datasets

Table 4 Calculation time

Footnotes

Acknowledgments

References

Table 3
MAE values for different datasets

Table 4
Calculation time