TSDNet: A two-stage decomposition-based hybrid deep neural network for long-term time series forecasting

Abstract

Due to recent developments in deep learning models, the field of time series forecasting has undergone significant advancements related to forecasting accuracy and reliability across various industrial sectors. Unfortunately, traditional deep learning models often encounter long-term and multivariate forecasting challenges due to complex temporal patterns. To overcome this, decomposition-based approaches have been proposed. However, there have been few attempts to utilize an appropriate network type for each decomposed component. In this paper, we propose the two-stage decomposition-based hybrid deep neural network (TSDNet) for enhancing the accuracy of long-term time series forecasting. To effectively manage complicated time series data with varying periodicities, TSDNet accommodates a single linear layer for forecasting smooth trend components and a convolutional module for complex seasonal components. Extensive experiments on various benchmark and real-world financial datasets show that TSDNet mostly improves the forecasting accuracy compared to the existing methods considered, particularly in long-term forecasting scenarios. Furthermore, ablation studies were conducted to examine the impact of the number of decomposition stages and the implementation of different modules on the decomposed elements, suggesting the effectiveness of the proposed approach.

Keywords

time series analysis time series decomposition long-term time series forecasting multivariate time series forecasting deep neural networks

1. Introduction

Time series analysis has increasingly gained much attention across various real-world decision-making domains, such as energy, transportation, finance, economics, meteorology, and healthcare.^1–5 This analytical process plays a crucial role in forecasting future scenarios by focusing on understanding past and present trends. For instance, precisely forecasting real-time demands and supplies in the energy sector is essential for efficient resource allocation.⁶ In finance and economics domains, time series forecasting forms the foundation for understanding market volatility and predicting future trends.⁷

Different from image and text data analytics, traditional models such as autoregressive integrated moving average (ARIMA)⁸ and state-space models were widely utilized in the time series forecasting domain. These models were demonstrated for their robustness in handling univariate data and effectiveness in capturing linear relationships over time. Despite their effectiveness, these traditional models might pose limitations in managing the complexities of modern time series forecasting tasks and understanding correlation, seasonality, and trends.^9–11 Moreover, they usually require extensive manual tuning to adjust internal components such as trends and seasonality.

In recent decades, remarkable advances in deep learning networks have introduced a new dimension in time series forecasting. Specifically, deep learning models such as feedforward neural networks, recurrent neural networks (RNNs), convolutional neural networks (CNNs), and Transformers have shown outstanding performance.¹² Yet, since most existing models focused on short-term and univariate forecasts based on past data, they face challenges in forecasting long-term and multivariate time series.¹³ Moreover, they often inadequately capture the complex interactions among different types of input data in multivariate time series.¹⁴

To address the shortcomings of standard deep learning models in time series forecasting, the following two major approaches were introduced. First, several researchers proposed new network architectures that are specifically designed for accommodating long-term dependencies and multivariate time series data. For example, time convolutional network models were devised to adapt CNNs for time series data.¹⁵ The authors in Wu et al.¹⁶ introduced the dimension transformation of time series analysis, which enables more sophisticated interpretations by navigating the complex scales and correlations inherent to multivariate data.

Second, time series decomposition approaches were adopted to address the limitations of traditional deep learning models in time series forecasting. In general, these approaches are performed by dividing time series into trends and seasonal components to analyze these elements separately as inputs for a forecasting model.¹⁷ From the perspective of combining deep neural networks and decomposition techniques, some studies utilize both the original and decomposed elements for predictions using complex networks.^18,19 More pertinently, DLinear²⁰ demonstrated the effectiveness of linear feedforward neural networks by utilizing seasonal and trend elements of multivariate time series data.

In this paper, we propose a novel two-stage decomposition-based hybrid deep neural network (TSDNet) that decomposes a single time series into four separate time series for long-term time series forecasting. Motivated by DLinear²⁰ and TimesNet Wu et al.,¹⁶ TSDNet utilizes both simple linear layers and a specialized convolutional module to effectively accommodate complex time series data with varying periodicities. Specifically, a single linear layer and a convolutional module are employed to forecast smooth trend components and complex seasonal components, respectively. Furthermore, TSDNet employs a parallel structure similar to DLinear for multivariate time series forecasting, enabling independent forecasts for each univariate series obtained from multivariate time series. Extensive experiments on six benchmarks and five real-world financial datasets demonstrated that the proposed method mostly outperformed the forecasting accuracy compared to existing time series forecasting methods.

The key contributions of this paper are listed as follows:

The proposed TSDNet treats multivariate time series as a collection of univariate series, each of which is then decomposed into multiple univariate sequences to enhance forecasting accuracy.

The effectiveness and validity of TSDNet are analyzed by conducting two ablation studies related to time series decomposition stages and deep learning modules.

The proposed approach demonstrates the superior performance over the recent long-term time series forecasting methods on several ETT and Weather datasets, along with five real-world financial datasets, while providing ease of implementation.

This paper is organized as follows. A review of recent advancements in time series forecasting models is provided in Section 2. In Section 3, we define the problem considered in this paper and introduce the structure of the proposed TSDNet. Section 4 presents the experimental design, including the datasets, evaluation metrics, and comparison models. Experimental results and their analysis are presented in Section 5, and finally, the conclusion is drawn in Section 6.

2. Related work

2.1. Traditional methods in time series forecasting

The field of time series prediction has attracted substantial interest due to its wide-ranging applications across various sectors. For instance, linear models such as exponential smoothing²¹ and the ARIMA model were foundational methods in time series analysis for decades, which are acclaimed for their simplicity and effectiveness in certain scenarios. However, these traditional linear models pose inherent limitations when they are confronted with nonlinear patterns in data as well as high-dimensional and multivariate time series. These limitations become more complicated as the complexity and volume of data increase. To overcome these limitations, more advanced models were developed and investigated, including the vector autoregressive moving average model,²² support vector regression,²³ ridge regression,²⁴ and nonparametric methods such as the Gaussian process.²⁵ Although these approaches can successfully deal with nonlinear and multidimensional data, there are still two challenges that need to be addressed. First, they often encounter computational complexity and difficulty in managing high-dimensional data. Second, their performance is highly likely to depend on the time-intensive feature engineering that is done by the domain experts.

2.2. Deep learning-based models for time series forecasting

In response to the challenges of the traditional time series forecasting models, there have been substantial attempts to develop deep learning-based models. In particular, RNN-based models such as the long short-term memory (LSTM)²⁶ and gated recurrent units²⁷ are known to be effective for extracting latent features from high-dimensional data. These models are recursively trained and updated at each time step for capturing temporal dependencies within the data. To further enhance the capability of a model by focusing on the significant interval of time series data, dual-stage attention-based recurrent neural networks²⁸ and dual self-attention temporal point processes recurrent neural networks²⁹ were proposed.

On the other hand, CNN-based models have gained significant attraction in time series analysis. These models offer a distinct approach to time series analysis, emphasizing the importance of spatial relationships and patterns in data interpretation. The authors of Borovykh³⁰ proposed the dilated CNN, which is characterized by its stacks of dilated convolutions to encompass a broad historical range. The dual self-attention network³¹ was designed to handle complex and nonlinear dependencies inherent in time series data. The sample convolution and interaction network (SCINet), which was introduced in Liu et al.,³² employs a series of connected convolutional and activation operations, allowing for the extraction of features from multiple perspectives. In a study by Silva et al.,³³ the original SCINet model was improved by incorporating skip connections into the structure. Building upon these developments, recently, TimesNet¹⁶ showcased an impressive performance, marking a significant advancement in time series forecasting. This model employed a modular architecture that effectively transforms one-dimensional time series data into two-dimensional tensors, utilizing multi-periodicity to capture and model complex temporal variations.

To take advantage of RNN and CNN-based approaches, several studies proposed hybrid deep learning models, such as a convolutional recurrent neural network,³⁴ by incorporating the unique capabilities of CNNs and RNNs. The network proposed in Lai et al.³⁵ aims to combine long and short-term patterns of time series data to support a more sophisticated analysis. Similarly, memory time series network³⁶ was designed to handle complex temporal patterns and dependencies in time series data. Gu et al.³⁷ proposed a novel hybrid approach that amalgamates RNNs, temporal convolutional neural networks, and neural differential equations to overcome the individual limitations of these techniques by integrating their strengths.

Recently, Transformer-based models and self-attention mechanisms³⁸ have been widely employed¹² due to their ability to capture long-term dependencies and balance predictability and model capability. Li et al.³⁹ proposed the LogSparse Transformer, which enhances the Transformer architecture with convolutional and sparse attention for improved efficiency. The Reformer⁴⁰ improved the original model by devising locality-sensitive hashing and reversible layers for computational efficiency. The authors in Zhou et al.⁴¹ proposed the Informer that employs prob-sparse self-attention and distilling techniques to reduce computational demands. The model proposed in Lim et al.⁴² utilized specialized components for feature selection and adopted gating layers to suppress unnecessary information. Further innovations were made by Pyraformer,⁴³ which utilizes a pyramidal attention structure to balance the analysis of short-term and long-term dependencies while minimizing computational complexity. The ETSformer⁴⁴ improved upon traditional Transformers by incorporating exponential smoothing, enhancing accuracy and interoperability. The authors of Chowdhury et al.⁴⁵ enhanced the Transformer model by focusing on task-aware data reconstruction to augment end-task performance. In addition,⁴⁶ proposed the non-stationary Transformers, which is a versatile framework characterized by making series stationary and adjusting attention modules. Finally, the CLformer was developed by Wang et al.,⁴⁷ which integrates convolution and locally grouped autocorrelation into the Transformer framework. More recently, by operating on variate tokens instead of temporal tokens, iTransformer⁴⁸ aims to enhance multivariate correlation modeling and achieves state-of-the-art performance in real-world forecasting.

In contrast to Transformer-based models, several studies proposed more straightforward approaches such as linear layers and multi-layer perceptron (MLP) block stacking to achieve forecasting performances comparable to those of Transformer-based models. N-Beats,⁴⁹ NBEATSx by Olivares et al.,⁵⁰ and N-Hits by Challu et al.⁵¹ initially verified the effectiveness of block stacking structures in streamlined MLP-based frameworks. Further emphasizing this trend,⁵² introduced LightTS, which utilizes a streamlined MLP structure with dual down-sampling strategies. These models demonstrated that complexity is not always a prerequisite for high performance in time series analysis.

2.3. Decomposition approaches for time series forecasting

Generally, the decomposition of time series data is achieved by separating the original data into several components. The decomposition-based approaches have been known to be highly effective for isolating and analyzing elements such as seasonal patterns, underlying trends, and residual factors in time series datasets.⁵³ One of the most prevalent decomposition methods is seasonal-trend decomposition using loess (STL)¹⁷ that aims to separate the time series into trend, seasonality, and residual parts. Additionally, advancements in decomposition techniques that employ Fourier transforms and wavelet-based multi-resolution analysis^54,55 offered alternative perspectives in decomposing time series data.

Due to the effectiveness of deep neural networks, there have been increasing attempts to integrate deep learning models with time series decomposition techniques.⁵⁶ This integration aims to harness the proficiency of deep learning in identifying complex patterns, particularly when employed to the decomposed trends and seasonal components of time series data. In this context, innovative models such as Autoformer¹⁸ and Fedformer⁵⁷ have emerged. Autoformer integrates a decomposition-based approach with an auto-correlation mechanism to significantly enhance long-term time series forecasting capabilities. Similarly, Fedformer combines seasonal-trend decomposition with frequency enhancement within a Transformer framework to capture global and detailed patterns for extended forecasting.

Another decomposition-based approach was introduced by utilizing contrastive learning methods for learning disentangled seasonal-trend representations.⁵⁸ The authors of Huang et al.⁵⁹ proposed a dual-channel network model designed explicitly for long-term forecasting in time series data. They employed a deep cross-decomposition method to effectively separate and extract features from seasonal and trend-cyclical components, demonstrating their adeptness in managing complex time series data. Building on these advancements, DLinear, which demonstrates its simplicity and robustness, has been recently proposed by Zeng et al.²⁰ This model, which is capable of predicting long sequences through a single time series decomposition, verified the potential of straightforward linear approaches in tackling complex time series forecasting tasks. As a result, the development of DLinear underscored the growing recognition of simpler yet effective models in time series forecasting. Thus, decomposition approaches were demonstrated to be instrumental since they contribute to understanding the inherent complexities of time series data and creating more accurate and efficient forecasting models. The proposed two-stage decomposition approach will be described in Section 3.2.

3. Methods

3.1. Problem definition and notations

We introduce the definition and notations for the proposed time series forecasting model. Let $t \in N$ be a time step, where $N$ represents the set of the natural numbers. A time series predictive model aims to forecast future values based on current and past states at given time steps. The input and target variables are denoted as $x_{t} \in R^{m}$ and $y_{t} \in R^{n}$ , respectively, where $m$ and $n$ respectively refer to the numbers of features and target variables. For example, both $x_{t}$ and $y_{t}$ include different types of information, such as sensor metrics, prices, or water temperatures.

Let denote the part of the time series from the first to the $t$ -th time point as $x_{1 : t}$ , and a fixed-length look-back window as $L \in N$ . For each $t$ , the objective of the proposed model is to forecast $y_{t + 1 : t + O} = {y_{t + 1}, \dots, y_{t + O}}$ $\in$ $R^{O \times n}$ by using the past data $x_{t - L + 1 : t} = {x_{t - L + 1}, \dots, x_{t}}$ $\in$ $R^{L \times m}$ . Here, $O \in N$ indicates the forecasting horizon. Finally, the predictive model $F$ is formally expressed as:

{\hat{y}}_{t + 1 : t + O} = F (x_{t - L + 1 : t}; θ),

(1)

where

{\hat{y}}_{t + 1 : t + O} = {{\hat{y}}_{t + 1}, \dots, {\hat{y}}_{t + O}}

\in R^{O \times n}

represents the series forecasted by the model, and

θ

symbolizes the set of trainable parameters within the model.

In this paper, we address the time series forecasting problem where $m = n$ . Note that if $m = n = 1$ , it corresponds to a univariate time series forecasting problem. In cases where $m = n > 1$ , the problem can be transformed into a multivariate time series forecasting, indicating that the joint histories of $m$ input variables are used to predict $m$ target variables.

3.2. The proposed model

3.2.1. The overview of TSDNet

Figure 1 provides a comprehensive overview of the proposed TSDNet. The core feature of TSDNet is a two-stage decomposition in which the input time series is split into four distinct subseries. Each subseries is individually processed through customized deep learning architectures to enhance forecasting accuracy. When TSDNet conducts multivariate time series forecasting with $m$ variables, they are divided into $m$ univariate time series. Finally, TSDNet constructs a forecasting result of multivariate time series by concatenating $m$ individual forecasts, as presented in Zeng et al.²⁰ Therefore, without loss of generality, we will present the structure and rationale of the proposed TSDNet from the perspective of univariate time series forecasting in the rest of this paper. Further details on specific functions and roles of each component within the architecture are described in the following sections.

Figure 1.

Simple overview of TSDNet architecture.

3.2.2. Instance normalization

To alleviate the impact of the scale for individual instances, we utilize instance normalization⁶⁰ in which the input data is normalized for each instance separately rather than the entire batch. This process is performed on the input $X \in R^{L \times 1}$ for each instance in a batch, resulting in the normalized form $\tilde{X} \in R^{L \times 1}$ as follows:

\tilde{X} = I N (X) = \frac{X - μ_{X}}{\sqrt{σ_{X}^{2} + ϵ}},

(2)

where

μ_{X}

and

σ_{X}^{2}

denote the mean and variance of

X

, respectively, and

ϵ

is a small constant introduced to ensure numerical stability. The prediction

\tilde{Y} \in R^{O \times 1}

is then generated from

\tilde{X}

as follows:

\tilde{Y} = F (\tilde{X}) .

(3)

Subsequently, a denormalization step is applied to revert the model output to the original scale of the time series, as suggested by Ogasawara et al.⁶¹ The final prediction, denoted as

\hat{Y} \in R^{O \times 1}

, is obtained by readjusting

\tilde{Y}

using the instance-specific parameters:

\hat{Y} = I N^{- 1} (\tilde{Y}) = \tilde{Y} \times \sqrt{σ_{X}^{2} + ϵ} + μ_{X} .

(4)

Note that

\tilde{Y}

is rescaled using the instance-specific standard deviation

\sqrt{σ_{X}^{2} + ϵ}

and recentered by the mean

μ_{X}

to produce the final forecast

\hat{Y}

3.2.3. Two-stage time series decomposition

Time series forecasting poses significant challenges due to factors such as seasonality and unpredictable fluctuations. To address these challenges, this paper proposes a systematic decomposition of time series, which identifies and isolates fundamental components to simplify the underlying structure and enhance the precision of subsequent predictive models. Drawing on the principles of DLinear,²⁰ we employ a two-stage procedure that extracts trends from seasonal components using a moving average and then repeats this process for more refined decomposition. Through the proposed two-stage decomposition, we can dissect the time series into elemental parts and elucidate different data characteristics, thereby enhancing the stationarity within the series.⁵³ The implementation of this two-stage decomposition is illustrated in Figure 2.

Figure 2.

The example of two-phase time series decomposition.

First, $\tilde{X}$ is decomposed into its trend and seasonal components that are described as follows:

\begin{aligned} T, S & = S e r i e s D e c o m p (\tilde{X}), \\ \tilde{X} & = T + S . \end{aligned}

(5)

In this first stage of decomposition, the trend component, denoted as

T

, is obtained by employing a moving average to

\tilde{X}

. On the other hand, the seasonal component, denoted as

S

, represents the remainder of the series after the trend component has been extracted. Next, the second decomposition is performed on the trend and seasonal components obtained from the first stage:

\begin{aligned} T_{t}, T_{s} & = S e r i e s D e c o m p (T), \\ T & = T_{t} + T_{s}, \end{aligned}

(6)

\begin{aligned} S_{t}, S_{s} & = S e r i e s D e c o m p (S), \\ S & = S_{t} + S_{s}, \end{aligned}

(7)

where (

T_{t}

T_{s}

) and (

S_{t}

S_{s}

) represent the subseries obtained by applying the second decomposition to

T

and

S

, respectively. As a result, the following equation holds for

\tilde{X}

\tilde{X} = T_{t} + T_{s} + S_{t} + S_{s} .

(8)

3.2.4. Linear module

Based on the findings from Zeng et al.²⁰ and Li et al.,¹⁴ it was demonstrated that utilizing a linear model structure can significantly enhance the forecasting accuracy of cyclical, long-term time series. Inspired by these insights, we adopt a linear module to the trend components of the time series, which is typically characterized by their smoother and more periodic natures. Let $f_{L i n} (\cdot)$ represent the linear module used in this paper, which is defined as follows:

f_{L i n} (X) = F e e d F o r w a r d (X) where f_{L i n} : R^{L \times 1} \to R^{O \times 1} .

(9)

3.2.5. Convolutional module

Different from trend components, seasonal components often contain various intricate and irregular patterns that are not fully addressed by trend decomposition. Therefore, these seasonal components are dealt with in a convolutional module to effectively capture their complex characteristics. To effectively accommodate these seasonal aspects, we utilize the $T i m e B l o c k$ from TimesNet,¹⁶ which has recently demonstrated promising results in forecasting seasonal components of time series. The following Equation represents the convolutional module used in this paper:

f_{C o n v} (X) = C r o p (T i m e B l o c k (F e e d F o r w a r d (X)), O) where f_{C o n v} : R^{L \times 1} \to R^{O \times 1} .

(10)

Subsequently, we will describe the details of equation (10).

Data length extension

To effectively extract the relevant information from time series data, a data length extension is applied to the input of $T i m e B l o c k$ . The rationale behind this is due to the following two reasons: First, convolutional networks need to match input and output dimensions. Since the input and output dimensions of TSDNet are $L$ and $O$ , respectively, they are required to be the same length. Second, extending the input sequence is highly likely to effectively capture more comprehensive information from the given time series. To achieve this, a single linear layer is incorporated by combining the total lengths of the input and output sequences. Specifically, the data length extension from the size $L$ to $L + O$ is mathematically expressed as follows:

f_{C_{1}} (X) = F e e d F o r w a r d (X) where f_{C_{1}} : R^{L \times 1} \to R^{(L + O) \times 1} .

(11)

Implementation of TimeBlock

CNNs are generally acclaimed for their proficiency in capturing two-dimensional (2D) variations using 2D kernels. However, the input of TSDNet corresponds to a one-dimensional (1D) form. To exploit the advantages of CNNs for time series forecasting, we incorporate $T i m e B l o c k$ structure.¹⁶ This structure transforms 1D time series data into a 2D format, thereby enabling the implementation of 2D kernel convolution layers. We note that TSDNet directly utilizes $T i m e B l o c k$ for the purpose of forecasting, which is different from TimesNet which employs $T i m e B l o c k$ for residual learning.

In $T i m e B l o c k$ , we employ the fast Fourier transform (FFT)⁶² to transform 1D time series data into a 2D format, and vice versa. This transformation process, depicted in Figure 3, is initiated by applying FFT to the 1D time series data to identify the $k$ most significant periodic functions, each characterized by the largest amplitude, and then calculate their corresponding period lengths using their frequencies. For the $i$ th frequency, denoted as $f_{i}$ among the top $k$ frequencies, the period length $p_{i}$ is calculated as:

p_{i} = ⌈ \frac{L + O}{f_{i}} ⌉, i \in {1, \dots, k} .

(12)

These calculated period lengths (

p_{i}

) are then utilized to reshape the original 1D data into

k

separate 2D data. This reshaping ensures complete 2D data formation, typically achieved through length adjustment, often involving rounding up to the nearest whole number. The resulting

k

2D data are then processed using 2D kernel convolution layers. This approach efficiently captures variations within and across periods, enabling the extraction of a broad spectrum of temporal 2D variations and enhancing the model forecasting capabilities.

Figure 3.

FFT-based 2D transformation procedure.

$T i m e B l o c k$ is defined as follows.

f_{C_{2}} (X) = T i m e B l o c k (X) where f_{C_{2}} : R^{(L + O) \times 1} \to R^{(L + O) \times 1} .

(13)

In equation (13),

f_{C_{2}} (X)

functions as

T i m e B l o c k

, as depicted in Figure 4. First, the 1D input data entering

T i m e B l o c k

are transformed into

k

2D tensors, based on the calculated frequencies and period lengths derived from FFT. Each tensor is then processed through separate CNNs, known as inception blocks,⁶³ which are utilized for their ability to capture and predict complex temporal patterns by processing data at multiple scales. The 2D tensors obtained from the inception blocks are transformed back into the original 1D data size, and a weighted sum using the top

k

amplitudes is performed to make the final prediction.

Figure 4.

$T i m e B l o c k$ structure employed in TSDNet.

After implementing $T i m e B l o c k$ , the cropping function is employed to adjust an $(L + O)$ -dimensional sequence to the output with length $O$ . The cropping operation, denoted as $C r o p (\cdot, O)$ , is defined as below.

f_{C_{3}} (X) = C r o p (X, O) = X [L + 1 : L + O] where f_{C_{3}} : R^{(L + O) \times 1} \to R^{O \times 1} .

(14)

This operation extracts the last

O

elements from the time series

X

. As a result, the length of the time series is reduced from

L + O

O

, which aligns with the final target prediction length.

Finally, the convolutional module used in this paper, as previously defined in equation (10), can be summarized as follows:

f_{C o n v} (X) = (f_{C_{3}} \circ f_{C_{2}} \circ f_{C_{1}}) (X) where f_{C o n v} : R^{L \times 1} \to R^{O \times 1} .

(15)

3.2.6. The overall forecasting procedure of TSDNet

The entire forecasting process of TSDNet is summarized in equations (16) to (18).

\begin{aligned} \tilde{X} & = I N (X) . \end{aligned}

(16)

\begin{aligned} \tilde{Y} & = F (\tilde{X}) = f_{L i n} (T_{t}) + f_{L i n} (T_{s}) + f_{C o n v} (S_{t}) + f_{C o n v} (S_{s}) . \end{aligned}

(17)

\begin{aligned} \hat{Y} & = I N^{- 1} (\tilde{Y}) . \end{aligned}

(18)

In addition, Figure 5 highlights the overall prediction procedure of TSDNet. Note that the instance normalization step is omitted in Figure 5 for the sake of conciseness. The procedure begins with a two-stage decomposition using simple moving averages to divide the input

\tilde{X}

into trend and seasonal components. This decomposition is performed again for trend and seasonal elements, which leads to four distinct subseries. As shown in Figure 5, trend components are processed through a linear module. In contrast, the seasonal components undergo a convolutional module to capture a broad spectrum of information. After aggregating predictions from the four subseries, the final forecast

\hat{Y}

is obtained by denormalizing

\tilde{Y}

Figure 5.

Entire structure of TSDNet.

The rationale behind the proposed two-stage decomposition approach is to effectively isolate stable long-term trends and short-term fluctuations, while reducing noise by separately handling residual elements. The trend subseries, which are characterized by their simple and repetitive structures, are processed using a linear module. This choice ensures efficient and accurate modeling of gradual and stable patterns that demand lower computational resources. On the other hand, the seasonal subseries, having been stripped of the trend component, retain irregular and complex fluctuations. A convolutional module is employed to better capture these intricate and diverse temporal dynamics. This module allows for extracting a broader range of features from the seasonal subseries, facilitating a deeper understanding of short-term variations. As a result, TSDNet provides the robust and accurate long-term forecasts $\hat{Y}$ by aggregating and denormalizing the subseries predictions.

4. Experiments

4.1. Datasets

4.1.1. Benchmark time series datasets

We prepared 6 datasets, where 4 datasets were collected from real-world datasets for long-term time series forecasting, and the others were from public benchmarks. The details of 6 datasets are described as follows:

Electricity Transformer Temperature (ETT): The ETT dataset⁴¹ comprises power transformer data collected between July 2016 and July 2018 from two counties in China, featuring seven variables including a target value of oil temperature and six power load parameters. The dataset is available in two granularities: ETTh1 and ETTh2 are recorded hourly, and ETTm1 and ETTm2 are recorded every 15 minutes, respectively.

Exchange-Rate: The collection of the daily exchange rates of eight foreign countries, including Australia, British, Canada, Switzerland, China, Japan, New Zealand, and Singapore, ranging from 1990 to 2016.³⁵

Weather: The weather dataset is climate data recorded every 10 min at weather stations within the Max-Planck Institute for Biogeochemistry throughout 2020. Each data point contains 21 climate indicators, such as barometric pressure, temperature, and humidity.

Table 1 lists a brief description of 6 datasets. In Table 1 the dataset size indicates the numbers of the training, validation, and test datasets, respectively.

Table 1.
Dataset descriptions.

Dataset Dim Dataset Size Data Frequency

ETTh1, ETTh2 7 (8545, 2881, 2881) Hourly

ETTm1, ETTm2 7 (34465, 11521, 11521) 15 mins

Exchange rate 8 (5120, 665, 1422) Daily

Weather 21 (36792, 5271, 10540) 10 mins

Dataset	Dim	Dataset Size	Data Frequency
ETTh1, ETTh2	7	(8545, 2881, 2881)	Hourly
ETTm1, ETTm2	7	(34465, 11521, 11521)	15 mins
Exchange rate	8	(5120, 665, 1422)	Daily
Weather	21	(36792, 5271, 10540)	10 mins

Table 2.

U.S. Stock dataset information.

U.S. Stocks
Sector	Symbol	Name	Training Period	Validation Period	Test Period
Technology	AAPL	Apple
Finance	JPM	JP Morgan Chase	2007-08-01	2020-01-01	2023-01-01
Healthcare	UNH	UnitedHealth Group	$\sim$	$\sim$	$\sim$
Consumer Goods	PG	Procter & Gamble	2019-12-31	2022-12-31	2024-03-31
Energy	XOM	ExxonMobil

4.1.2. Real-world financial datasets

The U.S. stock datasets utilized in this study consist of five major sectors—Technology, Finance, Healthcare, Consumer Goods, and Energy—featuring representative companies such as Apple (AAPL), JP Morgan Chase (JPM), UnitedHealth Group (UNH), Procter & Gamble (PG), and Exxon Mobil (XOM). Each of the datasets is partitioned to reflect diverse economic conditions, with a training period from August 1, 2007, to December 31, 2019, a validation period from January 1, 2020, to December 31, 2022, and a test period from January 1, 2023, to March 31, 2024. Details of the dataset are presented in Table 2.

4.2. Experiment configurations of TSDNet

In the experiments, TSDNet was implemented by using the PyTorch and trained with the ADAM optimizer⁶⁴ and MSE loss. Specifically, the batch size and initial learning rate were set to 32 and $1.0 \times 10^{- 4}$ , respectively. As mentioned in Section 3.2, for the first decomposition of TSDNet, the trend components were predicted using a single linear model, where a moving average period was set to 25. For $T i m e B l o c k$ , the channel and kernel sizes were set to be 8 and 6, respectively, and the top $k = 5$ frequencies were selected. The other hyperparameters are the same as those in TimesNet.¹⁶

4.3. Evaluation metrics

The two representative metrics, mean square error (MSE) and mean absolute error (MAE), are used to evaluate the performances of time series forecasting models. Generally, MSE is known to be more sensitive to outliers, while MAE provides a more linear perspective of errors. MSE is a common metric used to measure the average squared differences between the predicted and actual values, calculated using the following formula:

MSE = \frac{1}{n} \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})^{2}

(19)

where

y_{i}

and

{\hat{y}}_{i}

are the actual and predicted values, respectively, and

n

indicates the number of observations. Similarly, MAE quantifies the average absolute differences between the predicted and actual values, as expressed in the following equation:

MAE = \frac{1}{n} \sum_{i = 1}^{n} | y_{i} - {\hat{y}}_{i} |

(20)

4.4. Comparison models

To provide a broad perspective on current advancements and various methodologies in time series forecasting, the performance of the proposed method was compared with 16 models from existing literature. This selection spans a range of methodologies, from classical statistical techniques to modern deep learning approaches, facilitating diverse comparisons in both univariate and multivariate time series forecasting. Specifically, the baseline methods evaluated in this paper include ten transformer-based models: Transformer,³⁸ Reformer,⁴⁰ LogTrans,³⁹ Informer,⁴¹ Pyraformer,⁴³ Autoformer,¹⁸ Stationary,⁴⁶ Fedformer,⁵⁷ ETSformer,⁴⁴ and iTransformer.⁴⁸ Each model showcases unique adaptations and innovations within the transformer architecture. In addition, we compared two RNN-based models, LSTM²⁶ and LSSL,³⁷ known for their capability to capture time-dependent patterns in time series data. Furthermore, recent approaches, including TimesNet¹⁶ two MLP-based models, LightTS⁵² and DLinear,²⁰ were also included in the performance comparison.

5. Results

5.1. Performance comparison on multivariate and univariate time series forecasting

This section presents the results of time series forecasting experiments conducted on six diverse datasets, covering both univariate and multivariate scenarios. Throughout these experiments, a consistent input length ( $O$ ) of 96 was maintained, and various prediction lengths ( $L$ ), including 96, 192, 336, and 720 were evaluated. The best and second-best results for each prediction length and metric are highlighted in the results section, with the top performance in bold red and the following best underlined in blue.

For multivariate time series forecasting, TSDNet was assessed using six datasets, including four ETT datasets, along with Exchange and Weather data. Table 3 illustrates the performance of TSDNet in terms of MSE and MAE, compared with other methods. MAE and MSE values for comparison models were obtained from Wu et al.¹⁶ and Liu et al.⁴⁸

Table 3.
MSE and MAE results of TSDNet and other methods for multivariate time series forecasting, sourced from Wu et al.¹⁶ and Liu et al.⁴⁸

Models		TSDNet (Ours)		TimesNet¹⁶		iTransformer⁴⁸		ETSformer⁴⁴		LightTS⁵²		DLinear²⁰		FEDformer⁵⁷		Stationary⁴⁶		Autoformer¹⁸		Pyraformer⁴³		Informer⁴¹		LogTrans³⁹		Reformer⁴⁰		LSSL³⁷		LSTM²⁶
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm2	96	0.178	0.258	0.187	0.267	0.180	0.264	0.189	0.280	0.209	0.308	0.193	0.292	0.203	0.287	0.192	0.274	0.255	0.339	0.435	0.507	0.365	0.453	0.768	0.642	0.658	0.619	0.243	0.342	2.041	1.073
	192	0.242	0.300	0.249	0.309	0.250	0.309	0.253	0.319	0.311	0.382	0.284	0.362	0.269	0.328	0.280	0.339	0.281	0.340	0.730	0.673	0.533	0.563	0.989	0.757	1.078	0.827	0.392	0.448	2.249	1.112
	336	0.301	0.339	0.321	0.351	0.311	0.348	0.314	0.357	0.442	0.466	0.369	0.427	0.325	0.366	0.334	0.361	0.339	0.372	1.201	0.845	1.363	0.887	1.334	0.872	1.549	0.972	0.932	0.724	2.568	1.238
	720	0.398	0.394	0.408	0.403	0.412	0.407	0.414	0.413	0.675	0.587	0.554	0.522	0.421	0.415	0.417	0.413	0.433	0.432	3.625	1.451	3.379	1.338	3.048	1.328	2.631	1.242	1.372	0.879	2.720	1.287
	Avg	0.280	0.323	0.291	0.333	0.288	0.332	0.293	0.342	0.409	0.436	0.350	0.401	0.305	0.349	0.306	0.347	0.327	0.371	1.498	0.869	1.410	0.810	1.535	0.900	1.479	0.915	0.735	0.598	2.395	1.177
ETTm1	96	0.335	0.367	0.338	0.375	0.334	0.368	0.375	0.398	0.374	0.400	0.345	0.372	0.379	0.419	0.386	0.398	0.505	0.475	0.543	0.510	0.672	0.571	0.600	0.546	0.538	0.528	0.450	0.477	0.863	0.664
	192	0.375	0.385	0.374	0.387	0.377	0.391	0.408	0.410	0.400	0.407	0.380	0.389	0.426	0.441	0.459	0.444	0.553	0.496	0.557	0.537	0.795	0.669	0.837	0.700	0.658	0.592	0.469	0.481	1.113	0.776
	336	0.405	0.405	0.410	0.411	0.426	0.420	0.435	0.428	0.438	0.438	0.413	0.413	0.445	0.459	0.495	0.464	0.621	0.537	0.754	0.655	1.212	0.871	1.124	0.832	0.898	0.721	0.583	0.574	1.267	0.832
	720	0.472	0.445	0.478	0.450	0.491	0.459	0.499	0.462	0.527	0.502	0.474	0.453	0.543	0.490	0.585	0.516	0.671	0.561	0.908	0.724	1.166	0.823	1.153	0.820	1.102	0.841	0.632	0.596	1.324	0.858
	Avg	0.397	0.400	0.400	0.406	0.407	0.410	0.429	0.425	0.435	0.437	0.403	0.407	0.448	0.452	0.481	0.456	0.588	0.517	0.691	0.607	0.961	0.734	0.929	0.725	0.799	0.671	0.533	0.532	1.142	0.782
ETTh2	96	0.295	0.343	0.340	0.374	0.297	0.349	0.340	0.391	0.397	0.437	0.333	0.387	0.358	0.397	0.476	0.458	0.346	0.388	0.645	0.597	3.755	1.525	2.116	1.197	2.626	1.317	1.616	1.036	2.522	1.278
	192	0.380	0.397	0.402	0.414	0.380	0.400	0.430	0.439	0.520	0.504	0.477	0.476	0.429	0.439	0.512	0.493	0.456	0.452	0.788	0.683	5.602	1.931	4.315	1.635	11.12	2.979	2.083	1.197	3.312	1.384
	336	0.409	0.427	0.452	0.452	0.428	0.432	0.485	0.479	0.626	0.559	0.594	0.541	0.496	0.487	0.552	0.551	0.482	0.486	0.907	0.747	4.721	1.835	1.124	1.604	9.323	2.769	2.970	1.439	3.291	1.388
	720	0.436	0.448	0.462	0.468	0.427	0.445	0.500	0.497	0.863	0.672	0.831	0.657	0.463	0.474	0.562	0.560	0.515	0.511	0.963	0.783	3.647	1.625	3.188	1.540	3.874	1.697	2.576	1.363	3.257	1.357
	Avg	0.380	0.404	0.414	0.427	0.383	0.407	0.439	0.452	0.602	0.543	0.559	0.515	0.437	0.449	0.526	0.516	0.450	0.459	0.826	0.703	4.431	1.729	2.686	1.494	6.736	2.191	2.311	1.259	3.095	1.352
ETTh1	96	0.403	0.413	0.384	0.402	0.386	0.405	0.494	0.479	0.424	0.432	0.386	0.400	0.376	0.419	0.513	0.491	0.449	0.459	0.664	0.612	0.865	0.713	0.878	0.740	0.837	0.728	0.548	0.528	1.044	0.773
	192	0.456	0.441	0.436	0.429	0.441	0.436	0.538	0.504	0.475	0.462	0.437	0.432	0.420	0.448	0.534	0.504	0.500	0.482	0.790	0.681	1.008	0.792	1.037	0.824	0.923	0.766	0.542	0.526	1.217	0.832
	336	0.498	0.458	0.491	0.469	0.487	0.458	0.574	0.521	0.518	0.488	0.481	0.459	0.459	0.465	0.588	0.535	0.521	0.496	0.891	0.738	1.107	0.809	1.238	0.932	1.097	0.835	1.298	0.942	1.259	0.841
	720	0.492	0.476	0.521	0.500	0.503	0.491	0.562	0.535	0.547	0.533	0.519	0.516	0.506	0.507	0.643	0.616	0.514	0.512	0.963	0.782	1.181	0.865	1.135	0.852	1.257	0.889	0.721	0.659	1.271	0.838
	Avg	0.462	0.447	0.458	0.450	0.454	0.448	0.542	0.510	0.491	0.479	0.456	0.452	0.440	0.460	0.570	0.537	0.496	0.487	0.827	0.703	1.040	0.795	1.072	0.837	1.029	0.805	0.777	0.664	1.198	0.821
Weather	96	0.162	0.206	0.172	0.220	0.174	0.214	0.197	0.281	0.182	0.242	0.196	0.255	0.217	0.296	0.173	0.223	0.266	0.336	0.622	0.556	0.300	0.384	0.458	0.490	0.689	0.596	0.174	0.252	0.369	0.406
	192	0.207	0.247	0.219	0.261	0.221	0.254	0.237	0.312	0.227	0.287	0.237	0.296	0.276	0.336	0.245	0.285	0.307	0.367	0.739	0.624	0.598	0.544	0.658	0.589	0.752	0.638	0.238	0.313	0.416	0.435
	336	0.263	0.288	0.280	0.306	0.278	0.296	0.298	0.353	0.282	0.334	0.283	0.335	0.339	0.380	0.321	0.338	0.359	0.395	1.004	0.753	0.578	0.523	0.797	0.652	0.639	0.596	0.287	0.355	0.455	0.454
	720	0.341	0.338	0.365	0.359	0.358	0.347	0.352	0.288	0.352	0.386	0.345	0.381	0.403	0.428	0.414	0.410	0.419	0.428	1.420	0.934	1.059	0.741	0.869	0.675	1.130	0.792	0.384	0.415	0.535	0.520
	Avg	0.243	0.270	0.259	0.287	0.258	0.278	0.271	0.334	0.261	0.312	0.265	0.317	0.309	0.360	0.288	0.314	0.338	0.382	0.946	0.717	0.634	0.548	0.696	0.602	0.803	0.656	0.271	0.334	0.444	0.454
Exchange	96	0.089	0.210	0.107	0.234	0.086	0.206	0.085	0.204	0.116	0.262	0.088	0.218	0.148	0.278	0.111	0.237	0.197	0.323	1.748	1.105	0.847	0.752	0.968	0.812	1.065	0.829	0.395	0.474	1.453	1.049
	192	0.186	0.305	0.226	0.344	0.177	0.299	0.182	0.303	0.215	0.359	0.176	0.315	0.271	0.380	0.219	0.335	0.300	0.369	1.874	1.151	1.204	0.895	1.040	0.851	1.188	0.906	0.776	0.698	1.846	1.179
	336	0.343	0.423	0.367	0.448	0.331	0.417	0.348	0.428	0.377	0.466	0.313	0.427	0.460	0.500	0.421	0.476	0.509	0.524	1.943	1.172	1.672	1.036	1.659	1.081	1.357	0.976	1.029	0.797	2.136	1.231
	720	0.905	0.718	0.964	0.746	0.847	0.691	1.025	0.774	0.831	0.699	0.839	0.695	1.195	0.841	1.092	0.769	1.447	0.941	2.085	1.206	2.478	1.310	1.941	1.127	1.510	1.016	2.283	1.222	2.984	1.427
	Avg	0.381	0.414	0.416	0.443	0.360	0.403	0.410	0.427	0.385	0.447	0.354	0.414	0.519	0.500	0.461	0.454	0.613	0.539	1.913	1.159	1.550	0.998	1.402	0.968	1.280	0.932	1.121	0.798	2.105	1.221

TSDNet excelled in most datasets, particularly the ETT datasets. On ETTm2 and ETTh2 datasets, TSDNet achieved the best performance across all prediction lengths for both MSE and MAE metrics. Specifically, TSDNet showed a 3% improvement in MSE (from 0.288 to 0.280) on ETTm2 and a 1% improvement (from 0.383 to 0.380) on ETTh2, averaged across the four prediction lengths. On the ETTm1 dataset, TSDNet ranked first for prediction lengths of 336 and 720 and achieved the best performance in MAE when considering the average across the four lengths. On the ETTh1 dataset, TSDNet showed weaker performance for shorter prediction lengths. On the other hand, TSDNet performed well at the 336-length prediction for MAE and at 720-length predictions for both MSE and MAE.

In the Weather dataset, TSDNet ranked second in both MSE and MAE for the 720-length prediction but led in most other lengths, achieving the best performance across the four prediction lengths. On the other hand, TSDNet showed weaker performance on the Exchange dataset compared to the other datasets. This may be attributed to the high level of uncertainty and lack of long-term repetitive patterns in financial data, which makes it more complicated to forecast long-term time series.³⁵

Table 4 presents the univariate time series forecasting results on all ETT datasets: ETTm1, ETTm2, ETTh1, and ETTh2. In Table 4, since TimesNet primarily provides multivariate forecasting results, we implemented and tested several baseline models, including TimesNet, iTransformer, DLinear, Autoformer, Informer, and Transformer for univariate forecasting. The hyperparameters of baseline models were the same as those in Wu et al.¹⁶

Table 4.

MSE and MAE results of TSDNet and the other methods for univariate time series forecasting.

Models		TSDNet (Ours)		TimesNet¹⁶		iTransformer⁴⁸		DLinear²⁰		Autoformer¹⁸		Informer⁴¹		Transformer³⁸
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm2	96	0.0692	0.1891	0.0672	0.1879	0.0688	0.1894	0.0730	0.1981	0.0972	0.2391	0.0738	0.2048	0.0880	0.2345
	192	0.1005	0.2350	0.1026	0.2390	0.1095	0.2466	0.1056	0.2421	0.1706	0.3207	0.1366	0.2890	0.1217	0.2662
	336	0.1284	0.2723	0.1316	0.2769	0.1470	0.2911	0.1363	0.2806	0.1417	0.2928	0.1598	0.3101	0.1531	0.3023
	720	0.1782	0.3269	0.1815	0.3308	0.1830	0.3338	0.1906	0.3351	0.1872	0.3410	0.2096	0.3655	0.1807	0.3384
	Avg	0.1191	0.2558	0.1207	0.2587	0.1271	0.2652	0.1264	0.2640	0.1492	0.2984	0.1449	0.2924	0.1359	0.2854
ETTm1	96	0.0292	0.1274	0.0283	0.1257	0.0289	0.1271	0.0327	0.1333	0.0495	0.1720	0.0931	0.2484	0.0548	0.1792
	192	0.0434	0.1582	0.0441	0.1593	0.0489	0.1682	0.0523	0.1697	0.1005	0.2341	0.2100	0.3925	0.2261	0.3820
	336	0.0564	0.1829	0.0600	0.1882	0.0637	0.1933	0.0733	0.2018	0.0914	0.2471	0.2152	0.3827	0.2920	0.4587
	720	0.0792	0.2164	0.0810	0.2184	0.0833	0.2209	0.0985	0.2345	0.1051	0.2575	0.4456	0.5936	0.4861	0.6332
	Avg	0.0520	0.1712	0.0533	0.1729	0.0562	0.1774	0.0642	0.1849	0.0866	0.2277	0.2410	0.4043	0.2648	0.4133
ETTh2	96	0.1301	0.2764	0.1369	0.2871	0.1359	0.2876	0.1460	0.2961	0.1478	0.3003	0.2309	0.3902	0.1988	0.3617
	192	0.1825	0.3325	0.1926	0.3469	0.1847	0.3403	0.1981	0.3463	0.1850	0.3396	0.2636	0.4167	0.2753	0.4270
	336	0.2220	0.3754	0.2249	0.3757	0.2146	0.3704	0.2490	0.3962	0.2345	0.3823	0.3285	0.4594	0.2239	0.3827
	720	0.2366	0.3914	0.2597	0.4049	0.2395	0.3914	0.3382	0.4761	0.2554	0.4002	0.2914	0.4390	0.2300	0.3969
	Avg	0.1928	0.3439	0.2035	0.3536	0.1937	0.3474	0.2328	0.3787	0.2057	0.3556	0.2786	0.4263	0.2320	0.3921
ETTh1	96	0.0556	0.1785	0.0561	0.1822	0.0569	0.1825	0.0706	0.1978	0.0913	0.2403	0.1648	0.3395	0.5015	0.6396
	192	0.0736	0.2066	0.0717	0.2075	0.0726	0.2058	0.0929	0.2296	0.0967	0.2400	0.2884	0.4671	0.4372	0.5965
	336	0.0872	0.2307	0.0863	0.2304	0.0832	0.2233	0.1157	0.2651	0.1082	0.2670	0.2528	0.4266	0.4064	0.5618
	720	0.0845	0.2289	0.0789	0.2238	0.0819	0.2261	0.2036	0.3721	0.1319	0.2890	0.1888	0.3601	0.6510	0.7498
	Avg	0.0752	0.2112	0.0732	0.2110	0.0737	0.2094	0.1207	0.2661	0.1070	0.2591	0.2237	0.3983	0.4990	0.6369

For the ETTm2 dataset, TSDNet achieved the second-best performance for the 96-length prediction and the best for all other lengths, achieving a 1% improvement in mean performance across all four lengths in terms of MSE (from 0.1207 to 0.1191). Similarly, for the ETTm1 dataset, TSDNet ranked third for the 96-length prediction and first for all other lengths, which leads to a 2% improvement in average MSE (from 0.0533 to 0.0520). On the ETTh2 dataset, TSDNet ranked first in both metrics for the 96 and 192 lengths, second for the 336-length prediction, and second in MSE but first in MAE for the 720-length prediction, improving the mean MSE from 0.1937 to 0.1928. Finally, for the ETTh1 dataset, TSDNet demonstrated top performance for the 96-length prediction but ranked third across the remaining lengths, resulting in third-place performance on average for both MSE and MAE. Based on the above observations, TSDNet consistently yielded near-top performance across all datasets and prediction lengths, which demonstrates the robustness and adaptability of the proposed method in various forecasting scenarios.

5.2. Performance comparison on Real-time finance time series forecasting

Table 5 presents the performance of TSDNet and baseline methods in U.S. stock price forecasting. Each method was evaluated for forecast horizons of 10, 20, 30, 60, and 120 days. A 60-day input sequence was used for each method. TSDNet consistently outperformed the other baseline models on AAPL and PG in both MSE and MAE. On the other hand, TSDNet ranked first in MAE for JPM and in MSE for XOM while ranking second in both MSE and MAE for UNH. Among the comparison models, iTransformer yielded the lowest MAEs and MSEs for UNH and XOM, while TimesNet ranked first in MSE for JPM. TSDNet demonstrated stable performance across all forecast horizons and robust adaptability to various stocks, which suggests its efficiency and reliability for U.S. stock price forecasting.

Table 5.
MSE and MAE results of TSDNet and the other methods for U.S. stock closing price forecasting.

Models TSDNet (Ours) TimesNet¹⁶ iTransformer⁴⁸ DLinear²⁰ Autoformer¹⁸ Informer⁴¹ Transformer³⁸

Metric MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE

AAPL 10 0.151 0.287 0.158 0.294 0.177 0.315 0.389 0.542 0.245 0.424 55.73 7.438 44.37 6.617

20 0.266 0.385 0.287 0.395 0.289 0.410 0.506 0.636 0.531 0.629 62.09 7.856 45.51 6.711

30 0.345 0.462 0.397 0.492 0.355 0.471 0.598 0.688 0.713 0.738 66.23 8.116 46.68 6.801

60 0.632 0.646 0.616 0.643 0.588 0.635 0.840 0.769 1.017 0.833 73.06 8.528 55.03 7.394

120 0.581 0.608 0.777 0.723 0.612 0.623 0.946 0.771 1.191 0.877 77.63 8.793 60.00 7.726

Avg 0.395 0.478 0.447 0.509 0.404 0.491 0.656 0.681 0.739 0.700 66.95 8.146 50.32 7.050

JPM 10 0.029 0.131 0.034 0.146 0.030 0.137 0.087 0.242 0.056 0.192 2.017 1.322 0.886 0.809

20 0.060 0.189 0.061 0.194 0.059 0.191 0.112 0.271 0.149 0.308 2.960 1.630 1.016 0.876

30 0.091 0.236 0.092 0.239 0.096 0.245 0.138 0.300 0.148 0.305 2.870 1.639 0.888 0.857

60 0.205 0.358 0.228 0.381 0.243 0.389 0.237 0.378 0.322 0.432 3.929 1.909 1.638 1.189

120 0.182 0.333 0.146 0.297 0.167 0.319 0.183 0.338 0.286 0.428 4.082 1.974 1.895 1.317

Avg 0.113 0.250 0.112 0.251 0.119 0.256 0.151 0.306 0.192 0.333 3.171 1.695 1.265 1.010

PG 10 0.034 0.137 0.038 0.145 0.042 0.155 0.093 0.242 0.071 0.216 3.013 1.714 2.276 1.468

20 0.067 0.197 0.079 0.215 0.076 0.216 0.122 0.285 0.104 0.260 3.608 1.876 2.517 1.545

30 0.089 0.234 0.100 0.251 0.103 0.257 0.133 0.303 0.095 0.239 3.519 1.859 2.312 1.487

60 0.116 0.273 0.141 0.306 0.105 0.264 0.179 0.350 0.110 0.259 4.212 2.029 2.487 1.541

120 0.163 0.319 0.221 0.384 0.152 0.320 0.184 0.349 0.094 0.246 5.037 2.216 1.996 1.364

Avg 0.094 0.232 0.116 0.260 0.096 0.243 0.142 0.306 0.095 0.244 3.878 1.939 2.318 1.481

UNH 10 0.042 0.153 0.049 0.167 0.040 0.154 0.062 0.207 0.049 0.168 9.236 3.019 5.822 2.374

20 0.056 0.185 0.064 0.199 0.059 0.186 0.071 0.220 0.073 0.222 10.81 3.270 5.608 2.335

30 0.071 0.212 0.092 0.232 0.075 0.219 0.081 0.234 0.080 0.231 11.68 3.399 5.975 2.411

60 0.110 0.265 0.133 0.287 0.108 0.266 0.113 0.274 0.116 0.275 11.93 3.435 6.577 2.532

120 0.144 0.304 0.148 0.306 0.132 0.291 0.106 0.268 0.170 0.336 12.73 3.550 7.009 2.625

Avg 0.084 0.224 0.097 0.238 0.083 0.223 0.087 0.241 0.097 0.247 11.28 3.335 6.198 2.455

XOM 10 0.172 0.321 0.187 0.333 0.174 0.318 0.343 0.468 0.244 0.404 18.18 4.217 13.69 3.632

20 0.265 0.403 0.304 0.422 0.269 0.409 0.425 0.513 0.372 0.517 19.90 4.420 12.91 3.539

30 0.355 0.473 0.357 0.468 0.363 0.472 0.489 0.543 0.513 0.611 20.91 4.534 13.47 3.619

60 0.556 0.599 0.563 0.603 0.525 0.580 0.645 0.619 0.605 0.665 22.26 4.676 13.82 3.667

120 0.612 0.635 0.650 0.648 0.629 0.643 0.915 0.765 0.437 0.536 26.36 5.090 16.67 4.031

Avg 0.392 0.486 0.412 0.495 0.392 0.484 0.563 0.582 0.434 0.547 21.52 4.588 14.10 3.698

Models	TSDNet (Ours)	TimesNet¹⁶	iTransformer⁴⁸	DLinear²⁰	Autoformer¹⁸	Informer⁴¹	Transformer³⁸
AAPL	10	0.151	0.287	0.158	0.294	0.177	0.315	0.389	0.542	0.245	0.424	55.73	7.438	44.37	6.617
	20	0.266	0.385	0.287	0.395	0.289	0.410	0.506	0.636	0.531	0.629	62.09	7.856	45.51	6.711
	30	0.345	0.462	0.397	0.492	0.355	0.471	0.598	0.688	0.713	0.738	66.23	8.116	46.68	6.801
	60	0.632	0.646	0.616	0.643	0.588	0.635	0.840	0.769	1.017	0.833	73.06	8.528	55.03	7.394
	120	0.581	0.608	0.777	0.723	0.612	0.623	0.946	0.771	1.191	0.877	77.63	8.793	60.00	7.726
	Avg	0.395	0.478	0.447	0.509	0.404	0.491	0.656	0.681	0.739	0.700	66.95	8.146	50.32	7.050
JPM	10	0.029	0.131	0.034	0.146	0.030	0.137	0.087	0.242	0.056	0.192	2.017	1.322	0.886	0.809
	20	0.060	0.189	0.061	0.194	0.059	0.191	0.112	0.271	0.149	0.308	2.960	1.630	1.016	0.876
	30	0.091	0.236	0.092	0.239	0.096	0.245	0.138	0.300	0.148	0.305	2.870	1.639	0.888	0.857
	60	0.205	0.358	0.228	0.381	0.243	0.389	0.237	0.378	0.322	0.432	3.929	1.909	1.638	1.189
	120	0.182	0.333	0.146	0.297	0.167	0.319	0.183	0.338	0.286	0.428	4.082	1.974	1.895	1.317
	Avg	0.113	0.250	0.112	0.251	0.119	0.256	0.151	0.306	0.192	0.333	3.171	1.695	1.265	1.010
PG	10	0.034	0.137	0.038	0.145	0.042	0.155	0.093	0.242	0.071	0.216	3.013	1.714	2.276	1.468
	20	0.067	0.197	0.079	0.215	0.076	0.216	0.122	0.285	0.104	0.260	3.608	1.876	2.517	1.545
	30	0.089	0.234	0.100	0.251	0.103	0.257	0.133	0.303	0.095	0.239	3.519	1.859	2.312	1.487
	60	0.116	0.273	0.141	0.306	0.105	0.264	0.179	0.350	0.110	0.259	4.212	2.029	2.487	1.541
	120	0.163	0.319	0.221	0.384	0.152	0.320	0.184	0.349	0.094	0.246	5.037	2.216	1.996	1.364
	Avg	0.094	0.232	0.116	0.260	0.096	0.243	0.142	0.306	0.095	0.244	3.878	1.939	2.318	1.481
UNH	10	0.042	0.153	0.049	0.167	0.040	0.154	0.062	0.207	0.049	0.168	9.236	3.019	5.822	2.374
	20	0.056	0.185	0.064	0.199	0.059	0.186	0.071	0.220	0.073	0.222	10.81	3.270	5.608	2.335
	30	0.071	0.212	0.092	0.232	0.075	0.219	0.081	0.234	0.080	0.231	11.68	3.399	5.975	2.411
	60	0.110	0.265	0.133	0.287	0.108	0.266	0.113	0.274	0.116	0.275	11.93	3.435	6.577	2.532
	120	0.144	0.304	0.148	0.306	0.132	0.291	0.106	0.268	0.170	0.336	12.73	3.550	7.009	2.625
	Avg	0.084	0.224	0.097	0.238	0.083	0.223	0.087	0.241	0.097	0.247	11.28	3.335	6.198	2.455
XOM	10	0.172	0.321	0.187	0.333	0.174	0.318	0.343	0.468	0.244	0.404	18.18	4.217	13.69	3.632
	20	0.265	0.403	0.304	0.422	0.269	0.409	0.425	0.513	0.372	0.517	19.90	4.420	12.91	3.539
	30	0.355	0.473	0.357	0.468	0.363	0.472	0.489	0.543	0.513	0.611	20.91	4.534	13.47	3.619
	60	0.556	0.599	0.563	0.603	0.525	0.580	0.645	0.619	0.605	0.665	22.26	4.676	13.82	3.667
	120	0.612	0.635	0.650	0.648	0.629	0.643	0.915	0.765	0.437	0.536	26.36	5.090	16.67	4.031
	Avg	0.392	0.486	0.412	0.495	0.392	0.484	0.563	0.582	0.434	0.547	21.52	4.588	14.10	3.698

5.3. Hyper-parameter sensitivity analysis

We conducted a hyper-parameter sensitivity analysis on the ETTh1 dataset to evaluate the effects of optimizers and loss functions on the performance of TSDNet. To investigate the effectiveness of each optimizer, we tested the three well-known optimizers, including Adam, RMSprop,⁶⁴ and Adagrad,⁶⁵ while keeping the loss function fixed as MSE. As shown in Table 6, Adam consistently performed better than the others.

Table 6.
Comparison of optimizers based on MSE and MAE.

Optimizer Adagrad Adam RMSprop

Metric MSE MAE MSE MAE MSE MAE

96 0.6962 0.5559 0.4034 0.4129 0.4098 0.4173

192 0.7118 0.5691 0.4562 0.4413 0.4663 0.4471

336 0.7309 0.5851 0.4978 0.4575 0.5041 0.4638

720 0.7015 0.5909 0.4921 0.4764 0.5070 0.4886

Avg 0.7101 0.5752 0.4624 0.4470 0.4718 0.4542

Optimizer	Adagrad	Adam	RMSprop
96	0.6962	0.5559	0.4034	0.4129	0.4098	0.4173
192	0.7118	0.5691	0.4562	0.4413	0.4663	0.4471
336	0.7309	0.5851	0.4978	0.4575	0.5041	0.4638
720	0.7015	0.5909	0.4921	0.4764	0.5070	0.4886
Avg	0.7101	0.5752	0.4624	0.4470	0.4718	0.4542

Next, we evaluated the loss functions MSE, MAE, and Huber⁶⁶ by setting the optimizer to Adam based on the above results. Table 7 shows that MSE provided the best results among the tested loss functions. Based on these findings, the combination of Adam and MSE was used for TSDNet in the other experiments.

Table 7.

Comparison of loss functions based on MSE and MAE.

Loss	Huber		MAE		MSE
Metric	MSE	MAE	MSE	MAE	MSE	MAE
96	0.4104	0.4159	0.4069	0.4123	0.4034	0.4129
192	0.4619	0.4440	0.4632	0.4422	0.4562	0.4413
336	0.5047	0.4607	0.5040	0.4578	0.4978	0.4575
720	0.5037	0.4848	0.5054	0.4809	0.4921	0.4764
Avg	0.4702	0.4514	0.4699	0.4483	0.4624	0.4470

5.4. Computational complexity and scalability

Table 8 presents the training time and memory complexity of four representative methods, including TSDNet, TimesNet, DLinear, and iTransformer. The computational complexity of each model depends on three main parameters: $C$ , $L$ , and $D$ , where $C$ represents the number of variables in the dataset, $L$ is the length of the time series, and $D$ denotes the model dimension. Since TSDNet utilizes parts of the TimesNet module to forecast the seasonal component, the complexity $O (L \log L \cdot D)$ is determined from TimesNet whose complexity is lower than that of iTransformer. On the other hand, TSDNet deals with the multivariate components by employing the channel-independent structure of DLinear. Accordingly, $C$ is multiplied by the complexity of the TSDNet, which results in the complexity $O (C \cdot L \log L \cdot D)$ . Although TSDNet can handle each variable due to the $C$ factor, the large $C$ values might increase the training time and complexity of the model. Nevertheless, this impact becomes less significant for datasets with smaller $C$ , such as univariate or low-dimensional multivariate time series.

Table 8.
Comparison of computational complexity across forecasting models.

Methods Training Time Complexity Memory Complexity

TSDNet (Ours) $O (C \cdot L \log L \cdot D)$ $O (C \cdot L \log L \cdot D)$

TimesNet¹⁶ $O (L \log L \cdot D)$ $O (L \log L \cdot D)$

DLinear²⁰ $O (C \cdot L)$ $O (C \cdot L)$

iTransformer⁴⁸ $O (L^{2} \cdot D)$ $O (L^{2} \cdot D)$

Methods	Training Time Complexity	Memory Complexity
TSDNet (Ours)	$O (C \cdot L \log L \cdot D)$	$O (C \cdot L \log L \cdot D)$
TimesNet¹⁶	$O (L \log L \cdot D)$	$O (L \log L \cdot D)$
DLinear²⁰	$O (C \cdot L)$	$O (C \cdot L)$
iTransformer⁴⁸	$O (L^{2} \cdot D)$	$O (L^{2} \cdot D)$

5.5. Ablation study

5.5.1. Ablation of decomposition

To analyze the effects of employing time series decompositions and different modules on TSDNet, we carried out the two ablation studies in this subsection. In ablation studies, we utilized the ETTh2 dataset for both multivariate and univariate scenarios, with each experiment maintaining an input length of 96 and prediction lengths of 96, 192, 336, and 720. The first ablation study was carried out by changing the number of decompositions from 0 to 2, and the decompositions were performed on either trend or seasonal components as presented in Table 9. To investigate only the effects of decompositions on TSDNet, we note that only the linear module $f_{L i n}$ was used.

Table 9.
Configurations for an ablation study to evaluate the effects of time series decomposition frequency.

$X$

$T$ $S$

Experiment Number Number of Decomposition $T_{t}$ $T_{s}$ $S_{t}$ $S_{s}$

D1 0 $f_{L i n}$

D2 1 $f_{L i n}$ $f_{L i n}$

D3 2 $f_{L i n}$ $f_{L i n}$ $f_{L i n}$

D4 2 $f_{L i n}$ $f_{L i n}$ $f_{L i n}$

D5 2 $f_{L i n}$ $f_{L i n}$ $f_{L i n}$ $f_{L i n}$

		$X$
D1	0	$f_{L i n}$
D2	1	$f_{L i n}$	$f_{L i n}$
D3	2	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$
D4	2	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$
D5	2	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$

Tables 10 and 11 display the ablation study results for multivariate and univariate predictions, respectively. Interestingly, a single decomposition was more effective for $L = 720$ . Nevertheless, it was observed that D5 outperformed the other experiment scenarios except for the results on $L = 720$ , and D4 consistently ranked as the second-best. Moreover, as shown in Table 11, D5 and D4 achieved the top and second performance across all scenarios, respectively. Therefore, this implies that the two-stage decomposition is effective for univariate and multivariate time series forecasting.

Table 10.

Ablation study results across various decomposition scenarios for multivariate forecasting.

ETTh2	Output length
	96		192		336		720		Average
Experiment Number	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
D1	0.3215	0.3637	0.4073	0.4136	0.4408	0.4443	0.4440	0.4547	0.4034	0.4191
D2	0.3127	0.3571	0.3985	0.4077	0.4344	0.4402	0.4386	0.4512	0.3960	0.4140
D3	0.3110	0.3561	0.3975	0.4072	0.4333	0.4397	0.4391	0.4514	0.3952	0.4136
D4	0.3104	0.3555	0.3971	0.4069	0.4330	0.4395	0.4389	0.4513	0.3948	0.4133
D5	0.3091	0.3547	0.3959	0.4064	0.4321	0.4393	0.4401	0.4520	0.3943	0.4131

Table 11.

Ablation study results across various decomposition scenarios for univariate forecasting.

ETTh2	Output length
	96		192		336		720		Average
Experiment Number	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
D1	0.1361	0.2854	0.1874	0.3386	0.2308	0.3838	0.2657	0.4163	0.2050	0.3560
D2	0.1332	0.2812	0.1846	0.3351	0.2285	0.3813	0.2635	0.4142	0.2024	0.3529
D3	0.1325	0.2801	0.1838	0.3339	0.2278	0.3805	0.2626	0.4134	0.2017	0.3520
D4	0.1321	0.2797	0.1837	0.3340	0.2277	0.3804	0.2623	0.4132	0.2014	0.3518
D5	0.1318	0.2790	0.1831	0.3330	0.2271	0.3798	0.2617	0.4127	0.2009	0.3511

5.5.2. Ablation of modules

Next, we investigate the effects of employing different modules on the decomposed components. Based on the superiority of the two-stage decomposition shown in Tables 10 and 11, we evaluated the impact of using linear and convolutional modules on trend and seasonal components, Table 12 lists the experiment settings for the second ablation study, where $f_{L i n}$ and $f_{C o n v}$ represent linear and convolutional modules, respectively. Note that the experiment number D5 presented in Table 9 corresponds to M1 in Table 12.

Table 12.
Configurations for an ablation study for analyzing the impact of module combinations.

$X$

$T$ $S$

Experiment Number Number of Decomposition $T_{t}$ $T_{s}$ $S_{t}$ $S_{s}$

M1 2 $f_{L i n}$ $f_{L i n}$ $f_{L i n}$ $f_{L i n}$

M2 $f_{C o n v}$ $f_{C o n v}$ $f_{L i n}$ $f_{L i n}$

M3 (TSDNet) $f_{L i n}$ $f_{L i n}$ $f_{C o n v}$ $f_{C o n v}$

M4 $f_{C o n v}$ $f_{C o n v}$ $f_{C o n v}$ $f_{C o n v}$

		$X$
M1	2	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$
M2		$f_{C o n v}$	$f_{C o n v}$	$f_{L i n}$	$f_{L i n}$
M3 (TSDNet)		$f_{L i n}$	$f_{L i n}$	$f_{C o n v}$	$f_{C o n v}$
M4		$f_{C o n v}$	$f_{C o n v}$	$f_{C o n v}$	$f_{C o n v}$

Tables 13 and 14 present the results of the second ablation study. As shown in Table 13, the performances achieved by M1 and M4 were the near-best in 720-length predictions and shorter predictions, respectively. Nevertheless, it was observed that M3, which corresponds to the proposed TSDNet, exhibited the best average performance.

Table 13.

Multivariate forecasting results obtained by varying the combination of linear and convolutional modules.

ETTh2	Output length
	96		192		336		720		Average
Experiment Number	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
M1	0.3091	0.3547	0.3959	0.4064	0.4321	0.4393	0.4401	0.4520	0.3943	0.4131
M2	0.3056	0.3551	0.3943	0.4082	0.4408	0.4480	0.4614	0.4657	0.4005	0.4192
M3 (TSDNet)	0.2931	0.3423	0.3819	0.3976	0.4198	0.4333	0.4415	0.4519	0.3841	0.4063
M4	0.2926	0.3433	0.3810	0.3983	0.4218	0.4347	0.4533	0.4580	0.3872	0.4086

Table 14.

Univariate forecasting results obtained by varying the combination of linear and convolutional modules.

ETTh2	Output length
	96		192		336		720		Average
Experiment Number	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
M1	0.1318	0.2790	0.1831	0.3330	0.2271	0.3798	0.2671	0.4127	0.2009	0.3511
M2	0.1339	0.2815	0.1844	0.3348	0.2296	0.3825	0.2610	0.4123	0.2022	0.3528
M3 (TSDNet)	0.1298	0.2762	0.1826	0.3326	0.2241	0.3773	0.2459	0.3989	0.1956	0.3463
M4	0.1344	0.2823	0.1853	0.3365	0.2231	0.3760	0.2474	0.4011	0.1976	0.3490

Besides, Table 14 shows that M1 outperformed M4 when $L$ equals 96 and 192, indicating the effectiveness of linear modules in these scenarios. Although MSE and MAE yielded by M4 were lower than those of M3 in $L = 336$ , M3 outperformed the other experiment settings except for $L = 336$ . These findings imply the following two results. First, employing convolution modules across all components does not always reduce MAE and MSE. Second, the forecasting accuracy increases when convolutional modules are implemented for seasonal components. However, their application only to trend components might lead to performance degradation.

6. Conclusion

In this paper, we proposed a novel two-stage decomposition-based hybrid deep neural network for long-term time series forecasting. The proposed approach incorporates a two-stage time series decomposition into four distinct categories of trends and seasonalities. Specifically, we employed a single linear layer for the trend component, while for the seasonal component, we utilized a convolutional module based on $T i m e B l o c k$ . The final prediction is made by combining these decomposed elements to achieve the accuracy and reliability of long-term forecasting.

We conducted extensive experiments using six benchmark datasets and five real-world univariate financial datasets to compare the performance of the proposed method with the other baseline methods for both multivariate and univariate time series forecasting. For multivariate predictions, the proposed method outperformed other methods in four out of six benchmark datasets. For univariate predictions, it outperformed other methods in three out of four benchmark datasets. In the real-world financial datasets, our method ranked highest on two datasets, achieved either first or second place depending on forecast length for two datasets, and ranked second on one dataset. Furthermore, ablation studies were conducted to validate the effectiveness of the model, focusing on the number of decomposition stages and the implementation of different modules on the decomposed elements.

In future work, we plan to investigate the potential benefits of further decomposition stages on model performance. Additionally, as our experiments used a fixed moving average size for time series decomposition, future research will aim to understand how varying the moving average size impacts different datasets and decomposition stages. We also intend to reduce the computational complexity of TSDNet. In particular, optimizing the decomposition process and enhancing the efficiency of model components are promising directions for improving the proposed method.

Footnotes

Acknowledgements

This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) (No. NRF-2022R1F1A1066744), and in part by the Smart Manufacturing Innovation R&D program funded by the Korea Ministry of SMEs and Startups in 2024 (Project No. RS-2024-00433001).

ORCID iDs

Sukhyun Cho

Dokyun Kim

In-Beom Park

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) (No. NRF-2022R1F1A1066744), and in part by the Smart Manufacturing Innovation R&D program funded by the Korea Ministry of SMEs and Startups in 2024 (Project No. RS-2024-00433001).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Gao

. Machine learning applications for data center optimization. 2014.

Baek

Jang

Yoon

. End-to-end blood pressure prediction via fully convolutional networks. IEEE Access 2019; 7: 185458–185468.

Kim

Cho

, et al. A deep learning model for real-time mortality prediction in critically ill children. Critical Care 2019; 23: 1–10.

Sawhney

Agarwal

Wadhwa

, et al. Deep attentive learning for stock movement prediction from social media text and company correlations. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP). 2020, pp.8415–8426.

Lim

Zohren

. Time-series forecasting with deep learning: a survey. Philosop Trans R Soc A 2021; 379: 20200209.

Zonta

Da Costa

da Rosa Righi

, et al. Predictive maintenance in the industry 4.0: a systematic literature review. Comput Indust Eng 2020; 150: 106889.

Cao

. Financial time series forecasting model based on ceemdan and lstm. Phys A: Stat Mech Appl 2019; 519: 127–139.

Box

Jenkins

Reinsel

, et al. Time series analysis: forecasting and control. Hoboken, NJ: John Wiley & Sons, 2015.

Kusuma

Doherty

Ramchandran

. Distributed compression for sensor networks. In: Proceedings 2001 international conference on image processing (Cat. No. 01CH37205), 2021, vol.1, pp.82–85. IEEE.

10.

Goodfellow

Shlens

Szegedy

. Explaining and harnessing adversarial examples. arXiv preprint arXiv:14126572, 2014.

11.

Choi

Park

, et al. Deep learning for anomaly detection in time-series data: review, analysis, and guidelines. IEEE Access 2021; 9: 120043–120065.

12.

Wen

Zhou

Zhang

, et al. Transformers in time series: a survey. arXiv preprint arXiv:220207125, 2022.

13.

Sen

Dhillon

. Think globally, act locally: a deep neural network approach to high-dimensional time series forecasting. In: Proceedings of the international conference on neural information processing systems (NeurIPS), 2019, vol.32, Article No. 435, pp.4837–4846.

14.

, et al. Revisiting long-term time series forecasting: an investigation on linear mapping. arXiv preprint arXiv:230510721, 2023.

15.

Bai

Kolter

Koltun

. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:180301271, 2018.

16.

Liu

, et al. Timesnet: Temporal 2d-variation modeling for general time series analysis. In: Proceedings of the international conference on learning representations (ICLR), 2023.

17.

Cleveland

McRae

, et al. Stl: a seasonal-trend decomposition. J Off Stat 1990; 6: 3–73.

18.

Wang

, et al. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Adv Neural Inf Process Syst 2021; 34: 22419–22430.

19.

Wang

Peng

Huang

, et al. Micn: Multi-scale local and global context modeling for long-term series forecasting. In: International conference on learning representations (ICLR), 2023.

20.

Zeng

Chen

Zhang

, et al. Are transformers effective for time series forecasting? In: Proceedings of the AAAI conference on artificial intelligence, volume 37. 2023, pp.11121–11128.

21.

Gardner Jr

. Exponential smoothing: the state of the art. J Forecast 1985; 4: 1–28.

22.

Aboagye-Sarfo

Mai

Sanfilippo

, et al. A comparison of multivariate and univariate time series approaches to modelling and forecasting emergency department demand in western australia. J Biomed Inform 2015; 57: 62–73.

23.

Sapankevych

Sankar

. Time series prediction using support vector machines: a survey. IEEE Comput Intell Mag 2009; 4: 24–38.

24.

Bas

Yolcu

Egrioglu

. Picture fuzzy regression functions approach for financial time series based on ridge regression and genetic algorithm. J Comput Appl Math 2020; 370: 112656.

25.

Zhou

Zhang

Lin

. Time series forecasting by the novel gaussian process wavelet self-join adjacent-feedback loop reservoir model. Expert Syst Appl 2022; 198: 116772.

26.

Hochreiter

Schmidhuber

. Long short-term memory. Neural Comput 1997; 9: 1735–1780.

27.

Cho

Van Merriënboer

Gulcehre

, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:14061078, 2014.

28.

Qin

Song

Chen

, et al. A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:170402971, 2017.

29.

Liu

Gong

Yang

, et al. DSTP-RNN: a dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction. Expert Syst Appl 2020; 143: 113082.

30.

Borovykh

Bohte

Oosterlee

. Dilated convolutional neural networks for time series forecasting. J Comput Finance Forth 2019; 22: 73–101.

31.

Huang

Wang

, et al. Dsanet: dual self-attention network for multivariate time series forecasting. In: Proceedings of the 28th ACM international conference on information and knowledge management, 2019, pp.2129–2132.

32.

Liu

Zeng

Chen

, et al. Scinet: time series modeling and forecasting with sample convolution and interaction. Adv Neural Inf Process Syst 2022; 35: 5816–5828.

33.

Silva

AQB

Gonçalves

Matsubara

. Descinet: a hierarchical deep convolutional neural network with skip connection for long time series forecasting. Expert Syst Appl 2023; 228: 120246.

34.

Zhang

Dong

. Temperature forecasting via convolutional recurrent neural networks based on time-series data. Complexity 2020; 2020: 1–8.

35.

Lai

Chang

Yang

, et al. Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR conference on research & development in information retrieval, 2018, pp.95–104.

36.

Chang

Sun

, et al. A memory-network based solution for multivariate time-series forecasting. arXiv preprint arXiv:180902105, 2018.

37.

Goel

Ré

. Efficiently modeling long sequences with structured state spaces. In: ICLR, 2022.

38.

Vaswani

Shazeer

Parmar

, et al. Attention is all you need. Proceedings of International Conference on Neural Information Processing Systems (NIPS), 2017; 30: 6000–6010.

39.

Jin

Xuan

, et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In: Proceedings of the international conference on neural information processing systems (NeurIPS), 2019, vol.32, Article No. 471, pp.5243–5253.

40.

Kitaev

Kaiser

Levskaya

, et al. Reformer: the efficient transformer. In: ICLR, 2020.

41.

Zhou

Zhang

Peng

, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI conference on artificial intelligence, Vol.35, 2021, pp.11106–11115.

42.

Lim

Arık

SÖ

Loeff

, et al. Temporal fusion transformers for interpretable multi-horizon time series forecasting. Int J Forecast 2021; 37: 1748–1764.

43.

Liu

Liao

, et al. Pyraformer: low-complexity pyramidal attention for long-range time series modeling and forecasting. In: International conference on learning representations, 2022.

44.

Woo

Liu

Sahoo

, et al. Etsformer: exponential smoothing transformers for time-series forecasting. arXiv preprint arXiv:220201381, 2022.

45.

Chowdhury

Zhang

Shang

, et al. Tarnet: task-aware reconstruction for time-series transformer. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data Mining. 2022, pp.212–220.

46.

Liu

Wang

, et al. Non-stationary transformers: exploring the stationarity in time series forecasting. In Oh AH, Agarwal A, Belgrave D et al. (eds.) Advances in neural information processing systems. 2022, https://openreview.net/forum?id=ucNDIDRNjjv.

47.

Wang

Liu

, et al. Clformer: locally grouped auto-correlation and convolutional transformer for long-term multivariate time series forecasting. Eng Appl Artif Intell 2023; 121: 106042.

48.

Liu

Zhang

, et al. itransformer: inverted transformers are effective for time series forecasting. In: International conference on learning representations (ICLR), 2024.

49.

Oreshkin

Carpov

Chapados

, et al. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In: International conference on learning representations, 2020, https://openreview.net/forum?id=r1ecqn4YwB.

50.

Olivares

Challu

Marcjasz

, et al. Neural basis expansion analysis with exogenous variables: forecasting electricity prices with NBEATSx. Int J Forecast 2023; 39: 884–900.

51.

Challu

Olivares

Oreshkin

, et al. Nhits: Neural hierarchical interpolation for time series forecasting. Proc AAAI Conf Artif Intell 2023; 37: 6989–6997.

52.

Zhang

Cao

, et al. Less is more: fast multivariate time series forecasting with light sampling-oriented MLP structures. arXiv preprint arXiv:220701186, 2022.

53.

Korvesis

Besseau

Vazirgiannis

. Predictive maintenance in aviation: failure prediction from post-flight reports. In: 2018 IEEE 34th International conference on data engineering (ICDE). IEEE, 2018, pp.1414–1422.

54.

Singh

Joshi

Patney

, et al. The fourier decomposition method for nonlinear and non-stationary time series analysis. Proc R Soc A: Math Phys Eng Sci 2017; 473: 20160871.

55.

Wang

, et al. Multilevel wavelet decomposition network for interpretable time series analysis. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 2018, pp.2437–2446.

56.

Asadi

Regan

. A spatio-temporal decomposition based deep neural network for time series forecasting. Appl Soft Comput 2020; 87: 105963.

57.

Zhou

Wen

, et al. Fedformer: frequency enhanced decomposed transformer for long-term series forecasting. In: International conference on machine learning. PMLR, 2022, pp. 27268–27286.

58.

Woo

Liu

Sahoo

, et al. CoST: contrastive learning of disentangled seasonal-trend representations for time series forecasting. In: International conference on learning representations, 2022, https://openreview.net/forum?id=PilZY3omXV2.

59.

Huang

Liu

Zhang

, et al. Crosswavenet: a dual-channel network with deep cross-decomposition for long-term time series forecasting. Expert Syst Appl 2024; 238: 121642.

60.

Kim

Tae

, et al. Reversible instance normalization for accurate time-series forecasting against distribution shift. In: International conference on learning representations, 2022, https://openreview.net/forum?id=cGDAkQo1C0p.

61.

Ogasawara

Martinez

De Oliveira

, et al. Adaptive normalization: a novel data normalization approach for non-stationary time series. In: The 2010 international joint conference on neural networks (IJCNN). IEEE, 2010, pp.1–8.

62.

Smith

. Mathematics of the discrete Fourier transform (DFT): with audio applications. Stanford, CA: Julius Smith, 2008.

63.

Szegedy

Liu

Jia

, et al. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp.1–9.

64.

Kingma

. Adam: A Method for Stochastic Optimization. In: International conference on learning representations (ICLR), San Diega, CA, USA, 2015.

65.

Duchi

Hazan

Singer

. Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 2011; 12: 2121–2159.

66.

Huber

. Robust estimation of a location parameter. In: Breakthroughs in statistics: Methodology and distribution, Springer, 1992. pp.492–518.

		$X$
		$T$		$S$
Experiment Number	Number of Decomposition	$T_{t}$	$T_{s}$	$S_{t}$	$S_{s}$
D1	0	$f_{L i n}$
D2	1	$f_{L i n}$		$f_{L i n}$
D3	2	$f_{L i n}$		$f_{L i n}$	$f_{L i n}$
D4	2	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$
D5	2	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$	$f_{L i n}$

TSDNet: A two-stage decomposition-based hybrid deep neural network for long-term time series forecasting

Abstract

Keywords

1. Introduction

2. Related work

2.1. Traditional methods in time series forecasting

2.2. Deep learning-based models for time series forecasting

2.3. Decomposition approaches for time series forecasting

3. Methods

3.1. Problem definition and notations

3.2.1. The overview of TSDNet

Data length extension

Implementation of TimeBlock

4.1. Datasets

4.1.1. Benchmark time series datasets

Table 1. Dataset descriptions. Dataset Dim Dataset Size Data Frequency ETTh1, ETTh2 7 (8545, 2881, 2881) Hourly ETTm1, ETTm2 7 (34465, 11521, 11521) 15 mins Exchange rate 8 (5120, 665, 1422) Daily Weather 21 (36792, 5271, 10540) 10 mins

4.2. Experiment configurations of TSDNet

4.3. Evaluation metrics

5. Results

5.1. Performance comparison on multivariate and univariate time series forecasting

Table 3. MSE and MAE results of TSDNet and other methods for multivariate time series forecasting, sourced from Wu et al. 16 and Liu et al. 48

5.5.1. Ablation of decomposition

Footnotes

Acknowledgements

ORCID iDs

Funding

Declaration of Conflicting Interests

References

Table 1.
Dataset descriptions.

Dataset Dim Dataset Size Data Frequency

ETTh1, ETTh2 7 (8545, 2881, 2881) Hourly

ETTm1, ETTm2 7 (34465, 11521, 11521) 15 mins

Exchange rate 8 (5120, 665, 1422) Daily

Weather 21 (36792, 5271, 10540) 10 mins

Table 3.
MSE and MAE results of TSDNet and other methods for multivariate time series forecasting, sourced from Wu et al.¹⁶ and Liu et al.⁴⁸