Abstract
Energy or load disaggregation, as one essential part of non-intrusive load monitoring (NILM), is an efficient way to separate the consumption information of target appliances from the whole consumption data, and can accordingly help to regulate people’s energy consumption behaviors. However, the consumptions of the target appliances are usually affected by the variance of the opening time, working condition and user interference, so it is a difficult task to realize precise disaggregation. To further improve the energy disaggregation accuracy, this paper proposes a new parallel disaggregation strategy with two subnets for the energy consumption disaggregation of the target appliances in the residential buildings. In the proposed strategy, the parallel disaggregation network contains a long-term disaggregation network and a short-term disaggregation network, which can automatically and respectively learn the long-term trend features and short-term dynamic characteristics of the electrical appliances. This parallel structure can make full use of the advantages of different methods in feature extraction, so as to model the appliance features more comprehensively. To better extract the long-term and short-term features, in the long-term disaggregation subnet, we propose the double branch bi-directional temporal convolution network (DBB-TCN) which has a wider receptive field than the traditional temporal convolution networks (TCN), while in the short-term disaggregation subnet, we adopt the convolution auto-encoder to learn the short-term characteristics of the target appliances. Finally, detailed experiments and comparisons are made with two real-world datasets. Experimental results verified that the proposed parallel disaggregation method performs better than the existing methods under various evaluation criteria.
Keywords
Introduction
Due to the climate changes and shortage of resources, reducing carbon dioxide emissions and improving the utilization efficiency of energy have become the hotspot issue around the world. Energy management is an effective approach to alleviate energy shortages and is helpful for energy saving through reasonably regulating people’s daily energy consumption behaviors [1]. The goal of energy management is to optimize the appliances’ energy consumption rate, and further to reduce unnecessary energy consumption. In order to realize the reasonable energy management, it is necessary to obtain the energy consumption information of each appliance. However, the collected energy consumption information are often the mixture of multiple appliances operation signals. Therefore, it is necessary to adopt the energy disaggregation strategy to obtain the consumption information of each specific appliance from the whole mixed consumption data.
The ideal energy disaggregation method can extract the electricity consumption behavior information from the mixed signals of several appliances’ energy consumption. It can greatly help power companies to analyze the energy behaviors of the users, provide better power supply services for users and reduce the waste of unnecessary resources. Moreover, it has been proved that the suggestions based on the disaggregate data can help householders to reduce the cost of electricity wasting as much as 5-15% [2, 3]. Thus, the energy disaggregation plays an important role in the energy management.
Energy disaggregation can be seen as one essential part of the NILM [4]. Hart [4] first regarded NILM as a single-channel blind source separation problem and restored the appliance consumption information by identifying the state of the appliance. And finally, Hart successfully applied the energy consumption information of the appliances to optimize the users’ energy consumption behaviors. With the blooming of machine learning, a large number of machine learning methods of NILM have been proposed to solve the energy disaggregation problems in NILM. These methods can be divided into two categories: one is based on the Markov model [5], and the other is based on the neural network method [6].
For the Markov model based disaggregation methods, Kong [7] applied the hierarchical hidden markov model (HHMM) to represent the states of multi-state appliances. Bajović [8] proposed the hidden semi-markov model (HSMM) to reduce the impact of environmental noise. Furthermore, Singh [9] used the sparse coding approach to construct a new Markov method, which improved the traditional dictionary learning method and reduced the amount of data for training. Although the Markov methods have achieved some goals, it has obvious disadvantages. For example, it has a lower performance on the continuously varying loads, since these appliances have no fixed working states. In addition, the Markov methods on multi-states appliances require a great deal of expert knowledge.
For the neural network methods, the recurrent neural networks (RNN) [10] and its variants, such as the long short-term memory (LSTM) network [11] and the gated recurrent unit (GRU) [12], achieved some exciting achievements on NILM. Linh [13] built an improved RNN model for NILM and proved that the improved model performs better than the combinatorial optimization method. Kaselimi [14] proposed a context-aware LSTM model to enhance the representational ability of deep network. Moreover, Kelly [15] used sequence-to-sequence (seq2seq) learning to the NILM problem and compared its performance with LSTM, denoising auto-encoder (DAE) [16], combinatorial optimization method and regression method in the actual dataset. Chen [17] proposed a convolution neural network (CNN) disaggregation structure in sequence-to-sequence learning. Zhang [18] modified the seq2seq learning and proposed the sequence-to-point learning (seq2point) with the multi-layer convolution technique. Based on the studies of seq2point, Barber [19] proposed four approaches to optimize the network parameters. Inspired by the wavenet, Harell [20] proposed a causal 1-D convolutional neural network to realize the energy disaggregation. Kaselim [21] built a new multi-channel convolutional architecture for energy disaggregation by using multi-dimensional electrical statistical series as the input. Except CNN, as in [22] and [23], generative adversarial networks (GAN) [24] was adopted to reduce the influence of noise in disaggregation process.
The function of deep learning in NILM has been fully proved. In recent years, to explore the potential of CNN in time series tasks, the temporal convolution network (TCN) [25] has been proposed and successfully applied in many related fields [26–28]. In the NILM field, Zhou [29] proposed a muiti-scale TCN deep structure to learn the mixed appliance features. Jia [30] built an improved TCN disaggregation network according to the sequence to point learning. These studies verified that TCN has the stronger potebtial in NILM field than other deep learning methods, such as LSTM and CNN.
In summary, although the existing disaggregation methods have obtained some achievements, there are also some limitations. Relying on expert knowledge makes it difficult for Markov model based methods to perform energy disaggregation well in complex environments. And the existing methods have poor disaggregation performance on the long-term or low-usage-frequency appliances, which reflects the feature extraction ability of existing methods on these appliances needs to be strengthened. Last but not least, most existing disaggregation networks are single-line architectures with a fixed method, which adopt the same feature extraction mode for all appliances in the energy disaggregation. It is hard to ensure that the features of different appliances can be effectively learned.
Therefore, to further improve the energy disaggregation performance, this paper proposes a new parallel disaggregation network for the energy disaggregation. The main contributions of this study are as follows: In order to improve the accuracy of disaggregation, this paper proposes a parallel disaggregation network. The parallel disaggregation network contains a long-term disaggregation network and a short-term disaggregation network. Through the feature extraction of the two subnets, the long-term trend features and short-term dynamic characteristics of the appliances can be represented more comprehensively, thus, the disaggregation performance of the network can be efficiently improved. To enhance the feature extraction ability of TCN, we propose the double branch bi-directional TCN (DBB-TCN) in the long-term disaggregation subnet. Compared with traditional TCN, the DBB-TCN has two convolution branch to enlarge the receptive field in convolution layer, which improves the features extraction ability on different appliances. In addition, the DBB-TCN has a lighter structure than TCN when dealing with the same data. In the parallel disaggregation structure, a new peak-valley interval division method is employed. It divides data into the high and low energy consumption interval according to the statistical characteristics. Through the division of high and low consumption intervals, the users’ energy consumption characteristics can be understood more intuitively, and the disaggregation model can extract the appliance features more easily from the collected data. Moreover, this division method is helpful to enhance the pertinence of the disaggregation and improve the training efficiency. In order to verify the performance of the proposed method, comparisons are made with other existing popular methods, such as CNN, LSTM and TCN, in two residential electricity datasets. Experimental results show that the proposed method can obtain much better performance than the comparative methods.
The rests of this paper are organized as follows: Section 2 introduces the basic principles of TCN, Section 3 proposes the parallel disaggregation network in details, Section 4 reports the results of the comparative experiments in two real-world datasets. Finally, conclusions are summarized in Section 5.
Preliminaries
The Energy Disaggregation Problem
The purpose of energy disaggregation is to decompose the consumption of each specific electrical appliance from the total consumption signal. Letting M be the number of appliances in household, the total consumption signal can be expressed in term of the sum of all consumption appliances, which can be represented as

Schematic diagram of energy disaggregation. (The curves represent the aggregated signal and the energy consumption of the three target appliances respectively.)
Assuming that the M appliances’ power x
m
(t) and the total consumption power Z (t) are given, the energy disaggregation can be modeled as an optimization problem, which searches for the M-dimensional vector
Compared with the traditional CNN methods, TCN adopts dilated causal convolution for feature extraction and the residual block to correct network outputs, which can ensure its performance while preventing network degradation. Figure 2 shows the structure of TCN.

The structure of TCN, including the dilated causal convolution and the residual block.
The features of input data are extracted by the dilated causal convolution layer in TCN, which combines the causal convolution and the 1-D fully-convolutional network [31, 32]. In addition, the dilation coefficient is applied to the causal convolution process, which can enlarge the receptive field of TCN. Assuming that
Residual block is another important part of TCN, which can effectively avoid the gradient vanishing and explosion in the training process [33]. The outputs of the residual block are dependent on the inputs and a series of transformation Ψ, which is defined as
Framework
As shown in Fig. 3, the proposed parallel disaggregation architecture is constructed by the long-term disaggregation subnet and the short-term disaggregation subnet. In the long-term disaggregation subnet, we propose the double branch bi-directional TCN network (DBB-TCN) to improve the ability to extract the long-term trend features. In the short-term disaggregation subnet, the convolution auto-encoder is adopted to extract the short-term dynamic characteristics. Moreover, a statistical feature based data division method is given to enhance the pertinence of the disaggregation networks.

Schematic diagram of parallel disaggregation network.
The peak-valley interval division method aims to make use of the statistical characteristics for data classification. By this method, the collected energy consumption data are divided into the high energy consumption interval and the low energy consumption interval. The data division can not only help to understand the distribution of users’ energy consumption behaviors, but also can help the network to extract the appliances’ working characteristics and improve the network training efficiency.
There are two parts in the peak-valley interval division method, which are the boundary searching and the padding process. The boundary searching is to search the boundary points of the high and low energy consumption intervals according to the statistical characteristics. The boundary points are divided into the low boundary points and the high boundary points. The satisfying conditions of the boundary points in the low energy consumption interval can be expressed as
Similarly, the satisfying condition of the boundary points in the high energy consumption interval can be expressed as
Figure 4 shows the implement of the high and low energy consumption intervals. Moreover, the divided data are padded in padding process, which makes the length of the divided data be equal to the length of network input.

High energy consumption interval (blue box) and low energy consumption interval (green box) of the collected data by the peak-valley interval division method.
Although TCN expands the receptive field of the dilated causal convolution, it can only extract the information from previous data. In order to enhance the feature extraction ability of TCN, we propose a double branch bi-directional TCN in the long-term disaggregation subnet (DBB-TCN). There are two convolution branches in the convolution layer of DBB-TCN, which are the forward convolution branch and the backward convolution branch. The data processing process of the two convolution branches is the same as that of TCN, but the input data is different. The forward convolution branch extracts features from the current moment data and historical data, while the backward convolution branch extracts features from the current moment data and future data. Due to the double convolution branch structure, DBB-TCN has stronger feature extraction and data processing capabilities. Compared with the traditional TCN, the proposed DBB-TCN has a wider receptive field. In addition, when realizing the same receptive field, the network depth of DBB-TCN can be selected to be only half the depth of TCN.
Figure 5 shows the convolution process of DBB-TCN, in which, the input of DBB-TCN is realized by the sliding window mechanism, while the output corresponds to the input at the midpoint of the sliding window. Assuming that [xt-L+1, . . . , xt-1, x
t
, . . . , xt+L-1] is the input of the long-term disaggregation subnet at time t, correspondingly,

The convolution process of DBB-TCN.
Then,
Finally, the output of the residual block in the DBB-TCN can be calculated as

The structure of the long-term disaggregation subnet.
The purpose of the short-term disaggregation subnet is to extract the short-term dynamic characteristics and output the target sequences. Considering that the convolution auto-encoder has excellent performance on feature extraction, therefore, we adopt the convolution auto-encoder as the main component of the short-term disaggregation subnet. The structure of short-term disaggregation subnet is shown in Fig. 7.

The structure of the short-term disaggregation subnet, which includes three convolution layers, three max-pooling layers, three up-sampling layers and three deconvolution layers.
To make the size of the input data match the short-term disaggregation subnet, the longer sequence data are sliced into multiple short sequences, which can be considered that the inputs of the short-term disaggregation subnet are multi-dimensional short sequence data. In the encoding process, the potential features of the short sequence data
Max-pooling layer is applied after the convolution process, which can maintain the feature information from the upper layer and reduce the number of parameters.
Through the encoding process, the input data are transformed into the coding vector, then the output sequences are reconstructed in the decoding process. In the decoder, the deconvolution process can be represented as
The input data are disaggregated by the long-term disaggregation subnet and short-term disaggregation subnet simultaneously. The outputs of such two subnets are then connected in a linear function layer. It finally generates the disaggregation sequences of target appliance as

Framework of parallel disaggregation network.
Applied Datasets
The UK-DALE Dataset
The UK Domestic Appliance Level Electricity dataset (UK-DALE), contributed by Kelly et al [36], records the reading data in every 6 seconds from November 2012 to January 2015 for 5 households in the UK. And, the dataset contains the measurements of more than 10 types appliances. Each appliance has 4 million data recorded in 12 months.
In this paper, only four kinds of appliances are selected as the target appliances because they are the popular appliances for most existing evaluation models. The selected appliances are fridge, microwave, dishwasher and washing machine. House 1, 3, 4 and 5 are chosen as the training data and House 2 as test data, but only the House 1 and House 2 have the data of all type appliances selected. Furthermore, 90% data of House 1, 3, 4 and 5 are used for training, the rest data of these houses are selected as the verification data.
The REDD Dataset
The Reference Energy Disaggregation Dataset (REDD) [37] contains the load measurement of 6 buildings in US which recorded appliance readings in every 3 seconds and 1 second respectively. The length of each record is between 3 and 19 days and each appliance mentioned has about 1.2 million data samples in 4 months.
The training data and test data from the UK-DALE and REDD
The training data and test data from the UK-DALE and REDD
In the REDD dataset, the records of the energy consumption in House 2 and 3 are selected as the training data, House 1 is used for testing. And, the target appliances in these house are same as the appliances selected in the UK-DALE (fridge, microwave, dishwasher and washing machine). 90% data of House 2 and 3 are used for training and 10% data are selected as the verification data. The configuration of the training and test data in our experiments are listed in Table 1.
To reduce the impact of the original data distribution on network performance, the original data are normalized by subtracting the mean value and dividing the standard deviation value. Table 2 shows the mean power values, the standard deviation values and other parameters of the four target appliances used in this paper.
Parameters of the four appliances
Parameters of the four appliances
This paper evaluates the performance of the disaggregation network from two aspects. On one hand, we pay attention to the disaggregation error of the network, on the other hand, we evaluate the identification accuracy of the appliance states in the disaggregation process.
The mean absolute error (MAE) can reflect the average errors of the disaggregation results. Assuming that
The signal aggregate error (SAE) is another common metric to evaluate the relationship between the total disaggregation energy consumption and the actual total energy consumption in a period of time. SAE can be calculated as
Furthermore, recall (REC), precision (PRE), accuracy (ACC) and F1 score are applied to evaluate the performance of appliances state identification in disaggregation. In these four metrics, REC and PRE evaluate the identification ratio of positive samples, ACC represents the identification accuracy of all samples and F1 denotes the whole identification ability. These metrics can be computed as
In order to fully verify the disaggregation performance of the proposed method, on the one hand, we compare DBB-TCN with the existing non-parallel disaggregation methods in the UK-DALE datasets and REDD datasets, which are LSTM and traditional TCN. On the other hand, we construct three parallel methods (LSTM, traditional TCN and DBB-TCN) as the comparison methods to verify the effect of parallel structure in the energy disaggregation.
In order to effectively evaluate the training process of the disaggregation framework, the loss function used in this paper is defined as
The Adam optimization algorithm [38] is employed for training. The Adam optimizer adopts the first-order momentum and the second-order momentum on the basis of the random gradient descent method as the optimization mechanism, which makes it difficult to fall into local minimum. The four parameters in the Adam algorithm, i.e. the learning rate, β1, β2 and ∈, which are set to 0.001, 0.9, 0.999, and 10-8 respectively in this study. And the parallel disaggregation framework is precisely through the Adam optimization algorithm to minimize the loss function and find the optimal parameters for energy disaggregation
Parameters of parallel disaggregation network
In the peak-valley interval division preprocessing, according to the actual needs of the data, the thresholds of the high and low interval are set to be 1000w, the average ranges of the high and low energy consumption intervals are [0w, 600w] and [600w, 5000w] respectively, while the high and low fluctuation intervals are [0w, 300w] and [0w, 3000w]. The determination time of the boundary point of the interval S is set to be 30, which means that if the thirty sample points after the first sample point still satisfy the condition, that point will be a boundary point. In short-term disaggregation subnet, a seven-layer convolution auto-encoder is selected. The length of the input slice data is set to be 100, while the convolution filter size be (2, 2), and the step size be 1. Moreover, each convolution layer uses the leaky Relu function as the activation function. In long-term disaggregation subnet, the depth of DBB-TCN is set to be 8. The length of the sliding window is chosen as 512, which means that the lengths of forward and backward convolution window are 256. The size of the convolution kernel is set to be 20, while the number of the convolution kernel be 7, and the dropout value is selected to be 0.1. The detailed parameters of the peak-valley interval division method, short-term and long-term disaggregation subnet are listed in Table 3.
The UK-DALE Dataset
MAE and SAE for four types of appliances on the UK-DALE dataset
MAE and SAE for four types of appliances on the UK-DALE dataset
Bold values represent the optimal values of an index under different methods.
As mentioned above, to evaluate the performance of DBB-TCN in the disaggregation, we compare proposed disaggregation method with multiple disaggregation methods and Table 4 shows the results of MAE and SAE on the UK-DALE dataset.
In three non-parallel methods, the DBB-TCN has achieved better results on four appliances than other methods. Compared with TCN, the DBB-TCN reduces MAE by 65.60%, 18.60%, 24.72% and 19.95% on the fridge, microwave, dishwasher and washing machine, reflecting the impact of the double convolution branches in DBB-TCN for the energy disaggregation. And, it indicates that DBB-TCN has a significant improvement on the energy disaggregation.
Furthermore, to evaluate the effect of parallel structure, we construct three parallel network to make the comparison with non-parallel disaggregation methods. It can be seen from Table 4 that parallel structures have a higher performance than the non-parallel structure for the same method, which has improved by 57.15% and 50.58% on MAE and SAE at least. Accordingly, compared with the DBB-TCN non-parallel method, the DBB-TCN parallel method reduces the MAE by 58.49%, 61.42%, 69.74% and 14.12% on the four appliances, which obtains the best performance in all disaggregation methods.
Moreover, although DBB-TCN parallel method achieves the similar MAE to TCN parallel method on microwave, reduces SAE by 14.22%, which means that the DBB-TCN has the less total energy disaggregation error in the periodic time. Therefore, the results in Table 4 show that the better results can be achieved by the parallel disaggregation methods through using the long-term trend features and short-term dynamic characteristics in the energy disaggregation.
Performance comparison of the state identification for appliances in UK-DALE
Performance comparison of the state identification for appliances in UK-DALE
Bold values represent the optimal values of an index under different methods.
Figure 9 shows the disaggregation curve of four appliances disaggregated by six disaggregation methods on the UK-DALE dataset. It can be seen from the figure that most disaggregation methods disaggregate best on fridge, second on washing machines and dishwashers, and worst on microwave. It is mainly due to the different operating cycles and times of four appliances. Fridge has a fixed working period. Although the standby time of washing machines and dishwashers is longer, they also have fixed working conditions. But the working hours of the microwave are random, and it is difficult to determine operating times of the microwave in one day.

Disaggregation curve of four appliances disaggregated by six disaggregation methods in UK-DALE, p-Net is the abbreviation of parallel disaggregation network.
Specifically, in three non-parallel methods, TCN and DBB-TCN have a better fitting on washing machine and dishwasher, but poor disaggregation on microwave. Compared with the non-parallel methods, the parallel method greatly reduces error disaggregation and the disaggregation results are more excellent and stable. In all parallel methods, the DBB-TCN parallel method has the best disaggregation performance on the four appliances, which corresponds to the optimal MAE and SAE of DBB-TCN parallel disaggregation method obtained in Table 4.
Table 5 shows the results of the state identification on the UK-DALE dataset. The states of the appliances are determined by the threshold shown in Table 2. As shown in Table 5, compared with the non-parallel disaggregation methods, the parallel disaggregation methods are more accurate in identifying the states of the appliances, which proves that the parallel disaggregation methods have a better disaggregation than the non-parallel structures of the same method.
Clearly, three parallel methods have similar identification performance on the fridge. Although LSTM parallel method has the best identification on microwave, it performs worse on washing machine and dishwasher. It due to that LSTM parallel method fluctuates greatly in the low consumption period and classifies too many negative samples into positive samples. Compared with the other methods, the DBB-TCN method achieves the best identification on washing machine and dishwasher.
MAE and SAE for four types of appliances on the REDD Dataset
MAE and SAE for four types of appliances on the REDD Dataset
Bold values represent the optimal values of an index under different methods.
Performance comparison of the working state identification for appliances in REDD
Bold values represent the optimal values of an index under different methods.
The Disaggregation Performance
For the further verification, this paper also conducted the comparison experiments on the REDD dataset and the experiment results are shown in Table 6.
From Table 6, it can be inferred that DBB-TCN method performs best in all non-parallel methods, which achieved at least 37.24% and 20.13% overall performance improvement on MAE and SAE. The parallel methods reduce MAE and SAE by 64.78% and 22.03% overall, when compared with the non-parallel methods. Moreover, the DBB-TCN parallel disaggregation method obtains the optimal results on the disaggregation of four appliances, which proves the effectiveness of the parallel methods.
The Disaggregation Curve Observations
The disaggregation curve of appliances on the REDD dataset is shown in Fig. 10. It can be seen that the LSTM method fluctuates greatly when the appliance is not working. The performance of the DBB-TCN method is more stable than TCN on washing machine and dishwasher. Compared with the microwave in the UK-DALE dataset, the microwave in the REDD dataset has a more regular and longer work period, therefore, TCN and DBB-TCN has a better fitting on microwave. In addition, it can be inferred from the figure the parallel structure can obviously improve the disaggregation accuracy and reduce the fluctuation of the disaggregation curve.

Disaggregation curve of four appliances disaggregated by six disaggregation methods in REDD, p-Net is the abbreviation of parallel disaggregation network.
Table 7 shows the result of the state identification on the REDD dataset. It can be concluded that the proposed DBB-TCN parallel method can well achieve the identification task for different appliance. In all disaggregation methods, the DBB-TCN parallel method performs best on washing machine and dishwasher, while achieving competitive results on microwave. And, these three appliances are the low-usage-frequency appliances with high energy consumption. Hence, it can be concluded that the DBB-TCN parallel method can reduce the impact of the imbalanced data and extract features on these low-usage-frequency appliances.
The Effect of Peak-valley Interval Division Method in Training
Finally, in order to verify the function of peak-valley interval division method, we compare the losses of three type disaggregation methods using proposed division method. Figure 11 shows the training loss curve of three disaggregation methods on washing machine. It can be inferred from the figure that using peak-valley interval division method can greatly reduce the initial error value and training error for the same method.
Specifically, the loss of LSTM method has been greatly reduced when using the peak-valley interval division method. For TCN and DBB-TCN method, peak-valley interval division method accelerates the convergence speed of two methods, which makes TCN and DBB-TCN method spend less epochs obtaining the best performance. In addition, for the high energy consumption appliances with low frequency usage, such as microwave, dishwasher and washing machine, using peak-valley interval division method is helpful for the network to extract the appliance features and improve the disaggregation accuracy, since the high consumption interval often corresponds to the operating interval of these appliances.

Disaggregation loss curve of three disaggregation methods on washing machine, p-k process represents the abbreviation of the Peak-valley interval division method.
In this paper, we propose a new parallel disaggregation method for the energy disaggregation in the residential buildings, it includes the long-term and the short-term disaggregation subnet. In the long-term disaggregation subnet, we propose DBB-TCN to extract the long-term trend features. In the short-term disaggregation subnet, the convolution auto-encoder is selected to extract the short-term dynamic characteristics. In addition, we applied a new data division method in parallel method to improve the training efficiency. From experiment results of two real-world datasets, it can be proved that proposed parallel method achieves a better disaggregation performance under various evaluation criteria.
Although the proposed method in this paper has achieved some exciting results, there are still some limitations. For example, the types of energy-consuming appliances in real life are far more than the four types mentioned in this paper. Verifying the disaggregation performance of proposed method on other appliances will be one of our following work. Furthermore, we will also explore the disaggregation performance of this method on the other type buildings in the future.
Footnotes
Acknowledgment
This study is partly supported by the National Natural Science Foundation of China (Nos. 62003191, 61903226), the Taishan Scholar Project of Shandong Province (No. TSQN201812092), Key Research and Development Program of Shandong Province (No. 2021CXGC011205), and Innovation Capability Improvement Project for Science and Technology Small and Medium-sized Enterprises of Shandong Province (No. 2021TSGC1053).
