A deep learning-based interferometric synthetic aperture radar framework for abnormal displacement deformation prediction of bridges

Abstract

This paper first proposes a novel framework for combining deep learning approach and Synthetic Aperture Radar (SAR) technique to evaluate and predict the condition of bridge displacement . The Long short-term memory (LSTM) neural network and the Small Baseline Subsets InSAR (SBAS-InSAR) are used to predict the longitudinal deformation of the bridge. Firstly, the proposed framework based on LSTM is established to obtain the relationship between the longitudinal deformation of the bridge and the influence parameters (such as temperature, temporal baseline, and others). In particular, the residual connection (RC) module is adapted to form the residual long short-term memory (Res-LSTM) neural network to improve prediction accuracy and robustness. A case study in Nanjing is analyzed to verify the performance of the proposed framework. The extracted datasets obtained from Sentienl-1 are divided into the training, validation, and testing dataset, which can satisfy the requirements in the engineering field. The R² and MSE of the testing datasets utilizing the proposed model are 93.17% and 15.62. Furthermore, the results indicated that the proposed framework based on Res-LSTM could predict and warn the abnormal structural deformation, extend the lifespan of structures, and minimize structural damage due to excessive deformation.

Keywords

long short-term memory neural network small baseline subsets of interferometric synthetic aperture radar deformation prediction deep learning bridge health monitoring

Highlights

• The present study first proposed the method combined with the Res-LSTM and SBAS-InSAR to predict longitudinal abnormal and excessive displacement of bridges.

• The temperature, previous longitudinal displacement, and temporal baseline are considered as inputs to find the relationship between longitudinal bridge longitudinal and external factors.

• The root means square prop (Rmsprop) has the better performance and convergence efficiency than other optimizers.

• Assembling SAR data processing and Res-LSTM, a Graphical User Interface was designed to make the results of longitudinal displacement prediction of the bridge straightforward to comprehend.

Introduction

Bridges are an important part of infrastructure network that fundamentally promotes economic and social advancement (Wang et al., 2021). However, several factors, such as sudden earthquakes, vehicle overload, seasonal temperature variation, and frequent wind load may lead to large deformation during the service life of bridges, which will inevitably exacerbate bridge degradation (Zhang et al., 2021). Since the degradation of bridges will produce security risks, which requires a substantial funding budget of the asset owner for maintenance, so reliable and timely evaluation of bridge deformation is of great significance to bridge management. The latest statistics show that China’s bridge network includes more than 960,000 bridges (Ministry of Transport of the People’s Republic of China, nd), such as highway bridges, railway bridges, footbridges, etc. The bridge management department has high requirements for deformation monitoring, early warning, and evaluation of different kinds of bridges. In contrast, traditional structural health monitoring systems face problems, such as easy damage, poor robustness, and high price (C Zhang et al., 2022). Hence, these issues necessitate a need for cost-effectively monitoring the deformation of bridge and assessing the health condition.

The rapid development of satellite-based remote sensing or earth observation technology has the potential to monitor bridge deformation (Meng et al., 2018), which can work as alternative approaches for structural health monitoring and reduce financial budgets (Lan et al., 2012). For example, interferometric synthetic aperture radar (InSAR) analysis of acquired radar satellite images can accurately monitor the movement of bridge structures (Marinkovic et al., 2007). Furthermore, compared with traditional monitoring methods, the InSAR analysis approach can capture various images to obtain structural deformation regardless of weather conditions. The aforementioned satellite-based monitoring technique can screen abnormal deformation of various bridges and mark them for further on-site inspection (Cusson and Ozkan, 2019). Up to now, extensive studies have been conducted to detect and measure the deformation in bridges based on satellite-based technology. Qin et al. (Qin et al., 2019) first proposed a multi-temporal differential SAR interferometry approach to explore cable-stayed and arch bridge deformation characteristics. Huang et al. (Huang, Crosetto, et al., 2017) employed the persistent scatterer interferometry (PSI) approach to monitoring the displacement of a high-speed railway bridge and considered the influence of environmental temperature on the bridge displacement. Furthermore, Qin et al. (Qin et al., 2017, 2018) conducted many studies on the deformation of bridges caused by thermal dilation based on InSAR analysis. Although monitoring bridge deformation using satellite telemetry has proven feasible in practice, the early warning methods of bridge abnormal and excessive deformation based on InSAR still need further exploration.

Indeed, the bridge will be affected by many factors, such as corrosion, fire and temperature, wind etc., during the service period of the bridges (Ni et al., 2021; Xia et al., 2022). The influence of temperature, humidity, and wind on bridge deformation is periodic due to the replacement of seasons (Bai et al., 2022). Previous studies show an obvious relation between the bridge’s deformation and these periodic loads. Thus, it is possible to employ InSAR technology to monitor and predict the abnormal deformation of the bridges. However, due to the limited revisiting time of a single SAR satellite and large manual work, it is hard to conduct real-time monitoring and predict the bridge’s deformation utilizing single InSAR technology (Tan et al., 2018). Fortunately, the rapid advancement of data-driven technologies can address the issues mentioned above. Over the past years, machine and deep learning have performed well predicting mechanical properties (Bagherzadeh et al., 2023; Shafighfard et al., 2022) and structural response (Hou et al., 2022). As for bridge real-time response prediction, several algorithms have been successful in predicting and assessing bridge responses, such as auto-regression and moving-average (ARMA) model (Fan et al., 2016; Wang et al., 2013), support machine regression (SVR) (Feng et al., 2022), recurrent neural networks (RNNs) (Zhang et al., nd), and others (Lu et al., 2021; Ren et al., 2022). Among them, RNNs can remarkably simulate the model’s time-dependence and non-linearity, significantly improving the ability to predict long-sequence responses. In particular, the long short-term memory (LSTM) neural network, a unique RNN, can avoid vanishing and exploding gradients utilizing the gate units (Liao et al., 2023). Therefore, combining advanced algorithms and satellite technology can explore a time-saving and economical approach for bridge deformation monitoring and prediction.

For this purpose, this paper proposes a novel framework combined with LSTM and InSAR to realize bridges’ abnormal and excessive displacement assessment and early warning, considering the relationship between temperature and previous deformation. To achieve this, the authors first investigated the SAR data processing method to select the optimal technique for processing SAR signals for monitoring bridge deformation. The results extracted from InSAR are combined with corresponding temperature and bridge deformation features to evaluate bridge conditions. The present framework is packaged into the software with a graphical user interface to monitor and predict the deformation of bridges within the cyclic temperature loads. The proposed framework can improve the management of preventive maintenance, increase the lifespan of structures, minimize the structural damage caused by excessive deformation, and guarantee the safety of bridges.

The remaining of this paper is organized as follows. Firstly, the structure and principle of LSTM and SBAS-InSAR are illustrated in detail. Then, SBAS-InSAR and Res-LSTM are utilized to develop a surrogate model for predicting bridge displacement deformation. After obtaining an abnormal displacement prediction model, a typical numerical example is provided to validate the presented framework. In conclusion, the process of hyperparameter tuning, main conclusions, and discussion are presented.

Methodology

Monitoring the displacement of bridge bearing was crucial for bridge safety, and the longitudinal displacement of bridge was mainly affected by temperature. Thus, this paper proposed a framework combines the residual long short-term memory (Res-LSTM) neural network and the small baseline subsets InSAR (SBAS-InSAR) for predicting the longitudinal displacement of bridges caused by temperature. In this section, the basic concepts of the SBAS-InSAR and the LSTM neural network are introduced first. Then, the proposed surrogate model based on the Res-LSTM neural network is illustrated in the next section.

SBAS-InSAR

SBAS-InSAR, one of the time-series analysis techniques, was introduced by Berardino et al. (Berardino et al., 2002) in 2002. The fundamental principle behind this method is to select a small baseline interference pair when setting the normal and temporal baseline threshold. The time-series deformation data for the study areas are then obtained using the singular value decomposition algorithm (SVD) (Li and Sun, 2020). In this study, a set of N+1 (SLC) images covering the study areas is acquired by the Sentinel-1A (Xing et al., 2019). The number of interference pairs M should satisfy equation (1). Through choosing the optimal interferometric pairs, the phase value of the j-th interferometry obtained after differential interferometry, filtering, and unwrapping process can be expressed as equation (2).

\frac{N + 1}{2} \leq M \leq \frac{N (N + 1)}{2}

(1)

φ_{j} = φ_{t_{1}} - φ_{t_{2}} \approx \frac{4 π}{λ} (d_{t_{1}} - d_{t_{2}}) + δ φ_{j, a t m} + δ φ_{j, t o p o} + δ φ_{j, n o i s e}

(2)

where $φ_{t_{1}}$ and $φ_{t_{2}}$ are the phase value at $t_{1}$ and $t_{2}$ , respectively; $λ$ is the radar wavelength; $δ φ_{j, a t m}$ , $δ φ_{j, t o p o}$ , and $δ φ_{j, n o i s e}$ represent the components of atmospheric, topographic, and noises phases, respectively. The accuracy of the results can be increased by selecting better interferometric pairs with higher coherence. Furthermore, along the radar line of sight, $d_{t_{1}}$ and $d_{t_{2}}$ denote the cumulative displacement at $t_{1}$ and $t_{2}$ (the expression is given by equation (3)).

d_{t_{2}} - d_{t_{1}} = v_{i} (t_{2} - t_{1})

(3)

where $v_{i}$ denotes the average displacement rate along the radar of sight; when there are multiple interferometric pairs, the matrix form can be formulated as equation (4). In this case, the result of the displacement rate can be acquired by the SVD approach.

B v = d_{φ}

(4)

According to the findings of Monserrat and Huang et al. (Huang et al., 2017; Monserrat et al., 2011), the theoretical model was revised in this paper to add the influence of temperature into the phase component. Herein, the model can be expressed as follows:

φ_{j} = \frac{4 π}{λ} [d_{t_{1}} - d_{t_{2}}] + \frac{4 π}{λ} \cdot \frac{B^{j}}{R^{j} \sin θ^{j}} Δ_{D E M} + \frac{4 π}{λ} Δ T^{j} Δ_{T h e} + δ φ_{j, n o i s e}

(5)

where $B^{j}$ is the normal baseline; $Δ T^{j}$ defines the temperature difference between the two acquisitions; $θ^{j}$ and $R^{j}$ represent the j-th incidence angle and slant range of interferogram, respectively. The DEM error and the thermal dilation phase components are two unknown variables that can be denoted by $Δ_{D E M}$ and $Δ_{T h e}$ .

However, equation (5) is unsuitable for longitudinal displacement monitoring of long-span bridges because it is much more complex, and the solution has many limitations (Huang et al., 2017). Meanwhile, the corresponding longitudinal displacement can be ignored because the DEM error has little influence on the analysis results and the bridge settlement tends to be stable (Huang et al., 2017). Thus, the model considering only the expansion of the bridge can be simplified as follows:

φ_{j} = \frac{4 π}{λ} Δ T^{j} Δ_{T h e} + δ φ_{j, n o i s e}

(6)

LSTM neural network

Long short term memory (LSTM) neural network is a neural network with unique processing based on the traditional RNNs. The advantage of LSTM is that it can prevent gradient disappearance and gradient explosion issues during the training of plant time series. Consequently, LSTM has emerged as one of the popular methods for time series data prediction due to the use of a unique gate mechanism to learn the long-term dependencies of data. The structure of the LSTM unit cell is given in Figure 1.

Figure 1.

The structure of the LSTM unit cell in t time step (1-Forget Gate; 2-Input Gate; 3-Tanh Layer; 4-Output Gate).

Three distinct parts, introduced by LSTM, the input gate, output gate, and forget gate, can be used to regulate the transmission of information. The input gate and output gate, respectively, control the information inputs and final outputs. The forget gate’s function is to choose which data should be removed from the previous cell. The input and forget gates can be used to update the data in the current cell. The output gate then modifies the LSTM cell’s hidden output following the modified cell state. The updating formula of each gate is shown below:

f_{t} = σ (W_{f} \cdot [h_{t - 1}, x_{t}] + b_{f})

(7)

i_{t} = σ (W_{i} \cdot [h_{t - 1}, x_{t}] + b_{i})

(8)

{\tilde{c}}_{t} = \tanh (W_{c} \cdot [h_{t - 1}, x_{t}] + b_{c})

(9)

o_{t} = σ (W_{o} \cdot [h_{t - 1}, x_{t}] + b_{o})

(10)

c_{t} = f_{t} \otimes c_{t - 1} + {\tilde{c}}_{t} \otimes i_{t}

(11)

h_{t} = o_{t} \otimes \tanh (c_{t})

(12)

where

c_{t - 1}

is defined as the cell state of the previous step,

x_{t}

is the input of the LSTM cell,

h_{t}

is the hidden output,

f_{t}

i_{t}

, and

o_{t}

are forget gate, input gate, and output gate. Then,

{\tilde{c}}_{t}

represents the state of the cell. Additionally, the weight matrices for various gate units are represented by

W_{p a r a}

(para=f, i, c). σ(·) and tanh(·) denote the activation function of the LSTM (i.e., logistic sigmoid and hyperbolic tangent functions).

Equations (7)∼(12) can be used to show how information flows through the LSTM cell. When the results of the input $x_{t}$ and hidden output $h_{t - 1}$ are transmitted into the current LSTM cell, a vector $f_{t}$ between zero and one will be output in the forget gate. When the vector value in the forget gate comes to 0, the information will be entirely forgotten. On the contrary, if the input data is much more important, the vector value in the forget gate will be close to 1. Thus, this approach will preserve the significant long-term dependencies in sequential data.

Proposed framework of prediction of bridge abnormal displacement

The proposed framework based on the Res-LSTM neural network is shown in Figure 2. The input variables are previous longitudinal displacement of a bridge, the temperature, and the temporal baseline. The output is the longitudinal displacement of the bridge in the next scene for the result of SBAS-InSAR. The proposed neural network consists of an input layer, four hidden layers (two LSTM layers and two fully-connected layers), and an output layer. Each hidden layer has the same number of neurons. Furthermore, the numbers of hidden layers are often determined based on experience or by referring to existing models with excellent performance, and then adjusted according to trial calculation (Liao et al., 2022).

Figure 2.

The architecture of the Res-LSTM neural network.

Meanwhile, the residual connection (RC) module is implemented into the LSTM network to establish the Res-LSTM neural network. The RC module, proposed by He et al. (He et al., 2016), alleviates error back-propagation and gradient disappearance during the correction, improving the prediction accuracy and robustness. Then, RC modules can well preserve the detail information of the output of the previous hidden layer, thereby endowing Res-LSTM with better prediction performance. In this paper, some previous results obtained from SBAS-InSAR are needed as input variables to train the model to predict bridge displacement . Thus, compared with other algorithms, Res-LSTM is more suitable to solve the issue of predicting bridge displacement based on a few previous SBAS-InSAR results.

In the Res-LSTM, the RC module adds the output of two adjacent LSTM layers, and the expression is given by equations (13) and (14).

y_{L S T M 2} = ReLU (H_{L S T M 2} (y_{L S T M 1}))

(13)

y_{R C} = y_{L S T M 1} + y_{L S T M 2}

(14)

where

y_{R C}

y_{L S T M 1}

, and

y_{L S T M 2}

are the output of the RC module, the first LSTM layer, and the second LSTM layer, respectively;

H_{L S T M 2} (\cdot)

represents the non-linear mapping function and

ReLU (\cdot)

denotes the rectified linear function.

Furthermore, the details of the LSTM cell and each activation function and formula of the unit are given in Figure 1 and equations (7)–(12). The criteria indexes of the proposed Res-LSTM neural network are coefficient of determination(R²) and mean squared error (MSE). The two expressions of the selected criteria are as follows:

R^{2} = 1 - \frac{\sum_{i = 1}^{N} {(P_{i} - T_{i})}^{2}}{\sum_{i = 1}^{N} {(P_{i} - \bar{T})}^{2}}

(15)

M S E = \frac{1}{N} \sum_{i = 1}^{N} {(P_{i} - T_{i})}^{2}

(16)

where N denotes the number of the database;

T_{i}

represents the real value; and

P_{i}

indicates the predicted results of the purposed Res-LSTM.

Numerical example

Study area

A railway bridge in Nanjing, called Bridge-A, was undertaken as the case study to evaluate the predicted performance of the proposed framework. The bridge is a steel truss arch bridge with symmetrical structure. The longitudinal sliding supports are equipped on piers #4–#10 (except #7), while pier #7 is fixed with hinged support (as shown in Figure 3).

Figure 3.

Structure of the Bridge-A (unit: m).

Sentinel-1A SAR data

For acquiring the data for model training, the Sentinel-1A launched by the European Space Agency was selected in this paper (Table 1), which carries a C-band synthetic aperture Radar instrument and can collect the data in any weather, day or night. The 201 ascending single-look complex images were acquired by Sentinel-1A in interferometric wide swath mode between September 2015 and September 2022. The path and incidence angles are 142° and 44.89°, respectively. The range of the SAR data and the date of each image are shown in Figure 4. And the temperatures are acquired form the local weather station.

Figure 4.

Coverages of SAR datasets and locations of the Bridge-A.

Table 1.

Basic information on the Sentinel-1A satellite datasets.

Variable	Value
Path number	142
Incidence angle	44.89
Beam mode	Interferometric wide swath
Direction	Ascending
Polarization	VV
Start date	2015/09/16
End date	2022/09/15

The result of the SBAS-InSAR

The SBAS-InSAR is used to process the selected Sentinel-1A data. The processing is elaborated in Figure 5. The spatial and temporal baseline thresholds are set after resampling all SLC images for registration.

Figure 5.

The processing of the SBAS-InSAR.

Due to the long duration of this case study (roughly 7 years), we have restricted the temporal baseline length of the interferometric datasets to 180d in order to mitigate the impact of topography, atmospheric delay, and orbital deviation. Additionally, the max normal baseline was set as 45% because the Sentinel-1A satellite has a stable orbit and produces less deviation (P Zhang et al., 2022). In this study, the minimum cost flow (MCF) method (Darwish et al., 2021: 2), is used for phase unwrapping. The phase unwrapping results are then subjected to orbital refinement and residual phase removal. Finally, we select high-quality ground control points (GCP) to calculate the orbital parameters precisely and subtract the flat phase and orbit errors.

Figure 6(a) shows the coherence image of the Bridge-B (located in the lower right corner) and Bridge-A. Moreover, the result of the unwrapped phase image of the pair 2016/01/14 and 2016/01/26 is illustrated in Figure 6(b). It can be noted that Bridge-A with the truss structure has clear and better coherence and results. The results of another cable-stayed bridge are slightly worse due to its structural system, the low resolution of data, and stiffness. However, these drawbacks will be improved with the increase in data resolution and technology updates. Herein, this study aims to establish a data-driven surrogate model to predict the longitudinal displacement of bridges based on SBAS-InSAR technology, which has shown better performance in the steel bridges so far.

Figure 6.

Coherence image, unwrapped phase image, and interferogram filtered images of the pair 2016/01/14 and 2016/01/26.

Because the Bridge-A has better demonstration effect, the imaging geometry of the Sentinel-1A data at the Bridge-A is depicted in Figure 7. The following equation provides a simplified conversion of the LOS displacements into longitudinal displacements under the assumption that the longitudinal displacement along the bridge only brings in the LOS deformation:

D_{L on g} = \frac{D_{L O S}}{\sin θ} \cdot \frac{1}{\cos α}

(17)

where

D_{L on g}

and

D_{L O S}

denote the longitudinal deformation of bridge and the displacement of the LOS direction, respectively;

θ

represents the incidence angle of the Sentinel-1A;

α

is defined as the angle between the bridge’s longitudinal orientation and the satellite’s projected ground-based LOS direction.

Figure 7.

Sentinel-1A ascending imaging geometry of the Bridge-A and LOS decomposition.

The result of the longitudinal deformation of Bridge-A can be obtained according to equation (17) when the conversion of LOS deformation into Bridge-A’s longitudinal direction is followed. The time-series longitudinal deformation of Bridge-A during 2015–2022 is shown in Figure 8.

Figure 8.

Time-series longitudinal deformation of Bridge-A (the bearing in the end of bridge).

The temperature of each image is acquired from the local weather station. The red line denotes the day’s maximum temperature, and the green line represents the day’s minimum temperature. To our best knowledge, the temperature is a typical cyclic load, and the rise of temperature will lead to thermal expansion of the bridges, resulting in deformation. Note that the longitudinal displacement of Bridge-A changes periodically with temperature, as is evident in Figure 8. Thus, the surrogate model can be established by finding the relationship between temperature and response.

Data description

After obtaining the data for training the predicted model, a description of the distribution and composition of the dataset is necessary. The input variables are maximum temperature, minimum temperature, and the temporal baseline. The output is the longitudinal deformation of the target bridge. Note that the temporal baseline of each data is 12 or 24 days, due to the limitations of the Sentinel-1A. The details of the dataset are illustrated in Table 2 and Figure 9.

Table 2.

The description of the input parameters and output parameters.

Variable	Minimum	Maximum	Mean	Standard deviation	Type
Max Temperature (°C)	–3	39	21.46	9.14	Input
Min Temperature (°C)	–6	29	13.06	9.42	Input
Temporal Baseline (d)	0	36	12.72	3.32	Input
Longitudinal Deformation (mm)	–22.17	64.01	21.18	20.92	Output

Figure 9.

The description of the dataset.

Hyperparameter tuning

Data pre-processing

Figure 8 shows the results of the time-series longitudinal deformation of the Bridge-A utilizing the SBAS-InSAR. However, the results are raw data that may contain noise and need to be filtered before the following steps. Here, MATLAB toolbox smoothdata is adopted to smooth the raw data. Then, the dataset used for training Res-LSTM is created by the pre-processing shown in Figure 10. The created dataset includes input and output, both one-dimensional arrays. The sequence length i indicates the established dataset used for training surrogated model requires i sets of data obtained from the SBAS-InSAR results. The inputs contain i sets of prior environment variables (Input 1-Input 3) and i-1 sets of prior bridge deformation. Furthermore, the output is the i-th target bridge deformation. Following this principle, the final dataset contains n-i+1 groups of data.

Figure 10.

The process of creating the dataset based on SBAS-InSAR results.

Furthermore, input data is normalized to eliminate the influence of the scale effect on variables and then trained in the proposed surrogated model. In this paper, the authors introduced the min-max normalization method (Jain and Bhandare, 2011), defined by

{\hat{x}}_{i} = \frac{2 (x_{i} - \min_{1 \leq j \leq n} {x_{j}})}{\max_{1 \leq j \leq n} {x_{j}} - \min_{1 \leq j \leq n} {x_{j}}} - 1

(18)

where

{\hat{x}}_{i}

denotes the data after min-max normalization;

x_{i}

is the raw data without standardization;

\max_{1 \leq j \leq n} {x_{j}}

and

\min_{1 \leq j \leq n} {x_{j}}

represents the data’s maximum and minimum values, respectively. Noted that training the model utilizing the standardized data can improve the convergence rate and accuracy (Lin et al., 2020).

The optimal model selection

In order to avoid gradient vanishing, the Rectified Linear Unit (ReLU) is selected as the activation function of LSTM layers FC layers (Nair and Hinton, 2010). Furthermore, the dropout technique is implemented to prevent over-fitting (Srivastava et al., 2014). In this paper, the dropout value is set as 0.2. During the training of the Res-LSTM, the loss function is the mean square error (MSE) that is widely chosen in the regression task. Moreover, the database is split into training, validation, and testing databases in 7:1:2 ratios to improve the generalization capacity of the surrogate model.

Then, selecting the optimal sequence length is vital for the predicted results because LSTM can remember only sequences of a certain length. Thus, the authors study the quantitative comparisons among various sequence lengths in Res-LSTM. The results in the training dataset (as shown in Table 3 and Figure 11(a) and (b)) show that various models perform well on the training dataset using the selected hyperparameters. Some models suffer from an overfitting issue, though their R² of the training dataset exceeds 0.99. However, overfitting is a common issue during hyperparameter tuning (Adibimanesh et al., 2023). Thus, selecting the optimal model should consider the predictive performance of both the training and testing datasets. According to the performance in the testing dataset (as shown in Table 4 and Figure 11(c) and (d)), the predicted accuracy of the established model is optimal when the length of the sequence set and the number of neurons are 10 and 30, respectively.

Table 3.

Accuracy of predicted results in the case of various sample selections (Training dataset).

Accuracy (R²)		Number of neurons in the hidden layer
Accuracy (R²)		20	25	30	35	40	45
Sequence length	4	0.9999	0.9999	0.9996	0.9993	0.9994	0.9999
	6	0.9996	0.9989	0.9997	0.9999	0.9998	0.9999
	8	0.9998	0.9999	0.9999	0.9991	0.9996	0.9991
	10	0.9995	0.9999	0.9997	0.9999	0.9999	0.9998
	12	0.9999	0.9999	0.9999	0.9998	0.9979	0.9999

Figure 11.

Results with different numbers of neurons in training and testing datasets.

Table 4.

Accuracy of predicted results in the case of various sample selections (Testing dataset).

Accuracy (R²)		Number of neurons in the hidden layer
Accuracy (R²)		20	25	30	35	40	45
Sequence length	4	0.8182	0.7126	0.8579	0.8766	0.8236	0.8211
	6	0.8600	0.8218	0.8538	0.8978	0.7987	0.8849
	8	0.8937	0.8946	0.8938	0.8319	0.6432	0.8862
	10	0.9233	0.9106	0.9317	0.8816	0.9057	0.8769
	12	0.9039	0.8906	0.8331	0.9180	0.8742	0.9041

Figure 12 shows the results of loss functions in the training process with the optimal time step (as shown in Table 4). The maximum epoch for training the surrogate model is set as 10000. As shown in Figure 12, when the epoch reaches 30, the loss function of the proposed Res-LSTM is close to convergence. Furthermore, in this study, the learning and the decaying rates are 1 × 10⁻³ and 1 × 10⁻⁸ to ensure the convergence of the proposed Res-LSTM in the training process.

Figure 12.

Loss functions with different time steps during the training process.

In addition to finding the optimal number of neurons in the hidden layer, the training goal is to find the best group of parameters to minimize the loss function utilizing the optimization algorithm. In this paper, the authors selected adaptive moment estimation (Adam) (Kingma and Ba, 2014), root mean square prop (Rmsprop) (Tieleman et al., 2012), and stochastic gradient descent (SGD) (Rumelhart et al., 1988) to obtain the optimized algorithm. The results using these optimizers are shown in Table 5.

Table 5.

Comparison of the predicted performance of different optimizers.

Methods	R ²		MSE		Time (s)
Methods	Train	Test	Train	Test	Time (s)
LSTM (Adam)	0.9999	0.8022	0.3084	38.02	24.51
LSTM (Rmsprop)	0.9997	0.9317	0.1267	15.62	21.57
LSTM (Sgdm)	0.9378	0.3996	25.47	56.52	21.10

The max iterations of these conditions are 10000. Figure 13(a) shows that the models utilizing Adam and Rmsprop greatly converge in the training process. On the contrary, the model using SGD shows poor convergence efficiency. As shown in Figure 13(b), the model is converged when the epoch is close to 8000, which is quite slow compared with others. Meanwhile, when the maximum iterations of each condition were the same, the loss value of the SGD optimizer only decreased to the order of 10⁻². In contrast to the results of Adam and Rmsprop, the values of loss function during the training are both close to 10⁻⁸.

Figure 13.

Loss functions in the convergence process.

Overall, the Rmsprop has the most optimized algorithm with the highest predicted accuracy in this paper. Thus, the authors choose the Rmsprop algorithm for training the surrogate model. Moreover, the training process is conducted in Python utilizing Pytorch, the most widely used deep learning library at the moment. The configuration condition for model training is Intel (R) Core(TM) i7-10700K CPU @3.80 Hz, 3.79 GHz, 32.0 G RAM, and the NVIDIA GeForce RTX 3060, 12 G GPU. Moreover, the software is as follows: Windows 10 and Python 3.8.

Results and discussion

Bridges’ deformation prediction utilizing Res-LSTM

Obviously, the temperature greatly influences the longitudinal deformation of the bridge, especially for the truss structures. In this study, correlation analysis is carried out after obtaining the results of temperature and longitudinal displacement of the bridge based on SBAS-InSAR. The correlation coefficient $ρ_{T, Y}$ is the index that measures and describes the linear relationship between two variables. When the value is close to 1, it indicates a strong positive correlation between the two variables. On the contrary, the value indicates a strong negative correlation between the two variables when it is close to -1 (Jia and Wu, 2021). The expression of $ρ_{T, Y}$ can be expressed as follows:

ρ_{T, Y} = \frac{cov (T, Y)}{σ_{T} σ_{Y}} = \frac{E [(T - μ T) (Y - μ Y)]}{σ_{T} σ_{Y}}

(19)

where

ρ_{T, Y}

denotes the correlation index between the temperature T and Y which is the longitudinal deformation of Bridge. The

cov (T, Y)

and

σ_{T / Y}

are the covariance and the variance, respectively.

The correlation coefficient is obtained using equation (19) to analyze the relationship between temperature and the longitudinal deformation of Bridge-A based on SBAS-InSAR. The findings indicate that the correlation coefficient between the maximum/minimum temperature and the corresponding longitudinal deformation of Bridge-A are −0.65 and −0.66, respectively. Because the absolute values of the correlation mentioned above coefficients are all greater than 0.6 (the negative sign represents a negative correlation). Thus, there is a strong correlation between the temperature and the longitudinal displacement using the SBAS-InSAR technique.

Because of the strong correlation between temperature and longitudinal displacement, it is feasible to establish the surrogate model based on LSTM. The results of the proposed Res-LSTM surrogate model are as follows.

It can be seen from Table 6 that the R² and the MSE of the training dataset and the testing dataset. The value of R² and the corresponding MSE for the training dataset are 0.9997 and 0.1267, respectively. Meanwhile, for the testing dataset, the R² and its MSE are 0.9317 and 15.62, respectively. In addition, the detailed results of the prediction models are shown in Figure 14.

Table 6.

Performance of Res-LSTM.

Index	Training dataset	Testing dataset
R ²	0.9997	0.9317
MSE	0.1267	15.62

Figure 14.

The results of the training and testing dataset.

Indeed, the higher value of R² and the lower the corresponding MSE, the higher the accuracy of the prediction model. In this paper, the value of R² is 0.9317, and the predicted value of Bridge-A’s longitudinal deformation in the training set is close to the target value. Meanwhile, the longitudinal deformation of the bridge obtained by the SBAS-InSAR technique is affected by many factors, such as the resolution of the SAR image, the quality of the GCP points, the unwrapping methods and parameter settings, and others. In particular, setting parameters highly depends on the technician’s experience. In this case, the proposed Res-LSTM model can assist researchers in monitoring and predicting bridges’ longitudinal displacement utilizing multiple techniques.

Furthermore, to further verify the accuracy and performance of the proposed model, the authors count the distribution of prediction values versus target values. Figure 15 indicates that the predicted accuracy of the proposed surrogate model performs well on the training dataset, and the testing dataset’s prediction values are similar to target values. Thus, the predicted accuracy of the proposed approach can satisfy the engineering requirements for bridge health monitoring.

Figure 15.

Prediction values versus target values.

Model deployment

In order to facilitate the bridge’s owner to use the proposed framework to predict and warn abnormal deformation, a project with a graphical user interface was established utilizing the MATLAB GUI (Figure 16). Once time-series data is obtained, such as the previous deformation, temperature and the next predicted scenario, the deformation of the predicted scenario can be calculated to assist the bridge owner in monitoring and warning of abnormal deformation.

Figure 16.

SAR-Bridge user interface.

Discussion

Although the proposed Res-LSTM surrogate model’s predicted accuracy can meet the engineering field requirement, some strategies can still be used to improve the mode’s prediction accuracy. The following factors affect the accuracy of the Res-LSTM model: (1) The quality and quantity of the dataset for training the surrogate model are mediocre. As the data obtained in this paper is free and limited, the higher accuracy requires high-precision data for model training. (2) The maximum and minimum temperatures corresponding to each data are obtained from the local weather station utilizing Python rather than the sensor mounted on Bridge-A. As a result, there are some errors in the relationship between temperature and longitudinal deformation of the bridge and further affect training accuracy. Given the drawbacks above, the following three strategies can be considered.

1. Improve the dataset’s quality before the training applying high precision and resolution SAR image data such as TerraSAR-X, RADASAT2, COSMO-SkyMed, and others. As present study proposes a novel approach to tackle with bridge’s daily abnormal longitudinal deformation, resolution of the Sentinel-1 can meet our requirements and most importantly it is free. Commercial satellite data could be applied to further optimize the results once sufficient research funds are awarded.

2. Using the onsite real-time temperature data from sensors of the structural health monitoring system (SHMs). In present study, the temperature data from the weather station near the bridge is used for training instead of the data from Bridge-A’s SHMs. As there is might be some temperature differences between the weather station and temperature transmitters, the errors in predicted model of bridges are inevitable. Therefore, to capture the better prediction performance, the on-site data from SHMs need to be utilized.

3. Expand the number of databases of training model to further improve the proposed Res-LSTM surrogate model's performance. As we all know, the surrogate model training based on Res-LSTM is a data-driven process that needs a large number of data. However, Sentinel-1 was launched in 2015, and its revisit cycle was at least 12 days. Compared with other data-driven models, the volume of the database in this paper is relatively small. Fortunately, GF-3 satellite will enter the network soon and reduce the revisit cycles of the satellite (G Zhang et al., 2022).

Conclusions

This study first proposed the efficient surrogate model combined with the Res-LSTM and SBAS-InSAR to provide insight into predicting longitudinal abnormal and excessive deformation of bridges. The conclusions are as follows:

1. The framework proposed in this paper can effectively predict the abnormal displacement of bridges under the effect of temperature. Furthermore, this method has the potential to be an auxiliary approach with other traditional monitoring methods or as a cost-effective alternative to achieve bridge health assessment and early warning.

2. As SAR data processing is highly dependent on the experience of professionals, this framework can address the limitations that traditional InSAR techniques are hard to popularize widely. A MATLAB GUI was designed by assembling SAR data processing and deep learning algorithms, so the prediction of longitudinal deformation becomes fast and intelligently.

3. The proposed Res-LSTM surrogate model shows good prediction accuracy and efficiency in training and testing datasets. The R² of the training and testing datasets utilizing the proposed Res-LSTM model are 99.97% and 93.17%, respectively. Moreover, The MSE of the training and testing datasets are 0.1267 and 15.62, respectively. The results indicated that the proposed surrogate model could predict the longitudinal displacement of the bridge, which can help engineers maintain the bridge’s health easily and robustly. Furthermore, further improvements in the quality and quantity of the data are required to improve the predictive performance of the proposed Res-LSTM surrogate model.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (52127813), the Tencent Foundation through the XPLORER PRIZE, Natural Science Foundation of Jiangsu Province (BK20210254), Foundation of Science and Technology on Near-Surface Detection Laboratory (6142414210804) as well as the 2021 High-level Personnel Project Funding of Jiangsu Province (JSSCBS20210069). The authors are grateful for the financial support. In particular, we wish to thank Professor Xiaolin Meng (School of Instrument Science and Engineering, Southeast University) for the revisions suggestions to this manuscript.

ORCID iD

Gang Wu

Data availability statement

Data will be made available on request.

References

Adibimanesh

Polesek-Karczewska

Bagherzadeh

, et al. (2023) Energy consumption optimization in wastewater treatment plants: machine learning for monitoring incineration of sewage sludge. Sustainable Energy Technologies and Assessments 56: 103040. DOI: 10.1016/j.seta.2023.103040.

Bagherzadeh

Shafighfard

Khan

RMA

, et al. (2023) Prediction of maximum tensile stress in plain-weave composite laminates with interacting holes via stacked machine learning algorithms: a comparative study. Mechanical Systems and Signal Processing 195: 110315. DOI: 10.1016/j.ymssp.2023.110315.

Bai

Wang

Fang

, et al. (2022) Detecting seasonal and trend components in PS-InSAR displacement time series. Geocarto International 37(27): 16212–16231. DOI: 10.1080/10106049.2022.2106316.

Berardino

Fornaro

Lanari

, et al. (2002) A new algorithm for surface deformation monitoring based on small baseline differential SAR interferograms. IEEE Transactions on Geoscience and Remote Sensing 40(11): 2375–2383. DOI: 10.1109/TGRS.2002.803792.

Cusson

Ozkan

(2019) Satellite-based InSAR Monitoring –Validation on Victoria Bridge in Montreal, Canada. 5th Conference in Smart Monitoring, Assessment and Rehabilitation of Civil Structures (SMAR 2019)

Darwish

Kaiser

Koch

, et al. (2021) Assessing the accuracy of ALOS/PALSAR-2 and sentinel-1 radar images in estimating the land subsidence of coastal areas: a case study in alexandria city, Egypt. Remote Sensing 13(9): 1838. DOI: 10.3390/rs13091838.

Fan

Hao

(2016) Piezoelectric impedance based damage detection in truss bridges based on time frequency ARMA model. Smart Structures and Systems 18(3): 501–523. DOI: 10.12989/sss.2016.18.3.501.

Feng

Gao

, et al. (2022) Machine learning-based bridge cable damage detection under stochastic effects of corrosion and fire. Engineering Structures 264: 114421. DOI: 10.1016/j.engstruct.2022.114421.

Zhang

Ren

, et al. (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, June 2016, pp. 770–778. IEEE. DOI: 10.1109/CVPR.2016.90.

10.

Hou

Jiang

Wan

, et al. (2022) Deep learning and data augmentation based data imputation for structural health monitoring system in multi-sensor damaged state. Measurement 196: 111206. DOI: 10.1016/j.measurement.2022.111206.

11.

Huang

Crosetto

Monserrat

, et al. (2017a) Displacement monitoring and modelling of a high-speed railway bridge using C-band Sentinel-1 data. ISPRS Journal of Photogrammetry and Remote Sensing 128: 204–211. DOI: 10.1016/j.isprsjprs.2017.03.016.

12.

Huang

Ding

Wang

, et al. (2017b) InSAR-based longitudinal displacement monitoring and analysis on Nanjing Dashengguan bridge. Journal of Southeast University(Natural Science Edition 47(3): 584–589. DOI: 10.3969/j.issn.1001-0505.2017.03.028.

13.

Jain

Bhandare

(2011) Min max normalization based data perturbation method for privacy protection. International Journal of Computer and Communication Technology 2(8): 45–50.

14.

Jia

D-W

Z-Y

(2021) Seismic fragility analysis of RC frame-shear wall structure under multidimensional performance limit state based on ensemble neural network. Engineering Structures 246: 112975. DOI: 10.1016/j.engstruct.2021.112975.

15.

Kingma

(2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980.

16.

Lan

Liu

, et al. (2012) Complex urban infrastructure deformation monitoring using high resolution PSI. Ieee Journal of Selected Topics in Applied Earth Observations and Remote Sensing 5(2): 643–651. DOI: 10.1109/JSTARS.2011.2181490.

17.

Sun

(2020) Structural deformation reconstruction by the Penrose–Moore pseudo-inverse and singular value decomposition–estimated equivalent force. In: Structural Health Monitoring. London, England: Sage Publications Sage UK. 1475921720952333

18.

Liao

Zhang

Lin

, et al. (2022) A stacked residual LSTM network for nonlinear seismic response prediction of bridges. Engineering Mechanics 39: 1. DOI: 10.6052/j.issn.1000-4750.2022.04.0360.

19.

Liao

Lin

Zhang

, et al. (2023) Attention-based LSTM (AttLSTM) neural network for seismic response modeling of bridges. Computers and Structures 275: 106915. DOI: 10.1016/j.compstruc.2022.106915.

20.

Lin

Liu

, et al. (2020) Automatic lithology identification by applying LSTM to logging data: a case study in X tight rock reservoirs. IEEE Geoscience and Remote Sensing Letters 18(8): 1361–1365. DOI: 10.1109/LGRS.2020.3001282.

21.

Wang

Guo

(2021) Vibration characteristic analyses of medium-and small-span girder bridge groups in highway systems based on machine learning models. Advances in Structural Engineering 24(11). 2336–2350. DOI: 10.1177/1369433221997722.

22.

Marinkovic

Ketelaar

Van Leijen

, et al. (2007) InSAR Quality Control: Analysis of Five Years of Corner Reflector Time Series. Proceedings of the Fringe 2007 Workshop (ESA SP-649), Frascati, Italy.

23.

Meng

Nguyen

Xie

, et al. (2018) Design and implementation of a new system for large bridge monitoring—GeoSHM. Sensors 18(3): 3. DOI: 10.3390/s18030775.

24.

Ministry of Transport of the People’s Republic of China 2021 (nd) 2021 transport industry development statistical bulletin. https://xxgk.mot.gov.cn/2020/jigou/zhghs/202205/t20220524_3656659.html

25.

Monserrat

Crosetto

Cuevas

, et al. (2011) The thermal expansion component of persistent scatterer interferometry observations. IEEE Geoscience and Remote Sensing Letters 8(5): 864–868. DOI: 10.1109/LGRS.2011.2119463.

26.

Nair

Hinton

(2010) Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th international conference on machine learning (ICML-10)

27.

Zhang

Xin

(2021) Magnetic flux detection and identification of bridge cable metal area loss damage. Measurement 167: 108443. DOI: 10.1016/j.measurement.2020.108443.

28.

Qin

Liao

Zhang

, et al. (2017) Structural health and stability assessment of high-speed railways via thermal dilation mapping with time-series InSAR analysis. Ieee Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(6): 2999–3010. DOI: 10.1109/JSTARS.2017.2719025.

29.

Qin

Zhang

Ding

, et al. (2018) Mapping and characterizing thermal dilation of civil infrastructures with multi-temporal X-band synthetic aperture radar interferometry. Remote Sensing 10(6): 941. DOI: 10.3390/rs10060941.

30.

Qin

Ding

Liao

, et al. (2019) A bridge-tailored multi-temporal DInSAR approach for remote exploration of deformation characteristics and mechanisms of complexly structured bridges. ISPRS Journal of Photogrammetry and Remote Sensing 156: 27–50. DOI: 10.1016/j.isprsjprs.2019.08.003.

31.

Ren

Zhang

Zhu

, et al. (2022) Damaged cable identification in cable-stayed bridge from bridge deck strain measurements using support vector machine. Advances in Structural Engineering 25(4): 754–771. DOI: 10.1177/13694332211049996.

32.

Rumelhart

McClelland

Group

, et al. (1988) Parallel Distributed Processing. New York: IEEE.

33.

Shafighfard

Bagherzadeh

Rizi

, et al. (2022) Data-driven compressive strength prediction of steel fiber reinforced concrete (SFRC) subjected to elevated temperatures using stacked machine learning algorithms. Journal of Materials Research and Technology 21: 3777–3794. DOI: 10.1016/j.jmrt.2022.10.153.

34.

Srivastava

Hinton

Krizhevsky

, et al. (2014) Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15(1): 1929–1958.

35.

Tan

, et al. (2018) Semiautomated segmentation of sentinel-1 SAR imagery for mapping sea ice in labrador coast. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 11(5): 1419–1432. DOI: 10.1109/JSTARS.2018.2806640.

36.

Tieleman

Hinton

(2012) Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning 4(2): 26–31.

37.

Wang

Khoo

A-J

, et al. (2013) FEM calibrated ARMAX model updating method for time domain damage identification. Advances in Structural Engineering, 16(1): 51–60. DOI: 10.1260/1369-4332.16.1.51.

38.

Wang

Zhang

Liu

(2021) A most-unfavorable-condition method for bridge-damage detection and analysis using PSP-InSAR. Remote Sensing 14(1): 137. DOI: 10.3390/rs14010137.

39.

Xia

, et al. (2022) Temperature behaviors of an arch bridge through integration of field monitoring and unified numerical simulation. Advances in Structural Engineering, 25(16): 3492–3509. DOI: 10.1177/13694332221130797.

40.

Xing

Chang

H-C

Chen

, et al. (2019) Radar interferometry time series to investigate deformation of soft clay subgrade settlement—a case study of lungui highway, China. Remote Sensing 11(4): 429. DOI: 10.3390/rs11040429.

41.

Zhang

Wang

Han

, et al. (nd) An efficient method for predicting wheel-rail forces in coupled nonlinear train-track-bridge system using artificial neural networks. Advances in Structural Engineering 0(0): 13694332231156988. DOI: 10.1177/13694332231156989.

42.

Zhang

Cheng

(2021) A bridge weigh-in-motion method of motorway bridges considering random traffic flow based on long-gauge fibre Bragg grating sensors. Measurement 186: 110081. DOI: 10.1016/j.measurement.2021.110081.

43.

Zhang

Zhu

Shi

, et al. (2022a) Influence line estimation of bridge based on elastic net and vehicle-induced response. Measurement 202: 111883. DOI: 10.1016/j.measurement.2022.111883.

44.

Zhang

Wang

Chen

, et al. (2022b) Development of China’s spaceborne SAR satellite, processing strategy, and application: take Gaofen-3 series as an example. Geo-spatial Information Science 0(0): 1–16. DOI: 10.1080/10095020.2022.2124129.

45.

Zhang

Guo

, et al. (2022c) Land subsidence monitoring method in regions of variable radar reflection characteristics by integrating PS-InSAR and SBAS-InSAR techniques. Remote Sensing 14(14): 3265. DOI: 10.3390/rs14143265.